Similar presentations

2 Sessions on Grid What are we going to cover today?A brief historyWhy we are doing itApplicationsUsersChallengesMiddlewareWhat are you going to cover next week?technical talk on the specifics of our workIncluding application to e-Business and e-Science

3 An Overused Analogy Electrical Power GridComputing power might somehow be like electrical powerplug inswitch onhave access to unlimited powerWe don’t know who supplies the power, or where it comes fromjust pick up the bill at the end of the monthIs this the future of computing?We think that world we are in today is the way it has always been - not the caseDevelopment of societal structure is bound up with infrastructure.So one question - (C) why is there Chicago? (C) Onions, lakes, mossies - not very promising.In the US at that time, farming was starting, clearing forests, monoculture farming, bison killed5. Great lakes & rivers were previous means of travel.6. Emergence of railroads linked the lakes and news lines.7. Chicago became a transit centre - a cache for goods.8. People started to exchange different goods - empowered by the infrastructure.9. New institutions were formed such as CBT financial org.10. “Grid” technologies such as fridge vans made export easier, cheaper, further...11. New “middleware” created unexpected industries (great retailing etc)12. If there was ever a city that resulted from the emergence of infrastructure - it is Chicago.

4 Sounds great - but how long?Is the computing infrastructure available?Computing power1986: Cray X-MP ($8M)2000: Nintendo-64 ($149)2003: Earth Simulator (NEC), ASCI Q (LANL)2005: Blue Gene/L (IBM), 360 TeraflopsLook at for current supercomputers!1. It took Chicago over 100 years to do this2. Computing and comms is driven by exponential growth (its on steroids)3. For example, Cray (86) cost $8 million, had its own power substation, special cooling, no graphics.4. Its connection was a 56Kb/s NSF link.5. N64 in 2000 has the same processing power as the Cray X-MP and costs $150 bucks6. It uses 5 watts, not 60,000 wats7. It has 3 dimensional graphics8. Those with broadband or similar have more available than the NSF did only years ago.9. Think of todays compute power.10. Earth simulator (40Tflops peak, 35 sustained)11. ASCI LANL (20Tflops peak, 13 sustained)

6 Many Potential ResourcesTerra-bytedatabasesSpacetelescopes50M MobilePhonesMillions of PCs30% UtilisationGRID1. Lots of different resources type (heterogeneous resources)2. Mobile devices, large data sets, instrumentation, PCs, clusters, supercomputers, even playstations.3. Can these be linked in some sensible way?4. Well, it isn’t as simple as railroads and not all of this can be done.5. Sometimes the application can make it easy -> for example which highly partitionable.6. Most apps aren’t like that - plus they will have large data requirements.10k PS/2per weekSupercomputingCentres

11 What is a Computational Grid?A computational grid is a hardware and software infrastructure that provides dependable, consistent, pervasive and inexpensive access to high-end computational capabilities.The capabilities need not be high end.The infrastructure needs to be relatively transparent.1. Computational grids are for computing (naturally)2. Capability computing (productivity versus high performance)

23 End Systems State of the art Future directionsPrivileged OS; complete control of resources and servicesIntegrated nature allows high performancePlenty of high level languages and toolFuture directionsLack features for integration into larger systemsOS support for distributed computationMobile code (sandboxing)Reduction in network overheads

26 Internets State of the art Future directionsGeographical distribution, no central controlData sharing is very successfulManagement is difficultFuture directionsSharing other computing services (e.g. computation)Identification of resourcesTransparencyInternet services

27 Basic Grid services Authentication Acquiring resources SecurityCan the users use the system; what jobs can they run?Acquiring resourcesWhat resources are available?Resource allocation policy; schedulingSecurityIs the data safe? Is the user process safe?AccountingIs the service free, or should the user pay?

28 Research Challenges (#1)Grids computing is a relatively new areaThere are many challengesNature of ApplicationsNew methods of scientific and business computingProgramming models and toolsRethinking programming, algorithms, abstraction etc.Use of software components/servicesSystem ArchitectureMinimal demands should be placed on contributing sitesScalabilityEvolution of future systems and services

29 Research Challenges (#2)Problem solving methodsLatency- and fault-tolerant strategiesHighly concurrent and speculative executionResource managementHow are the resources shared?How do we achieve end-to-end performance?Need to specify QoS requirementsThen need to translate this to resource levelContention?

30 Research Challenges (#3)SecurityHow do we safely share data, resources, tasks?How is code transferred?How does licensing work?Instrumentation and performanceHow do we maintain good performance?How can load-balancing be controlled?How do we measure grid performance?Networking and infrastructureSignificant impact on networkingNeed to combine high and low bandwidth

32 Middleware Conceptual LayersWorkload Generation, Visualization…Discovery, Mapping, Scheduling, Security, Accounting…1. What is middleware2. Is the bit in between the resources and users.3. The Glue if you like.Computing, Storage, Instrumentation…

33 Requirements include:Offers up useful resourcesAccessible and useable resourcesStable and adequately supportedSingle user ‘Laptop feel’Middleware has much of this responsibility

34 Demanding management issuesUsers are (currently) likely to be sophisticatedbut probably not computer ‘techies’Need to hide detail & ‘obscene’ complexityProvide the vision of access of full resourcesProvide contract for level(s) of support (SLAs)

37 HPSG’s approach: Determine what resources are required(advertise)Determine what resources are available(discovery)Map requirements to available resources(scheduling)Maintain contract of performance(service level of agreement)Performance drives the middleware decisionsPACE