Transcription
Grids, Clouds and HPC The Munich NetworkManagement TeamDieter Kranzlmüllerkranzlmueller@ifi.lmu.deLehr- und Forschungseinheit für Kommunikationssysteme und Systemprogrammierung
About us - The MNM-Team.D. KranzlmüllerVisit @ University of Virginia2
Members of the MNM TeamProf. Dr. Dieter Kranzlmüller, Prof. Dr. Heinz-Gerd HegeringDr. Vitalian DanciuDr. Nils gentschen FeldeChristof KlauseckerSilvia KnittlAnnette KostelezkyThomas KöckerbauerRalf KönigTobias LindingerFeng LiuMartin MetzkerDr. Michael SchiffersChristoph SpielmanJohannes WatzlD. KranzlmüllerDr. Victor ApostolescuDr. Michael BrennerDr. Ernst BötschDr. Wolfgang HommelPatricia MarcuDr. Helmut ReiserChristian RichterDr. Thomas SchaafDr. David SchmitzDr. Mark YampolskiyProf.Dr. Gabi Dreo RodosekVolker EiselerFrank EyermannSebastian HanigkIris HochstatterRobert KochMichael KretzschmarBjörn StelteVisit @ University of Virginia3
rmanceComputingServiceManagementVirtualizationD. KranzlmüllerIT SecurityVisit @ University of Virginia4
MNM-Team LecturesD. KranzlmüllerVisit @ University of Virginia5
Munich Region: High-Tech Cluster Aerospace EADS ESG Galileo Industry Astrium IABG Eurocopter Information andCommunication Siemens Infineon Sun Intel HP General Electric Automotive BMW Audi MAN Finance Allianz Bay. Landesbank HypoVereinsbank MünchnerRück Research DLR FhG Max-Planck LMU FHM TUM Univ. derBundeswehr D. KranzlmüllerSoftware SoftLab Nemetschek Oracle Microsoft sd&mPatent European PatentOffice German PatentOfficeLife neering Visit @ University of VirginiaVenture Capitalists 3i Apax Atlas Venture EarlyBird TVM Wellington Partners 6
Leibniz Supercomputing Centreof the Bavarian Academyof Sciences and HumanitiesBy Ernst A.Graf7 2009 Leibniz Supercomputing Centre
The Leibniz Supercomputing Centre Computer Centre ( 175 employees) for all Munich Universities with-more than 80,000 students and-more than 26,000 employees-including 8,500 scientists Regional Computer Centre for all Bavarian Universities-Capacity computing-Special equipment-Backup and Archiving Centre(more than 7 petabyte, 5.5 billion files)-Distributed File Systems-Competence centre (Networks, HPC, IT Management) National Supercomputing Centre-Gauss Centre for Supercomputing-Integrated in European HPC and Grid projects(DEISA, PRACE, EGI)D. KranzlmüllerVisit @ University of Virginia8
LRZ Supercomputer HLRB2Foto Helmut Payer, produced by gsiComD. KranzlmüllerVisit @ University of Virginia9
The LRZ: a Supercomputing Centre National supercomputing system SGI Altix 4700-9728 compute cores (Intel Itanium2 Montecito)-62.3 TFlop/s peak performance-56.5 TFlop/s Linpack benchmark-39 TByte Total Memory-660 TByte attached Disk space-Weighing 103 metric tons-Consuming 1000 kVA-On 24 m x 12 m footprint 128 cpu SGI Altix 3700 Bx2for scientists in the state of Bavaria 128 dual core cpu SGI Altix 4700for scientists in the state of BavariaFoto Helmut Payer,produced by gsiCom Linux cluster with more than 3.500 cpus (resp. cores) mainlyfor the Munich universities and for scientists in the state of Bavaria More than 500 additional servers for general IT servicesD. KranzlmüllerVisit @ University of Virginia10
Examples of Applications @ LRZ Computational Fluid Dynamics: Optimisation of turbines and wings,noise reduction, air conditioning in trains Fusion: Plasma in a future fusion reactor (ITER) Astrophysics: Origin and evolution of stars and galaxies Solid State Physics: Superconductivity, surface properties Geophysics: Earth quake scenarios Material Science: Semiconductors Chemistry: Catalytic reactions Medicine and Medical Engineering: Blood flow, aneurysms, airconditioning of operating theatres Biophysics: Properties of viruses, genome analysis Climate research: Currents in oceansD. KranzlmüllerVisit @ University of Virginia11
Distribution of CPU-hoursD. KranzlmüllerVisit @ University of Virginia12
Extension of the Buildings(Computer: March 2011, Staff: Autumn 2011)FotomontageSüdansichtNordansichtD. KranzlmüllerVisit @ University of Virginia13
alizationD. KranzlmüllerIT SecurityVisit @ University of Virginia14
290 Sites55 Countries144.000 Cores25 PetaBytes Disk 17.000 Users 200 VOs 330.000 Jobs/DayArcheologyAstronomyAstrophysicsCivil ProtectionComp. ChemistryEarth SciencesFinanceFusionGeophysicsHigh Energy PhysicsLife SciencesMultimediaMaterial Sciences 25th NORDUnetConference,StatusJuly2009:CopenhagenD. id 417Jobs/month: 45%Sites: 5%Countries: 10%Visit @ University of VirginiaVOs: 29%inininina yeara yeara 15yeara year
European Grid InitiativeObjectives: Ensure the long-term sustainability of the European einfrastructure Coordinate the integration and interaction betweenNational Grid Infrastructures Operate the European level of the production Gridinfrastructure for a wide range of scientific disciplinesto link National Grid InfrastructuresEGI Grid Infrastructure should be a large-scale, production Grid infrastructure built on national grids that interoperate seamlessly atmany levels, offering reliable and predictable services to a widerange of applicationsD. KranzlmüllerVisit @ University of Virginia
EGI Operations TasksOperation of tools and services Grid configuration repositories Grid accounting repositoriesUser support Central ticket handling system Gathering requirements for usersupport tools Grid repositories for SLAcompliance and performancemonitoringEGI Blueprint Proposal(V3.0) tasksOther international Grid operations portalhttp://www.eu-egi.eu/blueprint.pdf MW deployment/roll-out and NGI Grid oversightsupport Functions of EGI Resource allocation & brokering Financingsupportof EGISecurity Interoperationsbetween NGI’s and TransitiontoEGI Security policy developmentandwith other gridsmaintenance Coordination of security andincident response Expert team for securityvulnerabilitiesD. Kranzlmüller Network support Definition of best practises,procedures, requirements Catch-all production grid coreservicesVisit @ University of Virginia
EGI OperationsEGI.euD. KranzlmüllerVisit @ University of VirginiaEGI.eu global tasksNGI international tasksNGI local tasks
EGI borationEuropean-level Grid ServicesEGI.euEGI.euMiddlewareXMiddleware BMiddleware BD. nalNationalNationalNational isessentialfor the gridGridGridGridGridInitiative22 continuesInitiativeNNto evolvewithInitiative Initiative new functionalities development is aVisit @ University of Virginialong-term process
D-Grid Project D-MONComputingService (DB)VO-specific View (OGSA-DAI)D. KranzlmüllerVisit @ University of Virginia
alizationD. CloudComputingIT SecurityVisit @ University of Virginia21
Cloud Research @ MNM-TeamQuestions: What is the cloud? What are the differencesbetween clouds and grids? Which technologies can beused within a cloud? How to use anonymousservices of cloudsefficiently? How to securely deployand use clouds?D. KranzlmüllerVisit @ University of Virginia22
Cloud Testbed @ MNM Example: Zimory-T-Labs spin-off-Small company ( 20 staff workers)-Interconnects 2 Telekom computing centers-Only enterprise-grade customers-Provides qualitatively high services-3 service-levels (Gold, Silver, Bronze)D. KranzlmüllerVisit @ University of Virginia23
Zimory Cloud (1/3)Management solution to manage multiple cloudsSource: Zimory GmbHD. KranzlmüllerVisit @ University of Virginia24
Zimory Cloud (2/3) Gateway to enterpise cloud computing Resources Support of SLAsSource: Zimory GmbHD. KranzlmüllerVisit @ University of Virginia25
Zimory Cloud (3/3)Wide AreaNetworkZimory PublicCloud SolutionCloud GatewayDate Center manager, Orchestrationlayer, Security manager, Data policymanager, Resource quality managerData Center 2Data Center 1Cloud ManagervCenter, Platespin, etc.vCenter, Platespin, etc.Infrastructure ManagerInfrastructure ManagerSLA manager, Resource planner,Application profiler, Network optimizer,Load balancer, Migration manager,Accounting, Authentication, Policymanager, Open APICloud HostVMware, Xen, KVM, Hyper-VVMware, Xen, KVM, Hyper-VVirtualized ResourcesVirtualized ResourcesOracle, MySQL, SANDBDBData StorageD. KranzlmüllerDBOracle, MySQL, SANDBDBDBSecurity management, Resourceoptimization, Interoperableaccess management, VMmigration,Database Hypervisor Data management, Datasynchronization, Data migration,Storage reliability and securityData StorageVisit @ University of Virginia26Source: Zimory GmbH
gementVirtualizationD. KranzlmüllerIT SecurityVisit @ University of Virginia27
Scalable Parallel Debugging with Integrated workbench framework to access the power ofexisting computing infrastructures Built on top of the Eclipse framework, released underEclipse Public License (EPL), continues as an EclipseTechnology Project Version 1.0 available for downloadhttp://www.eclipse.org/geclipse/D. KranzlmüllerVisit @ University of Virginia28
g-Eclipse Trace Viewer Tool to visualize andanalyze communication ofparallel message passingprograms Parallel debugging anddevelopment activitiescentered around TraceViewer Integrated into g-Eclipse Can be used asstandalone application Platform independentD. KranzlmüllerVisit @ University of Virginia29
g-Eclipse Trace Viewerphysical clockslogical clocksstatisticsD. KranzlmüllerVisit @ University of Virginia30
Scalability for Debugging Abstraction techniques are necessary Based on Actions and Markers extension points-reorder, group or hide processes-mark interesting eventsD. KranzlmüllerVisit @ University of Virginia31
Reorder, Group or Hide Processes By manually selecting the processes Automatically by criteriaD. KranzlmüllerVisit @ University of Virginia32
Communication Patterns Search for patterns defined using a description language-serves program understanding-helps to detect errors and inefficient communicationD. KranzlmüllerVisit @ University of Virginia33
g-Eclipse Components CombinedD. KranzlmüllerVisit @ University of Virginia34
rmanceComputingServiceManagementVirtualizationD. KranzlmüllerIT SecurityVisit @ University of Virginia35
Contactwww.nm.ifi.lmu.dekranzlmueller@ifi.lmu.deD. KranzlmüllerVisit @ University of Virginia36
VMware, Xen, KVM, Hyper-V VMware, Xen, KVM, Hyper-V DB DB DB Data Storage DB DB DB Data Storage Database Hypervisor Data management, Data synchronization, Data migration, Storage reliability and security Oracle, MySQL, SAN Oracle, MySQL, SAN Cloud Manager SLA manager, Resource planner, Application profiler, Network optimizer,
