2 CernVM Background Over the past couple of years, the industry has redefined the meaning of some familiar computing terms Shift from glorious ideas of a large public infrastructure and common middleware ( Grids ) towards end-to-end custom solutions and private corporate grids New buzzwords Amazon Elastic Computing Cloud (EC2) Everything is for rent (CPU, Storage, Network, Accounting) Blue Cloud (IBM) is coming Software as a Service (SaaS) Google App Engine Virtual Software Appliances and JeOS In all these cases, virtualizationemerged as a key enabling technology, and is supported by computer manufacturers Multiple cores Hardware virtualization (Intel VT, AMD-V) 2

3 CernVM Motivation LHC Experiment(s) Millions of lines of code Complicated software installation/update/configuration procedure, different from experiment to experiment Only a tiny portion of it is really used at runtime in most cases Often incompatible or lagging behind OS versions on desktop/laptop Multi core CPUs with hardware support for virtualization Making laptop/desktop ever more powerful and underutilised Using virtualization and extra cores to get extra comfort Zero effort to install, maintain and keep up to date the experiment software Reduce the cost of software development by reducing the number of compiler-platform combinations Decouple application lifecycle from evolution of system infrastructure 3

4 How do we do this? Build a thin Virtual Software Appliance for use by the LHC experiments This appliance should provide a complete, portable and easy to configure user environment for developing and running LHC data analysis locally and on the Grid be independent of physical software and hardware platforms (Linux, Windows, MacOS) This should minimize the number of platforms (compiler-os combinations) on which experiment software needs to be supported and tested, thus reducing the overall cost of LHC software maintenance All this is to be done in collaboration with the LHC experiments and OpenLab By reusing existing solutions where possible 4

18 The BOINC community Competition between individuals and teams for credit. Websites and regular updates on status of project by scientists. Forums for users to discuss the science behind the project. E.g. for the volunteers show great interest in CERN and the LHC. Supply each other with scientific information and even help debug the project. screensaver 18

19 Calculates stability of proton orbits in CERN s new LHC accelerator System is nonlinear and unstable so numerically very sensitive. Hard to get identical results on all platforms About users, PC s over 1500 CPU years of processing Objectives: extra CPU power and raising public awareness of CERN and the LHC -both successfully achieved. Started as an outreach project for CERN 50 th Anniversary 2004; used for Year of Physics (Einstein Year)

20 SixTrack program SixTrack is a Fortran program by F. Schmidt, based on DESY program SixTrack simulates 60 particles for 100k-1M LHC orbits Can include measured magnet parameters, beam-beam interactions revealed reproducibility issues, solved by E. McIntosh Phase space images of a particle for a stable orbit (left) and unstable chaotic orbit (right). 20

21 BOINC & LHC physics code Problems with normal BOINC used for LHC physics: 1) A project s application(s) must be ported to every volunteer platform of interest: most clients run Windows, but CERN runs Scientific Linux and porting to Windows is impractical. 2) The project s work must be fed into the BOINC server for distribution, and results must be recovered. Job submission scripts must be developed for this, but CERN physics experiments won t change their current setups. 3) Job management is very primitive in BOINC, whereas physicists want to know where their jobs are and be able to manage them. 21

27 BOINC Virtual Cloud Summary of the method: New BOINC wrapper (VMWrapper) used to start a guest Virtual machine in BOINC client PC, and execute a CernVM image. The CernVM image has all LHC software and CoPilot code. Host-to-VM communication/control provided for any BOINC PC. The new Vmwrapper gives BOINC client and server all the functions they need -they are unaware of VM s As before, the CoPilot allows LHC job production to proceed without changes. 27

29 Building a Volunteer Cloud Final Summary: Solved porting problem to all client platforms: Solved image size problem Solved job production interface problem All done without changing existing BOINC infrastructure (client or server side) All done without changing physicists code or procedures We have built a Volunteer Cloud 29

Virtualization Virtualization: extend or replace an existing interface to mimic the behavior of another system. Introduced in 1970s: run legacy software on newer mainframe hardware Handle platform diversity

Virtualization with Windows at CERN Juraj Sucik, Emmanuel Ormancey Internet Services Group Agenda Current status of IT-IS group virtualization service Server Self Service New virtualization features in

CSE 501 Monday, September 26, 2011 Kevin Cleary kpcleary@buffalo.edu What is Virtualization? What are the different types of virtualization. Practical Uses Popular virtualization products Demo Question,

Virtual Machines Uses for Virtual Machines Virtual machine technology, often just called virtualization, makes one computer behave as several computers by sharing the resources of a single computer between

Enabling multi-cloud resources at CERN within the Helix Nebula project D. Giordano (CERN IT-) HEPiX Spring 2014 Workshop This document produced by Members of the Helix Nebula consortium is licensed under

APPLICATION OF SERVER VIRTUALIZATION IN PLATFORM TESTING Application testing remains a complex endeavor as Development and QA managers need to focus on delivering projects on schedule, controlling costs,

Dynamic Extension of a Virtualized Cluster by using Cloud Resources CHEP 2012 Thomas Hauth,, Günter Quast IEKP KIT University of the State of Baden-Wuerttemberg and National Research Center of the Helmholtz

5. Implementation Implementation of the trust model requires first preparing a test bed. It is a cloud computing environment that is required as the first step towards the implementation. Various tools

Virtualization Dr. Yingwu Zhu What is virtualization? Virtualization allows one computer to do the job of multiple computers. Virtual environments let one computer host multiple operating systems at the

Running Windows on a Mac Why? 1. We still live in a mostly Windows world at work (but that is changing) 2. Because of the abundance of Windows software there are sometimes no valid Mac Equivalents. (Many

Virtualization and the U2 Databases Brian Kupzyk Senior Technical Support Engineer for Rocket U2 Nik Kesic Lead Technical Support for Rocket U2 Opening Procedure Orange arrow allows you to manipulate the

ovirt introduction FOSDEM 2013 Doron Fediuck Red Hat What is ovirt? Large scale, centralized management for server and desktop virtualization Based on leading performance, scalability and security infrastructure

Networks and Services Dr. Mohamed Abdelwahab Saleh IET-Networks, GUC Fall 2015 TOC 1 Infrastructure as a Service 2 Platform as a Service 3 Software as a Service Infrastructure as a Service Definition Infrastructure

How do Users and Processes interact with the Operating System? Users interact indirectly through a collection of system programs that make up the operating system interface. The interface could be: A GUI,

Customer Site Requirements for incontact Workforce Optimization 16.2 www.incontact.com Customer Site Requirements for incontact Workforce Optimization Version 16.2 Last Revision June 2016 About incontact

Citrix Training Course: Citrix Training Duration: 40 hours Mode of Training: Classroom (Instructor-Led) Virtualization has redefined the way IT resources are consumed and services are delivered. It offers

What is VM Upload? 1. VM Upload allows you to import your own VM and add it to your environment running on CloudShare. This provides a convenient way to upload VMs and appliances which were already built.

The CC1 system Solution for private cloud computing 1 Outline What is CC1? Features Technical details Use cases By scientist By HEP experiment System requirements and installation How to get it? 2 What

VMWARE APPLICATION NOTE VMware Player Using VMware Player This document contains the following sections: Work and Play in a Virtual World on page 1 Options and Features in VMware Player on page 4 Installing

The XenServer Product Family: A XenSource TM White Paper Virtualization Choice for Every Server: The Next Generation of Server Virtualization The business case for virtualization is based on an industry-wide

1 Virtual Machines A virtual machine (VM) is a "completely isolated guest operating system installation within a normal host operating system". Modern virtual machines are implemented with either software

University of Victoria Faculty of Engineering Winter 2013 Work Term Report Shoal: IaaS Cloud Cache Publisher Department of Physics University of Victoria Victoria, BC Mike Chester V00711672 Work Term 3