2 with Content sizes increase. The stakeholder and the developer decide increasing the network bandwidth Client Normal state reports 1.5 million requests /sec using SpecWEB benchmark reports 0.5 milli- second recovery time when faults happen reports 920 Mbps Faulty state Web server Heartbeat Stand-by The server must handle 1 million requests per second reports 1.5 million requests /sec using SpecWEB benchmark One more network link is added WEB Server is dependable Hardware components are argued HA server is used High availability and performance are argued When the system goes down, it must recover within 1 milli-second reports 0.5 millisecond recovery time when faults happen The network link offers at least 1.6 Gbps under the safe condition reports 1.8 Gbps in the normal state and 850 Mbps under one network link fails The network link offers at least 800Mbps reports 920 Mbps 3 An Overview of Anomaly situations hardware faults, software bugs, human errors Two Measurement Environments Target Hardware Specific Environment Cloud Environment Virtual Machines Key Features Benchmark Database Customization of Anomaly Scenario Benchmark Cloud 4

3 An Overview of Key Features Benchmark Database Customization of Anomaly Scenario Benchmark Cloud Benchmark Database Performance Benchmarks (Workload), Measurement Tools, Anomaly Generators, Anomaly Scripts Resultsof dependability d benchmarks The system maintainer can confirm whether the behavior observed at the operation phase has been reported during some previous anomaly situation. 5 An Overview of Key Features Benchmark Database Customization of Anomaly Scenario Benchmark Cloud Anomaly is constructed with anomaly generators and an anomaly script written in XML. An anomaly script gives instructions on how to use anomaly generators. A new anomaly script tfor a new application field is described using anomaly generators. Anomaly scripts may also be used and modified by other users for other target environments. 6

4 An Overview of Key Features Benchmark Database Customization of Anomaly Benchmark Cloud Considerable computing resources are required in order to test many dependability benchmarks. Some benchmarks may run on any computer that provides a virtual machine environment in which h a hardware simulator can operate. A benchmark cloud is a benchmark execution environment which uses general purpose computer resources, in which virtual machines run, and which can model application-specific hardware. Dependability benchmarks run in parallel with the benchmark cloud, and thus the benchmark cloud contributes to reducing the execution time of dependability benchmarks. 7 Limitations and Intentions The is only effective for anticipated anomaly conditions. It does not reveal weaknesses of the system in unexpected anomaly conditions It does not find the cause of such conditions. D-Logger D-Analyzer D-System D-Visor When an unknown cause of a new anomaly Monitor Anomalydetection Quick situation is faced, Analysis DEOS FrameworkTools 1. Finding the cause DEOS Process 2. Developing a program to generate the anomaly, Requirement/Environment i.e. anomaly load. Changeaccommodatingcycle 3. Registering the program in the database The range of testing for dependability in the target system is increased. Dependability in other systems is also increased using this new benchmark Stakeholders requirements/ System environment changes D-Effector D-Box Walker Evidence DCase Responsiveness management Execution Systemchangerequests basedonstakeholders agreement Causeanalysis Responsiveaction Failureprevention Design Anomalydetection/ Unexpectedfailure happen Editor DCase Update Implementation Type/Model Checker EvidencebyUnittest Failurereactingcycle Normal operation growthcycle Verification DS-Bench/ D-Cloud Evidenceby Integrationtest Test Achievementof accountability 8

5 The Rest of Talk Details of DS-Bench Details of D-Cloud 9 DETAILS OF DS-BENCH 10

8 DS-Bench: Supported Benchmarks and Anomaly Loads While benchmark programs and anomaly loads can be added to the framework by users, the following programs are supported by default. Name Description Bonnie++ Disk I/O benchmark cpustress A program just consuming a lot of CPU time (*) Hackbench System benchmark by generating a lot of processes simultaneously IMB Intel MPI Benchmark iperf Network bandwidth benchmark LinkRefuse Shutting off ports on a network switch (*) (*) Mainly used as anomaly loads LMBench System performance benchmark memstress A program just allocating a lot of memory (*) MPD MPI daemon NetCMD Adjusting network output bandwidth, injecting packet loss and packet reordering (*) NPB NAS Parallel Benchmark SupplyRefuse Shutting down power distribution to a target machine (*) terminator A program to unconditionally kill the specified process (*) 2010/11/4 Version 15 DETAILS OF D-CLOUD 16

12 Example Using D-Cloud with SpecC Device Model Simulation with SpecC device model can be realized. SpecC Hardware status SpecC description Linux console on VM Keypad Display Linux on VM(QEMU) select() read() User program Keypad Device Driver Interrupt signal SpecC simulator (SCRC) Interrupt Control Status / Control Registers Socket TCP write() Display Device Driver Output buffer Socket TCP 23 Future R&D Plan Description of Evidence for An evidence provided by is currently the set of the benchmark description and its result. How to describe evidences for to handle them How the stakeholder and developer agree using the results of Testing Coverage Integration of 24

Windows 2003 Performance Monitor The performance monitor, or system monitor, is a utility used to track a range of processes and give a real time graphical display of the results, on a Windows 2003 system.

3. The Lagopus SDN Software Switch Here we explain the capabilities of the new Lagopus software switch in detail, starting with the basics of SDN and OpenFlow. 3.1 SDN and OpenFlow Those engaged in network-related

The Monitis Monitoring Agent ver. 1.2 General principles, Security and Performance Monitis provides a server and network monitoring agent that can check the health of servers, networks and applications

WHITE PAPER IxChariot Virtualization Performance Test Plan Test Methodologies The following test plan gives a brief overview of the trend toward virtualization, and how IxChariot can be used to validate

Integration of and Mission- Critical IA Server V Masaru Sakai (Manuscript received May 20, 2005) Information Technology (IT) systems for today s ubiquitous computing age must be able to flexibly accommodate

Operating System for the K computer Jun Moroo Masahiko Yamada Takeharu Kato For the K computer to achieve the world s highest performance, Fujitsu has worked on the following three performance improvements

Ongoing evolution of Linux x86 machine check handling Sept. 2009 Andi Kleen LinuxCon 2009 What's a good error? User has to see it, of course That can be surprisingly difficult Also psychological barriers

2011 European HyperWorks Technology Conference Vladi Nosenzo, Roberto Vadori 20 Novembre, 2010 2011 ABSTRACT The work described below starts from an idea of a previous experience of Reply, developed in

WiFiPerf User Guide 1.5 AccessAgility LLC 2012 AccessAgility LLC. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any

Using VMWare VAAI for storage integration with Infortrend EonStor DS G7i Application Note Abstract: This document describes how VMware s vsphere Storage APIs (VAAI) can be integrated and used for accelerating

Performance and Bandwidth Testing for Data Circuits In some cases, customers report that they are experiencing slower than expected data connections. These can be caused by any number of factors, including

ServerPronto Cloud User Guide Virtual machines Virtual machines are based on templates and are deployed on hypervisors. Hypervisors give them access to CPU, disk and network resources. The ServerPronto

Spot server problems before they are noticed The system s really slow today! How often have you heard that? Finding the solution isn t so easy. The obvious questions to ask are why is it running slowly

Deploy the ExtraHop Discover Appliance with Hyper-V 2016 ExtraHop Networks, Inc. All rights reserved. This manual, in whole or in part, may not be reproduced, translated, or reduced to any machine-readable

1 VMWARE WHITE PAPER Introduction This paper outlines the considerations that affect network throughput. The paper examines the applications deployed on top of a virtual infrastructure and discusses the

Optimizing Linux Performance Why is Performance Important Regular desktop user Not everyone has the latest hardware Waiting for an application to open Application not responding Memory errors Extra kernel

Performance Evaluation of Linux Bridge James T. Yu School of Computer Science, Telecommunications, and Information System (CTI) DePaul University ABSTRACT This paper studies a unique network feature, Ethernet

Objectives At the end of this chapter, participants will be able to understand: Web server management options provided by Network Deployment Clustered Application Servers Cluster creation and management

16 Example of Standard API System Call Implementation Typically, a number associated with each system call System call interface maintains a table indexed according to these numbers The system call interface

NOUS INFOSYSTEMS LEVERAGING INTELLECT Mobile Performance Testing Approaches and Challenges ABSTRACT Mobile devices are playing a key role in daily business functions as mobile devices are adopted by most

Laboratory 12 Applications Network Application Performance Analysis Objective The objective of this lab is to analyze the performance of an Internet application protocol and its relation to the underlying