3 YFS (Reminder) Instructional distributed file system developed by MIT after a research distributed file system, called Frangipani Analogous to xv6 for OS courses When we discuss YFS, we really refer to Frangipani (or a simplified version thereof) YFS/Frangipani Goals: Aggregate many disks from many s Incrementally scalable Tolerates and recovers from node, network, disk failures 3

15 Lab 1: Lock Server Lock service consists of: Lock : grant a lock to clients, one at a time Lock client: talk to to acquire/release locks Correctness: At most one lock is granted to any client Additional requirement: acquire() at client does not return until lock is granted 15

16 Lab 1 Steps Step 1: Implement lock and client lock code Use in-house RPC mechanism: It s a bit more primitive than Thrift and the like (see next slide) Step 2: Implement at-most-once semantics How do we do that? Due next Friday, Sept 28 before midnight Absolutely no extensions! Lab is significantly more difficult than Lab 0, so start working on it NOW! Work in layers, submit imperfect but on time! 16

19 Thread Synchronization (Reminder) What s a thread? What s a race condition? How do we avoid race conditions? Locks are very low level, and often times you need more to accomplish a synchronization goal Hence, people have developed a variety of synchronization primitives that raise level of abstraction 19

20 Semaphores Integer variable x that allows interaction via 2 operations: x.p(): while (x == 0) wait; --x x.v(): ++x Both operations are done atomically All steps take place without any intervening operations Question: How do you implement a semaphore? When do we use semaphores? 20

21 Example: Thread-Safe FIFO Queue q.initialize(): initialize state q.insert(x): add item into queue q.remove(): block until queue not empty; return item at head of queue q.flush(): clear queue Assume that we already have a sequential queue implementation: sq But sq is not thread-safe! So, how do we make q thread-safe? 21

27 Condition Variables Condition variables provide synchronization point, where one thread suspends until activated by another Condition variable always associated with a mutex cvar.wait(): Must be called after locking mutex Atomically: { release mutex & suspend operation } When resume, lock the mutex (may have to wait for it) cvar.signal(): If no thread suspended, then no-op Wake up one suspended thread 27

30 Synchronization in Distributed Systems As we ve already seen in YFS Lab, distributed systems have similar issues: Multiple processes on different machines share the same resource: the data (or a printer, or the user s screen, ) Synchronization is even hairier in distributed systems, as you have to worry about failures, not just overlappings E.g.: when you get a lock, you may not even know you ve gotten it 30

31 Analogy: The Generals Dilemma e.g., carrier pigeon Let s attack together at 0500pm OK Hmm, should I attack? Did he get my OK? Same with distributed systems: machines need to agree on how to progress via an unreliable medium Things can get even messier if generals/machines can also fail (or even worse, go rogue) 31

32 Distributed Synchronization Mechanisms Logical clocks: clock synchronization is a real issue in distributed systems, hence they often maintain logical clocks, which count operations on the shared resource Consensus: multiple machines reach majority agreement over the operations they should perform and their ordering Data consistency protocols: replicas evolve their states in predefined ways so as to reach a common state despite different views of the input Distributed locking services: machines grab locks from a centralized, but still distributed, locking service, so as to coordinate their accesses to shared resources (e.g., files) Distributed transactions: an operation that involves multiple services either succeeds or fails at all of them We re going to look at all of these in the following lectures 32

We mean.network File System Introduction: Remote File-systems When networking became widely available users wanting to share files had to log in across the net to a central machine This central machine

Last Class: Semaphores A semaphore S supports two atomic operations: S Wait(): get a semaphore, wait if busy semaphore S is available. S Signal(): release the semaphore, wake up a process if one is waiting

Lab 2 : Basic File Server Introduction In this lab, you will start your file system implementation by getting the following FUSE operations to work: CREATE/MKNOD, LOOKUP, and READDIR SETATTR, WRITE and

Last Class: Synchronization Synchronization Mutual exclusion Critical sections Locks Synchronization primitives are required to ensure that only one thread executes in a critical section at a time. Lecture

CS11 Java Fall 2014-2015 Lecture 7 Today s Topics! All about Java Threads! Some Lab 7 tips Java Threading Recap! A program can use multiple threads to do several things at once " A thread can have local

1 1 Distributed Systems What are distributed systems? How would you characterize them? Components of the system are located at networked computers Cooperate to provide some service No shared memory Communication

BELONGS TO THE CYGNUS SOLUTIONS founded about 1989 initiative connected with an idea of free software ( commercial support for the free software ). Recently merged with RedHat. CYGNUS was also the original

Principles and characteristics of distributed systems and environments Definition of a distributed system Distributed system is a collection of independent computers that appears to its users as a single

Distributed File Systems Chapter 10 Distributed File System a) A distributed file system is a file system that resides on different machines, but offers an integrated view of data stored on remote disks.

Bertrand Mermet Sylvain Ract Linux Driver Devices. Why, When, Which, How? Since its creation in the early 1990 s Linux has been installed on millions of computers or embedded systems. These systems may

I/O Input/Output One of the functions of the OS, controlling the I/O devices Wide range in type and speed The OS is concerned with how the interface between the hardware and the user is made The goal in

Last Class: Introduction to Operating Systems User apps OS Virtual machine interface hardware physical machine interface An operating system is the interface between the user and the architecture. History

Running a Workflow on a PowerCenter Grid 2010-2014 Informatica Corporation. No part of this document may be reproduced or transmitted in any form, by any means (electronic, photocopying, recording or otherwise)

Operating Systems 4 th Class Lecture 1 Operating Systems Operating systems are essential part of any computer system. Therefore, a course in operating systems is an essential part of any computer science

The Operating System Level Virtual Memory File systems Parallel processing Case studies Due 6/3: 2, 3, 18, 23 Like other levels we have studied, the OS level is built on top of the next lower layer. Like

Processes and Non-Preemptive Scheduling Otto J. Anshus 1 Concurrency and Process Challenge: Physical reality is Concurrent Smart to do concurrent software instead of sequential? At least we want to have

Introduction Distributed Data Management Involves the distribution of data and work among more than one machine in the network. Distributed computing is more broad than canonical client/server, in that

Monitors, Java, Threads and Processes 185 An object-oriented view of shared memory A semaphore can be seen as a shared object accessible through two methods: wait and signal. The idea behind the concept

CSC 2233: Topics in Computer System Performance and Reliability: Storage Systems! Note: some of the slides in today s lecture are borrowed from a course taught by Greg Ganger and Garth Gibson at Carnegie

Review from last time CS 537 Lecture 3 OS Structure What HW structures are used by the OS? What is a system call? Michael Swift Remzi Arpaci-Dussea, Michael Swift 1 Remzi Arpaci-Dussea, Michael Swift 2

AC 2009-854: TEACHING AN OPERATING SYSTEM COURSE TO CET/EET STUDENTS Xuefu Zhou, University of Cincinnati American Society for Engineering Education, 2009 Page 14.1124.1 Teaching an Operating System Course

The Google File System (GFS) Google File System Example of clustered file system Basis of Hadoop s and Bigtable s underlying file system Many other implementations Design constraints Motivating application:

Network Attached Storage Jinfeng Yang Oct/19/2015 Outline Part A 1. What is the Network Attached Storage (NAS)? 2. What are the applications of NAS? 3. The benefits of NAS. 4. NAS s performance (Reliability

OPERATING SYSTEMS IIIT-Hyderabad OVERVIEW Introduction What is an OS/Kernel? Bootstrap program Interrupts and exceptions Volatile and Non volatile storage!!! Process Management What is a process/system

GOJAN SCHOOL OF BUSINESS AND TECHNOLOGY DEPARTMENT OF INFORMATION TECHNOLOGY CS2411-OPERATING SYSTEM QUESTION BANK UNIT-I (PROCESSES AND THREADS) 1. What is an Operating system? What are the various OS

Middleware: Past and Present a Comparison Hennadiy Pinus ABSTRACT The construction of distributed systems is a difficult task for programmers, which can be simplified with the use of middleware. Middleware

Real Time Programming: Concepts Radek Pelánek Plan at first we will study basic concepts related to real time programming then we will have a look at specific programming languages and study how they realize

Class 305 Focus On Threads: An Introduction Michel de Champlain ABSTRACT Threads are now supported by many new object-oriented languages. Java provides language-level support for multi-threading, resulting