MMU Final Presentation

Comments (0)

Transcript of MMU Final Presentation

l k Team E44 The Massive Memory Unit MembersRyan FeilJason FongDavid PolyakPaul SpillaneJack Weisse Advisor: Dr. Dennis Brylow Introduction The problem to be solved Network servers rely on largerepositories of data. The hard drive then seems to be the bottleneck in the system. Access to these drivestends to be slow. A new 7200 rpm hard drive cantransfer at around 75-80 MB/s A 1 Gb ethernet connection cantransfer at around 110-115 MB/s. This is a problem. So what's the solution? Why is it a problem? The Prototype What the device currently is. The Results How the prototype performs. Imagine you work for a company that employes over 1,000 people. Each of these people is going to be accessing data from somewhere on a company server on a daily basis. Many of these people are going to be making frequent transfers or transfers of rather large files. How much time (and money) does your company waste simply waiting for data to come from this server? We'll come back to this idea shortly. There's already a commonly used storage medium which is much faster than a hard drive An estimate* shows RAM as having a transfer rate of over 1 GB/s. *(800 MHz RAM with 64-bit channel / 2 clock cycles) Remember:A 7200 RPM hard drive’s actual transfer rate is around 75-80 MB/s. Why wouldn't you use RAM as a storage medium? 1 TB HD at $70 = $0.07/GB

8GB DDR2 RAM at $130 = $16.25/GB

RAM is over 230x more expensive per GB than a HD RAM is volatile.

That is, it stores its data only so long as it is being powered. If you made a network device that used RAM as it's primary storage medium. You would get our project: The MMU. Where we were at the start of the semester. The Current Prototype: The Prototype's Software How it works. Operating System: Linux, Ubuntu 9.10 (64-bit)

CPU: 3.0 GHz Intel Core 2 Quad

RAM: 8GB 800 MHz DDR2

HD: 1 TB, 7200 rpm

Tools: Custom coded server application and file transfer protocol. Putting together special purpose device would be expensive to the group, which has no actual budget. We repurposed a group member's computer and added additional RAM. A startup script which:Creates a temporary file system (TFS) in the prototype's RAM.Sets the TFS's size to user specifications.Mounts the TFS to the prototype's file system. A Cron* job which:Runs every 5 minutes (the time is editable by the user).Calls a script which runs an rsync** command on the MMU with a destination on the prototype's hard drive. *Cron is a tool scheduling tool for Linux.** rsync is a file backup command in Linux which is based on file differences. A server application which:Initializes itself on a user specified port and waits for a request for transfer.Handshakes and establishes a socket between the server and a requesting client.Locates requested files on the prototype's TFS and serializes it for network transfer.Transfers requested files to a client. A client application which:Initializes itself on a user specified port and attempts to meet the server at that port.Handshakes and establishes a socket between the itself and the server.Requests a specific file from the server.Receives and stores the requested file once it has been transferred from the server. The prototype's hard drive was outperforming the TFS on some network transfer tests. The team did not know where the slowdown was taking place. We decided to devote the semester to research and testing so as to find the bottleneck in the system. The test: transferring a 3.3 GB .iso file from the server to the client. Problem Areas The Transfer Mechanism The Network Setup The team was using a protocol called Samba to handle transfers between client and server hard drives. The Solution: The MMU FTP A specilized file transfer protocol written by the team. The prototype and client were, last semester, connected via Cat5 crossover cable. For proper testing, the team decided this setup needed to be more realistic. The Solution: It was written because Samba doesn't directly reflect the MMU's primary purpose. It is lightweight, simple, and allows for a client to store a transferred file in either it's own RAM or the hard drive. Test Results The prototype is connected to the 1 Gb network here in the lab. Allows any computer on the same network as the prototype to connect for file transfers. Allows for multiple clients and more realistic testing. But first: a demonstration. Also: Samba does not allow a file to be transferred directly to a client's RAM. If a client were accessing the MMU it would not need to store the file on its own hard drive. By simply sending and receiving random data without the use of a storage medium, we approximated our network throughput.

Network throughput: 111.53 MB/sIdeal 1 Gb network: 128 MB/sEfficiency: 87% By transferring from a storage medium to RAM, we get a good idea of the medium's maximum transfer rate. Old 7200 rpm HD: 65.13s at 50.12 MB/s

New 7200 rpm HD: 39.81s at 81.96 MB/s The MMU's TFS*:3.62s at 900.68 MB/s As a storage medium: The MMU's TFS is1699.1% faster than an old 7200 rpm hard drive.999.72% faster than a new 7200 rpm hard drive. *800 MHz DDR2 RAM This also means that a 7200 rpm hard drive can not saturate a 1 Gb network. Our MMU's TFS can easily saturate a 1 Gb network. Testing over the network revealed: Hard Drive:Average time: 39.78sAverage speed: 82.08 MB/s The MMU:Average time: 29.18sAverage speed: 111.63 MB/s If the medium cannot saturate, it transfer's near the max trasfer rate of the medium. Significance Why you should care. Calculating Time Saved With the current speed difference:111 MB/s for a TFS81 MB/s for a hard drive A corporation scenario: 5000 employees, each accessing 1GB daily.

Given the transfer speed of a hard drive as 81 MB/s, an employee spends 12.64s/GB on disk I/O.

Given the transfer speed of the MMU as 111 MB/s, an employee spends 9.23s/GB on disk I/O. With that many employees, the company loses 4 hours, 44 minutes, 10 seconds per day by using a traditional hard drive. This means you lose around 1 month, 14 days, 23 hours, 26 minutes of disk I/O per year. Meaning of our results. Questions? This translates to $32,670. The MMU's TFS can transfer at 900.68 MB/s On a 1 Gb network, the MMU is 36% faster than a traditional hard drive. A 10 Gb network, at 100% efficiency, could transfer at 1280 MB/s If we were to use a 10 Gb network, the MMU could transfer at its RAM's full capacity. We not only discovered, but circumvented the project's bottleneck. We have achieved the goal that we set out to meet last semester. For a 10 Gb network, using 850 MB/s for the MMU in the same calculation: These savings are $109,607 per year