Saturday, October 28, 2006

ZFS and SAM-QFS

With ZFS becoming popular and Sun StorageTek's SAM-QFS also rising in its own market, there are often questions from ISV's and customers who try to understand why Sun is having multiple technologies in File Systems arena. While the answer "Customer Choice" is a valid answer, lets talk about the philosophies pursued in both to make it more clear.

To explain the philosphical differences between ZFS and SAM-QFS, I would like to take the analogy of Virtual Memory and Memory Heirarchy Caching in Computer systems. Virtual Memory makes an abstraction of RAM -Random Accessible Memory and allows it to be used at an abstract level instead of tied of the physical memory. To use more memory you just add more DIMMs and the OS fits them into the overall realm of the Virtual Memory mapping. ZFS tries to something similar in the concept of Pool of devices and ZFS with its 128 bit mapping fits the actual physical spindles into the overall realm of the pool.

On the otherhand the memory heirarchy caching works with the fastest memory on the CPU, then the Level 2 cache on Chip, then the Level 3 cache off chip (if any) and finally the physical RAM. To get the fastest access the memory location has to be on the Level 1 cache on the CPU if it is not out there, it is checked in the Level 2 cache and if it is not out there it is checked on Level 3 cache and if not out there it checks from the physical RAM. If it finds it on the RAM it brings it to the level 1 cache. And eventually, the same memory reference location of the virtual memory. Similarly things happens with files in SAM-FS. It has different hierarchies of data storage depending on the cost and performance of the underlying storage like fiber channel disks, SATA drives and Tape drives (all products that are available from Sun). Depending on the policies set, a file is "aged" out from the fastest to the cheapest storage level automatically without user intervention and brought from the cheapest level to the fastest storage heirarchy level "on demand" and the aging process begins all over again.

So now the question comes how (if not now in near future) should SAM-QFS and ZFS work. The easiest way is SAM-QFS using ZFS pool as a heirachy category. Which means a ZFS pool of Fiber Channel Drives, another pool of SATA Drives and SAM-FS manages the heirarchy of the storages for "best" overall usage of the speed and cost of the storage system (the cheapest being the Sun StorageTek tape products) that all happens auto-magically within the system.

Of course I haven't really talked about the "QFS" part of SAM-QFS but that is like bringing SMP capability with its cache coherency into system. More about that in another entry.