Erez Zadok, Stony Brook University - September 2010

The value proposition of network storage has long been increasing bandwidth and IOPS for partitionable workloads by aggregating disks. When network latency, and even bandwidth, start to lag behind faster and larger local persistent stores such as Phase Change Memory or FLASH, the value proposition of network storage will have to change as blocking reads will be more quickly serviced by a local client-side IOPS tier than a network storage device. One of the most significant random read IOPS workloads will be out-of-RAM index lookups and insertions. There is a lot of interest in this area due to search, rapid file attribute indexing, deduplication, provenance, revision control, client-side cloud caching, and more. Therefore the number of indexes and the percentage of IOPS they will consume is only going to increase. This research work plans to conduct a comprehensive study of the performance of the most effective indexing technologies under mixed workloads across a multi-tier cache hierarchy including a client-side FLASH tier, a LAN network storage tier, and a remote backup or cloud tier. The project will explore several popular index technologies directly above a FLASH SSD and will provide an instrumentation framework suitable for capturing and analyzing IOPS-intensive workloads across a multi-tier storage architecture.