Stratos Idreos, Harvard - January 2018

NoSQL key-value stores are at the heart of numerous modern data-driven applications. Prof Stratos Idreos research shows that all existing designs are sub-optimal. They work towards a completely new data system design, CrimsonDB, which is 1) 10x faster than existing systems and gets faster for bigger data, 2) requires less hardware resources (memory) to produce the same or better results, and 3) it is automatically adaptable to new hardware and workloads, requiring no human-in-the-loop to tune the system. 

The innovation comes from a full break down of the “design space” of LSM-trees which is the primary data structure in log based storage. Through this break down they formalize the design space so that they can algorithmically and with closed form formulas argue about possible tunings as well as discover optimization opportunities.