Today the bulk of data in the cloud is still stored on magnetic storage media in the form of tape for archival data and Hard Disk Drives (HDD) for online data as magnetic media still provides the lowest cost storage. However, the areal density scaling that has driven the cost reduction in magnetic media over the last 40 years is now reaching its fundamental physical limits. In addition to this the lifetime of magnetic tape media (5-10 years) is too short for archival applications where a storage lifetime more than 100 years is required particularly as the amount of archival data scales into the ZettaByte region. For online data where faster access rates are required, we see that as hard disk drive capacity continues to scale, albeit more slowly than in the past, the data access rates per unit of stored data, measured in IOPS/TB, are declining. The access rates provided by the latest generation of 20TB HDDs are already too low to support all online workloads in the cloud. This is necessitating the deployment of more expensive hybrid storage solutions that use a combination of HDD and Solid State Drives (SSD) to achieve the required access rate performance. However, SSD capacity scaling is also slowing, as we come to the end of Moore’s Law and as such are unlikely to ever reach the same cost per GB as HDDs.

At Microsoft Research, we have been exploring new storage materials and methods of accessing them to design storage systems that meet the performance requirements of cloud storage. In this presentation, I will cover the work we have done in: Project Silica, where we have used the data storage durability of voxels written by femtosecond lasers in Silica Glass to create a new long lifetime archival storage technology. Our research to understand if we can use Holographic optical storage with Iron doped Lithium Niobate for online storage applications and the remaining challenges for such systems. Finally, I will present the framework that we have developed to evaluate the potential of future storage materials to address the challenges that we foresee for cloud scale storage applications.