Database Cracking

http://research.microsoft.com/apps/video/dl.aspx?id=148616
Adaptive Indexing targets dynamic environments where there is no workload knowledge and there is not enough time to invest in physical design preparations and tuning, e.g., due to very large data sets. With adaptive indexing, each query is seen as an advice of how data should be stored. With each incoming query, data is reorganized on-the-fly as part of the query operators. Future queries can exploit and enhance this knowledge. Autonomously, adaptively and without any external human administration, the system continuously adjusts to ever changing workload patterns, updates and storage restrictions. Adaptive indexing is designed on top of modern column-store architectures exploiting several new features such as one column at a time processing, vectorization, late tuple reconstruction and cache conscious algorithms.