Theory and practice of learning-based compressed data structures

Abstract

We revisit two fundamental and ubiquitous problems in data structure design: predecessor search and rank/select primitives. We show that real data present a peculiar kind of regularity based on geometric considerations. We name it “approximate linearity”. We thus expand the horizon of compressed data structures by presenting two solutions for the problems above that discover, or “learn”, in a principled algorithmic way, these approximate linearities. We provide a walkthrough of these new theoretical achievements, also with a focus on open-source libraries and their experimental improvements. We conclude by discussing the plethora of research opportunities that these new learning-based approaches to data structure design open up.