Big high-dimension data cube designs for hybrid memory systems

Author(s)

Silva, Rodrigo Rocha ; Hirata, Celso Massaki ; Lima, Joubert de Castro

• Document type: Journal article
• Source: KNOWLEDGE AND INFORMATION SYSTEMS; v. 62, n. 12, p. 30-pg., 2020-08-26.

Abstract

In Big Data cubes with hundreds of dimensions and billions of tuples, the indexing and query operations are a challenge and the reason is the time-space exponential complexity when a full cube is computed. Therefore, solutions based on RAM may not be practical and the solutions based on hybrid memory (RAM and disk) become viable alternatives. In this paper, we propose a hybrid approach, named bCubing, to index and query high-dimension data cubes with high number of tuples in a single machine and using RAM and disk memory systems. We evaluated bCubing in terms of runtime and memory consumption, comparing it with the Frag-Cubing, HIC and H-Frag approaches. bCubing showed to be faster and used less RAM than Frag-Cubing, HIC and H-Frag. bCubing indexed and allowed to query a data cube with 1.2 billion tuples and 60 dimensions, consuming only 84 GB of RAM, which means 35% less memory than HIC. The complex holistic measures mode and median were computed in multidimensional queries, and bCubing was, on average, 50% faster than HIC. (AU)

• FAPESP's process: 12/04260-4 - An approach based on invertex indexes for the computation, query, and update of data cubes (I-Cubing) and a technique to transform mobile applications with multi-touch interface to meet requirements of collaboration (CoMultiTouch)
• Grantee: Celso Massaki Hirata
• Support Opportunities: Regular Research Grants