Parallelism Strategies for Big Data Delayed Transfer Entropy Evaluation

Author(s)

Dourado, Jonas R. ^[1] ; de Oliveira Junior, Jordao Natal ^[1] ; Maciel, Carlos D. ^[1]

Affiliation

^[1] Univ Sao Paulo, Dept Elect & Computat Engn, BR-13566590 Sao Carlos, SP - Brazil

• Document type: Journal article
• Source: ALGORITHMS; v. 12, n. 9 SEP 2019.
• Web of Science Citations: 1

Abstract

Generated and collected data have been rising with the popularization of technologies such as Internet of Things, social media, and smartphone, leading big data term creation. One class of big data hidden information is causality. Among the tools to infer causal relationships, there is Delay Transfer Entropy (DTE); however, it has a high demanding processing power. Many approaches were proposed to overcome DTE performance issues such as GPU and FPGA implementations. Our study compared different parallel strategies to calculate DTE from big data series using a heterogeneous Beowulf cluster. Task Parallelism was significantly faster in comparison to Data Parallelism. With big data trend in sight, these results may enable bigger datasets analysis or better statistical evidence. (AU)

• FAPESP's process: 14/50851-0 - INCT 2014: National Institute of Science and Technology for Cooperative Autonomous Systems Applied in Security and Environment
• Grantee: Marco Henrique Terra
• Support type: Research Projects - Thematic Grants
• FAPESP's process: 18/19150-6 - Resilience of complex systems with the use of dynamic Bayesian Networks: a probabilistic approach
• Grantee: Carlos Dias Maciel
• Support type: Scholarships abroad - Research