Smart big data framework for insight discovery

Big Data deal with new challenges such as data variety, data veracity (correct, incorrect, misleading, etc.) and data completeness (provide a single part of the overall information.). In fact, the knowledge discovered from a single source that can offer incorrect or incomplete data, may have a negative impact on the quality of decisions based on it. Therefore, integrating data coming from multiple sources allows verifying the veracity and ensuring the completeness of the results and thus improving the quality of analysis and enhancing business decisions. In this paper, we present a smart framework that falls within the Big Data value chain process and aims to improve the quality of analytical results by focusing two main concerns regarding Big Data Integration; data completeness and data veracity. The framework integrates Big Data in order to build a complete global and correct insight from heterogeneous sources. The paper presents two implementations of the framework in the context of urban and highway traffic management systems. © 2021