HASSO: A highly-automated source selection and ordering system based on data quality factors

Big data integration gives access to a large number of data sources through a unified user interface. Answers include high-quality data, medium-quality data, and low-quality data. Selecting a subset of high accurate and consistent data sources and ordering them appropriately is critical to obtain as many high-quality answers as possible right after querying few data sources. However, the process of selecting and ordering data sources can be quite complicated and can present several challenges. The main challenge faced during that process is identifying the most adequate data quality factors to consider. In this paper, we present HASSO, a Highly-Automated Source Selection and Ordering System based on data quality factors. To produce consistent and high accurate answers, HASSO identifies, for each data source, its domain, data consistency and data accuracy using the schema matches. To maximize the total number of complete and non-redundant answers returned right after querying a small number of data sources, HASSO orders data sources in terms of their data overlap and in a decreasing order of their overall coverage. Experimental results in real-world domains show that HASSO produces high-quality answers at high speed. © 2020 IEEE.