基于内容相关的条件函数依赖的一致性清洗方法

doi:10.12068/j.issn.1005-3026.2016.12.003

东北大学学报:自然科学版 ›› 2016, Vol. 37 ›› Issue (12): 1683-1687.DOI: 10.12068/j.issn.1005-3026.2016.12.003

基于内容相关的条件函数依赖的一致性清洗方法

杜岳峰¹，申德荣¹，张亮²，于戈¹

(1. 东北大学信息科学与工程学院，辽宁沈阳110819； 2. 中国人民解放军 65154部队，辽宁凌源122513)

收稿日期:2015-07-31 修回日期:2015-07-31 出版日期:2016-12-15 发布日期:2016-12-23
通讯作者: 杜岳峰
作者简介:杜岳峰(1986-)，男，辽宁沈阳人，东北大学博士研究生；申德荣(1964-)，女，辽宁铁岭人，东北大学教授，博士生导师；于戈(1962-)，男，辽宁大连人，东北大学教授，博士生导师.
基金资助:
国家重点基础研究发展计划项目(2012CB316201)；国家自然科学基金资助项目(61033007).

A Consistency Cleaning Method Based on Content-related Conditional Functional Dependencies

DU Yue-feng¹， SHEN De-rong¹， ZHANG Liang²， YU Ge¹

1. School of Information Science & Engineering， Northeastern University， Shenyang 110819， China; 2. PLA 65154 Troops， Lingyuan 122513， China.

Received:2015-07-31 Revised:2015-07-31 Online:2016-12-15 Published:2016-12-23
Contact: DU Yue-feng
About author:-
Supported by:
-

摘要/Abstract

摘要： 基于条件函数依赖提出了一种内容相关的条件函数依赖，并给出基于内容相关的条件函数依赖的一致性清洗方法.通过分析条件函数依赖之间的关系，将相关联的条件函数依赖合并组成内容相关的条件函数依赖.内容相关的条件函数依赖可以检测多条件值下的数据一致性问题并提供可用于一致性修复的参考值.同时，提出了一种一致性修复的代价模型.模型参考内容相关的条件函数依赖对应元组的实际情况进行修复，实现代价最优，同时保证数据一致性.通过在两组真实数据集上进行试验测试，证明提出的基于内容相关的条件函数依赖的一致性清洗方法能够准确地检测数据的一致性问题并加以修复.

关键词: 数据清洗, 条件函数依赖, 内容相关, 数据一致性, 修复代价模型

Abstract: Based on conditional functional dependencies， content-related conditional functional dependencies (CCFDs) and the consistency cleaning method were presented based on CCFDs. By analyzing the relationship of the conditional functional dependencies， the related conditional functional dependencies were combined into CCFDs. The CCFDs can not only detect the consistencies under multi-conditional values， but also provide reference values for the consistency repairing. A consistency repairing-cost model was presented. Then the data was corrected to be consistent with the minimal repairing cost according to the actual data. And the repaired results are approved accuracy for both the inconsistency detection and the inconsistency repairing via the experimental evaluation on two real-life datasets.

Key words: data cleaning, conditional functional dependency, content relativity, data consistency, repairing-cost model

中图分类号:

TP311.13

杜岳峰，申德荣，张亮，于戈. 基于内容相关的条件函数依赖的一致性清洗方法[J]. 东北大学学报:自然科学版, 2016, 37(12): 1683-1687.

DU Yue-feng， SHEN De-rong， ZHANG Liang， YU Ge. A Consistency Cleaning Method Based on Content-related Conditional Functional Dependencies[J]. Journal of Northeastern University Natural Science, 2016, 37(12): 1683-1687.

参考文献

[1]Fan W.Data quality:theory and practice［C］// Proceedings of International Conference on Web-Age Information Management.Berlin:Springer-Verlag，2012:1-16.
[2]Fan W，Geerts F.Foundations of data quality management［M］.San Rafael:Morgan & Claypool，2012:1-201.
[3]Du Y F，Shen D R，Nie T，et al.Discovering condition-combined functional dependency rules［C］// Proceedings of the 16th Asia-Pacific Web Conference.Berlin:Springer-Verlag，2014:247-257.
[4]Fan W，Geerts F，Jia X，et al.Conditional functional dependencies for capturing data inconsistencies［J］.ACM Transactions on Database Systems Tods Homepage，2008，33(2):1-44.
[5]Fan W，Geerts F，Li J，et al.Discovering conditional functional dependencies［J］.IEEE Transactions on Knowledge & Data Engineering，2011，23(5):683-698.
[6]Flesca S，Furfaro F，Parisi F.Consistency checking and querying in probabilistic databases under integrity constraints［J］.Journal of Computer & System Sciences，2014，80(7):1448-1489.
[7]Fan W，Ma S，Tang N，et al.Interaction between record matching and data repairing［J］.Journal of Data and Information Quality，2014，4(4):1-16.
[8]Dallachiesa M，Ebaid A，Eldawy A，et al.NADEEF:a commodity data cleaning system［C］// Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data.New York:ACM，2013:541-552.
[9]Fan W.Dependencies revisited for improving data quality［C］// Proceedings of the 27th ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems.Vancouver，2008:159-170.
[10]Bohannon P，Fan W，Flaster M，et al.A cost-based model and effective heuristic for repairing constraints by value modification［C］// Proceedings of the 2005 ACM SIGMOD International Conference on Management of Data.New York:ACM，2005:143-154.

基于内容相关的条件函数依赖的一致性清洗方法

A Consistency Cleaning Method Based on Content-related Conditional Functional Dependencies

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 2

编辑推荐

Metrics

本文评价

[1]	刘晓琴，王大志，张翠玲，宁一. 基于模型预测和溯因推理网络的电网故障诊断方法[J]. 东北大学学报:自然科学版, 2016, 37(4): 472-477.
[2]	谷峪;李晓静;吕雁飞;于戈;. 基于RFID应用的综合性数据清洗策略[J]. 东北大学学报(自然科学版), 2009, 30(1): 34-37.