大数据环境下的不确定数据流在线分类算法

doi:10.12068/j.issn.1005-3026.2016.09.007

东北大学学报:自然科学版 ›› 2016, Vol. 37 ›› Issue (9): 1245-1249.DOI: 10.12068/j.issn.1005-3026.2016.09.007

大数据环境下的不确定数据流在线分类算法

吕艳霞，王翠荣，王聪，于长永

(东北大学信息科学与工程学院，辽宁沈阳110819)

收稿日期:2015-05-24 修回日期:2015-05-24 出版日期:2016-09-15 发布日期:2016-09-18
通讯作者: 吕艳霞
作者简介:吕艳霞(1982-)，女，河北沧州人，东北大学讲师，博士; 王翠荣(1964-)，女，河北唐山人，东北大学教授.
基金资助:
国家自然科学基金资助项目(61300195)；河北省自然科学基金资助项目(F2014501078)；辽宁省教育厅科学研究资助项目(L2013099)；东北大学秦皇岛分校科研基金资助项目(XNK201402).

Online Classification Algorithm for Uncertain Data Stream in Big Data

LYU Yan-xia， WANG Cui-rong， WANG Cong， YU Chang-yong

School of Information Science & Engineering， Northeastern University， Shenyang 110819， China.

Received:2015-05-24 Revised:2015-05-24 Online:2016-09-15 Published:2016-09-18
Contact: LYU Yan-xia
About author:-
Supported by:
-

摘要/Abstract

摘要： 在大数据环境下，由于隐私保护、数据丢失等原因，数据普遍存在不确定性；数据流系统中数据不断地到达系统，只扫描一遍且不能一次性全部获得；所以要构建一个增量分类模型来处理不确定数据流分类.本文基于VFDT算法提出了WBVFDTu算法，该算法在学习和分类阶段都可快速而有效地分析不确定信息.在学习期间，采用Hoeffding分解定理构造决策树模型；在分类期间，在决策树的叶子节点利用加权贝叶斯分类算法提高模型的分类准确率和算法的执行效率.最终证明该算法能够非常快速地学习不确定数据流，提高分类的准确率.

关键词: 不确定数据流, 加权贝叶斯, VFDT, 分类算法, 大数据

Abstract: Under the background of big data， there exist data uncertainties due to privacy protection， data loss and so on. In data stream system， data arrive at continuously and cannot be obtained all. In addition， all the inforation cannot be aquired with only one scan. Therefore， an incremental classification model is constructed to deal with uncertain data stream classification. The weighted Bayes based on VFDT (very fast decision tree) for uncertain data stream—WBVFDTu on the basis of VFDT algorithm is presented in the paper. The uncertain information can be analysed quickly and effectively in both the learning stage and classification stage. In the learning stage， a decision tree model for uncertain data stream is quickly constructed by using Hoeffding bound theory. In the classification stage， the weighted Bayes classifier in the tree leaves is used to improve the performance of the classification. Experimental results show that the proposed algorithm can very quickly learn uncertain data stream and improve the classification performance of the model.

Key words: uncertain data stream, weighted Bayes, VFDT(very fast decision tree), classification algorithm, big data

中图分类号:

TP311

吕艳霞，王翠荣，王聪，于长永. 大数据环境下的不确定数据流在线分类算法[J]. 东北大学学报:自然科学版, 2016, 37(9): 1245-1249.

LYU Yan-xia， WANG Cui-rong， WANG Cong， YU Chang-yong. Online Classification Algorithm for Uncertain Data Stream in Big Data[J]. Journal of Northeastern University Natural Science, 2016, 37(9): 1245-1249.

参考文献

[1]Tsang S，Kao B，Yip K Y，et al.Decision trees for uncertain data［J］.Knowledge & Data Engineering IEEE Transactions，2009，23(1):64-78.
[2]Hulten G，Spencer L，Domingos P.Mining time changing data streams［C］// Process of the Seventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.［S.l.］:ACM，2001:97-106.
[3]Qin B，Xia Y，Li F.DTU:a decision tree for uncertain data［J］.Advances in Knowledge Discovery and Data Mining，2009，5476:4-15.
[4]Gao C，Wang J.Direct mining of discriminative patterns for classifying uncertain data［C］// Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.［S.l.］:ACM，2010:861-870.
[5]Cao K Y，Wang G，Han D.An algorithm for classification over uncertain data based on extreme learning machine［J］. Neurocomputing，2016，174:194-202.
[6]Liang C，Zhang Y，Shi P，et al.Learning very fast decision tree from uncertain data streams with positive and unlabeled samples［J］. Information Sciences， 2012， 213(23):50-67.
[7]刘三民，孙知信，刘涛.基于样本不确定性的增量式数据流分类研究［J］.小型微型计算机系统，2015(2):193-196.(Liu San-min，Sun Zhi-xin，Liu Tao.Research of incremental data stream classification based on sample uncertainty［J］.Journal of Chinese Computer Systems，2015(2):193-196.)
[8]Pan S，Wu K，Zhang Y，et al.Classifier ensemble for uncertain data stream classification［J］.Lecture Notes in Computer Science，2010，6118(1):488-495.
[9]Hoeffding W.Probability inequalities for sums of bounded random variables［J］.Journal of the American Statistical Association，1962，58(301):13-30.
[10]He J，Zhang Y，Shi X L P.Learning naive Bayes classifiers from positive and unlabelled examples with uncertainty［J］.International Journal of Systems Science，2012，43(10):1805-1825.
[11]Liang C，Zhang Y，Hu P S Z.Learning accurate very fast decision trees from uncertain data streams［J］.International Journal of Systems Science，2015，46(16):3032-3050.
[12]卢惠林.基于加权Bayes分类器的流数据在线分类算法研究［J］.计算机科学，2014，41(5):227-229. (Lu Hui-lin.Weighted Bayes based data streaming online classification algorithm［J］.Computer Science，2014，41(5):227-229.)
[13]Aggarwal C C，Yu P S.A survey of uncertain data algorithms and applications［J］.IEEE Transactions on Knowledge & Data Engineering，2009，21(5):609-623.

大数据环境下的不确定数据流在线分类算法

Online Classification Algorithm for Uncertain Data Stream in Big Data

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 5

编辑推荐

Metrics

本文评价

[1]	黄川，胡平，连静. 一种基于车载信号还原机动车道3D地图的大数据方法[J]. 东北大学学报:自然科学版, 2020, 41(6): 771-777.
[2]	赵海，窦圣昶，李大舟，陈星池. 基于Lognormal函数的脉搏波数学建模[J]. 东北大学学报:自然科学版, 2016, 37(2): 169-173.
[3]	韩东红，王坤，邵崇雷，马畅. 一种面向不确定数据流的聚类算法[J]. 东北大学学报:自然科学版, 2016, 37(12): 1677-1682.
[4]	吴思炜，曹光明，周晓光，刘振宇. 基于大数据的C-Mn钢数据预处理及神经网络模型[J]. 东北大学学报:自然科学版, 2016, 37(12): 1710-1715.
[5]	张明卫;王波;张斌;朱志良;. 基于相关系数的加权朴素贝叶斯分类算法[J]. 东北大学学报(自然科学版), 2008, 29(7): 952-955.