A Local Outlier Detection Method Based on Objective Function

doi:10.12068/j.issn.1005-3026.2022.10.006

Abstract

Abstract: The traditional density based local outlier detection algorithm does not preprocess the original data set， which leads to the unsatisfactory detection effect when facing the unknown data set. Moreover， due to the need to calculate the outlier factor of each data point， the amount of calculation increases greatly when the amount of data is too large. Through the analysis of the local outlier detection algorithm， a local outlier detection method based on objective function FOLOF(FCM objective function-based LOF)is proposed. Firstly， the elbow rule is used to determine the optimal number of clusters in the data set.Then， the data set is pruned by the objective function of FCM to obtain the outlier candidate set.Finally， the weighted local outlier factor detection algorithm is used to calculate the outlier degree of each point in the candidate set. The relevant experiments are carried out on the artificial data set and UCI data sets. At the same time， the proposed method is compared with other methods.The results show that the proposed algorithm can improve the outlier detection accuracy， reduce the computational cost， and effectively achieve a better performance.

Key words: outlier detection; fuzzy C-means (FCM) algorithm; objective function; local outlier factor(LOF); pruning

CLC Number:

TP301.6

ZHOU Yu， ZHU Wen-hao， SUN Hong-yu. A Local Outlier Detection Method Based on Objective Function[J]. Journal of Northeastern University(Natural Science), 2022, 43(10): 1405-1412.

References

[1]Hawkins D.Identification of outliers［M］.Londen:Chapman and Hall，1980.
[2]Ji Z.Advancements of outlier detection:a survey［J］.ICST Transactions on Scalable Information Systems，2013，13(1):1-24.
[3]Vijayakumar V，Sri N，Sarojini P.Isolation forest and local outlier factor for credit card fraud detection system［J］.International Journal of Engineering and Advanced Technology，2020，4(9):261-265.
[4]Wang F，Wei Z，Zuo X.Anomaly IoT node detection based on local outlier factor and time series［J］.Computers， Materials & Continua，2020，2(64):1063-1073.
[5]Yuan X，Li J，Bai J，et，al.A local outlier factor-based detection of copy number variations from NGS data［J］.IEEE/ACM Transactions on Computational Biology and Bioinformatics，2019(99):1-1.
[6]吴强，张锐.基于局部异常因子的近地全天时星图小波去噪［J］.光学学报，2020，40(8):46-54.(Wu Qiang，Zhang Rui.Wavelet denoising of near earth all sky time star map based on local anomaly factor ［J］.Journal of Optics，2020，40(8):46-54.)
[7]Liu X，Cheung Y M，Peng S J，et al.Automatic mitral valve leaflet tracking in Echocardiography via constrained outlier pursuit and region-scalable active contours［J］.Neurocomputing，2014(144):47-57.
[8]周玉，朱文豪，房倩，等.基于聚类的离群点检测方法研究综述［J］，计算机工程与应用，2021，57(12):37-45.(Zhou Yu，Zhu Wen-hao，Fang Qian，et al.Research review of outlier detection methods based on clustering ［J］.Computer Engineering and Application，2021，57(12):37-45.)
[9]梅林，张凤荔，高强.离群点检测技术综述［J］，计算机应用研究，2020，37(2):3521-3527.(Mei Lin，Zhang Feng-li，Gao Qiang.Overview of outlier detection technology ［J］.Computer Application Research，2020，37(2):3521-3527.)
[10]Breuning M，Krigegel H P，Ng R，et al.LOF:identifying density based local outliers［C］//Proceeding of the ACM SIGMOD International Conference on Management of Data.Dallas，2000:93-104.
[11]Li L，Huang L，Yang W.Privacy-preserving LOF outlier detection［J］.Knowledge and Information Systems，2015，42(3):579-597.
[12]王敬华，赵新想，张国燕，等.NLOF:一种新的基于密度的局部离群点检测算法［J］.计算机科学，2013，40(8):181-185.(Wang Jing-hua，Zhao Xin-xiang，Zhang Guo-yan，et al.NLOF:a new density based local outlier detection algorithm ［J］.Computer Science，2013，40(8):181-185.)
[13]石鸿雁，马晓娟.改进的DBSCAN聚类和LAOF两阶段混合数据离群点检测方法［J］.小型微型计算机系统，2018，39(1):74-77.(Shi Hong-yan，Ma Xiao-juan.Improved DBSCAN clustering and LAOF two-stage mixed data outlier detection method ［J］.Small Microcomputer System，2018，39(1):74-77.)
[14]董泽，贾昊.基于EWT-LOF的热工过程数据异常值检测方法［J］.仪器仪表学报，2020，41(2):126-134.(Dong Ze，Jia Hao.Abnormal value detection method of thermal process data based on ewt-lof ［J］.Journal of Instrumentation，2020，41(2):126-134.)
[15]He Z Y，Xu X F， Deng S C.Discovering cluster-based local outliers［J］.Pattern Recognition Letters，2003，9(24):1641-1650.
[16]Li Q，Zhang J P，Feng G S .An outlier detection method based on fuzzy c-means clustering［J］.Key Engineering Materials，2010，419/420:165-168.
[17]Subramaniam M，Kathirvel A，Sabitha E，et al.Modified firefly algorithm and fuzzy c-means clustering based semantic information retrieval［J］.Journal of Web Engineering，2021，1(20):33-52.
[18]徐金东，赵甜雨，冯国政，等.基于上下文模糊C均值聚类的图像分割算法［J］，电子与信息学报.2021，43(7):2079-2086.(Xu Jin-dong，Zhao Tian-yu，Feng Guo-zheng，et al.Image segmentation algorithm based on context fuzzy c-means clustering ［J］.Journal of Electronics and Information，2021，43(7):2079-2086.)
[19]Kamolov A A，Park S.Prediction of depth of seawater using fuzzy c-means clustering algorithm of crowdsourced sonar data［J］.Sustainability，2021，13(11):5823-5823.
[20]Zhou Y，Ren Q C.Fuzzy c-means clustering algorithm for performance improvement of ENN［J］.　Cluster Computing，2019，22(5):11163-11174.
[21]王立英，石磊，伊静，等.NLOF:基于网格过滤的两阶段离群点检测算法［J］.计算机应用研究，2020，37(24):990-993.(Wang Li-ying，Shi Lei，Yi Jing，et al.NLOF:two-stage outlier detection algorithm based on grid filtering ［J］.Computer Application Research，2020，37(24):990-993.)
[22]Moh D，Bela L，Zoubi A L，et al.New outlier detection method based on fuzzy clustering［J］.WSEAS Transactions on Information Science and Applications，2010， 7(4/5/6): 681-690.
[23]Gan G J，Kwok-Po M.K-means clustering with outlier removal［J］.Pattern Recognition Letters，2017，90(15):8-14.
[24]Feng G，Li Z，Zhou W，et al.Entropy-based outlier detection using spark［J］.Cluster Computing，2020(23):409-419.
[25]古平，刘海波，罗志恒.一种基于多重聚类的离群点检测算法［J］.计算机应用研究，2013，30(3):751-753.(Gu Ping，Liu Hai-bo，Luo Zhi-heng.An outlier detection algorithm based on multi clustering［J］.Computer Application Research，2013，30(3):751-753.)