A1 Journal article (refereed)
Toolbox for Distance Estimation and Cluster Validation on Data With Missing Values (2022)

Niemelä, M., Äyrämö, S., & Kärkkäinen, T. (2022). Toolbox for Distance Estimation and Cluster Validation on Data With Missing Values. IEEE Access, 10, 352-367. https://doi.org/10.1109/ACCESS.2021.3136435

JYU authors or editors

Publication details

All authors or editors: Niemelä, Marko; Äyrämö, Sami; Kärkkäinen, Tommi

Journal or series: IEEE Access

eISSN: 2169-3536

Publication year: 2022

Volume: 10

Pages range: 352-367

Publisher: Institute of Electrical and Electronics Engineers (IEEE)

Publication country: United States

Publication language: English

DOI: https://doi.org/10.1109/ACCESS.2021.3136435

Research data link: https://github.com/markoniem/nanclustering_toolbox

Publication open access: Openly available

Publication channel open access: Open Access channel

Publication is parallel published (JYX): https://jyx.jyu.fi/handle/123456789/79601

Abstract

Missing data are unavoidable in the real-world application of unsupervised machine learning, and their nonoptimal processing may decrease the quality of data-driven models. Imputation is a common remedy for missing values, but directly estimating expected distances have also emerged. Because treatment of missing values is rarely considered in clustering related tasks and distance metrics have a central role both in clustering and cluster validation, we developed a new toolbox that provides a wide range of algorithms for data preprocessing, distance estimation, clustering, and cluster validation in the presence of missing values. All these are core elements in any comprehensive cluster analysis methodology. We describe the methodological background of the implemented algorithms and present multiple illustrations of their use. The experiments include validating distance estimation methods against selected reference methods and demonstrating the performance of internal cluster validation indices. The experimental results demonstrate the general usability of the toolbox for the straightforward realization of alternate data processing pipelines. Source code, data sets, results, and example macros are available on GitHub. https://github.com/markoniem/nanclustering_toolbox

Keywords: machine learning; algorithms; data; quality; validation; clusters; data processing; modelling (representation)

Free keywords: missing values; distance estimation; clustering; cluster validation

Contributing organizations

Related projects

Ministry reporting: Yes

Reporting Year: 2022

JUFO rating: 2