Communications in Information & Systems

Common Pitfalls Using the Normalized Compression Distance: What to Watch Out for in a Compressor

Manuel Cebrián, Manuel Alfonseca, and Alfonso Ortega

Full-text: Open access

Abstract

Using the mathematical background for algorithmic complexity developed by Kolmogorov in the sixties, Cilibrasi and Vitanyi have designed a similarity distance named normalized compression distance applicable to the clustering of objects of any kind, such as music, texts or gene sequences. The normalized compression distance is a quasi-universal normalized admissible distance under certain conditions. This paper shows that the compressors used to compute the normalized compression distance are not idempotent in some cases, being strongly skewed with the size of the objects and window size, and therefore causing a deviation in the identity property of the distance if we don’t take care that the objects to be compressed fit the windows. The relationship underlying the precision of the distance and the size of the objects has been analyzed for several well-known compressors, and specially in depth for three cases, bzip2, gzip and PPMZ which are examples of the three main types of compressors: block-sorting, Lempel-Ziv, and statistic.

Article information

Source
Commun. Inf. Syst. Volume 5, Number 4 (2005), 367-384.

Dates
First available: 5 April 2007

Permanent link to this document
http://projecteuclid.org/euclid.cis/1175791028

Mathematical Reviews number (MathSciNet)
MR2343930

Zentralblatt MATH identifier
1116.68037

Citation

Cebrián, Manuel; Alfonseca, Manuel; Ortega, Alfonso. Common Pitfalls Using the Normalized Compression Distance: What to Watch Out for in a Compressor. Communications in Information & Systems 5 (2005), no. 4, 367--384. http://projecteuclid.org/euclid.cis/1175791028.


Export citation