Institute of Mathematical Statistics Lecture Notes - Monograph Series

Scalable mining for classification rules in relational databases

Min Wang, Bala Iyer, and Jeffrey Scott Vitter

Full-text: Open access

Abstract

Data mining is a process of discovering useful patterns (knowledge) hidden in extremely large datasets. Classification is a fundamental data mining function, and some other functions can be reduced to it. In this paper we propose a novel classification algorithm (classifier) called MIND (MINing in Databases). MIND can be phrased in such a way that its implementation is very easy using the extended relational calculus SQL, and this in turn allows the classifier to be built into a relational database system directly. MIND is truly scalable with respect to I/O efficiency, which is important since scalability is a key requirement for any data mining algorithm.

We have built a prototype of MIND in the relational database management system DB2 and have benchmarked its performance. We describe the working prototype and report the measured performance with respect to the previous method of choice. MIND scales not only with the size of datasets but also with the number of processors on an IBM SP2 computer system. Even on uniprocessors, MIND scales well beyond dataset sizes previously published for classifiers.We also give some insights that may have an impact on the evolution of the extended relational calculus SQL.

Chapter information

Source
Anirban DasGupta, ed., A Festschrift for Herman Rubin (Beachwood, Ohio, USA: Institute of Mathematical Statistics, 2004), 348-377

Dates
First available in Project Euclid: 28 November 2007

Permanent link to this document
https://projecteuclid.org/euclid.lnms/1196285404

Digital Object Identifier
doi:10.1214/lnms/1196285404

Mathematical Reviews number (MathSciNet)
MR2126911

Zentralblatt MATH identifier
1268.62061

Subjects
Primary: 68P20: Information storage and retrieval

Keywords
data mining classification classifier relation database SQL

Rights
Copyright © 2004, Institute of Mathematical Statistics

Citation

Wang, Min; Iyer, Bala; Vitter, Jeffrey Scott. Scalable mining for classification rules in relational databases. A Festschrift for Herman Rubin, 348--377, Institute of Mathematical Statistics, Beachwood, Ohio, USA, 2004. doi:10.1214/lnms/1196285404. https://projecteuclid.org/euclid.lnms/1196285404


Export citation