Algorithms for Mining Association Rules for Binary Segmentations of Huge Categorical Databases
Yasuhiko Morimoto, Takeshi Fukuda, Hirofumi Matsuzawa, Takeshi Tokuyama, Kunikazu Yoda · 1998
We consider the problem of finding association rules that make nearly optimal binary segmen-tations of huge categorical databases. The op-timality of segmentation is defined by an ob-jective function suitable for the user’s objec-tive. An objective function is usually defined in terms of the distribution of a given target attribute. Our goal is to find association rules that split databases into two subsets, optimiz-ing the value of an objective function. The problem is intractable for general ob-jective functions, because letting N be the number of records of a given database, there are 2N possible binary segmentations, and we may have to exhaustively examine all of them. However, when the objective function is convex, there are feasible algorithms for finding nearly optimal binary segmentations, and we prove that typical criteria, such as “entropy (mutual information), ” “x2 (correla-tion), ” and “gini index (mean squared error),” are actually convex. We propose practical algorithms that use computational geometry techniques to handle