KnnTree Classification

Version 1.1, January 2003


In knnTree classification we build a classification tree, and then produce a k-nearest-neighbor (knn) classifier separately inside each leaf. Each of the knn classifiers separately uses variable selection (by either forward or backward selection, the same direction being used in each leaf), chooses whether the training set data should be scaled or not, and chooses k, all by leave-one-out cross-validation. Details can be found in Buttrey and Karo, "Using K-Nearest-Neighbor Classification in the Leaves of a Tree," Computational Statistics and Data Analysis, vol. 40 (2002), pp. 27-37.


The procedure to produce the knnTree classifier is implemented in a DLL (for Windows) and a shared library (for Unix-type systems) and is intended to run using R or S-Plus.

Current R instructions: please download this dll and this set of R commands. Then start R and use commands like the following (adjusting them for the proper locations of knnTree.dll and knnTree.R):

> dyn.load ("h:/temp/knnTree.dll")
> source ("h:/temp/knnTree.R")
> data (iris)
> set.seed (3)
> samp <- sample (nrow(iris), 75, replace=F)
> knn.var (iris[samp, c(5, 1:4)], theyre.the.same=T)
This knn classifier is based on 75 observations.
It uses 2 out of 4 variables without scaling.

Training rate is 0.01333 , achieved at k = 27.
> iris.knn <- knn.var (iris[samp, c(5, 1:4)], theyre.the.same=T)
> predict (iris.knn, iris[-samp, c(5, 1:4)], iris[samp, c(5, 1:4)])

[1] 0.08

Old instructions: don't do this. For R and unix, go to the CRAN home page and download the package. (Under "Software" choose "Package Sources" and choose "knnTree.") This seems to work fine. However the same code does not seem to work under Windows. For that you should download, which includes the R/S-Plus functions and their documentation as well as the DLL containing the compiled code. Details on the functions are included in the readme file which you can also view here.

If you prefer you can download the source files (sorry, temporarily unavailable) in ZIP form and compile them for your platform.