| DecTree {lazy.tree} | R Documentation |
This function recursively divides the sets of observations
into two subsets and create a decision tree.
Japanese help file: DecTree_JPH
DecTree(
y,
X,
impfuncname = "gini",
majestic = 0,
method = 4,
app = 0,
maxapp = 12,
maxncut = 50,
switch = 0,
eps = 0,
minnobs = 1,
maxdepth = 10,
maxdivide = 500,
minp = 1e-04,
nrepM = 50,
psampleM = 100,
rf = 0,
nxvarM = if (!is.null(y) && !is.factor(y)) max(floor(ncol(X)/3), 1) else
floor(sqrt(ncol(X))),
.R.sM = NULL,
minLeftM = 0.5,
maxRightM = 0.5,
minnobsM = 10,
title = "",
print = 1,
plot = 0,
debug = 0
)
y |
The criterion variable (as vector) |
X |
The regressor matrix (as data frame) |
impfuncname |
The name of impurity function name. |
majestic |
= 1, 2, 3, 4, or 5 to create a in-node resampling tree |
method |
Treatment of the categorical regressor variable. |
app |
= 1 or 9 to use all possible partitions
of categorical regressor levels. |
maxapp |
Maximum # of categories to use app. This will be used as the max # of categories of criterion when method=4 is used. |
maxncut |
Maximum # of cut point (threshold) candidates. |
eps |
The minimum improvement of impurity measure. |
minnobs |
The minimum # of observations in a node. |
maxdepth |
The maximum depth of a tree. |
maxdivide |
The maximum # of divisions. |
minp |
minimum value of multinomial probability |
nrepM |
# of divisions to try for majestic tree |
psampleM |
percentage of the sample for majestic tree |
rf |
= 1 to sample regressor variables (Random Forest) |
nxvarM |
# of variables to sample for majestic tree (Random Forest) |
.R.sM |
random number seed for majestic tree obtained as: |
minLeftM |
minimum proportion of x values to be classified as Left |
maxRightM |
maximum proportion of x values to be classified as Right |
minnobsM |
minimum # of obs in a node to use majestic |
title |
The title string |
print |
= 1 to print the result, = 2 to print the history = 3 to print the history in detail. |
plot |
= 1 to plot the tree. |
debug |
= 1 to print intermediate result |
Input Variables:
The criterion variable y must be a numeric vector or a factor.
The regressor variable X must be a data frame
with numeric or factor variables.
Division Details
A Greedy Algorithm with Depth-First Search (DFS) is used to find a tree.
For continuous regressor, all the unique values of it will be used as the candidate for the threshold.
Treatment of categorical regressor variables:
When app=0:
The categorical regressor with the # of levels greater than 2
will be quantified according to method.
Regardless of the method, if the criterion is continuous or
the # of levels of the criterion is equal to two,
the conditional mean of the criterion given the level of regressor
will be used as the quantified regressor.
(The # of quantified regressor = 1.)
If method=0 and if the # of levels of the criterion
is greater than 2, a categorical regressor will be dummy expanded
and the resulting dummy variables will be used as the quantified
regressor variables.
(The # of quantified regressor = # of levels of the regressor)
If method=1
When the # of levels of the criterion is greater than 2,
the first canonical variable of the canonical discriminant analysis
of the criterion on the dummy expanded regressor will be used.
(The # of quantified regressor = 1.)
If method=2, in addition to the above, the 2nd canonical
variable, if exists, and the sum and difference of those two canonical
variables will be added as the quantified regressors.
(The # of quantified regressors = 4 or 1.)
If method=3, the one-versus-rest dichotomization of the criterion
variable is used and the conditional means will be used as the quantified
regressors.
If any of the resulting quantified variables has zero variance
or duplicated,
the redundant quantified variables will be removed.
(The # of quantified regressors = 4 or 1 + # of levels of the criterion
or less.)
If method=4,
all possible bipartitions of the levels of the criterion
is used for the dichotomization
and the conditional means will be used as the quantified regressors.
If any of the resulting quantified variables has zero variance
or duplicated,
the redundant quantified variables will be removed.
(The # of quantified regressors = 2^nc-1-1 or less where nc is the
# of levels of the criterion variable.)
This seems to be the best option when the # of levels of the criterion
is not so large ( <= 10 or so. ) and the # of levels of the regressor
is very large.
When app=1:
If the levels of the criterion is greater than 2 and the levels of
the regressor is greater than 2, all possible partitions of size 2
will be tried.
When app=9:
Regardless the # of levels of the categorical variables
app will be used for the categorical regressors.
Stopping rule:
1) Relative improvement of the impurity measure due to a division
is less than or equal to eps
2) # of observations in a node is less than or equal to minnobs.
3) # of observations in either subset becomes 0 after division.
4) Maximum # of division is greater than maxdivide.
5) Maximum depth of the tree reached maxdepth.
Requires igraph package to draw a tree.
A class DecTree object containing a list of the following:
n: # of observations
ncaty: # of levels of y (0 means contimuous.)
levelsy: levels of y if y is categorical, or NULL
yname: the name of the criterion variable
yname2: the name of the actual y argument
nx: # of predictor variables
ncatx: # of levels of X (0 means contimuous.)
levelsX: levels of the regressor categorical variable, or NULL
xname: the names of predictor variables
xname2; the name of the actual X argument
impfuncname: the name of impurity function
nleaf: # of leaves
MaxDepth: The maximum depth of the tree
.R.s: .Random.seed vector
Use this to reproduce the majestic tree.
RNGkindM, .R.sM: random number seed info
node: A list of node information where node[[r]] contains:
node[[r]]$parent The parent of the r-th node.
node[[r]]$set The set of observations in the r-th node.
node[[r]]$nobs # of observations in the r-th node.
node[[r]]$j The variable to be used to divide the r-th node.
node[[r]]$k * The quantified variable number to be used.
node[[r]]$th * The value of the threshold or NA.
node[[r]]$nqx * # of unique quantified variables.
node[[r]]$valxj * The set of factor levels of categorical variable j
to be classified as left from this node.
node[[r]]$valxjR * The set of factor levels of categorical variable j
to be classified as right from this node.
node[[r]]$score The nobs weighted impurity measure.
node[[r]]$ImpMeasure score / nobs.
node[[r]]$scoreimp The improvement of impurity measure due to dividing
this node.
node[[r]]$impfuncname Name of the impurity measure.
node[[r]]$depth The distance from the top node
node[[r]]$Left * The left node name
node[[r]]$Right * The right node name
node[[r]]$switched = 1 when left and right were swiched.
node[[r]]$yhat ** The predicted value
node[[r]]$prob ** The post probability vector
* Not included in the final nodes.
** Not included in the middle nodes.
fimport: Feature Importance
Tab: Cross Table
margin: Marginal Frequency
cor: Correlation Coefficient between continuous y and yhat.
Gini: gini coefficient
mllh: minus log multinomial likelihood
nparam: # of parameters estimated
aic and bic:
tprecision: total precision
macroF1 and weightedF1: macro F1 statistics (unweighted and weighted)
SCT: a matrix of accuracy, precision, recall, specificity, and F1-score
for each levels of the criterion.
args: list of arguments to DecTree function.
title: title string generated in DecTree function.
y: input
yhat: predicted value of y
prob: probability vector
Node: node number
RNGkind: The name of randon mumber generator
.R.sM: The random number seed used for majestic.
Joe Suzuki (2020) Statistical Machine Learning with 100 Math and R Problems. Kyoritsu Shuppan.
# univariate regression tree;
# numerical y and numerical X
X <- data.frame(x=univar$x)
plot( univar$x, univar$y1, type="p" )
res01 <- DecTree( univar$y1,X, minnobs=1, eps=0.001
, print=3, plot=0, app=0, title="y1" )
# library(igraph)
# plot_dt(res01$node)
# categorical y and continuous X
res02 <- DecTree( as.factor(univar$y1),X, minnobs=1, eps=0.001, maxncut=100
, print=3, plot=0,, app=0, title="y1" )
# categorical y and categorical X: (method 1 and 2 are not available)
Xc <- data.frame(x=as.factor(univar$x))
res03 <- DecTree( as.factor(univar$y1),Xc, minnobs=1, eps=0.001, method=3
, print=3, plot=0,, app=0, title="y1" )
res04 <- DecTree( as.factor(univar$y1),Xc, minnobs=1, eps=0.001, method=4
, print=3, plot=0,, app=0, title="y1" )
# bivariate classification tree
# categorical y, categorical X
plot(square$x1,square$x2, type="n")
text(square$x1,square$x2,square$Class4, cex=2)
SQ2=matrix(square$class4, 4,, byrow=1)
heatmap( t(SQ2), Rowv=NA, Colv=NA, reorderfun=NA, symm=1
, col=c("darkred","darkgreen","orange","lightblue")
, labRow=4:1, labCol=1:4, revC=TRUE )
legend( "left", legend=1:4, pch=1,cex=2, pt.lwd=2, box.lwd=2
, col=c("darkred","darkgreen","orange","lightblue") )
Xc <- data.frame(x1=as.factor(square$x1),x2=as.factor(square$x2))
res0 <- DecTree( square$Class4, Xc, maxdepth=4, method=0, print=3
, plot=0, title="square: categorical y, categorical X" )
res1 <- DecTree( square$Class4, Xc, maxdepth=4, method=1, print=3
, plot=0, title="square: categorical y, categorical X" )
# categorical y, categorical X
res2 <- DecTree( square$Class4, Xc, maxdepth=4, method=2, print=3
, plot=0, title="square: categorical y, categorical X" )
# categorical y, categorical X
res3 <- DecTree( square$Class4, Xc, maxdepth=4, method=3, print=3
, plot=0, title="square: categorical y, categorical X" )
# categorical y, categorical X
res4 <- DecTree( square$Class4, Xc, maxdepth=4, method=4, print=3
, plot=0, title="square: categorical y, categorical X" )
# categorical y, categorical X: all possible partitions of x levels
res11 <- DecTree( square$Class4, Xc, maxdepth=4, eps=0, app=1, print=3
, plot=0, title="square: categorical y, categorical X" )
# categorical y, categorical X: all possible partitions of x levels
res12 <- DecTree( square$Class4, Xc,maxdepth=4, eps=0, app=9, print=3
, plot=0, title="square: categorical y, categorical X" )
# categorical y, numerical X
X <- data.frame(x1=square$x1,x2=square$x2)
res21 <- DecTree( square$Class4, X, minnobs = 2, eps=0, print=3, plot=0
, title="square: categorical y, numerical X" )
plot2d_dt( square$Class4, X, res21$node, ngrid=50, title="Border Lines" )
# numerical y, numerical X
res22 <- DecTree( square$class4, X, minnobs=2, eps=0, print=3, plot=0
, title="square: numerical y, numerical X" )
# slightly larger data
# categorical y, categorical X
Xc <- data.frame(x1=as.factor(square2$x1),x2=as.factor(square2$x2))
res31 <- DecTree( as.factor(square2$y), Xc, minnobs=50, eps=0.0001
, print=3, maxdepth=50, plot=0, app=9
, title="square: categorical y, categorical X" )
# categorical y, categorical X
Xc <- data.frame(x1=as.factor(square2$x1),x2=as.factor(square2$x2))
res32 <- DecTree( as.factor(square2$y), Xc, minnobs=50, eps=0.0001
, print=3, maxdepth=50, plot=0, app=0, method=4
, title="square: categorical y, categorical X" )
# iris data
Species <- iris$Species
levels(Species) <- c("set","ver","vir")
ncaty <- 3; ncatX <- rep(0,4)
resI1 <- DecTree( Species, iris[,1:4], eps=1e-5
, title="rect: categorical y, continuous X", plot=0 )
# iris: categorical y, mixed X
Species <- iris$Species
levels(Species) <- c("set","ver","vir")
Xc <- iris[,-5]
Xc[,3] <- cut(Xc[,3],9, labels=1:9)
Xc[,4] <- cut(Xc[,4],9, labels=1:9)
ncaty <- 3; ncatX <- c(0,0,8,8)
resI2 <- DecTree( Species, Xc, eps=1e-5, app=1
, title="iris: categorical y, mixed X", plot=0 )
resI3 <- DecTree( Species, Xc, eps=1e-5, app=0, method=1
, title="iris: categorical y, mixed X", plot=0 )
resI4 <- DecTree( Species, Xc, eps=1e-5, app=0, method=3
, title="iris: categorical y, mixed X", plot=0 )
resI5 <- DecTree( Species, Xc, eps=1e-5, app=0, method=4
, title="iris: categorical y, mixed X", plot=0 )