predictorImportance

Syntax

imp = predictorImportance(ens)
[imp,ma]
= predictorImportance(ens)

Description

imp = predictorImportance(ens) computes
estimates of predictor importance for ens by summing
these estimates over all weak learners in the ensemble. imp has
one element for each input predictor in the data used to train this
ensemble. A high value indicates that this predictor is important
for ens.

Input Arguments

Output Arguments

imp

A row vector with the same number of elements as the number
of predictors (columns) in ens.X.
The entries are the estimates of predictor importance, with 0 representing
the smallest possible importance.

ma

A P-by-P matrix of predictive
measures of association for P predictors. Element ma(I,J) is
the predictive measure of association averaged over surrogate splits
on predictor J for which predictor I is
the optimal split predictor. predictorImportance averages
this predictive measure of association over all trees in the ensemble.

Predictor Importance and Surrogate Splits

Estimate the predictor importance for all variables in the data and where the regression tree ensemble contains surrogate splits.

Load the carsmall data set.

load carsmall

Grow an ensemble of 100 regression trees for MPG using Acceleration, Cylinders, Displacement, Horsepower, Model_Year, and Weight as predictors. Specify tree stumps as the weak learners, and also identify surrogate splits.

More About

Predictor Importance

predictorImportance computes estimates of predictor
importance for ens by summing
changes in the mean squared error (MSE) due to
splits on every predictor and dividing the sum by
the number of branch nodes. If
ens is grown without
surrogate splits, this sum is taken over best
splits found at each branch node. If
ens is grown with surrogate
splits, this sum is taken over all splits at each
branch node including surrogate splits.
imp has one element for each
input predictor in the data used to train
ens. At each node, MSE is
estimated as node error weighted by the node
probability. Variable importance associated with
this split is computed as the difference between
MSE for the parent node and the total MSE for the
two children.

Predictive Measure of Association

The predictive measure of association is
a value that indicates the similarity between decision rules that
split observations. Among all possible decision splits that are compared
to the optimal split (found by growing the tree), the best surrogate decision
split yields the maximum predictive measure of association.
The second-best surrogate split has the second-largest predictive
measure of association.

Suppose xj and xk are
predictor variables j and k,
respectively, and j ≠ k.
At node t, the predictive measure of association
between the optimal split xj < u and
a surrogate split xk < v is

λjk=min(PL,PR)−(1−PLjLk−PRjRk)min(PL,PR).

PL is the
proportion of observations in node t, such that xj < u.
The subscript L stands for the left child of node t.

PR is the
proportion of observations in node t, such that xj ≥ u.
The subscript R stands for the right child of node t.

PLjLk is the proportion
of observations at node t, such that xj < u and xk < v.

PRjRk is the proportion
of observations at node t, such that xj ≥ u and xk ≥ v.

Observations with missing values for xj or xk do
not contribute to the proportion calculations.

λjk is a value
in (–∞,1]. If λjk >
0, then xk < v is
a worthwhile surrogate split for xj < u.

Algorithms

Element ma(i,j) is the predictive measure
of association averaged over surrogate splits on predictor j for
which predictor i is the optimal split predictor.
This average is computed by summing positive values of the predictive
measure of association over optimal splits on predictor i and
surrogate splits on predictor j and dividing by
the total number of optimal splits on predictor i,
including splits for which the predictive measure of association between
predictors i and j is negative.