-
Notifications
You must be signed in to change notification settings - Fork 1
/
Copy pathprecision.Rd
111 lines (96 loc) · 4.35 KB
/
precision.Rd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
% Generated by roxygen2: do not edit by hand
% Please edit documentation in R/precision.R
\name{precision}
\alias{precision}
\alias{precision.ridge}
\alias{precision.lm}
\title{Measures of Precision and Shrinkage for Ridge Regression}
\usage{
precision(object, det.fun, normalize, ...)
}
\arguments{
\item{object}{An object of class \code{ridge} or \code{lm}}
\item{det.fun}{Function to be applied to the determinants of the covariance
matrices, one of \code{c("log","root")}.}
\item{normalize}{If \code{TRUE} the length of the coefficient vector \eqn{\mathbf{\beta}_k} is
normalized to a maximum of 1.0.}
\item{\dots}{Other arguments (currently unused)}
}
\value{
An object of class \code{c("precision", "data.frame")} with the following columns:
\item{lambda}{The ridge constant}
\item{df}{The equivalent effective degrees of freedom}
\item{det}{The \code{det.fun} function of the determinant of the covariance matrix}
\item{trace}{The trace of the covariance matrix}
\item{max.eig}{Maximum eigen value of the covariance matrix}
\item{norm.beta}{The root mean square of the estimated coefficients, possibly normalized}
\item{norm.diff}{The root mean square of the difference between the OLS solution
(\code{lambda = 0}) and ridge solutions}
}
\description{
The goal of \code{precision} is to allow you to study the relationship between shrinkage of ridge
regression coefficients and their precision directly by calculating measures of each.
Three measures of (inverse) precision based on the \dQuote{size} of the
covariance matrix of the parameters are calculated. Let \eqn{V_k \equiv \text{Var}(\mathbf{\beta}_k)}
be the covariance matrix for a given ridge constant, and let \eqn{\lambda_i , i= 1,
\dots p} be its eigenvalues. Then the variance (= 1/precision) measures are:
\enumerate{
\item \code{"det"}: \eqn{\log | V_k | = \log \prod \lambda} (with \code{det.fun = "log"}, the default)
or \eqn{|V_k|^{1/p} =(\prod \lambda)^{1/p}} (with \code{det.fun = "root"})
measures the linearized volume of the covariance ellipsoid and corresponds conceptually to Wilks'
Lambda criterion
\item \code{"trace"}: \eqn{ \text{trace}( V_k ) = \sum \lambda} corresponds conceptually to Pillai's trace criterion
\item \code{"max.eig"}: \eqn{ \lambda_1 = \max (\lambda)} corresponds to Roy's largest root criterion.
}
Two measures of shrinkage are also calculated:
\itemize{
\item \code{norm.beta}: the root mean square of the coefficient vector \eqn{\lVert\mathbf{\beta}_k \rVert},
normalized to a maximum of 1.0 if \code{normalize == TRUE} (the default).
\item \code{norm.diff}: the root mean square of the difference from the OLS estimate
\eqn{\lVert \mathbf{\beta}_{\text{OLS}} - \mathbf{\beta}_k \rVert}. This measure is inversely related to \code{norm.beta}
}
A plot method, \code{\link{plot.precision}} facilitates making graphs of these quantities.
}
\note{
Models fit by \code{lm} and \code{ridge} use a different scaling for
the predictors, so the results of \code{precision} for an \code{lm} model
will not correspond to those for \code{ridge} with ridge constant = 0.
}
\examples{
longley.y <- longley[, "Employed"]
longley.X <- data.matrix(longley[, c(2:6,1)])
lambda <- c(0, 0.005, 0.01, 0.02, 0.04, 0.08)
lridge <- ridge(longley.y, longley.X, lambda=lambda)
# same, using formula interface
lridge <- ridge(Employed ~ GNP + Unemployed + Armed.Forces + Population + Year + GNP.deflator,
data=longley, lambda=lambda)
clr <- c("black", rainbow(length(lambda)-1, start=.6, end=.1))
coef(lridge)
(pdat <- precision(lridge))
# plot log |Var(b)| vs. length(beta)
with(pdat, {
plot(norm.beta, det, type="b",
cex.lab=1.25, pch=16, cex=1.5, col=clr, lwd=2,
xlab='shrinkage: ||b|| / max(||b||)',
ylab='variance: log |Var(b)|')
text(norm.beta, det, lambda, cex=1.25, pos=c(rep(2,length(lambda)-1),4))
text(min(norm.beta), max(det), "Variance vs. Shrinkage", cex=1.5, pos=4)
})
# plot trace[Var(b)] vs. length(beta)
with(pdat, {
plot(norm.beta, trace, type="b",
cex.lab=1.25, pch=16, cex=1.5, col=clr, lwd=2,
xlab='shrinkage: ||b|| / max(||b||)',
ylab='variance: trace [Var(b)]')
text(norm.beta, trace, lambda, cex=1.25, pos=c(2, rep(4,length(lambda)-1)))
# text(min(norm.beta), max(det), "Variance vs. Shrinkage", cex=1.5, pos=4)
})
}
\seealso{
\code{\link{ridge}}, \code{\link{plot.precision}}
}
\author{
Michael Friendly
}
\keyword{models}
\keyword{regression}