Using linear regression
A simple way to compute the sample partial correlation for some data is to solve the two associated linear regression problems and calculate the correlation between the residuals. Let X and Y be random variables taking real values, and let Z be the n-dimensional vector-valued random variable. Let xi, yi and zi denote the ith of i.i.d. observations from some joint probability distribution over real random variables X, Y, and Z, with zi having been augmented with a 1 to allow for a constant term in the regression. Solving the linear regression problem amounts to finding (n+1)-dimensional regression coefficient vectors and such that
where is the number of observations, and is the scalar product between the vectors and .
The residuals are then
and the sample partial correlation is then given by the usual formula for sample correlation, but between these new derived values:
In the first expression the three terms after minus signs all equal 0 since each contains the sum of residuals from an ordinary least squares regression.
Example
Consider the following data on three variables, X, Y, and Z:
More information X, Y ...
Close
Computing the Pearson correlation coefficient between variables X and Y results in approximately 0.970, while computing the partial correlation between X and Y, using the formula given above, gives a partial correlation of 0.919. The computations were done using R with the following code.
> X <- c(2,4,15,20)
> Y <- c(1,2,3,4)
> Z <- c(0,0,1,1)
> mm1 <- lm(X~Z)
> res1 <- mm1$residuals
> mm2 <- lm(Y~Z)
> res2 <- mm2$residuals
> cor(res1,res2)
[1] 0.919145
> cor(X,Y)
[1] 0.9695016
> generalCorr::parcorMany(cbind(X,Y,Z))
nami namj partij partji rijMrji
[1,] "X" "Y" "0.8844" "1" "-0.1156"
[2,] "X" "Z" "0.1581" "1" "-0.8419"
The lower part of the above code reports generalized nonlinear partial correlation coefficient between X and Y after removing the nonlinear effect of Z to be 0.8844. Also, the generalized partial correlation coefficient between X and Z after removing the nonlinear effect of Y to be 0.1581. See the R package `generalCorr' and its vignettes for details. Simulation and other details are in Vinod (2017) "Generalized correlation and kernel causality with applications in development economics," Communications in Statistics - Simulation and Computation, vol. 46, [4513, 4534], available online: 29 Dec 2015, URL https://doi.org/10.1080/03610918.2015.1122048.
It can be computationally expensive to solve the linear regression problems. Actually, the nth-order partial correlation (i.e., with |Z| = n) can be easily computed from three (n - 1)th-order partial correlations. The zeroth-order partial correlation ρXY·Ø is defined to be the regular correlation coefficient ρXY.
It holds, for any that[3]
Naïvely implementing this computation as a recursive algorithm yields an exponential time complexity. However, this computation has the overlapping subproblems property, such that using dynamic programming or simply caching the results of the recursive calls yields a complexity of .
Note in the case where Z is a single variable, this reduces to:[citation needed]
Using matrix inversion
The partial correlation can also be written in terms of the joint precision matrix. Consider a set of random variables, of cardinality n. We want the partial correlation between two variables and given all others, i.e., . Suppose the (joint/full) covariance matrix is positive definite and therefore invertible. If the precision matrix is defined as , then
-
|
|
(1) |
Computing this requires , the inverse of the covariance matrix which runs in time (using the sample covariance matrix to obtain a sample partial correlation). Note that only a single matrix inversion is required to give all the partial correlations between pairs of variables in .
To prove Equation (1), return to the previous notation (i.e. ) and start with the definition of partial correlation: ρXY·Z is the correlation between the residuals eX and eY resulting from the linear regression of X with Z and of Y with Z, respectively.
First, suppose are the coefficients for linear regression fit; that is,
Write the joint covariance matrix for the vector as
where
Then the standard formula for linear regression gives
Hence, the residuals can be written as
Note that has expectation zero because of the inclusion of an intercept term in . Computing the covariance now gives
-
|
|
(2) |
Next, write the precision matrix in a similar block form:
Then, by Schur's formula for block-matrix inversion,
The entries of the right-hand-side matrix are precisely the covariances previously computed in (2), giving
Using the formula for the inverse of a 2×2 matrix gives
So indeed, the partial correlation is
as claimed in (1).