Question about PLR

[garkavem]

Hello! I am sorry if my question is a bit dumb, but I don't understand how do we fit g0(X) model. I understand how we fit l0, m0 because we basically know X, Y, D... But we don't know \theta so we don't know what is Y-D\theta, no?

(screenshot from DoubleML -- An Object-Oriented Implementation of Double Machine Learning in R)

[garkavem]

Also I wonder if I understand correctly, solving equation (12) from the same paper is like setting psi(W, \theta, \eta) = Y - l(x) - \theta * (D - m(x)), but then on page 10 equation (17) is this expression multiplied by (D - m(x)) ?..

[SvenKlaassen]

In equation (12), the model is defined. This is not equal to the equation, we are solving.

The equation we are solving is defined via the regression of $\hat{W}$ on $\hat{V}$ (see step (2))

By regressing $\hat{W}$ on $\hat{V}$, we are trying to minimize the MSE (as fitting a univariate linear model):

$$ min_{\theta} \mathbb{E}\big[(\hat{W} - \theta\hat{V})^2\big] $$

This defines the equation via the first derivative:

$$ \mathbb{E}\big[(\hat{W} - \theta\hat{V})\hat{V}\big] $$

which corresponds to equation (17) (by the definition of $\hat{W}$ and $\hat{V}$).

[garkavem]

Ah yes, indeed, thank you!

[SvenKlaassen]

This is a good question.
You are correct, that it is quite hard to fit $g_0$ directly. The score is motivated from linear models where one would be able to obtain $g_0$ just via a regression of $Y$ on $D$ and $X$ (and then just use the prediction part from $X$).
As most learners do not have this additive structure, we have to rely on a two-step procedure.
First a PLR model with "partialling-out" score is fitted to obtain a preliminary estimate $\tilde{\theta}$. Afterwards, we compute $g_0$ via regression of $Y-\tilde{\theta}D$ on $X$. This is then used to obtain the final estimate $\hat{\theta}$.

[garkavem]

Thank you very much it is much clearer now!

[garkavem]

@SvenKlaassen I am sorry for asking so many question, but I am just curious, in the lecture slides from Tutorial on (Auto) DML based on Riesz Representers on the slide 18 if I understand correctly we have "a recipe" of getting $\psi$... For PLR should we be able to obtain eq. (17) or (18) (from the beforementioned paper) from it and if yes, how?

[SvenKlaassen]

Yes that is correct. This is the IV-type score.
If you check Slide 9 in the presentation you can see that for the PLR model (I use the notation from the presentation):

$$ \begin{align*} g(W)&=\theta D + f(X,A) \\ \alpha(W)&=\frac{D- \mathbb{E}[D|X,A]}{\mathbb{E}[(D- \mathbb{E}[D|X,A])^2]} \end{align*} $$

Further, we have

$$ m(W,g) = g(1, X, A) - g(0,X,A) = \theta $$

such that

$$ \psi_\theta(Z;\theta;\alpha,g) = (Y - D\theta - f(X,A) ) \frac{D- \mathbb{E}[D|X,A]}{\mathbb{E}[(D- \mathbb{E}[D|X,A])^2]} $$

Please remark that our score (see documentation) is defined slightly differently

$$ \psi_{IV}(Z;\theta;\alpha,g) = (Y - D\theta - f(X,A) ) (D- \mathbb{E}[D|X,A]) $$

as we solve $\mathbb{E}[\psi(\dots)]=0$. This implies, we are not using the complete Riez Representer $\alpha(W)$ but have to fit $\mathbb{E}[D|X,A]$ with ML methods.

[garkavem]

Thanks a lot for the explanation!