Sia

To explain step 3 in detail, we need to show that the hat matrices for $\boldsymbol{X}$ and $\boldsymbol{X}^{*}$ are the same, which implies that the fitted values from both models are also the same.

 Step 3: Show that the hat matrices are equal

1. 

: The hat matrix $\boldsymbol{H}$ for a design matrix $\boldsymbol{X}$ is given by:

   $$\boldsymbol{H} = \boldsymbol{X}(\boldsymbol{X}^{T} \boldsymbol{X})^{-1} \boldsymbol{X}^{T}$$

2. **Express the hat matrix for $\boldsymbol{X}^{*}$**: Since $\boldsymbol{X}^{*} = \boldsymbol{X} \boldsymbol{A}$, we can substitute this into the hat matrix formula:

   $$\boldsymbol{H}^{*} = \boldsymbol{X}^{*}(\boldsymbol{X}^{*T} \boldsymbol{X}^{*})^{-1} \boldsymbol{X}^{*T}$$

   Substituting $\boldsymbol{X}^{*} = \boldsymbol{X} \boldsymbol{A}$:

   $$\boldsymbol{H}^{*} = \boldsymbol{X} \boldsymbol{A} \left( (\boldsymbol{X} \boldsymbol{A})^{T} (\boldsymbol{X} \boldsymbol{A}) \right)^{-1} (\boldsymbol{X} \boldsymbol{A})^{T}$$

3. 

: We can simplify $(\boldsymbol{X} \boldsymbol{A})^{T} (\boldsymbol{X} \boldsymbol{A})$:

   $$(\boldsymbol{X} \boldsymbol{A})^{T} (\boldsymbol{X} \boldsymbol{A}) = \boldsymbol{A}^{T} \boldsymbol{X}^{T} \boldsymbol{X} \boldsymbol{A}$$

   Therefore, we have:

   $$\boldsymbol{H}^{*} = \boldsymbol{X} \boldsymbol{A} \left( \boldsymbol{A}^{T} \boldsymbol{X}^{T} \boldsymbol{X} \boldsymbol{A} \right)^{-1} \boldsymbol{A}^{T} \boldsymbol{X}^{T}$$

4. 

: Since $\boldsymbol{A}$ is nonsingular, we can use the property of inverses:

   $$\left( \boldsymbol{A}^{T} \boldsymbol{X}^{T} \boldsymbol{X} \boldsymbol{A} \right)^{-1} = \boldsymbol{A}^{-1} \left( \boldsymbol{X}^{T} \boldsymbol{X} \right)^{-1} \boldsymbol{A}^{-T}$$

5. 

: Substituting this back into our expression for $\boldsymbol{H}^{*}$:

   $$\boldsymbol{H}^{*} = \boldsymbol{X} \boldsymbol{A} \boldsymbol{A}^{-1} \left( \boldsymbol{X}^{T} \boldsymbol{X} \right)^{-1} \boldsymbol{A}^{-T} \boldsymbol{A}^{T} \boldsymbol{X}^{T}$$

   This simplifies to:

   $$\boldsymbol{H}^{*} = \boldsymbol{X} \left( \boldsymbol{X}^{T} \boldsymbol{X} \right)^{-1} \boldsymbol{X}^{T} = \boldsymbol{H}$$

 Conclusion

Since we have shown that $\boldsymbol{H}^{*} = \boldsymbol{H}$, it follows that the hat matrices for $\boldsymbol{X}$ and $\boldsymbol{X}^{*}$ are the same. Consequently, the fitted values from both models will also be the same, as they are derived from the same hat matrix. 

This completes the explanation for step 3! If you have any further questions or need clarification on any part, feel free to ask!