\documentclass[../Main.tex]{subfiles}
\graphicspath{{\subfix{Assets/img/}}}

\begin{document}
The following is a general description of how to solve highly dimensional 
bellman equations.
I currently only have it describing deterministic systems, but I will be 
adding stochastic versions soon.

Consider the following constrained Bellman Equation:

\begin{align}
    W(\theta_t) =& \max_{x_t} F(\theta_t,x_t) + \beta W(\theta_{t+1}) \\
    \text{Subjet To:}\\
    &\theta_{t+1} = G(\theta_t, x_t) %\\
    %&a \leq x_t \leq b %%TODO: Add this back in later.
\end{align}
Where $\theta_t$ is a $m \times 1$ vector of state variables and
$x_t$ is a $k\times 1$ vector of choice variables.

The resulting optimality conditions in matrix form are
\begin{align}
    [0] =& \underset{k\times 1}{\parder{F(\theta_t,x_t)}{x_t}{} }
        + \beta 
            \underset{k \times m}{\parder{G(\theta_t,x_t)}{x_t}{} }
            \cdot
            \underset{m\times 1}{\parder{W(\theta_{t+1})}{\theta_{t+1}}{}} \\
    [0] =& \vec f_{x_t} +\beta B_t \cdot \vec W_{\theta_{t+1}}
\end{align}

Similarly the envelope condations can be written as:
\begin{align}
    \underset{m\times 1}{\parder{W(\theta_{t})}{\theta_{t}}{}} 
        =& \underset{m\times 1}{\parder{F(\theta_t,x_t)}{\theta_t}{}}
        + \beta 
            \underset{m \times m}{\parder{G(\theta_t,x_t)}{\theta_t}{} }
            \cdot
            \underset{m\times 1}{\parder{W(\theta_{t+1})}{\theta_{t+1}}{}} \\
    \vec W_{\theta_t} =&  \vec f_{\theta_t} +\beta A_t \cdot \vec W_{\theta_{t+1}}
\end{align}
If $A_t$ is invertible, it gives the iteration condition
\begin{align}
    \vec W_{\theta_{t+1}} =& \beta^{-1} A_t^{-1} 
        \cdot \left[ \vec W_{\theta_t} - \vec f_{\theta_t}\right] 
\end{align}
If $A_t$ is not invertible, you don't have a solution leading to the standard euler equation.
But, assuming it is, we can now begin solving for the Euler Equation.

\subsubsection{Solving for Euler Equations}
The basic approach is to increment your optimality conditions 
forward a given number of periods, 
and then recurively apply the envelope conditions and laws of motion 
to define it in terms of the current period $t$ variables $\theta_t$. 
With the correct increments, you will get enough conditions to specify both
$\parder{W}{\theta_t}{}$ and $x(\theta_t)$.

This leaves two important questions:
\begin{itemize}
    \item How do you choose the number of periods to increment the optimality conditions?
    \item How do we use these to specify $x(\theta_t)$?
\end{itemize}

In response to the first question, there are $k+m$ unknown variables, 
the $k$ $x(\theta_t)$ policy functions and the $m$ $\parder{w}{\theta_t}{}$ value 
function derivatives.
%Thus we need $m+k$ conditions to solve the system.
By incrementing the optimality condition $j$ times such that $j\cdot m > m+k$,
and substituting the appropriate time-transition from the recursive envelope condition,
we can implicity define the $k+m$ unknowns through $k+m$ equations.

This provides the answer two the second question.
With the system of euler equations in hand\footnote{
    And appropriate existence results depending on the exact specification
}
we can choose any functional approximation approach that we care to use.
Options include euler equation iteration on a grid, 
neural networks, generalized least squares, etc. 
but we must solve for both the policy function and the partial 
derivatives together.

If we don't care about the $m$ partial derivatives, we can simplify
this by recognizing that the $m$ terms that determine them can be used 
to solve for $\parder{w}{\theta_t}{}$ in terms of $x()$, and then substituted into
the $k$ conditions that determine $x()$ in terms of $\parder{w}{\theta_t}{}$.
This gives us the standard euler equations.

\subsubsection{Inequality Constraints}
The discussion above holds for equality constraints.
How do we include inequality constraints, such as a lower bound on consumption 
or an upper bound on launch rates?

This can be addressed by using the KKT maximization technuite to create the optimality conditions.
and including the complementary slackness conditions in the system of equations
used to determine the euler equations.

A basic example is:
\begin{align}
    W(\theta_t) =& \max_{x_t} F(\theta_t,x_t) + \beta W(\theta_{t+1}) \\
    \text{Subjet To:}\\
    &\theta_{t+1} = G(\theta_t, x_t) \\
    &a_i \leq (x_t)_i \leq b_i ~\forall~ i \in \{1\dots k\} 
\end{align}


The resulting stationarity conditions in matrix form are
\begin{align}
    [0] =& \underset{k\times 1}{\parder{F(\theta_t,x_t)}{x_t}{} }
        + \beta 
            \underset{k \times m}{\parder{G(\theta_t,x_t)}{x_t}{} }
            \cdot
            \underset{m\times 1}{\parder{W(\theta_{t+1})}{\theta_{t+1}}{}} 
        + \underset{k \times 1}{\left[-\lambda_a + \lambda_b\right]} \\
    0 =& \lambda_a [(x_t)_i - a_i] ~\forall~ i \in \{1\dots k\} \\
    0 =& \lambda_b [(x_t)_i - b_i] ~\forall~ i \in \{1\dots k\}
\end{align}

In some situations (such as when using neural networks to approximate the policy function), 
this organization of the complementary slackness conditions is difficult to use. 
A useful equivalent condition is setting the equivalent Fisher-Burmeister function
equalt to zero.
\begin{align}
    0 = \Phi(a,b) = a + b - \sqrt{a^2 + b^2} 
\end{align}
A quick proof by contradiction shows that $\Phi(a,b)=0$ exists only when
$a\geq0, b\geq 0, ab=0$.
As it is differentiable, this works well for gradient descent.\footnote{
        https://notes.quantecon.org/submission/5ddb3c926bad3800109084bf
}




\end{document}
