You cannot select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
134 lines
5.5 KiB
TeX
134 lines
5.5 KiB
TeX
\documentclass[../Main.tex]{subfiles}
|
|
\graphicspath{{\subfix{Assets/img/}}}
|
|
|
|
\begin{document}
|
|
The following is a general description of how to solve highly dimensional
|
|
bellman equations.
|
|
I currently only have it describing deterministic systems, but I will be
|
|
adding stochastic versions soon.
|
|
|
|
Consider the following constrained Bellman Equation:
|
|
|
|
\begin{align}
|
|
W(\theta_t) =& \max_{x_t} F(\theta_t,x_t) + \beta W(\theta_{t+1}) \\
|
|
\text{Subjet To:}\\
|
|
&\theta_{t+1} = G(\theta_t, x_t) %\\
|
|
%&a \leq x_t \leq b %%TODO: Add this back in later.
|
|
\end{align}
|
|
Where $\theta_t$ is a $m \times 1$ vector of state variables and
|
|
$x_t$ is a $k\times 1$ vector of choice variables.
|
|
|
|
The resulting optimality conditions in matrix form are
|
|
\begin{align}
|
|
[0] =& \underset{k\times 1}{\parder{F(\theta_t,x_t)}{x_t}{} }
|
|
+ \beta
|
|
\underset{k \times m}{\parder{G(\theta_t,x_t)}{x_t}{} }
|
|
\cdot
|
|
\underset{m\times 1}{\parder{W(\theta_{t+1})}{\theta_{t+1}}{}} \\
|
|
[0] =& \vec f_{x_t} +\beta B_t \cdot \vec W_{\theta_{t+1}}
|
|
\end{align}
|
|
|
|
Similarly the envelope condations can be written as:
|
|
\begin{align}
|
|
\underset{m\times 1}{\parder{W(\theta_{t})}{\theta_{t}}{}}
|
|
=& \underset{m\times 1}{\parder{F(\theta_t,x_t)}{\theta_t}{}}
|
|
+ \beta
|
|
\underset{m \times m}{\parder{G(\theta_t,x_t)}{\theta_t}{} }
|
|
\cdot
|
|
\underset{m\times 1}{\parder{W(\theta_{t+1})}{\theta_{t+1}}{}} \\
|
|
\vec W_{\theta_t} =& \vec f_{\theta_t} +\beta A_t \cdot \vec W_{\theta_{t+1}}
|
|
\end{align}
|
|
If $A_t$ is invertible, it gives the iteration condition
|
|
\begin{align}
|
|
\vec W_{\theta_{t+1}} =& \beta^{-1} A_t^{-1}
|
|
\cdot \left[ \vec W_{\theta_t} - \vec f_{\theta_t}\right]
|
|
\end{align}
|
|
If $A_t$ is not invertible, you don't have a solution leading to the standard euler equation.
|
|
But, assuming it is, we can now begin solving for the Euler Equation.
|
|
|
|
\subsubsection{Solving for Euler Equations}
|
|
The basic approach is to increment your optimality conditions
|
|
forward a given number of periods,
|
|
and then recurively apply the envelope conditions and laws of motion
|
|
to define it in terms of the current period $t$ variables $\theta_t$.
|
|
With the correct increments, you will get enough conditions to specify both
|
|
$\parder{W}{\theta_t}{}$ and $x(\theta_t)$.
|
|
|
|
This leaves two important questions:
|
|
\begin{itemize}
|
|
\item How do you choose the number of periods to increment the optimality conditions?
|
|
\item How do we use these to specify $x(\theta_t)$?
|
|
\end{itemize}
|
|
|
|
In response to the first question, there are $k+m$ unknown variables,
|
|
the $k$ $x(\theta_t)$ policy functions and the $m$ $\parder{w}{\theta_t}{}$ value
|
|
function derivatives.
|
|
%Thus we need $m+k$ conditions to solve the system.
|
|
By incrementing the optimality condition $j$ times such that $j\cdot m > m+k$,
|
|
and substituting the appropriate time-transition from the recursive envelope condition,
|
|
we can implicity define the $k+m$ unknowns through $k+m$ equations.
|
|
|
|
This provides the answer two the second question.
|
|
With the system of euler equations in hand\footnote{
|
|
And appropriate existence results depending on the exact specification
|
|
}
|
|
we can choose any functional approximation approach that we care to use.
|
|
Options include euler equation iteration on a grid,
|
|
neural networks, generalized least squares, etc.
|
|
but we must solve for both the policy function and the partial
|
|
derivatives together.
|
|
|
|
If we don't care about the $m$ partial derivatives, we can simplify
|
|
this by recognizing that the $m$ terms that determine them can be used
|
|
to solve for $\parder{w}{\theta_t}{}$ in terms of $x()$, and then substituted into
|
|
the $k$ conditions that determine $x()$ in terms of $\parder{w}{\theta_t}{}$.
|
|
This gives us the standard euler equations.
|
|
|
|
\subsubsection{Inequality Constraints}
|
|
The discussion above holds for equality constraints.
|
|
How do we include inequality constraints, such as a lower bound on consumption
|
|
or an upper bound on launch rates?
|
|
|
|
This can be addressed by using the KKT maximization technuite to create the optimality conditions.
|
|
and including the complementary slackness conditions in the system of equations
|
|
used to determine the euler equations.
|
|
|
|
A basic example is:
|
|
\begin{align}
|
|
W(\theta_t) =& \max_{x_t} F(\theta_t,x_t) + \beta W(\theta_{t+1}) \\
|
|
\text{Subjet To:}\\
|
|
&\theta_{t+1} = G(\theta_t, x_t) \\
|
|
&a_i \leq (x_t)_i \leq b_i ~\forall~ i \in \{1\dots k\}
|
|
\end{align}
|
|
|
|
|
|
The resulting stationarity conditions in matrix form are
|
|
\begin{align}
|
|
[0] =& \underset{k\times 1}{\parder{F(\theta_t,x_t)}{x_t}{} }
|
|
+ \beta
|
|
\underset{k \times m}{\parder{G(\theta_t,x_t)}{x_t}{} }
|
|
\cdot
|
|
\underset{m\times 1}{\parder{W(\theta_{t+1})}{\theta_{t+1}}{}}
|
|
+ \underset{k \times 1}{\left[-\lambda_a + \lambda_b\right]} \\
|
|
0 =& \lambda_a [(x_t)_i - a_i] ~\forall~ i \in \{1\dots k\} \\
|
|
0 =& \lambda_b [(x_t)_i - b_i] ~\forall~ i \in \{1\dots k\}
|
|
\end{align}
|
|
|
|
In some situations (such as when using neural networks to approximate the policy function),
|
|
this organization of the complementary slackness conditions is difficult to use.
|
|
A useful equivalent condition is setting the equivalent Fisher-Burmeister function
|
|
equalt to zero.
|
|
\begin{align}
|
|
0 = \Phi(a,b) = a + b - \sqrt{a^2 + b^2}
|
|
\end{align}
|
|
A quick proof by contradiction shows that $\Phi(a,b)=0$ exists only when
|
|
$a\geq0, b\geq 0, ab=0$.
|
|
As it is differentiable, this works well for gradient descent.\footnote{
|
|
https://notes.quantecon.org/submission/5ddb3c926bad3800109084bf
|
|
}
|
|
|
|
|
|
|
|
|
|
\end{document}
|