|
|
|
@ -28,7 +28,7 @@ Thus the $M$-period (possibly infinite), problem is:
|
|
|
|
%
|
|
|
|
%
|
|
|
|
|
|
|
|
|
|
|
|
\subsection{Infinite Period (Bellman) Equation}
|
|
|
|
\subsection{Infinite Period (Bellman) Equation}
|
|
|
|
The problem above can be rewritten in the bellman form as
|
|
|
|
The inifinite period version of the problem above can be rewritten in the bellman form as
|
|
|
|
\begin{align}
|
|
|
|
\begin{align}
|
|
|
|
V^i(\vec s_t, \vec x^{\sim i}_t, D_t) = \max_{x^i_t} u^i(\vec s_t, D_t) -F(x)
|
|
|
|
V^i(\vec s_t, \vec x^{\sim i}_t, D_t) = \max_{x^i_t} u^i(\vec s_t, D_t) -F(x)
|
|
|
|
+ \beta \left[ V^i(\vec s_{t+1}, \vec x^{\sim i}_{t+1}, D_{t+1}) \right]
|
|
|
|
+ \beta \left[ V^i(\vec s_{t+1}, \vec x^{\sim i}_{t+1}, D_{t+1}) \right]
|
|
|
|
@ -42,61 +42,72 @@ To solve for the policy function, we have a variety of methods available.
|
|
|
|
Due to the computational method chosen later, I'm going to examine the conditions
|
|
|
|
Due to the computational method chosen later, I'm going to examine the conditions
|
|
|
|
for the existence of an euler equation.
|
|
|
|
for the existence of an euler equation.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
\subsubsection{Euler Equation}
|
|
|
|
\subsubsection{Euler Equation}
|
|
|
|
First, find the single optimality condition
|
|
|
|
Appendix \cref{Appendix} contains more details on the math involved.
|
|
|
|
|
|
|
|
What follows is just a sketch of the applied method in matrix notation.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
As there is only one choice variable, we get a single optimality condition.
|
|
|
|
|
|
|
|
It can be written in various formats, with the latter matching the appendix the best.
|
|
|
|
\begin{align}
|
|
|
|
\begin{align}
|
|
|
|
0 =& \parder{}{x^i_t}{} u^i(\vec s_t, D_t) -\parder{}{x^i_t}{}F(x)
|
|
|
|
% 0 =& \parder{}{x^i_t}{} u^i(\vec s_t, D_t) -\parder{}{x^i_t}{}F(x)
|
|
|
|
+ \beta \left[ \parder{}{x^i_t}{}
|
|
|
|
% + \beta \left[ \parder{}{x^i_t}{}
|
|
|
|
V^i(\vec s_{t+1}, \vec x^{\sim i}_{t+1}, D_{t+1})
|
|
|
|
% V^i(\vec s_{t+1}, \vec x^{\sim i}_{t+1}, D_{t+1})
|
|
|
|
\right] \\
|
|
|
|
% \right] \\
|
|
|
|
0 =& -\der{F}{x^i_t}{}
|
|
|
|
0 =& -\der{F}{x^i_t}{}
|
|
|
|
+ \beta \left[
|
|
|
|
+ \beta \left[
|
|
|
|
|
|
|
|
\nabla_{x^i_t} [ \vec s_{t+1}, \vec x^{\sim i}_{t+1}, D_{t+1} ]
|
|
|
|
|
|
|
|
\cdot
|
|
|
|
\nabla_{\vec s_{t+1}, \vec x^{\sim i}_{t+1}, D_{t+1}}
|
|
|
|
\nabla_{\vec s_{t+1}, \vec x^{\sim i}_{t+1}, D_{t+1}}
|
|
|
|
V^i(\vec s_{t+1}, \vec x^{\sim i}_{t+1}, D_{t+1})
|
|
|
|
V^i(\vec s_{t+1}, \vec x^{\sim i}_{t+1}, D_{t+1})
|
|
|
|
\cdot
|
|
|
|
|
|
|
|
\nabla_{x^i_t} [ \vec s_{t+1}, \vec x^{\sim i}_{t+1}, D_{t+1} ]
|
|
|
|
|
|
|
|
\right] \label{EQ:OptimalityCondition}\\
|
|
|
|
\right] \label{EQ:OptimalityCondition}\\
|
|
|
|
0 =& -\der{F}{x^i_t}{} + \beta \nabla V^i_{t+1} \cdot \vec a_t
|
|
|
|
0 =& -\der{F}{x^i_t}{} + \beta \vec a(\vec s_t,D_t) \cdot \nabla V^i_{t+1}
|
|
|
|
\label{EQ:SimplifiedOptimalityCondition}
|
|
|
|
\label{EQ:SimplifiedOptimalityCondition}\\
|
|
|
|
|
|
|
|
=& - f_{x_t} + \beta \vec a_t \cdot \nabla V^i_{t+1}
|
|
|
|
\end{align}
|
|
|
|
\end{align}
|
|
|
|
|
|
|
|
|
|
|
|
Second, the $2N$\footnote{recall that $N$ is the number of constellations.}
|
|
|
|
As there are $N$ constellations we get $N$ satellite stocks,
|
|
|
|
envelope conditions can also be found:
|
|
|
|
$N-1$ decisions $x^{\sim i}$,
|
|
|
|
|
|
|
|
and $1$ debris state for a total of $2N$ state variables\footnote{recall that $N$ is the number of constellations.}.
|
|
|
|
|
|
|
|
Thus there are $2N$ envelope conditions to be found:
|
|
|
|
\begin{align}
|
|
|
|
\begin{align}
|
|
|
|
|
|
|
|
% \nabla_{\vec s_t, \vec x^{\sim i}_t, D_t} V^i(\vec s_t, \vec x^{\sim i}_t, D_t)
|
|
|
|
|
|
|
|
% =& \nabla_{\vec s_t, \vec x^{\sim i}_t, D_t} u^i(\vec s_t, D_t) \notag \\
|
|
|
|
|
|
|
|
% &+ \beta \left[
|
|
|
|
|
|
|
|
% \nabla_{\vec s_{t+1}, \vec x^{\sim i}_{t+1}, D_{t+1} }
|
|
|
|
|
|
|
|
% V^i(\vec s_{t+1}, \vec x^{\sim i}_{t+1}, D_{t+1})
|
|
|
|
|
|
|
|
% \cdot
|
|
|
|
|
|
|
|
% \nabla_{\vec s_t, \vec x^{\sim i}_t, D_t}
|
|
|
|
|
|
|
|
% [ \vec s_{t+1}, \vec x^{\sim i}_{t+1}, D_{t+1} ]
|
|
|
|
|
|
|
|
% \right] \label{EQ:EnvelopeConditions}
|
|
|
|
|
|
|
|
% \\
|
|
|
|
\nabla_{\vec s_t, \vec x^{\sim i}_t, D_t} V^i(\vec s_t, \vec x^{\sim i}_t, D_t)
|
|
|
|
\nabla_{\vec s_t, \vec x^{\sim i}_t, D_t} V^i(\vec s_t, \vec x^{\sim i}_t, D_t)
|
|
|
|
=& \nabla_{\vec s_t, \vec x^{\sim i}_t, D_t} u^i(\vec s_t, D_t) \notag \\
|
|
|
|
=
|
|
|
|
&+ \beta \left[
|
|
|
|
\nabla \vec V^i_t
|
|
|
|
\nabla_{\vec s_{t+1}, \vec x^{\sim i}_{t+1}, D_{t+1} }
|
|
|
|
= \vec u^i
|
|
|
|
V^i(\vec s_{t+1}, \vec x^{\sim i}_{t+1}, D_{t+1})
|
|
|
|
+ \beta B_t \cdot \nabla \vec V^i_{t+1}
|
|
|
|
\cdot
|
|
|
|
|
|
|
|
\nabla_{\vec s_t, \vec x^{\sim i}_t, D_t}
|
|
|
|
|
|
|
|
[ \vec s_{t+1}, \vec x^{\sim i}_{t+1}, D_{t+1} ]
|
|
|
|
|
|
|
|
\right] \label{EQ:EnvelopeConditions}
|
|
|
|
|
|
|
|
\\
|
|
|
|
|
|
|
|
\nabla \vec V^i_t =& \vec u^i
|
|
|
|
|
|
|
|
+ \beta A \cdot \nabla \vec V^i_{t+1}
|
|
|
|
|
|
|
|
\label{EQ:SimplifiedEnvelopeConditions}
|
|
|
|
\label{EQ:SimplifiedEnvelopeConditions}
|
|
|
|
\end{align}
|
|
|
|
\end{align}
|
|
|
|
When interpreting this, note that
|
|
|
|
%When interpreting this, note that
|
|
|
|
$$
|
|
|
|
% $$
|
|
|
|
\nabla \vec V^i_{t+1} = \nabla_{[\vec s_{t+1}~ \vec x^{\sim i}_{t+1}~ D_{t+1}] }
|
|
|
|
% \nabla \vec V^i_{t+1} = \nabla_{[\vec s_{t+1}~ \vec x^{\sim i}_{t+1}~ D_{t+1}] }
|
|
|
|
V^i(\vec s_{t+1}, \vec x^{\sim i}_{t+1}, D_{t+1})
|
|
|
|
% V^i(\vec s_{t+1}, \vec x^{\sim i}_{t+1}, D_{t+1})
|
|
|
|
$$
|
|
|
|
% $$
|
|
|
|
is a $2N \times 1$ vector of first derivatives but
|
|
|
|
% is a $2N \times 1$ vector of first derivatives but
|
|
|
|
$$
|
|
|
|
% $$
|
|
|
|
A = \nabla_{\vec s_t, \vec x^{\sim i}_t, D_t}
|
|
|
|
% A = \nabla_{\vec s_t, \vec x^{\sim i}_t, D_t}
|
|
|
|
[ \vec s_{t+1}~ \vec x^{\sim i}_{t+1}~ D_{t+1} ]
|
|
|
|
% [ \vec s_{t+1}~ \vec x^{\sim i}_{t+1}~ D_{t+1} ]
|
|
|
|
$$
|
|
|
|
% $$
|
|
|
|
is a $2N \times 2N$ matrix of first derivatives.
|
|
|
|
% is a $2N \times 2N$ matrix of first derivatives.
|
|
|
|
|
|
|
|
|
|
|
|
By solving for $\vec V^i_{t+1}$ as a function of $\vec V^i_{t}$ we get the
|
|
|
|
% By solving for $\vec V^i_{t+1}$ as a function of $\vec V^i_{t}$ we get the
|
|
|
|
intertemporal condition:
|
|
|
|
% intertemporal condition:
|
|
|
|
\begin{align}
|
|
|
|
% \begin{align}
|
|
|
|
\frac{1}{\beta} A^{-1} \left(\nabla \vec V^i_t - \vec u^i_t \right)
|
|
|
|
% \frac{1}{\beta} A^{-1} \left(\nabla \vec V^i_t - \vec u^i_t \right)
|
|
|
|
= \nabla \vec V^i_{t+1}
|
|
|
|
% = \nabla \vec V^i_{t+1}
|
|
|
|
\end{align}
|
|
|
|
% \end{align}
|
|
|
|
Thus one crucial condition for the existence of a solution is that $A^{-1}$ exists for
|
|
|
|
% Thus one crucial condition for the existence of a solution is that $A^{-1}$ exists for
|
|
|
|
all values the laws of motion and choice functions can take.
|
|
|
|
% all values the laws of motion and choice functions can take.
|
|
|
|
|
|
|
|
|
|
|
|
% \subsection{Existence}
|
|
|
|
% \subsection{Existence}
|
|
|
|
% I need to do some more diving into conditions for existence.
|
|
|
|
% I need to do some more diving into conditions for existence.
|
|
|
|
@ -104,16 +115,19 @@ all values the laws of motion and choice functions can take.
|
|
|
|
% non-convergence.
|
|
|
|
% non-convergence.
|
|
|
|
%
|
|
|
|
%
|
|
|
|
|
|
|
|
|
|
|
|
Finally, to construct the euler equation, we take
|
|
|
|
To finish constructing the euler equation, we would use the intertemporal
|
|
|
|
\cref{EQ:SimplifiedOptimalityCondition}
|
|
|
|
transition function \cref{EQ:SimplifiedEnvelopeConditions} and iterated
|
|
|
|
and iterate it forward $2N-1$ times.
|
|
|
|
versions of \cref{EQ:OptimalityCondition,EQ:SimplifiedOptimalityCondition}
|
|
|
|
By substituting
|
|
|
|
to construct the $2N+1$ euler equations.\footnote{Double check numbers}
|
|
|
|
\cref{EQ:SimplifiedEnvelopeConditions}
|
|
|
|
Note that for even a small number of agents -- e.g. 3 -- this iterated substitution
|
|
|
|
into each iteration enough times
|
|
|
|
becomes relatively complex, requiring caculating an iterated intertemporal tranisition
|
|
|
|
you get a system that defines $\nabla V^i_t$
|
|
|
|
function and laws of motion 6 times.
|
|
|
|
By substituting this defined value of $\nabla V^i_t$ into
|
|
|
|
To solve this symbolicly involves inverting a $6 \times 6$ matrix.
|
|
|
|
\cref{EQ:SimplifiedOptimalityCondition}
|
|
|
|
As matrix inversion has approximately an $O(n^3)$ computational complexity,
|
|
|
|
one final time, we get a function that fully determines the policy function.
|
|
|
|
this becomes unsustainable very quickly.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Section \cref{SectionOnComputational} describes how to address this issue to generate
|
|
|
|
|
|
|
|
these euler equations using features of modern programming languages and linear algebra
|
|
|
|
|
|
|
|
libraries.
|
|
|
|
|
|
|
|
|
|
|
|
\end{document}
|
|
|
|
\end{document}
|
|
|
|
|