You cannot select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
105 lines
4.5 KiB
TeX
105 lines
4.5 KiB
TeX
\documentclass[../Main.tex]{subfiles}
|
|
\graphicspath{{\subfix{Assets/img/}}}
|
|
|
|
\begin{document}
|
|
\subsection{Introduction to }
|
|
The computational approach I have decided to take is an application of
|
|
\cite{MALIAR2018}, where the policy function is approximated using a
|
|
neural network.
|
|
|
|
The approach uses the fact that the euler equation implicitly defines the
|
|
optimal policy function, for example:
|
|
$[0] = f(x(\theta),\theta)$.
|
|
This can easily be turned into a mean square loss function by squaring both
|
|
sides,
|
|
$0 = f^2(x(\theta),\theta)$,
|
|
allowing one to find $x(\dot)$ as the solution to a minimization problem.
|
|
|
|
By choosing a neural network as the functional approximation, we are able to
|
|
take advantage of the significant computational and practical improvements
|
|
currently revolutionizing Machine Learning.
|
|
In particular, we can now use common frameworks, such as python, PyTorch,
|
|
and various online accerators (Google Colab)
|
|
which have been optimized for relatively high performance and
|
|
straightforward development.
|
|
|
|
\subsection{Computational Plan}
|
|
I have decided to use python and the PyTorch Neural Network library for this project.
|
|
|
|
The most difficult step is creating the euler equations.
|
|
When working with high dimensioned problems involving differentiation,
|
|
three general computational approaches exist:
|
|
\begin{itemize}
|
|
\item Using a symbolic library (sympy) or language (mathematica) to create the
|
|
euler equations.
|
|
This has the disadvantage of being (very) slow, but the advantage that
|
|
for a single problem specification it only needs completed once.
|
|
It requires taking a matrix inverse, which can easily complicate formulas,
|
|
and is computationally expensive.
|
|
\item Using numerical differentiation (ND).
|
|
The primary issue with ND is that errors can grow quite quickly when
|
|
performing algebra on numerical derivatives.
|
|
This requires tracking how errors can grow and compound within your
|
|
specific formulation of the problem.
|
|
\item Using automatic differentiation (AD) to differentiate the computer code
|
|
directly.
|
|
This approach has a few major benefits.
|
|
\begin{itemize}
|
|
\item Precision is high, because you are calcuating symbolic
|
|
derivatives of your computer functions.
|
|
\item ML is heavily dependent on AD, thus the tools are plentiful
|
|
and tested.
|
|
\item The coupling of AD and ML lead to a tight integration with
|
|
the neural network libraries, simplifying the calibration procedure.
|
|
\end{itemize}
|
|
\end{itemize}
|
|
I have chosen to use the AD to generate a euler equation function, which will
|
|
then be the basis of our loss function.
|
|
|
|
|
|
The first step is to construct the intertemporal transition functions
|
|
(e.g \ref{put_refs_here}).
|
|
%Not sure how much detail to use.
|
|
%I'm debating on describing how it is done.
|
|
These take derivatives of the value function at time $t$ as an input, and output
|
|
derivatives of the value function at time $t+1$.
|
|
Once this function has been finished, it can be combined with the laws of motion
|
|
in an iterated manner to transition between times $t$ and times $t+k$.
|
|
I did so by coding a function that iteratively compose the transition
|
|
and laws of motion functions, retuning a $k$-period transition function.
|
|
|
|
The second step is to generate functions that represent the optimality conditions.
|
|
By taking the appropriate derivatives with respect to the laws of motion and
|
|
utility functions, this can be constructed explicitly.
|
|
|
|
Once these two functions are completed, they can be combined to create
|
|
the euler equations, as described in appendix \ref{appx??}.
|
|
|
|
\subsection{Training}
|
|
|
|
With the euler equation and resulting loss function in place,
|
|
standard training approachs can be used to fit the function.
|
|
I plan on using some variation on stochastic gradient descent.
|
|
|
|
Normally, neural networks are trained on real world data.
|
|
As this is a synthetic model, I am planning on training it on random selections
|
|
from the state space.
|
|
If I can data on how satellites are and have been distributed, I plan on
|
|
selecting from that distribution.
|
|
|
|
\subsections{Extensions}
|
|
One key question is how to handle the case of heterogeneous agents.
|
|
I believe I can address this in the constellation operator's case
|
|
by solving for the policy functions of each class of operator
|
|
simultaneously.
|
|
I still have some questions about this approach and have not dived into
|
|
some of the mathemeatics that deeply.
|
|
|
|
|
|
|
|
\subsection{Existence concerns}
|
|
%check matrix inverses etc.
|
|
%
|
|
|
|
\end{document}
|