You cannot select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
563 lines
20 KiB
TeX
563 lines
20 KiB
TeX
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|
|
% Beamer Presentation
|
|
% LaTeX Template
|
|
% Version 1.0 (10/11/12)
|
|
%
|
|
% This template has been downloaded from:
|
|
% http://www.LaTeXTemplates.com
|
|
%
|
|
% License:
|
|
% CC BY-NC-SA 3.0 (http://creativecommons.org/licenses/by-nc-sa/3.0/)
|
|
%
|
|
% Changed theme to WSU by William King
|
|
%
|
|
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|
|
|
|
%----------------------------------------------------------------------------------------
|
|
% PACKAGES AND THEMES
|
|
%----------------------------------------------------------------------------------------
|
|
|
|
\documentclass[xcolor=dvipsnames,aspectratio=169]{beamer}
|
|
|
|
|
|
%Import Preamble bits
|
|
\input{../Assets/preambles/FormattingPreamble.tex}
|
|
\input{../Assets/preambles/TikzitPreamble.tex}
|
|
\input{../Assets/preambles/MathPreamble.tex}
|
|
\input{../Assets/preambles/BibPreamble.tex}
|
|
\input{../Assets/preambles/GeneralPreamble.tex}
|
|
|
|
|
|
|
|
|
|
|
|
%----------------------------------------------------------------------------------------
|
|
% TITLE PAGE
|
|
%----------------------------------------------------------------------------------------
|
|
|
|
\title[MDP Constellations]{Modeling decisions in operating satellite constellations}
|
|
|
|
\author{Will King} % Your name
|
|
\institute[WSU] % Your institution as it will appear on the bottom of every slide, may be shorthand to save space
|
|
{
|
|
Washington State University \\ % Your institution for the title page
|
|
\medskip
|
|
\textit{william.f.king@wsu.edu} % Your email address
|
|
}
|
|
\date{\today} % Date, can be changed to a custom date
|
|
|
|
|
|
|
|
|
|
|
|
\begin{document}
|
|
\begin{frame}
|
|
\titlepage % Print the title page as the first slide
|
|
\end{frame}
|
|
|
|
|
|
|
|
%-------------------------------------------------------------------------------------
|
|
%%%%%%%%%%%%%%%%%%%% Developing the Model%%%%%%%%%%%%%%%%%%%%%%%%
|
|
\section{Background}
|
|
% Why should we care?
|
|
% Uses of space
|
|
% Pollution in space and it's impacts
|
|
% Kessler Syndrome
|
|
% What is different now
|
|
%
|
|
%
|
|
%
|
|
%-------------------------------------------------------------------------------------
|
|
\begin{frame}
|
|
\frametitle{Orbital Debris}
|
|
%Story from monday ISS
|
|
|
|
\href{https://edition.cnn.com/2021/11/15/politics/russia-anti-satellite-weapon-test-scn/index.html}{ISS threatened by debris cloud - Monday Nov 15th, 2021}
|
|
\begin{itemize}
|
|
\item Russia conducts an Anti-Satellite Missle Test generating at least 1,500 items of trackable debris
|
|
\item The Astronauts and Cosmonauts on the ISS entered lockdown, including donning pressure suits.
|
|
\item The situation is still being monitored although the immediate danger appears to have passed.
|
|
\end{itemize}
|
|
\end{frame}
|
|
%-------------------------------
|
|
\begin{frame}
|
|
\frametitle{Orbital Debris}
|
|
Other events involving the ISS highlight the dangers from orbital debris:
|
|
\begin{itemize}
|
|
\item ISS conducts 3 evasive maneuvers to doge debris in 2020-
|
|
\href{https://www.jpost.com/science/international-space-station-nearly-struck-by-chinese-satellite-debris-684809}{Jerusalem Post}
|
|
\item ISS hit by debris, May 2021- \href{https://www.asc-csa.gc.ca/eng/iss/news.asp}{Canadian Space Agency}
|
|
\item ISS dodged debris from 2007 Anti-Sat Missile, Nov 2021-
|
|
\href{https://www.jpost.com/science/international-space-station-nearly-struck-by-chinese-satellite-debris-684809}{Jerusalem Post (Same as above)}
|
|
\end{itemize}
|
|
|
|
%This isn't a unique experience
|
|
%list of other issues the ISS has faced
|
|
|
|
\end{frame}
|
|
%-------------------------------
|
|
\begin{frame}
|
|
\frametitle{Orbital Debris}
|
|
%Talk about Fregat breakup
|
|
Not just an issue for manned space flight.
|
|
|
|
|
|
\href{https://www.orbitaldebris.jsc.nasa.gov/quarterly-news/pdfs/odqnv25i1.pdf}{Orbital Debris Quarterly News - NASA}
|
|
\begin{itemize}
|
|
\item In May of 2020, the Satellite SL-23 Zenit Fregat's
|
|
tank suffered a second breakup event.
|
|
\item While only 65 large pieces of debris were initially identified, by Feb. 2021
|
|
over 325 had been attributed to the breakup.
|
|
\item Debris was spread in orbits between 500km and 6,000km.
|
|
\end{itemize}
|
|
|
|
\href{https://www.yahoo.com/news/space-debris-russian-missile-test-175253044.html}{Starlink and recent Anti-Sat test}
|
|
\begin{itemize}
|
|
\item Estimated that there will likely be some impact to Starlink operations.
|
|
\item 1,500 large pieces of debris initially identified.
|
|
\end{itemize}
|
|
\end{frame}
|
|
|
|
%-------------------------------
|
|
\begin{frame}
|
|
\frametitle{Why now?}
|
|
%launch costs
|
|
%nano-satellites
|
|
|
|
In recent years two major changes have occured
|
|
\begin{enumerate}
|
|
\item New launch providers: SpaceX, RocketLab, etc have lead to plummeting launch costs
|
|
\item CubeSates and other Nano-Satellites.
|
|
\end{enumerate}
|
|
\end{frame}
|
|
%-------------------------------
|
|
\begin{frame}
|
|
\frametitle{Goals}
|
|
%Model decision making of satellite operators to be able to investigate policies to reduce kessler syndrome.
|
|
|
|
Goals:
|
|
\begin{itemize}
|
|
\item Model the choices
|
|
facing Satellite Constellation Operators
|
|
and optimal policy policy response.
|
|
\item Investigate the effect of various policies on debris pollution
|
|
\end{itemize}
|
|
\end{frame}
|
|
%-------------------------------
|
|
%-------------------------------------------------------------------------------------
|
|
%%%%%%%%%%%%%%%%%%%% Developing the Model%%%%%%%%%%%%%%%%%%%%%%%%
|
|
\section{TOC}
|
|
%-------------------------------------------------------------------------------------
|
|
\begin{frame}[allowframebreaks] %Allow frame breaks
|
|
\frametitle{Overview} % Table of contents slide, comment this out to remove it
|
|
\tableofcontents
|
|
%Planned TOC
|
|
% See ../outline2.txt
|
|
\end{frame}
|
|
%-------------------------------------------------------------------------------------
|
|
%%%%%%%%%%%%%%%%%%%% Developing the Model%%%%%%%%%%%%%%%%%%%%%%%%
|
|
\section{Literature}
|
|
%-------------------------------------------------------------------------------------
|
|
\begin{frame}
|
|
\frametitle{Past Literature}
|
|
|
|
Key elements of recent literature.
|
|
\begin{enumerate}
|
|
\item \cite{Kessler1978}: Raised issue of runaway orbital pollution.
|
|
\item \cite{Adilov2015}: Described 2 period salop model of interactions.
|
|
\item \cite{Adilov2018,Adilov2018a}: Described an infinite period model with symmetric competitive interactions.
|
|
\item \cite{RaoRondina2020}: Describe a symetric infinite period model (first to do so).
|
|
\item \cite{Rao2020}: Examine the effect of Orbital-Use fees, find it would quadruple long term value produced of the space industry.
|
|
\end{enumerate}
|
|
\end{frame}
|
|
%-------------------------------
|
|
%-------------------------------------------------------------------------------------
|
|
%%%%%%%%%%%%%%%%%%%% Developing the Model%%%%%%%%%%%%%%%%%%%%%%%%
|
|
\section{Model}
|
|
%-------------------------------------------------------------------------------------
|
|
\begin{frame}
|
|
\frametitle{Overview}
|
|
|
|
\begin{itemize}
|
|
\item Mathematical Notation
|
|
\item Law of motion for debris
|
|
\item Law of motion for satellite stocks
|
|
\item Kessler Syndrome
|
|
\item Markov Decision Problems
|
|
\end{itemize}
|
|
\end{frame}
|
|
%-------------------------------
|
|
\begin{frame}
|
|
\frametitle{Mathematical Notation}
|
|
\begin{itemize}
|
|
\item $S_t$: The set of constellation satellites stocks.
|
|
\item $s^i_t$: The number of satellites (stock) for constellation $i$.
|
|
\item $D_t$: The level of debris.
|
|
\item $X_t$: The set of launches.
|
|
\item $x^i_t$: The launches from constellation $i$.
|
|
\end{itemize}
|
|
\end{frame}
|
|
%-------------------------------
|
|
%------------------------------------
|
|
\subsection{Laws of Motion}
|
|
%------------------------------------
|
|
\begin{frame}
|
|
\frametitle{Debris}
|
|
Law of motion for debris
|
|
\begin{align}
|
|
D_{t+1} =& (1-\delta)D_t \tag{Debris decay.} \\
|
|
&+ g\cdot D_t \tag{Debris produced by collision with debris.} \\
|
|
&+ \gamma \sum^N_{i=1} \left(1-R^i(S_t,D_t) \right) s^i_t \tag{Debris produced by satellite destruction.} \\
|
|
&+ \Gamma \sum^n_{j=1} x^j_t \tag{Debris produced by launches.}
|
|
\end{align}
|
|
\end{frame}
|
|
%-------------------------------
|
|
\begin{frame}
|
|
\frametitle{Satellite Stocks}
|
|
|
|
Law of motion for satellite stocks
|
|
\begin{align}
|
|
s^i_{t+1} =& \left(
|
|
R^i(S_t,D_t,X_t)
|
|
- \eta
|
|
\right) \cdot s^i_t
|
|
+ x^i_t
|
|
\end{align}
|
|
\begin{itemize}
|
|
\item $\eta$ is the orbit decay rate of satellites.
|
|
\end{itemize}
|
|
\end{frame}
|
|
%-------------------------------
|
|
%------------------------------------
|
|
\subsection{Kessler Syndrome}
|
|
%------------------------------------
|
|
\begin{frame}
|
|
\frametitle{Explanation of Kessler Syndrome}
|
|
\begin{block}{Kessler Syndrome}
|
|
The situation in which collisions between
|
|
objects in orbit produced debris and this debris begins collisions
|
|
with other objects, leading to a runaway growth in debris.
|
|
As debris can persist for millenia, this may make some orbits unusable.
|
|
|
|
\autocite{Kessler1978}
|
|
\end{block}
|
|
Often described as a condition with an exponential growth of debris.
|
|
|
|
\end{frame}
|
|
%-------------------------------
|
|
\begin{frame}
|
|
\frametitle{Past approaches to Kessler Syndrome}
|
|
\begin{itemize}
|
|
\item \cite{Adilov2018}:\\
|
|
Develops an analog of kessler syndrome where the condition is met when satellites
|
|
are destroyed immediately after launch by debris.
|
|
\begin{align}
|
|
\left\{ (S_t,D_t) : R^i(S_t,D_t) = 0 \forall i\right\}
|
|
\end{align}
|
|
\item \cite{RaoRondina2020}:\\
|
|
A working paper in which the authors develop a dynamic model and a definition of
|
|
kessler syndrome that captures all increasing debris levels.
|
|
\begin{align}
|
|
\left\{ (S_t,D_t) :
|
|
\lim_{t \rightarrow \infty} D_{t+1}(S_t,D_t) = \infty
|
|
\right\}
|
|
\end{align}
|
|
\end{itemize}
|
|
\end{frame}
|
|
%-------------------------------
|
|
\begin{frame}
|
|
\frametitle{My contributions}
|
|
I propose two refinements of these definitions to
|
|
simplify analyzing kessler syndrome in computational models.
|
|
\begin{itemize}
|
|
\item $\epsilon$-Kessler Region
|
|
\item Proto Kessler Region
|
|
\end{itemize}
|
|
\end{frame}
|
|
%-------------------------------------------------------
|
|
\begin{frame}
|
|
\frametitle{$\epsilon$-Kessler Region}
|
|
\begin{block}{$\epsilon$-Kessler Region}
|
|
\begin{align}
|
|
\kappa_\epsilon = \left\{ \left(S_t, D_t \right) : \forall k \geq 0,
|
|
~~ D_{t+k+1} - D_{t+k} \geq \epsilon > 0 \right\}
|
|
\end{align}
|
|
\end{block}
|
|
|
|
Notable Features
|
|
\begin{itemize}
|
|
\item $\epsilon$ can be calibrated to capture only economically significant growth.
|
|
\item Requires an explicit description of what is considered economically significant.
|
|
\item Guarantees divergent behavior.
|
|
\item Simulated transition paths can identify the region.
|
|
\end{itemize}
|
|
\end{frame}
|
|
%-------------------------------------------------------
|
|
\begin{frame}
|
|
\frametitle{Proto Kessler Region}
|
|
|
|
\begin{block}{Proto Kessler Region}
|
|
\begin{align}
|
|
\kappa_\text{proto} =
|
|
\left\{
|
|
\left(S_t,D_t \right) : ~~ D_{t+1} - D_{t} \geq \epsilon_\text{proto}
|
|
\right\}
|
|
\end{align}
|
|
\end{block}
|
|
|
|
Notable Features
|
|
\begin{itemize}
|
|
\item $\epsilon_\text{proto}$ can be calibrated to capture only economically significant growth.
|
|
\item Requires an explicit description of what is considered economically significant.
|
|
\item Does not guarantee divergent behavior.
|
|
\item Easily computable kessler regions.
|
|
\end{itemize}
|
|
\end{frame}
|
|
%-------------------------------------------------------
|
|
\begin{frame}
|
|
\frametitle{Proto Kessler Region}
|
|
With the given law of motion for debris, the proto-kessler region is:
|
|
\begin{align}
|
|
\left\{
|
|
\left(S_t,D_t \right) :
|
|
(g-\delta) D_t
|
|
+ \gamma \sum^n_{i=1} 1-R^i(S_t,D_t)
|
|
+ \Gamma \sum^n_{i=1} x^i_t(S_t,D_t)
|
|
\geq \epsilon_\text{proto}
|
|
\right\}
|
|
\end{align}
|
|
\end{frame}
|
|
%------------------------------------
|
|
\subsection{Markov Decision Problem Formulation}
|
|
%------------------------------------
|
|
\begin{frame}
|
|
\frametitle{Operator's Problem}
|
|
\begin{align}
|
|
V^i(S_t, x^{\sim i}_t, D_t) = \max_{x^i_t} u^i(S_t, D_t) -F(x^i_t)
|
|
+ \beta \left[ V^i(S_{t+1}, x^{\sim i}_{t+1}, D_{t+1}) \right]
|
|
\end{align}
|
|
\end{frame}
|
|
%-------------------------------
|
|
\begin{frame}
|
|
\frametitle{Benefit Functions}
|
|
|
|
Possible benefit functions
|
|
\begin{itemize}
|
|
\item Linear (Currently working on this one)
|
|
\item Cournot Profits
|
|
\item Profits under Partial substitutability
|
|
\item Military capabilities (Keeping up with the Jones')
|
|
\end{itemize}
|
|
|
|
\end{frame}
|
|
%-------------------------------
|
|
\begin{frame}
|
|
\frametitle{Planner's Problem}
|
|
\begin{align}
|
|
W(S_t, D_t) =& \max_{X_t} \left[
|
|
\sum^N_{i=1} \left(u^i(S_t, D_t) - F(x^i_t) \right)
|
|
+ \beta \left[ W(S_{t+1}, D_{t+1}) \right]\right] \notag \\
|
|
&\text{subject to:} \notag \\
|
|
& s^i_{t+1} = (R^i(S_t, D_t)) s^i_t +x^i_t ~~~ \forall i \notag \\
|
|
& D_{t+1} = (1-\delta + g)D_t
|
|
+ \gamma \sum^N_{i=1} \left(1-R^i(\vec s_t, D_t)\right) s^i_t
|
|
+ \Gamma \sum^N_{i=1} x^i_t
|
|
\end{align}
|
|
\end{frame}
|
|
%-------------------------------
|
|
\begin{frame}
|
|
\frametitle{Planned model expansions}
|
|
\begin{itemize}
|
|
\item Multiple interacting orbital shells and debris terms.
|
|
\item Stochastic laws of motion
|
|
\item Multiple types of operators
|
|
\item Operators benefit functions include taxation
|
|
\end{itemize}
|
|
\end{frame}
|
|
%-------------------------------
|
|
%-------------------------------------------------------------------------------------
|
|
%%%%%%%%%%%%%%%%%%%% Developing the Model%%%%%%%%%%%%%%%%%%%%%%%%
|
|
\section{Analysis}
|
|
%-------------------------------------------------------------------------------------
|
|
\begin{frame}
|
|
\frametitle{Issues}
|
|
The following issues characterize the Operators' and Planner's problem
|
|
\begin{itemize}
|
|
\item Curse of Dimensionality
|
|
\item Strategic Interaction (operators only)
|
|
\end{itemize}
|
|
\end{frame}
|
|
%-------------------------------
|
|
\begin{frame}
|
|
\frametitle{Possible approaches}
|
|
|
|
Possible approaches
|
|
\begin{itemize}
|
|
\item Standard VFI/Howards algorithm.
|
|
\item VFI with sparse state space (dimensionality reduction).
|
|
\item Reinforcement Learning.
|
|
\item \cite{MALIAR2018} approaches using machine learning.
|
|
\end{itemize}
|
|
\end{frame}
|
|
%-------------------------------
|
|
\begin{frame}
|
|
\frametitle{Chosen approach}
|
|
% maliar et al - Bellman Residuals
|
|
% present basic approach (using my notation)
|
|
% Discuss basic training loop (use sutton and bartos term Generalized Policy Iteration)
|
|
|
|
Bellman Residual minimization due to \autocite{MALIAR2018}
|
|
|
|
Use NN to approximate $V(S_t,D_t|\theta_1)$ and $X(S_t,D_t|\theta_2)$.
|
|
|
|
The loss function is:
|
|
\begin{align}
|
|
0 =& \left[
|
|
V(S_t, D_t) - F(S_t, D_t, X_t)- \beta V(S_{t+1}, D_{t+1})
|
|
\right]^2 \notag\\
|
|
&- v \left[
|
|
F(S_t, D_t, X_t) + \beta V(S_{t+1}, D_{t+1})
|
|
\right] \\
|
|
0 =& \left[
|
|
V(S_t, D_t) - F(S_t, D_t, X_t)- \beta V(S_{t+1}, D_{t+1})
|
|
- \frac{v}{2}
|
|
\right]^2
|
|
- v \left[
|
|
V(S_{t}, D_{t}) + \frac{v}{4}
|
|
\right]
|
|
\end{align}
|
|
|
|
\end{frame}
|
|
%-------------------------------
|
|
\begin{frame}
|
|
\frametitle{Training Loop: Planner}
|
|
|
|
For each training epoch
|
|
\begin{enumerate}
|
|
\item Draw random data
|
|
\item train policy function
|
|
\item train value function
|
|
\end{enumerate}
|
|
\end{frame}
|
|
%-------------------------------
|
|
\begin{frame}
|
|
\frametitle{Training Loop: Operators}
|
|
|
|
For each training epoch
|
|
\begin{enumerate}
|
|
\item Draw random data
|
|
\item For each operator
|
|
\begin{enumerate}
|
|
\item train policy function
|
|
\item train value function
|
|
\end{enumerate}
|
|
\end{enumerate}
|
|
\end{frame}
|
|
%-------------------------------
|
|
%------------------------------------
|
|
\subsection{Analysis so far}
|
|
%------------------------------------
|
|
\begin{frame}
|
|
\frametitle{State of the Code}
|
|
Currently functioning
|
|
\begin{itemize}
|
|
\item Planner Value and Policy training
|
|
\end{itemize}
|
|
|
|
|
|
Almost functioning
|
|
\begin{itemize}
|
|
\item Operator Value and Policy training
|
|
\item Proto-Kessler Region analysis
|
|
\end{itemize}
|
|
%Planner training
|
|
%Operator training
|
|
\end{frame}
|
|
%-------------------------------
|
|
\begin{frame}
|
|
\frametitle{Results}
|
|
%simulated debris paths (increase the ammount of debris a couple orders of magnitude)
|
|
%Protokessler region plots?
|
|
|
|
Results are currently waiting on finishing the code.
|
|
|
|
\vspace{12pt}
|
|
|
|
Some analyses I plan on completing include
|
|
\begin{itemize}
|
|
\item Kessler Region analysis
|
|
\item Free Entry conditions analysis
|
|
\end{itemize}
|
|
\end{frame}
|
|
%-------------------------------
|
|
%-------------------------------------------------------------------------------------
|
|
%%%%%%%%%%%%%%%%%%%% Developing the Model%%%%%%%%%%%%%%%%%%%%%%%%
|
|
\section{Conclusion}
|
|
%-------------------------------------------------------------------------------------
|
|
%-------------------------------
|
|
\begin{frame}
|
|
\frametitle{Summary}
|
|
Summary
|
|
\begin{enumerate}
|
|
\item Created Dynamic model of the MDP facing satellite operators.
|
|
\item Defined new Kessler Regions for computational analysis.
|
|
\item Currently developing solution and simulation tools.
|
|
\item Much work left to do.
|
|
\end{enumerate}
|
|
\end{frame}
|
|
%-------------------------------
|
|
\begin{frame}
|
|
\frametitle{Other Areas Needing Work}
|
|
|
|
Related Orbits Work
|
|
\begin{enumerate}
|
|
\item Adding stochastic elements to the model.
|
|
\item Parameter Estimation.
|
|
\item Rights of Way.
|
|
\item Satellite Lifetimes and constellation management.
|
|
\end{enumerate}
|
|
|
|
Related computational work
|
|
\begin{enumerate}
|
|
\item Automating the Euler Equation Residuals method.
|
|
\end{enumerate}
|
|
\end{frame}
|
|
%-------------------------------
|
|
\begin{frame}
|
|
\frametitle{Questions?}
|
|
|
|
\center
|
|
Any remaining questions?
|
|
\end{frame}
|
|
%-------------------------------
|
|
\begin{frame}[allowframebreaks]
|
|
\frametitle{References}
|
|
\printbibliography
|
|
\end{frame}
|
|
%\begin{frame}
|
|
% \frametitle{MarginalRevenue}
|
|
% \begin{figure}
|
|
% \tikzfig{../Assets/owned/ch8_MarginalRevenue}
|
|
% \includegraphics[height=\textheight]{../Assets/copyrighted/KrugmanObsterfeldMeliz_fig8-7.jpg}
|
|
% \label{FIG:costs}
|
|
% \caption{Average Cost Curve as firms enter.}
|
|
% \end{figure}
|
|
%\end{frame}
|
|
\end{document}
|
|
% \begin{frame}
|
|
% \frametitle{Columns}
|
|
% \begin{columns}
|
|
% \begin{column}{0.5\textwidth}
|
|
% \end{column}
|
|
% \begin{column}{0.5\textwidth}
|
|
% \begin{figure}
|
|
% \tikzfig{../Assets/owned/ch7_EstablishedAdvantageExample2}
|
|
% \label{FIG:costs}
|
|
% \caption{Setting the Stage}
|
|
% \end{figure}
|
|
% \end{column}
|
|
% \end{columns}
|
|
% \end{frame}
|
|
% %---------------------------------------------------------------
|