JobMarketPaper/Paper/sections/06_Results.tex

\documentclass[../Main.tex]{subfiles}

\begin{document}

In this section 
I describe the model fitting, the posteriors of the parameters of interest,
and intepret the results.


\subsection{Estimation Procedure}
I fit the econometric model using mc-stan 
\cite{standevelopmentteam_StanModelling_2022}
through the rstan 
\cite{standevelopmentteam_RStanInterface_2023}
interface.

I had X Trials with X snapshots in total. \todo{Fill out.} 

%describe  
X\todo{UPDATE VALUES} 
warmup iterations and
X\todo{UPDATE VALUES} 
sampling iterations in six chains.

% \subsection{Data Exploration} 
% \todo{fill this out later.}
%look at trial 


\subsection{Primary Results}

The primary, causally-identified value we can estimate is the change in 
the probability of termination caused by (counterfactually) keeping enrollment
open instead of closing enrollment when observed. 
In figure \ref{fig:pred_dist_diff_delay} below, we see this impact of 
keeping enrollment open.


\begin{figure}[H]
    \includegraphics[width=\textwidth]{../assets/img/current/pred_dist_diff-delay}
    \small{
        Values near 1 indicate a near perfect increase in the probability 
        of termination. 
        Values near 0 indicate little change in probability,
        while values near -1, represent a decrease in the probability
        of termination. 
        The scale is in probability points, thus a value near 1 is a change 
        from unlikely to terminate under control, to highly likely to 
        terminate.
    }
    \caption{Distribution of Predicted Differences}
    \label{fig:pred_dist_diff_delay}
\end{figure}

We can see from figure 
\ref{fig:pred_dist_diff_delay} 
That there are roughly four regimes. 
The first consists of trials that experiences nearly no effect,
i.e. have values near zero.
Trials in the second regime experience a mild to large reduction in 
the probability of termination, with X percent of the probability mass 
between about 5 percentage points and 50 percentage point  reductions.
The third regime is those trials that experience a mild to large 
increase in the probability of termination, 
from an increase o 5 percentage points to about 75 percentage points. 
The fourth and final regime is the X\% of trials that experience a significant
(greater than 75 percentage point) increase in the probability of 
termination.
%Notes on interpretation
% - increase vs decrease on graph 
% - 
% - 
% - 
% - 

% The probability mass associated with a each 10 percentage point change are in table \ref{tab:regimes}
% \begin{table}[H]
% \caption{Regimes and associated probability masses}\label{tab:regimes}
% \begin{center}
% \begin{tabular}[c]{l|l}
% 	\hline
% 	\multicolumn{1}{c|}{\textbf{Interval}} & 
% 	\multicolumn{1}{c}{\textbf{Probability Mass}} \\
% 	\hline
% 	$[,]$ & b \\
% 	$[,]$ & b \\
% 	$[,]$ & b \\
% 	$[,]$ & b \\
% 	$[,]$ & b \\
% 	\hline
% \end{tabular}
% \end{center}
% \end{table}

Figure \ref{fig:pred_dist_dif_delay2} shows how this overall
result comes from different disease categories.
\begin{figure}[H]
    \includegraphics[width=\textwidth]{../assets/img/current/pred_dist_diff-delay-group}
    \caption{Distribution of Predicted differences by Disease Group}
    \label{fig:pred_dist_dif_delay2}
\end{figure}

Overall, we can see that there appear to be some trials or situations 
that are highly suceptable to enrollment difficulties, and this 
appears to hold for all disease categories for which I have data.
This relative homogeneity of results may be due to the 
partial pooling effect from the hierarchal model 
and the fact that the sample size per disease is rather small.
An additional explanation is that the variance of the parameter distributions
might be high enough for each trial to have a few situation in which they have
a high probability of terminating.



% Although it is not causally identified due to population interactions,
% we can examine the direct effect from adding a single generic competitior drug
% and how the similar result decomposes very differently.
% This is shown just as a contrast to the enrollment results.
% Figure 
% \label{fig:pred_dist_diff_generic}
% shows a very similar result with roughly the same regimes,
% while 
% \label{fig:pred_dist_dif_generic2}
% shows that this breakdown is different.
% \todo{
%     Consider moving these to an appendix as they are 
%     just additions at this point.
% }
%
% \begin{figure}[H]
%     \includegraphics[width=\textwidth]{../assets/img/current/pred_dist_diff-generic}
%     \caption{
%         Distribution of Predicted Differences for one additional generic 
%         competitor
%     }
%     \label{fig:pred_dist_diff_generic}
% \end{figure}
%
% \begin{figure}[H]
%     \includegraphics[width=\textwidth]{../assets/img/current/pred_dist_diff-generic-group}
%     \caption{}
%     \label{fig:pred_dist_dif_generic2}
% \end{figure}
%


\end{document}