Began updating to fixed analysis run

added the analysis (see html file added)
and added new image results.

Paper/sections/06_Results.tex

@@ -100,15 +100,15 @@ keeping enrollment open.
 
 \begin{table}[H]
     \centering
-    \caption{Boxplot Summary Statistics}
+    \caption{Boxplot Summary Statistics: percentage point due to intervention}
     \label{table:boxplotsummary}
     \begin{tabular}{ | c c c c c c c c | }
         \hline
         5th & 10th & 25th & median & 
         75th & 90th & 95th & mean \\
         \hline
-        -0.376 & -0.264 & -0.129 & -0.023 &
-         0.145 &  0.925 & 0.982  & 0.096 \\
+        -2.1 & -0.8 & 0.0 & 1.2 &
+         4.2 &  8.2 & 11.0  & 2.5 \\
         \hline
     \end{tabular}
 \end{table}
@@ -119,66 +119,16 @@ The key figures from the boxplot in figure
 \ref{fig:pred_dist_diff_delay}
 are sumarized in table \ref{table:boxplotsummary}
 There are a few interesting things to point out here. 
-Let's start by getting aquainted with the details of the distribution above.
-A couple more points
-First, 63\% of the probability mass is equal to or below zero.
-Seconds, about 13\% of the probability mass is contained within the interval 
+First, over 75\% of the probability mass is equal to or above zero, suggesting
+that most trials will experience some harm from a delay in closing enrollment.
+Seconds, about 39.1\% of the probability mass is contained within the interval 
 [-0.01,0.01].
 The full 5\% percentile table can be found in  table
 \ref{TABLE:PercentilesOfDistributionOfDifferences}
+\todo{fix table}
 in appendix
 \ref{Appendix:Results}
 
-It can also be devided into a few different regimes.
-% - spike at 0
-% - the boxplot
-% - 63% of mass below 0 : find better way to say that
-%   - For a random trial, there is a 63% chance that the impact is to reduce the probability of a termination.
-% - 2 pctg-point wide band centered on 0 has ~13% of the masss
-% - mean represents 9.x% increase in probability of termination. A quick simulation gives about the same pctg-point increase in terminated trials.
-% - there are 3 regimes: low impact (near zero), medium impact (concentrated in decreased probability of termination), and high impact (concentrated in increased probability of termination). 
-The first regime consists of the low impact results, i.e. those values of $\delta_p$ 
-near zero. 
-About 13\% of trials lie within a single percentage point change of zero, 
-suggesting that there is a reasonable chance that delaying 
-a close of enrollment has no impact. 
-The second regime consists of the moderate impact on clinical trials'
-probabilities of termination, say values in the interval $[-0.5, 0.5]$ 
-on the graph.
-Most of this probability mass is represents a decrease in the probability of 
-a termination, some of it rather large decreases.
-The third regime consists of the high impact region, 
-almost exclusively concentrated above increases in the probability of 
-termination $\delta_p > 0.75$. 
-These represent cases where delaying the close of enrollemnt changes a trial
-from a case where they were highly likely to complete their primary objectives to 
-a case where they were likely or almost certain to terminate the trial early.
-%   - the high impact regime is strange because it consists of trials that moved from unlikely (<20% chance) of termination to a high chance (>80% chance) of termination. Something like 5% of all trials have a greater than 98 percentage point increase in termination. Not sure what this is doing. 
-
-% Looking at the spike around zero, we find that 13.09% of the probability mass 
-% is contained within the band from [-1,1]. 
-% Additionally, there was 33.4282738% of the probability above that 
-% – representing those with a general increase in the 
-% probability of termination – and 53.4817262% of the probability mass 
-% below the band – representing a decrease in the probability of termination. 
-% On average, if you keep the trial open instead of closing it, 0.6337363% of
-% trials will see a decrease in the probability of termination, but, due to 
-% the high increase in probability of termination given termination was 
-% increased, the mean probability of termination increases by 0.0964726. 
-
-
-% Pulled the data from the report
-% ```{r}
-% summary(pddf_ib$value) 
-% Min.      1st Qu.   Median   Mean    3rd Qu. Max. 
-% -0.99850 -0.12919 -0.02259 0.09647 0.14531 1.00000 
-% quants <- quantile(pddf_ib$value, probs = seq(0,1,0.05), type=4) 
-%    # Convert to a data frame 
-%    quant_df <- data.frame( Percentile = names(quants),  Value = quants ) 
-%    kable(quant_df)
-%    Percentile Value 
-%   SEE TABLE IN APPENDIX
-%```
 
 Figure \ref{fig:pred_dist_dif_delay2} shows how the different disease categories
 tend to have a similar results:
@@ -187,85 +137,81 @@ tend to have a similar results:
     \caption{Distribution of Predicted differences by Disease Group}
     \label{fig:pred_dist_dif_delay2}
 \end{figure}
-Again, note the high mass near zero, the general decrease in the probability
-of termination, and then the strong upper tails.
-
-Continuing to the $\beta$ parameters in figure 
-\ref{fig:parameters_ANR_by_group}, 
-we can see the estimated distributions 
-the status: \textbf{Active, not recruiting}.
-The prior distributions were centered on zero, but we can see that the 
-pooled learning has moved the mean
-values negative, representing reductions in the probability of termination 
-across the board. 
-This decrease in the probability of termination is strongest in the categories of Neoplasms ($n=49$),
-Musculoskeletal diseases ($n=17$), and Infections and Parasites ($n=20$), the three categories with the most data.
-As this is a comparison against the trial status XXX, we note that YYY.
-\todo{The natural comparison I want to make is against the Recruting status. Do I want to redo this so that I can read that directly?It shouldn't affect the $\delta_p$ analysis, but this could probably use it. YES, THIS UPDATE NEEDS TO HAPPEN. The base needs to be ``active not recruiting.''}
-Overall, this is consistent with the result that extending a clinical trial's enrollment period will reduce the probability of termination.
 
-\begin{figure}[H]
-    \includegraphics[width=\textwidth]{../assets/img/betas/parameter_across_groups/parameters_12_status_ANR}
-    \caption{Distribution of parameters associated with ``Active, not recruiting'' status, by ICD-10 Category}
-    \label{fig:parameters_ANR_by_group}
-\end{figure}
-% - 
+% Continuing to the $\beta$ parameters in figure 
+% \ref{fig:parameters_ANR_by_group}, 
+% we can see the estimated distributions 
+% the status: \textbf{Recruiting}.
+% %TOFIX: Discuss how this is a fixed effect with no comparator,i.e. compared 
+% %to the "average" conditions it is an "increase/decrease" in the probability of termination.
+% This xxx in the probability of termination is strongest in the categories of Neoplasms ($n=49$),
+% Musculoskeletal diseases ($n=17$), and Infections and Parasites ($n=20$), the three categories with the most data.
+% % As this is a comparison against the trial status XXX, we note that YYY.
+% % \todo{The natural comparison I want to make is against the Recruting status. Do I want to redo this so that I can read that directly?It shouldn't affect the $\delta_p$ analysis, but this could probably use it. YES, THIS UPDATE NEEDS TO HAPPEN. The base needs to be ``active not recruiting.''}
+% Overall, this is consistent with the result that extending a clinical trial's enrollment period will reduce the probability of termination.
+%
+% \begin{figure}[H]
+%     \includegraphics[width=\textwidth]{../assets/img/betas/parameter_across_groups/parameters_12_status_ANR}
+%     \caption{Distribution of parameters associated with ``Active, not recruiting'' status, by ICD-10 Category}
+%     \label{fig:parameters_ANR_by_group}
+% \end{figure}
+% % - 
 
 
 %   - Potential Explanations for high impact regime:
-This leads to the question:
-``How could this intervention have such a wide range in the intensity 
-and direction of impacts?''
-The most likely explanations in my mind are that either
-some trials are highly suceptable to enrollment struggles or that this is a 
-modelling artifact.
-%       - Some trials are highly suceptable. This is the face value effect
-The first option -- that some trials are more suceptable to 
-issues with participant enrollment -- should allow us to 
-isolate categories or trials that contribute the most to this effect.
-This is not what we find when we inspect the categories
-in figure 
-\ref{fig:pred_dist_dif_delay2}.
-Instead it appears that most of the categories have this high 
-impact regime when $\delta_p > 0.75$, although the maximum value
-of this regime varies considerably.
-
-Another explanation is that this is a modelling artefact due to priors 
-with strong tails and the relatively low number of trials in 
-each ICD-10 categories.
-In short, there might be high levels of uncertanty in some parameter values,
-which manifest as fat tails in the distributions of the $\beta$ parameters. 
-Because of the logistic format of the model, these fat tails lead to 
-extreme values of $p$, and potentally large changes $\delta_p$. 
-I believe that this second explanation -- a model artifact due to uncertanty --
-is likely to be the cause. 
-A few things lead me to believe this:
-\begin{itemize}
-    \item The low fractions of E-BFMI suggest that the sampler is struggling 
-        to explore some regions of the posterior. 
-        According to 
-        \cite{standevelopmentteam_runtimewarningsconvergence_2022}
-        this is 
-        often due to thick tails of posterior distributions. 
-        During earlier analysis, when I had about 100 trials, the number of 
-        warnings was significantly higher.
-    \item When we examine the results across different ICD-10 category, 
-        \ref{fig:pred_dist_dif_delay2}
-        we note that most categories have the same upper tail spike.
-    \item In Figure 
-        % \ref{fig:betas_delay}, 
-        \ref{fig:parameters_ANR_by_group},
-        we see that most ICD-10 categories
-        have fat tails in the $\beta$s, even among the categories 
-        relatively larger sample sizes.
-\end{itemize}
-
-Overally it is hard to escape the conclusion that more data is needed across
-many -- if not all -- of the disease categories.
-At the same time, the median result is a decrease in the probability 
-of termination when the enrollment period is held open.
-My inclination is to believe that the overall effect is to reduce the 
-probability of termination.
+% This leads to the question:
+% ``How could this intervention have such a wide range in the intensity 
+% and direction of impacts?''
+% The most likely explanations in my mind are that either
+% some trials are highly suceptable to enrollment struggles or that this is a 
+% modelling artifact.
+% %       - Some trials are highly suceptable. This is the face value effect
+% The first option -- that some trials are more suceptable to 
+% issues with participant enrollment -- should allow us to 
+% isolate categories or trials that contribute the most to this effect.
+% This is not what we find when we inspect the categories
+% in figure 
+% \ref{fig:pred_dist_dif_delay2}.
+% Instead it appears that most of the categories have this high 
+% impact regime when $\delta_p > 0.75$, although the maximum value
+% of this regime varies considerably.
+%
+% Another explanation is that this is a modelling artefact due to priors 
+% with strong tails and the relatively low number of trials in 
+% each ICD-10 categories.
+% In short, there might be high levels of uncertanty in some parameter values,
+% which manifest as fat tails in the distributions of the $\beta$ parameters. 
+% Because of the logistic format of the model, these fat tails lead to 
+% extreme values of $p$, and potentally large changes $\delta_p$. 
+% I believe that this second explanation -- a model artifact due to uncertanty --
+% is likely to be the cause. 
+% A few things lead me to believe this:
+% \begin{itemize}
+%     \item The low fractions of E-BFMI suggest that the sampler is struggling 
+%         to explore some regions of the posterior. 
+%         According to 
+%         \cite{standevelopmentteam_runtimewarningsconvergence_2022}
+%         this is 
+%         often due to thick tails of posterior distributions. 
+%         During earlier analysis, when I had about 100 trials, the number of 
+%         warnings was significantly higher.
+%     \item When we examine the results across different ICD-10 category, 
+%         \ref{fig:pred_dist_dif_delay2}
+%         we note that most categories have the same upper tail spike.
+%     \item In Figure 
+%         % \ref{fig:betas_delay}, 
+%         \ref{fig:parameters_ANR_by_group},
+%         we see that most ICD-10 categories
+%         have fat tails in the $\beta$s, even among the categories 
+%         relatively larger sample sizes.
+% \end{itemize}
+%
+% Overally it is hard to escape the conclusion that more data is needed across
+% many -- if not all -- of the disease categories.
+% At the same time, the median result is a decrease in the probability 
+% of termination when the enrollment period is held open.
+% My inclination is to believe that the overall effect is to reduce the 
+% probability of termination.
 
 
 \end{document}