Began updating to fixed analysis run

added the analysis (see html file added)
and added new image results.
cbo
will king 1 year ago
parent 2d67a2db95
commit 7959a32c81

@ -100,15 +100,15 @@ keeping enrollment open.
\begin{table}[H]
\centering
\caption{Boxplot Summary Statistics}
\caption{Boxplot Summary Statistics: percentage point due to intervention}
\label{table:boxplotsummary}
\begin{tabular}{ | c c c c c c c c | }
\hline
5th & 10th & 25th & median &
75th & 90th & 95th & mean \\
\hline
-0.376 & -0.264 & -0.129 & -0.023 &
0.145 & 0.925 & 0.982 & 0.096 \\
-2.1 & -0.8 & 0.0 & 1.2 &
4.2 & 8.2 & 11.0 & 2.5 \\
\hline
\end{tabular}
\end{table}
@ -119,66 +119,16 @@ The key figures from the boxplot in figure
\ref{fig:pred_dist_diff_delay}
are sumarized in table \ref{table:boxplotsummary}
There are a few interesting things to point out here.
Let's start by getting aquainted with the details of the distribution above.
A couple more points
First, 63\% of the probability mass is equal to or below zero.
Seconds, about 13\% of the probability mass is contained within the interval
First, over 75\% of the probability mass is equal to or above zero, suggesting
that most trials will experience some harm from a delay in closing enrollment.
Seconds, about 39.1\% of the probability mass is contained within the interval
[-0.01,0.01].
The full 5\% percentile table can be found in table
\ref{TABLE:PercentilesOfDistributionOfDifferences}
\todo{fix table}
in appendix
\ref{Appendix:Results}
It can also be devided into a few different regimes.
% - spike at 0
% - the boxplot
% - 63% of mass below 0 : find better way to say that
% - For a random trial, there is a 63% chance that the impact is to reduce the probability of a termination.
% - 2 pctg-point wide band centered on 0 has ~13% of the masss
% - mean represents 9.x% increase in probability of termination. A quick simulation gives about the same pctg-point increase in terminated trials.
% - there are 3 regimes: low impact (near zero), medium impact (concentrated in decreased probability of termination), and high impact (concentrated in increased probability of termination).
The first regime consists of the low impact results, i.e. those values of $\delta_p$
near zero.
About 13\% of trials lie within a single percentage point change of zero,
suggesting that there is a reasonable chance that delaying
a close of enrollment has no impact.
The second regime consists of the moderate impact on clinical trials'
probabilities of termination, say values in the interval $[-0.5, 0.5]$
on the graph.
Most of this probability mass is represents a decrease in the probability of
a termination, some of it rather large decreases.
The third regime consists of the high impact region,
almost exclusively concentrated above increases in the probability of
termination $\delta_p > 0.75$.
These represent cases where delaying the close of enrollemnt changes a trial
from a case where they were highly likely to complete their primary objectives to
a case where they were likely or almost certain to terminate the trial early.
% - the high impact regime is strange because it consists of trials that moved from unlikely (<20% chance) of termination to a high chance (>80% chance) of termination. Something like 5% of all trials have a greater than 98 percentage point increase in termination. Not sure what this is doing.
% Looking at the spike around zero, we find that 13.09% of the probability mass
% is contained within the band from [-1,1].
% Additionally, there was 33.4282738% of the probability above that
% representing those with a general increase in the
% probability of termination and 53.4817262% of the probability mass
% below the band representing a decrease in the probability of termination.
% On average, if you keep the trial open instead of closing it, 0.6337363% of
% trials will see a decrease in the probability of termination, but, due to
% the high increase in probability of termination given termination was
% increased, the mean probability of termination increases by 0.0964726.
% Pulled the data from the report
% ```{r}
% summary(pddf_ib$value)
% Min. 1st Qu. Median Mean 3rd Qu. Max.
% -0.99850 -0.12919 -0.02259 0.09647 0.14531 1.00000
% quants <- quantile(pddf_ib$value, probs = seq(0,1,0.05), type=4)
% # Convert to a data frame
% quant_df <- data.frame( Percentile = names(quants), Value = quants )
% kable(quant_df)
% Percentile Value
% SEE TABLE IN APPENDIX
%```
Figure \ref{fig:pred_dist_dif_delay2} shows how the different disease categories
tend to have a similar results:
@ -187,85 +137,81 @@ tend to have a similar results:
\caption{Distribution of Predicted differences by Disease Group}
\label{fig:pred_dist_dif_delay2}
\end{figure}
Again, note the high mass near zero, the general decrease in the probability
of termination, and then the strong upper tails.
Continuing to the $\beta$ parameters in figure
\ref{fig:parameters_ANR_by_group},
we can see the estimated distributions
the status: \textbf{Active, not recruiting}.
The prior distributions were centered on zero, but we can see that the
pooled learning has moved the mean
values negative, representing reductions in the probability of termination
across the board.
This decrease in the probability of termination is strongest in the categories of Neoplasms ($n=49$),
Musculoskeletal diseases ($n=17$), and Infections and Parasites ($n=20$), the three categories with the most data.
As this is a comparison against the trial status XXX, we note that YYY.
\todo{The natural comparison I want to make is against the Recruting status. Do I want to redo this so that I can read that directly?It shouldn't affect the $\delta_p$ analysis, but this could probably use it. YES, THIS UPDATE NEEDS TO HAPPEN. The base needs to be ``active not recruiting.''}
Overall, this is consistent with the result that extending a clinical trial's enrollment period will reduce the probability of termination.
\begin{figure}[H]
\includegraphics[width=\textwidth]{../assets/img/betas/parameter_across_groups/parameters_12_status_ANR}
\caption{Distribution of parameters associated with ``Active, not recruiting'' status, by ICD-10 Category}
\label{fig:parameters_ANR_by_group}
\end{figure}
% -
% Continuing to the $\beta$ parameters in figure
% \ref{fig:parameters_ANR_by_group},
% we can see the estimated distributions
% the status: \textbf{Recruiting}.
% %TOFIX: Discuss how this is a fixed effect with no comparator,i.e. compared
% %to the "average" conditions it is an "increase/decrease" in the probability of termination.
% This xxx in the probability of termination is strongest in the categories of Neoplasms ($n=49$),
% Musculoskeletal diseases ($n=17$), and Infections and Parasites ($n=20$), the three categories with the most data.
% % As this is a comparison against the trial status XXX, we note that YYY.
% % \todo{The natural comparison I want to make is against the Recruting status. Do I want to redo this so that I can read that directly?It shouldn't affect the $\delta_p$ analysis, but this could probably use it. YES, THIS UPDATE NEEDS TO HAPPEN. The base needs to be ``active not recruiting.''}
% Overall, this is consistent with the result that extending a clinical trial's enrollment period will reduce the probability of termination.
%
% \begin{figure}[H]
% \includegraphics[width=\textwidth]{../assets/img/betas/parameter_across_groups/parameters_12_status_ANR}
% \caption{Distribution of parameters associated with ``Active, not recruiting'' status, by ICD-10 Category}
% \label{fig:parameters_ANR_by_group}
% \end{figure}
% % -
% - Potential Explanations for high impact regime:
This leads to the question:
``How could this intervention have such a wide range in the intensity
and direction of impacts?''
The most likely explanations in my mind are that either
some trials are highly suceptable to enrollment struggles or that this is a
modelling artifact.
% - Some trials are highly suceptable. This is the face value effect
The first option -- that some trials are more suceptable to
issues with participant enrollment -- should allow us to
isolate categories or trials that contribute the most to this effect.
This is not what we find when we inspect the categories
in figure
\ref{fig:pred_dist_dif_delay2}.
Instead it appears that most of the categories have this high
impact regime when $\delta_p > 0.75$, although the maximum value
of this regime varies considerably.
Another explanation is that this is a modelling artefact due to priors
with strong tails and the relatively low number of trials in
each ICD-10 categories.
In short, there might be high levels of uncertanty in some parameter values,
which manifest as fat tails in the distributions of the $\beta$ parameters.
Because of the logistic format of the model, these fat tails lead to
extreme values of $p$, and potentally large changes $\delta_p$.
I believe that this second explanation -- a model artifact due to uncertanty --
is likely to be the cause.
A few things lead me to believe this:
\begin{itemize}
\item The low fractions of E-BFMI suggest that the sampler is struggling
to explore some regions of the posterior.
According to
\cite{standevelopmentteam_runtimewarningsconvergence_2022}
this is
often due to thick tails of posterior distributions.
During earlier analysis, when I had about 100 trials, the number of
warnings was significantly higher.
\item When we examine the results across different ICD-10 category,
\ref{fig:pred_dist_dif_delay2}
we note that most categories have the same upper tail spike.
\item In Figure
% \ref{fig:betas_delay},
\ref{fig:parameters_ANR_by_group},
we see that most ICD-10 categories
have fat tails in the $\beta$s, even among the categories
relatively larger sample sizes.
\end{itemize}
Overally it is hard to escape the conclusion that more data is needed across
many -- if not all -- of the disease categories.
At the same time, the median result is a decrease in the probability
of termination when the enrollment period is held open.
My inclination is to believe that the overall effect is to reduce the
probability of termination.
% This leads to the question:
% ``How could this intervention have such a wide range in the intensity
% and direction of impacts?''
% The most likely explanations in my mind are that either
% some trials are highly suceptable to enrollment struggles or that this is a
% modelling artifact.
% % - Some trials are highly suceptable. This is the face value effect
% The first option -- that some trials are more suceptable to
% issues with participant enrollment -- should allow us to
% isolate categories or trials that contribute the most to this effect.
% This is not what we find when we inspect the categories
% in figure
% \ref{fig:pred_dist_dif_delay2}.
% Instead it appears that most of the categories have this high
% impact regime when $\delta_p > 0.75$, although the maximum value
% of this regime varies considerably.
%
% Another explanation is that this is a modelling artefact due to priors
% with strong tails and the relatively low number of trials in
% each ICD-10 categories.
% In short, there might be high levels of uncertanty in some parameter values,
% which manifest as fat tails in the distributions of the $\beta$ parameters.
% Because of the logistic format of the model, these fat tails lead to
% extreme values of $p$, and potentally large changes $\delta_p$.
% I believe that this second explanation -- a model artifact due to uncertanty --
% is likely to be the cause.
% A few things lead me to believe this:
% \begin{itemize}
% \item The low fractions of E-BFMI suggest that the sampler is struggling
% to explore some regions of the posterior.
% According to
% \cite{standevelopmentteam_runtimewarningsconvergence_2022}
% this is
% often due to thick tails of posterior distributions.
% During earlier analysis, when I had about 100 trials, the number of
% warnings was significantly higher.
% \item When we examine the results across different ICD-10 category,
% \ref{fig:pred_dist_dif_delay2}
% we note that most categories have the same upper tail spike.
% \item In Figure
% % \ref{fig:betas_delay},
% \ref{fig:parameters_ANR_by_group},
% we see that most ICD-10 categories
% have fat tails in the $\beta$s, even among the categories
% relatively larger sample sizes.
% \end{itemize}
%
% Overally it is hard to escape the conclusion that more data is needed across
% many -- if not all -- of the disease categories.
% At the same time, the median result is a decrease in the probability
% of termination when the enrollment period is held open.
% My inclination is to believe that the overall effect is to reduce the
% probability of termination.
\end{document}

Binary file not shown.

Before

Width:  |  Height:  |  Size: 399 KiB

After

Width:  |  Height:  |  Size: 358 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 361 KiB

After

Width:  |  Height:  |  Size: 319 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 379 KiB

After

Width:  |  Height:  |  Size: 340 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 366 KiB

After

Width:  |  Height:  |  Size: 326 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 296 KiB

After

Width:  |  Height:  |  Size: 269 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 292 KiB

After

Width:  |  Height:  |  Size: 263 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 342 KiB

After

Width:  |  Height:  |  Size: 308 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 341 KiB

After

Width:  |  Height:  |  Size: 334 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 290 KiB

After

Width:  |  Height:  |  Size: 287 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 218 KiB

After

Width:  |  Height:  |  Size: 196 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 217 KiB

After

Width:  |  Height:  |  Size: 197 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 207 KiB

After

Width:  |  Height:  |  Size: 187 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 198 KiB

After

Width:  |  Height:  |  Size: 181 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 220 KiB

After

Width:  |  Height:  |  Size: 199 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 217 KiB

After

Width:  |  Height:  |  Size: 196 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 219 KiB

After

Width:  |  Height:  |  Size: 197 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 206 KiB

After

Width:  |  Height:  |  Size: 187 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 201 KiB

After

Width:  |  Height:  |  Size: 182 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 217 KiB

After

Width:  |  Height:  |  Size: 198 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 212 KiB

After

Width:  |  Height:  |  Size: 192 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 205 KiB

After

Width:  |  Height:  |  Size: 186 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 198 KiB

After

Width:  |  Height:  |  Size: 180 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 219 KiB

After

Width:  |  Height:  |  Size: 200 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 217 KiB

After

Width:  |  Height:  |  Size: 198 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.9 MiB

After

Width:  |  Height:  |  Size: 2.0 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.4 MiB

After

Width:  |  Height:  |  Size: 1.3 MiB

Some files were not shown because too many files have changed in this diff Show More

Loading…
Cancel
Save