Independent Reviews: unintended consequences
Medians exaggerated
In an oncology trial, whenever you’ve looked at a Kaplan Meier curve of a blinded, independent, central review (BICR) of progression those curves will be exaggerated. More specifically, the median, as estimated by the BICR, will be longer than it should be and longer than it would have been had those same independent reviewers been assessing the patients at the centres. What’s more, the stronger the correlation, between the local(investigator) assessment and the BICR, the worse the bias becomes.
The table shows an example of how the median is over-estimated substantially even when there was no actual bias on the part of investigators.
In an open label trial this could play to the advantage of the comparator if there was a tendency for investigators to call progressions early more often in that arm – this can happen when there are high hopes for the experimental arm, which is given the benefit of the doubt by investigators. The net result could be that the experimental arm fails to show its true advantage even though the intended purpose of the independent review was to get closer to the true relative benefit.
Huh why does this happen!
This happens due to a phenomenon called informative censoring (Dodd*), and it is a consequence of investigators, quite understandably, ceasing to scan patients once they believe they have progressed. For those patients who progress according to the local but not BICR review, who we’ll call local only progressors (LOPs), the natural approach is to censor such patients for the BICR analysis. It’s how those censored patients are treated in the analysis which is the source of the problem.
If you want to fully understand why, read on
We need to delve into the mechanics and workings of the log-rank/cox analysis to uncover what’s going on.
Here goes! The standard tests assume that any censored patient, ie those without the event of interest, would subsequently have progressed at the same rate as patients who continued to be followed. You can think of this as imputing a time based on the average outcome of patients who were followed fully. This is OK for patients censored for reasons unrelated to outcome such as those ‘administratively’ censored who are known not to have progressed at the time of data-cut-off.
Now in our example, this is the same as assuming the LOPs (remember them - the local only progressors), would have progressed at the same rate as patients who, at that timepoint, neither the investigator or BICR believe had progressed. This is an untenable assumption. There will be a +ve correlation between the local and BICR assessments so the analysis will effectively be imputing events too late for the LOPs. The net result being the BICR KM curves are exaggerated and explains why the bias gets bigger as the correlation does.
So what can you do about it?
take a look at my next blog
So in summary great care is required when interpreting BICR data and despite the best of intentions, the results can be misleading due to a mismatch with the assumptions being made in the standard statistical approaches. The BICR might not always be longer than the local review but it will always be longer than it should be.
* Dodd LE et al J Clin Oncol 2008;26:3791–6.
