Mathematical window dressing: the art of dealing with recalcitrant data.
A close examination of the 2026 JAMA target-trial emulation shows that mathematical over-correction is being used to substitute real-world biology with statistical fiction.
This article concerns a new paper, published a few weeks ago in JAMA:
Here’s a PDF:
This has been widely reported in breathless and excited terms, as illustrated by the results of a search on Google:
Looking at the top result in the above - the European Medical Journal - by way of example:
The article repeats the study’s contention that the covid vaccine was associated with lower risks of:
major adverse cardiovascular events.
all-cause major adverse cardiovascular events
all-cause hospitalisation
all-cause death
I think it’s worthwhile looking at this study in some detail, as this paper - and the reaction to it - tells us much about what is wrong with science today.
Sanjeev Sabhlok’s claims about the study
Ever since I saw him make a fascinating presentation to Panda a few years ago, I have been following Sanjeev Sabhlok on X, Telegram and Facebook.
He has been giving regular updates on a textbook of public health he is writing. Through analysis of primary sources going back centuries, he seems to be coming to similar conclusions as myself (and no doubt much of the readership here):
Most of what “establishment” medicine has told us - especially if accompanied by a label “public health” - is exaggerated or simply false.
This is a result of the evidence being viewed through the lenses of either ideological or commercial interests.
I’d certainly recommend subscribing to his Telegram channel (here).
Though I applaud the work he is doing on public health, I believe he is still suffering from a blind spot (which I described here) in relation to vaccination.
On this very topic, this post by Sanjeev on Facebook caught my attention:
To save you having to click through to Facebook, it reads as follows:
A bunch of nonsensical claims seem to be floating around on the internet re: a new study: (Link ) eg [link to X post]… which has totally misrepresented the study results.
Here is the study’s MAIN chart (see the data supplement). The study shows that covid vaccines save lives from covid-related causes. But it also shows a CLEAR (BUT SMALL) BENEFIT in terms of all-cause mortality (which is what ultimately counts).
Given the very large size of the sample, this is a powerful result.
Why did this happen? The study suggests that the “vaccine’s protective association extends to the hidden burden of undetected SARS-CoV-2 and its sequelae.”
In other words, many people catch covid routinely and overcome it - therefore it doesn’t show up in the data - but that episode might have a negative effect on bodily systems. This makes sense.
Each time the body has to devote precious resources to fight a bug, there is necessarily a small reduction of resources/ effort on other things.
The covid vaccine seems to provide a much greater benefit (than merely in terms of covid-related outcomes) by reducing the intensity of these minor episodes of covid. This is FAIRLY SMALL, but adds up in the end and might fractionally extend the life expectancy of those who take the covid vaccine.
I could not detect any obvious methodological error in the paper.
This paper has helped me to formulate a new hypothesis re: the immune system - that while catching a mild disease might provide a benefit in terms of generating antibodies, it might also impose a cost in terms of resources diverted away from the maintenance of other bodily systems (I assume the same nutrients used in fighting disease could have otherwise gone into building stronger heart muscles, or better bones). I will note this hypothesis in the PH textbook and look for any validation of this theory.
The following graph from the paper was then inserted below:
I then replied (on Facebook) as follows:
With the greatest respect, I am extremely surprised you cannot see methodological flaws (or at least major questions which need asking).
This is NOT a randomised trial. It is essentially a model using observational data.
The word “randomised” - inserted with abandon into the methodological statement - is designed to dupe the naive.
Time and time again such studies have been shown to be non-reproducible in the field.
Prasad1 writes quite eloquently (and briefly) about this here:
The obvious explanation - that the groups are different at baseline - cannot be adequately adjusted for a priori. This is WHY we do RCTs in the first place.
The idea that off-target benefits are a better explanation than differences between the populations stretches credulity. Even more so since ACM benefits are said to begin accruing by about week 5.
In response, Sanjeev has doubled down on his praise for this study, stating (here) that he “still think[s] the JAMA study makes sense” and here that “the paper is not defective as you [referring to me] seem to think it is”.
However, I do believe that Sanjeev is intellectually honest and willing to consider alternative viewpoints. Later, he asked Miguel Hernán (a Harvard Professor specialising in causality) to opine on the study, stating (here) that “since he has written on the methodology, he can tell us if the methodology was used correctly”.
Miguel Hernan can essentially be regarded as the architect of the methodology employed in the JAMA trial which Sanjeev admired so greatly; in 2016 his seminal paper on the topic was published in the American Journal of Epidemiology (downloadable from here.)
However, I don’t think the question as to whether the methodology was used correctly is the relevant one. A much bigger and more pertinent question is whether such emulated randomised trials are (for all practical purposes) capable of answering complex causal questions in the field of biology2 - not whether the methodology was followed properly.
Asking Hernan to opine on this matter is a bit like asking a proponent of frontal lobotomies to adjudge the validity of a paper reporting on the benefits of the procedure, based on how technically correct the surgeon was in carrying it out.
Prasad has similar doubts (FWIW)
In this piece, Prasad (correctly in my view) calls for trials of this nature to be clearly spelt out as being what they really are - observational studies. He makes the point that time and time again real-world observations have failed to reproduce the effects predicted by “target trials”.
He is clearly as irritated as I am by the language chosen to describe these efforts, which somehow suggests that something akin to randomisation is being done. These are not randomised trials at all, but by the use of that word a veneer of respectability / credibility is being used to hide the true nature of these emulated trials - low quality observational evidence.
A particularly vocal critic of the methodology has been Frank Harrell3, Professor of Biostatistics at Vanderbilt University School of Medicine - who has also had experience advsing FDA on biostatistics (LinkedIn profile). There’s an interesting discussion on the methodology here. Harrel writes:
and
The essential problem with any study design which does not involve randomisation is that there will always be confounding which cannot be corrected for, that is to say a priori differences between the groups being compared which can or do explain some or all of the differences observed.
The authors of papers employing methodology such as that described by Hernan (and used in the JAMA paper we are discussing here) go to huge lengths to correct for such differences, but the fact is that they can only correct for differences which are known, observable and measurable, and for which the relationship between the difference and the effect on the outcome of interest has been reliably and accurately characterised.
He goes into more detail in this piece which is worth a read:
In that, he concludes that:
But straight-up randomized experiments are causal by design. Observational studies should be prospectively designed so that causal inference has a chance. Target trial emulation runs the risk of serving as window dressing for inadequately designed observational studies.
How “good” was the 2026 JAMA trial in terms of methodology?
I doubt very much that Hernan will answer Sanjeev’s request for help on X, so I had a go myself.
The essential nature of the study was as follows:
The study was a “target-trial emulation” using US Veterans Affairs electronic health records from just over 1m veterans, with average age 70. Nearly all (92%) were male.
The study compared adults receiving the 2024–2025 COVID-19 and influenza vaccines together against those receiving just the the influenza vaccine alone.
Attempts to adjust for confounders were made using inverse probability of treatment and censoring weights.
In their methodology, the authors say they corrected for a huge range of covariates:
However, my point is that there will always be unknowns, and, moreover, for the known covariates the mathematical relationship between them and the outcome of interest might not be “nailed down”, as the authors do somewhat cursorily acknowledge (see below).
In 2023 Hansford et al published a systematic review “Reporting of Observational Studies Explicitly Aiming to Emulate Randomized Trials”.
In this, they evaluated how observational studies which explicitly aimed to emulate a target trial were reported in the literature. They discovered widespread inconsistency and detailed exactly where these publications fell short.
I have (with the help of Gemini) created a list of the elements which Hansford et al say should be addressed in an ideal study:
(To be clear, I am not judging the JAMA covid booster study’s validity in terms of assertions of causality by this exercise. I am just answering the question as to how well this study stacks up against the recipe for the methodology, the validity of which I do not accept to answer such complex questions.)
Having read through the paper, it seems that the authors did address all these requirements4 except for one: the paper falls short in terms of discussion over and identification of residual confounders (ie differences between the groups which aren’t corrected for by the methdology), the only mention of the possibility of which can be found in their final sentence of the “Strengths and Limitations” section:
Although analyses were adjusted for a wide range of covariates with validated variable definitions, residual confounding and misclassification cannot be excluded. We did not assess variant-specific vaccine effectiveness. We assumed no interaction between COVID-19 and influenza vaccines. We did not examine vaccine adverse events.
What problems might there have been with residual confounding5?
1. How the known confounders are treated:
The way in which the known confounders are corrected for depends on what is assumed about the relationship between those characteristics and the outcome of interest. In this study, the authors cite a number of papers (numbers 9, 28–36 in their reference list) in this respect.
The quality of these papers would seem to be a critical factor in how well the differences between the groups have been “corrected,” since the model inherits its assumptions from those papers about which factors matter.
I am not going to exhaustively describe those papers here, but would point out that many of them rely on reliable and meaningful testing and categorisation practices, and most utilise test-negative designs (which have many issues6).
Moreover, these papers do in any event themselves specifically mention the following known issues:
The inability to capture behavioral selection trends
Missing data filled in using modeling based on mathematical assumptions
Frailty which persists in the adjusted data, suggesting incomplete correction and hence unmatched groups.
2. Known differences which are simply ignored
Secondly, there are known differences here which have simply been ignored, most notably prior vaccine injury / adverse effects and the corresponding depletion of susceptibles in the treatment group.
Those in the “no covid vaccine” group did not mean “completely unvaccinated since 2020”; it meant “did not receive the updated 2024–2025 seasonal dose on day zero.” As a result, anyone who had received a prior version of the vaccine (like the 2021 booster or the 2023 XBB dose) more than 90 days prior to enrollment was indeed eligible to be placed in the “no vaccine” control group.
The authors did actually know this, so they adjusted for it in the analysis. But rather than separating out and reporting the subgroups, they lumped everyone together, artificially balancing the control and treatment groups so they have the same proportion of prior vaccine exposure. Whilst this does indeed balance things up (whilst potentially hiding certain effects in specific subgroups) the bigger problem here is that this totally ignores the biological reality of why these individuals did or didn’t have a further dose.
If an individual took a COVID vaccine in 2021 or 2022 and suffered an adverse cardiovascular event, a subclinical injury, or significant side effects, they would be much less likely to have a further dose. Therefore, the cohort of people who show up in late 2024 to get a same-day co-administration of both a flu shot and a new COVID booster are, by definition, a highly self-selected group of people who have successfully tolerated multiple prior doses without overt injury.
By pooling everyone who didn’t get the 2024–2025 booster into the control arm, the control arm inherently absorbs those who stopped vaccinating due to prior frailty, side effects, or clinical changes.
Even though the mathematical manipulation (ie inverse probability weighting) matches them “on paper” in terms of having received prior doses, it cannot measure the unrecorded biological reality of why they chose not to get the 2024 dose.
In other words, it treats health history as a static checklist, structurally blind to the reality that those who stop vaccinating frequently do so because they suffered prior adverse experiences, completely unbalancing the comparison groups on day zero. Put differently, the weighting procedure creates a mathematical fiction of balance on paper while leaving massive residual confounding intact in the real world.
3. Unknown unknowns
Thirdly, there may be differences between the groups (ie those who chose to have the covid vaccine along with the flu vaccine and those who didn’t) which are simply unknown. Biology is a complex system, our understanding of which is incomplete. To paraphrase Donald Rumsfeld: “We don’t know what we don’t know.”
Of course, a perfectly effective treatment can disappear in the presence of a perfectly opposing confounder, but equally a completely ineffective treatment can appear highly effective in the presence of a perfectly favourable confounder. Because there are unknown confounders, the debate around studies like this inevitably centres on the plausibility of the data and the biological effects claimed.
Notably, when studies have been published which suggest significant harms from vaccines, they are frequently dismissed as being observational in nature, or otherwise the spectre of missing and / or unknown confounders is raised. This would seem to represent a double standard.
A close look at the data shows that the JAMA trial spews out preposterous conclusions - and even more so than Prasad points out.
In the below article, Prasad makes the point that ACM reductions beginning so soon after dosing are an imprimateur of mismatched groups.
To quote Prasad:
In the supplement it is clear they do not achieve adequate matching. All cause mortality curves separate by day 10. That is simply impossibly fast. The MACE separates by week 5, and the all cause death separates by week 2. The only explanation is that the no vaccine group is fundamentally different from the outset of the study. The healthy vaccinee effect. The paper failed to achieve a suitable control arm.
That’s all that needs to be said. Do not pass go, do not collect 200 dollars. The entire paper, and all included results, is flawed. Not only should it not be in the news, the paper should not be in print. It is hopelessly flawed.
Now, it is clear from his responses, that Sanjeev does NOT think this invalidates the study. In response to this specific point, he states (here):
Not true. Most vaccines start having a partial impact from a very early stage. Vaccines are a super-fast way of producing antibodies. Each severe episode (in this case of covid) averted, saves immune system resources which can be deployed elsewhere. When a sufficiently large population is considered, this partial impact can indeed start showing a small effect on all-cause mortality from its very early days. (E.g. imagine someone with co-morbidities who has a mild episode of covid because of the vaccine - that would enable them to fight other co-morbidities better: the overall difference is minuscule, hence not out of the realm of possibility).
I am not going to comment further on that, other than to say that believing that some cardio-protective effect - from an injection of something with accepted cardio-toxic mechanisms - better explains lower all-cause mortality than differences between the two groups is only explainable by seriously biased priors.
However, the preposterousness of the data isn’t just observable in that particular measure.
In this short NEJM article, the authors (one of which is Prasad) illustrate the absurdity of data reported for a booster trial from Israel, by showing that the data implies that the vaccinated cohort had a 94.8% lower rate of dying from non-COVID causes (like cancer, accidents, or heart attacks) than the unvaccinated group - something which cannot possibly be a biological effect of the injection.
In the recent 2026 JAMA study, a close analysis of the data reveals a similar preposterousness.
Here’s the raw and adjusted data from the paper. Remember that the purpose of the study was “To determine whether the 2024-2025 COVID-19 vaccine is associated with reduced risk of COVID-19–associated MACE7”.
A critical analysis of these findings reveals a stark divergence: the narrative constructed by multivariate mathematical models stands in direct opposition to the raw empiricism of the observed data.
The raw data demonstrates an unadjusted signal of elevated all-cause mortality and cardiovascular events within the vaccinated cohort relative to the unvaccinated. The investigators attribute this asymmetry to baseline confounding, asserting that the vaccinated cohort was inherently older, more comorbid, and characteristically frail. To neutralize this imbalance, the study deploys a complex algorithmic adjustment designed to construct an artificial parity between the groups.
This adjustment, however, forces a total inversion of the data. Excess (all-cause) mortality is reversed, and a small elevated rate of MACE is transformed to suggest significant protection.
When a mathematical adjustment actually reverses the observations, we should be sceptical and ask whether the magnitude of the reversal suggests that the methodology is not simply addressing confounding, but rather introducing artefact through over-correction?
The All-Cause mortality paradox
An analysis of the source of the claimed benefit is instructive. The study makes the following claims for the covid vaccine booster (per 10,000 individuals):
1 prevented covid-related cardiovascular deaths
16 prevented all-cause deaths
24 prevented all-cause cardiovascular events
The intervention appears to prevent far more total deaths than can be attributed to the target pathogen itself.
For this model to mirror reality, one must accept a biological impossibility: that the intervention prevents 16 times more total deaths and 24 times more total cardiac events than can be directly attributed to the target pathogen itself.
To address this paradox, the investigators point to a massive, speculative reservoir of undocumented, subclinical covid-related pathology. In other words, they are asking us to accept that this unmeasured, unverified variable simultaneously serves as the missing confounder and the primary justification for the model’s skewed output.
This is surely circular reasoning. The discrepancies between measured and estimated effects are being re-interpreted as evidence for hidden disease, rather than limitations of the model. The hypothesis has been made impossible to falsify.
Conclusion: modeling for the desired result?
Ultimately, these weighted estimates do not demonstrate that confounding has been eradicated; rather, they expose the extreme sensitivity of observational epidemiology to baseline modeling assumptions.
The more extreme the difference between the raw empirical inputs and the weighted outputs, the more heavily the conclusions rely on the absolute completeness (and correctness) of the model employed.
When a statistical apparatus yields conclusions that happen to align perfectly with establishment narrative but which contradicts the raw data, one has to ask whether the authors have simply devised a mechanism for “dealing with” recalcitrant data.
I should point out that I do have my concerns about Prasad. I think he is much too pharma-friendly and prioritises his postion in the establishment heirarchy over telling the whole truth. I am blocked on X by him - see here.
An under-appreciation of the complexity of systems is a common and recurring theme in much of my writing - see here for example.
A point which only goes to the quality of methodological adherence with the target trial framework, not to its validity as regards identifying causality.
It’s worth emphasising here that proper randomisation deals with all these issues.
Most notably in this context, their design means they only look at the measure of interest, and are blind to “off-target” effects. For instance, they might detect and claim a 5% reduction in “covid deaths” (using covid test positivity as the main measure of interest) from the intervention, while entirely missing a 10% increase in non-covid mortality which may have been caused by the intervention.
Major Cardiac Event