Heterogeneity in long COVID and inadequate assessment of post-exertional malaise contribute to issues of indirectness in trials with high risk of bias
Dear Editor
Zeraatkar et al. report that moderate certainty evidence suggests that CBT and physical and mental health rehabilitation probably improve symptoms of long covid [1]. However, despite being presented as a (living) systematic review, the present conclusions are based on results from only a single trial in each case (REGAIN for physical and mental health rehabilitation [2]; and a Dutch trial of CBT [3]). Both of these studies have a high risk of bias, being open-labelled (non-blinded) and reliant on patient-reported outcome measures (subjective questionnaires) [4].
Moreover, the REGAIN study only included patients who experienced severe COVID-19 infection, requiring hospitalization. This review “opted not to rate down the certainty of evidence for indirectness because there is no evidence that currently suggests the effects of the intervention may be different based on severity of the acute COVID-19 infection.” We disagree with this decision and the reasoning behind it. Only a quarter of long COVID patients required hospitalisation for their initial infection [5], and there are data to suggest that non-hospitalization is associated with a higher risk of developing post-COVID-19 ME/CFS [6]. This is likely why the REGAIN study reported no instances of post-exertional malaise (PEM), the cardinal feature of ME/CFS, during the trial or follow-up period, despite PEM being one of the most frequently reported symptoms in patients with long COVID. For these reasons, we suggest that the REGAIN cohort is not representative of the wider long COVID population, and that REGAIN’s conclusions should therefore not have been extrapolated to all long COVID patients, as is the case in this review.
The authors of this review will be aware that similar questions about the quality and certainty of trial evidence are central in the longstanding and ongoing debate around the efficacy and suitability of CBT and graded exercise therapy (GET) for ME/CFS. In fact the decision of the 2021 NICE ME/CFS guideline committee to downgrade the certainty of trial evidence was in part based on the same issue of indirectness concerning the representativeness of trial cohorts with regard to PEM [7].
The heterogeneity of long COVID and current lack of clinical biomarkers to characterise different phenotypes provide challenges in trial design, but adequate identification and reporting of PEM (which should include determining whether patients meet established ME/CFS diagnostic criteria) can help address potential questions about representativeness and patient concerns about the suitability of different interventions. Once again, these trial findings highlight the issue of reliance on patient-reported outcome measures in non-blinded randomized trials, and the importance of objective outcome measures [4].
References
[1] Zeraatkar D, Ling M, Kirsh S, et al. Interventions for the management of long covid (post-covid condition): living systematic review. bmj 2024;387. doi:https://doi.org/10.1136/bmj-2024-081318
[2] McGregor G, Sandhu H, Bruce J, et al, Clinical effectiveness of an online supervised group physical and mental health rehabilitation programme for adults with post-covid-19 condition (REGAIN study): multicentre randomised controlled trial. BMJ 2024;384:e076506. doi:10.1136/bmj-2023-076506
[3] Kuut TA, Müller F, Csorba I, et al, Efficacy of Cognitive-Behavioral Therapy Targeting Severe Fatigue Following Coronavirus Disease 2019: Results of a Randomized Controlled Trial. Clin Infect Dis 2023;77:687-95. doi:10.1093/cid/ciad257
[4] Tack M, Tuller DM, Struthers C. Bias caused by reliance on patient-reported outcome measures in non-blinded randomized trials: an in-depth look at exercise therapy for chronic fatigue syndrome. Fatigue Biomed Health Behav. 2020;8(4):181-192. doi:10.1080/21641846.2020.1848262.
[5] FAIR Health, Patients Diagnosed with Post-COVID Conditions: An Analysis of Private Healthcare Claims Using the Official ICD-10 Diagnostic Code (FAIR Health, 2022). Available at https://s3.amazonaws.com/media2.fairhealth.org/whitepaper/asset/Patients...
[6] Chen CW, Lee HH, Chang SH, Chen YL, Wang YH, Leong PY, Wei JC. Risk of chronic fatigue syndrome after COVID-19: A retrospective cohort study of 3227281 patients. Journal of Infection and Public Health. 2024 Nov 1;17(11):102559. doi:10.1016/j.jiph.2024.102559
[7] Barry PW, Kelley K, Tan T, Finlay I. NICE guideline on ME/CFS: robust advice based on a thorough review of the evidence. Journal of Neurology, Neurosurgery & Psychiatry. 2024 Jul 1;95(7):671-4. doi:10.1136/jnnp-2023-332731
Rapid Response:
Heterogeneity in long COVID and inadequate assessment of post-exertional malaise contribute to issues of indirectness in trials with high risk of bias
Dear Editor
Zeraatkar et al. report that moderate certainty evidence suggests that CBT and physical and mental health rehabilitation probably improve symptoms of long covid [1]. However, despite being presented as a (living) systematic review, the present conclusions are based on results from only a single trial in each case (REGAIN for physical and mental health rehabilitation [2]; and a Dutch trial of CBT [3]). Both of these studies have a high risk of bias, being open-labelled (non-blinded) and reliant on patient-reported outcome measures (subjective questionnaires) [4].
Moreover, the REGAIN study only included patients who experienced severe COVID-19 infection, requiring hospitalization. This review “opted not to rate down the certainty of evidence for indirectness because there is no evidence that currently suggests the effects of the intervention may be different based on severity of the acute COVID-19 infection.” We disagree with this decision and the reasoning behind it. Only a quarter of long COVID patients required hospitalisation for their initial infection [5], and there are data to suggest that non-hospitalization is associated with a higher risk of developing post-COVID-19 ME/CFS [6]. This is likely why the REGAIN study reported no instances of post-exertional malaise (PEM), the cardinal feature of ME/CFS, during the trial or follow-up period, despite PEM being one of the most frequently reported symptoms in patients with long COVID. For these reasons, we suggest that the REGAIN cohort is not representative of the wider long COVID population, and that REGAIN’s conclusions should therefore not have been extrapolated to all long COVID patients, as is the case in this review.
The authors of this review will be aware that similar questions about the quality and certainty of trial evidence are central in the longstanding and ongoing debate around the efficacy and suitability of CBT and graded exercise therapy (GET) for ME/CFS. In fact the decision of the 2021 NICE ME/CFS guideline committee to downgrade the certainty of trial evidence was in part based on the same issue of indirectness concerning the representativeness of trial cohorts with regard to PEM [7].
The heterogeneity of long COVID and current lack of clinical biomarkers to characterise different phenotypes provide challenges in trial design, but adequate identification and reporting of PEM (which should include determining whether patients meet established ME/CFS diagnostic criteria) can help address potential questions about representativeness and patient concerns about the suitability of different interventions. Once again, these trial findings highlight the issue of reliance on patient-reported outcome measures in non-blinded randomized trials, and the importance of objective outcome measures [4].
References
[1] Zeraatkar D, Ling M, Kirsh S, et al. Interventions for the management of long covid (post-covid condition): living systematic review. bmj 2024;387. doi:https://doi.org/10.1136/bmj-2024-081318
[2] McGregor G, Sandhu H, Bruce J, et al, Clinical effectiveness of an online supervised group physical and mental health rehabilitation programme for adults with post-covid-19 condition (REGAIN study): multicentre randomised controlled trial. BMJ 2024;384:e076506. doi:10.1136/bmj-2023-076506
[3] Kuut TA, Müller F, Csorba I, et al, Efficacy of Cognitive-Behavioral Therapy Targeting Severe Fatigue Following Coronavirus Disease 2019: Results of a Randomized Controlled Trial. Clin Infect Dis 2023;77:687-95. doi:10.1093/cid/ciad257
[4] Tack M, Tuller DM, Struthers C. Bias caused by reliance on patient-reported outcome measures in non-blinded randomized trials: an in-depth look at exercise therapy for chronic fatigue syndrome. Fatigue Biomed Health Behav. 2020;8(4):181-192. doi:10.1080/21641846.2020.1848262.
[5] FAIR Health, Patients Diagnosed with Post-COVID Conditions: An Analysis of Private Healthcare Claims Using the Official ICD-10 Diagnostic Code (FAIR Health, 2022). Available at https://s3.amazonaws.com/media2.fairhealth.org/whitepaper/asset/Patients...
[6] Chen CW, Lee HH, Chang SH, Chen YL, Wang YH, Leong PY, Wei JC. Risk of chronic fatigue syndrome after COVID-19: A retrospective cohort study of 3227281 patients. Journal of Infection and Public Health. 2024 Nov 1;17(11):102559. doi:10.1016/j.jiph.2024.102559
[7] Barry PW, Kelley K, Tan T, Finlay I. NICE guideline on ME/CFS: robust advice based on a thorough review of the evidence. Journal of Neurology, Neurosurgery & Psychiatry. 2024 Jul 1;95(7):671-4. doi:10.1136/jnnp-2023-332731
Competing interests: No competing interests