Establishing diagnostic accuracy of haemoglobin in faecal immunochemical tests (FITs) and establishing a receiver operating characteristic (ROC) curve for FITs in the English primary care symptomatic population: A cohort study in the South West of England
Start Date May 2018
Code E1-C
Status Completed
Introduction
Recently, the NICE guidance DG30 ‘approved’ faecal immunochemical tests (FITs) for use in primary care for persons aged 18 years and over with symptoms potentially indicative of colorectal cancer, without rectal bleeding, who do not meet NICE criteria for referral under the suspected cancer pathway.(1) DG30 presents a review of studies of the diagnostic performance of various FIT assays, but much (if not all) of the evidence presented is from screening or secondary care-based studies. The spectrum effect (the variation in measures of test performance by the prevalence and distribution of disease in the test population)(2) means that the diagnostic performance achieved in those studies may not apply to the symptomatic primary care population. To date, one study has specifically investigated the diagnostic performance and optimum cut off for FIT in a symptomatic primary care population.(3)
The underlying prevalence of colorectal cancer is higher in the secondary care population (and lower in an asymptomatic population undergoing screening) than it will be in the symptomatic primary care population. Therefore, it may not be valid to apply the evidence identified in DG30 to primary care FITs. Additionally, commonly used threshold values for Hb in studies of FIT performance are 50 or 100 ng/ml (ng of Hb per ml of suspension), but there is a high level of between-study variation in this and other features. The NICE guidance (DG30) currently uses a threshold of 10ng/ml. It is difficult to draw any conclusions from the current literature base due to variation in study settings and samples, recruitment strategies, definitions of symptomatic, thresholds, assays, and other factors. Determining the clinically optimum threshold in a primary care population will enable FITs to be used as triage tests to stratify patients by risk and aid GPs in selecting only those with a high risk of colorectal cancer for colonoscopy (current models of care cannot support the sharp increases in urgent referrals to colonoscopy seen over the last few years).(4)
There is an urgent need to develop a policy for use of FIT for symptomatic patients in primary care that is safe in terms of the threshold used for an abnormal value (the balance between sensitivity and specificity; not missing significant numbers of colorectal cancers whilst protecting endoscopic services from overload with healthy patients), and is acceptable to primary care physicians and patients, therefore likely to have a high rate of acceptance when offered.
The proposed diagnostic accuracy study will comprise two phases, each following the same protocol. In phase 1, we will use our international CanTest links to investigate FIT performance in countries currently using FITs on symptomatic patients. Phase 2 will involve applying learnings from the international arm of the study to an English study. For this, we will collect and analyse data from English labs, where FIT testing is only just being introduced. Results from this cohort study will feed into a complementary health economic evaluation of FITs.
Aims & objectives
To determine the clinically optimum threshold for faecal immunochemical tests (FITs) in primary care patients with symptoms potentially indicative of colorectal cancer.
Methodology
A multicentre cohort study, initially in our partner countries and jurisdictions who are already using FITs, and later in England where FIT testing is currently being introduced. We will approach partners in Australia, the USA, Canada, Denmark, Sweden, and the Netherlands to determine current FIT use, policy, and assays, as well as the availability of test data, patient demographic data, and cancer outcome data. Some countries may have additional symptom data. The study sample will comprise patients aged 18 years and over, who have a primary care-ordered FIT at a participating laboratory. As DG30 recommends FIT testing for patients with symptoms that indicate colorectal cancer (but do not qualify for referral), it is to be expected that patients in our English study sample will be symptomatic. In the international arms of the study, we will collect data on reason for FITs, if those data are available. Entry to the study will be at the point of the FIT being carried out. Patients with FITs for screening purposes or ordered from secondary care will be excluded.
The exposure variable will be the patients’ FIT result, and the assay type used by the laboratory. This is crucial, as assays differ in their lower thresholds for detecting Hb. There is variation between assays in how the samples are prepared which affect the value that is returned – and so the clinically optimum threshold value for further investigation. Covariates will include patient age and sex.
The primary outcomes of interest will be diagnoses of colorectal cancer or pre-colorectal cancer (low, medium, or high-risk adenoma), collected from local cancer registries. We will also collect diagnoses of serious bowel disease (inflammatory bowel disease) if these are available. The time at which these diagnoses are made relative to the FIT will be captured; patient follow-up will span one year to capture all incident cases and to estimate false negative rates associated with different Hb threshold values.
A simulation approach was used to establish the sample size to give 95% confidence intervals around a cancer risk of 3%. Assuming a linear relationship in a logistic regression suggests a sample size of 2250 will be sufficient so long as the Hb threshold is contained within the central core of the distribution of FIT levels. This sample size will be applied in each stratum. Whilst a non-linear relationship may increase uncertainty it is expected that by using interactions in logistic regression models, information can be borrowed across strata improving precision.
Data analysis will investigate the relationship between FIT result and the probability of a colorectal cancer diagnosis and assess the diagnostic accuracy of the test in the symptomatic primary care population. Secondary analyses will examine the other disease outcomes. A ROC analysis will be used to address the primary aim of determining a threshold value (or values) for FIT in symptomatic primary care patients, with subgroup analyses by age and sex. Initially this will include all patient data from all included countries and jurisdictions; following this, the study sample will be stratified by assay type. NICE approved assays are: OC Sensor, HM-JACKarc, and FOB Gold. RIDASCREEN has not been approved by NICE. Other assays exist in the international literature and ROC curves specific to each type will be estimated. The variation in thresholds and testing equipment used in different regions will enable us to establish diagnostic performance measures for different thresholds, and for different assays.
In phase 2 of the study (the English arm), for consistency with UK NICE referral guidelines (NG12), we will investigate the threshold FIT level with a positive predictive value (PPV) of 3%, overall and per assay-type. We will also examine other thresholds and their associated PPVs (as our previously published work suggests that the public would request colonoscopy at risks as low as 1%). Currently available FITs are only able to report Hb values as low as 10ng/ml, which may limit the scope of this aspect of the study.
Outputs & impact
Knowledge of FIT performance in symptomatic primary care populations.
Optimum thresholds for this population, stratified by age and sex.
Next steps
The work described above will be complemented by a health economics study led by Marije Van Melle at Cambridge University.
Publications
- Bailey SER, Abel GA, Atkins A, Byford R, Davies S-J, Mays J, McDonald TJ, Miller J, Neck C, Rennison J, Thomas P, Walter FM, Warren S, Hamilton W. Diagnostic performance of a faecal immunochemical test for patients with low-risk symptoms of colorectal cancer in primary care: an evaluation in the South West of England. British Journal of Cancer, 2021, doi: 10.1038/s41416-020-01221-9
Related projects