Clinical relevance of the borderline results of the Hybrid Capture 2 High-Risk HPV DNA assay with cervical samples collected in Specimen Transport Medium
Article Category: Research Article
Published Online: Sep 24, 2019
Page range: 316 - 322
Received: Jul 10, 2019
Accepted: Aug 06, 2019
DOI: https://doi.org/10.2478/raon-2019-0044
© 2019 Jerneja Varl, Urska Ivanus, Ziva Pohar Marinsek, Tine Jerman, Anja Ostrbenk Valencak, Mario Poljak, Veronika Kloboves Prevodnik, published by Sciendo
This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License.
The Slovenian national cervical cancer screening programme ZORA uses the Hybrid Capture II (HC2) assay (Qiagen, Hilden, Germany) as a triage test that stratifies women with low-grade cervical changes into those with high and low risks for developing cervical cancer.1 The results of the HC2 test are presented as relative light units/ cut-off (RLU/CO) values, and the cut-off value for a positive result is 1.0 RLU/CO2. Many studies have confirmed the high reproducibility of the HC2 test results both within and between laboratories.3, 4, 5 Nevertheless, some studies have noted that analytical accuracy is significantly lower in the vicinity of the cut-off value.3, 4, 5, 6, 7, 8 Therefore, the manufacturer has published instructions for the further management of samples with borderline results that differ regarding which medium is used for sample collection. For PreservCyt specimens (Hologic Inc., Marlborough, United States), the manufacturer proposed the implementation of a borderline RLU/CO area called the “grey zone” in the range of 1.0–2.5 RLU/CO and recommended retesting the samples when results fall within this range.2 However, when storing the samples in the Specimen Transport Medium (STM) (Qiagen, Hilden, Germany), the manufacturer’s instructions are different, recommending retesting only the samples with suspected HPV infection and those with HC2 results near but below the 1.0 RLU/CO value. Retesting can be performed with the HC2 test or using another method.2
Several authors9, 10, 11, 12, 13, 14, 15, 16 have investigated the reproducibility and clinical significance of retesting PreservCyt samples with borderline HC2 results. However, we have not found a single study where the same problem was addressed for specimens collected in STM. A previous Slovenian study by Seme
The aims of our study were (I) to determine the clinical relevance of the HC2 test results within three different “grey zones” for STM samples and (II) to determine whether the current algorithm for retesting “grey zone” STM specimens with the HC2 assay is clinically relevant.
The study population included 596 women who participated in the Slovenian HPV self-sampling project L3-5512 from April 2014 to July 2016. The L3-5512 study protocol has been previously described.17 All women were referred to colposcopy where smears for high-risk HPV testing were obtained. The indications for colposcopy followed the Slovenian national guidelines, including high-grade cytology, HPV positive triage test after repeated low-grade cytology, positive HPV test for the surveillance of the women treated for high-grade intraepithelial lesion (HSIL) and a positive HPV test on self-sampling. Women with abnormal colposcopy underwent colposcopy-guided biopsy, followed by histological evaluation according to WHO recommendations.18 The reported high-grade histological outcomes within one year after colposcopy included cervical intraepithelial neoplasia grade 2 (CIN2), cervical intraepithelial neoplasia grade 3 (CIN3), squamous cell carcinoma, carcinoma with origin outside the cervix, cervical glandular intraepithelial neoplasia grade 2/adenocarcinoma
The study was approved by the National Medical Ethics Committee at the Slovenian Ministry of Health (consents Nos. 155/03/13 and 136/04/14). All participating women provided written informed consent.
For the detection of high-risk HPV, we used the HC2 assay (Qiagen). The HC2 assay was performed according to the manufacturer’s instructions, and the results were reported as positive or negative using 1.0 RLU/CO as the cut-off value.2 Briefly, the test is a nucleic acid hybridization assay with signal amplification using microplate chemiluminescence for the qualitative detection of 13 high-risk types of HPV DNA in cervical and vaginal specimens.2 The results were also interpreted according to the “grey zone” ranges proposed by the manufacturer for PreservCyt (1–2.5 RLU/CO2), Seme
The results are presented as ranges of RLU/CO values for the HPV test result; a proportion of HPV test results in a specific range for all HPV test results; a proportion of women with CIN2/3+ in a specific range for all women with CIN2/3+ in one year since colposcopy; and a risk for CIN2/3+ as a proportion of women with CIN2/3+ for all women in a specific range. Cohen’s kappa with a 95% confidence interval (CI) was calculated as a measure of agreement between the original and retested HPV test results as a binary variable at a 1.0 RLU/CO cut-off value. Sensitivities and specificities were calculated with a 95% CI at different RLU/CO cut-off values, and the ROC curve was plotted. All analyses were conducted with R v3.5.3.19
Our final study group included 594 women after we excluded one woman who had undergone hysterectomy prior to colposcopy and another with missing data from colposcopy. Biopsy was performed in 352 women (59.3%). Out of 594 women, 291 (49.0%) were negative either on colposcopy (242) or on histology (49). A histologically confirmed low-grade intraepithelial lesion (LSIL) was diagnosed in 141 (23.7%) women and CIN2+ was diagnosed in 162 (27.3%) women. There were 48 CIN2, 102 CIN3, 1 VAIN3, 1 VIN3, 4 AIS and 6 squamous carcinomas.
The number of HPV test results within and outside the three “grey zone” ranges are presented in Table 1. The proportion of samples located in 1.0–2.5 RLU/CO, 0.4–4.0 RLU/CO, and 0.7–2.0 RLU/CO was 3.9% (23/594), 10.8% (64/594), and 4.5% (27/594), respectively.
Number of HPV test results according to “grey zones” proposed by the manufacturer (PreservCyt)I, Seme
| RLU/CO value I | N and % women (Ntot = 594) |
|---|---|
| 283 (47.6) | |
| 23 (3.9) | |
| 288 (48.5) |
| RLU/CO value II | N and % women (Tot. N = 594) |
|---|---|
| 253 (42.6) | |
| 30 (5.1) | |
| 34 (5.7) | |
| 277 (46.6) |
| RLU/CO value Ill | N and % women (Tot. N = 594) |
|---|---|
| 271 (45.6) | |
| 12 (2.0) | |
| 15 (2.5) | |
| 296 (49.8) |
N = number; Ntot = total number
All 64 samples with results between 0.4–4.0 RLU/ CO values were retested. In 8/64 (12.5%) retested samples, the results differed from the original results (Figure 1). Six samples for which the results changed from positive to negative originated from patients without CIN2+ lesions, while the seventh case was obtained from a patient with CIN3+ diagnosis. The sample for which the result changed from negative to positive was from a patient without a CIN2+ diagnosis. Kappa agreement between the results before and after retesting was 0.75 (95% CI: 0.59–0.91). Retesting samples with results within the 0.4–4.0 RLU/CO range did not detect any additional CIN2+ cases.
Figure 1
Changes in the RLU/CO values after retesting samples within the 0.4–4.0 RLU/CO range. The red arrow represents the samples with changes in the results from negative to positive, the blue arrows represent samples with changes in the results from positive to negative, and the black arrows represent samples that retained the original result.
The distribution of women with a CIN2+/3+ diagnosis and the risk for CIN2+/3+ based on the RLU/CO values of their HPV test results are presented in Table 2. The majority of women with CIN2+ (85%) had RLU/CO values above 10, and 1.2%, 2.5% % and 5.6% of CIN2+ cases were found within RLU/CO intervals of 0.7–2.0, 1.0–2.5 and 0.4–4.0, respectively. The risk for CIN2+ in women within the 0.4–0.69 and 0.7–0.99 RLU/CO ranges was 5.6% and 8.3%, respectively; however, these results represent one woman per range.
The distribution of women with a CIN2+/3+ diagnosis and the risk for CIN2+/3+ based on the RLU/CO values of their HPV test results
| RLU/CO value | N and % women (Ntot = 594) | N and % CIN2+ (Ntot = 162) | Risk for CIN2+ (%) | N and % CIN3+ (Ntot = 114) | Risk for CIN3+ (%) |
|---|---|---|---|---|---|
| RLU/CO < 0.4 | 253 (42.6) | 8 (4.9) | 3.2 | 2 (1.8) | 0.8 |
| 0.4 ≤ RLU/CO < 0.7 | 18 (3.0) | 1 (0.6) | 5.6 | 1 (0.9) | 5.6 |
| 0.7 ≤ RLU/CO < 1.0 | 12 (2.0) | 1 (0.6) | 8.3 | 1 (0.9) | 8.3 |
| 1.0 ≤ RLU/CO ≤ 2.0 | 15 (2.5) | 1 (0.6) | 6.7 | 1 (0.9) | 6.7 |
| 2.0 < RLU/CO ≤ 2.5 | 8 (1.3) | 3 (1.9) | 37.5 | 2 (1.8) | 25.0 |
| 2.5 < RLU/CO ≤ 4.0 | 11 (1.9) | 3 (1.9) | 27.3 | 2 (1.8) | 18.2 |
| 4.0 < RLU/CO ≤ 10.0 | 26 (4.4) | 7 (4.3) | 26.9 | 6 (5.3) | 23.1 |
| 10.0 < RLU/CO ≤ 100 | 112 (18.9) | 48 (29.6) | 42.9 | 33 (28.9) | 29.5 |
| 100 < RLU/CO ≤ 1000 | 105 (17.7) | 68 (42.0) | 64.8 | 53 (46.5) | 50.5 |
| 1000 < RLU/CO | 34 (5.7) | 22 (13.6) | 64.7 | 13 (11.4) | 38.2 |
CIN2+ = cervical intraepithelial neoplasia grade 2 or greater; CIN3+ = cervical intraepithelial neoplasia grade 3 or greater; N = number; Ntot = total number
Calculations of the sensitivity and specificity of HC2 for CIN2+ at various RLU/CO values are presented in Figure 2. At the threshold recommended by the manufacturer (RLU/CO = 1.0), the sensitivity was 93.8% (95% CI: 90.1–97.5%), and the specificity was 63.2% (95% CI: 58.6–67.6%). Increasing or decreasing the threshold in the vicinity of 1.0 RLU/ CO did not significantly improve one value without lowering the other. Increasing the cut-off value from 1.0 to 2.5 RLU/CO decreased the sensitivity from 93.8% to 91.4% (95% CI: 87.0–95.7%), while the specificity increased from 63.2% to 67.6% (95% CI: 63.0–72.0%). Within the interval of 0.7–2.0 RLU/ CO, the sensitivity remained nearly constant, with 94.4% (95% CI: 90.7–97.5%) at a cut-off value of 0.7
Figure 2
ROC curve demonstrating the sensitivity and specificity of the HC2 test for CIN2+ with marked RLU/CO cutoff values that represent the lower and upper borders of the “grey zone” ranges.
Our results showed that relatively few HC2 test results fell within the “grey zone” ranges currently used by the Institute of Oncology Ljubljana and those proposed by the manufacturer for PreservCyt specimens (3.7% and 4.5%, respectively). The “grey zone” proposed by Seme
We have not found any studies that examined the proportion of STM specimens with HC2 results located in the vicinity of a cut-off value. However, a few studies have reported on the number of HC2 results found in the “grey zone” interval proposed by the manufacturer (1.0–2.5 RLU/CO) for PreservCyt specimens.9, 10, 11 Muldrew
From a clinical point of view, the number of CIN2+ cases found within a “grey zone” is much more relevant than the number of HC2 equivocal results. Among all CIN2+ cases found in our whole study group, only 1.2% was within 0.7–2.0 RLU/ CO, 2.5% was within 1.0–2.5 RLU/CO and 5.6% was within 0.4–4.0 RLU/CO. Although several studies have evaluated the proportion of women with CIN2+ diagnoses among those with HC2 results near the cut-off point, it is difficult to compare our results to theirs. The results vary to a moderate extent, and some differences could be attributed to differences in the studied populations and to the definition of the equivocal results. Knoepp
Since the manufacturer recommends retesting PreservCyt specimens with results within the “grey zone”, several authors have already evaluated whether the retesting algorithm is effective.11, 12, 13, 14, 15, 16 Some authors have performed one retest15, 16, while others have reported two retests13, and some retested only those samples where the results were found within the “grey zone” after the first retest11 or below 1.0 RLU/CO.14 The range of the “grey zone” varied from 0.8 to 3.0 RLU/CO. Some authors have found that the majority of their specimens retained the original positive (87%–97%) HC2 result.11, 12, 13, 14 These results are comparable to our results for the STM specimens, which showed that 87.5% of specimens retained the original HC2 test result after retest, even though our “grey zone” was wider. Ramirez
The expected added value of retesting equivocal HC2 results is a potential increase in the accuracy of the triage test and a better identification of women who have low-risk for developing CIN2+ lesions since these individuals could be returned to screening or to a less intensive follow-up. However, an additional reason that speaks against retesting samples within the “grey zones” is the finding that sensitivity and specificity do not change much within these “grey zone” ranges. Calculating the sensitivity of the HC2 test at different RLU/CO values demonstrated that decreasing the cut-off for positivity to the lower border of the “grey zone” range would achieve a small increase in the sensitivity and therefore would not add a higher accuracy of the test. The sensitivity was 95.1% at a cut-off value of 0.4, 94.4% at the cut-off value of 0.7 and 93.8% at the cut-off value of 1.0 RLU/CO. Several authors have performed similar studies20, 21, 22, 23, 24, 25, 26 and a systematic review by Rebolj
The strength of our study is in reporting results within “grey zones” for STM specimens and investigating two “grey zones” that extended below 1.0 RLU/CO. The results within “grey zones” for STM samples have not been presented before, and most reports included “grey zones” above the proposed cut-off value. The limitation of our study is a small study population, comprising women invited to colposcopy, which showed a higher incidence of CIN2+ lesions. The risk for developing CIN2+ was higher within the investigated “grey zones” (7.4%, 17.4% and 14.1%, respectively) than among the general population since the prevalence of the disease was higher in women referred to colposcopy than in the population of women in our study. Therefore, our findings need to be tested on a larger population with the risk for CIN2+ comparable to that of the population where triage is recommended. An additional limitation of our study is the use of residual samples that were stored in the freezer (-30°C) after the denaturation step for HC2 retesting. This storage procedure could have caused sample degradation and might have influenced our results. These findings will be important for cervical cancer screening programmes that use the HC2 assay as a primary or triage test and collect cervical specimens in STM. The results will help specify the best protocol for handling STM specimens with results within the “grey zone”.
In conclusion, our results show that retesting STM samples within the “grey zones” is not necessary for several reasons. The majority of samples within the “grey zone” retain the original HC2 results after retest. The number of CIN2+ lesions among women with “grey zone” HC2 results is low. There is limited additional value of the retesting algorithm since sensitivity and specificity of HC2 for CIN2+ do not change much within the “grey zone”. Retesting samples with HC2 results in the negative range of the “grey zone” does not increase sensitivity, while retesting in the positive “grey zone” does not add to a less intensive management of women. Women with HC2 results above 1.0 RLU/CO but within the “grey zone” will be followed in the same manner, regardless of the outcome of the retest. Only women with two negative HC2 results will return to regular screening. Furthermore, according to Slovenian clinical guidelines, the management of women with discordant results between the original test and the repeated HC2 test does not allow women to be returned to screening or to a less intensive follow-up, since at least one additional test is needed before the decision could be made about further management.