Utility of electronic medical recordbased fibrosis scores in predicting advanced cirrhosis in patients with hepatitic C virus infection

Open access

Abstract

Objective

To determine whether advanced cirrhosis - defined by the detection of nodular liver contours or portal venous collaterals on imaging studies - could be predicted by fibrosis algorithms, calculated using laboratory and demographic features extracted from patients’ electronic medical records. To this end, we compared seven EMR-based fibrosis scores with liver imaging studies in a cohort of HCV patients.

Methods

A search of our health system’s patient data warehouse identified 867 patients with chronic HCV infection. A total of 565 patients had undergone at least one liver imaging study and had no confounding medical condition affecting the imaging features or fibrosis scores. Demographic and laboratory data were used to calculate APRI, Fib4, Fibrosis Index, Forns, GUCI, Lok Index and Vira-HepC scores for all viremic patients who had undergone liver imaging. Data points selected for the calculation of these scores were based on laboratory results obtained within the shortest possible time from the imaging study. Areas under the receiver operating curves (AUROC), optimum cut-offs, sensitivities, specificities and positive and negative predictive values were calculated for each score.

Results

Seven algorithms were performed similarly in predicting cirrhosis. Sensitivities ranged from 0.65 to 1.00, specificities from 0.67 to 0.90, positive predictive values from 0.33 to 0.38, and negative predictive values from 0.93 to 1.00. No individual test was superior, as the confidence intervals of all AUROCs overlapped.

Conclusions

EMR-based scoring systems performed relatively well in ruling out advanced, radiologically-defined cirrhosis. However, their moderate sensitivity and positive predictive values limit their reliability for EMR-based diagnosis.

Introduction

Fibrosis staging represents a critical step in the evaluation of patients with chronic hepatitis C virus (HCV) infection. It identifies patients with advanced disease stages who require screening for complications such as hepatocellular cancer or varices, while also being used to prioritize patients for treatment with direct-acting antiviral drugs. [1-3] The conventional staging process typically involves multiple steps, including laboratory testing, liver imaging, elastography or liver biopsy. In turn, this process requires numerous, dedicated patient visits, and is therefore not easily applicable to large patient populations. In recent years, a number of serum-based fibrosis tests have been developed and validated. Many studies have reported reasonable performance characteristics and good agreement with biopsy or elastography data.[4, 5] With the advent of electronic medical record systems, fibrosis scores can be easily calculated after extracting the pertinent test results for each patient. This process does not require additional patient visits, and can be applied to large patient cohorts. As such, EMR-based fibrosis staging might simplify and expedite the population-wide staging of patients with HCV. We recently tested the performance characteristics of several serum-based fibrosis prediction scores in a cohort of patients with chronic hepatitis C, using liver biopsy as the gold standard. [6] We noticed that the scores were better suited to detect higher fibrosis stages, although their overall performance as judged by AUROCs was only moderate. We hypothesized that an EMR-based approach could be used to identify patients with the most advanced fibrosis stage. This hypothesis was tested in a cohort of HCV-infected patients, in a large, integrated healthcare system.

Materials and methods

Ethics committee approval

The Institutional Review Board and Ethics Committee approved the study. It was determined that informed consent need not be obtained from individual patients, since all the patient identifiers had been removed from the data file.

Identification of patients with chronic hepatitis C infection

NorthShore University Heath System maintains a patient database containing key data extracted from the EPIC medical record system. A search of the patient data warehouse was carried out to identify all the patients who had received inpatient or outpatient care between 2003 and 2013, using the search terms for hepatitis C antibody and hepatitis C RNA polymerase chain reaction (PCR) testing. Patients with documented active HCV infection were further analyzed.

Review of imaging studies to determine the absence or presence of cirrhosis

The imaging data of each patient were reviewed. Abdominal ultrasound, CT scan or MRI studies obtained after the establishment of active HCV infection were identified. The radiology reports in the patients’ EMR were reviewed with regard to the presence or absence of descriptive terms of a “nodular liver” or “porto-systemic collaterals”. No additional radiologic review or independent interpretation was performed.[7-9]

Exclusion criteria

The list of patients’ problems, biopsy reports, progress notes and discharge summaries were reviewed to identify the confounding factors that could potentially affect the accuracy of radiologic diagnosis or the calculation of fibrosis scores. We excluded patients who had concomitant causes of liver disease, including alcoholic hepatitis, primary biliary cirrhosis, hemochromatosis and Wilson’s disease. Similarly, patients with documented infiltrative or metastatic liver disease, ongoing chemotherapy, myelofibrosis, aplastic anemia, idiopathic thrombocytopenia and chronic coumadin therapy were excluded, as were patients with cachexia, defined as a BMI of <18.5. HCV viremia, around the time of the liver imaging study, was ascertained. Patients who had undergone HCV treatment prior to their imaging study were excluded.

Calculation of fibrosis scores

Laboratory and demographic data were extracted on all viremic patients who had undergone liver imaging studies. These data were used to calculate the following, previously validated fibrosis scores: aminotransferase to platelet ratio index (APRI),[10] Forns,[11] Vira-HepC,[12] Fibroindex,[13] Lok Index,[14] Göteborg University Cirrhosis Index (GUCI),[15] and Fib-4.[16] The upper limits of normal for AST and ALT used in the calculations were 19IU/ml for women and 30IU/ml for men. We selected laboratory values that were obtained within the shortest possible time span from the imaging study.

Statistical analysis

Statistical software by the SAS Institute (Cary, North Carolina) was used to calculate sensitivity, specificity, positive and negative predictive values, as well as receiver operating characteristics (ROC) associated with each test. The test characteristics were applied to differentiate the presence or absence of radiologic cirrhosis. The 95% confidence interval for the area under the receiver operating characteristics (AUROC) was calculated as suggested by Hanley and McNeil.[17, 18] The statistical methods of this study were reviewed by Dr. Amnon Sonnenberg from Portland VA Medical Center and the Oregon Health & Science University.

Results

Identification of HCV-infected patients

The total NorthShore patient population (2003 to 2013) comprised of 1,040,458 individuals. Of these, 10,571 patients – corresponding to approximately 1% of the overall population – had undergone HCV antibody testing. A total of 867 patients were actively infected upon confirmatory testing. As expected, the majority (604/867, 70%) of viremic patients belonged to the baby boomer generation (1945-1965). Pre- and post-baby boomers contributed 171 (20%) and 92 (10%) patients, respectively.

A total of 565 patients had undergone at least one liver imaging study and met the inclusion criteria. For 147 of the excluded 302 patients, no imaging studies were available in the EMR system. The remaining 155 patients were excluded for a variety of reasons, as listed in Figure 1.

Figure 1
Figure 1

HCV study population. The figure summarizes the inclusion and exclusion criteria of the study. Of 867 patients with proven chronic HCV infection, 147 patients were excluded due to the lack of liver imaging studies on file. 155 patients were excluded due to the following reasons: successful HCV treatment prior to imaging study (n=44, 28.3%; unavailability of imaging studies after establishment of HCV diagnosis (n=42, 27.1%); lab data obtained during hospitalization for an acute illness (n = 17, 10.9%); BMI < 18.5 (n = 12, 7.7%); concomitant hepatocellular carcinoma (n=8, 5.2%); prior liver transplantation (n = 7, 4.5%); end-stage renal disease or nephrotic syndrome (n=7, 4.5%); metastatic liver disease (n =5, 3.2%); active alcohol abuse during lab testing (n=3, 1.9%); and miscellaneous reasons (n=10; 6.5%). HCV: hepatitis C virus.

Citation: Journal of Translational Internal Medicine 5, 1; 10.1515/jtim-2017-0011

ROC curves for detection of advanced fibrosis

Figure 2 shows the receiver operating characteristics of the seven fibrosis tests. The ROC curves were similar, with the exception of the Forns and Fibroindex indices, both of which could be calculated only in a minority of patients.

Figure 2
Figure 2

Receiver Operating Characteristics of serum-based fibrosis scores.

Citation: Journal of Translational Internal Medicine 5, 1; 10.1515/jtim-2017-0011

Table 1 shows a statistical comparison of the areas under the receiver operating curves. Excluding Fibroindex and Forns, the AUROCs ranged from 0.78 to 0.84, corresponding to a “fair” to “good” performance. No single test was statistically superior, as the confidence intervals of all AUROCs overlapped.

Table 1

Area under the receiver operating curves for detection of cirrhosis

Fibrosis indexNTPTNAUROC95% CI
APRI548734750.78(0.71 -0.84)
FIB-4548734750.84(0.78 -0.90)
Fibroindex7714630.78(0.62 -0.93)
Forns623590.96(0.80 -1.00)
GUCI460663940.79(0.72 -0.85)
Lok460663940.83(0.76 -0.89)
Vira-HepC548734750.81(0.75 -0.87)

N: number of patients; TP: true positive; TN: true negative; AUROC: area under the receiver operating curve; CI: 95% confidence interval.

Sensitivity, specificity, negative and positive predictive values

Table 2 summarizes the performance characteristics of the scoring algorithms. Optimal cut-offs were derived from the AUROC analysis. Moderate sensitivities were observed, ranging from 0.66 to 0.84. Specificities ranged from 0.76 to 0.82. Positive predictive values ranged from 0.33 to 0.38, and negative predictive values from 0.93 to 0.97. In general, fibrosis scores were better suited to rule out than rule in advanced radiologic cirrhosis.

Table 2

Performance characteristics of seven fibrosis scores

Fibrosis indexNcut-offsensitivityspecificityPPVNPV
APRI5482.160.660.810.350.94
FIB-45483.000.840.760.350.97
Fibroindex771.570.930.670.380.98
Forns628.481.000.900.331.00
GUCI4602.550.650.820.380.93
Lok4600.560.730.800.380.95
Vira-HepC5480.810.710.780.330.95

N: number of patients; PPV: positive predictive value; NPV: negative predictive value; GUCI: Göteborg University cirrhosis index.

Discussion

The present analysis was prompted by the results of our previous study on the performance characteristics of serum-based fibrosis tests in HCV-infected patients.[6] We observed that the scores performed better at higher fibrosis stages, with the highest AUROCs observed for the detection of cirrhosis. Given this trend, we hypothesized that EMR-based scores might perform best in detecting advanced cirrhosis. Testing this hypothesis in a clinical patient cohort is challenging since such patients rarely undergo a diagnostic liver biopsy. For example, in our previous study, only 5 of 191 (2.6%) clinic patients with chronic HCV who underwent a liver biopsy had Ishak stage 6 fibrosis. In the absence of a tissue diagnosis, we reasoned that the presence of unequivocal radiologic criteria of cirrhosis would represent an acceptable alternative. In our review of the radiological literature, a nodular liver (often described as “undulating liver surface”) and the presence of porto-systemic collaterals were the two most frequently used imaging criteria for cirrhosis.[7-9]

Our study reveals a modest ability of the seven scores to detect advanced cirrhosis, with sensitivities ranging from 0.66 to 0.84, excluding Fibroindex and Forns. Based on this result, the serum-based testing would identify majority of the patients, but would miss approximately one third of patients in this category. A second finding of interest was the relatively poor positive predictive value of all tests (ranging from 0.33 to 0.38), indicating that only approximately one of three patients with high scores would be confirmed to have radiologically-advanced fibrosis. We speculate that some of the “false positive” patients might have earlier-stage or compensated cirrhosis. This interpretation is supported by the known, low sensitivity of imaging studies for cirrhosis.[19]

A third finding of our study was that no single algorithm was significantly superior, as the confidence intervals of the AUROCs for APRI, Fib-4, GUCI, Lok, and Vira-HepC overlapped. We did not have sufficient data to fully evaluate Fibroindex and Forns scores – these tests incorporate less commonly ordered tests such as gamma globulin, total cholesterol and GGT. However, based on the comparison of their positive and negative predictive values with those of the remaining five algorithms, it is doubtful that a larger sample size would have yielded different outcomes. Therefore, we suggest that all the tested algorithms have similar limitations in their utility. This finding is in agreement with a recent meta-analysis by Chou,[19] and with our own recent study.[6]

The fourth major finding was the demonstration that all scoring algorithms had high negative predictive values, indicating that they are well-suited to rule out advanced cirrhosis. This feature could be incorporated into an EMR-based staging approach, as it would identify a lower-risk group. However, the relatively low sensitivities and positive predictive values indicate that additional tests, such as imaging studies or elastography, would still be required for patients with scores above the cut-off, in order to reliably identify patients with advanced cirrhosis.

Our study has a few potential limitations. With regard to the radiologic test results, we accepted the initial radiologist’s interpretation as documented in the patient’s charts, and did not reassess the imaging studies. This approach may result in inter-observer variability, although it appears unlikely that it would result in major differences in interpretation as we included only easily apparent diagnostic imaging features. We are aware of more sophisticated imaging analyses that would result in higher accuracy, sensitivity, and specificity. Such modalities include parenchymal texture analyses, elastography and vascular perfusion studies.[9] However, such measurements would require additional testing of large numbers of patients, which in turn would result in significant logistical obstacles and delays. By comparison, our approach has the advantage of utilizing already existing, “real life” imaging data that can be obtained from standard electronic medical records.

Another limitation pertains to the clinical utility of our findings, given that many patients with advanced imaging features of cirrhosis may already have been diagnosed based on a history of cirrhosis manifestations, complications, or laboratory abnormalities. Ultimately, the clinical value of predicting advanced cirrhosis using this approach will need to be addressed in a prospective study of un-staged HCV patients, including patients in whom no imaging, biopsy, or elastography data may be available. We are in the process of conducting such a study.

In conclusion, our data suggests that fibrosis algorithms are well-suited to rule out advanced cirrhosis. This feature may be of value for an initial, EMR-based staging protocol. However, due to their modest positive predictive values and sensitivities, patients with scores above the cutoff would still have to undergo additional testing to reliably establish the presence of advanced disease. Our data strongly suggest that a definitive “in silico” staging system may not be feasible using the currently available algorithms.

Source of Foundation

Claus J. Fimmel received funding from Gilead Sciences Inc. (Investigator-initiated grant (IN-US-334-1585)) for this project.

Conflict of Interest The authors have no other conflicts of interest to report.

Disclosure

An abstract of this work was presented at the AASLD’s 2015 Liver Meeting in San Francisco, CA. in November, 2015. This abstract received the Presidential Poster of Distinction Award at the aforementioned conference.

References

  • 1

    Foster GR Irving WL Cheung MC Walker AJ Hudson BE Verma S et al. Cohort study of the impact of direct acting antiviral therapy in patients with chronic hepatitis C and decompensated cirrhosis. J Hepatol 2016; 64:1224-31.

    • Crossref
    • Export Citation
  • 2

    Coffin PO Reynolds A. Ending hepatitis C in the United States: the role of screening. Hepat Med 2014;6:79-87.

  • 3

    Sussman NL Remien CH Kanwal F. The end of hepatitis C. Clin Gastroenterol Hepatol 2014;12: 533-6.

    • Crossref
    • Export Citation
  • 4

    Castera L. Non-invasive assessment of liver fibrosis in chronic hepatitis C. Hepatol Int 2011;5:625-34.

    • Crossref
    • Export Citation
  • 5

    Chou R Wasson N. Blood tests to diagnose fibrosis or cirrhosis in patients with chronic hepatitis C virus infection. Ann Int Med 2013;159:372.

    • Crossref
    • Export Citation
  • 6

    Anand V Hyun C Khan QM Hall C Hessefort N Sonnenberg A et al. Identification and Fibrosis Staging of Hepatitis C Patients Using the Electronic Medical Record System. J Clin Gastroenterol 2016.;50:664-9.

    • Crossref
    • Export Citation
  • 7

    Wang JH Changchien CS Hung CH Eng HL Tung WC Kee KM et al. FibroScan and ultrasonography in the prediction of hepatic fibrosis in patients with chronic viral hepatitis. J Gastroenterol 2009;44:439-46.

    • Crossref
    • Export Citation
  • 8

    Colli A Fraquelli M Andreoletti M Marino B Zuccoli E Conte D. Severe liver fibrosis or cirrhosis: accuracy of US for detection--analysis of 300 cases. Radiology. 2003;227:89-94.

    • Crossref
    • Export Citation
  • 9

    Yeom SK Lee CH Cha SH Park CM. Prediction of liver cirrhosis using diagnostic imaging tools. World J Hepatol 2015;7:2069-79.

    • Crossref
    • Export Citation
  • 10

    Wai CT Greenson JK Fontana RJ Kalbfleisch JD Marrero JA Conjeevaram HS et al. A simple noninvasive index can predict both significant fibrosis and cirrhosis in patients with chronic hepatitis C. Hepatology 2003;38:518-26.

    • Crossref
    • Export Citation
  • 11

    Forns X Ampurdanes S Llovet JM Aponte J Quinto L Martinez-Bauer E et al. Identification of chronic hepatitis C patients without hepatic fibrosis by a simple predictive model. Hepatology 2002;36(4 Pt 1):986-92.

  • 12

    Fontana RJ Kleiner DE Bilonick R Terrault N Afdhal N Belle SH et al. Modeling hepatic fibrosis in African American and Caucasian American patients with chronic hepatitis C virus infection. Hepatology 2006;44:925-35.

    • Crossref
    • Export Citation
  • 13

    Koda M Matunaga Y Kawakami M Kishimoto Y Suou T Murawaki Y. FibroIndex a practical index for predicting significant fibrosis in patients with chronic hepatitis C. Hepatology 2007;45:297-306.

    • Crossref
    • Export Citation
  • 14

    Lok AS Ghany MG Goodman ZD Wright EC Everson GT Sterling RK et al. Predicting cirrhosis in patients with hepatitis C based on standard laboratory tests: results of the HALT-C cohort. Hepatology 2005;42:282-92.

    • Crossref
    • Export Citation
  • 15

    Westin J Ydreborg M Islam S Alsio A Dhillon AP Pawlotsky JM et al. A non-invasive fibrosis score predicts treatment outcome in chronic hepatitis C virus infection. Scand J Gastroenterol 2008;43:73-80.

    • Crossref
    • Export Citation
  • 16

    Vallet-Pichard A Mallet V Nalpas B Verkarre V Nalpas A Dhalluin-Venier V et al. FIB-4: an inexpensive and accurate marker of fibrosis in HCV infection. comparison with liver biopsy and fibrotest. Hepatology 2007;46:32-6.

    • Crossref
    • Export Citation
  • 17

    Hanley JA McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982;143:29-36.

    • Crossref
    • Export Citation
  • 18

    Hanley JA McNeil BJ. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology 1983;148: 839-43.

    • Crossref
    • Export Citation
  • 19

    Chou R Wasson N. Blood tests to diagnose fibrosis or cirrhosis in patients with chronic hepatitis C virus infection: a systematic review. Ann Intern Med 2013;158:807-20.

    • Crossref
    • Export Citation

If the inline PDF is not rendering correctly, you can download the PDF file here.

  • 1

    Foster GR Irving WL Cheung MC Walker AJ Hudson BE Verma S et al. Cohort study of the impact of direct acting antiviral therapy in patients with chronic hepatitis C and decompensated cirrhosis. J Hepatol 2016; 64:1224-31.

    • Crossref
    • Export Citation
  • 2

    Coffin PO Reynolds A. Ending hepatitis C in the United States: the role of screening. Hepat Med 2014;6:79-87.

  • 3

    Sussman NL Remien CH Kanwal F. The end of hepatitis C. Clin Gastroenterol Hepatol 2014;12: 533-6.

    • Crossref
    • Export Citation
  • 4

    Castera L. Non-invasive assessment of liver fibrosis in chronic hepatitis C. Hepatol Int 2011;5:625-34.

    • Crossref
    • Export Citation
  • 5

    Chou R Wasson N. Blood tests to diagnose fibrosis or cirrhosis in patients with chronic hepatitis C virus infection. Ann Int Med 2013;159:372.

    • Crossref
    • Export Citation
  • 6

    Anand V Hyun C Khan QM Hall C Hessefort N Sonnenberg A et al. Identification and Fibrosis Staging of Hepatitis C Patients Using the Electronic Medical Record System. J Clin Gastroenterol 2016.;50:664-9.

    • Crossref
    • Export Citation
  • 7

    Wang JH Changchien CS Hung CH Eng HL Tung WC Kee KM et al. FibroScan and ultrasonography in the prediction of hepatic fibrosis in patients with chronic viral hepatitis. J Gastroenterol 2009;44:439-46.

    • Crossref
    • Export Citation
  • 8

    Colli A Fraquelli M Andreoletti M Marino B Zuccoli E Conte D. Severe liver fibrosis or cirrhosis: accuracy of US for detection--analysis of 300 cases. Radiology. 2003;227:89-94.

    • Crossref
    • Export Citation
  • 9

    Yeom SK Lee CH Cha SH Park CM. Prediction of liver cirrhosis using diagnostic imaging tools. World J Hepatol 2015;7:2069-79.

    • Crossref
    • Export Citation
  • 10

    Wai CT Greenson JK Fontana RJ Kalbfleisch JD Marrero JA Conjeevaram HS et al. A simple noninvasive index can predict both significant fibrosis and cirrhosis in patients with chronic hepatitis C. Hepatology 2003;38:518-26.

    • Crossref
    • Export Citation
  • 11

    Forns X Ampurdanes S Llovet JM Aponte J Quinto L Martinez-Bauer E et al. Identification of chronic hepatitis C patients without hepatic fibrosis by a simple predictive model. Hepatology 2002;36(4 Pt 1):986-92.

  • 12

    Fontana RJ Kleiner DE Bilonick R Terrault N Afdhal N Belle SH et al. Modeling hepatic fibrosis in African American and Caucasian American patients with chronic hepatitis C virus infection. Hepatology 2006;44:925-35.

    • Crossref
    • Export Citation
  • 13

    Koda M Matunaga Y Kawakami M Kishimoto Y Suou T Murawaki Y. FibroIndex a practical index for predicting significant fibrosis in patients with chronic hepatitis C. Hepatology 2007;45:297-306.

    • Crossref
    • Export Citation
  • 14

    Lok AS Ghany MG Goodman ZD Wright EC Everson GT Sterling RK et al. Predicting cirrhosis in patients with hepatitis C based on standard laboratory tests: results of the HALT-C cohort. Hepatology 2005;42:282-92.

    • Crossref
    • Export Citation
  • 15

    Westin J Ydreborg M Islam S Alsio A Dhillon AP Pawlotsky JM et al. A non-invasive fibrosis score predicts treatment outcome in chronic hepatitis C virus infection. Scand J Gastroenterol 2008;43:73-80.

    • Crossref
    • Export Citation
  • 16

    Vallet-Pichard A Mallet V Nalpas B Verkarre V Nalpas A Dhalluin-Venier V et al. FIB-4: an inexpensive and accurate marker of fibrosis in HCV infection. comparison with liver biopsy and fibrotest. Hepatology 2007;46:32-6.

    • Crossref
    • Export Citation
  • 17

    Hanley JA McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982;143:29-36.

    • Crossref
    • Export Citation
  • 18

    Hanley JA McNeil BJ. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology 1983;148: 839-43.

    • Crossref
    • Export Citation
  • 19

    Chou R Wasson N. Blood tests to diagnose fibrosis or cirrhosis in patients with chronic hepatitis C virus infection: a systematic review. Ann Intern Med 2013;158:807-20.

    • Crossref
    • Export Citation
Search
Journal information
Figures
  • View in gallery

    HCV study population. The figure summarizes the inclusion and exclusion criteria of the study. Of 867 patients with proven chronic HCV infection, 147 patients were excluded due to the lack of liver imaging studies on file. 155 patients were excluded due to the following reasons: successful HCV treatment prior to imaging study (n=44, 28.3%; unavailability of imaging studies after establishment of HCV diagnosis (n=42, 27.1%); lab data obtained during hospitalization for an acute illness (n = 17, 10.9%); BMI < 18.5 (n = 12, 7.7%); concomitant hepatocellular carcinoma (n=8, 5.2%); prior liver transplantation (n = 7, 4.5%); end-stage renal disease or nephrotic syndrome (n=7, 4.5%); metastatic liver disease (n =5, 3.2%); active alcohol abuse during lab testing (n=3, 1.9%); and miscellaneous reasons (n=10; 6.5%). HCV: hepatitis C virus.

  • View in gallery

    Receiver Operating Characteristics of serum-based fibrosis scores.

Cited By
Metrics
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 344 181 4
PDF Downloads 92 47 3