Prediction of Human Papillomavirus (HPV) Status in Oropharyngeal Squamous Cell Carcinoma Based on Radiomics and Machine Learning Algorithms: A Multi-Cohort Study

Liang Zhinan, Zhang Wei, You Yudi, Dong Yabing, Xiao Yuanzhe and Liu Xiulan^{* *}Correspondence: Liu Xiulan, Department of Radiation Oncology, Fengcheng Hospital of Fengxian District, Shanghai, China, Email:

Author info »

Abbreviations

ACC: Accuracy ; AUC: Area Under the Receiver Operating Characteristic (ROC) Curve; AJCC: American Joint Cancer Committee; CT: Computed Tomography; DICOM: Digital Imaging and Communication in Medicine; DT: Decision Tree; ET: Extra Trees; GLCM: Gray-Level Co-Occurrence Matrix; GLDM: Gray-Level Dependence Matrix; GLSZM: Gray-Level Size-Zone Matrix; GLRLM: Gray-Level Run-Length Matrix; GTV: Gross Tumor Volume; GE: General Electric; HPV: Human Papillomavirus; IBSI: Image Biomarkers Standardization Initiative; KNN: K-Nearest Neighbors; LGBM: Light Gradient Boosting Machine; LoG: Laplacian Gaussian; LR: Logistic Regression; MRI: Magnetic Resonance Imaging; NB: Naive Bayes; NGTDM: Neighboring GrayTone Difference Matrix; OPSCC: Oropharyngeal Squamous Cell Carcinoma; RTSTRUCT: Radiation Therapy Structures; RF: Random Forest; RFE: Recursive Feature Elimination; SGD: Stochastic Gradient Descent; SVM: Support Vector Machine; SNPH: Shanghai Ninth People’s Hospital; TCIA: The Cancer Imaging Archive; TNM: Tumor Node Metastasis; XGB: Extreme Gradient Boosting

Introduction

Oropharyngeal Squamous Cell Carcinoma (OPSCC), which is often associated with Human Papillomavirus (HPV) infection, has been increasing rapidly over the past 20 years (Lechner M, et al., 2022). The prognosis of patients with HPV-positive OPSCC is better than that of those with HPV-negative OPSCC, as the former is more likely to respond to radiotherapy and chemotherapy. American Joint Cancer Committee found that HPV-related OP-SCC is different from HPV-negative cancer, so it is classified as a unique entity with independent staging rules. Therefore, the differentiation of these two cancer types has significant implications for prognostic evaluation and clinical decision-making (Ang KK, et al., 2010).

The radiomics method, which provides a reliable assessment of HPV status viaquantitative imaging analysis, may be an attractive approach to determining HPV types. Some studies have reported the use of preoperative Computed Tomography (CT) or Magnetic Resonance Imaging (MRI) for determining the HPV status of patients with OPSCC (Freihat O, et al., 2021; Bagher‐Ebadian H, et al., 2021; Fomin I, et al., 2021). However, the majority of studies investigating the imaging phenotype of tumors are limited to single centers, which lack reproducibility and validation of current radiomic models (Avanzo M, et al., 2020). In addition, when constructing classification models for differentiating HPV status, there are often controversies in the choice of the best machine learning algorithm. Therefore, the aim of our study was to develop and independently validate a better machine learning-based radiomics model for the prediction of HPV phenotype in OPSCC based on CT scans from multi-institutional and multi-national cohorts.

Materials and Methods

Patient population

Three OPSCC cohorts were included in this research: The Cancer Imaging Archive (TCIA) OPC-Radiomics cohort (n=606) (Kwan JY, et al., 2018; Clark K, et al., 2013), TCIA Head-Neck-Radiomics-HN1 cohort from the Netherlands (“MAASTRO” cohort, n=89) (Aerts HJ, et al., 2014), and our institution cohort from Shanghai Ninth People’s Hospital (“SNPH cohort”, n=108). For the SNPH cohort, 108 consecutive patients, who underwent pretreatment contrast CT from January 2018 to January 2021 for newly diagnosed and pathologically confirmed OPSCC, were involved in this study. Patients with the following criteria were included: (1) the pretreatment CT scan of the neck is available; (2) biopsy-proven OPSCC; (3) and the HPV status was determined by p16 immunohistochemistry. Patients with the following criteria were excluded: (1) Radiation Therapy Structures (RTSTRUCT) of Gross Tumor Volume (GTV) is not available; (2) lacking HPV status by p16 immunohistochemistry or (3) CT images containing obvious artifacts. Based on the patient inclusion and exclusion criteria, 497 patients from the OPC-Radiomics cohort, 74 patients from the MAASTRO cohort, and 80 patients from the SNPH cohort were included for further analysis. The OPC-Radiomics cohort was randomly sampled into training cohort (n=332) and validation cohort (n=165) with a ratio of 2:1. MAASTRO cohort and SNPH cohort were used as independent external testing cohorts. The process of patient registration is shown in Figure 1.

Figure 1: Flow chart of patient selection process

The data is identified by TCIA collection and custody, and the provider is responsible for consent and approval. Institutional review board approval for retrospective analysis of clinical data was granted, and informed consent was waived.

Tumor segmentation

In the two TCIA cohorts, GTV was obtained using the RTSTRUCT. GTV segmentation in the SNPH cohort was manually segmented by one radiation oncologist with 5 years of head-and-neck imaging experience and revised by another radiation oncologist with 20 years of clinical expertise using 3D Slicer software (Fedorov A, et al., 2012).

CT imaging

The CT scanners of the SNPH cohort included General Electric (GE) Discovery CT750 HD, GE Light Speed 16, and Siemens SOMATOM Definition Flash with the following specifications: 0.625 mm thickness, 60-280 mA at 120 kVp, and matrix size=512 × 512 pixels. Helical CT scans were performed with a slice thickness of 0.625 mm (GE) and 2 mm (Toshiba) at a peak power of 120 kV and a tube current of 300 milliamperes (Kwan JY, et al., 2018). The detailed descriptions of CT imaging and scanners of OPC-Radiomics cohort and MAASTRO cohort are provided in TCIA (Clark K, et al., 2013).

Image preprocessing

When evaluating texture features related to intensity and spatial information in radiomic research, The differences in voxel size and intensity are of great crucial due to image acquisition of CT images on different scanners. To reduce the impact of different slice thicknesses and voxel sizes on radiomics features, 1 × 1 × 1 mm voxels resampling was performed on Digital Imaging and Communication in Medicine (DICOM) images in CT scans. Five derivates generated by each image mode were mainly obtained by edge-enhanced Laplacian Gaussian (LoG) filtering with sigma setting 1, 2, 3, 4, and 5 mm for CT scanning. A fixed bin width of 25 units was applied to the analysis, which gave better repeatability than a fixed bin number (Avanzo M, et al., 2020). Data coordination was achieved through the Image Biomarkers Standardization Initiative (IBSI), which focuses primarily on achieving high-throughput quantitative image analysis (radiomics), particularly for standardizing the extraction of image biomarkers from acquired images (Bagher‐Ebadian H, et al., 2020).

Texture feature extraction

Radiomics, a Python package derived from the open-source package Pyradiomics, was used to extract texture features in the 3D slicer. From the CT images of each patient, 1316 radiomics features were extracted, which including 14 shape features,18 first-order histogram features, 24 GrayLevel Co-Occurrence Matrix (GLCM) features, 14 Gray-Level Dependence Matrix(GLDM) features, 16 Gray-Level Size-Zone Matrix (GLSZM) features, 16 Gray-Level Run-Length Matrix(GLRLM) features, 744 wavelet features, 5 Neighboring Gray-Tone Difference Matrix(NGTDM) features, and 465 Laplacian of Gaussian (LoG sigma=1,2,3,4,5) features. Detailed descriptions of radiomic features are provided elsewhere (http://www. radiomics.io/pyradiomics.html).

Feature selection

In order to mitigate selection bias or overfitting of our radiomics model, a wrapper-based algorithm was applied to feature selection in the training cohort (Zhang Y, et al., 2021). Logistic Regression combined with Recursive Feature Elimination (RFE) algorithms was used to remove redundant and irrelevant features. Last, 20 features were kept for subsequent radiomic analysis. All radiomics features were scaled to z-score conversion before feature selection was applied.

Construction of Human Papillomavirus (HPV) classification models

Ten kinds of machine-learning classification algorithms were compared with optimized hyperparameters, in order to develop an optimized classification model in the training cohort. The final model was then applied to the independent validation and testing cohorts respectively. The ten machine learning classifiers fall into three different families: Linear, nonlinear, and ensemble. The linear classifiers were Logistic Regression (LR), Naive Bayes (NB), and Stochastic Gradient Descent (SGD). Nonlinear classifiers include the Support Vector Machine (SVM), Decision Tree (DT), and K-Nearest Neighbors (KNN). The rest are ensemble models including Extra Trees (ET), Random Forest (RF), Extreme Gradient Boosting (XGB), and Light Gradient Boosting Machine (LGBM). The machine learning classification algorithm is shown in Figure 2, and hyperparameter descriptions are presented.

Figure 2: The workflow of this study

Statistical analysis

Evaluation of prediction performance was done using the Area Under the Receiver Operating Characteristic (ROC) Curve (AUC). A predicted probability of the validation and testing cohorts was used to calculate diagnostic performance (AUC, Accuracy (ACC)). The p values were calculated by comparing paired AUCs with DeLong’s test. Mann-Whitney U test was utilized for comparisons between continuous variables, and the chi-squared test was performed for categorical variables between groups. Statistical tests were conducted bilaterally, and P values less than 0.05 were considered statistically significant.

Results

Characteristics of patients

A total of 651 patients with OPSCC met our inclusion criteria-497 from OPC-Radiomics cohort, 74 from MAASTRO cohort and 80 from SNPH cohort. OPC-Radiomics cohort was randomized into training cohort (261 men, 71 women; median age, 61.1 years) and validation cohort (131 men, 34 women; median age, 59.7 years) with a ratio of 2:1. MAASTRO cohort (60 men, 14 women; median age, 60.4 years) and SNPH cohort (66 men, 14 women; median age, 60.5 years) were used as independent external testing cohorts. No significant difference was found in age and gender among the four cohorts. As most patients were diagnosed before 2017 in this study, we used the 7^th edition of UICC TNM classification for research and analysis of all included patients. The institutional protocol was used for OPC-Radiomics and MAASTRO cohorts, and also provided the overall HPV annotation in TCIA. Table 1 summarizes the baseline characteristics of patients in training cohort, validation cohort, testing cohort 1 and testing cohort 2.

Table 1: Baseline characteristics of the included patients

Performance of different models

In the training cohort, by comparing of the ten machine-learning classifiers, the top five models with the largest AUC and ACC are selected as the final models to predict the HPV status of OPSCC and are listed in Table 2. The top five AUCs of LR, SVM, RF, XGB, and LGBM classifier were 0.86, 0.92, 0.97, 1.0 and 1.0, respectively. The ACCs of the top 5 models above are 0.8, 0.82, 0.88, 1.0 and 1.0, respectively.

Table 2: Performance of the top 5 machine-learning classifiers

In the independent validation cohort, the performance of RF (AUC=0.72) was the best among the five models, followed by LR (AUC=0.68), SVM (AUC=0.66), LGBM (AUC=0.65), and XGB (AUC=0.65). In the valida tion cohort, significant differences were found in terms of AUC between RF and the other classifiers (P<0.05, DeLong’s test) except for LR. In the external testing cohort 1, no significant difference was found between the AUC of the RF and other models (P>0.05, DeLong’s test), with the AUCs of 0.54, 0.64, 0.63, 0.60, and 0.61 for LR, SVM, RF, XGB, and LGBM, re spectively. In the external testing cohort 2, we found that the RF-based model yielded the best classification performance, reaching an AUC of 0.78, and details are listed in Table 3 and Figure 3.

Table 3: Discrimination performance comparison of RF and other four classifiers

Figure 3: The ROC curve of different models. RF: Random Forest; SVM: Support Vector Machine; XGB: Extreme Gradient Boosting; DT: Decision Tree; LGBM: Light Gradient Boosting Machine; ET: Extra Trees; SGD: Stochastic Gradient Descent; KNN: K-Nearest Neighbors; NB: Naïve Bayes; ROC: Receiver Operating Characteristic Curve. Note: Training cohort, Validation cohort, Testing cohort 1, Testing cohort 2, Reference

Overall, the RF-based model was significantly superior or not significantly inferior to other machine-learning classifiers in the four cohorts. Thus, the RF-based model was selected as the final model for predicting the HPV status of OPSCC with the AUCs of 0.97, 0.72, 0.63, and 0.78 in the training cohort, validation cohort, testing cohort 1, and testing cohort 2, respectively.

Discussion

Since radiomics is an innovative quantitative imaging analysis technique that utilizes quantitative metrics through high-throughput extraction, numerous researches have investigated its predictive value of HPV status for OPSCC. In a study by Bolin et al, radiomic biomarkers on CT images were applied to predict HPV status of OPSCC patients using a linear discriminant analysis machine-learning classifier, performing well in both training (AUC=0.84) and validation cohorts (AUC=0.70) (Song B, et al., 2021). Sohn B, et al., 2021, reported that MRI phenotyping, based on radiomics, can also discriminate between HPV positive and negative OPSCC, and may serve as a potential imaging biomarker (Sohn B, et al., 2021). Reza found that Random Forest based model could successfully predict HPV status, even using different scanner manufacturers (AUC=0.74-0.79) (Reiazi R, et al., 2021). Previous studies have shown that radiomics has the potential in predicting HPV types of OPSCC.

In the present study, three cohorts of different countries and medical institutions were used to construct the classification model based on ten machine-learning algorithms. Random Forest classifier exhibited favorable classification performances with AUC values of 0.97 for the training cohort and 0.63-0.78 for the validation and testing cohorts. These results might suggest that the information from medical images may have value as surrogates or signatures for HPV status in clinical practice (Ren J, et al., 2020; Mu W, et al., 2021; Haider SP, et al., 2020).

While radiomics is growing in clinical interest, the reproducibility and validity of previously published studies on radiomics remain problematic (Gillies RJ, et al., 2016; Bodalal Z, et al., 2019; Welch ML, et al., 2019). Radiomics or machine-learning algorithm has the inherent vulnerability of being susceptible to overfitting, so extensive validation is required to prove the algorithms’ reliability (Zwanenburg A, et al., 2020; O’Connor JP, et al., 2017; Feng XL, et al., 2022). Our study was designed to minimize the potential for overfitting using completely separate validation and testing cohorts. Additionally, previous studies have been conducted with proprietary software that cannot be obtained by the public; this also leads to the difficulty in unifying the methods in the area of radiomics (Welch ML, et al., 2019). In this study, we focused on this problem by using free and opensource software to extract radiomic features from image data. Hence, these open-source software systems may facilitate transparency and decrease methodology heterogeneity, in particular in the field of radiomics.

Our study compared ten machine learning algorithms based on CT radiomics analysis, and the RF model was found to be the most accurate prediction of HPV status for OPSCC. As shown in the training cohort, most of the machine learning classifiers could accurately predict HPV status, while the RF radiomic model performed better in the validation cohort with an AUC of 0.72 and in the testing cohort 2 with an AUC of 0.78. It demonstrated that models, which performed well based on the training cohort, may not necessarily work well in the validation or testing cohort (Feng XL, et al., 2022). Random Forest, the classifier used in this study, is robust against noises and outliers, and is capable of dealing quickly with high-dimensional data (Capitaine L, et al., 2021). A number of classification problems involving medical imaging-based texture analysis were solved using RF (Goyal R, et al., 2022; Chen AY, et al., 2022; Wang J, et al., 2022). Accordingly, Random Forest was recommended to be employed in CT texture analysis for the prediction of HPV status in OPSCC (Ren J, et al., 2020; Dai H, et al., 2020; Haider SP, et al., 2021).

This study had a few limitations. First, this study was retrospective, and the models have to be validated in prospective studies. Second, Most radiomics studies have had trouble interpreting radiomics, particularly when it comes to its pathophysiological significance. Radiomic features are not computed from a pathophysiological perspective, but rather from a mathematical context which quantifies the heterogeneities and shapes of pixels. Therefore, we only speculate that radiomic features provide additional information concerning tumor heterogeneity and shape, and this information can be obtained from measuring tumor image characteristics. In addition, there may be other biological or clinical elements specific to the development of HPV ± head and neck cancers, which are not covered in this study. It might not always be possible to obtain reliable demographic information from all patients, and some may be prone to cultural or geographic differences, which may restrict the use of the models; to this end, we limited ourselves to utilizing only radiomics features extracted from clinical CT scans in order to develop an objective biomarker.

Conclusion

By comparing 10 kinds of machine-learning classifiers, we found that the best performance was achieved when using a Random Forest-based model, with the Area Under the Receiver Operating Characteristic (ROC) Curves (AUCs) of 0.97, 0.72, 0.63, and 0.78 in the training cohort, validation cohort, testing cohort 1 (MAASTRO cohort), and testing cohort 2 (SNPH cohort), respectively. The Random Forest-based radiomics model was effective in differentiating Human Papillomavirus status of Oropharyngeal Squamous Cell Carcinoma in multi-national population, which provides the possibility for this non-invasive method to be widely applied in clinical practice.

Declarations

Acknowledgments

We wish to thank the timely help given by Jiang Wei and Wu Kailong in analyzing the large number of samples. Besides, the construction funding for Oncology and Radiotherapy Center of Fengxian District of Shanghai is gratefully acknowledged.

Authors’ contributions

ZW and XY contributed to the conception and design of the study; LZ and LX analyzed and interpreted the clinical data and imaging findings, and they were major contributors in writing the manuscript. YY and DY helped perform the analysis with constructive discussions. All authors read and approved the final manuscript.

Funding

This study was supported by the construction funding for the Oncology and Radiotherapy Center of the Fengxian District of Shanghai.

Availability of data and materials

All data generated or analyzed during this study are included in this published article.

Ethics approval and consent to participate

Our study complied with the Declaration of Helsinki. The study was approved by the medical ethics committee of the Fengcheng hospital of Fengxian district. (NO.2021-051: The registration number of ethics board) and written informed consent from the patients for use of data and the accompanying images was waived by the medical ethics committee of the Fengcheng hospital of Fengxian district due to retrospective nature of the study.

References

1. Lechner M, Liu J, Masterson L, Fenton TR. HPV-associated oropharyngeal cancer: Epidemiology, molecular biology and clinical management. Nat Rev Clin Oncol. 2022; 19(5): 306-327.

   [Crossref] [Google Scholar] [Pubmed]

2. Ang KK, Harris J, Wheeler R, Weber R, Rosenthal DI, Nguyen-Tân PF, et al. Human papillomavirus and survival of patients with oropharyngeal cancer. N Engl J Med. 2010; 363(1): 24-35.

   [Crossref] [Google Scholar] [Pubmed]

3. Freihat O, Tóth Z, Pintér T, Kedves A, Sipos D, Cselik Z, et al. Pre-treatment PET/MRI based FDG and DWI imaging parameters for predicting HPV status and tumor response to chemoradiotherapy in primary Oropharyngeal Squamous Cell Carcinoma (OPSCC). Oral Oncol. 2021; 116: 105239.

   [Crossref] [Google Scholar] [Pubmed]

4. Bagher-Ebadian H, Zhu S, Siddiqui F, Lu M, Movsas B, Chetty IJ. On the development of an outcome-driven frequency filter for improving radiomics-based modeling of Human Papillomavirus (HPV) in patients with oropharyngeal squamous cell carcinoma. Med Phys. 2021; 48(11): 7552-7562.

   [Crossref] [Google Scholar] [Pubmed]

5. Fomin I, Hughes I, Bhuta S. Oropharyngeal squamous cell carcinomas: Can imaging findings predict HPV status? J Med Imaging Radiat Oncol. 2021; 65(2): 175-181.

   [Crossref] [Google Scholar] [Pubmed]

6. Avanzo M, Wei L, Stancanello J, Vallieres M, Rao A, Morin O, et al. Machine and deep learning methods for radiomics. Med Phys. 2020; 47(5): 185-202.

   [Crossref] [Google Scholar] [Pubmed]

7. Kwan JY, Su J, Huang SH, Ghoraie LS, Xu W, Chan B, et al. Radiomic biomarkers to refine risk models for distant metastasis in HPV-related oropharyngeal carcinoma. Int J Radiat Oncol Biol Phys. 2018; 102(4): 1107-1116.

   [Crossref] [Google Scholar] [Pubmed]

8. Clark K, Vendt B, Smith K, Freymann J, Kirby J, Koppel P, et al. The Cancer Imaging Archive (TCIA): Maintaining and operating a public information repository. J Digit Imaging. 2013; 26(6): 1045-1057.

   [Crossref] [Google Scholar] [Pubmed]

9. Aerts HJ, Velazquez ER, Leijenaar RT, Parmar C, Grossmann P, Carvalho S, et al. Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach. Nat Commun. 2014; 5(1): 1-9.

   [Crossref] [Google Scholar] [Pubmed]

10. Fedorov A, Beichel R, Kalpathy-Cramer J, Finet J, Fillion-Robin JC, Pujol S, et al. 3D Slicer as an image computing platform for the Quantitative Imaging Network. Magn Reson Imaging. 2012; 30(9): 1323-1341.

    [Crossref] [Google Scholar] [Pubmed]

11. Bagher-Ebadian H, Lu M, Siddiqui F, Ghanem AI, Wen N, Wu Q, et al. Application of radiomics for the prediction of HPV status for patients with head and neck cancers. Med Phys. 2020; 47(2): 563-575.

    [Crossref] [Google Scholar] [Pubmed]

12. Zhang Y, Wei X, Cao C, Yu F, Li W, Zhao G, et al. Identifying discriminative features for diagnosis of Kashin-Beck disease among adolescents. BMC Musculoskelet Disord. 2021; 22(1): 1-10.

    [Crossref] [Google Scholar] [Pubmed]

13. Song B, Yang K, Garneau J, Lu C, Li L, Lee J, et al. Radiomic features associated with HPV status on pretreatment computed tomography in oropharyngeal squamous cell carcinoma inform clinical prognosis. Front Oncol. 2021; 11: 744250.

    [Crossref] [Google Scholar] [Pubmed]

14. Sohn B, Choi YS, Ahn SS, Kim H, Han K, Lee SK, et al. Machine learning based radiomic HPV phenotyping of oropharyngeal SCC: A feasibility study using MRI. The Laryngoscope. 2021; 131(3): E851-E856.

    [Crossref] [Google Scholar] [Pubmed]

15. Reiazi R, Arrowsmith C, Welch M, Abbas-Aghababazadeh F, Eeles C, Tadic T, et al. Prediction of Human Papillomavirus (HPV) Association of Oropharyngeal Cancer (OPC) using radiomics: The impact of the variation of CT scanner. Cancers. 2021; 13(9): 2269.

    [Crossref] [Google Scholar] [Pubmed]

16. Ren J, Yuan Y, Qi M, Tao X. Machine learning-based CT texture analysis to predict HPV status in oropharyngeal squamous cell carcinoma: Comparison of 2D and 3D segmentation. Eur Radiol. 2020; 30(12): 6858-6866.

    [Crossref] [Google Scholar] [Pubmed]

17. Mu W, Jiang L, Shi Y, Tunali I, Gray JE, Katsoulakis E, et al. Non-invasive measurement of PD-L1 status and prediction of immunotherapy response using deep learning of PET/CT images. J Immunother Cancer. 2021; 9(6).

    [Crossref] [Google Scholar] [Pubmed]

18. Haider SP, Zeevi T, Baumeister P, Reichel C, Sharaf K, Forghani R, et al. Potential added value of PET/CT radiomics for survival prognostication beyond AJCC 8th edition staging in oropharyngeal squamous cell carcinoma. Cancers. 2020; 12(7): 1778.

    [Crossref] [Google Scholar] [Pubmed]

19. Gillies RJ, Kinahan PE, Hricak H. Radiomics: Images are more than pictures, they are data. Radiology. 2016; 278(2): 563.

    [Crossref] [Google Scholar] [Pubmed]

20. Bodalal Z, Trebeschi S, Nguyen-Kim TD, Schats W, Beets-Tan R. Radiogenomics: Bridging imaging and genomics. Abdom Radiol. 2019; 44(6): 1960-84.

    [Crossref] [Google Scholar] [Pubmed]

21. Welch ML, McIntosh C, Haibe-Kains B, Milosevic MF, Wee L, Dekker A, et al. Vulnerabilities of radiomic signature development: The need for safeguards. Radiother Oncol. 2019; 130: 2-9.

    [Crossref] [Google Scholar] [Pubmed]

22. Zwanenburg A, Vallières M, Abdalah MA, Aerts HJ, Andrearczyk V, Apte A, et al. The image biomarker standardization initiative: Standardized quantitative radiomics for high-throughput image-based phenotyping. Radiology. 2020; 295(2): 328-338.

    [Crossref] [Google Scholar] [Pubmed]

23. O'Connor JP, Aboagye EO, Adams JE, Aerts HJ, Barrington SF, Beer AJ, et al. Imaging biomarker roadmap for cancer studies. Nat Rev Clin Oncol. 2017; 14(3): 169-186.

    [Crossref] [Google Scholar] [Pubmed]

24. Feng XL, Wang SZ, Chen HH, Huang YX, Xin YK, Zhang T, et al. Optimizing the radiomics-machine-learning model based on non-contrast enhanced CT for the simplified risk categorization of thymic epithelial tumors: A large cohort retrospective study. Lung Cancer. 2022; 166: 150-160.

    [Crossref] [Google Scholar] [Pubmed]

25. Capitaine L, Genuer R, Thiébaut R. Random forests for high-dimensional longitudinal data. Stat Methods Med Res. 2021; 30(1): 166-184.

    [Crossref] [Google Scholar] [Pubmed]

26. Goyal R, Mui LW, Riyahi S, Prince MR, Lee HK. Machine learning based prediction model for closed-loop small bowel obstruction using computed tomography and clinical findings. J Comput Assist Tomogr. 2022; 46(2): 169-174.

    [Crossref] [Google Scholar] [Pubmed]

27. Chen AY, Lee J, Damjanovic A, Brooks BR. Protein pKa prediction by tree-based machine learning. J Chem Theory Comput. 2022; 18(4): 2673-2686.

    [Crossref] [Google Scholar] [Pubmed]

28. Wang J, Ren J, Liu J, Zhang L, Yuan Q, Dong B. Identification and verification of the ferroptosis-and pyroptosis-associated prognostic signature for low-grade glioma. Bosn J Basic Med Sci. 2022.

    [Crossref] [Google Scholar] [Pubmed]

29. Dai H, Huang Y, Xiao G, Lan B, Jiang G, Tian J. Predictive features of thymic carcinoma and high-risk thymomas using random forest analysis. J Comput Assist Tomogr. 2020; 44(6): 857-864.

    [Crossref] [Google Scholar] [Pubmed]

30. Haider SP, Sharaf K, Zeevi T, Baumeister P, Reichel C, Forghani R, et al. Prediction of post-radiotherapy locoregional progression in HPV-associated oropharyngeal squamous cell carcinoma using machine-learning analysis of baseline PET/CT radiomics. Transl Oncol. 2021; 14(1): 100906.

    [Crossref] [Google Scholar] [Pubmed]