Effects of Interobserver Segmentation Variability and Intensity Discretization on MRI-Based Radiomic Feature Reproducibility of Lipoma and Atypical Lipomatous Tumor

Atypical lipomatous tumor (ALT) and lipoma are the most common soft-tissue lesions [1]. According to the 2020 edition of the World Health Organization classification [2], the term ALT is reserved for low-grade adipocytic neoplasms arising at anatomical sites for which surgery is generally curative, including the extremities and trunk [2]. ALTs have a relatively indolent disease course compared to well-differentiated liposarcomas, namely lipomatous lesions with the same histology but located in deep anatomical sites such as the retroperitoneum, mediastinum, and spermatic cord, where there is a higher risk for recurrence and dedifferentiation related to lower chances of achieving negative surgical margins [2]. In line with this relatively indolent clinical behavior, treatment strategy has progressively shifted from extensive surgery to marginal excision in ALTs, which is now considered an appropriate option to achieve local control while taking into account the morbidity rates associated with surgery [3]. On the other hand, lipomas are benign lipomatous lesions, which do not require any treatment unless symptomatic or due to cosmetic concerns [3]. Lipomas are rare in deep locations, such as the retroperitoneum, but very common in the extremities and trunk [1]. Thus, an accurate distinction between ALT and lipoma is desirable to offer optimal patient care.

In the diagnostic pathway of lipomatous soft-tissue lesions, magnetic resonance imaging (MRI) is the imaging method of choice for diagnosis and differentiating ALT from lipoma, with high sensitivity and substantial specificity [4‐6]. In detail, according to a recent meta-analysis, the sensitivity and specificity of radiologists evaluating multiple combined imaging parameters (called “radiologist gestalt”) range from 76 to 100% and 37 to 77%, respectively, if only studies focusing on lipoma and ALT are considered [4]. Nonetheless, a certain degree of interobserver variability has emerged even among expert readers [5‐7], with kappa values ranging from 0.23 to 0.7 according to this meta-analysis [4]. Preliminary imaging studies applying radiomics have shown promise for improving diagnostic accuracy and characterizing lipomatous soft-tissue lesions more objectively [8]. Radiomics includes the extraction and analysis of quantitative parameters from medical images, known as radiomic features [9‐11]. A crucial step of radiomic workflows is feature reproducibility assessment, as these quantitative parameters may suffer from interobserver variability, particularly regarding tumor delineation while performing manual segmentation [12‐15]. Segmentation margins are also critical because the peritumoral area may influence the reproducibility of radiomic features and their diagnostic performance [15, 16]. Furthermore, in radiomic workflows, the effects of different image intensity discretization methods on feature reproducibility are debated [17‐19]. In literature, the intraclass correlation coefficient (ICC) is commonly employed to evaluate radiomic feature reproducibility [16, 20‐23].

The aim of this study is to investigate the influence of interobserver manual segmentation variability on the reproducibility of MRI-based radiomic features in lipoma and ALT, also considering the impact of different image intensity discretization methods.

The main finding of our study is that the rates of stable radiomic features extracted from T1-weighted and T2-weighted MRI sequences were very high (90% or higher) regardless of the discretization method and segmentation approach. The discretization method with fixed bin width yielded higher stable feature rates and higher feature ICC values compared to fixed bin number, regardless of the segmentation approach with or without marginal erosion (p < 0.001). Additionally, no difference in stable feature rates was found between the segmentation approaches, regardless of the discretization method (p ≥ 0.175). Overall, a small but still not negligible degree of segmentation variability highlighted the need to include a reliability analysis in radiomic studies.

Radiomics has a great potential as a non-invasive biomarker to quantify several tumor characteristics, both standalone and combined with artificial intelligence methods such as machine learning [31‐33]. However, it faces challenges to clinical implementation [34]. A great variability in radiomic features has emerged as a major issue across studies, and image segmentation is the most critical step [11]. As segmentation is time-consuming if performed manually, prior to conducting radiomic studies, methodological analyses would be desirable to preliminarily evaluate the robustness of different segmentation approaches and avoid biases due to non-reproducible, noisy features. Similar analyses were previously performed in kidney [16], lung and head and neck [14], and cartilaginous bone [15] lesions. Regarding lipomatous soft-tissue tumors, most radiomic studies included a feature reproducibility assessment as a dimensionality-reduction method in their radiomic workflow, which was built with the aim of differentiating benign from malignant (including low-grade) lesions [35‐42]. More recently, Sudjai et al. compared the effects of intra- and interobserver segmentation variability on the reproducibility of 2D and 3D MRI-based radiomic feature reproducibility in lipoma and ALT [43]. A region growing-based semiautomatic contour-focused segmentation was performed on T1-weighted sequences by two readers and only original images were used for feature extraction, resulting in 43 out of 93 (46.2%) 2D features and 76 out of 107 (71%) 3D features with an absolute agreement ICC ≥ 0.75, which defined feature stability [43]. Based on their findings, we focused our study on 3D segmentations only, as they yielded higher stable feature rates. We compared two image intensity discretization methods (fixed bin number vs. fixed bin width) and two segmentation approaches (contour-focused vs. margin shrinkage) on both T1-weighted and T2-weighted sequences, involving three different readers as suggested by the ICC guidelines by Koo and Li [24]. After extraction of features from original, filtered and transformed images (1106 features per sequence compared to 107 in the previous study [43]), we found higher rates of stable features (90% or higher per sequence, regardless of the discretization method and segmentation approach) using ICC 95% CI lower bound ≥ 0.75 as a stricter cutoff to define feature stability. This difference could be attributed to the use of filtered and transformed (in addition to the original) images for feature extraction in our study, as well as to the different experiences of the readers involved in image segmentation, namely a statistician and a research scientist in the previous study [43] and three physicians in our study. Despite these differences, a common conclusion that can be drawn from the previous [43] and our studies is that most 3D MRI radiomic features of lipoma and ALT have good reproducibility, although a certain degree of segmentation variability exists.

In our study, T1-weighted and T2-weighted MRI sequences demonstrated good reproducibility regardless of the image intensity discretization method employed in image pre-processing, which was performed using both options of fixed bin number and fixed bin width, with stable feature rates respectively ranging from 90.69 to 95.66% and from 95.39 to 97.65%. The discretization method with fixed bin width resulted in higher stable feature rates and higher feature ICC values, thus providing more robust features compared to discretization with fixed bin number in our series. This finding is in line with previous positron emission tomography and MRI studies showing better feature reproducibility when implementing fixed bin width [44, 45]. Margin shrinkage led to an improvement in terms of feature ICC values compared to contour-focused segmentation only when implementing discretization with fixed bin width on T2-weighted images. Conversely, no difference in terms of feature ICC values was found between the two segmentation approaches when implementing discretization with fixed bin width on T1-weighted images or fixed bin number regardless of the employed MRI sequence. Additionally, no difference in terms of stable feature rates was found between the two segmentation approaches, regardless of the discretization method. Thus, a definite conclusion regarding the superiority of one segmentation approach over the other cannot be drawn. This confirms the need for a preliminary assessment of feature reproducibility in radiomic workflows and is in line with literature emphasizing the importance of reproducibility in artificial intelligence and radiology [46‐48].

Some limitations of our study should be addressed. First, it has a retrospective design, as a prospective analysis is not strictly necessary for radiomic studies [49]. Second, the retrospective design accounts for the exclusion of contrast-enhanced MRI, which was not performed consistently in our series of lipomas and ALTs. This is in line with recent studies suggesting that the value of contrast administration may be limited in lipoma and ALT [6, 50], with no clear improvement in diagnostic accuracy following the addition of contrast-enhanced sequences to a non-contrast MRI protocol [50]. Finally, due to its scope, this was a single institution study, and the generalizability of our results should be confirmed on more varied datasets.

Radiomic features of lipoma and ALT extracted from T1-weighted and T2-weighted MRI sequences are reproducible regardless of the segmentation approach and segmentation method, although a minimal degree of segmentation variability exists and highlights the need to perform a preliminary reproducibility analysis in radiomic studies. As stable feature rates were similar between contour-focused and margin shrinkage segmentations, it could be reasonable to prefer the former approach for ease of use in clinical practice. Image intensity discretization with fixed bin width provided higher stable feature rates and feature ICC values compared to discretization with fixed bin number. Thus, the former discretization method might be favored when performing image pre-processing in future radiomic studies dealing with lipomatous soft-tissue tumors.