In research settings, histological analysis is a commonly used technique for evaluation of cartilage health, determining severity of osteoarthritis (OA), and evaluating efficacy of therapeutic approaches. It is common practice with this approach to evaluate cartilage cellularity and chondrocyte organization. Chondrocyte density estimated from histologic sections has been accepted as a measure of articular cartilage health. Previous work indicates that optimal chondrocyte density is necessary to maintain cartilage tissue, while declining chondrocyte density is associated with degeneration of articular cartilage in OA and with aging.¹ In contrast, chondrocyte cloning - formation of non-linear clusters of chondrocytes - is considered one of the markers of osteoarthritis,¹ although the significance and events responsible for cloning are not well understood. Semi-quantitative, subjective, categorical evaluations of cartilage cellularity, such as those performed during application of Mankin¹ and OARSI² scoring, have limitations including uncertain accuracy and reproducibility.³ For these reasons, the ability to rapidly quantitate chondrocyte density and cloning could help advance understanding of these phenomena in the pathophysiology of osteoarthritis and in articular cartilage regeneration, repair, and aging.

One previously reported approach uses image analysis algorithms to automatically identify chondrocyte density and red intensity values of pixels within the segmented cartilage and assigns cellularity and PG depletion Mankin sub-scores based on deviation of those quantitative values from pre-defined normative values.^{4, 5} While that program achieved good agreement with human experts for the Mankin structural sub-score (linear regression; R²: 0.87, slope: 0.84) and the PG depletion sub-score (R²: 0.63, slope: 0.70), there was poor agreement with human experts on the cellularity sub-score (R²: 0.07, slope: 0.21).⁴ This poor agreement may result from the simple thresholding segmentation and edge detection used to identify chondrocytes,⁴ two image analysis techniques which are far from being robust to the huge variation in tissue appearance during cartilage degeneration. Erroneous chondrocyte density calculations could artificially elevate the assigned cellularity sub-score by classifying it as either hypercellular or hypocellular.¹ Furthermore, difficulty in accurately segmenting chondrocytes substantially reduces the ability to automatically identify chondrocyte cloning. To accurately quantify cartilage cellularity and/or accurately assign cellularity scores, more sophisticated image analysis algorithms that can accommodate the wide variety of chondrocyte appearances and organization in degenerating cartilage are required.

Conveniently, modern deep-learning-based approaches have been reported to have achieved great success in a variety of computer vision tasks.^6–10 A convolutional neural network (CNN) is one common deep learning method in which the model is trained to analyze a given image type by using pairs of similar input images and the corresponding correct output data (ground truth) – a technique called supervised learning. With this technique, a CNN model is able to learn inter-related hierarchical features from the image, which allows it to accurately classify, detect, and segment objects with variable appearances.^{8, 9} CNN-based approaches have previously been applied very successfully in orthopedic research to perform tasks such as grading/classifying radiographic knee images^{11, 12} and segmenting bone and soft tissues from MR images.^13–15 CNN-based approaches have also achieved tremendous success in a variety of digital pathology applications. For example, Bejnordi et al have assessed different CNN models for detecting lymph node metastases in women with breast cancer, and achieved a model that outperforms expert pathologists.¹⁶ Similarly, Nagpal et al developed and validated a CNN model to stage prostate cancer, which resulted in higher accuracy than a group of pathologists.¹⁷ There are also CNN-based studies ranging from detecting sub-cellular (nuclei) components^{18, 19} to segmentations of glandular structures^{20, 21} in histology images.

However, applications to cartilage analysis have been more limited. Recent work by Powel et al ²² describes using a CNN-based image classifier to automatically assign a Bern score to evaluate chondrogenicity of engineered cartilage,²³ and Rytky et al has developed a CNN model to automatically segment calcified cartilage to study its modification during OA progression.²⁴ Based on this previous success, our goal in this work was to develop and implement methods to accurately map chondrocyte density and chondrocyte cloning across the entire articular cartilage surface using a combination of CNN models and spatial information. We aimed to adapt our previously trained U-net model²⁵ and a “you only look once” (YOLO)⁸ model to segment individual chondrocytes and identify chondrocyte clones in cartilage. We hypothesized that fully automated chondrocyte density and clone detection achieved with CNN models would have excellent agreement with gold-standard human expert cell/clone identification. Further, we hypothesized that application of the trained models would be able to identify subtle progressive changes in chondrocyte cellularity and organization associated with development of post-traumatic osteoarthritis after a joint injury.

The optimal U-net was selected at the 82nd epoch with a median F1 score of 0.89 (95% CI: 0.88 to 0.93) on the training validation images, and the optimal YOLO was selected at the 105^th epoch with a median F1 score of 0.80 (95% CI: 0.67 to 1.00) on the training validation images. Those F1 score values indicated good detection accuracy (good = 0.70–0.90)²¹ by each model. In the secondary validation, the U-net achieved an excellent agreement with the average counts by the experts (Figure 4; R² = 0.93, y = 0.90× – 0.69) and a close-to-excellent median F1 score (median: 0.87; 95% CI: 0.84 to 0.89). The average IOU between the U-net segmentation and manual segmentation of each chondrocyte was 0.849, indicating high accuracy of U-net chondrocyte segmentations.³⁴ The three expert researchers’ agreement identifying clones was good (ICC = 0.76).³⁵ Despite relatively a lower median F1 score (median: 0.67; 95% CI: 0.57 to 0.89) for clone detection, the YOLO still achieved good agreement with the average counts by the experts (Figure 4; R² = 0.79, y = 0.95×). Both models were found to be able to identify chondrocytes/clones of variable size, shape, and appearance (Figure 4).

Differences in cellularity were found between the normal and the different post-operative timepoints (Table 1). Average chondrocyte density one week after fracture (mean: 856 cells/mm²; 95% CI: 748–963 cells/mm²) was slightly higher than normal chondrocyte density (mean: 774 cells/mm²; 95% CI: 742–806 cells/mm²; p=0.573) and significantly higher than at 3 months (mean: 610 cells/mm²; 95% CI: 491–729 cells/mm²; p=0.006), 6 months (mean: 638 cells/mm²; 95% CI: 470–806 cells/mm²; p=0.016), and 12 months (mean: 683 cells/mm²; 95% CI: 556–810 cells/mm²; p=0.077) after fracture (Figure 5-A). Clone density and percentage of chondrocytes in clones 3, 6, and 12 months after fracture was higher than in normal cartilage or at one week postoperatively, however these differences did not reach statistical significance (Figure 5-B, C; p>0.1 for all pair-wise comparisons).

Local cellularity differences were noted among the different post-injury timepoints. At the anterior positions, chondrocyte densities were lower 3, 6, and 12 months after fracture than after one week or in normal tissue. Cell density was the lowest 3 months after fracture and was significantly lower than densities in normal tissue or one week after fracture at the 10% (p=0.032/0.012), 20% (p=0.107/0.024), and 30% (p=0.006/0.039) positions (Figure 6-A). One week after fracture, chondrocyte densities in the posterior locations were higher than in normal tissue and significantly higher than at 3, 6 and 12 months: 60% (p=0.137, 0.049, 0.649 for 3, 6, 12 months, respectively), 70% (p=0.164, 0.096, 0.556), 80% (p=0.108, 0.010, 0.428), 90% (p=0.023, <0.0001, 0.046), and 100% (p=0.002, < 0.0001, 0.067). Chondrocyte densities 3, 6, and 12 months after fracture were also lower than the normal group at the posterior positions, although these differences did not reach statistical significance.

Locally, significantly higher clone density/percentage of chondrocytes in clones was found 3, 6, and 12 months after fracture (Figure 6-B, C; Table 2). Local clone density at 3 months was significantly higher than the normal group at the 70% position (p=0.013) which is moving into weightbearing tissue that was not directly affected by the fracture. By 6 and 12 months after fracture, significantly higher clone densities than in the normal and/or one-week groups were found more peripherally in the joint (the 20% and 80% positions). Clone density at 6 months was significantly higher than in normal or one week at the 20% position (p=0.026, 0.021) and at 12 months clone density was higher than in the normal (p=0.057) and one-week (p=0.047) groups at the 80% position. Variations in the percentage of chondrocytes in a clone were very similar to the local variations of clone density (Figure 6-C).

Histological analysis remains among the most commonly used research techniques to evaluate the health of cartilage. In this work, we trained, validated, and implemented two different CNN models to quantify the progressive chondrocyte cellularity changes after joint/cartilage injury. The trained U-net and YOLO models developed for this purpose were validated to be accurate for identifying chondrocytes and clones by achieving good agreement with manual assessment by expert researchers. These algorithms proved to be extensible for use identifying chondrocytes and chondrocyte clones in degenerating minipig cartilage, and they were able to identify progressive cellularity changes with time after joint injury.

While there are several CNN models that could be suitable for chondrocyte identification,^{7, 10} the U-net was selected for chondrocyte detection because it has demonstrated state-of-art performance on a similar cell detection task,⁹ and it provides boundary information from chondrocyte segmentation that can be useful for future studies related to co-localization of functional cellular stains and specific morphological structures. In contrast, as relatively little is known about the importance of the morphology of clones, a simple bounding-box-based detection method was used for clone detection in this work. The YOLO was selected over other bounding-box-based methods for its developmental and implementation simplicity. Specifically, it is a single CNN model that can be developed and trained end-to-end to predict bounding box probability and coordinates simultaneously.⁸ Similar methods have two stages, requiring independent development of a bounding box generator and an image classifier,^{7, 36} which can be more laborious and slower. If clone morphology, rather than simple detection, becomes more important in future work, adapting the U-net trained to segment chondrocytes using transfer learning would likely be successful for that purpose.

The U-net model reported here is a retrained version of our previously reported model which had been selected based on providing the lowest average loss value during training.²⁵ However, that model, chosen to minimize weighted loss at each pixel, did not correspond to the highest detection accuracy at the cell level. Therefore, in addition to modifications made to emphasize detection of small, closely packed chondrocytes, the retrained model was selected based on providing the highest F1 score, which reflected the best cell detection accuracy as compared with the gold-standard manual segmentations. To maximize functional accuracy, a similar process was followed during the training of the YOLO. This approach of selecting a model based on performance was used to avoid choosing an overfit model, while the secondary validation of the selected model (on different testing images) was intended to assess generalizability to unseen image data.

Estimated chondrocyte densities in pig cartilage were close in magnitude to previously reported values in a large animal (~1000 cells/mm²).³⁷ Reduced chondrocyte density and more active cloning was found beginning 3 months after intra-articular fracture, which corroborates the elevated histological scoring of PTOA previously reported in this animal model at the same time point.³¹ Chondrocyte density began to progressively, though not significantly, rebound by 6 months (increased 28 cells/mm²; p=0.716) and 12 months (increased 73 cells/mm²; p=0.5734) after the fracture. Despite greater clone densities at 6 and 12 months, the progressive increases in chondrocyte density were found to be mainly from increases in chondrocytes not associated with clones (non-clone related chondrocyte density: 580, 596, 642 cells/mm² for 3, 6, and 12 months after fracture, respectively). At 6 and 12 month postoperative timepoints, the combination of lower than normal chondrocyte density, increased number of chondrocyte clones, and increasing chondrocyte cellularity outside clones indicate progressive arthritic changes, but not necessarily a linear progression through the stages of cellularity outlined in the Mankin scoring system.¹ Local increases in clone density at 3 months were found close to the region that articulates with the healing fracture line, and then more anteriorly and posteriorly relative to that location by 6 months and 12 months after fracture. These findings would indicate spatial progression of cartilage abnormalities through the joint that are associated with time after injury, however determining the mechanism responsible for this would require a separate study.

There are several limitations in this work that need to be considered. The first is that we have trained the YOLO model to identify a chondrocyte clone based on our empirical definition of >3 non-linearly organized chondrocytes in a single lacuna. These criteria were chosen to provide a standardized morphological description of the appearance of a specific feature of osteoarthritic cartilage identified in the Mankin scoring scheme as “cloning”.¹ This term “clone” has been used interchangeably with the term “cluster” or “proliferation” in other scoring schemes.² A true clone would imply that all the chondrocytes present in any given instance of this feature were derived from a single cell, and such an assessment would require a detailed analysis of cell proliferation markers that is not feasible in Safranin-O sections. While determining the true clonal/proliferative nature of the cells in these features is outside the scope of this work, a future study combining this automated feature detection approach with specific stages of chondrocyte proliferation/activity could provide important information about the time course of chondrocyte function throughout the course of osteoarthritic cartilage degeneration after joint injury. It would also be interesting to relate such findings to local collagen disruption or proteoglycan concentrations as such covariates would presumably be closely related to cellular response. However, our model is presently trained to identify clones independent of surrounding tissue appearance, and as such it cannot provide any mechanistic or associative information relating chondrocyte proliferation to tissue structure/composition.

Secondly, the inter-observer agreement on clone identification using this definition was relatively lower (ICC = 0.76) than expert agreement achieved for identification of individual chondrocytes (ICC = 0.87).²⁵ However, given the much smaller number of clones in an image compared to the total number of chondrocytes, this reduction in ICC value is a function of disagreement on a very small number of clones in the full data set. Another limitation was that the YOLO detected some false positive clones among sparsely distributed chondrocytes and some false detections of small clone sizes (Figure 4). A possible reason for the false positives is that the training images included mostly clustered chondrocytes, leaving the model without sufficient training to recognize sparser chondrocyte patterns. A possible reason for identifying artificially small clones was that the YOLO was trained using rabbit cartilage and clones from rabbit cartilage are smaller than those that can develop in much thicker pig cartilage. Retraining using additional images of sparse chondrocytes, larger clones, and multiple species could potentially further improve performance over the good agreement with experts that was achieved with this version of the YOLO. Finally, there were few statistically significant differences in clone density associated with time after fracture in the minipig model of PTOA, which is attributed both to the presence of some clones in normal minipig talar cartilage, and to the small number of animals available for each study group (n = 5) in this secondary analysis of existing histological sections. As hypothesized, progressive chondrocyte/clone density changes were identified, although adequately powered future studies will be needed to fully document the natural history of chondrocyte activity in joints sustaining intra-articular fractures.

In conclusion, two CNN models were developed, validated, and implemented to document progressive changes in chondrocyte density & cloning in a minipig model of intra-articular fracture, which is known develop post-traumatic OA. The accuracy of the resulting chondrocyte segmentation and clone identification was very similar to the gold-standard of human expert chondrocyte/clone identification. This fully objectively obtained cellularity data could be incorporated into an automated image-analysis based histological scoring system for OA progression, or used as a stand-alone technique to quantify changes in cellularity and chondrocyte organization in articular cartilage. This deep-learning-powered approach provides objective and accurate cartilage health information and can thus better facilitate studies of cartilage injury and regeneration.