Discussion
As patients and physicians enthusiastically pursue AS as a strategy to minimize morbidity associated with overtreatment of PCa, the ability to accurately select men suitable for this approach is paramount. Predictive models to help select men for AS have been published with several easily accessible to clinicians and patients on the internet. However, despite evidence in the literature suggesting caution with application of predictive models outside their derived populations, many clinicians and patients may be utilizing these tools inappropriately.
In our analysis, from a collaborative database of men who underwent upfront radical prostatectomy, we analyzed four models purported to predict men with insignificant PCa, a surrogate for suitability for AS. Unlike several of the original model analyses, we adopted the approach by Kattan et al. and restricted inclusion to men who truly might be considered for AS. Thus men who had high-risk PCa at biopsy (for example, Gleason 8 disease), who would never be clinically considered for AS, were not analyzed, as previous work has indicated the low probability of low-risk disease in this scenario. In addition, as the boundaries defining 'insignificant cancer' and suitability for AS are being challenged, we tested the performance of these predictive models using four progressively lenient definitions of insignificant cancer (Table 2).
Our receiver operator characteristic curves with calculated AUC (Figure 1) showed at best, modest performance of the different models (maximal AUC of 0.664). Not surprisingly, the models performed better in predicting the more stringent definitions of insignificant cancer (Epstein, and cancer volume ≤1.3 ml), as this is what they were originally designed for. The P-values for nearly all models, across the four definitions, were significant or approached significance. Thus the AUCs found, although moderate, are likely to statistically be better than tossing a coin (AUC=0.5).
These results highlight the need for careful validation of predictive models. Internal validation, using the data set the model was originally derived from, statistically tends to overestimate both discriminative and calibrative ability. In our external validation of both Chun and O'Brien nomograms, the AUCs calculated (0.563 and 0.633, respectively) were markedly lower than original reports (0.904 and 0.933, respectively). Their performance was even poorer than that demonstrated on similar analysis by Iremashvilli et al. (AUC 0.729 and 0.739, respectively). Furthermore, the calibration curves for all models (Figure 2), with the exception of Kattan, had restricted range of predicted probabilities to <0.4. This indicates that their originally reported high AUC was driven by the pre-test probability of insignificant disease in the derived population, rather than any discriminatory power. In both Chun and O'Brien models, only ~5% of their cohorts had defined 'insignificant disease'. Conversely, in an example of a model that was statistically sound but of limited clinical value, a different nomogram restricted its analysis to men with only one positive core at biopsy. To our knowledge, there is no published AS cohort that restricts eligibility to a single biopsy core. At last, the decision analysis curve for the best performing model (Kattan predicting Epstein insignificant disease) was no better than assuming all men had indolent disease. To put it simply, these models lack the ability to identify men with a high probability of insignificant disease.
Another possible reason for disparities between internal and external validation are true differences between the cohorts. The Kattan model was published in 2003, but derived from a population of men who had radical prostatectomy between 1986 and 2000. Changes in biopsy technique (sextant to extended) and Gleason grade reporting have occurred, and hence the patient seated in front of clinicians today differs to those this model was derived from. Geographical differences may also exist. The Steyerberg model updates the Kattan, but is derived from men with screen detected cancer. Hence, it may not be appropriate for populations such as the United Kingdom (a component of our validation cohort) with low prevalence of PSA testing. Furthermore, attempted validation was performed for definitions of insignificant disease with minor variations to what the nomograms were originally designed for. However, interestingly the models showed similar performance in predicting Epstein and Wolters' definitions of insignificant disease. Similarly, work to investigate whether these models could be used to predict pathological progression for men on AS was limited. At last, models can only be as good as the data available to create or validate them. It is increasingly apparent that transrectal ultrasound-guided prostate biopsy is an imperfect test with significant sampling error. The discrepancy in both grade and volume between biopsy and radical prostatectomy is difficult to model statistically.
Incorporation of advances in imaging, biomarkers and genetic tests may be able enhance these models. Developments in multiparametric magnetic resonance imaging, with standardized scoring and proposed ability to differentially detect more aggressive cancers, offer the greatest potential for incorporation into individualized predictive models. Magnetic resonance imaging-guided biopsy, be it cognitive, ultrasound-guided or in-bore, in addition presents a potential solution to overcoming the sampling error associated with standard transrectal prostate biopsy. Furthermore, magnetic resonance imaging may be able to pre-operatively directly assess tumor volume as a predictive variable. Transperineal template biopsies sample more of the gland but are still a 'blinded' form of biopsy. The emergence of biomarkers that can improve the predictive ability of established clinical pathological variables, and possibly predict more aggressive disease, are likely to be incorporated into models in the future.
Our study sample size (n=460) limits the statistical power of our analysis. However, it does represent multi-institutional data and is one of the largest cohorts of men with 'low-risk disease' (using Kattan pre-selection criteria) that has undergone upfront radical prostatectomy published in the literature. The Kattan nomogram was itself derived from a similar cohort of 409 men. As fewer and fewer men undergo radical prostatectomy for 'low-risk disease', this data is of value. The cohorts were collected over 6–12 years, and thus more recently AS has become acceptable and increasingly adopted. Thus it must be remembered that although the patients analyzed had 'low-risk disease', they represent a surgical cohort, with all its unforeseen biases compared with a strict AS cohort. These may include age, family history, comorbidities, patient anxiety, imaging findings and institutional preferences for treatment. Being retrospective, differences in data collection between individual centers could not be controlled. Percentage of core involved by cancer, reported by Cambridge and Toronto, required conversion to millimeters of cancer using a median total core length and thus could have affected calculations. Furthermore, adverse pathology at radical prostatectomy may not correlate with longer-term outcomes such as biochemical recurrence and cancer-specific mortality, more clinically relevant outcomes.
For this investigation, we set out to ascertain the most effective published predictive model to guide clinicians in identifying men with insignificant PCa. Our results showed that the four models found all performed moderately at best, and only the Kattan model had any useable predictive range at calibration for our cohort. We would counsel to use these easily accessible models with care, as there is a danger that uneducated use, without understanding of how they are derived or their limitations, could result in false reassurance or anxiety.