Michael W. Kattan

The performance of prediction models can be assessed using a variety of methods and metrics. Traditional measures for binary and survival outcomes include the Brier score to indicate overall model performance, the concordance (or c) statistic for discriminative ability (or area under the receiver operating characteristic [ROC] curve), and goodness-of-fit statistics for calibration.Several new measures have recently been proposed that can be seen as refinements of discrimination measures, including variants of the c statistic for survival, reclassification tables, net reclassification improvement (NRI), and integrated discrimination improvement (IDI). Moreover, decision-analytic measures have been proposed, including decision curves to plot the net benefit achieved by making decisions based on model predictions.We aimed to define the role of these relatively novel approaches in the evaluation of the performance of prediction models. For illustration, we present a case study of predicting the presence of residual tumor versus benign tissue in patients with testicular cancer (n = 544 for model development, n = 273 for external validation).We suggest that reporting discrimination and calibration will always be important for a prediction model. Decision-analytic measures should be reported if the predictive model is to be used for clinical decisions. Other measures of performance may be warranted in specific applications, such as reclassification metrics to gain insight into the value of adding a novel predictor to an established model.

BACKGROUND: Few published studies have combined clinical prognostic factors into risk profiles that can be used to predict the likelihood of recurrence or metastatic progression in patients following treatment of prostate cancer. We developed a nomogram that allows prediction of disease recurrence through use of preoperative clinical factors for patients with clinically localized prostate cancer who are candidates for treatment with a radical prostatectomy. METHODS: By use of Cox proportional hazards regression analysis, we modeled the clinical data and disease follow-up for 983 men with clinically localized prostate cancer whom we intended to treat with a radical prostatectomy. Clinical data included pretreatment serum prostate-specific antigen levels, biopsy Gleason scores, and clinical stage. Treatment failure was recorded when there was clinical evidence of disease recurrence, a rising serum prostate-specific antigen level (two measurements of 0.4 ng/mL or greater and rising), or initiation of adjuvant therapy. Validation was performed on a separate sample of 168 men, also from our institution. RESULTS: Treatment failure (i.e., cancer recurrence) was noted in 196 of the 983 men, and the patients without failure had a median follow-up of 30 months (range, 1-146 months). The 5-year probability of freedom from failure for the cohort was 73% (95% confidence interval = 69%-76%). The predictions from the nomogram appeared accurate and discriminating, with a validation sample area under the receiver operating characteristic curve (i.e., comparison of the predicted probability with the actual outcome) of 0.79. CONCLUSIONS: A nomogram has been developed that can be used to predict the 5-year probability of treatment failure among men with clinically localized prostate cancer treated with radical prostatectomy.

PURPOSE: Laparoscopic partial nephrectomy is an increasingly performed, minimally invasive alternative to open partial nephrectomy. We compared early postoperative outcomes in 1,800 patients undergoing open partial nephrectomy by experienced surgeons with the initial experience with laparoscopic partial nephrectomy in patients with a single renal tumor 7 cm or less. MATERIALS AND METHODS: Data on 1,800 consecutive open or laparoscopic partial nephrectomies were collected prospectively or retrospectively in tumor registries at 3 large referral centers. Demographic, intraoperative, postoperative and followup data were compared between the 2 groups. RESULTS: Compared to the laparoscopic partial nephrectomy group of 771 patients the 1,028 undergoing open partial nephrectomy were a higher risk group with a greater percent presenting symptomatically with decreased performance status, impaired renal function and tumor in a solitary functioning kidney (p<0.0001). More tumors in the open partial nephrectomy group were more than 4 cm and centrally located and more proved to be malignant (p<0.0001 and 0.0003, respectively). Based on multivariate analysis laparoscopic partial nephrectomy was associated with shorter operative time (p<0.0001), decreased operative blood loss (p<0.0001) and shorter hospital stay (p<0.0001). The chance of intraoperative complications was comparable in the 2 groups. However, laparoscopic partial nephrectomy was associated with longer ischemia time (p<0.0001), more postoperative complications, particularly urological (p<0.0001), and an increased number of subsequent procedures (p<0.0001). Renal functional outcomes were similar 3 months after laparoscopic and open partial nephrectomy with 97.9% and 99.6% of renal units retaining function, respectively. Three-year cancer specific survival for patients with a single cT1N0M0 renal cell carcinoma was 99.3% and 99.2% after laparoscopic and open partial nephrectomy, respectively. CONCLUSIONS: Early experience with laparoscopic partial nephrectomy is promising. Laparoscopic partial nephrectomy offered the advantages of less operative time, decreased operative blood loss and a shorter hospital stay. When applied to patients with a single renal tumor 7 cm or less, laparoscopic partial nephrectomy was associated with additional postoperative morbidity compared to open partial nephrectomy. However, equivalent functional and early oncological outcomes were achieved.

PURPOSE: We analyzed the long-term progression-free probability after radical retropubic prostatectomy in a consecutive series of patients with localized prostate cancer. MATERIALS AND METHODS: From 1983 to 1998, 1,000 patients (median age 62.9 years, range 37.7 to 81.4) with clinical stage T1 to T2 prostate cancer were treated with radical retropubic prostatectomy and pelvic lymphadenectomy, without other cancer related therapy before recurrence. Mean followup was 53.2 months (median 46.9, range 1 to 170). RESULTS: Ten years after radical retropubic prostatectomy the mean probability +/- 2 standard errors that patients remained free of progression and of any further treatment was 75.0% +/- 3.7% and of metastasis 84.2% +/- 4.4%. Mean actuarial cancer specific survival rate +/- 2 standard error was 97.6% +/- 1.7%. In a multivariate analysis pretreatment prostate specific antigen level (p <0.0001), biopsy Gleason sum (p <0.0001) and clinical stage (p=0.0071) were independent prognostic factors for progression. After prostatectomy independent risk factors were Gleason sum in the prostatectomy specimen (p=0.0008), extracapsular extension (p=0.0019), seminal vesical involvement (p <0.0001), lymph node metastasis (p <0.0001) and surgical margin status (p <0.0001). Margins were positive in 12.8% of cases. At 10 years postoperatively radical retropubic prostatectomy was effective for cancer confined to the prostate (92.2% progression-free probability) and also not confined (52.8%), including 71.4% progression-free probability for patients with only extracapsular extension and 37.4% with seminal vesicle invasion without lymph node metastasis. CONCLUSIONS: Radical retropubic prostatectomy provided long-term cancer control in 75% of patients with clinically localized prostate cancer and was effective in the majority of those with high risk cancer, including T2c or biopsy Gleason sum 8 to 10, or PSA greater than 20 ng./ml. Further research should address identifying patients who can safely avoid aggressive therapy.

PURPOSE: An increasing serum prostate-specific antigen (PSA) level is the initial sign of recurrent prostate cancer among patients treated with radical prostatectomy. Salvage radiation therapy (SRT) may eradicate locally recurrent cancer, but studies to distinguish local from systemic recurrence lack adequate sensitivity and specificity. We developed a nomogram to predict the probability of cancer control at 6 years after SRT for PSA-defined recurrence. PATIENTS AND METHODS: Using multivariable Cox regression analysis, we constructed a model to predict the probability of disease progression after SRT in a multi-institutional cohort of 1,540 patients. RESULTS: The 6-year progression-free probability was 32% (95% CI, 28% to 35%) overall. Forty-eight percent (95% CI, 40% to 56%) of patients treated with SRT alone at PSA levels of 0.50 ng/mL or lower were disease free at 6 years, including 41% (95% CI, 31% to 51%) who also had a PSA doubling time of 10 months or less or poorly differentiated (Gleason grade 8 to 10) cancer. Significant variables in the model were PSA level before SRT (P < .001), prostatectomy Gleason grade (P < .001), PSA doubling time (P < .001), surgical margins (P < .001), androgen-deprivation therapy before or during SRT (P < .001), and lymph node metastasis (P = .019). The resultant nomogram was internally validated and had a concordance index of 0.69. CONCLUSION: Nearly half of patients with recurrent prostate cancer after radical prostatectomy have a long-term PSA response to SRT when treatment is administered at the earliest sign of recurrence. The nomogram we developed predicts the outcome of SRT and should prove valuable for medical decision making for patients with a rising PSA level.