Skip to content

Classification and survival

Most of the classification metrics come from scikit-learn and the survival estimate from lifelines, and depictr redraws both under the shared theme. The survival figure adds a number-at-risk table beneath the curves in one call.

import numpy as np

import depictr as dp

ct = dp.clinical_trial()
score = 1 / (1 + np.exp(-ct["biomarker"]))  # a probability-like score

ROC curve

An ROC curve with the area under the curve.

p = dp.roc_curve_plot(ct["adverse_event"], score)
print(show(p))
depictr plot

Precision-recall on a rare outcome

When the positive class is rare the precision-recall curve is more informative than the ROC curve, because it ignores the many true negatives.

p = dp.pr_curve_plot(ct["adverse_event"], score)
print(show(p))
depictr plot

Gains, lift and choosing a threshold

The cumulative gains and lift charts read off how many positives a ranked campaign captures at each depth.

p = dp.gain_plot(ct["adverse_event"], score)
print(show(p))
depictr plot 
p = dp.lift_plot(ct["adverse_event"], score)
print(show(p))
depictr plot

The threshold plot sweeps the decision cut-off, so the trade-off between the metrics as the operating point moves can be read straight off the curves.

p = dp.threshold_plot(ct["adverse_event"], score)
print(show(p))
depictr plot

Calibration

A calibration curve asks whether a predicted probability of, say, 0.3 is borne out by an event in about 30% of such cases, so it needs probabilities from a fitted model rather than the ranking score the charts above are happy with. We fit a logistic regression and pass its predicted probabilities.

from sklearn.linear_model import LogisticRegression

X = ct[["biomarker", "age"]]
fit = LogisticRegression().fit(X, ct["adverse_event"])
p = dp.calibration_plot(ct["adverse_event"], fit.predict_proba(X)[:, 1],
                        n_bins=5)
print(show(p))
depictr plot

Each of the five bins holds 60 patients, and each point is sized by its count. The mean predicted risks run from 0.03 to 0.24 and the observed rates from 0.02 to 0.28. Every 95% Wilson interval includes its bin's predicted risk, so the curve is consistent with a well-calibrated model. Equal-width bins (strategy="uniform") would put 263 patients in the lowest bin and leave the highest with four, none of whom had an event.

Confusion matrix

A confusion-matrix heatmap, normalised by the true class.

p = dp.confusion_matrix_plot(ct["adverse_event"], (score > 0.6).astype(int),
                             normalise="true")
print(show(p, width=6))
depictr plot

Survival

Kaplan-Meier curves with a log-rank test and a number-at-risk table. The dotted steps are 95% confidence limits, built on the log-log scale as in the R package, and each + marks a censored patient. The legend sits in the empty top-right the descending curves leave behind.

p = dp.survival_plot(ct["time"], ct["event"], group=ct["arm"],
                     risk_table=True, legend_inside=True, logrank=True,
                     title="Survival by arm")
print(show(p, width=8, height=6))
depictr plot