confusion_matrix

TabularPredictor.confusion_matrix(data: str | TabularDataset | DataFrame | None = None, model: str | None = None, decision_threshold: float | None = None, normalize: str | Literal['true', 'pred', 'all'] | None = None, labels: list | None = None, display: bool = True, save_path: str | None = None, **kwargs)[source]

Compute and optionally plot the confusion matrix for classification models.

Parameters:

  • data (str or TabularDataset or pd.DataFrame, optional) –

    Data to evaluate. If None:

    • If validation data exists (predictor.has_val), it is used.

    • If no validation data exists and predictor is in bagged mode, strictly out-of-fold (OOF) predictions are used.

    • If no validation data exists and predictor is NOT in bagged mode, training data is used (Note: this effectively evaluates on the data the model was trained on, which will likely result in overly optimistic scores).

    Can be a file path, pd.DataFrame or TabularDataset object.

  • model (str, optional) – The name of the model to get predictions from. Defaults to None, which uses the highest scoring model. Valid models are listed in this predictor by calling predictor.model_names()

  • decision_threshold (float, optional) – The decision threshold used to convert prediction probabilities to predictions. Only relevant for binary classification, otherwise ignored. If None, defaults to predictor.decision_threshold. Valid values are in the range [0.0, 1.0]

  • normalize ({'true', 'pred', 'all'}, optional) – Normalization mode (see sklearn.metrics.confusion_matrix). If None, no normalization is applied.

  • labels (list, optional) – Class label ordering. If None, uses the predictor’s known class labels.

  • display (bool, default=True) – If True, displays the confusion matrix plot.

  • save_path (str, optional) – File path to save the confusion matrix plot. If None, the plot is not saved.

  • **kwargs (dict) – Additional keyword arguments passed to sklearn.metrics.confusion_matrix.

Returns:

cm – Confusion matrix where entry (i, j) indicates the number of samples with true label i and predicted label j.

Return type:

ndarray of shape (n_classes, n_classes)

Raises:

ValueError – If the problem type is not ‘binary’ or ‘multiclass’. If no data is available to compute the confusion matrix. If the label column is missing in the provided data.

Examples

>>> predictor = TabularPredictor(label='class').fit(train_data)
>>> cm = predictor.confusion_matrix()  # Uses data (validation if available, else OOF or train)
>>> cm = predictor.confusion_matrix(test_data)  # Uses test data
>>> cm = predictor.confusion_matrix(test_data, normalize='true', save_path='cm.png')
>>> cm = predictor.confusion_matrix(model='LightGBM', decision_threshold=0.8)

References

https://scikit-learn.org/stable/modules/generated/sklearn.metrics.confusion_matrix.html