TabularPredictor.calibrate_decision_threshold#

TabularPredictor.calibrate_decision_threshold(data: str | TabularDataset | pd.DataFrame | None = None, metric: str | Scorer | None = None, model: str = 'best', decision_thresholds: int | List[float] = 50, verbose: bool = True) → float[source]#

Calibrate the decision threshold in binary classification to optimize a given metric. You can pass the output of this method as input to predictor.set_decision_threshold to update the predictor. Will raise an AssertionError if predictor.problem_type != ‘binary’.

Note that while calibrating the decision threshold can help to improve a given metric, other metrics may end up having worse scores. For example, calibrating on balanced_accuracy will often harm accuracy. Users should keep this in mind while leveraging decision threshold calibration.

Parameters

  • data (Union[str, pd.DataFrame], default = None) – The data to use for calibration. Must contain the label column. We recommend to keep this value as None unless you are an advanced user and understand the implications. If None, will use internal data such as the holdout validation data or out-of-fold predictions.

  • metric (autogluon.core.metrics.Scorer or str, default = None) – The metric to optimize during calibration. If None, uses predictor.eval_metric.

  • model (str, default = 'best') – The model to use prediction probabilities of when calibrating the threshold. If ‘best’, will use predictor.get_model_best().

  • decision_thresholds (Union[int, List[float]], default = 50) – The number of decision thresholds on either side of 0.5 to search. The default of 50 will result in 101 searched thresholds: [0.00, 0.01, 0.02, …, 0.49, 0.50, 0.51, …, 0.98, 0.99, 1.00] Alternatively, a list of decision thresholds can be passed and only the thresholds in the list will be searched.

  • verbose (bool, default = True) – If True, will log information about the calibration process.

Returns

  • Decision Threshold (A float between 0 and 1 defining the decision boundary for predictions that)

  • maximizes the metric score on the data for the model.