aif360.algorithms.postprocessing

Calibrated Equality of Odds

class aif360.algorithms.postprocessing.CalibratedEqOddsPostprocessing(unprivileged_groups, privileged_groups, cost_constraint='weighted', seed=None)[source]

Calibrated equalized odds postprocessing is a post-processing technique that optimizes over calibrated classifier score outputs to find probabilities with which to change output labels with an equalized odds objective [7].

References

[7]G. Pleiss, M. Raghavan, F. Wu, J. Kleinberg, and K. Q. Weinberger, “On Fairness and Calibration,” Conference on Neural Information Processing Systems, 2017

Adapted from: https://github.com/gpleiss/equalized_odds_and_calibration/blob/master/calib_eq_odds.py

Parameters:
  • unprivileged_groups (dict or list(dict)) – Representation for unprivileged group.
  • privileged_groups (dict or list(dict)) – Representation for privileged group.
  • cost_contraint – fpr, fnr or weighted
  • seed (int, optional) – Seed to make predict repeatable.
fit(dataset_true, dataset_pred)[source]

Compute parameters for equalizing generalized odds using true and predicted scores, while preserving calibration.

Parameters:
Returns:

Returns self.

Return type:

CalibratedEqOddsPostprocessing

fit_predict(dataset_true, dataset_pred, threshold=0.5)[source]

fit and predict methods sequentially.

predict(dataset, threshold=0.5)[source]

Perturb the predicted scores to obtain new labels that satisfy equalized odds constraints, while preserving calibration.

Parameters:
  • dataset (BinaryLabelDataset) – Dataset containing scores that needs to be transformed.
  • threshold (float) – Threshold for converting scores to labels. Values greater than or equal to this threshold are predicted to be the favorable_label. Default is 0.5.
Returns:

transformed dataset.

Return type:

dataset (BinaryLabelDataset)

Equality of Odds

class aif360.algorithms.postprocessing.EqOddsPostprocessing(unprivileged_groups, privileged_groups, seed=None)[source]

Equalized odds postprocessing is a post-processing technique that solves a linear program to find probabilities with which to change output labels to optimize equalized odds [8] [9].

References

[8]M. Hardt, E. Price, and N. Srebro, “Equality of Opportunity in Supervised Learning,” Conference on Neural Information Processing Systems, 2016.
[9]G. Pleiss, M. Raghavan, F. Wu, J. Kleinberg, and K. Q. Weinberger, “On Fairness and Calibration,” Conference on Neural Information Processing Systems, 2017.
Parameters:
  • unprivileged_groups (list(dict)) – Representation for unprivileged group.
  • privileged_groups (list(dict)) – Representation for privileged group.
  • seed (int, optional) – Seed to make predict repeatable.
fit(dataset_true, dataset_pred)[source]

Compute parameters for equalizing odds using true and predicted labels.

Parameters:
Returns:

Returns self.

Return type:

EqOddsPostprocessing

fit_predict(dataset_true, dataset_pred)[source]

fit and predict methods sequentially.

predict(dataset)[source]

Perturb the predicted labels to obtain new labels that satisfy equalized odds constraints.

Parameters:
  • dataset (BinaryLabelDataset) – Dataset containing labels that needs to be transformed.
  • dataset – Transformed dataset.

Reject Option Classification

class aif360.algorithms.postprocessing.RejectOptionClassification(unprivileged_groups, privileged_groups, low_class_thresh=0.01, high_class_thresh=0.99, num_class_thresh=100, num_ROC_margin=50, metric_name='Statistical parity difference', metric_ub=0.05, metric_lb=-0.05)[source]

Reject option classification is a postprocessing technique that gives favorable outcomes to unpriviliged groups and unfavorable outcomes to priviliged groups in a confidence band around the decision boundary with the highest uncertainty [10].

References

[10]F. Kamiran, A. Karim, and X. Zhang, “Decision Theory for Discrimination-Aware Classification,” IEEE International Conference on Data Mining, 2012.
Parameters:
  • unprivileged_groups (dict or list(dict)) – Representation for unprivileged group.
  • privileged_groups (dict or list(dict)) – Representation for privileged group.
  • low_class_thresh (float) – Smallest classification threshold to use in the optimization. Should be between 0. and 1.
  • high_class_thresh (float) – Highest classification threshold to use in the optimization. Should be between 0. and 1.
  • num_class_thresh (int) – Number of classification thresholds between low_class_thresh and high_class_thresh for the optimization search. Should be > 0.
  • num_ROC_margin (int) – Number of relevant ROC margins to be used in the optimization search. Should be > 0.
  • metric_name (str) – Name of the metric to use for the optimization. Allowed options are “Statistical parity difference”, “Average odds difference”, “Equal opportunity difference”.
  • metric_ub (float) – Upper bound of constraint on the metric value
  • metric_lb (float) – Lower bound of constraint on the metric value
fit(dataset_true, dataset_pred)[source]

Estimates the optimal classification threshold and margin for reject option classification that optimizes the metric provided.

Note

The fit function is a no-op for this algorithm.

Parameters:
Returns:

Returns self.

Return type:

RejectOptionClassification

fit_predict(dataset_true, dataset_pred)[source]

fit and predict methods sequentially.

predict(dataset)[source]

Obtain fair predictions using the ROC method.

Parameters:dataset (BinaryLabelDataset) – Dataset containing scores that will be used to compute predicted labels.
Returns:Output dataset with potentially fair predictions obtain using the ROC method.
Return type:dataset_pred (BinaryLabelDataset)