Welcome to AI Fairness 360’s documentation!

aif360.algorithms

aif360.algorithms.preprocessing

Disparate Impact Remover

class aif360.algorithms.preprocessing.DisparateImpactRemover(repair_level=1.0, sensitive_attribute='')[source]

Disparate impact remover is a preprocessing technique that edits feature values increase group fairness while preserving rank-ordering within groups [1].

References

[1]M. Feldman, S. A. Friedler, J. Moeller, C. Scheidegger, and S. Venkatasubramanian, “Certifying and removing disparate impact.” ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2015.
Parameters:
  • repair_level (float) – Repair amount. 0.0 is no repair while 1.0 is full repair.
  • sensitive_attribute (str) – Single protected attribute with which to do repair.
fit_transform(dataset)[source]

Run a repairer on the non-protected features and return the transformed dataset.

Parameters:dataset (BinaryLabelDataset) – Dataset that needs repair.
Returns:Transformed Dataset.
Return type:dataset (BinaryLabelDataset)

Note

In order to transform test data in the same manner as training data, the distributions of attributes conditioned on the protected attribute must be the same.

Learning Fair Representations

class aif360.algorithms.preprocessing.LFR(unprivileged_groups, privileged_groups, k=5, Ax=0.01, Ay=1.0, Az=50.0, print_interval=250, verbose=1, seed=None)[source]

Learning fair representations is a pre-processing technique that finds a latent representation which encodes the data well but obfuscates information about protected attributes [2].

References

[2]R. Zemel, Y. Wu, K. Swersky, T. Pitassi, and C. Dwork, “Learning Fair Representations.” International Conference on Machine Learning, 2013.

Based on code from https://github.com/zjelveh/learning-fair-representations

Parameters:
  • unprivileged_groups (tuple) – Representation for unprivileged group.
  • privileged_groups (tuple) – Representation for privileged group.
  • k (int, optional) – Number of prototypes.
  • Ax (float, optional) – Input recontruction quality term weight.
  • Az (float, optional) – Fairness constraint term weight.
  • Ay (float, optional) – Output prediction error.
  • print_interval (int, optional) – Print optimization objective value every print_interval iterations.
  • verbose (int, optional) – If zero, then no output.
  • seed (int, optional) – Seed to make predict repeatable.
fit(dataset, **kwargs)[source]

Compute the transformation parameters that leads to fair representations.

Parameters:dataset (BinaryLabelDataset) – Dataset containing true labels.
Returns:Returns self.
Return type:LFR
fit_transform(dataset, seed=None)[source]

fit and transform methods sequentially

transform(dataset, threshold=0.5, **kwargs)[source]

Transform the dataset using learned model parameters.

Parameters:
  • dataset (BinaryLabelDataset) – Dataset containing labels that needs to be transformed.
  • threshold (float, optional) – threshold parameter used for binary label prediction.
Returns:

Transformed Dataset.
Return type:

dataset (BinaryLabelDataset)

Optimized Preprocessing

class aif360.algorithms.preprocessing.OptimPreproc(optimizer, optim_options, unprivileged_groups=None, privileged_groups=None, verbose=False, seed=None)[source]

Optimized preprocessing is a preprocessing technique that learns a probabilistic transformation that edits the features and labels in the data with group fairness, individual distortion, and data fidelity constraints and objectives [3].

References

[3]F. P. Calmon, D. Wei, B. Vinzamuri, K. Natesan Ramamurthy, and K. R. Varshney. “Optimized Pre-Processing for Discrimination Prevention.” Conference on Neural Information Processing Systems, 2017.

Based on code available at: https://github.com/fair-preprocessing/nips2017

Parameters:
  • optimizer (class) – Optimizer class.
  • optim_options (dict) – Options for optimization to estimate the transformation.
  • unprivileged_groups (dict) – Representation for unprivileged group.
  • privileged_groups (dict) – Representation for privileged group.
  • verbose (bool, optional) – Verbosity flag for optimization.
  • seed (int, optional) – Seed to make fit and predict repeatable.

Note

This algorithm does not use the privileged and unprivileged groups that are specified during initialization yet. Instead, it automatically attempts to reduce statistical parity difference between all possible combinations of groups in the dataset.

fit(dataset, sep='=')[source]

Compute optimal pre-processing transformation based on distortion constraint.

Parameters:
  • dataset (BinaryLabelDataset) – Dataset containing true labels.
  • sep (str, optional) – Separator for converting one-hot labels to categorical.
Returns:

Returns self.
Return type:

OptimPreproc
fit_transform(dataset, sep='=', transform_Y=True)[source]

Perfom fit() and transform() sequentially.

transform(dataset, sep='=', transform_Y=True)[source]

Transform the dataset to a new dataset based on the estimated transformation.

Parameters:
  • dataset (BinaryLabelDataset) – Dataset containing labels that needs to be transformed.
  • transform_Y (bool) – Flag that mandates transformation of Y (labels).

Reweighing

class aif360.algorithms.preprocessing.Reweighing(unprivileged_groups, privileged_groups)[source]

Reweighing is a preprocessing technique that Weights the examples in each (group, label) combination differently to ensure fairness before classification [4].

References

[4]F. Kamiran and T. Calders, “Data Preprocessing Techniques for Classification without Discrimination,” Knowledge and Information Systems, 2012.
Parameters:
  • unprivileged_groups (list(dict)) – Representation for unprivileged group.
  • privileged_groups (list(dict)) – Representation for privileged group.
fit(dataset)[source]

Compute the weights for reweighing the dataset.

Parameters:dataset (BinaryLabelDataset) – Dataset containing true labels.
Returns:Returns self.
Return type:Reweighing
fit_transform(dataset)

Train a model on the input and transform the dataset accordingly.

Equivalent to calling fit(dataset) followed by transform(dataset).

Parameters:dataset (Dataset) – Input dataset.
Returns:Output dataset. metadata should reflect the details of this transformation.
Return type:Dataset
transform(dataset)[source]

Transform the dataset to a new dataset based on the estimated transformation.

Parameters:dataset (BinaryLabelDataset) – Dataset that needs to be transformed.
Returns:Transformed dataset.
Return type:dataset (BinaryLabelDataset)

aif360.algorithms.inprocessing

Adversarial Debiasing

class aif360.algorithms.inprocessing.AdversarialDebiasing(unprivileged_groups, privileged_groups, scope_name, sess, seed=None, adversary_loss_weight=0.1, num_epochs=50, batch_size=128, classifier_num_hidden_units=200, debias=True)[source]

Adversarial debiasing is an in-processing technique that learns a classifier to maximize prediction accuracy and simultaneously reduce an adversary’s ability to determine the protected attribute from the predictions [5]. This approach leads to a fair classifier as the predictions cannot carry any group discrimination information that the adversary can exploit.

References

[5]B. H. Zhang, B. Lemoine, and M. Mitchell, “Mitigating Unwanted Biases with Adversarial Learning,” AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society, 2018.
Parameters:
  • unprivileged_groups (tuple) – Representation for unprivileged groups
  • privileged_groups (tuple) – Representation for privileged groups
  • scope_name (str) – scope name for the tenforflow variables
  • sess (tf.Session) – tensorflow session
  • seed (int, optional) – Seed to make predict repeatable.
  • adversary_loss_weight (float, optional) – Hyperparameter that chooses the strength of the adversarial loss.
  • num_epochs (int, optional) – Number of training epochs.
  • batch_size (int, optional) – Batch size.
  • classifier_num_hidden_units (int, optional) – Number of hidden units in the classifier model.
  • debias (bool, optional) – Learn a classifier with or without debiasing.
fit(dataset)[source]

Compute the model parameters of the fair classifier using gradient descent.

Parameters:dataset (BinaryLabelDataset) – Dataset containing true labels.
Returns:Returns self.
Return type:AdversarialDebiasing
fit_predict(dataset)

Train a model on the input and predict the labels.

Equivalent to calling fit(dataset) followed by predict(dataset).

Parameters:dataset (Dataset) – Input dataset.
Returns:Output dataset. metadata should reflect the details of this transformation.
Return type:Dataset
predict(dataset)[source]

Obtain the predictions for the provided dataset using the fair classifier learned.

Parameters:dataset (BinaryLabelDataset) – Dataset containing labels that needs to be transformed.
Returns:Transformed dataset.
Return type:dataset (BinaryLabelDataset)

ART Classifier

class aif360.algorithms.inprocessing.ARTClassifier(art_classifier)[source]

Wraps an instance of an art.classifiers.Classifier to extend Transformer.

Initialize ARTClassifier.

Parameters:art_classifier (art.classifier.Classifier) – A Classifier object from the adversarial-robustness-toolbox.
fit(dataset, batch_size=128, nb_epochs=20)[source]

Train a classifer on the input.

Parameters:
  • dataset (Dataset) – Training dataset.
  • batch_size (int) – Size of batches (passed through to ART).
  • nb_epochs (int) – Number of epochs to use for training (passed through to ART).
Returns:

Returns self.
Return type:

ARTClassifier
fit_predict(dataset)

Train a model on the input and predict the labels.

Equivalent to calling fit(dataset) followed by predict(dataset).

Parameters:dataset (Dataset) – Input dataset.
Returns:Output dataset. metadata should reflect the details of this transformation.
Return type:Dataset
predict(dataset, logits=False)[source]

Perform prediction for the input.

Parameters:
  • dataset (Dataset) – Test dataset.
  • logits (bool, optional) – True is prediction should be done at the logits layer (passed through to ART).
Returns:

Dataset with predicted labels in the labels field.
Return type:

Dataset

Meta Fair Classifier

class aif360.algorithms.inprocessing.MetaFairClassifier(tau=0.8, sensitive_attr='', type='fdr')[source]

The meta algorithm here takes the fairness metric as part of the input and returns a classifier optimized w.r.t. that fairness metric [11].

References

[11]L. E. Celis, L. Huang, V. Keswani, and N. K. Vishnoi. “Classification with Fairness Constraints: A Meta-Algorithm with Provable Guarantees,” 2018.
Parameters:
  • tau (double, optional) – Fairness penalty parameter.
  • sensitive_attr (str, optional) – Name of protected attribute.
  • type (str, optional) – The type of fairness metric to be used. Currently “fdr” (false discovery rate ratio) and “sr” (statistical rate/disparate impact) are supported. To use another type, the corresponding optimization class has to be implemented.
fit(dataset)[source]

Learns the fair classifier.

Parameters:dataset (BinaryLabelDataset) – Dataset containing true labels.
Returns:Returns self.
Return type:MetaFairClassifier
fit_predict(dataset)

Train a model on the input and predict the labels.

Equivalent to calling fit(dataset) followed by predict(dataset).

Parameters:dataset (Dataset) – Input dataset.
Returns:Output dataset. metadata should reflect the details of this transformation.
Return type:Dataset
predict(dataset)[source]

Obtain the predictions for the provided dataset using the learned classifier model.

Parameters:dataset (BinaryLabelDataset) – Dataset containing labels that needs to be transformed.
Returns:Transformed dataset.
Return type:BinaryLabelDataset

Prejudice Remover

class aif360.algorithms.inprocessing.PrejudiceRemover(eta=1.0, sensitive_attr='', class_attr='')[source]

Prejudice remover is an in-processing technique that adds a discrimination-aware regularization term to the learning objective [6].

References

[6]T. Kamishima, S. Akaho, H. Asoh, and J. Sakuma, “Fairness-Aware Classifier with Prejudice Remover Regularizer,” Joint European Conference on Machine Learning and Knowledge Discovery in Databases, 2012.
Parameters:
  • eta (double, optional) – fairness penalty parameter
  • sensitive_attr (str, optional) – name of protected attribute
  • class_attr (str, optional) – label name
fit(dataset)[source]

Learns the regularized logistic regression model.

Parameters:dataset (BinaryLabelDataset) – Dataset containing true labels.
Returns:Returns self.
Return type:PrejudiceRemover
fit_predict(dataset)

Train a model on the input and predict the labels.

Equivalent to calling fit(dataset) followed by predict(dataset).

Parameters:dataset (Dataset) – Input dataset.
Returns:Output dataset. metadata should reflect the details of this transformation.
Return type:Dataset
predict(dataset)[source]

Obtain the predictions for the provided dataset using the learned prejudice remover model.

Parameters:dataset (BinaryLabelDataset) – Dataset containing labels that needs to be transformed.
Returns:Transformed dataset.
Return type:dataset (BinaryLabelDataset)

aif360.algorithms.postprocessing

Calibrated Equality of Odds

class aif360.algorithms.postprocessing.CalibratedEqOddsPostprocessing(unprivileged_groups, privileged_groups, cost_constraint='weighted', seed=None)[source]

Calibrated equalized odds postprocessing is a post-processing technique that optimizes over calibrated classifier score outputs to find probabilities with which to change output labels with an equalized odds objective [7].

References

[7]G. Pleiss, M. Raghavan, F. Wu, J. Kleinberg, and K. Q. Weinberger, “On Fairness and Calibration,” Conference on Neural Information Processing Systems, 2017

Adapted from: https://github.com/gpleiss/equalized_odds_and_calibration/blob/master/calib_eq_odds.py

Parameters:
  • unprivileged_groups (dict or list(dict)) – Representation for unprivileged group.
  • privileged_groups (dict or list(dict)) – Representation for privileged group.
  • cost_contraint – fpr, fnr or weighted
  • seed (int, optional) – Seed to make predict repeatable.
fit(dataset_true, dataset_pred)[source]

Compute parameters for equalizing generalized odds using true and predicted scores, while preserving calibration.

Parameters:
Returns:

Returns self.
Return type:

CalibratedEqOddsPostprocessing
fit_predict(dataset_true, dataset_pred, threshold=0.5)[source]

fit and predict methods sequentially.

predict(dataset, threshold=0.5)[source]

Perturb the predicted scores to obtain new labels that satisfy equalized odds constraints, while preserving calibration.

Parameters:
  • dataset (BinaryLabelDataset) – Dataset containing scores that needs to be transformed.
  • threshold (float) – Threshold for converting scores to labels. Values greater than or equal to this threshold are predicted to be the favorable_label. Default is 0.5.
Returns:

transformed dataset.
Return type:

dataset (BinaryLabelDataset)

Equality of Odds

class aif360.algorithms.postprocessing.EqOddsPostprocessing(unprivileged_groups, privileged_groups, seed=None)[source]

Equalized odds postprocessing is a post-processing technique that solves a linear program to find probabilities with which to change output labels to optimize equalized odds [8] [9].

References

[8]M. Hardt, E. Price, and N. Srebro, “Equality of Opportunity in Supervised Learning,” Conference on Neural Information Processing Systems, 2016.
[9]G. Pleiss, M. Raghavan, F. Wu, J. Kleinberg, and K. Q. Weinberger, “On Fairness and Calibration,” Conference on Neural Information Processing Systems, 2017.
Parameters:
  • unprivileged_groups (list(dict)) – Representation for unprivileged group.
  • privileged_groups (list(dict)) – Representation for privileged group.
  • seed (int, optional) – Seed to make predict repeatable.
fit(dataset_true, dataset_pred)[source]

Compute parameters for equalizing odds using true and predicted labels.

Parameters:
Returns:

Returns self.
Return type:

EqOddsPostprocessing
fit_predict(dataset_true, dataset_pred)[source]

fit and predict methods sequentially.

predict(dataset)[source]

Perturb the predicted labels to obtain new labels that satisfy equalized odds constraints.

Parameters:
  • dataset (BinaryLabelDataset) – Dataset containing labels that needs to be transformed.
  • dataset – Transformed dataset.

Reject Option Classification

class aif360.algorithms.postprocessing.RejectOptionClassification(unprivileged_groups, privileged_groups, low_class_thresh=0.01, high_class_thresh=0.99, num_class_thresh=100, num_ROC_margin=50, metric_name='Statistical parity difference', metric_ub=0.05, metric_lb=-0.05)[source]

Reject option classification is a postprocessing technique that gives favorable outcomes to unpriviliged groups and unfavorable outcomes to priviliged groups in a confidence band around the decision boundary with the highest uncertainty [10].

References

[10]F. Kamiran, A. Karim, and X. Zhang, “Decision Theory for Discrimination-Aware Classification,” IEEE International Conference on Data Mining, 2012.
Parameters:
  • unprivileged_groups (dict or list(dict)) – Representation for unprivileged group.
  • privileged_groups (dict or list(dict)) – Representation for privileged group.
  • low_class_thresh (float) – Smallest classification threshold to use in the optimization. Should be between 0. and 1.
  • high_class_thresh (float) – Highest classification threshold to use in the optimization. Should be between 0. and 1.
  • num_class_thresh (int) – Number of classification thresholds between low_class_thresh and high_class_thresh for the optimization search. Should be > 0.
  • num_ROC_margin (int) – Number of relevant ROC margins to be used in the optimization search. Should be > 0.
  • metric_name (str) – Name of the metric to use for the optimization. Allowed options are “Statistical parity difference”, “Average odds difference”, “Equal opportunity difference”.
  • metric_ub (float) – Upper bound of constraint on the metric value
  • metric_lb (float) – Lower bound of constraint on the metric value
fit(dataset_true, dataset_pred)[source]

Estimates the optimal classification threshold and margin for reject option classification that optimizes the metric provided.

Note

The fit function is a no-op for this algorithm.

Parameters:
Returns:

Returns self.
Return type:

RejectOptionClassification
fit_predict(dataset_true, dataset_pred)[source]

fit and predict methods sequentially.

predict(dataset)[source]

Obtain fair predictions using the ROC method.

Parameters:dataset (BinaryLabelDataset) – Dataset containing scores that will be used to compute predicted labels.
Returns:Output dataset with potentially fair predictions obtain using the ROC method.
Return type:dataset_pred (BinaryLabelDataset)

Base Class

class aif360.algorithms.Transformer(**kwargs)[source]

Abstract base class for transformers.

Transformers are an abstraction for any process which acts on a Dataset and returns a new, modified Dataset. This definition encompasses pre-processing, in-processing, and post-processing algorithms.

Initialize a Transformer object.

Algorithm-specific configuration parameters should be passed here.

fit(dataset)[source]

Train a model on the input.

Parameters:dataset (Dataset) – Input dataset.
Returns:Returns self.
Return type:Transformer
fit_predict(dataset)[source]

Train a model on the input and predict the labels.

Equivalent to calling fit(dataset) followed by predict(dataset).

Parameters:dataset (Dataset) – Input dataset.
Returns:Output dataset. metadata should reflect the details of this transformation.
Return type:Dataset
fit_transform(dataset)[source]

Train a model on the input and transform the dataset accordingly.

Equivalent to calling fit(dataset) followed by transform(dataset).

Parameters:dataset (Dataset) – Input dataset.
Returns:Output dataset. metadata should reflect the details of this transformation.
Return type:Dataset
predict(dataset)[source]

Return a new dataset with labels predicted by running this Transformer on the input.

Parameters:dataset (Dataset) – Input dataset.
Returns:Output dataset. metadata should reflect the details of this transformation.
Return type:Dataset
transform(dataset)[source]

Return a new dataset generated by running this Transformer on the input.

This function could return different dataset.features, dataset.labels, or both.

Parameters:dataset (Dataset) – Input dataset.
Returns:Output dataset. metadata should reflect the details of this transformation.
Return type:Dataset

aif360.datasets

Base Class

class aif360.datasets.Dataset(**kwargs)[source]

Abstract base class for datasets.

copy(deepcopy=False)[source]

Convenience method to return a copy of this dataset.

Parameters:deepcopy (bool, optional) – deepcopy() this dataset if True, shallow copy otherwise.
Returns:A new dataset with fields copied from this object and metadata set accordingly.
Return type:Dataset
export_dataset()[source]

Save this Dataset to disk.

split(num_or_size_splits, shuffle=False)[source]

Split this dataset into multiple partitions.

Parameters:
  • num_or_size_splits (array or int) – If num_or_size_splits is an int, k, the value is the number of equal-sized folds to make (if k does not evenly divide the dataset these folds are approximately equal-sized). If num_or_size_splits is an array of type int, the values are taken as the indices at which to split the dataset. If the values are floats (< 1.), they are considered to be fractional proportions of the dataset at which to split.
  • shuffle (bool, optional) – Randomly shuffle the dataset before splitting.
Returns:

Splits. Contains k or len(num_or_size_splits) + 1 datasets depending on num_or_size_splits.
Return type:

list(Dataset)
validate_dataset()[source]

Error checking and type validation.

Structured Dataset

class aif360.datasets.StructuredDataset(df, label_names, protected_attribute_names, instance_weights_name=None, scores_names=[], unprivileged_protected_attributes=[], privileged_protected_attributes=[], metadata=None)[source]

Base class for all structured datasets.

A StructuredDataset requires data to be stored in numpy.ndarray objects with dtype as float64.

features

Dataset features for each instance.

Type:numpy.ndarray
labels

Generic label corresponding to each instance (could be ground-truth, predicted, cluster assignments, etc.).

Type:numpy.ndarray
scores

Probability score associated with each label. Same shape as labels. Only valid for binary labels (this includes one-hot categorical labels as well).

Type:numpy.ndarray
protected_attributes

A subset of features for which fairness is desired.

Type:numpy.ndarray
feature_names

Names describing each dataset feature.

Type:list(str)
label_names

Names describing each label.

Type:list(str)
protected_attribute_names

A subset of feature_names corresponding to protected_attributes.

Type:list(str)
privileged_protected_attributes

A subset of protected attribute values which are considered privileged from a fairness perspective.

Type:list(numpy.ndarray)
unprivileged_protected_attributes

The remaining possible protected attribute values which are not included in privileged_protected_attributes.

Type:list(numpy.ndarray)
instance_names

Indentifiers for each instance. Sequential integers by default.

Type:list(str)
instance_weights

Weighting for each instance. All equal (ones) by default. Pursuant to standard practice in social science data, 1 means one person or entity. These weights are hence person or entity multipliers (see: https://www.ibm.com/support/knowledgecenter/en/SS3RA7_15.0.0/com.ibm.spss.modeler.help/netezza_decisiontrees_weights.htm) These weights may not be normalized to sum to 1 across the entire dataset, rather the nominal (default) weight of each entity/record in the data is 1. This is similar in spirit to the person weight in census microdata samples. https://www.census.gov/programs-surveys/acs/technical-documentation/pums/about.html

Type:numpy.ndarray
ignore_fields

Attribute names to ignore when doing equality comparisons. Always at least contains ‘metadata’.

Type:set(str)
metadata

Details about the creation of this dataset. For example:

{
    'transformer': 'Dataset.__init__',
    'params': kwargs,
    'previous': None
}
Type:dict
Parameters:
  • df (pandas.DataFrame) – Input DataFrame with features, labels, and protected attributes. Values should be preprocessed to remove NAs and make all data numerical. Index values are taken as instance names.
  • label_names (iterable) – Names of the label columns in df.
  • protected_attribute_names (iterable) – List of names corresponding to protected attribute columns in df.
  • instance_weights_name (optional) – Column name in df corresponding to instance weights. If not provided, instance_weights will be all set to 1.
  • unprivileged_protected_attributes (optional) – If not provided, all but the highest numerical value of each protected attribute will be considered not privileged.
  • privileged_protected_attributes (optional) – If not provided, the highest numerical value of each protected attribute will be considered privileged.
  • metadata (optional) – Additional metadata to append.
Raises:
  • TypeError – Certain fields must be np.ndarrays as specified in the class description.
  • ValueError – ndarray shapes must match.
align_datasets(other)[source]

Align the other dataset features, labels and protected_attributes to this dataset.

Parameters:other (StructuredDataset) – Other dataset that needs to be aligned
Returns:New aligned dataset
Return type:StructuredDataset
convert_to_dataframe(de_dummy_code=False, sep='=', set_category=True)[source]

Convert the StructuredDataset to a pandas.DataFrame.

Parameters:
  • de_dummy_code (bool) – Performs de_dummy_coding, converting dummy- coded columns to categories. If de_dummy_code is True and this dataset contains mappings for label and/or protected attribute values to strings in the metadata, this method will convert those as well.
  • set_category (bool) – Set the de-dummy coded features to categorical type.
Returns:

  • pandas.DataFrame: Equivalent dataframe for a dataset. All columns will have only numeric values. The protected_attributes field in the dataset will override the values in the features field.
  • dict: Attributes. Will contain additional information pulled from the dataset such as feature_names, label_names, protected_attribute_names, instance_names, instance_weights, privileged_protected_attributes, unprivileged_protected_attributes. The metadata will not be returned.
Return type:

(pandas.DataFrame, dict)
export_dataset(export_metadata=False)[source]

Export the dataset and supporting attributes TODO: The preferred file format is HDF

import_dataset(import_metadata=False)[source]

Import the dataset and supporting attributes TODO: The preferred file format is HDF

split(num_or_size_splits, shuffle=False, seed=None)[source]

Split the dataset into multiple datasets :param num_or_size_splits: :type num_or_size_splits: list or int :param shuffle: :type shuffle: bool :param seed: takes the same argument as numpy.random.seed() :type seed: int or array_like :param function:

Returns:Each element of this list is a dataset obtained during the split
Return type:list
temporarily_ignore(*fields)[source]

Temporarily add the fields provided to ignore_fields.

To be used in a with statement. Upon completing the with block, ignore_fields is restored to its original value.

Parameters:*fields – Additional fields to ignore for equality comparison within the scope of this context manager, e.g. temporarily_ignore(‘features’, ‘labels’). The temporary ignore_fields attribute is the union of the old attribute and the set of these fields.

Examples

>>> sd = StructuredDataset(...)
>>> modified = sd.copy()
>>> modified.labels = sd.labels + 1
>>> assert sd != modified
>>> with sd.temporarily_ignore('labels'):
>>>     assert sd == modified
>>> assert 'labels' not in sd.ignore_fields
validate_dataset()[source]

Error checking and type validation.

Raises:
  • TypeError – Certain fields must be np.ndarrays as specified in the class description.
  • ValueError – ndarray shapes must match.

Binary Label Dataset

class aif360.datasets.BinaryLabelDataset(favorable_label=1.0, unfavorable_label=0.0, **kwargs)[source]

Base class for all structured datasets with binary labels.

Parameters:
  • favorable_label (float) – Label value which is considered favorable (i.e. “positive”).
  • unfavorable_label (float) – Label value which is considered unfavorable (i.e. “negative”).
  • **kwargs – StructuredDataset arguments.
validate_dataset()[source]

Error checking and type validation.

Raises:
  • ValueError – labels must be shape [n, 1].
  • ValueError – favorable_label and unfavorable_label must be the only values present in labels.

Standard Datasets

Base Class

class aif360.datasets.StandardDataset(df, label_name, favorable_classes, protected_attribute_names, privileged_classes, instance_weights_name='', scores_name='', categorical_features=[], features_to_keep=[], features_to_drop=[], na_values=[], custom_preprocessing=None, metadata=None)[source]

Base class for every BinaryLabelDataset provided out of the box by aif360.

It is not strictly necessary to inherit this class when adding custom datasets but it may be useful.

This class is very loosely based on code from https://github.com/algofairness/fairness-comparison.

Subclasses of StandardDataset should perform the following before calling super().__init__:

  1. Load the dataframe from a raw file.

Then, this class will go through a standard preprocessing routine which:

  1. (optional) Performs some dataset-specific preprocessing (e.g. renaming columns/values, handling missing data).
  2. Drops unrequested columns (see features_to_keep and features_to_drop for details).
  3. Drops rows with NA values.
  4. Creates a one-hot encoding of the categorical variables.
  5. Maps protected attributes to binary privileged/unprivileged values (1/0).
  6. Maps labels to binary favorable/unfavorable labels (1/0).
Parameters:
  • df (pandas.DataFrame) – DataFrame on which to perform standard processing.
  • label_name – Name of the label column in df.
  • favorable_classes (list or function) – Label values which are considered favorable or a boolean function which returns True if favorable. All others are unfavorable. Label values are mapped to 1 (favorable) and 0 (unfavorable) if they are not already binary and numerical.
  • protected_attribute_names (list) – List of names corresponding to protected attribute columns in df.
  • privileged_classes (list(list or function)) – Each element is a list of values which are considered privileged or a boolean function which return True if privileged for the corresponding column in protected_attribute_names. All others are unprivileged. Values are mapped to 1 (privileged) and 0 (unprivileged) if they are not already numerical.
  • instance_weights_name (optional) – Name of the instance weights column in df.
  • categorical_features (optional, list) – List of column names in the DataFrame which are to be expanded into one-hot vectors.
  • features_to_keep (optional, list) – Column names to keep. All others are dropped except those present in protected_attribute_names, categorical_features, label_name or instance_weights_name. Defaults to all columns if not provided.
  • features_to_drop (optional, list) – Column names to drop. Note: this overrides features_to_keep.
  • na_values (optional) – Additional strings to recognize as NA. See pandas.read_csv() for details.
  • custom_preprocessing (function) – A function object which acts on and returns a DataFrame (f: DataFrame -> DataFrame). If None, no extra preprocessing is applied.
  • metadata (optional) – Additional metadata to append.

Adult Dataset

class aif360.datasets.AdultDataset(label_name='income-per-year', favorable_classes=['>50K', '>50K.'], protected_attribute_names=['race', 'sex'], privileged_classes=[['White'], ['Male']], instance_weights_name=None, categorical_features=['workclass', 'education', 'marital-status', 'occupation', 'relationship', 'native-country'], features_to_keep=[], features_to_drop=['fnlwgt'], na_values=['?'], custom_preprocessing=None, metadata={'label_maps': [{1.0: '>50K', 0.0: '<=50K'}], 'protected_attribute_maps': [{1.0: 'White', 0.0: 'Non-white'}, {1.0: 'Male', 0.0: 'Female'}]})[source]

Adult Census Income Dataset.

See aif360/data/raw/adult/README.md.

See StandardDataset for a description of the arguments.

Examples

The following will instantiate a dataset which uses the fnlwgt feature:

>>> from aif360.datasets import AdultDataset
>>> ad = AdultDataset(instance_weights_name='fnlwgt',
... features_to_drop=[])
WARNING:root:Missing Data: 3620 rows removed from dataset.
>>> not np.all(ad.instance_weights == 1.)
True

To instantiate a dataset which utilizes only numerical features and a single protected attribute, run:

>>> single_protected = ['sex']
>>> single_privileged = [['Male']]
>>> ad = AdultDataset(protected_attribute_names=single_protected,
... privileged_classes=single_privileged,
... categorical_features=[],
... features_to_keep=['age', 'education-num'])
>>> print(ad.feature_names)
['education-num', 'age', 'sex']
>>> print(ad.label_names)
['income-per-year']

Note: the protected_attribute_names and label_name are kept even if they are not explicitly given in features_to_keep.

In some cases, it may be useful to keep track of a mapping from float -> str for protected attributes and/or labels. If our use case differs from the default, we can modify the mapping stored in metadata:

>>> label_map = {1.0: '>50K', 0.0: '<=50K'}
>>> protected_attribute_maps = [{1.0: 'Male', 0.0: 'Female'}]
>>> ad = AdultDataset(protected_attribute_names=['sex'],
... privileged_classes=[['Male']], metadata={'label_map': label_map,
... 'protected_attribute_maps': protected_attribute_maps})

Now this information will stay attached to the dataset and can be used for more descriptive visualizations.

Bank Dataset

class aif360.datasets.BankDataset(label_name='y', favorable_classes=['yes'], protected_attribute_names=['age'], privileged_classes=[<function BankDataset.<lambda>>], instance_weights_name=None, categorical_features=['job', 'marital', 'education', 'default', 'housing', 'loan', 'contact', 'month', 'day_of_week', 'poutcome'], features_to_keep=[], features_to_drop=[], na_values=['unknown'], custom_preprocessing=None, metadata=None)[source]

Bank marketing Dataset.

See aif360/data/raw/bank/README.md.

See StandardDataset for a description of the arguments.

By default, this code converts the ‘age’ attribute to a binary value where privileged is age >= 25 and unprivileged is age < 25 as in GermanDataset.

Compas Dataset

class aif360.datasets.CompasDataset(label_name='two_year_recid', favorable_classes=[0], protected_attribute_names=['sex', 'race'], privileged_classes=[['Female'], ['Caucasian']], instance_weights_name=None, categorical_features=['age_cat', 'c_charge_degree', 'c_charge_desc'], features_to_keep=['sex', 'age', 'age_cat', 'race', 'juv_fel_count', 'juv_misd_count', 'juv_other_count', 'priors_count', 'c_charge_degree', 'c_charge_desc', 'two_year_recid'], features_to_drop=[], na_values=[], custom_preprocessing=<function default_preprocessing>, metadata={'label_maps': [{1.0: 'Did recid.', 0.0: 'No recid.'}], 'protected_attribute_maps': [{0.0: 'Male', 1.0: 'Female'}, {1.0: 'Caucasian', 0.0: 'Not Caucasian'}]})[source]

ProPublica COMPAS Dataset.

See aif360/data/raw/compas/README.md.

See StandardDataset for a description of the arguments.

Note: The label value 0 in this case is considered favorable (no recidivism).

Examples

In some cases, it may be useful to keep track of a mapping from float -> str for protected attributes and/or labels. If our use case differs from the default, we can modify the mapping stored in metadata:

>>> label_map = {1.0: 'Did recid.', 0.0: 'No recid.'}
>>> protected_attribute_maps = [{1.0: 'Male', 0.0: 'Female'}]
>>> cd = CompasDataset(protected_attribute_names=['sex'],
... privileged_classes=[['Male']], metadata={'label_map': label_map,
... 'protected_attribute_maps': protected_attribute_maps})

Now this information will stay attached to the dataset and can be used for more descriptive visualizations.

German Dataset

class aif360.datasets.GermanDataset(label_name='credit', favorable_classes=[1], protected_attribute_names=['sex', 'age'], privileged_classes=[['male'], <function GermanDataset.<lambda>>], instance_weights_name=None, categorical_features=['status', 'credit_history', 'purpose', 'savings', 'employment', 'other_debtors', 'property', 'installment_plans', 'housing', 'skill_level', 'telephone', 'foreign_worker'], features_to_keep=[], features_to_drop=['personal_status'], na_values=[], custom_preprocessing=<function default_preprocessing>, metadata={'label_maps': [{1.0: 'Good Credit', 2.0: 'Bad Credit'}], 'protected_attribute_maps': [{1.0: 'Male', 0.0: 'Female'}, {1.0: 'Old', 0.0: 'Young'}]})[source]

German credit Dataset.

See aif360/data/raw/german/README.md.

See StandardDataset for a description of the arguments.

By default, this code converts the ‘age’ attribute to a binary value where privileged is age >= 25 and unprivileged is age < 25 as proposed by Kamiran and Calders [1].

References

[1]F. Kamiran and T. Calders, “Classifying without discriminating,” 2nd International Conference on Computer, Control and Communication, 2009.

Examples

In some cases, it may be useful to keep track of a mapping from float -> str for protected attributes and/or labels. If our use case differs from the default, we can modify the mapping stored in metadata:

>>> label_map = {1.0: 'Good Credit', 0.0: 'Bad Credit'}
>>> protected_attribute_maps = [{1.0: 'Male', 0.0: 'Female'}]
>>> gd = GermanDataset(protected_attribute_names=['sex'],
... privileged_classes=[['male']], metadata={'label_map': label_map,
... 'protected_attribute_maps': protected_attribute_maps})

Now this information will stay attached to the dataset and can be used for more descriptive visualizations.

aif360.explainers

Metric Text Explainer

class aif360.explainers.MetricTextExplainer(metric)[source]

Class for explaining metric values with text.

These briefly explain what a metric is and/or how it is calculated unless it is obvious (e.g. accuracy) and print the value.

This class contains text explanations for all metric values regardless of which subclass they appear in. This will raise an error if the metric does not apply (e.g. calling true_positive_rate if type(metric) == DatasetMetric).

Initialize a MetricExplainer object.

Parameters:metric (Metric) – The metric to be explained.

Metric JSON Explainer

class aif360.explainers.MetricJSONExplainer(metric)[source]

Class for explaining metric values in JSON format.

These briefly explain what a metric is and/or how it is calculated unless it is obvious (e.g. accuracy) and print the value.

This class contains JSON explanations for all metric values regardless of which subclass they appear in. This will raise an error if the metric does not apply (e.g. calling true_positive_rate if type(metric) == DatasetMetric).

Initialize a MetricExplainer object.

Parameters:metric (Metric) – The metric to be explained.

aif360.metrics

Dataset Metric

class aif360.metrics.DatasetMetric(dataset, unprivileged_groups=None, privileged_groups=None)[source]

Class for computing metrics based on one StructuredDataset.

Parameters:
  • dataset (StructuredDataset) – A StructuredDataset.
  • privileged_groups (list(dict)) – Privileged groups. Format is a list of dicts where the keys are protected_attribute_names and the values are values in protected_attributes. Each dict element describes a single group. See examples for more details.
  • unprivileged_groups (list(dict)) – Unprivileged groups in the same format as privileged_groups.
Raises:

Examples

>>> from aif360.datasets import GermanDataset
>>> german = GermanDataset()
>>> u = [{'sex': 1, 'age': 1}, {'sex': 0}]
>>> p = [{'sex': 1, 'age': 0}]
>>> dm = DatasetMetric(german, unprivileged_groups=u, privileged_groups=p)
num_instances(privileged=None)[source]

Compute the number of instances, \(n\), in the dataset conditioned on protected attributes if necessary.

Parameters:privileged (bool, optional) – Boolean prescribing whether to condition this metric on the privileged_groups, if True, or the unprivileged_groups, if False. Defaults to None meaning this metric is computed over the entire dataset.
Raises:AttributeError – privileged_groups or unprivileged_groups must be must be provided at initialization to condition on them.

Binary Label Dataset Metric

class aif360.metrics.BinaryLabelDatasetMetric(dataset, unprivileged_groups=None, privileged_groups=None)[source]

Class for computing metrics based on a single BinaryLabelDataset.

Parameters:
  • dataset (BinaryLabelDataset) – A BinaryLabelDataset.
  • privileged_groups (list(dict)) – Privileged groups. Format is a list of dicts where the keys are protected_attribute_names and the values are values in protected_attributes. Each dict element describes a single group. See examples for more details.
  • unprivileged_groups (list(dict)) – Unprivileged groups in the same format as privileged_groups.
Raises:

TypeError – dataset must be a BinaryLabelDataset type.
base_rate(privileged=None)[source]

Compute the base rate, \(Pr(Y = 1) = P/(P+N)\), optionally conditioned on protected attributes.

Parameters:privileged (bool, optional) – Boolean prescribing whether to condition this metric on the privileged_groups, if True, or the unprivileged_groups, if False. Defaults to None meaning this metric is computed over the entire dataset.
Returns:Base rate (optionally conditioned).
Return type:float
consistency(n_neighbors=5)[source]

Individual fairness metric from [1] that measures how similar the labels are for similar instances.

\[1 - \frac{1}{n\cdot\text{n_neighbors}}\sum_{i=1}^n |\hat{y}_i - \sum_{j\in\mathcal{N}_{\text{n_neighbors}}(x_i)} \hat{y}_j|\]
Parameters:n_neighbors (int, optional) – Number of neighbors for the knn computation.

References

[1]R. Zemel, Y. Wu, K. Swersky, T. Pitassi, and C. Dwork, “Learning Fair Representations,” International Conference on Machine Learning, 2013.
disparate_impact()[source]
\[\frac{Pr(Y = 1 | D = \text{unprivileged})} {Pr(Y = 1 | D = \text{privileged})}\]
mean_difference()[source]

Alias of statistical_parity_difference().

num_negatives(privileged=None)[source]

Compute the number of negatives, \(N = \sum_{i=1}^n \mathbb{1}[y_i = 0]\), optionally conditioned on protected attributes.

Parameters:privileged (bool, optional) – Boolean prescribing whether to condition this metric on the privileged_groups, if True, or the unprivileged_groups, if False. Defaults to None meaning this metric is computed over the entire dataset.
Raises:AttributeError – privileged_groups or unprivileged_groups must be must be provided at initialization to condition on them.
num_positives(privileged=None)[source]

Compute the number of positives, \(P = \sum_{i=1}^n \mathbb{1}[y_i = 1]\), optionally conditioned on protected attributes.

Parameters:privileged (bool, optional) – Boolean prescribing whether to condition this metric on the privileged_groups, if True, or the unprivileged_groups, if False. Defaults to None meaning this metric is computed over the entire dataset.
Raises:AttributeError – privileged_groups or unprivileged_groups must be must be provided at initialization to condition on them.
statistical_parity_difference()[source]
\[Pr(Y = 1 | D = \text{unprivileged}) - Pr(Y = 1 | D = \text{privileged})\]

Classification Metric

class aif360.metrics.ClassificationMetric(dataset, classified_dataset, unprivileged_groups=None, privileged_groups=None)[source]

Class for computing metrics based on two BinaryLabelDatasets.

The first dataset is the original one and the second is the output of the classification transformer (or similar).

Parameters:
  • dataset (BinaryLabelDataset) – Dataset containing ground-truth labels.
  • classified_dataset (BinaryLabelDataset) – Dataset containing predictions.
  • privileged_groups (list(dict)) – Privileged groups. Format is a list of dicts where the keys are protected_attribute_names and the values are values in protected_attributes. Each dict element describes a single group. See examples for more details.
  • unprivileged_groups (list(dict)) – Unprivileged groups in the same format as privileged_groups.
Raises:

TypeError – dataset and classified_dataset must be BinaryLabelDataset types.
_between_group_generalized_entropy_index(groups, alpha=2)[source]

Between-group generalized entropy index is proposed as a group fairness measure in [2] and is one of two terms that the generalized entropy index decomposes to.

Parameters:
  • groups (list) – A list of groups over which to calculate this metric. Groups should be disjoint. By default, this will use the privileged_groups and unprivileged_groups as the only two groups.
  • alpha (int) – See generalized_entropy_index().

References

[2]T. Speicher, H. Heidari, N. Grgic-Hlaca, K. P. Gummadi, A. Singla, A. Weller, and M. B. Zafar, “A Unified Approach to Quantifying Algorithmic Unfairness: Measuring Individual and Group Unfairness via Inequality Indices,” ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2018.
accuracy(privileged=None)[source]

\(ACC = (TP + TN)/(P + N)\).

Parameters:privileged (bool, optional) – Boolean prescribing whether to condition this metric on the privileged_groups, if True, or the unprivileged_groups, if False. Defaults to None meaning this metric is computed over the entire dataset.
Raises:AttributeError – privileged_groups or unprivileged_groups must be provided at initialization to condition on them.
average_abs_odds_difference()[source]

Average of absolute difference in FPR and TPR for unprivileged and privileged groups:

\[\tfrac{1}{2}\left[|FPR_{D = \text{unprivileged}} - FPR_{D = \text{privileged}}| + |TPR_{D = \text{unprivileged}} - TPR_{D = \text{privileged}}|\right]\]

A value of 0 indicates equality of odds.

average_odds_difference()[source]

Average of difference in FPR and TPR for unprivileged and privileged groups:

\[\tfrac{1}{2}\left[(FPR_{D = \text{unprivileged}} - FPR_{D = \text{privileged}}) + (TPR_{D = \text{unprivileged}} - TPR_{D = \text{privileged}}))\right]\]

A value of 0 indicates equality of odds.

between_all_groups_coefficient_of_variation()[source]

The between-group coefficient of variation is two times the square root of the between_all_groups_generalized_entropy_index() with \(\alpha = 2\).

between_all_groups_generalized_entropy_index(alpha=2)[source]

Between-group generalized entropy index that uses all combinations of groups based on self.dataset.protected_attributes. See _between_group_generalized_entropy_index().

Parameters:alpha (int) – See generalized_entropy_index().
between_all_groups_theil_index()[source]

The between-group Theil index is the between_all_groups_generalized_entropy_index() with \(\alpha = 1\).

between_group_coefficient_of_variation()[source]

The between-group coefficient of variation is two times the square root of the between_group_generalized_entropy_index() with \(\alpha = 2\).

between_group_generalized_entropy_index(alpha=2)[source]

Between-group generalized entropy index that uses self.privileged_groups and self.unprivileged_groups as the only two groups. See _between_group_generalized_entropy_index().

Parameters:alpha (int) – See generalized_entropy_index().
between_group_theil_index()[source]

The between-group Theil index is the between_group_generalized_entropy_index() with \(\alpha = 1\).

binary_confusion_matrix(privileged=None)[source]

Compute the number of true/false positives/negatives, optionally conditioned on protected attributes.

Parameters:privileged (bool, optional) – Boolean prescribing whether to condition this metric on the privileged_groups, if True, or the unprivileged_groups, if False. Defaults to None meaning this metric is computed over the entire dataset.
Returns:Number of true positives, false positives, true negatives, false negatives (optionally conditioned).
Return type:dict
coefficient_of_variation()[source]

The coefficient of variation is two times the square root of the generalized_entropy_index() with \(\alpha = 2\).

disparate_impact()[source]
\[\frac{Pr(\hat{Y} = 1 | D = \text{unprivileged})} {Pr(\hat{Y} = 1 | D = \text{privileged})}\]
equal_opportunity_difference()[source]

Alias of true_positive_rate_difference().

error_rate(privileged=None)[source]

\(ERR = (FP + FN)/(P + N)\)

Parameters:privileged (bool, optional) – Boolean prescribing whether to condition this metric on the privileged_groups, if True, or the unprivileged_groups, if False. Defaults to None meaning this metric is computed over the entire dataset.
Raises:AttributeError – privileged_groups or unprivileged_groups must be provided at initialization to condition on them.
error_rate_difference()[source]

Difference in error rates for unprivileged and privileged groups, \(ERR_{D = \text{unprivileged}} - ERR_{D = \text{privileged}}\).

error_rate_ratio()[source]

Ratio of error rates for unprivileged and privileged groups, \(\frac{ERR_{D = \text{unprivileged}}}{ERR_{D = \text{privileged}}}\).

false_discovery_rate(privileged=None)[source]

\(FDR = FP/(TP + FP)\)

Parameters:privileged (bool, optional) – Boolean prescribing whether to condition this metric on the privileged_groups, if True, or the unprivileged_groups, if False. Defaults to None meaning this metric is computed over the entire dataset.
Raises:AttributeError – privileged_groups or unprivileged_groups must be provided at initialization to condition on them.
false_discovery_rate_difference()[source]

\(FDR_{D = \text{unprivileged}} - FDR_{D = \text{privileged}}\)

false_discovery_rate_ratio()[source]

\(\frac{FDR_{D = \text{unprivileged}}}{FDR_{D = \text{privileged}}}\)

false_negative_rate(privileged=None)[source]

\(FNR = FN/P\)

Parameters:privileged (bool, optional) – Boolean prescribing whether to condition this metric on the privileged_groups, if True, or the unprivileged_groups, if False. Defaults to None meaning this metric is computed over the entire dataset.
Raises:AttributeError – privileged_groups or unprivileged_groups must be provided at initialization to condition on them.
false_negative_rate_difference()[source]

\(FNR_{D = \text{unprivileged}} - FNR_{D = \text{privileged}}\)

false_negative_rate_ratio()[source]

\(\frac{FNR_{D = \text{unprivileged}}}{FNR_{D = \text{privileged}}}\)

false_omission_rate(privileged=None)[source]

\(FOR = FN/(TN + FN)\)

Parameters:privileged (bool, optional) – Boolean prescribing whether to condition this metric on the privileged_groups, if True, or the unprivileged_groups, if False. Defaults to None meaning this metric is computed over the entire dataset.
Raises:AttributeError – privileged_groups or unprivileged_groups must be provided at initialization to condition on them.
false_omission_rate_difference()[source]

\(FOR_{D = \text{unprivileged}} - FOR_{D = \text{privileged}}\)

false_omission_rate_ratio()[source]

\(\frac{FOR_{D = \text{unprivileged}}}{FOR_{D = \text{privileged}}}\)

false_positive_rate(privileged=None)[source]

\(FPR = FP/N\)

Parameters:privileged (bool, optional) – Boolean prescribing whether to condition this metric on the privileged_groups, if True, or the unprivileged_groups, if False. Defaults to None meaning this metric is computed over the entire dataset.
Raises:AttributeError – privileged_groups or unprivileged_groups must be provided at initialization to condition on them.
false_positive_rate_difference()[source]

\(FPR_{D = \text{unprivileged}} - FPR_{D = \text{privileged}}\)

false_positive_rate_ratio()[source]

\(\frac{FPR_{D = \text{unprivileged}}}{FPR_{D = \text{privileged}}}\)

generalized_binary_confusion_matrix(privileged=None)[source]

Compute the number of generalized true/false positives/negatives, optionally conditioned on protected attributes. Generalized counts are based on scores and not on the hard predictions.

Parameters:privileged (bool, optional) – Boolean prescribing whether to condition this metric on the privileged_groups, if True, or the unprivileged_groups, if False. Defaults to None meaning this metric is computed over the entire dataset.
Returns:Number of generalized true positives, generalized false positives, generalized true negatives, generalized false negatives (optionally conditioned).
Return type:dict
generalized_entropy_index(alpha=2)[source]

Generalized entropy index is proposed as a unified individual and group fairness measure in [3]. With \(b_i = \hat{y}_i - y_i + 1\):

\[\begin{split}\mathcal{E}(\alpha) = \begin{cases} \frac{1}{n \alpha (\alpha-1)}\sum_{i=1}^n\left[\left(\frac{b_i}{\mu}\right)^\alpha - 1\right],& \alpha \ne 0, 1,\\ \frac{1}{n}\sum_{i=1}^n\frac{b_{i}}{\mu}\ln\frac{b_{i}}{\mu},& \alpha=1,\\ -\frac{1}{n}\sum_{i=1}^n\ln\frac{b_{i}}{\mu},& \alpha=0. \end{cases}\end{split}\]
Parameters:alpha (int) – Parameter that regulates the weight given to distances between values at different parts of the distribution.

References

[3]T. Speicher, H. Heidari, N. Grgic-Hlaca, K. P. Gummadi, A. Singla, A. Weller, and M. B. Zafar, “A Unified Approach to Quantifying Algorithmic Unfairness: Measuring Individual and Group Unfairness via Inequality Indices,” ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2018.
generalized_false_negative_rate(privileged=None)[source]

\(GFNR = GFN/P\)

Parameters:privileged (bool, optional) – Boolean prescribing whether to condition this metric on the privileged_groups, if True, or the unprivileged_groups, if False. Defaults to None meaning this metric is computed over the entire dataset.
Raises:AttributeError – privileged_groups or unprivileged_groups must be provided at initialization to condition on them.
generalized_false_positive_rate(privileged=None)[source]

\(GFPR = GFP/N\)

Parameters:privileged (bool, optional) – Boolean prescribing whether to condition this metric on the privileged_groups, if True, or the unprivileged_groups, if False. Defaults to None meaning this metric is computed over the entire dataset.
Raises:AttributeError – privileged_groups or unprivileged_groups must be provided at initialization to condition on them.
generalized_true_negative_rate(privileged=None)[source]

\(GTNR = GTN/N\)

Parameters:privileged (bool, optional) – Boolean prescribing whether to condition this metric on the privileged_groups, if True, or the unprivileged_groups, if False. Defaults to None meaning this metric is computed over the entire dataset.
Raises:AttributeError – privileged_groups or unprivileged_groups must be provided at initialization to condition on them.
generalized_true_positive_rate(privileged=None)[source]

Return the ratio of generalized true positives to positive examples in the dataset, \(GTPR = GTP/P\), optionally conditioned on protected attributes.

Parameters:privileged (bool, optional) – Boolean prescribing whether to condition this metric on the privileged_groups, if True, or the unprivileged_groups, if False. Defaults to None meaning this metric is computed over the entire dataset.
Raises:AttributeError – privileged_groups or unprivileged_groups must be provided at initialization to condition on them.
negative_predictive_value(privileged=None)[source]

\(NPV = TN/(TN + FN)\)

Parameters:privileged (bool, optional) – Boolean prescribing whether to condition this metric on the privileged_groups, if True, or the unprivileged_groups, if False. Defaults to None meaning this metric is computed over the entire dataset.
Raises:AttributeError – privileged_groups or unprivileged_groups must be provided at initialization to condition on them.
num_false_negatives(privileged=None)[source]

\(FN = \sum_{i=1}^n \mathbb{1}[y_i = \text{favorable}]\mathbb{1}[\hat{y}_i = \text{unfavorable}]\)

Parameters:privileged (bool, optional) – Boolean prescribing whether to condition this metric on the privileged_groups, if True, or the unprivileged_groups, if False. Defaults to None meaning this metric is computed over the entire dataset.
Raises:AttributeError – privileged_groups or unprivileged_groups must be provided at initialization to condition on them.
num_false_positives(privileged=None)[source]

\(FP = \sum_{i=1}^n \mathbb{1}[y_i = \text{unfavorable}]\mathbb{1}[\hat{y}_i = \text{favorable}]\)

Parameters:privileged (bool, optional) – Boolean prescribing whether to condition this metric on the privileged_groups, if True, or the unprivileged_groups, if False. Defaults to None meaning this metric is computed over the entire dataset.
Raises:AttributeError – privileged_groups or unprivileged_groups must be provided at initialization to condition on them.
num_generalized_false_negatives(privileged=None)[source]

Return the generalized number of false negatives, \(GFN\), the weighted sum of predicted scores where true labels are ‘favorable’, optionally conditioned on protected attributes.

Parameters:privileged (bool, optional) – Boolean prescribing whether to condition this metric on the privileged_groups, if True, or the unprivileged_groups, if False. Defaults to None meaning this metric is computed over the entire dataset.
Raises:AttributeError – privileged_groups or unprivileged_groups must be provided at initialization to condition on them.
num_generalized_false_positives(privileged=None)[source]

Return the generalized number of false positives, \(GFP\), the weighted sum of predicted scores where true labels are ‘favorable’, optionally conditioned on protected attributes.

Parameters:privileged (bool, optional) – Boolean prescribing whether to condition this metric on the privileged_groups, if True, or the unprivileged_groups, if False. Defaults to None meaning this metric is computed over the entire dataset.
Raises:AttributeError – privileged_groups or unprivileged_groups must be must be provided at initialization to condition on them.
num_generalized_true_negatives(privileged=None)[source]

Return the generalized number of true negatives, \(GTN\), the weighted sum of predicted scores where true labels are ‘favorable’, optionally conditioned on protected attributes.

Parameters:privileged (bool, optional) – Boolean prescribing whether to condition this metric on the privileged_groups, if True, or the unprivileged_groups, if False. Defaults to None meaning this metric is computed over the entire dataset.
Raises:AttributeError – privileged_groups or unprivileged_groups must be provided at initialization to condition on them.
num_generalized_true_positives(privileged=None)[source]

Return the generalized number of true positives, \(GTP\), the weighted sum of predicted scores where true labels are ‘favorable’, optionally conditioned on protected attributes.

Parameters:privileged (bool, optional) – Boolean prescribing whether to condition this metric on the privileged_groups, if True, or the unprivileged_groups, if False. Defaults to None meaning this metric is computed over the entire dataset.
Raises:AttributeError – privileged_groups or unprivileged_groups must be provided at initialization to condition on them.
num_pred_negatives(privileged=None)[source]

\(\sum_{i=1}^n \mathbb{1}[\hat{y}_i = \text{unfavorable}]\)

Parameters:privileged (bool, optional) – Boolean prescribing whether to condition this metric on the privileged_groups, if True, or the unprivileged_groups, if False. Defaults to None meaning this metric is computed over the entire dataset.
Raises:AttributeError – privileged_groups or unprivileged_groups must be provided at initialization to condition on them.
num_pred_positives(privileged=None)[source]

\(\sum_{i=1}^n \mathbb{1}[\hat{y}_i = \text{favorable}]\)

Parameters:privileged (bool, optional) – Boolean prescribing whether to condition this metric on the privileged_groups, if True, or the unprivileged_groups, if False. Defaults to None meaning this metric is computed over the entire dataset.
Raises:AttributeError – privileged_groups or unprivileged_groups must be provided at initialization to condition on them.
num_true_negatives(privileged=None)[source]

\(TN = \sum_{i=1}^n \mathbb{1}[y_i = \text{unfavorable}]\mathbb{1}[\hat{y}_i = \text{unfavorable}]\)

Parameters:privileged (bool, optional) – Boolean prescribing whether to condition this metric on the privileged_groups, if True, or the unprivileged_groups, if False. Defaults to None meaning this metric is computed over the entire dataset.
Raises:AttributeError – privileged_groups or unprivileged_groups must be provided at initialization to condition on them.
num_true_positives(privileged=None)[source]

Return the number of instances in the dataset where both the predicted and true labels are ‘favorable’, \(TP = \sum_{i=1}^n \mathbb{1}[y_i = \text{favorable}]\mathbb{1}[\hat{y}_i = \text{favorable}]\), optionally conditioned on protected attributes.

Parameters:privileged (bool, optional) – Boolean prescribing whether to condition this metric on the privileged_groups, if True, or the unprivileged_groups, if False. Defaults to None meaning this metric is computed over the entire dataset.
Raises:AttributeError – privileged_groups or unprivileged_groups must be provided at initialization to condition on them.
performance_measures(privileged=None)[source]

Compute various performance measures on the dataset, optionally conditioned on protected attributes.

Parameters:privileged (bool, optional) – Boolean prescribing whether to condition this metric on the privileged_groups, if True, or the unprivileged_groups, if False. Defaults to None meaning this metric is computed over the entire dataset.
Returns:True positive rate, true negative rate, false positive rate, false negative rate, positive predictive value, negative predictive value, false discover rate, false omission rate, and accuracy (optionally conditioned).
Return type:dict
positive_predictive_value(privileged=None)[source]

\(PPV = TP/(TP + FP)\)

Parameters:privileged (bool, optional) – Boolean prescribing whether to condition this metric on the privileged_groups, if True, or the unprivileged_groups, if False. Defaults to None meaning this metric is computed over the entire dataset.
Raises:AttributeError – privileged_groups or unprivileged_groups must be provided at initialization to condition on them.
power(privileged=None)[source]

Alias of num_true_positives().

precision(privileged=None)[source]

Alias of positive_predictive_value().

recall(privileged=None)[source]

Alias of true_positive_rate().

selection_rate(privileged=None)[source]

\(Pr(\hat{Y} = \text{favorable})\)

Parameters:privileged (bool, optional) – Boolean prescribing whether to condition this metric on the privileged_groups, if True, or the unprivileged_groups, if False. Defaults to None meaning this metric is computed over the entire dataset.
Raises:AttributeError – privileged_groups or unprivileged_groups must be provided at initialization to condition on them.
sensitivity(privileged=None)[source]

Alias of true_positive_rate().

specificity(privileged=None)[source]

Alias of true_negative_rate().

statistical_parity_difference()[source]
\[Pr(\hat{Y} = 1 | D = \text{unprivileged}) - Pr(\hat{Y} = 1 | D = \text{privileged})\]
theil_index()[source]

The Theil index is the generalized_entropy_index() with \(\alpha = 1\).

true_negative_rate(privileged=None)[source]

\(TNR = TN/N\)

Parameters:privileged (bool, optional) – Boolean prescribing whether to condition this metric on the privileged_groups, if True, or the unprivileged_groups, if False. Defaults to None meaning this metric is computed over the entire dataset.
Raises:AttributeError – privileged_groups or unprivileged_groups must be provided at initialization to condition on them.
true_positive_rate(privileged=None)[source]

Return the ratio of true positives to positive examples in the dataset, \(TPR = TP/P\), optionally conditioned on protected attributes.

Parameters:privileged (bool, optional) – Boolean prescribing whether to condition this metric on the privileged_groups, if True, or the unprivileged_groups, if False. Defaults to None meaning this metric is computed over the entire dataset.
Raises:AttributeError – privileged_groups or unprivileged_groups must be provided at initialization to condition on them.
true_positive_rate_difference()[source]

\(TPR_{D = \text{unprivileged}} - TPR_{D = \text{privileged}}\)

Sample Distortion Metric

class aif360.metrics.SampleDistortionMetric(dataset, distorted_dataset, unprivileged_groups=None, privileged_groups=None)[source]

Class for computing metrics based on two StructuredDatasets.

Parameters:
  • dataset (StructuredDataset) – A StructuredDataset.
  • distorted_dataset (StructuredDataset) – A StructuredDataset.
  • privileged_groups (list(dict)) – Privileged groups. Format is a list of dicts where the keys are protected_attribute_names and the values are values in protected_attributes. Each dict element describes a single group. See examples for more details.
  • unprivileged_groups (list(dict)) – Unprivileged groups in the same format as privileged_groups.
Raises:

TypeError – dataset and distorted_dataset must be StructuredDataset types.
euclidean_distance(privileged=None, returned=False)[source]

Compute the average Euclidean distance between the samples from the two datasets.

mahalanobis_distance(privileged=None, returned=False)[source]

Compute the average Mahalanobis distance between the samples from the two datasets.

manhattan_distance(privileged=None, returned=False)[source]

Compute the average Manhattan distance between the samples from the two datasets.

mean_euclidean_distance_difference(privileged=None)[source]

Difference of the averages.

mean_euclidean_distance_ratio(privileged=None)[source]

Ratio of the averages.

mean_mahalanobis_distance_difference(privileged=None)[source]

Difference of the averages.

mean_mahalanobis_distance_ratio(privileged=None)[source]

Ratio of the averages.

mean_manhattan_distance_difference(privileged=None)[source]

Difference of the averages.

mean_manhattan_distance_ratio(privileged=None)[source]

Ratio of the averages.

Utility Functions

This is the helper script for implementing metrics.

aif360.metrics.utils.compute_boolean_conditioning_vector(X, feature_names, condition=None)[source]

Compute the boolean conditioning vector.

Parameters:
  • X (numpy.ndarray) – Dataset features
  • feature_names (list) – Names of the features.
  • condition (list(dict)) – Specifies the subset of instances we want to use. Format is a list of dicts where the keys are feature_names and the values are values in X. Elements in the list are clauses joined with OR operators while key-value pairs in each dict are joined with AND operators. See examples for more details. If None, the condition specifies the entire set of instances, X.
Returns:

Boolean conditioning vector. Shape is [n] where n is X.shape[0]. Values are True if the corresponding row satisfies the condition and False otherwise.
Return type:

numpy.ndarray(bool)

Examples

>>> condition = [{'sex': 1, 'age': 1}, {'sex': 0}]

This corresponds to (sex == 1 AND age == 1) OR (sex == 0).

aif360.metrics.utils.compute_distance(X_orig, X_distort, X_prot, feature_names, dist_fun, condition=None)[source]

Compute the distance element-wise for two sets of vectors.

Parameters:
Returns:

  • Element-wise distances (1-D).
  • Condition vector (1-D).
Return type:

(numpy.ndarray(numpy.float64), numpy.ndarray(bool))
aif360.metrics.utils.compute_num_TF_PN(X, y_true, y_pred, w, feature_names, favorable_label, unfavorable_label, condition=None)[source]

Compute the number of true/false positives/negatives optionally conditioned on protected attributes.

Parameters:
Returns:

Number of positives/negatives (optionally conditioned).
aif360.metrics.utils.compute_num_gen_TF_PN(X, y_true, y_score, w, feature_names, favorable_label, unfavorable_label, condition=None)[source]

Compute the number of generalized true/false positives/negatives optionally conditioned on protected attributes. Generalized counts are based on scores and not on the hard predictions.

Parameters:
  • X (numpy.ndarray) – Dataset features.
  • y_true (numpy.ndarray) – True label vector.
  • y_score (numpy.ndarray) – Predicted score vector. Values range from 0 to 1. 0 implies prediction for unfavorable label and 1 implies prediction for favorable label.
  • w (numpy.ndarray) – Instance weight vector - the true and predicted datasets are supposed to have same instance level weights.
  • feature_names (list) – names of the features.
  • favorable_label (float) – Value of favorable/positive label.
  • unfavorable_label (float) – Value of unfavorable/negative label.
  • condition (list(dict)) – Same format as compute_boolean_conditioning_vector().
Returns:

Number of positives/negatives (optionally conditioned).
aif360.metrics.utils.compute_num_instances(X, w, feature_names, condition=None)[source]

Compute the number of instances, \(n\), conditioned on the protected attribute(s).

Parameters:
Returns:

Number of instances (optionally conditioned).
Return type:

int
aif360.metrics.utils.compute_num_pos_neg(X, y, w, feature_names, label, condition=None)[source]

Compute the number of positives, \(P\), or negatives, \(N\), optionally conditioned on protected attributes.

Parameters:
Returns:

Number of positives/negatives (optionally conditioned)
Return type:

int

Indices and tables