In previous blog posts, we discussed the presence of AI for social good and AI ethics at NeurIPS 2018 and alluded to a growing set of AI ethics definitions, metrics, and tools from companies, organizations, research institutions, and governments. In this post, we dive more into these tools and when you might use them. A number of individuals helped with the curation of this list, including representatives from the Data for Democracy and Partnership on AI groups. Even with this collaborative effort, it is highly possible that a tool was missed, and if that is the case, please let us know and we will update the list!
The tools range in their complexity, technicality, and topic. In general, AI ethics includes topics of fairness, accountability, transparency, privacy, human rights, and security, among others. Regardless of topic, modality, or origin, most of the tools include additional links to other tools and papers and highlight that the tool is meant to aid and promote investigation and discussion, not necessarily to be a standalone solution. Below is a list of the various tools in approximate order of technicality.
Deon, developed by Driven Data, provides a command line tool that allows you to add an ethics checklist to projects. The checklist is customizable for each project and provides a list of examples to illustrate potentially harmful results. Deon covers most topics of AI ethics and spans the duration of a project. It is a bit lighter weight than the Ethics and Algorithms Toolkit, but covers a wider range of topics.
The Digital Impact Toolkit from the Digital Civil Society Lab at the Stanford Center on Philanthropy and Civil Society helps foundations and nonprofits manage their data. The toolkit provides these organizations with tools for using data ethically and safely. It primarily consists of a series of worksheets to elicit discussion and provide best known methods for conducting a data/AI project with ethics in mind. It focuses on privacy, transparency, and human rights.
The Ethics and Algorithms Toolkit, created by GovEx, the City and County of San Francisco, Harvard DataSmart, and Data Community DC was created to help cities understand the implications and mitigate potential risks of using algorithms. The toolkit first assesses and then manages the algorithm risk. It delves more deeply into potential harms and human rights considerations than most of the other tools. The main goal of the toolkit is to elicit conversation, and in our experience does a good job of raising questions and considerations.
Aequitas, from the University of Chicago Center for Data Science and Public Policy, provides machine learning developers, analysts, and policymakers with an open source bias audit toolkit. The audit looks at four measures of fairness, and assesses a categorical dataset based on these metrics. It also includes a handy decision tree to help you decide which of these metrics is more important in your situation, though the audit tool also lets you explore each measure and provides an explanation for why each is important and how the dataset failed the metric.
This extensible, open source toolkit developed by IBM Research assesses discrimination and bias in machine learning models. The goal of the toolkit is to provide researchers with a framework to share and evaluate algorithms. The included interactive Web experience provides industry-specific tutorials for researchers and data scientists to incorporate the most relevant tools into their work. This toolkit mostly covers fairness, but has the widest variety of ways to learn about the topic.
The What-If Tool from Google is built into the open source TensorBoard Web application and allows users to analyze a machine learning model. With the What-If Tool, users can test algorithmic fairness constraints, visualize inference results, edit a datapoint to see how a model performs (aka counterfactuals), and more. As the tool is incorporated into TensorFlow*, it provides the most visual experience of the toolkits listed here. It primarily focuses on fairness as opposed to other AI ethics topics.
Lime (local interpretable model-agnostic explanations), from the University of Washington, is an open source project that allows users to understand the reasons behind why a model makes certain predictions. These explanations provide insights that help users detect and improve untrustworthy models. The algorithm works for categorical, text, and image datasets. It’s been around since 2016, making it one of the oldest tools on this list, but is still being actively developed. While the algorithm primarily is built for transparency, it can be utilized by practitioners to understand if models are fair.
There are a number of tools that we weren’t able to access, because they are available only to customers or employees of a company. These include Fairness Flow from Facebook, similar to Google’s What-If Tool, which helps Facebook employees assess if their datasets and predictions are fair. Another is the Fairness Tool from Accenture which allows its staff and clients to determine how their prediction algorithms may be relying on sensitive variables like race and gender, along with correlated variables such as location, occupation, or others. A third is Weights and Biases which focuses on bias and transparency, though a key focus of the tool is hyperparameter tuning and experiment tracking. This is a tool which is free to use by an individual, but has a team or company fee.
The tools available to support AI ethics will only grow as the discussion continues. Though this blog primarily focuses on multi-functional toolkits, there are additional techniques and algorithms in this space. In future blog posts, we’ll explore these and dive deeper on various AI ethics topics, including references to research and additional open source resources.
Intel and the Intel logo are trademarks of Intel Corporation or its subsidiaries in the U.S. and/or other countries.
*Other names and brands may be claimed as the property of others.