United States: Proposed Algorithmic Accountability Act Targets Bias In Artificial Intelligence

In Short

The Situation: There have been numerous reports that computer algorithms used in artificial intelligence ("AI") systems have created or contributed to biased and discriminatory outcomes. To reduce such bias and discrimination, Senators Cory Booker (D-NJ) and Ron Wyden (D-OR) recently proposed the Algorithmic Accountability Act of 2019 ("Act"), with Rep. Yvette Clarke (D-NY) sponsoring an equivalent bill in the House. The Act is the first federal legislative effort to regulate AI systems across industries in the United States, and it reflects a growing and legitimate concern regarding the lawful and ethical implementation of AI.

The Result: The Act authorizes and directs the Federal Trade Commission ("FTC") to issue and enforce regulations that will require certain persons, partnerships, and corporations using, storing, or sharing consumers' personal information to conduct impact assessments and "reasonably address in a timely manner" any identified biases or security issues.

Looking Ahead: The Act would affect AI systems used not only by technology companies, but also by banks, insurance companies, retailers, and many other consumer businesses. Entities that develop, acquire, and/or utilize AI must be cognizant of the potential for biased decision-making and outcomes resulting from its use. Such entities should make efforts now to mitigate such potential biases and take corrective action when it is found.



Background: The Potential for Bias in AI

Employed across industries, AI applications unlock smartphones using facial recognition, make driving decisions in autonomous vehicles, recommend entertainment options based on user preferences, assist the process of pharmaceutical development, judge the creditworthiness of potential homebuyers, and screen applicants for job interviews. AI automates, quickens, and improves data processing by finding patterns in the data, adapting to new data, and learning from experience. In theory, AI is objective—but in reality, AI systems are informed by human intelligence, which is of course far from perfect. Humans typically select the data used to train machine learning algorithms and create parameters for the machines to "learn" from new data over time. Even without discriminatory intent, the training data may reflect unconscious or historic bias. For example, if the training data shows that people of a certain gender or race have fulfilled certain criteria in the past, the algorithm may "learn" to select those individuals at the exclusion of others.

The Intent of the Act

The Act was introduced on the heels of several widely publicized reports of AI leading to biased or discriminatory outcomes. Concerned that algorithms may exacerbate discrimination, Senator Wyden explained that the Act "requires companies to study the algorithms they use, identify bias in these systems and fix any discrimination or bias they find." Senator Booker noted that the Act represents a "key step toward ensuring more accountability" from companies using AI software. As Representative Clarke put it: "Algorithms shouldn't have an exemption from our anti-discrimination laws."

Key Provisions of the Act

As a preliminary matter, the Act would apply to any "covered entity"—namely, any person, partnership, or corporation that is subject to the FTC's jurisdiction and makes more than $50 million per year, possesses or controls personal information on at least one million people or devices, or primarily acts as a data broker that buys and sells consumer data.

Within two years of the Act's enactment, the FTC is charged with requiring such "covered entities" to conduct impact assessments for any existing or new: (i)"high-risk" automated decision systems; and (ii) "high-risk" information systems.

An "automated decision system" is any computational process that employs AI or machine learning to make or facilitate making a decision that affects consumers (e.g., product recommendations based on a user's search history or past buying habits). An "information system" is a "process, automated or not, that involves personal information" but does not qualify as an "automated decision system."

"High risk" systems subject to the impact assessments include those that: 

  • Pose a significant risk to the privacy or security of personal information of consumers, or of resulting in or contributing to inaccurate, unfair, biased, or discriminatory decisions affecting consumers;
  • Make or facilitate human decision-making based on systematic and extensive evaluations of consumers (including attempts to analyze/predict sensitive aspects of their lives) and alter legal rights of consumers or otherwise affect consumers;
  • Involve the personal information of consumers regarding race, religion, health, gender, gender identity, criminal convictions or arrests, and other factors;
  • Monitor public places; and
  • Meet any other criteria established by the FTC.

An impact assessment of a "high-risk" automated decision system must evaluate the system and its training data "for impacts on accuracy, fairness, bias, discrimination, privacy, and security" and must include (among other things) a description of the duration for which the system stores personal information and results, what information about the system is available to consumers, the extent to which consumers have access to the results of the system and may correct or object to its results, etc. An impact assessment for a "high-risk" information system (called a "data protection impact assessment" under the Act) evaluates the extent to which the system protects the privacy and security of the personal information that it processes.

The Act provides that "if reasonably possible," impact assessments are to be performed in consultation with external third parties, including independent auditors and independent technology experts. Moreover, the assessments are not "one and done"; rather, they will have to be conducted as often as the FTC determines is necessary (and likely as often as necessary to assess the ever-changing state of the AI system, its decision-making, and outcomes). Finally, the assessments cannot be avoided with consumer waivers. However, consumers would not necessarily be privy to any impact assessment, as the decision to make an assessment public would be left to the covered entity in its sole discretion.

What Now?

Employers utilizing AI already must be cognizant of the potential for disparate treatment or, more likely, disparate impact claims under equal employment opportunity laws such as Title VII, the Age Discrimination in Employment Act, or the Americans with Disabilities Act as a result of algorithmic bias. Whether or not the Act (or a modified version of it) ultimately becomes law, companies utilizing AI should be prepared for additional government oversight of AI and increased consumer scrutiny regarding the lawful and ethical use of AI. The Act does not offer a perfect solution—significant questions remain as to whether it goes far enough in imposing neutral or third-party "checks" on AI systems, how its requirements would be policed and enforced, whether it requires sufficient transparency to consumers and the public, and whether even the most well-meaning companies can keep pace with their constantly evolving AI technologies. Nevertheless, the Act marks a meaningful step in advancing the dialogue on this important issue.

Entities reliant on AI systems can take steps now to proactively address potential bias issues. While the steps ultimately taken would depend on the entity, the nature of the AI, and other facts, companies with direct input and control over the development of AI can:  

  • Evaluate the development processes used for the systems and the system outputs;
  • Develop training programs for those engaged in AI development and data processing to raise awareness of inherent biases in the data;
  • Implement an audit system for regularly checking the input data and results generated by the AI;
  • Document key decision-making in AI software development;
  • Develop AI tools that improve the traceability of AI decisions to provide real-time insights into how decisions are made; and 
  • Increase transparency to consumers regarding data and AI use.

Companies that do not develop AI in-house, but instead utilize AI systems developed by third parties, can make efforts to understand the third party's policies and procedures for eliminating bias in such systems. They can also try to structure contracts with these third parties to limit the company's liability for legal violations arising from decisions made by the system. Moreover, employers who utilize AI in making employment decisions (e.g., hiring, discipline, termination) must be aware of the potential for algorithmic bias and should consider evaluating whether the AI system is producing a disparate impact based on age, race, sex, national origin, or any other protected class.



Three Key Takeaways
  1. As AI becomes ubiquitous in its applications across industries, so does its potential for bias and discrimination. Understanding the inherent biases in underlying data and developing automated decision systems with explainable results will be key to addressing and correcting the potential for unfair, inaccurate, biased, and discriminatory AI systems.
  2. Whether or not the Act becomes law in its present form or becomes law at all, algorithmic bias is a significant issue—and consumer businesses can take steps to address it now.
  3. Qualified individuals both within and outside the company should be selected and empowered to investigate and rectify bias and security flaws in AI systems. Outside consultants should be consulted on best practices. Legal counsel should be consulted regarding potential legislation and regulation (within and outside the United States) and regarding how to handle document assessments, disclosures, and corrective actions.

The content of this article is intended to provide a general guide to the subject matter. Specialist advice should be sought about your specific circumstances.

To print this article, all you need is to be registered on Mondaq.com.

Click to Login as an existing user or Register so you can print this article.

Authors
 
In association with
Related Topics
 
Related Articles
 
Related Video
Up-coming Events Search
Tools
Print
Font Size:
Translation
Channels
Mondaq on Twitter
 
Mondaq Free Registration
Gain access to Mondaq global archive of over 375,000 articles covering 200 countries with a personalised News Alert and automatic login on this device.
Mondaq News Alert (some suggested topics and region)
Select Topics
Registration (please scroll down to set your data preferences)

Mondaq Ltd requires you to register and provide information that personally identifies you, including your content preferences, for three primary purposes (full details of Mondaq’s use of your personal data can be found in our Privacy and Cookies Notice):

  • To allow you to personalize the Mondaq websites you are visiting to show content ("Content") relevant to your interests.
  • To enable features such as password reminder, news alerts, email a colleague, and linking from Mondaq (and its affiliate sites) to your website.
  • To produce demographic feedback for our content providers ("Contributors") who contribute Content for free for your use.

Mondaq hopes that our registered users will support us in maintaining our free to view business model by consenting to our use of your personal data as described below.

Mondaq has a "free to view" business model. Our services are paid for by Contributors in exchange for Mondaq providing them with access to information about who accesses their content. Once personal data is transferred to our Contributors they become a data controller of this personal data. They use it to measure the response that their articles are receiving, as a form of market research. They may also use it to provide Mondaq users with information about their products and services.

Details of each Contributor to which your personal data will be transferred is clearly stated within the Content that you access. For full details of how this Contributor will use your personal data, you should review the Contributor’s own Privacy Notice.

Please indicate your preference below:

Yes, I am happy to support Mondaq in maintaining its free to view business model by agreeing to allow Mondaq to share my personal data with Contributors whose Content I access
No, I do not want Mondaq to share my personal data with Contributors

Also please let us know whether you are happy to receive communications promoting products and services offered by Mondaq:

Yes, I am happy to received promotional communications from Mondaq
No, please do not send me promotional communications from Mondaq
Terms & Conditions

Mondaq.com (the Website) is owned and managed by Mondaq Ltd (Mondaq). Mondaq grants you a non-exclusive, revocable licence to access the Website and associated services, such as the Mondaq News Alerts (Services), subject to and in consideration of your compliance with the following terms and conditions of use (Terms). Your use of the Website and/or Services constitutes your agreement to the Terms. Mondaq may terminate your use of the Website and Services if you are in breach of these Terms or if Mondaq decides to terminate the licence granted hereunder for any reason whatsoever.

Use of www.mondaq.com

To Use Mondaq.com you must be: eighteen (18) years old or over; legally capable of entering into binding contracts; and not in any way prohibited by the applicable law to enter into these Terms in the jurisdiction which you are currently located.

You may use the Website as an unregistered user, however, you are required to register as a user if you wish to read the full text of the Content or to receive the Services.

You may not modify, publish, transmit, transfer or sell, reproduce, create derivative works from, distribute, perform, link, display, or in any way exploit any of the Content, in whole or in part, except as expressly permitted in these Terms or with the prior written consent of Mondaq. You may not use electronic or other means to extract details or information from the Content. Nor shall you extract information about users or Contributors in order to offer them any services or products.

In your use of the Website and/or Services you shall: comply with all applicable laws, regulations, directives and legislations which apply to your Use of the Website and/or Services in whatever country you are physically located including without limitation any and all consumer law, export control laws and regulations; provide to us true, correct and accurate information and promptly inform us in the event that any information that you have provided to us changes or becomes inaccurate; notify Mondaq immediately of any circumstances where you have reason to believe that any Intellectual Property Rights or any other rights of any third party may have been infringed; co-operate with reasonable security or other checks or requests for information made by Mondaq from time to time; and at all times be fully liable for the breach of any of these Terms by a third party using your login details to access the Website and/or Services

however, you shall not: do anything likely to impair, interfere with or damage or cause harm or distress to any persons, or the network; do anything that will infringe any Intellectual Property Rights or other rights of Mondaq or any third party; or use the Website, Services and/or Content otherwise than in accordance with these Terms; use any trade marks or service marks of Mondaq or the Contributors, or do anything which may be seen to take unfair advantage of the reputation and goodwill of Mondaq or the Contributors, or the Website, Services and/or Content.

Mondaq reserves the right, in its sole discretion, to take any action that it deems necessary and appropriate in the event it considers that there is a breach or threatened breach of the Terms.

Mondaq’s Rights and Obligations

Unless otherwise expressly set out to the contrary, nothing in these Terms shall serve to transfer from Mondaq to you, any Intellectual Property Rights owned by and/or licensed to Mondaq and all rights, title and interest in and to such Intellectual Property Rights will remain exclusively with Mondaq and/or its licensors.

Mondaq shall use its reasonable endeavours to make the Website and Services available to you at all times, but we cannot guarantee an uninterrupted and fault free service.

Mondaq reserves the right to make changes to the services and/or the Website or part thereof, from time to time, and we may add, remove, modify and/or vary any elements of features and functionalities of the Website or the services.

Mondaq also reserves the right from time to time to monitor your Use of the Website and/or services.

Disclaimer

The Content is general information only. It is not intended to constitute legal advice or seek to be the complete and comprehensive statement of the law, nor is it intended to address your specific requirements or provide advice on which reliance should be placed. Mondaq and/or its Contributors and other suppliers make no representations about the suitability of the information contained in the Content for any purpose. All Content provided "as is" without warranty of any kind. Mondaq and/or its Contributors and other suppliers hereby exclude and disclaim all representations, warranties or guarantees with regard to the Content, including all implied warranties and conditions of merchantability, fitness for a particular purpose, title and non-infringement. To the maximum extent permitted by law, Mondaq expressly excludes all representations, warranties, obligations, and liabilities arising out of or in connection with all Content. In no event shall Mondaq and/or its respective suppliers be liable for any special, indirect or consequential damages or any damages whatsoever resulting from loss of use, data or profits, whether in an action of contract, negligence or other tortious action, arising out of or in connection with the use of the Content or performance of Mondaq’s Services.

General

Mondaq may alter or amend these Terms by amending them on the Website. By continuing to Use the Services and/or the Website after such amendment, you will be deemed to have accepted any amendment to these Terms.

These Terms shall be governed by and construed in accordance with the laws of England and Wales and you irrevocably submit to the exclusive jurisdiction of the courts of England and Wales to settle any dispute which may arise out of or in connection with these Terms. If you live outside the United Kingdom, English law shall apply only to the extent that English law shall not deprive you of any legal protection accorded in accordance with the law of the place where you are habitually resident ("Local Law"). In the event English law deprives you of any legal protection which is accorded to you under Local Law, then these terms shall be governed by Local Law and any dispute or claim arising out of or in connection with these Terms shall be subject to the non-exclusive jurisdiction of the courts where you are habitually resident.

You may print and keep a copy of these Terms, which form the entire agreement between you and Mondaq and supersede any other communications or advertising in respect of the Service and/or the Website.

No delay in exercising or non-exercise by you and/or Mondaq of any of its rights under or in connection with these Terms shall operate as a waiver or release of each of your or Mondaq’s right. Rather, any such waiver or release must be specifically granted in writing signed by the party granting it.

If any part of these Terms is held unenforceable, that part shall be enforced to the maximum extent permissible so as to give effect to the intent of the parties, and the Terms shall continue in full force and effect.

Mondaq shall not incur any liability to you on account of any loss or damage resulting from any delay or failure to perform all or any part of these Terms if such delay or failure is caused, in whole or in part, by events, occurrences, or causes beyond the control of Mondaq. Such events, occurrences or causes will include, without limitation, acts of God, strikes, lockouts, server and network failure, riots, acts of war, earthquakes, fire and explosions.

By clicking Register you state you have read and agree to our Terms and Conditions