In Short

The Development: On 21 April 2021, the European Commission ("Commission") unveiled a proposal for a "Regulation laying down harmonized rules on Artificial Intelligence" ("AI Regulation"), which sets out how AI systems and their outputs can be introduced to and used in the European Union ("EU"). The AI Regulation is accompanied by a proposal for a new Regulation on Machinery Products, which focuses on the safe integration of the AI system into machinery, as well as a new Coordinated Plan on AI outlining the necessary policy changes and investment at Member State level to strengthen the EU's leading position in trustworthy AI.

Background: The draft AI Regulation is part of a wider regulatory agenda in the EU focusing on availability and use of industrial data. Reflecting input from various stakeholders, it aims to establish a European model for the development and use of AI systems that ensures an EU market for AI systems that balances related benefits and risks. Among other things, the draft AI Regulation broadly defines AI systems, specifically prohibits certain uses of AI systems (such as social scoring by public authorities) and foresees a regime for introducing "high risks" AI in the EU.

Looking Ahead: If adopted by the EU Parliament and the Council (which could take two to three years), the proposed AI Regulation would condition how AI systems (or products integrating AI) are commercialized and used in the EU and could lead to global repercussions, as with the European General Data Protection Regulation ("GDPR"). Organizations exploring, developing or using AI systems should consider contributing to the public consultation which is open until 22 June 2021. In any event, they should closely follow these developments which, if adopted, will apply to their activities in addition to key regulations such as the GDPR and possibly the proposed Digital Services and Digital Market Acts.

Proposed AI Regulation

The draft AI Regulation introduces a set of rules, following a risk-based approach, to establish the conditions for an ecosystem of trust regarding the placing on the market, putting into service and use of AI systems in the EU. The main building blocks of the proposed regime are summarized below.

Potential extra-territorial scope: The draft AI Regulation would apply to providers placing on the market or putting into service AI systems in the EU, irrespective of the location of these providers, to all EU users of AI systems; and to both providers and users of AI systems located outside the EU if the output produced by the AI system is used in the EU.

Wide definition of AI: The draft AI Regulation broadly defines AI systems as all software developed with techniques and approaches such as "machine learning", "logic- and knowledge-based" and "statistical" approaches, that can, for a given set of human-defined objectives, generate outputs such as content, predictions, recommendations, or decisions influencing the environments in which they interact.

Prohibited AI practices: The draft AI Regulation proposes to ban AI practices that consist of (i) deploying subliminal techniques beyond a person's consciousness, or exploiting the vulnerabilities of a specific group of persons, in order to distort these persons' behavior in a manner that causes or is likely to cause them harm; (ii) social scoring by public authorities; and (iii) using real-time remote biometric identification systems in publicly accessible spaces for the purpose of law enforcement, unless justified for a targeted search for victims of crimes, the prevention of threats to people's lives and physical safety or of terrorist attacks, and the detection and identification of perpetrators of serious crimes.

Focus on "high-risk" AI systems: The draft AI Regulation introduces a specific regime for placing high-risk AI systems on the market or putting these into service. A number of AI applications qualify as such under the draft AI Regulation, including safety components of products or products covered by existing EU product safety legislation (e.g., for machinery, toys, radio equipment, cars and other types of vehicles, and medical devices) when subject to third-party conformity assessment. High-risk AI systems also include so-called "stand-alone AI systems" used for:

  • "Real-time" and "post" remote biometric identification of natural persons;
  • Safety in the management and operation of critical infrastructures;
  • Educational and vocational training (access to institutions or student assessments);
  • Recruiting or making other human resources decisions;
  • Evaluating creditworthiness of persons;
  • Evaluating a person's eligibility for public assistance benefits and services;
  • Enforcing laws in ways that may interfere with a person's fundamental rights;
  • Processing and examining asylum and visa applications and border control management; and
  • Assisting judges in researching and interpreting facts and the law and in applying the laws to the facts.

The list of high-risk AI systems appears comprehensive and covers applications in various industries like banking and finance, social media, HR, and public services, but the Commission could update these.

Qualification as a high-risk AI system triggers a series of mandatory requirements, and compliance with these must be assessed before the products are placed on the market or put into service. These obligations include:

  • Establishment of an adequate risk management system;
  • Use of high quality training, validation and testing data sets;
  • Preparation of technical documentation providing all necessary information on the system and its purpose to assess its compliance with the requirements;
  • Development of logging capabilities enabling automatic recording to ensure traceability of the functioning of the system;
  • Provision of appropriate transparency on the operation of the AI system and clear information to users;
  • Guarantee of human oversight to minimize risk; and
  • Attainment of a high level of accuracy, robustness and cybersecurity.

Providers of high-risk AI systems must assess compliance with these requirements in accordance with the conformity assessment procedures set out in the draft AI Regulation. Depending on the type of system concerned, these procedures can either take the form of a self-assessment or a third-party assessment through the involvement of a notified body.

High-risk AI systems that are deemed to comply with the mandatory requirements following assessment by their providers should bear the "CE" quality marking to indicate their conformity with European rules. Stand-alone high-risk AI systems must also register with a publicly available EU database on high-risk AI systems.

In addition to the above obligations borne by providers, the draft AI Regulation also imposes obligations on importers, distributors, and users of high-risk AI systems to ensure that these products comply with regulatory requirements before their placing or making available on the market and to ensure safe use of the products.

Non-high-risk AI systems: Unlike high-risk AI systems, the draft AI Regulation regulates non-high-risk AI systems only to a limited extent by imposing transparency obligations for such AI systems in order to protect the users of, or persons exposed to, such technology. This covers AI intended to interact with natural persons, emotion recognition systems, a biometric categorization systems, and deepfakes. All other AI systems can be developed and used without additional legal obligations.

Measures in support of innovation: To promote innovation, the draft AI Regulation would enable national regulators to establish regulatory sandboxes schemes and require Member States to provide certain services and facilities to small-scale providers, start-ups, and users.

Enforcement: The draft AI Regulation delegates most enforcement powers to Member States, who will designate competent EU Member State authorities (most likely the data protection authorities) and determine the penalties applicable to infringements of the AI Regulation. Notably, despite Member State powers to decide on penalties, the draft AI Regulation provides that failure to comply with certain sensitive provisions (i.e., prohibited AI practices and high quality of data sets) will result in maximum fines of up to EUR 30 million or 6% of a company's worldwide annual turnover. Non-compliance with any other requirements applicable to AI systems would result in fines of up to EUR 20 million or 4% of a company's worldwide annual turnover.

National monitoring and enforcement will be supervised by a contemplated European Artificial Intelligence Board, whose role will be to facilitate an effective and harmonized implementation of the draft AI Regulation e.g., through the issuance of recommendations.

Machinery Regulation

The draft Machinery Regulation complements the draft AI Regulation and is intended to replace the Machinery Directive. It aims at ensuring a safe integration of the AI system into machinery as a whole, towards safeguarding against compromising the safety of the overall machinery for users and consumers. Businesses would need to undertake only one conformity assessment for both the AI Regulation and the Machinery Regulation. The draft Machinery Regulation would also respond to market needs by bringing greater legal clarity to current provisions and simplifying the administrative burden and costs for companies.

Next Steps

The European Parliament and the Council of the EU will now review and discuss the Commission's proposals, which could result in modifications. Both institutions must approve the final text under qualified majority before the AI Regulation and the Machinery Regulation take effect. This process could take two to three years.

A Global Trend

This EU initiative takes place within a broader global discussion on the need to adopt AI-specific rules. For example, in November 2020, the U.S. White House, through its Office of Management and Budget, issued Guidance for Regulation of AI Applications, which establishes a framework for federal agencies to assess potential regulatory and non-regulatory approaches to emerging AI issues. All federal agencies with authority over these issues are directed to provide compliance plans by May 2021. Additional U.S. AI-driven initiatives concern the use of AI in the Federal Government and the creation of a new National AI Initiative Office for federal AI coordination, which may play an important role in the governance of AI.

Five Key Takeaways

  1. The Commission has released a proposal to define the first EU-wide regulatory framework on AI. The proposed centerpiece AI Regulation aims to guarantee user safety and safeguard fundamental EU values and rights, while strengthening AI uptake and innovation across the EU. The proposed AI Regulation would not require further implementation into each national law of the Member States. However, certain grandfathering rights would apply for legacy AI systems that are not subject to significant changes in their design or intended purpose.
  2. The draft AI Regulation deals with core considerations such as defining AI and high-risk applications, regulatory obligations for providers of AI systems, and the conformity assessment of high-risk AI applications. This ensemble would implicate a broad cross-section of industries.
  3. The draft AI Regulation proposes to ban certain uses of high-risk AI systems altogether, while making others subject to mandatory requirements and increased scrutiny.
  4. Failure to comply with the draft AI Regulation could result in significant administrative fines of up to EUR 30 million or 6% of a company's annual worldwide turnover.
  5. The proposed AI Regulation could create a new impetus towards increased enforcement and advocacy for safety and user rights in all Member States, as influenced by the envisaged European Artificial Intelligence Board.

The content of this article is intended to provide a general guide to the subject matter. Specialist advice should be sought about your specific circumstances.