Internalizing AI Governance: The Practical Thinking So Far

Clients, recent speaking engagements, the explosion of state AI regulation and guidance from financial authorities have all forced me to think and re-think how companies should practically approach their AI governance.

On the one hand, AI-powered tools promise to advance productivity for most tech-powered companies, and most companies find themselves eager to harness the power of these solutions. On the other hand, the regulatory, legal, and reputational risks are increasingly non-trivial, including the potential for private litigation and enforcement, failed customer engagement strategy, and other challenges.

The answer, in part, lies in the possibility of building repeatable processes around the assessment and decision to deploy AI, wherever and however it may be deployed within an organization. It is time to move beyond governance policies and documents that exist – untouched and unused, for all purposes dead and inert – on a shelf. This article instead offers a few emerging practical ideas to help companies develop repeatable, living governance processes that actually serve the organization in its decision to deploy AI solutions. This general framework is meant to be flexible in that it could work with other approaches, such as detailed control or audit frameworks, with more or less elaborate internal reporting structures, and with more or less internal technical and legal expertise on hand. But, in my experience, some elements of this framework need to be present for real, working internal AI governance to exist.

Build an AI Inventory / Register.

The Financial Services AI Risk Model Framework centrally recommends having a registry of AI systems. This idea deserves wider attention, and should not just be limited to financial institutions or FinTech businesses. A registry might be used to support more or less complex oversight processes, but some central source of truth often makes sense in any case. The registry could identify systems and their purpose, internal ownership, the status of assessments or review at different stages of the deployment/deployment/management lifecycle, and memorialize the decision to deploy the AI. The registry might also track subsequent confirmation of the deployment decision at various key stages in the AI lifecycle (beta, full deployment, major version upgrades, or identification of some substantial new risks).

Assign Business Ownership.

Without appropriate ownership, any governance system will fail. The decision to deploy particular AI tools ultimately need to be ‘owned’ at an appropriate level. For large organizations, AI ‘owners’ could be focused on different functional areas (like product, sales, HR) or fall under the responsibility of a single Chief AI Officer. Ideally, such owners would have some alignment of responsibility (to make the decision to deploy an AI) and power (to oversee that deployment and any related assessments). Ideally, such owners will be able to form real, all-things-considered business judgments about the company’s rational appetite for risk in making the decision to deploy particular AI. For larger organizations, these decisions could be subject to further guidance and oversight at more senior levels of the business. Procedurally, owners will ideally have some responsibility for maintaining their entries within the AI Inventory and for advancing related assessments, advising and decision-making.

Conduct Ongoing Assessments.

The idea that AI should be assessed or advised in some way is hardly novel. AI should be assessed and advised. There is a potential pitfall, however, in viewing assessment and advising as a strictly one-time activity that usually occurs at some early developmental stage and is then deemed permanently completed. Systems – including AI systems – inevitably change and develop over time, in response to customer demands, consideration of prior assessments and advice, technical and legal development, and for other reasons. This change has to be managed. The key to a living governance process, therefore, is to have an assessment form / questionnaire / process that can be “versioned up” as facts change, and which will trigger additional expert analysis and decision-making at that juncture.

Assessment Forms and Materials Need Careful Consideration.

Legal risks around AI are increasingly deep and wide. Assessment forms and questionnaires should therefore be considerately designed and managed over time to provide legal and other advisors with critical, key facts under a broad range of laws and risk scenarios – ideally, without being unduly burdensome.

Avoid Setting Up Legal and Compliance Functions for Failure.

Though particular advice always varies with context,assigning “ownership” of ultimate business risk and AI deployment decision-making to legal and compliance functions is often ill-advised. Instead, the business managers and executives need to own the risks / rewards of their own business decisions. Legal and compliance need to maintain their roles as objective, independent advisors and assistants to the business, not as substitute decision-makers. Assigning inappropriate ownership to legal and compliance functions also tends to be irrational insofar as it makes legal and compliance responsible for systems, personnel, and relationships not directly under their control. Responsibility without actual power or control over the area of responsibility is usually a trap resulting in predictable failure.

The Positive Process-Supporting Role of Legal and Compliance.

Legal and compliance can appropriately manage the documentation and records created by the above governance processes. Legal and compliance can form analysis and advice on the basis of completed assessment questionnaires / processes. Legal and compliance can facilitate review and updates to the form of the inventory and assessment to help improve their usefulness within the organization. Legal and compliance can help advise and execute on certain directions from the business (e.g., a direction to enhance contractual protection with key vendors). Legal and compliance can help train, educate and guide processes entailed by the above framework. Consider assigning these (appropriate) roles to internal or external legal and other advisors.

Top-Level Leadership is Key.

Consistent with the above, C-Level officers and senior management may appropriately decide to delegate some level of AI assessment and decision-making ownership to subordinate levels of the organization. However, executives need to oversee these delegations. Such oversight includes ensuring appropriate resources and governing policies; it also includes, in any human organization, ensuring that there are consequences for non-performance.

There is undoubtedly more to say about what implementation might look like in particular organizations, or how these framework considerations might work with other approaches, audit standards, or the like. This framework does not provide in advance the answer to what particular laws require of particular applications, or how businesses might rationally decide to approach their particular risks. The general thought is only that, when these parts work together jointly, then organizations may really start to have a complete, practical, AI governance system integrated appropriately into the business as a whole as a real, living process.

Taft will continue to monitor developments and similar trends. If you have questions, Taft’s Privacy, Security & AI attorneys are available to assist. As always, please sign up to receive emails of our latest posts here on Privacy and Data Security Insights, and follow us on LinkedIn for the latest in privacy, security and artificial intelligence legal news.