Organisations are investing significant time and resources into understanding what data they hold, where it is stored, who has access to it, how it may be used and how it should be protected, making data governance a boardroom issue not merely an IT one. With increasing regulatory scrutiny, cybersecurity risk and dependence on data-driven technologies, that focus is understandable.
But there is a gap that is receiving far less attention.
As artificial intelligence (“AI”) becomes embedded into the systems and tools used by organisations, many businesses appear to assume that if their data is properly governed, their AI is too. That is not necessarily the case.
Data governance and AI governance are closely connected, but they are not the same thing.
Data governance focuses primarily on the management, quality, availability, security, and lawful use of data, together with the roles and controls that support this objective. AI governance has a broader system-level focus: how AI is selected, developed, procured, monitored and retired, how its outputs affect people and decisions, and who remains responsible throughout its lifecycle.
The distinction matters because an organisation can have excellent controls around its data and still have very little control over how AI is being used.
An organisation may, for example, know exactly what information employees are permitted to access, while having little visibility over whether employees are uploading that information into generative AI tools. It may have carefully considered how customer information is stored and protected, while giving limited consideration to how an AI-enabled system is using that information to generate recommendations or influence decisions. It may even procure a technology solution without fully appreciating that AI functionality has been embedded into the product.
The result is that AI can begin operating across an organisation faster than the governance structures intended to manage it. This introduces additional and sometimes overlapping risks that conventional data controls may not fully address. AI systems may produce incorrect or misleading outputs. They may rely on information in ways that are not always transparent to the user. Their functionality may change over time as suppliers update models and introduce new features. AI may also increasingly influence decisions that were previously made entirely by people.
Traditional data governance controls do not necessarily answer the questions that follow.
Who is permitted to use AI and for what purpose? When can an AI-generated output be relied on? When must a person review or override it? Who is responsible if the output is wrong? Does the organisation understand which AI systems are already being used within its technology environment? What happens when a supplier introduces AI functionality into an existing service?
In South Africa, these questions may also have direct legal consequences. The Protection of Personal Information Act, 2013 ('POPIA') does not regulate AI as a technology in itself, but its requirements apply where an AI system processes personal information. Section 71 (Automated decision-making) is particularly relevant where a decision that has legal consequences, or substantially affects a person, is based solely on automated processing intended to profile that person. Depending on the circumstances, organisations may also need to consider other POPIA requirements such as transparency, security, purpose limitation, operator oversight and cross-border processing requirements.
These are AI governance questions.
The concern is not that organisations need another layer of governance for the sake of it. The concern is that AI is already becoming part of day-to-day business operations, often before organisations have properly considered how it should be controlled.
Effective AI governance does not necessarily require organisations to reinvent their existing governance structures. In many cases, it should build on them. Existing data protection, information security, procurement, technology and risk frameworks can provide an important foundation, but these should not obscure the need for AI-specific expertise, testing, documentation and escalation routes.
However, organisations still need to look specifically at AI.
At a minimum, organisations should:
- identify AI systems and use cases (including embedded and employee-adopted tools);
- classify them according to the purpose, affected persons, data used and potential impact;
- assign an accountable business owner;
- assess and control risk before deployment or material change;
- monitor performance and supplier changes; and
- remediate or retire AI systems that no longer meet the organisation’s requirements.
Organisations should also consider whether appropriate human oversight exists where AI produces outputs, makes recommendations or influences decisions.
Contracts should also form part of that assessment. Where AI is provided by a third party, organisations should assess the rights and remedies available to parties, including: what the supplier is doing with their data; whether AI functionality can change during the term of the agreement; what responsibility the supplier accepts for the operation of the technology; and whether the contractual protections negotiated for a conventional technology service remain appropriate once AI is introduced.
The starting point is straightforward: do not assume that governing the data means that the AI using that data is governed too.
Data governance establishes how information going into an AI system is managed. AI governance addresses whether the AI technology and systems, outputs and effects are appropriately controlled.
The content of this article is intended to provide a general guide to the subject matter. Specialist advice should be sought about your specific circumstances.
[View Source]
