AI, Personal Data And The DPDP Act: Navigating Legal Uncertainty

Artificial Intelligence (AI) systems are fundamentally dependent on data, and a significant portion of the information used to develop and deploy such systems may relate to identifiable individuals. This creates an important intersection between AI technologies and India’s evolving data protection framework.

With the enactment of the Digital Personal Data Protection Act, 2023 (DPDP Act), organisations developing or deploying AI systems are increasingly required to assess how their use of personal data aligns with existing legal obligations. The challenge, however, is that the DPDP Act was not drafted specifically with generative AI, large language models, or large-scale machine learning systems in mind.

At the same time, India does not currently have a comprehensive standalone legislation specifically regulating AI. Consequently, businesses are required to apply a general data protection framework to technologies that raise several questions not expressly addressed by existing legislation. This uncertainty is particularly relevant in relation to AI training data, consent, anonymisation, transparency, and accountability.

The Challenge of Consent in AI Training

The DPDP Act establishes a framework under which personal data is generally processed on the basis of consent or for certain specified legitimate uses. Where consent is relied upon, a Data Fiduciary is required to provide appropriate information regarding the purpose for which personal data is being processed.

This framework functions relatively effectively in conventional commercial situations. However, AI training presents a considerably different challenge.

Modern AI systems may be trained using extremely large datasets obtained from websites, online forums, social media platforms, review sites, and other digital sources. Organisations may have no direct relationship with many of the individuals whose information ultimately appears within such datasets. Obtaining meaningful consent from every individual whose personal data may be included therefore becomes practically difficult, particularly where information is collected at scale.

The fact that information is publicly accessible also does not necessarily mean that it can automatically be used for any purpose. Public availability and lawful processing are not necessarily the same thing. Organisations should therefore exercise caution before assuming that personal data available on the internet can be freely collected and incorporated into AI training datasets.

The Limited Scope of Legitimate Uses

The DPDP Act permits the processing of personal data without fresh consent in certain specifically identified circumstances under Section 7, commonly referred to as “certain legitimate uses.”

These include circumstances such as processing for employment-related purposes, compliance with legal obligations, and other situations expressly identified under the legislation. However, large-scale AI training is not specifically identified as an independent legitimate use.

This creates an important legal gap for AI developers. Where meaningful consent cannot realistically be obtained, organisations may find it difficult to identify a clear statutory basis for processing personal data for model training.

While certain AI-related processing activities may potentially fall within specific provisions depending on the facts of a particular case, broadly categorising AI training under an existing legitimate use without a clear legal basis may involve compliance risks. Until further regulatory guidance or judicial interpretation emerges, businesses should avoid relying on overly expansive interpretations of the exemptions available under the DPDP Act.

Anonymisation and the Risk of Re-Identification

Anonymisation may provide an important mechanism for reducing data protection risks associated with AI systems. The DPDP Act applies to information relating to an identifiable individual. Where data has been genuinely anonymised and can no longer be used to identify a person, it may fall outside the scope of personal data regulated under the Act.

For AI developers, this creates a clear incentive to minimise the use of identifiable personal information and adopt anonymisation or aggregation techniques wherever possible.

However, anonymisation should not be regarded as a permanent or absolute compliance solution. Advances in data analytics and AI technologies have demonstrated that individuals may potentially be re-identified by combining supposedly anonymous datasets with other available information.

Indian law does not presently prescribe a detailed technical standard for determining when information has been sufficiently anonymised in the context of AI systems. Organisations should therefore treat anonymisation as an ongoing risk-management exercise rather than a one-time technical process.

The “Black Box” Problem and Accountability

Complex AI systems also create challenges relating to transparency and accountability. Many machine learning systems operate through highly complex computational processes, making it difficult to explain precisely how a particular decision or output was generated. This is commonly referred to as the “black box” problem.

The issue becomes particularly significant where AI systems are used to influence decisions relating to recruitment, credit, insurance, fraud detection, or other activities that may materially affect individuals.

The DPDP Act does not currently contain a detailed statutory regime specifically addressing automated decision-making or providing individuals with a general right to obtain an explanation for AI-generated decisions. However, this absence should not necessarily be interpreted as excluding automated processing from broader data protection obligations.

Organisations remain responsible for ensuring that personal data is processed lawfully and that appropriate measures are adopted to protect such information. Maintaining documentation regarding AI systems, their intended purpose, the categories of data used, and their operational limitations may therefore become increasingly important.

Accountability Across the AI Supply Chain

Determining responsibility can become particularly complicated where several organisations are involved in developing and deploying an AI system.

A typical AI ecosystem may involve a data provider, model developer, cloud infrastructure provider, technology vendor, and the business ultimately deploying the system. If personal data is improperly processed or exposed, determining which entity bears responsibility may not always be straightforward.

The DPDP Act places primary obligations on the Data Fiduciary, being the entity responsible for determining the purpose and means of processing personal data. However, AI systems frequently involve multiple entities performing different functions throughout the data lifecycle.

Organisations should therefore clearly identify the responsibilities of technology providers, vendors, processors, and other participants. Appropriate contractual arrangements should address authorised processing purposes, security obligations, data retention, breach reporting, and regulatory responsibilities.

The Road Ahead

Waiting for complete regulatory clarity may not be a practical strategy for organisations already developing or deploying AI technologies. Businesses should instead begin establishing internal AI governance practices based on the existing obligations under the DPDP framework.

A useful starting point is to understand the complete data lifecycle associated with an AI system, including the source of the data, the purpose for which it is processed, and the entities involved. Businesses should document the legal basis for datasets used in AI development, apply data minimisation principles, adopt anonymisation wherever appropriate, and avoid assuming that publicly available personal data can automatically be used without regulatory consideration.

The DPDP Act provides an important foundation for regulating personal data, but several questions concerning AI systems remain unresolved. Issues relating to AI training datasets, anonymisation, automated decision-making, transparency, and liability will likely require further clarification through regulatory guidance, enforcement decisions, or judicial interpretation.

Until greater clarity emerges, organisations should adopt a cautious and proactive approach. In a regulatory environment evolving alongside rapidly advancing technology, proactive and documented compliance may be the most effective way to manage the legal uncertainty surrounding AI and personal data.

The content of this article is intended to provide a general guide to the subject matter. Specialist advice should be sought about your specific circumstances.