Actionable Privacy Practice Takeaways From The Joint Investigation Of OpenAI: A Practical Compliance Guide For AI Developers

On May 6, 2026, the Office of the Privacy Commissioner of Canada (OPC), together with the Commission d’accès à l’information du Québec (“CAI”), the Office of the Information and Privacy Commissioner for British Columbia (“OIPC-BC”), and the Office of the Information and Privacy Commissioner of Alberta (“OIPC-AB”) (the “Offices”), jointly released findings from their investigation into OpenAI’s privacy practices related to ChatGPT.

The Offices’ investigation was originally commenced in April 2023 as a result of an individual complaint to the OPC. By May 2023, the OPC closed the individual complaint and initiated a broader joint inquiry with privacy regulators from Québec, British Columbia, and Alberta. The investigation examined compliance with Canada’s Personal Information Protection and Electronic Documents Act (“PIPEDA”), Québec’s Act Respecting the Protection of Personal Information in the Private Sector (“Québec’s Private Sector Act”), British Columbia’s Personal Information Protection Act (“PIPA-BC”) and Alberta’s Personal Information Protection Act (“PIPA-AB”).

The Offices examined practices across seven key compliance principles: appropriate purposes, consent, openness and transparency, accuracy, individual rights (access, correction, and deletion), retention, and accountability. Within these areas, the investigation addressed several notable areas for AI developers, including consent requirements for scraping publicly accessible information for training of AI models, privacy-by-default obligations in Québec, accuracy standards for AI-generated outputs containing personal information, and the technical limits of access, correction, and deletion rights in the context of LLMs.

The Offices’ lengthy decision contains a wide range of useful guidance on the privacy compliance expectations of regulators across Canada, and points where they diverge, relevant for any organization that develops, deploys, or integrates AI software processing personal information throughout the lifecycle of its large language models (LLMs).

Key findings for AI developers

The following self-assessment checklist translates the decision's key findings into actionable privacy compliance guidance, helping AI software owners map current practices to regulator expectations, close gaps, and document compliance.

1. Jurisdictional considerations

All Offices found they had jurisdiction to investigate and make recommendations or orders, where an organization is collecting, using, and disclosing the personal information of individuals in respect of a province, in the course of commercial activity, and where services were made available to Canadian users, even when it does not have an established physical presence in Canada.

• Have we assessed whether we have Canadian users, whether our terms of service address Canadian users, and whether personal information is transmitted between Canada and another jurisdiction—that would subject us to Canadian privacy laws even without a physical presence in the country?
• Have we mapped which provincial privacy statutes apply to our operations, recognizing that PIPEDA, PIPA-BC, PIPA-AB, and Québec's Private Sector Act impose distinct requirements—particularly around consent—that may produce different compliance outcomes?

2. Training data collection and use

a. Purpose limitation and proportionality

The Offices accepted that developing and deploying an LLM may represent an appropriate purpose and may be effective in advancing a legitimate need. AI developers should nevertheless be able to establish that the collection, use and disclosure of personal information is limited to what is necessary and proportionate. The Offices found that collection from publicly accessible websites and licensed datasets may be overbroad where mitigation measures are not sufficient to limit the scope of personal information to what is necessary; by contrast, the use of user interactions for model training may be proportional where residual privacy risks are significantly mitigated.

The regulators also recognized the necessity and proportionality of using a certain level of user interaction data (e.g., prompts) to improve model outputs, particularly where organizations disassociate interactions from user accounts, remove personal identifiers using filtering tools, allow users to opt out of training, instruct users not to include sensitive information, and limit training to a small subset of interactions.

• Have we clearly documented the specific purposes for which we collect personal information for model training, and are those purposes legitimate, important, and real?
• Are the legitimacy of our needs and the effectiveness of our practices proportionate to the sensitivity of the information being collected? Have we considered whether less invasive alternatives exist?
• Have we implemented technical measures to screen out websites aimed at children, social media platforms, or sites containing sensitive information about vulnerable groups? If not, can we demonstrate why less privacy-invasive alternatives are insufficient?
• Have we implemented filtering tools to detect and mask identifying information about private individuals in training data? Have we documented the recall and false positive rates of those tools?

b. Consent for public or licensed training data

The Offices diverged materially in their analysis of valid consent for collection and use of training data that includes personal information. The OPC accepted, in a pragmatic and flexible interpretation of PIPEDA, that implied consent may support use of publicly available and licensed data for future dataset training where privacy risks are significantly and meaningfully mitigated (such as by training models to refuse to provide private or sensitive information in their outputs). However, it can be relied upon only where the information is not sensitive and the collection, use and disclosure falls within users’ reasonable expectations.

The OIPC-BC and OIPC-AB concluded that PIPA-BC and PIPA-AB do not contain the PIPEDA concept of implied consent and instead impose specific requirements for “implicit,” “deemed,” or “notice” consent—all of which require that individuals provide their information for the purpose of training AI models, following proper notice. These requirements must be complied with to ensure valid consent for publicly accessible and licensed training data.

The CAI, applying the distinct requirements of Québec's Private Sector Act, emphasized that organizations should not assume that personal information posted online may be collected and used for AI training. Whether valid consent exists depends on the circumstances in which the information was originally collected, including the notice provided to individuals and the applicable website terms and privacy policies. Organizations should therefore verify that individuals were informed their information could be made publicly available and used by third parties, including for AI training where applicable, and ensure that the information was not disclosed without valid consent, including parental or tutor consent where required for individuals under the age of 14.

• Can we demonstrate valid, express or implied, consent for our collection and use of personal information from publicly accessible websites?
• Have we identified whether the personal information in our training datasets includes sensitive categories—such as financial and health data, ethnic and racial origins, political opinions, sexual information, religious beliefs, or personal information pertaining to young people—that would require express consent?
• Have we considered whether individuals reasonably expected their publicly accessible information to be used for AI training when it was posted, especially where the information predates widespread awareness of generative AI?
• Are we relying on “publicly available information” exemptions? If so, have we confirmed that the sources fall within the narrowly defined regulatory categories, rather than assuming that all internet-accessible information qualifies?
• When relying on third-party data sources, have we implemented contractual assurances requiring the third party to obtain lawful consent, and do we monitor the third party’s compliance with these requirements?
• For organizations subject to PIPA-BC or PIPA-AB: have we established that individuals voluntarily provided their information for the purpose of training, or that we provided individuals with notice of our intended collection, as required for implicit or deemed consent under those statutes?
• For organizations subject to Québec’s Private Sector Act: have we checked on the sources from which they collect personal information to ensure that individuals were clearly informed, at the time of collection, that their information would be made public and could be collected and used by third parties including for the purpose of training AI models.

3. User interaction data

a. Consent and notice

To ensure valid consent to use user interaction data for training purposes, AI developers should obtain express consent where the information collected and used for training includes sensitive personal information and/or the use is not within users' reasonable expectations. Prior to collection, AI developers must provide explicit notice to users stating the purposes for, and the means by, which the information is collected, used or disclosed. The notice must also provide details of how users can exercise their rights of access and rectification/correction provided by law and their right to withdraw consent to the disclosure or use of the information collected. A one-time notification at account creation or first use is not sufficient where users are unlikely to understand the nature, purposes and consequences of the processing.

• Do we obtain express consent for the collection and use of personal information from user interactions for model training purposes, where those interactions may involve sensitive personal information?
• Is consent obtained prior to the user submitting any prompts or personal information?
• Are our notifications about data use for training purposes displayed prominently and repeatedly, rather than shown only once at account creation?
• Do we offer users the ability to opt out of training, and if so, is this feature accessible, not deceptively designed, and consistently available?

b. Privacy settings and privacy by default

For Québec users, AI developers should configure technological products or services so that privacy settings provide the highest level of privacy by default, without any intervention by the user. This obligation, introduced by the Law 25 amendments to Québec’s Private Sector Act (in force since September 2023), is codified in section 9.1. The CAI concluded that section 9.1 applies to the entire lifecycle of personal information, including collection, use, disclosure and retention, and that settings relating to the use of user interactions for training of AI models fall within the scope of that obligation.

• Can users easily review and manage whether their interaction data is used for model training?
• For users in Québec: are privacy settings configured to provide the highest level of confidentiality by default, without requiring any intervention by the user? Is the use of interaction data for training purposes deactivated by default, consistent with this requirement applying to privacy settings covering the entire lifecycle of personal information?

4. AI model outputs

a. Consent for disclosure

AI developers should treat model outputs as potential disclosures of personal information, including where outputs contain opinions, rumours, sensitive information or information outside an individual's reasonable expectations. Consent from the individual whose information is disclosed is therefore required under all four privacy statutes. The OPC found that the challenges with ensuring consent may be resolved by the implementation of mitigating measures to significantly reduce the risk that private or sensitive information is disclosed as model outputs. The OIPC-BC and OIPC-AB found that the requirements for implicit, deemed or notice consent also apply to disclosure, and declined to make a similar finding. The CAI similarly reiterated that the rules governing consent for collection and use apply to communication of personal information and concluded that disclosure of personal information in model output without consent contravenes Québec's Private Sector Act.

• Have we assessed whether our AI model outputs may disclose personal information, including opinions, rumours, or other non-traditional categories that fall within the statutory definition of personal information?
• Do our output filtering categories align with the full breadth of the applicable privacy statutes' definitions of personal information?
• Do we obtain express consent where outputs may disclose sensitive personal information or where the disclosure is outside the individual's reasonable expectations?
• Do we inform individuals what personal information is collected and how, when, and why it is collected, used or disclosed throughout any stage of the generative AI lifecycle?
• Do we maintain and publish documentation about the datasets used to develop or train the generative AI tool, including the categories and sources of any personal information, the legal authority for their collection and use, whether there are any licensing agreements or other restrictions on the acceptable uses of the datasets, and any modification, filtering or other curation practices applied to the datasets?
• Do we take appropriate steps to make the outputs from our generative AI systems traceable and explainable?
• Do we ensure that all information communicated about a generative AI system is designed to be understandable by the intended audience, and made readily available both before, during and after use of the system?

b. Accuracy safeguards

The Offices found that AI developers should take reasonable steps to ensure that personal information generated or disclosed by a model is accurate, complete and up to date as necessary for the purposes for which it is to be used. Developers should not rely on inconspicuous or inconsistent disclaimers as to accuracy. They should assess the level of accuracy of personal information in outputs, prominently inform users of limitations and the need to verify facts, and provide mechanisms to verify sources.

• Do we provide sufficient notification to users regarding the potential for response information to be inaccurate?
• Do we clearly and consistently inform users of the need to verify the accuracy of facts provided?
• Do we provide a viable mechanism for users to effectively and reliably verify the accuracy of facts in our outputs?
• Have we conducted assessments specifically validating the accuracy of personal information in our model outputs—as distinct from general factuality benchmarks?
• Are accuracy disclaimers displayed prominently (not in small or inconspicuous formatting) and consistently across all outputs, rather than only in certain contexts?
• Have we considered whether the conversational tone used by our AI product increases the risk of user over-reliance on potentially inaccurate information?
• Have we identified and mitigated the risk that inaccurate outputs could be used to make a decision affecting an individual?

5. Individual rights: Access, correction, and deletion

a. Access requests

AI developers should ensure that individuals can inform themselves of the existence, use and disclosure of their personal information without unreasonable effort. Self-service export tools may be helpful, but they should be accessible, user-friendly and complete enough to explain what personal information is held, used or disclosed. Self-service access data extract tools do not provide a blanket answer to access obligations, where they do not address potential disclosure to third parties, and provide technical and confusing outputs.

• Upon request, can users access the existence, use, and disclosure of their personal information?
• Do we provide an account of the use that has been made or is being made of this information and an account of the third parties to which it has been disclosed?
• Is information relating to the management of personal information, including the means to gain access to personal information held by the organization, readily available, easily accessible, and generally understandable?
• Do our data export tools provide information in accessible formats that allow individuals to understand what personal information is held?
• Do our data export tools cover all categories of personal information held?
• Do we clearly communicate the option to challenge an access response?

b. Correction and deletion

AI developers should provide procedures that allow individuals to challenge the accuracy and completeness of personal information and have it amended as appropriate, and should permit individuals to withdraw consent and seek deletion where applicable. Where model architecture makes true correction, unlearning or deletion difficult or impossible, developers should implement pragmatic alternatives such as blocklists and output filters for individuals’ verified personal information. Certain measures may be taken—including blocking mechanisms, output filtering, and exclusion from future training datasets—that may satisfactorily reduce the risk of disclosing individuals’ personal information.

• Do we provide procedures for individuals to access and correct any information collected about them during their use of the system, and develop processes to permit individuals to exercise their ability to access or correct personal information contained in an AI model?
• Where the architecture of our models prevents true correction of inaccurate personal information, have we implemented pragmatic alternatives such as blocking mechanisms and verified-information blocklists?
• Where deletion from a trained model is not technically feasible, have we implemented measures to prevent the individual's information from appearing in outputs and to filter it from future training runs?
• Can we verify the identity of individuals making correction or deletion requests? Have we documented any gaps in our ability to do so?

6. Data retention

AI developers should establish and implement retention and disposal policies, including retention schedules for training datasets, system prompts and outputs, and should destroy, de-identify or anonymize personal information once it is no longer required to fulfil identified purposes. Retaining unstructured raw data indefinitely for successive model iterations increases risks, especially where datasets may include sensitive, inaccurate or outdated information.

However, for PIPEDA purposes, the OPC indicated that it may be acceptable to retain data as a historical benchmark for scientific integrity purposes, provided that the information is segregated and stored within a secure archive with limited access, strong protections are in place to ensure the data is not used for other purposes including model development, data subject rights continue to apply to segregated datasets, and the organization regularly re-evaluates whether it needs to retain each dataset. The CAI found that individuals must be informed prior to collection of all purposes for dataset retention, including historical reference and scientific integrity purposes, and personal information should be anonymized once collection purposes have been fulfilled.

• Do we destroy, de-identify, or anonymize personal information (as required under applicable legislation) once it is no longer needed for the identified purposes?
• Do we maintain and follow appropriate retention schedules for personal information, including data contained in training datasets, system prompts, and outputs?
• Do these schedules both limit retention of information that is no longer required and ensure data is retained long enough for individuals to exercise access rights?
• Do we have a formal, documented retention and deletion policy that specifically addresses personal information in training datasets, including unstructured data collected from publicly accessible sources?
• Have we established defined retention periods for all categories of training data, rather than retaining data indefinitely for successive model iterations?
• Have we assessed the heightened breach risk associated with indefinite retention of datasets that may include sensitive, inaccurate, or outdated information—such as webpages removed from the internet since being scraped?
• For organizations subject to Québec's Private Sector Act: have we provided users with proper notice regarding retention for all purposes for which we retain datasets? If not, are we anonymizing personal information once the original collection purposes have been fulfilled?

7. Accountability and governance

Accountability requires AI developers to designate one or more individuals to oversee compliance and to implement policies and practices that give effect to the applicable Acts. Developers should have privacy management, accuracy assessment, consent, retention and mitigation measures in place before deployment, rather than relying on remedial steps after harms arise.

• Have we designated one or several individuals, to oversee our organization’s compliance with the Acts and implement policies and procedures to that effect?
• Have we established compliance processes and practices, including accuracy assessments, retention policies, and consent mechanisms, before deploying our AI product?
• Have we completed privacy impact assessments for all material data processing activities associated with our AI product?
• Do we respond to identified privacy compliance issues promptly, in a manner that demonstrates accountability?

8. Third-party AI deployers

Organizations that deploy third-party AI tools—rather than developing their own models—remain accountable for personal information processed through those tools. Under PIPEDA and provincial statutes, an organization that transfers personal information to an AI service provider must use contractual or other means to ensure comparable protection. Deployers should conduct due diligence on their AI vendors’ privacy practices, including training data sources, consent mechanisms, and data retention policies.

• If we integrate third-party AI tools into our products or services, have we conducted due diligence on the provider’s privacy practices, including their training data sources, consent mechanisms, and compliance with Canadian privacy law?
• Do our agreements with third-party AI providers clearly allocate responsibility for privacy compliance, including responding to access, correction, and deletion requests?

Read the original article on GowlingWLG.com