AI Innovation Meets Privacy Regulation: Key Takeaways from the OPC’s Investigation into OpenAI

Overview  

In November 2022, ChatGPT, a generative AI chatbot designed to allow users to input prompts and receive an AI-generated response, was first released to the public. While the platform has transformed how individuals and businesses interact with AI, it has also raised significant concerns regarding the collection, use and disclosure of personal information in the development and deployment of AI models. 

On April 4, 2023, the Office of the Privacy Commissioner of Canada (the “OPC”) announced that it had launched an investigation of Open AI OpCo, LLC (“OpenAI”), the company behind the AI-powered chatbot ChatGPT. On May 6, 2026, the OPC, together with the Commission d’accès à l’information du Québec (the “CAI”), the Office of the Information and Privacy Commissioner for British Columbia (the “BC IPC”), and the Office of the Information and Privacy Commissioner of Alberta (the “AB IPC”), (collectively, the “Privacy Commissioners” and individually each a “Privacy Regulator”), revealed their findings following a joint investigation into how OpenAI sourced training data for ChatGPT models GPT-3.5 and GPT-4, and whether those practices complied with applicable Canadian federal and provincial privacy laws.   

A Question of Jurisdiction

OpenAI initially questioned the Privacy Regulators’ authority to conduct an investigation given that it is a US-based company. However, Canada’s private sector federal privacy legislation,  the Personal Information Protection and Electronic Documents Act (“PIPEDA”), and equivalent private sector provincial privacy statutes, British Columbia’s Personal Information Protection Act (“PIPA-BC”) and Alberta’s Personal Information Protection Act (“PIPA-AB”), do apply to organizations operating outside of Canada where a “real and substantial connection” exists.

The Privacy Commissioners concluded that such a connection existed for several reasons, as follows:

• OpenAI offers services in Canada, including paid subscription services such as ChatGPT Plus;
• OpenAI’s Terms of Service apply to Canadian users;
• a physical presence in Canada is not required to establish a real and substantial connection; and
• OpenAI’s operations require the transmission and receipt of personal information between Canada and the United States in connection with the collection and disclosure of information through ChatGPT. 

The Privacy Commissioners’ Findings and Recommendations

Findings

While each Privacy Regulator investigated compliance with the specific applicable privacy law that it each oversees, they collectively identified several concerns regarding the manner in which OpenAI trained ChatGPT. More specifically, the Privacy Commissioners’ investigation examined the following issues:

Appropriate Purposes: The Privacy Commissioners accepted that OpenAI’s purpose of developing and deploying ChatGPT was appropriate, however, the Privacy Commissioners also evaluated OpenAI’s collection, use and disclosure of personal information collected from different sources. OpenAI represented to the Privacy Commissioners that it uses the following sources to train its ChatGPT model: (a) information from publicly available sources; (b) information that it licenses from third parties; (c) users’ interactions with ChatGPT; and (d) conversations generated by human AI trainers (both OpenAI’s employees and contractors). The Privacy Commissioners ultimately found that the manner in which OpenAI collected personal information from internet sources and third parties for the purposes of training ChatGPT was overbroad and contravened applicable law.

Consent and Notice: The Privacy Regulators found that OpenAI did not obtain valid consent for its collection, use and disclosure of personal information for the purpose of developing and deploying ChatGPT. The OPC, BC IPC and AB IPC concluded that OpenAI failed to obtain valid consent for the collection and use of personal information it obtained from publicly available sources; further, these Privacy Regulators found that OpenAI should have obtained express consent for the disclosure of personal information where it was either sensitive or outside the reasonable expectations of the individuals concerned. Additionally, the CAI concluded that OpenAI did not sufficiently document the context in which the duty to inform under applicable Quebec laws had been fulfilled or the context in which the consent of individuals concerned had been obtained.

Openness (Model Transparency): The OPC, BC IPC and AB IPC found that OpenAI did not meet the openness and transparency requirements under applicable privacy laws. Despite the fact that OpenAI had privacy communications to users that were readily accessible, the investigation concluded that key information was either incomplete or not clear.  

Accuracy: The OPC, BC IPC and AB IPC found that OpenAI failed to meet the accuracy requirements under applicable laws. Specifically, OpenAI failed (a) to provide sufficient information to users regarding the potential for a response generated by ChatGPT to be inaccurate; (b) to clearly inform users of the need to verify the facts provided by ChatGPT; and (c) to consistently provide a mechanism for users to verify those facts. 

Access, Correction and Deletion: The Privacy Commissioners collectively found that OpenAI failed to provide users with sufficient ability to access, correct and delete their personal information. OpenAI was found to not have used sufficient mitigation measures to comply with its access obligations under applicable law, specifically OpenAI was not able to verify when personal information related to a user requesting the access, correction or deletion of their data.   

Retention: The Privacy Commissioners found that OpenAI failed to establish appropriate retention and disposal policies for the personal information collected, used and disclosed for the purposes of training ChatGPT.  

Accountability: The Privacy Commissioners collectively found that OpenAI failed to meet its accountability obligations with respect to personal information under its control under applicable laws. The failure to be accountable exposed individuals whose personal information was at issue to potential harm, including breaches of personal information, inaccuracy of information, discrimination on the basis of accurate and inaccurate information, and other easily foreseeable social harms.

Recommendations

The Privacy Commissioners ultimately found that OpenAI deployed ChatGPT in a manner that contravened applicable privacy laws and made several recommendations with a view to allowing the development and deployment of generative AI technology in Canada in a sufficiently privacy-protective manner, including:

• Limiting Collection/Necessity: Develop a plan to limit the personal information used to train models to that which has been established to be necessary and proportional for the purpose for research and testing. Proposed measures included: the implementation of processes that minimize the collection of personal information for the purpose of training its models, including ceasing the collection of personal information from sources containing significant personal information (e.g., social media and discussion forums).
• Limiting collection of sensitive information via user interactions: Ensure that users are informed and can reasonably understand the consequences of disclosing sensitive personal information when interacting with ChatGPT (e.g. a prominent notice).
• Consent: Develop a plan for the implementation of measures to ensure that valid consent is obtained from individuals in Canada whose personal information is collected, used and disclosed for the purpose of training or deploying ChatGPT.
• Model Transparency: Provide the public with plain language, comprehensive and easily accessible information about (a) the categories of personal information used to build training data; (b) how models function; and (c) the existing limitations on model explainability.
• Accuracy: Implement measures to ensure that users are made aware of the current limitations of its models’ level of accuracy of personal information included in outputs.
• Access: Implement measures to ensure that the format of exported data is accessible and user friendly for the general public.
• Retention: Develop a formal retention policy for personal information about individuals in Canada to ensure that data is only retained as long as necessary for the identified purpose.
• Accountability: Implement accountability measures to address each of the above concerns by developing governance models with respect to the changes made to comply with the above noted recommendations.

OpenAI’s Response

In response to the investigation, the Privacy Commissioners made a number of recommendations aimed at bringing OpenAI into compliance with Canadian privacy laws.

Although OpenAI disagreed with portions of the Privacy Commissioners’ findings, it cooperated throughout the investigation and implemented several measures during its course, including:

• retiring the GPT-3.5 and GPT-4 models in July 2024 and July 2025;
• implementing filtering tools designed to detect and redact personal identifiers from user interactions used to fine-tune newer models;
• introducing a web search feature that conducts real-time internet searches to generate responses;
• adopting formal retention policies governing the retention and deletion of personal information; and
• implementing measures to ensure inactive datasets are no longer used in model development and are retained solely for historical benchmarking purposes.

OpenAI also committed to implementing additional privacy measures, including:

• publishing a bilingual Canadian blog post explaining its privacy practices in Canada in conjunction with the release of the report;
• expanding its “How ChatGPT and our foundation models are developed” article to be more user friendly;
• providing users with clearer notices that prompts may be used to train AI models and advise users not to input sensitive information;
• testing additional protective measures for minors; and
• providing quarterly reports to the Privacy Commissioners detailing its compliance efforts until all commitments have been satisfied.

Key Takeaways

The OPC ultimately concluded that the issues identified in its investigation of OpenAI and ChatGPT were well-founded and conditionally resolved based on OpenAI’s new mitigation measures. In contrast, the BC IPC and AB IPC, determined that OpenAI’s reliance on scraped data fails to meet foundational consent obligations under PIPA BC and PIPA AB, which contain more explicit consent requirements than PIPEDA. The BC IPC and AB IPC found that the measures taken by OpenAI in response to the investigation were not sufficient to meet the foundational consent requirements under PIPA BC and PIPA AB. The CAI found that several issues identified by the investigation to be resolved, but similar to the BC IPC and AB IPC maintained that the issues identified by the investigation with respect to consent and retention to be well-founded and unresolved. The CAI has undertaken to continue to monitor OpenAI’s implementation of the joint recommendation and CAI-specific recommendations moving forward. Across all jurisdictions, the Privacy Commissioners expect OpenAI to continue to strengthen its privacy protections and acknowledge that broader societal and ethical questions surrounding the deployment and development of generative AI will require an ongoing collective effort to shape a robust governance framework.

The investigation by the Privacy Commissioners serves as an important reminder that Canadian privacy regulators are actively scrutinizing the development and deployment of AI systems. Businesses should not assume that third-party AI platforms are automatically compliant with Canadian privacy laws. Organizations using AI tools should carefully assess:

• how personal information is collected, used and retained;
• whether confidential or sensitive information may be disclosed through AI systems; and
• whether appropriate safeguards and governance policies are in place.

As regulatory scrutiny surrounding AI continues to evolve, businesses should consider seeking legal advice when adopting or integrating AI technologies into their operations.

For more information about leveraging AI technology and privacy compliance in Canada, please contact Lisa R., Lifshitz, Roland Hung or Laura Crimi of Torkin Manes’ Technology and Privacy & Data Management Groups.

The authors would like to acknowledge Torkin Manes’ Articling Student Kayla Oliveira for her invaluable contribution in drafting this bulletin.