A Grok and a Hard Place: Navigating Canadian Privacy Scrutiny

Overview

On January 15, 2026, the Office of the Privacy Commissioner of Canada (“OPC”) launched an investigation against X Corp., the operator of the platform X, and X.AI LLC (“xAI”, together with X Corp, “X”), the developer of the artificial intelligence (“AI”) chatbot Grok. The investigation began due to multiple reports to the OPC alleging that Grok had been responsible for generating and publicly releasing millions of sexually explicit deepfakes depicting real and identifiable individuals.

The OPC’s investigation ultimately found that X failed to obtain valid consent for the collection, use and disclosure of personal information and that a reasonable person would deem this use of personal information by Grok inappropriate. 

Following its investigation, the OPC made several recommendations to X to bring them into compliance with Canada’s Personal Information Protection and Electronic Documents Act (“PIPEDA”). X disagreed with the OPC’s conclusions and recommendations, claiming they had taken all reasonable steps to implement safeguards aimed at reducing the risk of misuse of their services. Absent any demonstration of effectiveness of these safeguards, the OPC concluded it will continue to monitor the implementation of safeguards promised by X as this matter has not been resolved.

Background

Grok is a multimodal generative AI system powered by a large language model capable of generating text, images, code and other content in response to user prompts. It is accessible through three access points, including Grok on the X platform, the @Grok account operated by xAI, and as a standalone app.

Grok’s image-generation capabilities evolved rapidly between 2024 and 2025, transitioning from requiring a third-party AI model to an in-house AI model “Aurora”, and later to an enhanced image generation and editing model called Grok Imagine. Grok Imagine was made available for use within standalone Grok, Grok on the X platform and the @Grok account. In late December 2025, X introduced a feature allowing users to edit images directly on the X platform, including editing images posted by other users.

Shortly thereafter, users began exploiting this tool by using prompts designed to evade safeguards, enabling the generation of explicit, sexualized deepfakes of identifiable individuals. These deepfakes often targeted children and women, leading the @Grok account to publish a public apology for a circulating image generated by its services of two young girls in sexualized attire.

Key issues

The OPC sought to examine the following two issues:

1. Whether X obtained valid consent for the collection, use and disclosure of personal information for the purpose of generating sexualized deepfakes; and
2. Whether a reasonable person would consider the collection, use and disclosure of personal information for the purpose of an image generation service capable of generating sexualized deepfakes, to be appropriate in the circumstances.

Outcomes

The OPC concluded that X did not obtain valid consent for the collection, use and disclosure of personal information for the purpose of generating sexualized deepfakes.

Under PIPEDA, knowledge and consent are required for the collection, use and disclosure of personal information, which is defined by PIPEDA as “information about an identifiable individual”, therefore including deepfakes regardless of whether the image is accurate or not.

PIPEDA additionally provides that the form of consent required by an organization for the collection, use and disclosure of personal information may vary based on the sensitivity of the information, the individual’s reasonable expectations, and the individual’s reasonable expectation to understand the nature, purpose and consequences of the collection, use or disclosure of the personal information to which they are consenting.

Organizations must obtain express consent where the information is sensitive, the practice of obtaining the information falls outside the reasonable expectations of the individual, and where the collection of the information may create a risk of significant harm. The OPC ultimately found that X did not obtain valid consent and contravened PIPEDA for the reasons below.

• Sensitive information: Sexualized deepfakes contain sensitive information, and as such information may pertain to an individual’s sex life or sexual orientation. Additionally, young people’s personal information is considered sensitive and in need of higher protection.
• Reasonable expectations: Individuals would not reasonably expect sexualized images of themselves to be posted on social media platforms by others, particularly when the original images were posted for entirely unrelated purposes. The OPC finds this especially true where those deepfakes pertain to minors. 
• Risk of significant harm: The generation and publication of sexualized deepfakes without the knowledge and consent of the depicted individuals can create a risk of significant harm to them. Such harm may include reputational, psychology, or financial.

The OPC concluded that a reasonable person would consider the collection, use and disclosure of personal information by Grok in these circumstances to be inappropriate.

Further, under PIPEDA, an organization can collect, use or disclose personal information only for purposes that a reasonable person would consider are appropriate.

The OPC considered the sensitivity of the personal information involved, whether the organizations had a legitimate need or bona fide business interest for its image generation services, the availability of less privacy-invasive alternatives, and whether the invasion of privacy was proportionate to the benefits achieved.

The OPC came to the following conclusions accordingly:

• The personal information contained in sexualized deepfakes is considered sensitive, as discussed previously.
• Despite the OPC’s acceptance that the organizations’ purpose of their image and video generation represents a legitimate need/bona fide business interest, this purpose does not justify non-consensual sexualized deepfakes of identifiable individuals.
• The privacy risks associated with Grok’s image-generation tools could have been substantially reduced through more proactive design approaches. The organizations failed to adequately anticipate foreseeable misuse during the development stage.
• The loss of privacy and risk of significant harm to individuals arising from the organizations’ services significantly outweigh the benefits to the organizations.

Grok’s response and recommendations

In finding that X contravened PIPEDA, the OPC recommended a series of compliance measures including suspending Grok Imagine until adequate safeguards are demonstrated, strengthening privacy risk-management processes, undergoing independent audits, and scheduling ongoing monitoring. X disagreed with the OPC’s findings and argued that existing and newly implemented safeguards are sufficient. The organizations have committed to providing the OPC with quarterly reports on the effectiveness of their safety measures, and the OPC will continue to monitor the implementation of these commitments.

Key takeaways – What does this mean for AI users and/or businesses?

First, AI image-generation platforms are subject to rigorous safeguards, given the heightened risks associated with misuse of public visual content. Second, responsibility does not shift away from an AI platform merely because harmful outputs are user generated. The platform remains ultimately accountable for foreseeable misuse, given that the platform is responsible for collecting and disclosing individuals’ personal information. Third, where an organization has a legitimate or bona fide interest in providing its services, that interest will be outweighed if the potential harm from its services exceeds its produced benefits. Finally, regulatory compliance requires more than proposed or anticipated safeguards. There must be demonstrable evidence that protective measures are actively implemented, operational and effective in practice.

For more information about AI and compliance with Canada’s privacy legislation, please contact Roland Hung or Laura Crimi of Torkin Manes’ Technology and Privacy & Data Management Groups.

The authors would like to acknowledge Torkin Manes’ Summer Student Arielle Teperman for her invaluable contribution in drafting this bulletin.