Ontario Human Rights Commission: Warns Surveillance Pricing Could Violate Code

• The Ontario Human Rights Commission warns that "surveillance pricing" could violate the Human Rights Code.
• Surveillance pricing uses AI and personal data to set individualized prices, potentially charging consumers differently for identical products.
• Violations could occur if pricing relies on protected characteristics like race, sex, disability, age, or family status.
• The Commission emphasizes that discriminatory impact, not intent, determines if a practice violates the Code.
• The OHRC has released resources, including an AI Impact Assessment, to help organizations mitigate discrimination risks from AI systems.

The Commission specifically articulated that surveillance pricing could breach the Human Rights Code if businesses utilize data linked to protected characteristics to impose elevated prices.

The Ontario Human Rights Commission (OHRC) has issued a significant warning regarding "surveillance pricing," cautioning that this emerging practice could contravene the province's Human Rights Code. This pricing model, which leverages artificial intelligence (AI) and consumers' personal data to determine individualized prices, raises serious concerns about potential discrimination. The OHRC highlighted that retailers might employ automated algorithms to assign varying costs for identical goods based on an individual's personal circumstances and observed behaviors, often without the consumer's awareness.

This approach differs fundamentally from traditional "dynamic pricing," which adjusts prices primarily in response to market demand fluctuations. Surveillance pricing, conversely, utilizes specific personal information to set prices in real-time, creating a scenario where consumers may be unable to compare prices effectively or even ascertain if others are being charged differently for the same item. The Commission's intervention underscores a growing apprehension about how advanced data analytics and AI could inadvertently lead to unfair practices.

The Commission specifically articulated that surveillance pricing could breach the Human Rights Code if businesses utilize data linked to protected characteristics to impose elevated prices. These protected grounds include, but are not limited to, race, sex, disability, age, and family status. A critical aspect of the Commission's stance is its emphasis on discriminatory impact rather than the business's intent. This means that even if a company does not deliberately aim to discriminate when implementing surveillance pricing, the practice could still lead to human rights violations if its effects are discriminatory.

To illustrate these concerns, the Commission presented several hypothetical scenarios. One example involved a woman potentially being charged higher fares for ride-sharing services late at night compared to men. Another scenario depicted a person with a mobility disability facing increased charges for food delivery services. A third hypothetical outlined an Asian consumer being asked to pay more for decorative items in the period leading up to Lunar New Year. These examples underscore the potential for AI-driven pricing to inadvertently penalize individuals based on their protected attributes.

In response to the evolving landscape of technology and its intersection with human rights, the Ontario Human Rights Commission has intensified its efforts to examine how emerging technologies, including surveillance pricing and other AI applications, align with existing human rights protections. This proactive approach reflects the Commission's mandate to address new risks and prevent discrimination as organizations increasingly explore the benefits of artificial intelligence.

As part of these efforts, the OHRC, in collaboration with other prominent human rights, privacy, and law reform organizations, has published key resources. These include a Human Rights AI Impact Assessment and a set of Principles for the Responsible Use of Artificial Intelligence. These documents are designed to offer practical guidance, assisting organizations in identifying and mitigating potential discriminatory outcomes from AI systems while simultaneously upholding privacy standards and human rights. The Commission strongly advises Ontario businesses and organizations considering the use of personal data or AI technologies to consult these resources to ensure full compliance with the Human Rights Code.

Source: Original reporting via Ontario Human Rights Commission.