Bias in the Code: Algorithmic Fairness in Relationship Technology

Every AI in your love life learned from someone else's. Matching algorithms learn from who got swiped on; advice models learn from what people wrote online; tone models learn from labelled examples of what "angry" looks like. If those records tilt by race, gender, culture or dialect, the model tilts with them. This is not a hypothetical failure; it is the default one.

Key takeaways

Bias enters through data and design, not malice. Models reproduce the preferences and blind spots in their training data (Buolamwini & Gebru, 2018).
On dating platforms, design choices such as ethnicity filters and "similar to what you liked" recommendations can amplify race as a factor in who you see (Hutson et al., 2018).
Fairness is not one number. Reasonable definitions of it conflict, and choosing between them is a values decision, not a technical one (Mitchell et al., 2021).
A fair tool shows its evidence. Treat any relationship score as a prompt to look at the messages yourself, never as a verdict.

How bias gets in

Buolamwini and Gebru's Gender Shades study is the canonical example: commercial face classifiers that were near-perfect on lighter-skinned men had error rates of up to 34.7% on darker-skinned women, because the data they were built on under-represented them. The lesson generalises. A model's accuracy follows its data, and its errors land hardest on whoever the data left out.

Relationship technology has three places for that to happen: matching (who you are shown), advice (what a model tells you to do), and analysis (how a model labels your messages).

Matching

Hutson, Taft, Barocas and Levy (2018) reviewed the design of popular dating platforms and found that features like ethnicity filters, and recommenders trained on past swipes, can make race more salient in who gets seen, not less. Read the paper. An algorithm that learns "people like you swiped right on people like this" reproduces the aggregate preferences of everyone before you.

Advice

Advice models are trained largely on English-language text from particular cultures, so their default picture of a healthy relationship carries those norms: who should initiate, what counts as too clingy, how long a reply should take. The same models are also inconsistent. ChatGPT ranked relationship advice differently on repeat runs in a 2024 study.

Analysis

Tone and sentiment models are the ones most likely to touch your own messages. They are trained on labelled examples, and they misread whatever the labellers did not speak: dialect, sarcasm, code-switching, non-native phrasing, in-jokes. A model that scores "k." as hostile for everyone will be wrong for the couple who use it as a sign-off.

What fairness research says can be done

Pick a definition and say which. Mitchell and colleagues (2021) catalogue the choices behind every fairness metric and show that common ones, such as equal error rates across groups and equal calibration, cannot all hold at once. Read the review. A team that says "our model is fair" without saying which kind is not saying much.
Fix the data and measure by group. Chouldechova and Roth (2020) survey the technical toolkit: diversifying and reweighting training data, testing accuracy separately for each group, and constraining models during training. Read the survey
Look past the model. Selbst and colleagues (2019) argue that fairness fails when it is treated as a property of the algorithm alone; how a tool is deployed, who reads its output, and what they do with it are part of the system. Read the paper. For a relationship tool, the human reading the score matters as much as the score.

Five questions to ask any relationship AI

What was it trained on, and who is missing? If the answer is "the internet", assume the defaults of the loudest group.
Does it show its evidence? A score with no way to see the messages behind it is unfalsifiable. Prefer tools that put the timeline next to the number.
Does it treat both people the same? An analyzer that rates one partner's short replies as "disengaged" and the other's as "efficient" is applying a norm, not a measurement.
Can you correct it? If it misreads your dialect, sarcasm or shared shorthand, is there a way to say so, or does the verdict stand?
Who decides what healthy means? A model has a default. A relationship gets to have its own.

At MosaicChats we try to keep the second and last of those honest: the tone timeline and reply patterns are shown week by week so you can check a result against the conversation you remember, and Myrah, the assistant, is there to explain a result rather than hand one down.

Look at the evidence, not the score

MosaicChats shows the week-by-week tone and reply patterns behind every result, so you can judge them against the conversation you actually had.

Analyze your chat

Frequently asked questions

What is algorithmic bias in dating apps?

When a matching or recommendation system reproduces the preferences and blind spots in its training data, so certain groups are shown less, ranked lower, or filtered out. Hutson et al. (2018) found that design choices such as ethnicity filters and swipe-based recommenders can amplify race as a factor in who users see.

Can an AI relationship analyzer be biased against me?

Yes, most often through language. Tone and sentiment models trained on one dialect or culture can misread sarcasm, code-switching, non-native phrasing or a couple's private shorthand. Prefer tools that show the messages or timeline behind a score so you can check it.

How can I tell if AI relationship advice is biased?

Ask it the same question twice and compare; ask it to state the assumptions behind its answer; and notice whether its idea of a healthy relationship matches yours or a generic default. A 2024 study found ChatGPT's advice rankings barely matched human judgments and changed between runs.

Is there a standard for fair AI?

Not a single one. Fairness research offers several definitions, such as equal error rates or equal calibration across groups, and shows they cannot all be satisfied at once (Mitchell et al., 2021). Responsible teams pick one, say which, and measure accuracy by group.

References & Sources

Buolamwini, J., & Gebru, T. (2018). Gender shades: Intersectional accuracy disparities in commercial gender classification. Proceedings of Machine Learning Research, 81, 77–91. Source
Chouldechova, A., & Roth, A. (2020). A snapshot of the frontiers of fairness in machine learning. Communications of the ACM, 63(5), 82–89. Source
Hutson, J. A., Taft, J. G., Barocas, S., & Levy, K. (2018). Debiasing desire: Addressing bias & discrimination on intimate platforms. Proceedings of the ACM on Human-Computer Interaction, 2(CSCW), Article 73. Source
Mitchell, S., Potash, E., Barocas, S., D'Amour, A., & Lum, K. (2021). Algorithmic fairness: Choices, assumptions, and definitions. Annual Review of Statistics and Its Application, 8, 141–163. Source
Selbst, A. D., boyd, d., Friedler, S. A., Venkatasubramanian, S., & Vertesi, J. (2019). Fairness and abstraction in sociotechnical systems. Proceedings of the Conference on Fairness, Accountability, and Transparency, 59–68. Source

Bias in the CodeAlgorithmic Fairness in Relationships