Day 3 — Targeting, Experimentation & Algorithmic Bias

The situation

Alex Campbel, CEO of Artea, knows that 87% of website visitors never buy anything. She wants to know whether a 20% off coupon changes that.

She cannot just send coupons and see what happens.

She runs an A/B test.

Why does she need a control group?

Average outcomes by experimental condition

5,000 customers. Half received the coupon.

The coupon increased transactions.

The coupon did not increase revenue.

Are you confident in both findings?

Does the coupon effect differ by how the customer found Artea?

Coefficients show treatment effect relative to Google baseline. Source: regression with interaction terms, AB_test data.

Facebook and Instagram customers respond. Google customers do not — and their revenue goes down.

Does having an abandoned cart change how customers respond?

29% of the 5,000 customers had abandoned a cart.

For customers with abandoned carts, the coupon works on both transactions and revenue. For everyone else, it barely moves the needle.

What you did — in two phases

Phase 1: Training

You had 5,000 customers with known outcomes. You used that data to find patterns — which customers responded to the coupon, and which didn’t.

The AB_test tab was your training data. You were the algorithm.

Phase 2: Inference

You took the rule you learned and applied it to 6,000 different customers — customers with no outcome data, because they were never in the experiment.

The Next_Campaign tab was your test set. You were making predictions, not observations.

This is the structure of every supervised learning problem: learn a function on labeled data, apply it to unlabeled data.

Standard policies ranked by revenue

Source: New Campaign Instructor tab, counterfactual outcomes.

What you had

AB_test — 5,000 customers Behavioral data: acquisition channel, past purchases, cart status, browsing time. Outcomes after the experiment. No demographics. This is what you used to find patterns.

Next_Campaign — 6,000 customers Same behavioral variables. No outcomes. No demographics. You added a 1 or 0 to each row. This is what you applied your rule to.

What both datasets share:

Neither contained gender. Neither contained race or ethnicity. Not because Artea doesn’t have customers with those characteristics — but because that data was held separately and was not used for targeting.

The business case

Artea’s customer base is 58% female and 21% minority.

The two best-performing policies systematically undertarget both groups — particularly minorities, who receive coupons at roughly one-third their population share under policy 3b.

These are not low-value customers to exclude. They are the majority of the customer base.

A promotional strategy that concentrates offers on a narrow demographic slice creates three business risks:

• Loyalty erosion among the majority who never receive offers
• Brand perception among customers who notice the pattern
• Concentration of discount dependency in a narrow segment, increasing margin risk over time

Optimizing for short-term revenue lift is not the same as optimizing for long-term customer value.

Is this fair?

Your best policy gave coupons to 42% of female customers and 7% of minority customers.

The baseline: 58% female, 21% minority.

No one used gender or race as a variable. The algorithm optimized revenue.

Independence

\[D \perp A\]

Prediction is independent of group membership.

Plain language: Equal outcomes across groups.

Decision question: Is the probability of receiving a coupon the same regardless of gender or minority status?

Also called: Demographic parity

Separation

\[D \perp A \mid R\]

Prediction is independent of group, given true outcome.

Plain language: Equal errors across groups.

Decision question: Among customers who would have responded to the coupon, are they missed at the same rate regardless of group? Among those who would not respond, are they wrongly targeted at the same rate?

Also called: Equalized odds

Sufficiency

\[R \perp A \mid D\]

True outcome is independent of group, given the prediction.

Plain language: Scores mean the same thing for everyone.

Decision question: Among all customers assigned the same predicted response probability, do they actually respond at the same rate regardless of group?

Also called: Calibration

Independence \(D \perp A\)

\[P(D=1 \mid A=female) = P(D=1 \mid A=male)\]

A model satisfies independence if the probability of receiving the offer is the same across groups. If 58% of customers are female, this implies roughly 58% of coupons should go to female customers.

This is the only criterion you can evaluate without knowing who would have responded.

Separation \(D \perp A \mid R\)

\[P(D=0 \mid R=1, A=female) = P(D=0 \mid R=1, A=male)\]

A model satisfies separation if, among customers who would respond to the offer (\(R=1\)), the share who were never given it is the same across groups.

\(R=1\) means: this customer would respond to the offer — they are a persuadable customer. This is the causal outcome: response to the treatment.

In Artea: Among customers who would respond to the coupon, the share who were never targeted should be equal across gender.

Requires an experiment — \(R\) is not observable in deployment data without randomization.

Sufficiency \(R \perp A \mid D\)

\[P(R=1 \mid D=1, A=female) = P(R=1 \mid D=1, A=male)\]

A model satisfies sufficiency if, among customers who received the offer (\(D=1\)), the response rate is the same across groups.

When we condition on \(D=1\), \(R=1\) is directly observable as “actually responded” — no experiment needed. The offer either worked or it did not.

In Artea: Among customers who were offered the coupon, the share who actually bought should be the same for female and male customers.

Female customers respond at 12.3%. Male customers at 9.3%. The base rates differ.

Sufficiency requires that a coupon means the same thing for everyone. But if women buy more often, sending coupons to women will always “work better” for women than for men.

Separation requires missing equal shares of buyers across groups. But a model that maximizes response will concentrate on groups with higher base rates — which mechanically creates unequal miss rates across groups.

Independence requires sending equal shares of coupons to each group. But forcing that equal split means ignoring who actually responds — which reopens the other two gaps.

Each fix creates a new violation. That is not a design flaw. It is arithmetic.

Chouldechova (2017) and Kleinberg et al. (2016) proved this formally. The Artea data shows it empirically.

What happened

Goldman Sachs issued Apple Card in August 2019. The credit algorithm assigned limits based on individual credit history, income, and spending patterns — not gender.

In November 2019, programmer David Heinemeier Hansson tweeted that Apple Card had given him a credit limit 20× higher than his wife — despite filing joint taxes and her having the higher credit score.

Apple co-founder Steve Wozniak reported the same disparity.

The New York Department of Financial Services launched an investigation.

Goldman’s response:

“We do not know your gender during the Apple Card application process.”

What happened

Hospitals and insurers used a widely deployed algorithm from Optum (UnitedHealth) to identify patients who needed high-risk care management.

The algorithm was race-blind. Race was not an input.

A study published in Science (Obermeyer et al., 2019) found that at identical risk scores, Black patients were 26% sicker than white patients — yet the algorithm ranked them as equally low-risk.

Estimated impact: the number of Black patients identified for additional care was reduced by more than half.

What happened

Twitter used a saliency algorithm from 2018 to automatically crop images in the timeline — showing the most “interesting” part of a photo without requiring users to click through.

In September 2020, users noticed the algorithm consistently favored white faces over Black faces in cropped previews. The thread went viral. Twitter investigated.

In May 2021, Twitter published its own findings and decommissioned the algorithm.

Yee, Tantipongpipat & Mishra (2021), Twitter ML Ethics team.

What the internal audit found:

• 4% difference from parity favoring white individuals over Black individuals in cropped previews
• 8% difference from parity favoring women over men
• Evidence of “male gaze” bias: algorithm cropped to women’s bodies rather than faces in some images

In Week 1 we built this framework:

Perception – raw input from the environment

Representation – how inputs are encoded for computation

Model – the function learned from data

Constraints – rules that limit allowable outputs

Behavior – what the system does in the world

Where each failure entered

The representation defines what the model can see. What it cannot see, it can still learn through correlation. What the constraints do not specify, the policy will not enforce.

Do not ignore protected attributes

Excluding a variable does not remove its effect. Correlated behavior carries it back in.

Make the tradeoff explicit

Equal outcomes, equal errors, equal meaning. You cannot have all three.

Measure what you are doing

You cannot audit what you do not observe. Check who is helped — and who is missed.

Fix the right problem

Bias can come from the data, the target, or the policy — not just the model.