Why the customer most likely to buy may be the wrong person to discount

A retailer wants to send a £5 offer. Its AI identifies customers who are very likely to buy, and the campaign produces plenty of orders. That sounds encouraging until someone asks how many of those customers would have bought anyway. A discount given to an already committed buyer can reduce the return from a sale without creating one.

Causal machine learning addresses that second question. EconML, an open-source project for estimating how effects differ across people or situations, uses interventions such as discounts as a motivating example. A predictive model estimates an outcome under the conditions represented in its data. An effect model tries to estimate how changing a particular condition would change that outcome.[1]

Two successful-looking campaigns, different economics

Consider this fictional experiment. A shop divides customers into two groups using information recorded before the offer: regular and occasional buyers. Within each group it randomly assigns 500 people to receive a £5 discount and 500 to receive no offer. It counts whether each person places one order during the next seven days. All figures below are invented for this illustration.

The regular buyers are far more likely to order even after the discount. But the occasional buyers show the larger estimated response to receiving it: ten percentage points rather than four. Selecting people by their predicted purchase rate would favour a different group from selecting them by the estimated change the offer produces.

Now suppose an order contributes £20 after its ordinary costs, before the discount. Every buyer who receives the offer redeems it, reducing that contribution to £15. Assume one order per customer, equal order values, no additional campaign costs and no later effects. These deliberately simple assumptions let us isolate the immediate arithmetic.

For regular buyers, the contribution from 20 additional orders is overwhelmed by discounts on purchases the comparison suggests would have happened anyway. The relevant quantity is therefore incremental value per offer, not simply the number of recipients who buy. Work by Daniel Baier and Björn Stöcker on profit uplift similarly distinguishes economic value from purchase and revenue outcomes; costs such as returns affect the calculation.[5]

How a model learns a change it cannot directly observe

There is a fundamental difficulty: the same person cannot both receive and not receive the same offer in the same situation. Athey and Imbens explain this missing-counterfactual problem in their work on causal trees. Random assignment makes groups comparable on average, allowing their outcomes to support an effect estimate. It does not reveal exactly which particular people were persuaded.[2]

One approach is a T-learner. As EconML’s documentation explains, it fits one model to outcomes among treated people and another to outcomes among controls. For a customer’s characteristics, it predicts both outcomes and subtracts one from the other. Flexible machine-learning methods can represent differences that a single overall campaign average would conceal.[3]

That flexibility creates another challenge: a pattern discovered in a sample can be noise. Athey and Imbens’ honest causal-tree method separates the data used to discover customer groups from the data used to estimate their effects. This limits the optimism that can arise when a method searches for striking differences and then uses those same observations to report how striking they are.[2]

The resulting prediction is an estimated average effect for people with particular observed characteristics. Calling an individual a certain persuadable buyer would imply knowledge the experiment does not supply. A useful marketing score can still be uncertain about the next person who receives the offer.[3]

The data must describe the decision you want to make

Public data make the distinction tangible. Criteo released an uplift-prediction dataset built from advertising incrementality experiments, with a randomly selected part of the population prevented from being targeted. Its corrected release contains 13,979,592 rows. The earlier release had a confounding problem: information in the features could reveal the advertiser, whose incrementality differed across campaigns. Criteo explains the issue in an erratum.[6]

The release also uses anonymisation and non-uniform subsampling. Criteo explicitly says these transformations prevent recovery of the original advertising incrementality. That makes it a useful research dataset without making it a direct estimate of a retailer’s likely return. We examined the documentation; we did not download the data or run a targeting model.[6]

Validation is harder here than checking whether a sales forecast matched an observed sale. EconML’s guidance stresses that the true counterfactual is unavailable and that observational data cannot generally prove away unobserved confounding. If customers who received an offer already differed in an unrecorded, relevant way, a sophisticated model may still attribute that difference to the offer.[4]

The unit of comparison matters as well. In our experiment, the treatment is receiving an offer. Comparing people who redeem a coupon with everyone who does not would discard the random assignment: redemption happens after the offer, and buyers select themselves into it. Keep the randomly assigned groups when estimating the effect of sending the campaign.[2]

The shop’s decision is now more precise. It wants an offer that changes purchases enough to justify its cost, with evidence that survives testing on new customers. The AI contribution is to help find where that change may occur. Even a highly accurate prediction of who will buy cannot, by itself, answer whether giving them a discount was worthwhile.

Report an error or suggest a correction