You can have an excellent churn model, an excellent value model and an excellent recommender, and still leave the product team staring at three numbers with no idea what to actually do for the player in front of them. Scores are not decisions. A decisioning layer closes that gap by ranking the next best action for a player by its expected value, and, this is the part worth reading for, it knows the difference between an assumption and a learned fact and refuses to let the first wear the costume of the second. This note is about both halves: the economics, and the honesty in the middle of the economics.
From scores to a decision
For any inbound player event, the layer assembles the candidate actions, such as serving a personalised recommendation, extending a retention offer, sending a light-touch re-engagement nudge, or doing nothing, and scores each by a single, transparent expected value: the probability the action succeeds multiplied by the value at stake, minus the cost of acting. Every term is sourced, not invented. The probability of success comes from the player's own precomputed model scores; the value and cost come from a small, editable table of action economics that the product team owns.
A retention offer aimed at a high-value player at genuine churn risk carries a large positive expected value; the identical offer aimed at a low-value, low-risk player carries a negative one, because the cost outweighs the tiny expected gain, and so it is correctly not chosen. Actions are ranked by expected value, the top positive one becomes the recommendation, and every action carries both its computed expected value and a plain-English reason for it. This replaces the usual sprawling pile of hand-tuned priority rules with an economic calculation a product manager can read, audit and adjust by editing the economics table. And because the same ordering is wired into the real-time serving path, not just a console view, the decision a stakeholder sees explained on screen is the same one the live system is making.
Pure expected-value maximisation is not, however, the whole story, and pretending it were would be its own kind of dishonesty. Some actions are imperatives: they must fire when their condition is met regardless of what the arithmetic says, because they encode a policy the business or the player's wellbeing requires rather than a revenue bet. A wellbeing-oriented check-in for a player showing signs of disengagement is not something you skip because a flashier offer scored a few cents higher. So the layer treats those as a separate, higher tier: imperatives are surfaced first, and the expected-value ranking orders everything that remains. The economics decide the discretionary actions; they are not allowed to override the non-negotiable ones.
The honesty problem in the middle of it
Now look again at the expected-value formula. The value and the cost are known quantities. But the effect of the action, how much a retention offer actually moves the outcome versus doing nothing, is a causal quantity, and you cannot read a causal effect off historical scores no matter how good the scores are. Players who received offers in the past were not a random sample; they were selected, often by exactly the kind of rule this system is replacing. The easy, dishonest move is to bury a made-up uplift multiplier in the formula and present the polished result as if it were precise.
We do the opposite, and we do it visibly. The treatment effect starts life as an explicitly labelled, population-level assumption, a single tunable constant, and every action's output carries a source field that at this stage reads as assumed. Nothing about the decision pretends to be a per-player causal estimate when it is in fact a platform-wide guess. A stakeholder reading the output can see, in the output itself, that this number rests on an assumption rather than a measurement.
The flywheel that earns its learning
The entire point of labelling the assumption is that it creates a gap the platform can then close with data, honestly, and only when the data is real. Every served decision is logged as a treatment; a fraction of eligible players are deliberately held back as a control; and when the real outcome is later known, the customer reports it back through a feedback endpoint. That is the raw material for an actual, measured uplift estimate rather than a guessed one.
A label-gated learner then does the obvious, honest thing: it computes the uplift of an action as the success rate among treated players minus the success rate among the held-back control, per action, but only once both arms have accumulated enough labelled outcomes to make the difference meaningful, a per-arm minimum so that a handful of lucky outcomes cannot flip the decision. Below that threshold the learner returns nothing and the decisioning falls back to the assumed constant, with the source field still reading assumed. Above it, the learned value takes over and the field flips to learned.
Critically, the learner is dormant until real treated-and-control outcomes accrue; it never trains on synthetic or back-filled data to look clever on day one. On a fresh deployment it is honestly inert, and as the flywheel turns, decide, hold out a control, observe outcomes, measure the gap, it wakes up on its own, action by action, as each one earns enough evidence. The naive version of AI decisioning fabricates a causal number, ships it and hopes the results roughly work out. The honest version ships a transparent baseline, tells you in the output that it is a baseline, builds the experimental apparatus to measure the real effect, and only swaps in the learned number once the data has genuinely earned the upgrade. One of these you can put in front of a skeptical stakeholder and defend line by line; the other you cannot.