Use cases
CounterFact evaluates repeated decision systems where actions, context, and outcomes can be joined from historical logs. It works best when teams make the same kind of decision many times, choose from a discrete set of actions, and can measure what happened afterward.
Ranking and recommendation
What it is
Ranking and recommendation systems decide what users see: which items appear first, which content is promoted, and which offers are surfaced. Teams often have many policy ideas and limited experiment capacity.
Typical data
User context, displayed items and positions, and engagement or conversion outcomes. The most common gaps are missing assignment probabilities and incomplete action-set information.
What CounterFact checks
Whether historical impression and outcome data is strong enough to support a credible offline comparison of a proposed ranking policy, or whether key logging gaps need to be fixed first. Ranking scenarios are planned for a future release.
Recommendation exampleRenewal and save-offer workflows
What it is
As a subscription or contract approaches renewal, a team or system chooses a retention action such as standard terms, a discount, escalation, or no intervention. These decisions repeat at scale and directly affect retention and margin.
Typical data
CRM history, billing data, product usage, support history, the action taken, and the renewal outcome. Common gaps include missing records of which actions were available, how the choice was made, and where manual overrides occurred.
What CounterFact checks
Whether the historical logs support a credible offline evaluation of a proposed renewal policy. That includes assignment reconstructability, outcome maturity, candidate-action visibility, and support for the proposed policy in past data. Available now.
SaaS renewal sample (well instrumented)More playground samples (other domains)
The same evaluation engine powers additional frozen CSVs. Scenario IDs are historical, but the logs are synthetic debt collections and support routing—not renewals.
Debt collections (partial logging)Support routing (minimal logging)Best fit for repeated decisions with discrete actions, measurable outcomes, and historical logs.