Part I

The Repeat-Run Volatility Rule

Ask the exact same question twice and about 50% of the recommended brands change. In software specifically, identical repeat runs retained 63–73% of brands — meaning roughly a third of the set differed.

~50%

of brands change across identical repeat runs (all industries)

27–37%

of the recommended-brand set differed on identical software queries

Two forces drive change in AI recommendation lists:

Same query, new run — inherent model variance between runs.
Same intent, new wording — phrasing shifts roughly half the recommended brands.

Repetition tests reliability. Phrasing tests coverage. Measure both.

Ask the exact same question twice and roughly half the recommended brands change — AI recommendation is a probability game, not a fixed ranking.

Query A vs. Query B (identical text): the recommended-brand sets differ run to run.

Source: Fractl research: “Where AI Recommendations Actually Come From”