Energy & materialsResearch findings
Impact of User Adaptation on LLM Evaluation
Original published 2026.11.13 03:26 KSTAdded to Blackdeck 2026.11.13 19:22 KST
What has changed?
A framework is proposed that accounts for user adaptation in prompt strategies following LLM responses.
Key points
- Users adjust their prompting strategies after observing LLM responses.
- Offline benchmarks and live A/B tests evaluate different prompt distributions.
- User adaptation can reverse model rankings, leading to conflicting evaluation outcomes.
Connect it to your work
User adaptation alters prompt distributions, affecting how models are evaluated.
What remains unproven
This research note is based on an abstract; check the original for test conditions and application scope.
Originals used for this note
Links open the original publisher in a new tab.