AI & softwareResearch findings
MisleadQA: Diagnosing LLM Misleading Failures from Factually True but Out-of-Scope Context
Original published 2026.10.18 04:58 KSTAdded to Blackdeck 2026.10.20 13:03 KST
What has changed?
A dataset that identifies how factually correct but out-of-scope context misleads LLMs, even when the content is true.
Key points
- MisleadQA diagnoses how factually true but out-of-scope context misleads LLMs.
- Out-of-scope context (S) misleads 29.3% of cases, while in-scope false context (F) misleads 75.4%.
- Adding correct context (G) reduces S's effect significantly, but F still misleads in 21.0% of cases.
Connect it to your work
Factually true but contextually irrelevant information can mislead LLMs as effectively as false information.
What remains unproven
This research note is based on an abstract; check the original for test conditions and application scope.
Originals used for this note
Links open the original publisher in a new tab.