Mislearning and Overconfidence
Modeling how agents learn the wrong lessons and trust them too much#
Together, they explain why intelligent agents persist in failure.
Purpose#
This module exists to:
- model false learning trajectories
- explain confidence divorced from competence
- capture institutional and cultural blind spots
- support delayed collapse and sudden failure
- prevent agents from converging on truth by default
Mislearning is not a bug.
It is a structural feature of bounded cognition.
Mislearning as Substrate Expression (S / E / R)#
Structure (S)#
- flawed mental models
- brittle heuristics
- institutional dogma
- narrative shortcuts
Activation (E)#
- success reinforcement
- stress‑induced simplification
- social validation
- threat‑driven certainty
Relational Time (R)#
- reinforcement lag
- delayed feedback
- generational transmission
- error accumulation
Mislearning compounds quietly over time.
Common Mislearning Pathways#
1. Success‑Based Mislearning#
Mechanism: early success reinforces incorrect causal models
Outcome: confidence grows faster than understanding
Winning for the wrong reason is dangerous.
2. Overgeneralization#
Mechanism: narrow lessons applied too broadly
Outcome: brittle strategies fail under novelty
What worked once becomes doctrine.
3. Identity‑Protected Error#
Mechanism: beliefs shielded from correction by identity
Outcome: evidence is reinterpreted or ignored
Error becomes loyalty.
4. Socially Reinforced Falsehood#
Mechanism: group validation outweighs feedback
Outcome: synchronized mislearning
Groups can learn together — incorrectly.
5. Institutional Lock‑In#
Mechanism: procedures persist despite mismatch
Outcome: adaptation lags reality
Institutions remember success longer than relevance.
Overconfidence Dynamics#
Overconfidence increases when:
- feedback is delayed
- failure is externalized
- dissent is punished
- narratives explain away anomalies
Confidence is cheaper than correction.
Mislearning and Collapse#
Mislearning contributes to collapse by:
- masking early warning signals
- narrowing perceived option space
- accelerating commitment to failing paths
- delaying corrective action
Collapse often arrives after peak confidence.
Mislearning Metrics (Simulation Hooks)#
Trackable indicators include:
- confidence‑accuracy divergence
- error persistence rate
- dissent suppression index
- narrative rigidity
- correction latency
These metrics predict fragility under shock.
Failure Modes#
Mislearning modeling fails when:
- agents always self‑correct
- confidence tracks accuracy
- dissent is always effective
- institutions abandon doctrine easily
Error must be sticky and rewarded.
Integration Notes#
Mislearning and overconfidence:
- distort learning curves
- harden identity
- polarize social interaction
- delay regime transition
This module explains why systems fail loudly after succeeding quietly.
Status#
Canonical mislearning and overconfidence framework for cognitive agent simulation.
Designed for individual, institutional, and civilizational agents.