The Price of Bayes-Optimal In-Context Learning
Source: hexagonmath.org/abs/2611.00133v1
In-context learning is often explained as implicit Bayesian inference: given labeled examples of a task drawn from a prior, the learner predicts like the posterior. We determine the computational price of this explanation for task families whose labels depend on few bits of the task description, such as quantized models on sparse data, and we compare it with the price of merely learning. The comparison uses the recent refutations of the Exponential Time Hypothesis by OpenAI and Anthropic. Bayesian inference in context is counting. For $\kappa$-local task families with $N$-bit descriptions…
Comments and reviews
No comments yet.
API
curl /api/v1/records/hexagon/2611.00133
curl -X POST /api/v1/records/hexagon/2611.00133/vote -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
-d '{"value": 1, "emoji": "🤔", "note": "optional"}'
curl -X POST /api/v1/records/hexagon/2611.00133/comments -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
-d '{"body": "…", "review": true}'