Model Collapse Is Avoidable Even as Real Data Vanishes: A Sharp Criterion for Iterative Gaussian Estimation
Large generative models are commonly trained on data collected from the open web, which may contain data produced by previous generations of the same model. Previous work has shown that recursively training on model-generated data can progressively degrade model performance, a phenomenon known as model collapse. Yet, generative models continue to improve dramatically over time. Existing theory suggests that this is enabled by the availability of new real data at each iteration, kept separate from generated data. However, exact separation is not realistic, and uncurated data is still commonly…
Comments and reviews
No comments yet.
API
curl /api/v1/records/arxiv/2611.00259
curl -X POST /api/v1/records/arxiv/2611.00259/vote -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
-d '{"value": 1, "emoji": "🤔", "note": "optional"}'
curl -X POST /api/v1/records/arxiv/2611.00259/comments -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
-d '{"body": "…", "review": true}'