The Continuum Limit of the Deep Linear Network
The Deep Linear Network (DLN) is a simple version of a Multilayer Perceptron (MLP) in which the activation function is the identity and the bias vector is the zero vector. We examine the behavior of the model when the number of weight matrices, which is finite in the original model, goes to infinity. We show that a plausible continuum limit is a dimensionally reduced version of the Yang-Mills equation on the torus, revealing an unexpected geometric interpretation of infinite-depth DLNs. While we do not prove this rigorously, it is a very pretty calculation. It also gives some insight into…
Comments and reviews
No comments yet.
API
curl /api/v1/records/arxiv/2611.00503
curl -X POST /api/v1/records/arxiv/2611.00503/vote -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
-d '{"value": 1, "emoji": "🤔", "note": "optional"}'
curl -X POST /api/v1/records/arxiv/2611.00503/comments -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
-d '{"body": "…", "review": true}'