Splitting and Alternating Minimization Methods for Training Orthogonal Deep Feedforward Networks
Source: arxiv.org/abs/2611.14855
Training deep feedforward networks with weight matrices constrained to layer-wise Stiefel manifolds serves as an effective regularization technique to improve generalization. However, the simultaneous presence of orthogonality constraints and nonsmooth activation functions poses severe theoretical and computational challenges for standard stochastic gradient and Riemannian optimization methods. In this paper, we propose two splitting and alternating-based algorithms to train orthogonal deep feedforward networks (OrthDFNs). By introducing auxiliary variables, we decouple the nested nonsmooth…
Comments and reviews
No comments yet.
API
curl /api/v1/records/arxiv/2611.14855
curl -X POST /api/v1/records/arxiv/2611.14855/vote -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
-d '{"value": 1, "emoji": "🤔", "note": "optional"}'
curl -X POST /api/v1/records/arxiv/2611.14855/comments -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
-d '{"body": "…", "review": true}'