Fix Multiclass layer gradient

[andrewdalpino]

Softmax activation function is incompatible with any non-Cross Entropy-based loss function. Remove Softmax activation function. Subsume functionality into Multiclass layer - correct the gradient computation for non-Cross Entropy-based loss function such as Relative Entropy.

[Copilot]

🟡 Changes recommended

The updated Multiclass::gradient() reduces along the wrong axis for the Softmax Jacobian-vector product, and the new test data currently allows that incorrect implementation to pass.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR removes the standalone Softmax activation function from the NeuralNet API and moves Softmax behavior into the Multiclass output layer, aiming to fix backpropagation for non-cross-entropy classification losses (e.g., Relative Entropy/KL divergence).

Changes:

• Removed Softmax activation function (code, tests, benchmarks, and docs navigation/page).
• Implemented Softmax computation inside Multiclass and updated its gradient logic; added a new test covering a non-cross-entropy loss.
• Performed small formatting/doc touch-ups (initializers, docs, changelog) to reflect the removal.

File summaries

File	Description
tests/NeuralNet/Layers/MulticlassTest.php	Adds a new gradient test intended to cover Softmax-Jacobian behavior with RelativeEntropy.
tests/NeuralNet/ActivationFunctions/SoftmaxTest.php	Removes unit tests for the deleted Softmax activation function.
src/NeuralNet/Layers/Multiclass.php	Removes dependency on Softmax activation class, adds internal Softmax, and updates gradient computation.
src/NeuralNet/Initializers/Xavier1.php	Updates doc text to remove Softmax mention; minor formatting.
src/NeuralNet/Initializers/Uniform.php	Minor formatting.
src/NeuralNet/Initializers/Normal.php	Minor formatting.
src/NeuralNet/Initializers/LeCun.php	Minor formatting.
src/NeuralNet/Initializers/He.php	Minor formatting.
src/NeuralNet/ActivationFunctions/Softmax.php	Deletes the Softmax activation function implementation.
mkdocs.yml	Removes Softmax activation function from docs nav.
docs/neural-network/initializers/xavier-1.md	Removes Softmax link/mention in Xavier 1 docs.
docs/neural-network/activation-functions/softmax.md	Removes the Softmax activation function docs page.
docs/classifiers/softmax-classifier.md	Removes the link to the deleted Softmax activation docs page.
CHANGELOG.md	Notes Softmax removal and Multiclass gradient fix.
benchmarks/NeuralNet/ActivationFunctions/SoftmaxBench.php	Removes benchmarks for the deleted Softmax activation function.

Review details

• Files reviewed: 15/15 changed files
• Comments generated: 2
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.