Break apart cross entropy

[andrewdalpino]

No description provided.

[Copilot]

Pull request overview

This PR splits the previously shared cross-entropy loss into two explicit cost functions—BinaryCrossEntropy and MulticlassCrossEntropy—and updates neural net layers, classifiers, tests, and documentation to use the appropriate variant.

Changes:

• Introduces BinaryCrossEntropy and MulticlassCrossEntropy cost functions and updates default layer/classifier wiring accordingly.
• Updates backprop “softmax/sigmoid + cross entropy” fast-path checks to trigger only for the correct new cost-function type.
• Migrates/extends unit tests and docs (MkDocs nav + new pages) to reflect the split.

Reviewed changes

Copilot reviewed 24 out of 24 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tests/NeuralNet/SnapshotTest.php	Updates snapshot wiring to use BinaryCrossEntropy for binary output.
tests/NeuralNet/NetworkTest.php	Updates network test to use MulticlassCrossEntropy for multiclass output.
tests/NeuralNet/Layers/MulticlassTest.php	Updates multiclass layer tests and string expectations for the new cost function name.
tests/NeuralNet/Layers/BinaryTest.php	Updates binary layer tests and string expectations for the new cost function name.
tests/NeuralNet/FeedForwardTest.php	Updates feed-forward integration test to use MulticlassCrossEntropy.
tests/NeuralNet/CostFunctions/MulticlassCrossEntropyTest.php	Renames/refocuses the prior cross-entropy tests onto MulticlassCrossEntropy and updates expected gradients.
tests/NeuralNet/CostFunctions/BinaryCrossEntropyTest.php	Adds dedicated tests for binary cross entropy compute/differentiate behavior.
tests/Classifiers/SoftmaxClassifierTest.php	Updates classifier tests to expect MulticlassCrossEntropy in params/defaults.
tests/Classifiers/MultilayerPerceptronTest.php	Updates MLP tests to expect MulticlassCrossEntropy in params/defaults.
tests/Classifiers/LogisticRegressionTest.php	Updates logistic regression tests to expect BinaryCrossEntropy in params/defaults.
src/NeuralNet/Layers/Multiclass.php	Switches default cost function to MulticlassCrossEntropy and updates the fast-path type check.
src/NeuralNet/Layers/Binary.php	Switches default cost function to BinaryCrossEntropy and updates the fast-path type check.
src/NeuralNet/CostFunctions/MulticlassCrossEntropy.php	Adds a dedicated multiclass cross-entropy loss implementation.
src/NeuralNet/CostFunctions/BinaryCrossEntropy.php	Renames/updates implementation to represent binary cross entropy.
src/Classifiers/SoftmaxClassifier.php	Defaults to MulticlassCrossEntropy and rejects BinaryCrossEntropy.
src/Classifiers/MultilayerPerceptron.php	Defaults to MulticlassCrossEntropy and rejects BinaryCrossEntropy.
src/Classifiers/LogisticRegression.php	Defaults to BinaryCrossEntropy.
mkdocs.yml	Updates docs navigation to include the two new cost-function pages.
docs/neural-network/cost-functions/multiclass-cross-entropy.md	Adds docs page for multiclass cross entropy.
docs/neural-network/cost-functions/cross-entropy.md	Removes the old unified cross-entropy page.
docs/neural-network/cost-functions/binary-cross-entropy.md	Adds docs page for binary cross entropy.
docs/classifiers/softmax-classifier.md	Updates references/examples to MulticlassCrossEntropy.
docs/classifiers/multilayer-perceptron.md	Updates references/examples to MulticlassCrossEntropy.
docs/classifiers/logistic-regression.md	Updates references/examples to BinaryCrossEntropy.

Suppressed comments (1)

src/NeuralNet/CostFunctions/BinaryCrossEntropy.php:57

• BinaryCrossEntropy::compute() clips the target values, but differentiate() uses the original (unclipped) target. That means the gradient is not the derivative of the loss being computed, which can lead to incorrect optimization dynamics. Clipping the output is sufficient here (log terms are already protected), so the target should not be clipped (or must be clipped consistently in both methods).

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.