3.0

[andrewdalpino]

- Integers are now considered a categorical data type
- K Nearest Neighbors and KNN Regressor inference is now parallelized
- Isolation Forest training and inference is now parallelized
- Added disk-based streaming neural network snapshotting
- Can now clear neural network Adaptive optimizer state
- Added validation interval parameter to MLPs and GBM Learners
- Cross Entropy loss function now split into Binary and Multiclass
- Logistic Regression, Softmax, and Adaline now use hold out set
- Adaboost now uses validation set with early stopping window
- Renamed TF-IDF dampening parameter to sublinear
- Exportable Extractors now append by default with option to overwrite
- RBX Serializer tracks major library version number, not minor
- Added Class/Cluster Purity clustering metrics
- V-measure, Completeness, and Homogeneity now use entropy-based formula
- Fixed KDTree edge pruning + optimize traversal
- Ball and Vantage Trees now require Subadditive kernels
- K-d Trees now require Monotonic distance kernels
- Optimize Dataset sort(), sorting is now unstable
- Added per-class smoothing to Gaussian Naive Bayes
- Added per-cluster smoothing to Gaussian Mixture
- You can now exclude certain categories from one-hot encoding
- Fixed SVC save/load using class map sidecar
- Parallel Backends now default to max physical cores not logical
- Added workers() method to the Backend interface (Serial returns 1)
- No longer save/load Backend state, transient per environment
- Added Emoji preset to Regex Filter
- Added Float Type Converter numeric string and ints to float
- Boolean Converter now converts truthy and falsy
- Interval Discretizer now encodes values as integers
- Polynomial Expander now limited to 10'th degree
- Updated to PSR-3 Log version 3
- Update Amp Backend to Amp version 2.0
- Removed Word Stemmer tokenizer
- Removed window early stopping from TSNE
- Removed output layer L2 Penalty parameter from MLP Learners
- RBX serializer now emits warning on class revision mismatch
- Class revision hash now compensates for circular references
- Filesystem Persister now does atomic writes
- Added cleanup() method to remove neural network residual state

Update:

• Murmur3 new default Token Hashing Vectorizer hash function
• Dataset fold() now returns excess samples in last fold
• Increase default Decision Tree max leaf node size from 3 to 5
• Canonicalized He and Xavier 2 initializers
• Xavier 2 now extends He as a deprecated alias

[apphp]

Another found with LLM.

WDYT about this? Does it make sense or relevant at all?

Fix: do the same as AdaBoost — score the model after the new tree is in the ensemble, remember tree count, and roll back only if quality got worse.

Three changes in train():

1. Move “add tree + update out / outTest” above validation
   Right now you score, then maybe break, then append. Swap that: train booster → $this->ensemble[] = $booster → update predictions → then evalThisStep.

2. Save size, not epoch

$bestEnsembleSize = 0;
// when score improves (do this BEFORE breaking on max score):
if ($score > $bestScore) {
    $bestScore = $score;
    $bestEpoch = $epoch;
    $bestEnsembleSize = count($this->ensemble);
    $numWorseEpochs = 0;
} else {
    ++$numWorseEpochs;
}
if ($score >= $maxScore) {
    break;
}

3. Restore by that size, only if the last score is worse

if ($this->scores and end($this->scores) < $bestScore) {
    $this->ensemble = array_slice($this->ensemble, 0, $bestEnsembleSize);
}

Use <, not <=. Use $bestEnsembleSize, not $bestEpoch.

[andrewdalpino]

Which algo are you talking about @apphp?

[apphp]

Which algo are you talking about @apphp?

src/Regressors/GradientBoost.php