NTDLS.Determinet
This is open source software licensed as MIT License. You can obtain the source code from Determinet
📦 Be sure to check out the NuGet package: https://www.nuget.org/packages/NTDLS.Determinet
Determinet is versatile multilayer perception neural network designed for ease of use and extendibility. In addition the the library, you'll find a test harness which includes a character recognition trainer, validator and visual testing tool. These provide working examples of training a model as well as generating predictions.
Built-in Activation Functions
| Activation Type | What It Does | When To Use It | Where To Use It |
|---|---|---|---|
| Identity | Passes values through unchanged. The neuron is basically “transparent.” | Use for pure regression or as a linear output layer when predicting real numbers — e.g., predicting house prices, speed, or temperature. | Output |
| PiecewiseLinear | Acts linear within a range but limits slope outside (like a gentle LeakyReLU with bounds). | Use when you want bounded but mostly linear responses — e.g., controlling actuator strength, image intensity normalization, or testing stable transitions. | Hidden (sometimes Output for bounded regression) |
| Linear | Scales input by a slope and optionally clamps to a range. | Great for controlled continuous outputs where scaling matters — e.g., steering angle, torque output, or color channel prediction. | Output (regression), Hidden (feature scaling) |
| ReLU (Rectified Linear Unit) | Keeps positive values, zeros out negatives. Very fast and stable. | The default for most deep learning tasks. Ideal for vision models, embeddings, and dense MLPs — e.g., image recognition, feature extraction, signal processing. | Hidden |
| Sigmoid | Squashes output between 0 and 1 (probability curve). | Use for binary classification (“yes/no,” “on/off,” “spam/not spam”) or when modeling probability per neuron in multi-label tasks. | Output (binary classification), Hidden (gating in LSTMs or attention) |
| Tanh | Squashes output between -1 and 1, centered around 0. | Use when values can be positive or negative and symmetry matters — e.g., RNNs, control systems, or directional outputs (-1 left, +1 right). | Hidden |
| LeakyReLU | Like ReLU, but negative values leak through slightly instead of being zeroed. | Use in deep MLPs to avoid “dead neurons” and keep gradients flowing — especially in dense or deep architectures. | Hidden |
| SoftMax | Converts outputs into a probability distribution that sums to 1. | Use for multi-class classification — e.g., recognizing characters, digits, objects, or anything where exactly one class is correct. | Output |
| SimpleSoftMax | A simplified SoftMax (no temperature scaling). | Use when you need fast, stable classification and temperature tuning isn’t required. Works the same for most classification tasks. | Output |
| ELU (Exponential Linear Unit) | Works like ReLU for positive inputs but smoothly bends for negatives instead of dropping to zero. | Use when you want ReLU-like behavior but prefer smoother gradients and mean activations closer to zero — e.g., deep networks prone to “dead neurons.” | Hidden |
| Gaussian (RBF) | Outputs a bell curve centered around zero (f(x) = e-x²). | Use for radial basis networks, feature clustering, or anomaly detection where proximity to a center value matters. | Hidden |
| HardSigmoid | A linear, fast approximation of the sigmoid function. | Use in lightweight models or embedded systems where performance matters, or when full sigmoid precision isn’t required. | Hidden, Output (binary) |
| HardTanh | Like Tanh but clipped to the range [-1, 1] using straight lines. | Use when you need bounded outputs but want faster computation — e.g., control tasks, quantized models, or when smoothness isn’t critical. | Hidden |
| Mish | Smoothly blends ReLU and Swish behavior (f(x) = x·tanh(ln(1+eˣ))). | Use when you want smoother gradients and slightly better generalization in deep MLPs or CNNs — can outperform ReLU/Swish on some tasks. | Hidden |
| SELU (Scaled ELU) | A self-normalizing version of ELU that keeps activations around mean=0, var=1. | Use in deep fully-connected networks without batch normalization; helps stabilize training automatically. | Hidden |
| SoftSign | Smoothly scales inputs as x / (1 + | Use as a lightweight, stable alternative to tanh when you want smooth gradients without exponential cost — good for RNNs or bounded outputs. | Hidden |
| Swish | Smooth, self-gated function (f(x) = x · sigmoid(x)). | Use in modern deep networks (CNNs, Transformers, MLPs) where you want ReLU-like performance but with better gradient flow and smoother transitions. | Hidden |
| SoftPlus | Smooth version of ReLU (f(x) = ln(1 + eˣ)); never zeroes out completely. | Use when you want ReLU-like behavior but need continuous derivatives — e.g., probabilistic networks, VAEs, or any model requiring gradient stability. | Hidden, sometimes Output (positive-only regression) |