NTDLS.Determinet

Status: Stable
Released: 2024-12-31
License: MIT License

This is open source software licensed as MIT License. You can obtain the source code from GitHub or browse the releases for source code associated with specific versions. If you make any changes which you feel improves this application, please feel free to submit a pull - request.

Determinet

📦 Be sure to check out the NuGet package: https://www.nuget.org/packages/NTDLS.Determinet

Determinet is versatile multilayer perception neural network designed for ease of use and extendibility. In addition the the library, you'll find a test harness which includes a character recognition trainer, validator and visual testing tool. These provide working examples of training a model as well as generating predictions.

image

Built-in Activation Functions

Activation Type What It Does When To Use It Where To Use It
Identity Passes values through unchanged. The neuron is basically “transparent.” Use for pure regression or as a linear output layer when predicting real numbers — e.g., predicting house prices, speed, or temperature. Output
PiecewiseLinear Acts linear within a range but limits slope outside (like a gentle LeakyReLU with bounds). Use when you want bounded but mostly linear responses — e.g., controlling actuator strength, image intensity normalization, or testing stable transitions. Hidden (sometimes Output for bounded regression)
Linear Scales input by a slope and optionally clamps to a range. Great for controlled continuous outputs where scaling matters — e.g., steering angle, torque output, or color channel prediction. Output (regression), Hidden (feature scaling)
ReLU (Rectified Linear Unit) Keeps positive values, zeros out negatives. Very fast and stable. The default for most deep learning tasks. Ideal for vision models, embeddings, and dense MLPs — e.g., image recognition, feature extraction, signal processing. Hidden
Sigmoid Squashes output between 0 and 1 (probability curve). Use for binary classification (“yes/no,” “on/off,” “spam/not spam”) or when modeling probability per neuron in multi-label tasks. Output (binary classification), Hidden (gating in LSTMs or attention)
Tanh Squashes output between -1 and 1, centered around 0. Use when values can be positive or negative and symmetry matters — e.g., RNNs, control systems, or directional outputs (-1 left, +1 right). Hidden
LeakyReLU Like ReLU, but negative values leak through slightly instead of being zeroed. Use in deep MLPs to avoid “dead neurons” and keep gradients flowing — especially in dense or deep architectures. Hidden
SoftMax Converts outputs into a probability distribution that sums to 1. Use for multi-class classification — e.g., recognizing characters, digits, objects, or anything where exactly one class is correct. Output
SimpleSoftMax A simplified SoftMax (no temperature scaling). Use when you need fast, stable classification and temperature tuning isn’t required. Works the same for most classification tasks. Output
ELU (Exponential Linear Unit) Works like ReLU for positive inputs but smoothly bends for negatives instead of dropping to zero. Use when you want ReLU-like behavior but prefer smoother gradients and mean activations closer to zero — e.g., deep networks prone to “dead neurons.” Hidden
Gaussian (RBF) Outputs a bell curve centered around zero (f(x) = e-x²). Use for radial basis networks, feature clustering, or anomaly detection where proximity to a center value matters. Hidden
HardSigmoid A linear, fast approximation of the sigmoid function. Use in lightweight models or embedded systems where performance matters, or when full sigmoid precision isn’t required. Hidden, Output (binary)
HardTanh Like Tanh but clipped to the range [-1, 1] using straight lines. Use when you need bounded outputs but want faster computation — e.g., control tasks, quantized models, or when smoothness isn’t critical. Hidden
Mish Smoothly blends ReLU and Swish behavior (f(x) = x·tanh(ln(1+eˣ))). Use when you want smoother gradients and slightly better generalization in deep MLPs or CNNs — can outperform ReLU/Swish on some tasks. Hidden
SELU (Scaled ELU) A self-normalizing version of ELU that keeps activations around mean=0, var=1. Use in deep fully-connected networks without batch normalization; helps stabilize training automatically. Hidden
SoftSign Smoothly scales inputs as x / (1 + Use as a lightweight, stable alternative to tanh when you want smooth gradients without exponential cost — good for RNNs or bounded outputs. Hidden
Swish Smooth, self-gated function (f(x) = x · sigmoid(x)). Use in modern deep networks (CNNs, Transformers, MLPs) where you want ReLU-like performance but with better gradient flow and smoother transitions. Hidden
SoftPlus Smooth version of ReLU (f(x) = ln(1 + eˣ)); never zeroes out completely. Use when you want ReLU-like behavior but need continuous derivatives — e.g., probabilistic networks, VAEs, or any model requiring gradient stability. Hidden, sometimes Output (positive-only regression)