Normalization
Normalization in the COBRA subsystem appears in two distinct forms in the current source:
- reusable normalizer classes under
cobra/core/normalizers/; - scalar normalization constants used directly by
GradientCOBRAandMixCOBRARegressor.
These mechanisms are related conceptually, but they are not currently wired together.
The reusable normalizer package provides:
while the main regression COBRA estimators currently use:
and direct scalar multiplication.
This distinction is important when reading or extending the package.
Normalization in the COBRA pipeline
Normalization happens before distance computation.
Conceptually:
flowchart LR
X["Input or prediction representation"]
N["Normalization"]
D["Distance"]
A["Kernel adapter"]
K["Kernel"]
G["Aggregator"]
X --> N --> D --> A --> K --> GChanging the scale of the representation changes the pairwise distances seen by the downstream COBRA components.
Two normalization systems
-
Reusable normalizers
Stateful feature-wise transforms:
created through:
-
COBRA scalar scaling
A single multiplicative constant computed by:
and used directly by:
Normalizer package
The reusable normalizer components live under:
The exported classes are:
BaseNormalizer
All reusable normalizers inherit:
and implement:
The base class also supplies:
Interface
class BaseNormalizer(ABC):
@abstractmethod
def fit(
self,
x,
**kwargs,
):
...
@abstractmethod
def transform(
self,
x,
**kwargs,
):
...
def fit_transform(
self,
x,
**kwargs,
):
self.fit(
x,
**kwargs,
)
return self.transform(
x,
**kwargs,
)
The intended input shape is:
and the transformed result has the same shape.
Normalizers are stateful
The source documentation explicitly describes normalizers as stateful.
For example:
Those statistics are learned during:
and reused during:
NormalizerFactory
Reusable normalizers are registered through:
which subclasses the package's shared:
Create a normalizer by name:
from kfc_procedure.cobra.core.normalizers import (
NormalizerFactory,
)
normalizer = NormalizerFactory.create(
"standard"
)
Registered names
The current source registers:
where:
are aliases for the same implementation.
Inspect available normalizers
The current registry should include:
after the package has been imported.
Standard normalization
StandardNormalizer is registered as:
It implements feature-wise Z-score normalization:
Fitted attributes
The class initializes:
After fitting:
contains one mean per feature and:
contains one standard deviation per feature.
Standard fit()
The implementation is:
and returns:
Population standard deviation
The source calls:
without specifying:
so NumPy's default:
is used.
That means std_ is based on the population-style standard deviation rather
than the sample standard deviation with ddof=1.
Standard transform()
The implementation is:
The learned vectors broadcast across rows.
Numerical stability
The source adds:
to every feature standard deviation:
This prevents direct division by zero for constant features.
Constant feature behavior
Suppose one feature has the same value in every fitted row.
Then:
and the stored value becomes approximately:
For training values equal to the learned mean:
so that feature transforms to:
for the fitted data.
Direct standard example
import numpy as np
from kfc_procedure.cobra.core.normalizers import (
StandardNormalizer,
)
X = np.array([
[1.0, 10.0],
[2.0, 20.0],
[3.0, 30.0],
])
normalizer = StandardNormalizer()
X_norm = normalizer.fit_transform(
X
)
print(
normalizer.mean_
)
print(
normalizer.std_
)
print(
X_norm
)
Transform new data
After fitting:
uses the original:
rather than recomputing them.
This is the expected fit/transform separation in the reusable normalizer API.
No fit-state validation
StandardNormalizer.transform() does not call:
and does not manually check whether:
have been learned.
Calling:
before fit() therefore fails through ordinary NumPy/Python arithmetic rather
than a package-specific fitted-state error.
Min-max normalization
MinMaxNormalizer is registered as:
It applies feature-wise scaling:
For values inside the fitted range, this maps the fitted minimum to 0 and the
fitted maximum to approximately 1.
Fitted attributes
The constructor initializes:
Min-max fit()
The implementation is:
and returns:
Min-max transform()
The implementation is:
Constant min-max features
When:
the denominator would normally be zero.
The source adds:
to avoid direct division by zero.
For a value equal to the fitted constant:
the transformed value is:
Direct min-max example
import numpy as np
from kfc_procedure.cobra.core.normalizers import (
MinMaxNormalizer,
)
X = np.array([
[10.0, 100.0],
[20.0, 150.0],
[30.0, 200.0],
])
normalizer = MinMaxNormalizer()
X_norm = normalizer.fit_transform(
X
)
print(
normalizer.min_
)
print(
normalizer.max_
)
print(
X_norm
)
New values can fall outside [0, 1]
MinMaxNormalizer.transform() does not clip transformed values.
If a new observation is below the fitted minimum:
the output can be negative.
If it is above the fitted maximum:
the output can exceed 1.
Example:
produces approximately:
rather than clipping to:
No min-max fit-state validation
Like the standard normalizer, MinMaxNormalizer does not explicitly validate
fitted state inside:
Using it before:
fails through arithmetic with uninitialized attributes.
Shared validation behavior
The two current reusable normalizers are intentionally lightweight.
They both begin with:
but do not perform a shared validation layer for:
Those conditions are therefore handled implicitly by NumPy operations.
One-dimensional input
The class documentation describes input as:
but the implementation does not enforce two dimensions.
For a one-dimensional vector:
the source computes statistics over:
and returns a one-dimensional transformed vector.
So 1D data can work numerically even though the documented primary shape is 2D.
Feature-count compatibility
After fitting on:
features, the learned statistics have shape:
Transforming data with a different number of columns will fail or broadcast according to NumPy's shape rules.
No explicit feature-count error is raised by the normalizer classes.
NaN and infinity
Neither reusable normalizer filters:
before calculating statistics.
For example:
can propagate non-finite values into learned attributes.
The current source does not use:
or other sanitization.
Reusable normalizers are not currently used by main COBRA estimators
A package-wide source search shows that:
are referenced inside the normalizer package itself, but not by the current implementations of:
Those estimators do not currently accept constructor parameters such as:
and do not create a normalizer_ fitted attribute.
Current integration status
The reusable normalizer classes exist as extension components, but the
current main COBRA estimators do not route their internal representations
through NormalizerFactory.
Their actual normalization behavior is described below.
Scalar normalization utility
The active normalization helper used by the regression COBRA estimators is:
from:
Its signature is:
Scalar formula
The source first computes:
Then it chooses the numerator:
Finally:
where:
in the current estimator calls.
Implementation
max_val = (
np.max(
np.abs(
y
)
)
+
1e-12
)
c = (
norm_constant
if norm_constant is not None
else scale_factor
)
return c / (
max_val
*
M
)
Important norm_constant meaning
The parameter name:
can suggest that it directly supplies the final multiplier.
That is not what the current helper does.
When provided, it replaces:
in the numerator.
The returned multiplier is still:
Important
norm_constant is a normalization numerator in the current helper,
not the final scaling constant.
Direct helper example
import numpy as np
from kfc_procedure.cobra.utils.preprocessing import (
compute_normalization_constant,
)
y = np.array([
-10.0,
5.0,
8.0,
])
c = compute_normalization_constant(
y,
scale_factor=30.0,
M=3,
)
print(
c
)
Ignoring the tiny epsilon:
Custom numerator example
Because norm_constant is not None, the helper uses:
instead of:
as the numerator.
Scale is global, not feature-wise
Unlike:
the scalar helper returns one number.
The estimator then performs:
for every coordinate.
So it does not learn separate feature-wise means, standard deviations, minima, or maxima.
GradientCOBRA normalization
GradientCOBRA exposes:
in its constructor.
During fitting, after creating the prediction-space matrix, the source calls:
self.normalize_constant_ = (
compute_normalization_constant(
y=y,
norm_constant=self.norm_constant,
scale_factor=30.0,
M=prediction_space.shape[1],
)
)
GradientCOBRA formula
Let:
Then the default scalar is:
If constructor parameter:
is provided:
GradientCOBRA normalized prediction space
The calibration prediction matrix is stored as:
Then the distance matrix is computed on:
rather than the raw predictions.
GradientCOBRA flow
flowchart LR
P["Prediction space P"]
C["Scalar c"]
PN["Y_l_norm_ = cP"]
D["Distance matrix"]
K["Kernel pipeline"]
P --> PN
C --> PN
PN --> D --> KGradientCOBRA prediction-time consistency
During prediction:
Then:
So new predictions are multiplied by the same fitted scalar used for the calibration prediction space.
This preserves scale consistency between:
Inspect GradientCOBRA normalization
After fitting:
Inspect the normalized calibration space:
Reconstruct GradientCOBRA normalization
In standard feature mode:
Then:
should normally return:
as_predictions=True in GradientCOBRA
When:
the supplied:
is already treated as prediction space.
The same scalar calculation is still applied:
The helper still derives its magnitude from:
and:
MixCOBRA uses two scalar constants
MixCOBRARegressor combines:
It therefore computes two separate scaling constants:
from constructor settings:
MixCOBRA input-space constant
The current fit code calls:
self.normalize_constant_x_ = (
compute_normalization_constant(
X,
norm_constant=self.norm_constant_x,
scale_factor=5.0,
M=prediction_space.shape[1],
)
)
Note carefully that the first argument is:
even though the helper parameter is named:
The function simply applies:
to whatever array is supplied.
MixCOBRA input formula
Let:
have \(M\) columns.
The default input-space scalar is:
The maximum is taken over the entire supplied array X.
MixCOBRA prediction-space constant
The source also calls:
self.normalize_constant_y_ = (
compute_normalization_constant(
y,
norm_constant=self.norm_constant_y,
scale_factor=50.0,
M=prediction_space.shape[1],
)
)
So the default prediction-space multiplier is:
MixCOBRA default scaling factors
The source therefore uses different default numerators:
The two spaces are deliberately scaled differently before distance calculation.
MixCOBRA normalized calibration spaces
The source stores:
and:
Two-parameter mode
With:
the package computes distances separately:
distance_matrix_x_ = distance(
X_l_norm_,
X_l_norm_,
)
distance_matrix_y_ = distance(
Y_l_norm_,
Y_l_norm_,
)
Then the adapter combines them as:
Normalization therefore affects both terms before alpha and beta are
optimized.
One-parameter MixCOBRA
With:
the source concatenates:
and computes one distance matrix on that joined representation.
So the scalar normalization constants control the relative numerical scale of the two blocks even before the one-parameter bandwidth is optimized.
MixCOBRA prediction-time scaling
At prediction time:
The fitted scalar constants are reused for new observations.
Inspect MixCOBRA normalization
After fitting:
Inspect the stored spaces:
Reconstruct MixCOBRA normalized input space
reconstructed_x = (
model.X_l_
*
model.normalize_constant_x_
)
print(
np.allclose(
reconstructed_x,
model.X_l_norm_,
)
)
Reconstruct MixCOBRA prediction space
In normal feature mode:
Then:
should normally return:
Important MixCOBRA data-source detail
The input-space constant is computed from the X argument passed to:
not from:
specifically.
So when explicit calibration data are supplied:
the source computes:
from:
because that is the X argument to fit().
It then applies that scalar to:
for calibration distances.
MixCOBRA target-scaling detail
Similarly:
is computed from the y argument passed to fit().
With explicit calibration data:
while the prediction-space geometry belongs to:
The source therefore derives the scalar from training-side targets and applies it to calibration-side estimator predictions.
This page documents that exact implementation.
MixCOBRA with as_predictions=True
When:
the shared training resolver sets:
and:
MixCOBRA then sets:
but still computes:
from the supplied X and:
from the supplied y.
As a result:
and:
are both based on the supplied prediction matrix, but multiplied by two different scalar constants.
Current source behavior
In MixCOBRA's as_predictions=True fit path, the same supplied matrix
participates as both:
and is therefore stored in two differently scaled representations.
CombinedClassifier does not normalize prediction space
The current:
does not call:
and does not use:
Its calibration prediction matrix is stored directly as:
and distance is computed on:
Why that matters
With the default:
distance, CombinedClassifier operates on hard-label mismatch patterns such as:
The current source applies no standardization, min-max scaling, or scalar normalization before that distance.
Normalization and distance scaling
A scalar multiplier changes many distance metrics predictably.
For Euclidean distance:
So multiplying every prediction coordinate by:
scales the Euclidean distance by:
The package then separately applies the kernel adapter parameter, such as:
Thus both normalization and optimized bandwidth affect the numerical scale that reaches the kernel.
Effective GradientCOBRA RBF scale
For raw prediction-space Euclidean distance:
scalar normalization by:
gives approximately:
The one-parameter adapter then gives:
With the default RBF kernel:
so:
for Euclidean geometry.
This shows why normalization and bandwidth should be interpreted together.
Effect of number of prediction columns
The scalar helper divides by:
where M is passed as:
So, all else equal, adding more prediction columns reduces the scalar:
That behavior is explicit in the current source.
Zero-magnitude data
The helper computes:
If every supplied value is zero:
then:
This produces a very large scalar.
However, multiplying an all-zero representation by that scalar still produces zeros.
If the target array is zero while the prediction matrix is not, the resulting prediction scaling can become very large.
The current helper does not cap the returned constant.
Empty arrays
compute_normalization_constant() calls:
directly.
An empty input therefore fails through NumPy's reduction rules.
The helper does not provide a custom empty-array validation error.
M=0
The helper does not validate:
before division.
Passing:
causes division by zero behavior.
The current high-level calls normally obtain M from:
so an ordinary non-empty prediction space has a positive value.
Negative custom numerator
The helper does not require:
If a negative numerator is supplied, the returned scalar is negative.
With distance metrics based on differences and absolute magnitudes, sign may cancel mathematically; with arbitrary custom components, the effect depends on the downstream implementation.
The current source does not restrict this parameter.
Difference from Z-score normalization
StandardNormalizer performs:
feature by feature.
compute_normalization_constant() performs:
with one global scalar.
So the scalar helper:
does not center features
does not equalize per-column variance
does not store per-feature statistics.
Difference from MinMax normalization
MinMaxNormalizer performs:
The COBRA scalar helper instead scales every value by the same number.
It does not force coordinates into:
Why StandardNormalizer and MinMaxNormalizer are still useful
Although the main estimators do not currently use them internally, they can be used directly when building custom COBRA workflows.
For example:
normalizer = NormalizerFactory.create(
"standard"
)
P_train_norm = normalizer.fit_transform(
P_train
)
P_test_norm = normalizer.transform(
P_test
)
Those normalized matrices can then be passed to lower-level distance or custom pipeline components.
Pre-normalizing external prediction space
For an estimator that accepts:
you can preprocess a matrix yourself.
For example:
normalizer = NormalizerFactory.create(
"standard"
)
P_train_norm = normalizer.fit_transform(
P_train
)
P_test_norm = normalizer.transform(
P_test
)
Then:
and:
use your transformed representation.
Important
GradientCOBRA will still apply its own scalar
compute_normalization_constant() after receiving P_train_norm.
So this is additional preprocessing, not a replacement for the estimator's built-in scalar scaling.
External normalization with CombinedClassifier
Because CombinedClassifier does not apply its own normalization, a manually
transformed prediction matrix passed via:
is used directly by its distance component.
Whether that transformation makes sense depends on the configured distance.
For example, standardizing hard class labels before Hamming distance changes the representation away from the package's normal hard-label pattern.
The source does not impose or recommend such a transformation automatically.
Custom normalizer
A custom normalizer can subclass:
and register with:
Example:
import numpy as np
from kfc_procedure.cobra.core.normalizers import (
BaseNormalizer,
NormalizerFactory,
)
@NormalizerFactory.register(
"maxabs"
)
class MaxAbsNormalizer(
BaseNormalizer
):
def __init__(
self,
):
self.scale_ = None
def fit(
self,
x,
**kwargs,
):
x = np.asarray(
x
)
self.scale_ = (
np.max(
np.abs(
x
),
axis=0,
)
+
1e-12
)
return self
def transform(
self,
x,
**kwargs,
):
return (
np.asarray(
x
)
/
self.scale_
)
Then:
can resolve it.
Register aliases
The factory supports multiple names:
Both names point to the same class.
Factory registration does not integrate it automatically
Registering a new normalizer makes it available through:
but current:
do not query NormalizerFactory.
So a custom normalizer is not automatically activated by registration alone.
The estimator source would need an explicit normalizer configuration path.
Inspect standard fitted state
normalizer = NormalizerFactory.create(
"standard"
)
normalizer.fit(
X
)
print(
normalizer.mean_
)
print(
normalizer.std_
)
Inspect min-max fitted state
normalizer = NormalizerFactory.create(
"minmax"
)
normalizer.fit(
X
)
print(
normalizer.min_
)
print(
normalizer.max_
)
Inspect GradientCOBRA fitted state
There is no:
in the current implementation.
Inspect MixCOBRA fitted state
print(
model.normalize_constant_x_
)
print(
model.normalize_constant_y_
)
print(
model.X_l_norm_
)
print(
model.Y_l_norm_
)
Again, these are scalar-normalized arrays, not outputs from the reusable
NormalizerFactory classes.
Debugging normalization
Check prediction-space width
GradientCOBRA:
Remember that M enters the denominator of the scalar normalization formula.
Recompute GradientCOBRA constant
from kfc_procedure.cobra.utils.preprocessing import (
compute_normalization_constant,
)
c = compute_normalization_constant(
y_train,
norm_constant=model.norm_constant,
scale_factor=30.0,
M=prediction_space.shape[1],
)
print(
c
)
print(
model.normalize_constant_
)
Use the same y array that was passed as the main fit() argument.
Recompute MixCOBRA constants
c_x = compute_normalization_constant(
X,
norm_constant=model.norm_constant_x,
scale_factor=5.0,
M=prediction_space.shape[1],
)
c_y = compute_normalization_constant(
y,
norm_constant=model.norm_constant_y,
scale_factor=50.0,
M=prediction_space.shape[1],
)
Check magnitude
For MixCOBRA:
Current implementation comparison
| Mechanism | Learned state | Transform | Current main estimator use |
|---|---|---|---|
StandardNormalizer |
mean_, std_ |
\((x-\mu)/\sigma\) | not wired in |
MinMaxNormalizer |
min_, max_ |
\((x-min)/(max-min)\) | not wired in |
compute_normalization_constant() |
one scalar | \(cx\) | GradientCOBRA, MixCOBRA |
| CombinedClassifier | none | raw prediction space | no normalization |
Scalar defaults by estimator
| Estimator / space | Numerator | Denominator source |
|---|---|---|
| GradientCOBRA prediction space | 30.0 |
max(abs(y)) * M |
| MixCOBRA input space | 5.0 |
max(abs(X)) * M |
| MixCOBRA prediction space | 50.0 |
max(abs(y)) * M |
| CombinedClassifier | — | no normalization |
All denominators include the source's added:
inside the maximum-magnitude term.
Important current-source caveats
| Area | Current behavior |
|---|---|
| reusable normalizers available | Standard, MinMax |
| reusable normalizers used internally by main estimators | No |
| Standard epsilon | 1e-12 added to std |
| MinMax epsilon | 1e-12 added to range |
| MinMax clipping | none |
| fitted-state checks | none |
| shared finite-value validation | none |
| GradientCOBRA scaling | one scalar |
| MixCOBRA scaling | two scalars |
| CombinedClassifier normalization | none |
custom norm_constant |
replaces numerator, not final scalar |
normalization divided by prediction width M |
Yes |
| negative custom numerator validation | none |
| MixCOBRA explicit-calibration constant source | main X/y arguments, not X_l/y_l |
Quick reference
| Goal | Current source mechanism |
|---|---|
| feature-wise zero mean / unit variance | StandardNormalizer |
feature-wise [0,1] scaling on fitted range |
MinMaxNormalizer |
| create normalizer by name | NormalizerFactory.create(...) |
| GradientCOBRA internal prediction scaling | normalize_constant_ |
| customize GradientCOBRA numerator | norm_constant= |
| MixCOBRA input scaling | normalize_constant_x_ |
| MixCOBRA prediction scaling | normalize_constant_y_ |
| customize MixCOBRA numerators | norm_constant_x=, norm_constant_y= |
| inspect normalized GradientCOBRA representation | Y_l_norm_ |
| inspect normalized MixCOBRA representations | X_l_norm_, Y_l_norm_ |
Mental model
The current package has reusable feature-wise normalizers, but its main COBRA regression estimators actually normalize geometry with global scalar multipliers before distance computation.
Reusable normalization:
GradientCOBRA:
MixCOBRA:
and:
Understanding which normalization path is actually active is important when interpreting distances, bandwidths, and MixCOBRA's input/prediction-space balance.