Aggregators
Aggregators are the final numerical step in the COBRA pipeline.
After a distance matrix has been transformed by a kernel, each query has one weight for each calibration observation. The aggregator combines those weights with the calibration targets or class labels.
The current source provides two built-in aggregators:
implemented by:
under:
They are registered through:
Where aggregators sit in the pipeline
For one query, COBRA first computes a kernel-weight row:
where the \(m\) entries correspond to the calibration observations.
The aggregator then receives:
and returns one prediction.
flowchart LR
Q["Query"]
D["Distance to calibration data"]
K["Kernel"]
W["Weight vector"]
V["Calibration values"]
A["Aggregator"]
Y["Prediction"]
Q --> D --> K --> W
W --> A
V --> A
A --> YBuilt-in aggregators
| Registry name | Alias | Class | Task used by default |
|---|---|---|---|
weighted_mean |
wm |
WeightedMeanAggregator |
regression |
weighted_vote |
wv |
WeightedVoteAggregator |
classification |
The factory itself does not attach explicit regression/classification categories to these registrations.
The task distinction comes from which estimator selects which aggregator by default.
Default aggregator by estimator
The current high-level defaults are:
| Estimator | Default aggregator |
|---|---|
GradientCOBRA |
weighted_mean |
MixCOBRARegressor |
weighted_mean |
CombinedClassifier |
weighted_vote |
So the normal mapping is:
BaseAggregator
Every aggregator inherits:
The abstract interface centers on:
The base class also provides:
for batch aggregation and defines:
as an optional classification extension.
aggregate()
The required method is:
It aggregates one query.
Conceptually:
and returns one scalar or class prediction.
aggregate_matrix()
The base implementation supports batch prediction.
Its signature is:
with:
The method applies:
independently for each query row.
Base batch validation
The current base implementation requires:
otherwise it raises:
The terminology in the base docstring says n_models, but in the actual COBRA
usage the second dimension corresponds to calibration observations.
Base batch output dtype
The base class creates:
fills it query by query, and finally returns:
So the generic batch path supports either numeric or object-valued aggregate outputs.
WeightedVoteAggregator overrides this method with its own vectorized
implementation.
aggregate_proba()
The base class defines:
but the default implementation raises:
Classification aggregators may override this method.
Both current built-ins define an aggregate_proba() implementation, although
the regression weighted-mean version is only meaningful when the supplied
values are already probability vectors.
AggregatorFactory
Create a registered aggregator with:
from kfc_procedure.cobra.core.aggregators import (
AggregatorFactory,
)
aggregator = AggregatorFactory.create(
"weighted_mean"
)
or:
Aliases
The current registrations are:
and:
So these pairs are equivalent:
Inspect available aggregators
The current registry should include:
after the aggregator package has been imported.
Weighted mean aggregation
WeightedMeanAggregator is the default regression aggregator.
It is registered as:
The source formula is:
Here:
- \(v_i\) is a calibration target;
- \(w_i\) is its kernel weight for the query.
Single-query implementation
The implementation begins with:
So the target values are flattened into a one-dimensional numeric vector.
Empty values
If:
the source raises:
No weights: arithmetic mean
If:
the aggregator returns:
So without a weight vector:
Default fallback
If:
the source computes:
Therefore the default fallback for a single weighted-mean call is the
unweighted mean of the supplied values.
Weight sanitization
Weights are converted with:
then sanitized using:
So:
weights are replaced with:
before aggregation.
Important consequence
Non-finite weights do not directly propagate into the regression prediction.
They simply contribute zero weight.
This sanitization occurs inside:
Zero denominator
The source computes:
and checks:
If the total weight is effectively zero, the method returns:
instead of dividing by zero.
Weighted result
Otherwise:
which is exactly:
Direct weighted-mean example
import numpy as np
from kfc_procedure.cobra.core.aggregators import (
WeightedMeanAggregator,
)
values = np.array([
10.0,
20.0,
30.0,
])
weights = np.array([
0.8,
0.1,
0.1,
])
aggregator = WeightedMeanAggregator()
prediction = aggregator.aggregate(
values,
weights,
)
print(
prediction
)
The first calibration target contributes most strongly because it has the largest kernel weight.
Regression fallback example
values = np.array([
10.0,
20.0,
30.0,
])
weights = np.zeros(3)
prediction = aggregator.aggregate(
values,
weights,
fallback=15.0,
)
Because:
the result is:
Default fallback example
If no fallback is supplied:
the source uses:
which in this example is:
GradientCOBRA regression aggregation
GradientCOBRA resolves its aggregator with:
Its default is:
GradientCOBRA cross-validation
During bandwidth evaluation:
So inside cross-validation the explicit fallback is:
rather than the local mean.
GradientCOBRA final prediction
At prediction time:
preds = self.aggregator_.aggregate_matrix(
values=self.y_l_,
weights=K,
fallback=self.global_mean_,
)
Therefore the final prediction fallback is:
when a query has effectively zero total kernel weight.
This distinction matters:
MixCOBRA regression aggregation
MixCOBRARegressor also defaults to:
and uses the same batch aggregator.
During cross-validation:
is passed.
During final prediction:
is passed.
So its regression fallback pattern matches the current GradientCOBRA implementation.
Weighted-mean batch aggregation
WeightedMeanAggregator does not override:
so it uses the base-class loop.
For:
the base code performs one:
call per query.
Conceptually:
for every query \(q\).
Weighted-mean probability aggregation
WeightedMeanAggregator also defines:
The source explicitly notes that this is only meaningful when:
Probability input shape
Here values is not treated as a one-dimensional target vector.
The code uses:
without flattening.
A natural expected shape is:
No weights
If:
the method returns:
which averages the supplied probability vectors class by class.
With weights
Weights are converted and sanitized:
Then they are normalized with:
and the result is:
Probability normalization caveat
Because the source divides by:
rather than explicitly falling back when the sum is zero, an all-zero weight vector produces a zero probability vector rather than a fallback distribution.
The method does not apply a final probability renormalization after summing.
Weighted vote aggregation
WeightedVoteAggregator is the default classifier aggregator.
It is registered as:
The single-query rule is:
Here:
- \(v_i\) is a calibration class label;
- \(w_i\) is the corresponding query-to-calibration kernel weight.
Unweighted vote
If:
the source computes:
and returns:
So this path is an ordinary majority vote over the supplied values.
Tie behavior without weights
np.unique() returns classes in sorted order for sortable data.
np.argmax(counts) selects the first maximum.
Therefore in an equal-count tie, the current unweighted path tends to select
the first class in the np.unique() ordering.
The package does not implement a separate explicit tie-policy parameter.
Weighted vote input validation
The values are flattened:
If:
the method raises:
Weight length validation
Weights are converted with:
Then the source requires:
otherwise it raises:
Non-finite weighted-vote weights
The implementation computes:
then:
So calibration observations with:
weights are removed entirely from the voting calculation.
This differs from WeightedMeanAggregator, which converts those weights to
zero.
Weighted class scores
After filtering, the source obtains:
and constructs a one-hot matrix:
If there are:
the one-hot matrix has shape:
Score calculation
The class scores are:
which gives one total weight per class:
The final class is:
Direct weighted-vote example
import numpy as np
from kfc_procedure.cobra.core.aggregators import (
WeightedVoteAggregator,
)
values = np.array([
0,
1,
1,
0,
])
weights = np.array([
0.1,
0.4,
0.3,
0.2,
])
aggregator = WeightedVoteAggregator()
prediction = aggregator.aggregate(
values,
weights,
)
print(
prediction
)
Class 1 receives:
while class 0 receives:
so the result is:
Negative weights
The source does not constrain weighted-vote weights to be non-negative.
If the supplied kernel or custom weighting stage produces negative finite values, those values participate directly in:
and can reduce a class score.
The built-in similarity kernels normally return non-negative outputs, but the aggregator itself does not enforce that assumption.
Empty data after finite-weight filtering
There is an important edge case in the current source.
The method checks:
before removing non-finite weights.
If every weight is non-finite, then:
can produce empty arrays afterward.
The source does not perform a second empty check at that point.
Subsequent operations such as:
can therefore fail.
This is a current implementation edge case rather than a documented fallback behavior.
Weighted-vote batch aggregation
Unlike WeightedMeanAggregator,
WeightedVoteAggregator overrides:
with a vectorized implementation.
Its inputs are:
Batch implementation
The source computes the class one-hot matrix once:
Then all query scores are computed with:
giving:
Finally:
returns one class per query.
Batch validation
The override checks only:
and raises:
if not.
It does not explicitly check:
before matrix multiplication.
A mismatch will therefore fail through NumPy's matrix multiplication rules.
Batch path does not filter non-finite weights
The single-query:
filters non-finite weights.
But the vectorized:
does not apply:
or:
before:
This means single and batch weighted-vote behavior differs for non-finite weights.
Current source inconsistency
WeightedVoteAggregator.aggregate() removes non-finite weights.
WeightedVoteAggregator.aggregate_matrix() uses the weight matrix directly.
Non-finite values can therefore propagate differently depending on which method is called.
CombinedClassifier uses single weighted vote
The current CombinedClassifier does not use the vectorized
aggregate_matrix() path for ordinary hard-label prediction.
Instead, it iterates over query rows and calls:
for each query.
Therefore the single-query finite-weight filtering logic applies to the main CombinedClassifier hard-prediction path.
CombinedClassifier zero-weight fallback
Before calling the aggregator, the classifier checks:
Otherwise it calls:
So an all-zero or non-positive-sum kernel row is handled by
CombinedClassifier itself rather than by WeightedVoteAggregator.
CombinedClassifier cross-validation fallback
During cross-validation, the source similarly performs:
if np.sum(w) <= 0:
pred = self.global_majority_class_
else:
pred = self.aggregator_.aggregate(
y_train,
w,
)
Thus its fallback class is the global majority class computed by the fitted classifier.
Probability aggregation in WeightedVoteAggregator
The current:
implementation deserves special attention.
The method receives:
but the source does not actually use:
when constructing the probabilities.
Current implementation
The method computes:
Then:
and:
Next:
and finally:
Consequence of ignoring weights
For values:
the returned class frequencies are approximately:
regardless of whether the kernel weights are:
or:
because the current method does not use the supplied weight vector.
Current source behavior
WeightedVoteAggregator.aggregate_proba() currently returns normalized
unweighted class frequencies.
It does not compute weighted class probabilities from the provided kernel weights.
Effect on CombinedClassifier.predict_proba()
CombinedClassifier.predict_proba() computes the query kernel row:
and then calls:
However, with the current default:
the weights=w argument is ignored by aggregate_proba().
Therefore non-fallback CombinedClassifier.predict_proba() rows are based on
the class distribution of:
rather than query-specific kernel weighting.
Zero-weight probability fallback
Before calling aggregate_proba(), CombinedClassifier checks:
and sets probability 1.0 on:
for that query.
So the current probability behavior is:
zero/non-positive kernel sum
-> one-hot global-majority fallback
positive kernel sum
-> unweighted calibration class frequencies
with the default weighted-vote aggregator.
aggregate_proba_batch()
WeightedVoteAggregator also defines:
This method does use the weight matrix.
It computes:
then:
So batch probability aggregation is weighted.
Current API mismatch
The base BaseAggregator class does not define:
as part of its abstract or shared interface.
CombinedClassifier also does not call it in its standard probability path.
Therefore the current weighted probability behavior implemented in
aggregate_proba_batch() is not used by the normal CombinedClassifier
predict_proba() loop.
Probability batch zero-score behavior
If one query has all zero weighted class scores:
then division by:
returns an all-zero probability row.
There is no majority-class fallback inside:
itself.
Aggregator configuration
The high-level COBRA estimators accept:
and:
For example:
or:
from kfc_procedure.cobra import CombinedClassifier
classifier = CombinedClassifier(
aggregator="weighted_vote",
)
Current built-in aggregator parameters
The two concrete built-ins do not define custom __init__() methods.
So the current source exposes no specialized constructor parameters for:
Although the high-level estimators accept:
passing arbitrary parameters to either built-in can fail through the factory constructor call because the classes do not define corresponding constructor arguments.
Inspect the fitted aggregator
After fitting a COBRA estimator:
For regression, the default type is:
For classification:
Direct factory example
from kfc_procedure.cobra.core.aggregators import (
AggregatorFactory,
)
mean_aggregator = AggregatorFactory.create(
"wm"
)
vote_aggregator = AggregatorFactory.create(
"wv"
)
Aggregation values are calibration targets, not base-model predictions
A subtle but important distinction:
The aggregator in GradientCOBRA or CombinedClassifier usually does not aggregate the original base-estimator prediction columns directly.
Instead:
- prediction-space representations determine distances;
- distances produce kernel weights over calibration observations;
- the aggregator combines the calibration targets or labels.
For regression:
For classification:
The difference is the type of values and the aggregation rule.
Regression example
Suppose one query has kernel weights:
over calibration targets:
Then weighted mean returns:
Classification example
Suppose one query has weights:
over labels:
Then class scores are:
and:
The weighted-vote output is class:
Weight normalization
The two built-ins treat weight normalization differently.
WeightedMeanAggregator.aggregate() does not explicitly normalize W first.
Instead it computes:
which is algebraically equivalent to using normalized weights when the sum is nonzero.
WeightedVoteAggregator.aggregate() does not normalize weights at all because
multiplying all class weights by the same positive constant leaves:
unchanged.
Kernel weights need not sum to one
The kernel layer can return arbitrary non-negative similarity values.
For example:
The aggregators handle those raw values directly.
No separate normalization step is required before:
for hard prediction.
Compact kernels and aggregator fallback
Compact kernels such as:
can produce all-zero query rows.
This makes fallback behavior particularly important.
For regression:
For classification:
The fallback behavior belongs partly to the estimator and partly to the aggregator.
Naive kernel interaction
The source's:
kernel returns the adapted distance unchanged.
That means larger distances become larger "weights."
The aggregators do not know whether their input came from a proper similarity kernel.
They simply consume the numeric weights they are given.
So with a naive kernel, weighted mean and weighted vote will give more influence to larger distance values unless some earlier transformation changes them.
This follows directly from the current component separation.
Custom aggregator
A custom aggregator should subclass:
and implement:
The base batch implementation can then be inherited if query-by-query looping is acceptable.
Example:
import numpy as np
from kfc_procedure.cobra.core.aggregators import (
BaseAggregator,
AggregatorFactory,
)
@AggregatorFactory.register(
"median"
)
class MedianAggregator(
BaseAggregator
):
def aggregate(
self,
values,
weights=None,
**kwargs,
):
values = np.asarray(
values,
dtype=float,
)
return float(
np.median(
values
)
)
Then:
can resolve it through the normal factory.
Custom weighted regression aggregator
For example:
@AggregatorFactory.register(
"positive_weighted_mean"
)
class PositiveWeightedMean(
BaseAggregator
):
def aggregate(
self,
values,
weights=None,
fallback=None,
**kwargs,
):
V = np.asarray(
values,
dtype=float,
).reshape(-1)
if weights is None:
return float(
np.mean(V)
)
W = np.maximum(
np.asarray(
weights,
dtype=float,
).reshape(-1),
0.0,
)
denom = np.sum(W)
if np.isclose(
denom,
0.0,
):
return (
float(np.mean(V))
if fallback is None
else fallback
)
return float(
W @ V
/ denom
)
Custom classification aggregator
A custom classifier aggregator can implement both:
and:
Example interface:
class MyVoteAggregator(
BaseAggregator
):
def aggregate(
self,
values,
weights=None,
**kwargs,
):
...
def aggregate_proba(
self,
values,
weights=None,
classes=None,
**kwargs,
):
...
This is the appropriate extension point if query-specific weighted probabilities are required.
Implementing weighted probabilities correctly
A custom probability aggregator can compute:
For example:
def aggregate_proba(
self,
values,
weights=None,
classes=None,
**kwargs,
):
V = np.asarray(
values
).reshape(-1)
if classes is None:
classes = np.unique(V)
classes = np.asarray(
classes
)
if weights is None:
W = np.ones(
len(V),
dtype=float,
)
else:
W = np.asarray(
weights,
dtype=float,
).reshape(-1)
scores = np.array([
np.sum(
W[
V == c
]
)
for c in classes
])
total = np.sum(scores)
if np.isclose(
total,
0.0,
):
return np.zeros(
len(classes),
dtype=float,
)
return scores / total
This is an extension example; it is not what the current built-in
aggregate_proba() implementation does.
Custom batch optimization
If a custom aggregator can be vectorized efficiently, override:
instead of inheriting the base Python loop.
WeightedVoteAggregator demonstrates this pattern.
Likewise, a classification aggregator may define a custom batch probability method, although the current base interface does not standardize one.
Shape conventions
Single regression query
returns:
Batch regression queries
returns:
Single classification query
returns:
Classification probabilities
values:
calibration class labels
weights:
one query's calibration weights
classes:
complete output class ordering
returns:
for the current WeightedVoteAggregator.aggregate_proba() API.
Debugging regression aggregation
Inspect target values
Inspect kernel weights
Inspect weight sum
A near-zero total can trigger a fallback in weighted mean.
Reproduce one prediction
prediction = model.aggregator_.aggregate(
values=model.y_l_,
weights=K[0],
fallback=model.global_mean_,
)
Debugging classification aggregation
Inspect calibration labels
Compute weighted class scores
values = model.y_l_
weights = K[0]
classes = np.unique(
values
)
scores = {
c: weights[
values == c
].sum()
for c in classes
}
print(
scores
)
Compare the largest score with:
Debugging probabilities
For the current built-in weighted-vote probability method:
proba = model.aggregator_.aggregate_proba(
values=model.y_l_,
weights=K[0],
classes=model.classes_,
)
Then compare with unweighted calibration class frequencies:
freq = np.array([
np.mean(
model.y_l_ == c
)
for c in model.classes_
])
print(
proba
)
print(
freq
)
With the current implementation, these should match apart from the final tiny
1e-12 normalization adjustment.
Current implementation differences
| Behavior | weighted_mean |
weighted_vote |
|---|---|---|
| values converted to float | Yes | No |
| no weights | arithmetic mean | majority vote |
| weight-length validation | No explicit check | Yes |
| non-finite weights | replaced by zero | filtered in single-query path |
| zero-total handling | fallback | caller usually handles |
| batch override | No | Yes |
| probability method | weighted probability vectors | unweighted label frequencies |
| batch probability method | No | Yes, weighted |
Important current-source caveats
Weighted mean does not validate length equality
WeightedMeanAggregator.aggregate() flattens both arrays but does not
explicitly check:
If they differ, the later:
operation raises through NumPy.
Weighted vote batch path differs from single path
Single:
filters non-finite weights.
Batch:
does not.
Weighted vote probability ignores weights
Single-query:
does not use the supplied weights.
Batch:
does use them.
Built-in classes have no custom constructor parameters
So:
currently has little practical use with the two built-in aggregator classes.
It remains useful as an extension mechanism for custom registered aggregators.
Quick reference
| Property | Weighted Mean | Weighted Vote |
|---|---|---|
| Registry | weighted_mean, wm |
weighted_vote, wv |
| Default task | regression | classification |
| Single rule | normalized weighted numeric mean | weighted class argmax |
| No-weight behavior | arithmetic mean | majority vote |
| Final regression fallback | supplied by estimator | — |
| Final classification fallback | — | supplied by CombinedClassifier |
| Vectorized batch override | No | Yes |
aggregate_proba() |
weighted mean of probability vectors | ignores weights |
aggregate_proba_batch() |
not defined | weighted |
| Built-in custom params | none | none |
Mental model
The kernel decides how strongly each calibration observation should influence the query; the aggregator decides how those weighted calibration values become the final prediction.
For regression:
For classification:
The current source implements these two hard-prediction rules directly, while
its classification probability path has a separate implementation caveat:
WeightedVoteAggregator.aggregate_proba() currently ignores the supplied
weights.