COBRA API¶

`kfc_procedure.cobra.GradientCOBRA` ¶

Bases: BaseEstimator, RegressorMixin

`kfc_procedure.cobra.MixCOBRARegressor` ¶

Bases: ABC, BaseEstimator, RegressorMixin

MixCOBRA regressor that learns mixing weights across input/output spaces.

This estimator implements the MixCOBRA pattern: it trains an ensemble of base estimators, constructs a prediction-space representation, computes distances in input and output (prediction) spaces, learns mixing coefficients (alpha, beta) that balance those spaces, and aggregates neighbor targets with a kernel-weighted aggregator.

Parameters:

Name	Type	Description	Default
`estimators`	`list[str \| BaseEstimator] \| None`	List of estimator identifiers or estimator instances used as the expert pool. When a string is provided the ``EstimatorFactory`` is used to instantiate the implementation.	`None`
`estimators_params`	`dict[str, Any] \| None`	`Optional mapping of estimator name -> init parameters passed to the corresponding factory when string identifiers are used.`	`None`
`distance`	`str`	Identifier for the distance metric used to compute pairwise similarity in feature/prediction spaces (resolved via ``DistanceFactory``).	`'euclidean'`
`distance_params`	`dict \| None`	`Additional keyword arguments forwarded to the chosen distance class.`	`None`
`kernel`	`str`	Kernel identifier used to convert distances into weights (resolved via ``KernelFactory``).	`'rbf'`
`kernel_params`	`dict \| None`	`Keyword arguments forwarded to the kernel implementation.`	`None`
`aggregator`	`str`	`Aggregation strategy identifier used to combine neighbor targets.`	`'weighted_mean'`
`aggregator_params`	`dict \| None`	`Keyword arguments forwarded to the aggregator implementation.`	`None`
`loss`	`str`	`Loss identifier used during hyperparameter optimization.`	`'mse'`
`loss_params`	`dict \| None`	`Keyword arguments forwarded to the loss implementation.`	`None`
`optimizer`	`str`	`Optimizer identifier (gradient or search based) used to tune mixing parameters; resolved via optimizer factories.`	`'grad'`
`optimizer_params`	`dict \| None`	`Parameters forwarded to the optimizer implementation.`	`None`
`alpha_list`	`ndarray \| None`	`Optional candidate grids used by grid/search optimizers. Not used for gradient-based optimizers.`	`None`
`beta_list`	`ndarray \| None`	`Optional candidate grids used by grid/search optimizers. Not used for gradient-based optimizers.`	`None`
`norm_constant_x`	`float \| None`	Optional normalization constants for input (X) and output (Y) spaces. When None, the configured ``SpaceNormalizer`` determines appropriate scaling.	`None`
`norm_constant_y`	`float \| None`	Optional normalization constants for input (X) and output (Y) spaces. When None, the configured ``SpaceNormalizer`` determines appropriate scaling.	`None`
`opt_method`	`str`	High-level optimization mode: either ``'grad'`` for gradient-based tuning or ``'search'`` for grid/search strategies.	`'grad'`
`one_parameter`	`bool`	If True, only optimize ``alpha`` and keep ``beta`` fixed to zero.	`False`
`random_state`	`int \| None`	`PRNG seed for reproducible components (splitters, optimizers, etc.).`	`None`

Notes

The implementation follows a pipeline of: estimator training, prediction matrix construction, space normalization, distance/kernel computation, optimizer-driven parameter selection, and aggregation.
All components are created via factory classes; to extend behaviour, register new implementations with the appropriate *Factory.

`init` ¶

Initialize MixCOBRA model and store configuration.

`fit` ¶

Fit MixCOBRA model with hyperparameter optimization.

This method trains base estimators and learns optimal mixing weights (alpha, beta) that balance input-space and output-space distances for aggregation.

Parameters:

Name	Type	Description	Default
`X`	`ndarray`	`Training features. Shape: (n_samples, n_features).`	required
`y`	`ndarray`	`Training targets. Shape: (n_samples,).`	required
`X_l`	`ndarray \| None`	`External calibration features. If provided, used for aggregation instead of internal split. Shape: (n_cal_samples, n_features).`	`None`
`y_l`	`ndarray \| None`	`External calibration targets. Shape: (n_cal_samples,).`	`None`
`pred_features`	`ndarray \| None`	`Pre-computed model predictions to use instead of internal estimator outputs. Useful for integrating external predictions.`	`None`

Returns:

Name	Type	Description
`self`	`MixCOBRARegressor`	Fitted model instance.

Workflow

Split data into training (X_k) and calibration (X_l) subsets
Train base estimators on X_k
Generate prediction matrix on X_l
Normalize input and output spaces
Initialize distance, kernel, adapter, and aggregator components
Optimize alpha/beta mixing parameters

Examples:

>>> model = MixCOBRARegressor()
>>> model.fit(X_train, y_train)

`predict` ¶

Predict target values using fitted MixCOBRA aggregator.

For each test sample, this method computes distances in both input and output (prediction) spaces, combines them using optimized alpha/beta weights, and aggregates neighbor training targets.

Parameters:

Name	Type	Description	Default
`X`	`ndarray`	`Test features. Shape: (n_samples, n_features).`	required
`pred_X`	`ndarray \| None`	`Pre-computed predictions for test samples (e.g., from external models). If None, predictions are generated using internal estimators.`	`None`
`alpha`	`float \| None`	`Optional parameter overrides (currently unused; kept for API compatibility).`	`None`
`beta`	`float \| None`	`Optional parameter overrides (currently unused; kept for API compatibility).`	`None`
`bandwidth`	`float \| None`	`Optional parameter overrides (currently unused; kept for API compatibility).`	`None`

Returns:

Type	Description
`ndarray`	Predicted target values. Shape: (n_samples,).

Workflow

Generate or use provided predictions for test samples
Normalize input and prediction spaces
Compute distances in both spaces
Combine distances using learned alpha/beta weights
Transform combined distance via kernel adapter
Apply kernel to generate similarity weights
Aggregate calibration targets using kernel weights

Examples:

>>> y_pred = model.predict(X_test)

`kfc_procedure.cobra.CombinedClassifier` ¶

Bases: ABC, BaseEstimator

CombineClassifier¶

A kernel-based ensemble aggregation classifier that operates on prediction-space representations of base estimators. The model works in three main stages: 1. Train base estimators on kernel dataset 2. Transform data into prediction space 3. Perform kernel-weighted aggregation using optimized bandwidth

Parameters:

Name	Type	Description	Default
`estimators`	`list of str or BaseEstimator`	List of base estimators used to generate prediction space.	`None`
`estimators_params`	`dict`	Hyperparameters for base estimators.	`None`
`distance`	`str`	Distance metric used in prediction space.	`"hamming"`
`distance_params`	`dict`	Parameters for distance function.	`None`
`kernel`	`str`	Kernel function used to transform distances into similarities.	`"rbf"`
`kernel_params`	`dict`	Parameters for kernel function.	`None`
`aggregator`	`str`	Aggregation strategy for combining weighted labels.	`"weighted_vote"`
`aggregator_params`	`dict`	Parameters for aggregator.	`None`
`loss`	`str`	Loss function used for cross-validation optimization.	`"mse"`
`loss_params`	`dict`	Parameters for loss function.	`None`
`optimizer`	`str`	Optimization strategy for bandwidth selection.	`"grid"`
`optimizer_params`	`dict`	Parameters for optimizer.	`None`
`n_jobs`	`int`	Number of parallel jobs.	`1`
`bandwidth_list`	`array - like`	Candidate bandwidth values for optimization.	`None`
`max_iter`	`int`	Maximum iterations for optimizer search.	`300`
`n_cv`	`int`	Number of cross-validation folds.	`5`
`random_state`	`int`	Random seed for reproducibility.	`None`

`fit` ¶

Fit the CombineClassifier model.

This method: - Splits or resolves training context - Fits base estimators - Constructs prediction space - Builds kernel similarity matrix - Optimizes bandwidth via cross-validation

Returns:

Type	Description
`self`

`kfc_procedure.cobra.SuperLearner` ¶

Bases: BaseEstimator

`init` ¶

This is a class of the implementation of SuperLearner by van der Laan, M., Polley, E. and Hubbard, A. (2007): https://doi.org/10.2202/1544-6115.1309.

Parameters:

- `random_state`: (default is `None`) set the random state of the random generators in the class.

- `base_learners`: (default is None) the list of candidate learners or estimators. 
    If it is None, intial learners including 'linear_regression', 'ridge', 'lasso', 'tree', and 'random_forest' are used with default parameters.
    It should be a sublist of the following list: L = ['linear_regression', 'knn', 'ridge', 'lasso', 'tree', 'random_forest', 'svm', 'sgd', 'bayesian_ridge', 'adaboost', 'gradient_boost'].

- `base_params`: (default is `None`) a dictionary containing the parameters of the candidate learners given in the `base_learners` argument. 
    It must be a dictionary with:
    - `key`     : the name of the base learners defined in `base_learners`, 
    - `value`   : a dictionary with (key, value) = (parameter, value).

- `meta_learners`: (default is `None` and linear regression is used) meta learners that are trained on predicted features $(y_i, z_i)$ where $z_i = (r_1(x_i), ..., r_M(x_i))$ of $\mathbb{R}^M$ for $i=1,...,n$.
    It is the model that takes predicted features given by all the candidate learners as inputs. It must be an element of the list L of all the base learners.
    If a list of predictors (subset of L) is given, then the best one will be selected using CV error defined by `cv_folds`.

- `meta_params_cv`: (default is `None`) a dictionary with "keys" be the name of the candidate meta learners given in `meta_learners` argument, and the "value" is the parameter dictionary.
    For example, if two meta learners are proposed in `meta_learners = ['ridge', 'lasso']`, then this argument should be the following dictionary:
    `meta_params_cv = {
        'ridge' : {'alpha' : 2 ** np.linspace(-10,10,100)},
        'lasso' : {'alpha' : 2 ** np.linspace(-10,10,100)}
    }`
    where in this case, the panalization strenght `alpha = 2 ** np.linspace(-10,10,100)` is to be tuned using cross validation technique.

- `cv_folds`: (default is `None`) a list or an array `I` of size $n$ (observation size) whose elements are in {0,1,...,K-1}. 
    Then, $I[i]=k$ if and only if observation $i$ belongs to fold $k$ in cross-validation procedure. If `None`, then the folds are selected randomly.

 - `kernel`: (default is 'radial') the kernel function used for the aggregation. 
    It should be an element of the list ['exponential', 'gaussian', 'radial', 'cauchy', 'reverse_cosh', 'epanechnikov', 'biweight', 'triweight', 'triangular', 'cobra', 'naive'].
    Some options such as 'gaussian' and 'radial' lead to the same radial kernel function. 
    For 'cobra' or 'naive', they correspond to Biau et al. (2016).

- `loss_function`: (default is None) a function or string defining the cost function to be optimized for estimating the optimal bandwidth parameter.
    By defalut, the K-Fold cross-validation MSE is used. Otherwise, it must be either:
    - a function of two argumetns (y_true, y_pred) or
    - a string element of the list ['mse', 'mae', 'mape', 'weighted_mse']. If it is `weighted_mse`, one can define the weight for each training point using `loss_weight` argument below.

- `loss_weight`: (default is None) a list of size equals to the size of the training data defining the weight for each individual data point in the loss function. 
    If it is None and the `loss_function = weighted_mse`, then a normalized weight W(i) = 1/PDF(i) is assigned to individual i of the training data.

Returns:

self : returns an instance of self.

Methods:

- fit : fitting the super learner on the design features (original data or predicted features). The argument of this method are described below.
- train_base_learners : build base learners on CV data. It is also possible to set the values of (hyper) parameters for each base learner in `base_params`.
- add_extra_learners : to add additional learners to the list of base learner to train meta learner therefore build super learner. This can be class method or estimator, list, array or data frame of the same numer of rows as the training data.
- train_meta_learner : to train meta learner on (y_i z_i), CV predicted features. This method must be called if you add any axtra-learners to the list of base learner after calling `fit` method.
- draw_learning_curve : for plotting the graphic of learning algorithm (error vs parameter).

`fit` ¶

This method builds base and meta learner of Super learning algorithm.

Parameters:

- `X, y`: the training input and out put. If the argument `as_predictions = True`, then the input `X` is treated as predicted features `Z`. 
    In this case, the meta learner is trained directly on (X,y) without building any base learners. 
- `train_meta_learners`: a boolean variable controlling whether to directly train the meta learner or not after training the base learners given in `base_learners` argument.
    This is useful when you want to add extra learners to the list of base learners before training the meta learner. 
- `as_predictions` : a boolean variable controlling whether `X` should be treated as predicted features `Z` or not. If it is `True`, then meta learners can be trained directly on (X,y).

Important note:

You can perform CV over a list of meta learner of meta_learners argument by providing its corresponding dictionary of parameters in meta_params arguement. Moreover, you can also add features that were obtained from anonymous models by specifying in the fit method using train_meta_learners = False. In this case, fit method only train the base learners and provide predicted features (Z_i) for meta learners. After that, you can use add_extra_learners method to add extra learners to the list of base learners. These extra learners can be any "sklearn" classes, or pandas data frame, numpy arrays or list containing the same observation as the training data. If the data frame are added as additional learners, then it will be concatenated to the predicted features (Z_i) for training meta models. It is important to notice that if extra features are added as extra learners, it the corresponding extra features of the testing data must also be provided.

COBRA API¶

kfc_procedure.cobra.GradientCOBRA ¶

kfc_procedure.cobra.MixCOBRARegressor ¶

__init__ ¶

fit ¶

predict ¶

kfc_procedure.cobra.CombinedClassifier ¶

CombineClassifier¶

fit ¶

kfc_procedure.cobra.SuperLearner ¶

__init__ ¶

fit ¶

`kfc_procedure.cobra.GradientCOBRA` ¶

`kfc_procedure.cobra.MixCOBRARegressor` ¶

`init` ¶

`fit` ¶

`predict` ¶

`kfc_procedure.cobra.CombinedClassifier` ¶

`fit` ¶

`kfc_procedure.cobra.SuperLearner` ¶

`init` ¶

`fit` ¶