Majority Vote

MajorityVoteCombiner is the simplest classification consensus strategy in the KFC C-Step.

It combines the divergence-specific class predictions produced by the F-Step using hard voting:

\[ \boxed{ \text{F-Step class predictions} \rightarrow \text{row-wise majority vote} \rightarrow \text{final class label} } \]

The implementation is registered under:

majority_vote

and lives in:

kfc_procedure/core/combiner/classification/majority_vote.py

The source describes it as:

Hard voting ensemble combiner.

Each sample is assigned the most frequent label among base models.

Core idea

Suppose the F-Step produces one class prediction per divergence.

For one observation:

Euclidean     -> 1
GKL           -> 1
Logistic      -> 0
Itakura-Saito -> 1

The prediction row is:

[1, 1, 0, 1]

and the majority-vote result is:

Mathematically, for prediction row

\[ (c_1,\ldots,c_M), \]

the combiner returns the most frequently occurring label.

Input matrix

The combiner expects a two-dimensional prediction matrix:

\[ P \in \mathbb{R}^{n\times M}, \]

where:

\(n\) is the number of observations;
\(M\) is the number of prediction columns.

In KFC, \(M\) is normally the number of configured divergences.

For example:

             euclidean      gkl      logistic      is
sample 1          0           0           1          0
sample 2          1           1           1          1
sample 3          2           1           2          2

The combiner applies voting independently to each row.

Implementation

The source converts the input with:

X = np.asarray(X)

then validates:

if X.ndim != 2:
    raise ValueError(
        f"Expected 2D array, got {X.shape}"
    )

It allocates one output value per sample:

outputs = np.empty(
    n_samples,
    dtype=object,
)

Then, for each row:

row = X[i]

outputs[i] = Counter(
    row
).most_common(1)[0][0]

The complete behavior is therefore a direct row-wise mode calculation.

Stateless fitting

MajorityVoteCombiner does not learn parameters.

Its fit() method is:

def fit(
    self,
    X,
    y=None,
):
    return self

The target values are not used.

So:

combiner.fit(
    P_train,
    y_train,
)

does not change the voting rule.

This differs from learned classification combiners such as:

stacking_classifier
combined_classifier

Direct example

import numpy as np

from kfc_procedure.core.combiner.classification import (
    MajorityVoteCombiner,
)


P = np.array([
    [0, 0, 1],
    [1, 1, 1],
    [2, 1, 2],
])


combiner = MajorityVoteCombiner()

combiner.fit(P)

prediction = combiner.predict(P)

print(
    prediction
)

The result is conceptually:

[0, 1, 2]

`predict()` and `combine()`

MajorityVoteCombiner inherits:

predict()

from:

BaseCombiner

The base implementation is:

def predict(
    self,
    X,
):
    return self.combine(X)

Therefore:

combiner.predict(P)

and:

combiner.combine(P)

use exactly the same majority-vote logic.

Tie behavior

The current source does not implement a separate tie-breaking rule.

It uses:

Counter(
    row
).most_common(1)[0][0]

directly.

For example, a row such as:

[0, 1]

contains a tie.

The result follows the behavior of Python's:

collections.Counter.most_common()

for equally frequent elements.

No KFC-specific tie policy

The package does not define a separate rule such as:

smallest class wins
first divergence wins
random tie-breaking
confidence-weighted tie-breaking

If your application needs an explicit tie policy, implement a custom combiner.

Output dtype

The combiner creates:

np.empty(
    n_samples,
    dtype=object,
)

for its result.

That means the combiner itself can represent labels such as:

0
1
2

or:

"class_a"
"class_b"

provided those labels reach the combiner successfully.

Important upstream label limitation

Although MajorityVoteCombiner itself uses an object-dtype output array, the current F-Step initializes each divergence prediction vector with:

np.full(
    X.shape[0],
    np.nan,
)

which produces a floating-point array by default.

Therefore string-valued local classifier predictions may fail during F-Step assignment before they ever reach the majority-vote combiner.

Current full-pipeline behavior

Numeric class labels are the safest choice with the current KFC classification pipeline.

The majority-vote combiner is more permissive than the upstream F-Step prediction buffer.

KFC usage

Conceptually, the registered KFC configuration is:

from kfc_procedure import KFCClassifier

model = KFCClassifier(
    divergences=[
        "euclidean",
        "gkl",
    ],
    local_model="logistic_regression",
    combiner="majority_vote",
    random_state=42,
)

However, the current C-Step source has a constructor-parameter issue for this string path.

Current C-Step `random_state` issue

When C-Step receives a string combiner, it copies:

combiner_params

and then injects:

random_state=self.random_state

if the key is not already present.

The relevant logic is:

params = dict(
    self.combiner_params
)

if "random_state" not in params:
    params["random_state"] = self.random_state

return CombinerFactory.create(
    name,
    **params
)

But MajorityVoteCombiner defines no constructor at all.

It therefore inherits the default constructor behavior and does not accept:

random_state

as a keyword argument.

Result

A string-based C-Step can attempt the equivalent of:

MajorityVoteCombiner(
    random_state=42
)

or:

MajorityVoteCombiner(
    random_state=None
)

and raise a TypeError.

Current source limitation

In the current source version, the string identifier:

majority_vote

is affected by C-Step's automatic random_state injection.

This is a constructor-forwarding issue, not a limitation of the voting algorithm.

Current workaround

Pass a pre-instantiated combiner object:

from kfc_procedure import KFCClassifier

from kfc_procedure.core.combiner.classification import (
    MajorityVoteCombiner,
)


model = KFCClassifier(
    divergences=[
        "euclidean",
        "gkl",
    ],
    local_model="logistic_regression",
    combiner=MajorityVoteCombiner(),
    random_state=42,
)

For non-string combiners, C-Step does:

if not isinstance(
    self.combiner,
    str,
):
    return self.combiner

so no constructor arguments are injected.

How KFC produces the vote matrix

KFC first fits K-Step and F-Step using the internal K/F subset.

For new data:

clusters = model.kstep_.predict(
    X_test
)

produces one cluster assignment per divergence.

Then:

P = model.fstep_.predict(
    X_test,
    clusters,
)

produces one class prediction per divergence.

Finally:

y_pred = model.cstep_.predict(
    P
)

runs majority voting.

Prediction flow

flowchart LR
    X["New sample"]

    K1["Euclidean cluster"]
    K2["GKL cluster"]
    K3["Other divergences"]

    F1["Local classifier"]
    F2["Local classifier"]
    F3["Local classifier"]

    P["Prediction row"]
    V["Majority vote"]
    Y["Final label"]

    X --> K1 --> F1 --> P
    X --> K2 --> F2 --> P
    X --> K3 --> F3 --> P

    P --> V --> Y

The majority combiner does not know which cluster produced each label.

It receives only the class-prediction row.

Inspect the vote inputs

After fitting:

clusters = model.kstep_.predict(
    X_test
)

P = model.fstep_.predict(
    X_test,
    clusters,
)

print(
    P[:10]
)

This shows the exact rows sent to majority voting.

Inspect divergence column names

The column order can be recovered with:

divergence_names = list(
    model.fstep_.models_.keys()
)

print(
    divergence_names
)

Then inspect one sample:

row = P[0]

for divergence, label in zip(
    divergence_names,
    row,
):
    print(
        divergence,
        label,
    )

Inspect final majority decisions

final = model.cstep_.predict(
    P
)

for i in range(
    min(
        10,
        len(P),
    )
):
    print(
        P[i],
        "->",
        final[i],
    )

This is one of the easiest ways to understand the C-Step behavior for a fitted classification model.

One divergence

If KFC uses:

divergences=[
    "euclidean"
]

then the prediction matrix has one column:

(n_samples, 1)

For a row such as:

[1]

the majority vote is necessarily:

So with one divergence, MajorityVoteCombiner does not alter the F-Step prediction.

Two divergences

With two divergences:

[0, 0]

has an unambiguous majority.

But:

[0, 1]

is a tie.

Because no explicit KFC tie rule is implemented, the output follows Counter.most_common() behavior.

If ties are common in a two-divergence configuration, consider whether an explicit custom rule is needed.

Odd vs even numbers of prediction columns

The package does not enforce an odd number of divergences.

An odd number can reduce the frequency of simple two-class ties, but KFC does not automatically add, remove, or reorder divergences for voting purposes.

Likewise, multiclass voting can still produce ties even with an odd number of prediction columns.

This is a property of the prediction pattern, not a special check in the combiner.

Majority vote does not use confidence

The combiner receives hard class labels only.

It does not consume:

class probabilities
decision scores
distance to cluster center
local classifier confidence
kernel weights

The source simply counts labels in each row.

So these two rows are identical to the combiner:

classifier A predicts class 1 with probability 0.51
classifier B predicts class 1 with probability 0.99

if both enter the C-Step as:

No learned divergence weights

Unlike a learned meta-classifier, majority vote gives no trainable importance parameter to one divergence over another.

Each prediction occupies one position in the row and contributes one vote.

Conceptually:

\[ \text{vote weight per column} = 1. \]

There is no fitted coefficient vector.

No target use during C-Step fitting

Even though CStep.fit() calls:

strategy.fit(
    X,
    y,
)

the majority-vote implementation ignores y.

So the C-Step target labels do not modify the voting rule.

No `predict_proba()`

MajorityVoteCombiner implements only:

fit()
combine()

and inherits:

predict()

from BaseCombiner.

It does not implement:

predict_proba()

Therefore:

CStep.predict_proba()

cannot delegate probability prediction to this strategy.

C-Step behavior

For a classification C-Step:

cstep.predict_proba(P)

checks:

hasattr(
    self.strategy_,
    "predict_proba"
)

For MajorityVoteCombiner, this is false.

The C-Step raises an AttributeError indicating that the strategy does not support probability prediction.

Majority vote vs stacking classifier

The two strategies use the same kind of F-Step input matrix but process it very differently.

majority_vote
    prediction row
        ↓
    count labels
        ↓
    most frequent label

versus:

stacking_classifier
    prediction row
        ↓
    fitted meta-classifier
        ↓
    learned final label

Majority vote is stateless.

Stacking is supervised and can learn different relationships between prediction columns.

See:

Stacking

Majority vote vs CombinedClassifier

CombinedClassifier is also learned.

Its KFC wrapper uses the F-Step matrix as prediction-space input and applies distance/kernel-based consensus.

Conceptually:

majority_vote
    row-wise class counts

versus:

combined_classifier
    compare prediction vectors
        ↓
    distance
        ↓
    kernel weighting
        ↓
    weighted class consensus

For the full algorithm, see:

CombinedClassifier

Majority vote with multiclass labels

The source makes no binary-only assumption.

For example:

[2, 1, 2, 2]

produces:

and:

[0, 1, 2, 2, 2]

also produces:

The only requirement at the combiner level is that labels be countable by Counter.

Direct string-label example

The combiner itself can handle object labels:

import numpy as np

from kfc_procedure.core.combiner.classification import (
    MajorityVoteCombiner,
)


P = np.array(
    [
        ["cat", "cat", "dog"],
        ["dog", "dog", "cat"],
    ],
    dtype=object,
)

combiner = MajorityVoteCombiner()

prediction = combiner.predict(P)

print(
    prediction
)

Conceptually:

["cat", "dog"]

Again, this demonstrates the combiner itself; the current full F-Step pipeline has the separate string-label buffer caveat described earlier.

Custom tie policy

If you need explicit tie handling, subclass BaseCombiner.

For example:

import numpy as np
from collections import Counter

from kfc_procedure.core.combiner.base import (
    BaseCombiner,
)


class SmallestLabelVote(
    BaseCombiner
):

    def fit(
        self,
        X,
        y=None,
    ):
        return self

    def combine(
        self,
        X,
    ):
        X = np.asarray(X)

        output = np.empty(
            len(X),
            dtype=object,
        )

        for i, row in enumerate(X):
            counts = Counter(row)

            highest = max(
                counts.values()
            )

            tied = [
                label
                for label, count
                in counts.items()
                if count == highest
            ]

            output[i] = min(tied)

        return output

This example uses an explicit smallest-label rule rather than relying on Counter.most_common() tie behavior.

Register a custom vote combiner

To use a custom strategy by name:

from kfc_procedure.core.combiner import (
    CombinerFactory,
)


@CombinerFactory.register(
    "smallest_label_vote",
    categories={"classification"},
)
class SmallestLabelVote(
    BaseCombiner
):

    def __init__(
        self,
        random_state=None,
    ):
        self.random_state = random_state

    ...

The random_state constructor parameter is included because the current C-Step injects that keyword into all string-based combiner construction.

Debugging majority vote

1. Check F-Step matrix shape

print(
    P.shape
)

Expected:

(n_samples, n_divergences)

2. Inspect rows directly

print(
    P[:10]
)

3. Check divergence order

print(
    list(
        model.fstep_.models_.keys()
    )
)

4. Compare manual counts

from collections import Counter

row = P[0]

print(
    Counter(row)
)

print(
    Counter(row).most_common(1)
)

5. Compare with C-Step output

print(
    model.cstep_.predict(
        P[:1]
    )
)

6. Check constructor error

If the error contains:

unexpected keyword argument 'random_state'

and the configured string is:

majority_vote

use:

combiner=MajorityVoteCombiner()

with the current source version.

Quick reference

Property	Current behavior
Registry name	`majority_vote`
Task	classification
Learning	none
Uses `y` during fit	No
Input	2D class-prediction matrix
Aggregation	row-wise mode
Implementation	`Counter(row).most_common(1)[0][0]`
Explicit tie policy	No
Output dtype	`object`
`predict_proba()`	No
Learned coefficients	none
Current string KFC path	affected by C-Step `random_state` injection

Mental model

Majority vote treats every divergence prediction as one hard vote and returns the label that appears most often in the row.

\[ \boxed{ [c_1,c_2,\ldots,c_M] \rightarrow \operatorname{mode} \rightarrow \widehat c } \]

It is the simplest classification consensus rule in the current source: stateless, unweighted, and based only on hard labels.

Majority Vote

Core idea

Input matrix

Implementation

Stateless fitting

Direct example

predict() and combine()

Tie behavior

Output dtype

Important upstream label limitation

KFC usage

Current C-Step random_state issue

Result

Current workaround

How KFC produces the vote matrix

Prediction flow

Inspect the vote inputs

Inspect divergence column names

Inspect final majority decisions

One divergence

Two divergences

Odd vs even numbers of prediction columns

Majority vote does not use confidence

No learned divergence weights

No target use during C-Step fitting

No predict_proba()

C-Step behavior

Majority vote vs stacking classifier

Majority vote vs CombinedClassifier

Majority vote with multiclass labels

Direct string-label example

Custom tie policy

Register a custom vote combiner

Debugging majority vote

1. Check F-Step matrix shape

2. Inspect rows directly

3. Check divergence order

4. Compare manual counts

5. Compare with C-Step output

6. Check constructor error

Quick reference

Mental model

`predict()` and `combine()`

Current C-Step `random_state` issue

No `predict_proba()`