Skip to content

GAFeatureSelectionCV ​

Genetic algorithm wrapper-based feature selection with cross-validation.

GAFeatureSelectionCV selects a subset of columns from the input data that maximises the cross-validation score of the given estimator. After fitting, it exposes support_ (a boolean mask of selected features) and behaves as a fitted scikit-learn estimator for prediction.

Class Signature ​

python
from sklearn_genetic import GAFeatureSelectionCV

GAFeatureSelectionCV(
    estimator,
    *,
    cv=5,
    scoring=None,
    refit=True,
    verbose=0,
    keep_top_k=1,
    elitism=True,
    error_score=np.nan,
    return_train_score=False,
    random_state=None,
    evolution_config=None,
    population_config=None,
    runtime_config=None,
    optimization_config=None,
)

Parameters ​

ParameterTypeDefaultDescription
estimatorestimator—A scikit-learn estimator with a fit method
cvint or CV splitter5Cross-validation strategy
scoringstr, callable or dictNoneMetric to evaluate. None uses the estimator's default scorer
refitbool or strTrueRefit the best estimator on the full training data after search; name the metric for multi-metric scoring
verboseint0Verbosity level
keep_top_kint1Number of hall-of-fame individuals to keep across generations
elitismboolTrueWhether to carry over the best individual to the next generation
error_scorefloat or "raise"np.nanScore to assign when a candidate raises an exception
random_stateint, RandomState or NoneNoneSeeds the whole search at fit time for reproducible runs. None is non-deterministic
evolution_configEvolutionConfigNonePopulation size, generations, crossover/mutation rates
population_configPopulationConfigNoneInitialization strategy and diversity settings
runtime_configRuntimeConfigNoneParallelism, caching, verbosity
optimization_configOptimizationConfigNoneLocal search, fitness sharing

Search correctness ​

Each fresh fit recomputes candidate scores for the supplied data and scoring settings. Caching avoids repeated evaluations within that fit; explicit checkpoint resumes retain saved evaluations. Both elitist and non-elitist searches use tournament selection, so negative scores and criteria="min" remain meaningful. Scikit-learn's negative loss scorers normally use the default criteria="max".

Smart initialization spreads target mask sizes across the allowed range. It samples column indices without replacement, drawing the omitted columns for dense masks. This reduces initialization cost on wide data. Within the same version, set random_state to reproduce the initial masks. If caching is enabled and the population exceeds the number of possible masks, smart initialization reuses duplicate draws to fill the remaining slots with independent individuals.

max_features must be None or an integer from 1 through the number of input columns. Initialisation, crossover and mutation repair masks to keep at least one feature and respect that upper bound.

Attributes After fit ​

AttributeDescription
support_Boolean mask — True for selected features
best_score_Winning finalist CV score, available when final_selection=True and refit=True
best_estimator_Estimator fitted with the selected features on the full training data
cv_results_Dict with per-candidate results
historyDictionary of lists aligned by generation. Pair history["cross_validate_calls"] with history["fitness_best"] to track the best search score against the number of mask evaluations.
logbookDEAP logbook with per-candidate evaluation records in logbook.chapters["parameters"]
fit_stats_Evaluation counters
n_features_Number of selected features
feature_names_in_Feature names seen during fit when X has string column names

See plotting score against evaluation cost to compare runs with different cache-hit rates.

Methods ​

MethodDescription
fit(X, y, callbacks=None)Run the genetic feature selection
cv_results_dataframe()Return feature masks and scores as a pandas DataFrame sorted by test rank
transform(X)Return X with only the selected features
get_feature_names_out(input_features=None)Return the selected feature names
predict(X)Predict using best_estimator_ on selected features
predict_proba(X)Predict class probabilities
score(X, y)Score the selected features with the configured scorer, or the refit metric for multi-metric scoring

With a sparse-compatible estimator, you can pass a SciPy sparse matrix directly to fit. The selector keeps CSR and CSC training matrices in their original formats and converts other sparse formats, including COO, to CSC before selecting columns.

score(X, y) evaluates the refitted estimator on new data, so it can differ from the candidate CV scores in cv_results_ and from best_score_ when finalist selection is enabled. With a custom scorer that includes a feature-count penalty, score(X, y) includes that penalty too; calculate an unpenalized test metric separately when needed.

cv_results_dataframe() uses the selected refit metric for multi-metric searches and keeps evaluation order for tied ranks. Its params column contains each candidate's feature mask. It leaves cv_results_ unchanged and also works after a single-metric fit with refit=False.

Example ​

python
import numpy as np
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.svm import SVC

from sklearn_genetic import EvolutionConfig, GAFeatureSelectionCV

X, y = load_iris(return_X_y=True)
rng = np.random.default_rng(42)
noise = rng.uniform(0, 10, size=(X.shape[0], 10))
X = np.hstack((X, noise))

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=0)

selector = GAFeatureSelectionCV(
    estimator=SVC(gamma="auto"),
    cv=3,
    scoring="accuracy",
    evolution_config=EvolutionConfig(population_size=6, generations=2),
    random_state=42,
)

selector.fit(X_train, y_train)

print("Selected features:", selector.support_)
print("Search CV evaluations:", selector.history["cross_validate_calls"])
print("Best search CV accuracy:", round(np.nanmax(selector.cv_results_["mean_test_score"]), 4))
print("Test accuracy:", round(selector.score(X_test, y_test), 4))
print(selector.cv_results_dataframe()[["n_features", "mean_test_score", "rank_test_score"]].head())

See Also ​

Released under the MIT License.