GAFeatureSelectionCV
Genetic algorithm wrapper-based feature selection with cross-validation.
GAFeatureSelectionCV selects a subset of columns from the input data that maximises the cross-validation score of the given estimator. After fitting, it exposes support_ (a boolean mask of selected features) and behaves as a fitted scikit-learn estimator for prediction.
Class Signature
from sklearn_genetic import GAFeatureSelectionCV
GAFeatureSelectionCV(
estimator,
*,
cv=5,
scoring=None,
refit=True,
verbose=0,
keep_top_k=1,
elitism=True,
error_score=np.nan,
return_train_score=False,
random_state=None,
evolution_config=None,
population_config=None,
runtime_config=None,
optimization_config=None,
)Parameters
| Parameter | Type | Default | Description |
|---|---|---|---|
estimator | estimator | — | A scikit-learn estimator with a fit method |
cv | int or CV splitter | 5 | Cross-validation strategy |
scoring | str, callable or dict | None | Metric to evaluate. None uses the estimator's default scorer |
refit | bool or str | True | Refit the best estimator on the full training data after search; name the metric for multi-metric scoring |
verbose | int | 0 | Verbosity level |
keep_top_k | int | 1 | Number of hall-of-fame individuals to keep across generations |
elitism | bool | True | Whether to carry over the best individual to the next generation |
error_score | float or "raise" | np.nan | Score to assign when a candidate raises an exception |
random_state | int, RandomState or None | None | Seeds the whole search at fit time for reproducible runs. None is non-deterministic |
evolution_config | EvolutionConfig | None | Population size, generations, crossover/mutation rates |
population_config | PopulationConfig | None | Initialization strategy and diversity settings |
runtime_config | RuntimeConfig | None | Parallelism, caching, verbosity |
optimization_config | OptimizationConfig | None | Local search, fitness sharing |
Search correctness
Each fresh fit recomputes candidate scores for the supplied data and scoring settings. Caching avoids repeated evaluations within that fit; explicit checkpoint resumes retain saved evaluations. Both elitist and non-elitist searches use tournament selection, so negative scores and criteria="min" remain meaningful. Scikit-learn's negative loss scorers normally use the default criteria="max".
Smart initialization spreads target mask sizes across the allowed range. It samples column indices without replacement, drawing the omitted columns for dense masks. This reduces initialization cost on wide data. Within the same version, set random_state to reproduce the initial masks. If caching is enabled and the population exceeds the number of possible masks, smart initialization reuses duplicate draws to fill the remaining slots with independent individuals.
max_features must be None or an integer from 1 through the number of input columns. Initialisation, crossover and mutation repair masks to keep at least one feature and respect that upper bound.
Attributes After fit
| Attribute | Description |
|---|---|
support_ | Boolean mask — True for selected features |
best_score_ | Winning finalist CV score, available when final_selection=True and refit=True |
best_estimator_ | Estimator fitted with the selected features on the full training data |
cv_results_ | Dict with per-candidate results |
history | Dictionary of lists aligned by generation. Pair history["cross_validate_calls"] with history["fitness_best"] to track the best search score against the number of mask evaluations. |
logbook | DEAP logbook with per-candidate evaluation records in logbook.chapters["parameters"] |
fit_stats_ | Evaluation counters |
n_features_ | Number of selected features |
feature_names_in_ | Feature names seen during fit when X has string column names |
See plotting score against evaluation cost to compare runs with different cache-hit rates.
Methods
| Method | Description |
|---|---|
fit(X, y, callbacks=None) | Run the genetic feature selection |
cv_results_dataframe() | Return feature masks and scores as a pandas DataFrame sorted by test rank |
transform(X) | Return X with only the selected features |
get_feature_names_out(input_features=None) | Return the selected feature names |
predict(X) | Predict using best_estimator_ on selected features |
predict_proba(X) | Predict class probabilities |
score(X, y) | Score the selected features with the configured scorer, or the refit metric for multi-metric scoring |
With a sparse-compatible estimator, you can pass a SciPy sparse matrix directly to fit. The selector keeps CSR and CSC training matrices in their original formats and converts other sparse formats, including COO, to CSC before selecting columns.
score(X, y) evaluates the refitted estimator on new data, so it can differ from the candidate CV scores in cv_results_ and from best_score_ when finalist selection is enabled. With a custom scorer that includes a feature-count penalty, score(X, y) includes that penalty too; calculate an unpenalized test metric separately when needed.
cv_results_dataframe() uses the selected refit metric for multi-metric searches and keeps evaluation order for tied ranks. Its params column contains each candidate's feature mask. It leaves cv_results_ unchanged and also works after a single-metric fit with refit=False.
Example
import numpy as np
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.svm import SVC
from sklearn_genetic import EvolutionConfig, GAFeatureSelectionCV
X, y = load_iris(return_X_y=True)
rng = np.random.default_rng(42)
noise = rng.uniform(0, 10, size=(X.shape[0], 10))
X = np.hstack((X, noise))
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=0)
selector = GAFeatureSelectionCV(
estimator=SVC(gamma="auto"),
cv=3,
scoring="accuracy",
evolution_config=EvolutionConfig(population_size=6, generations=2),
random_state=42,
)
selector.fit(X_train, y_train)
print("Selected features:", selector.support_)
print("Search CV evaluations:", selector.history["cross_validate_calls"])
print("Best search CV accuracy:", round(np.nanmax(selector.cv_results_["mean_test_score"]), 4))
print("Test accuracy:", round(selector.score(X_test, y_test), 4))
print(selector.cv_results_dataframe()[["n_features", "mean_test_score", "rank_test_score"]].head())See Also
- Basic Usage — feature selection tutorial
- Config Objects — configuration objects
- GASearchCV — hyperparameter tuning variant