Imbalanced Datasets: Practical Machine Learning Techniques

You have 10,000 transactions. 9,500 are legitimate. 500 are fraudulent. You train a model. It achieves 95% accuracy by predicting "legitimate" for everything.

It catches zero fraud.

This is the imbalanced dataset problem. When one class dominates, models learn to ignore the minority class.

This article explains 5 practical techniques to handle imbalanced data, shows Python examples for each, and helps you choose the right approach.

The Imbalanced Data Problem

import pandas as pd
import numpy as np

# Simulate imbalanced dataset
np.random.seed(42)
n_samples = 10000

# 95% legitimate, 5% fraudulent
X = np.random.randn(n_samples, 10)
y = np.array([0] * 9500 + [1] * 500)

print(f"Class distribution:")
print(f"  Legitimate (0): {sum(y == 0)} ({sum(y == 0)/len(y)*100:.1f}%)")
print(f"  Fraudulent (1): {sum(y == 1)} ({sum(y == 1)/len(y)*100:.1f}%)")

# A naive model that always predicts 0
from sklearn.metrics import accuracy_score, f1_score

y_pred_naive = np.zeros(len(y))
print(f"\nNaive model accuracy: {accuracy_score(y, y_pred_naive):.3f}")
print(f"Naive model F1: {f1_score(y, y_pred_naive):.3f}")
# Accuracy: 0.950, F1: 0.000

Technique 1: Oversampling

Oversampling increases the number of minority class samples by duplicating them.

from imblearn.over_sampling import RandomOverSampler

# Oversample minority class
ros = RandomOverSampler(random_state=42)
X_resampled, y_resampled = ros.fit_resample(X, y)

print(f"Before: {np.bincount(y)}")
print(f"After:  {np.bincount(y_resampled)}")
# Before: [9500  500]
# After:  [9500 9500]

Pros and Cons

Technique 2: Undersampling

Undersampling reduces the number of majority class samples.

from imblearn.under_sampling import RandomUnderSampler

# Undersample majority class
rus = RandomUnderSampler(random_state=42)
X_resampled, y_resampled = rus.fit_resample(X, y)

print(f"Before: {np.bincount(y)}")
print(f"After:  {np.bincount(y_resampled)}")
# Before: [9500  500]
# After:  [500 500]

Pros and Cons

Technique 3: SMOTE

SMOTE (Synthetic Minority Over-sampling Technique) creates synthetic samples by interpolating between existing minority samples.

from imblearn.over_sampling import SMOTE

# SMOTE creates synthetic minority samples
smote = SMOTE(random_state=42)
X_resampled, y_resampled = smote.fit_resample(X, y)

print(f"Before: {np.bincount(y)}")
print(f"After:  {np.bincount(y_resampled)}")
# Before: [9500  500]
# After:  [9500 9500]

How SMOTE Works

1. Select a minority sample
2. Find its k nearest minority neighbors
3. Randomly select a neighbor
4. Create a synthetic point on the line between them
5. Repeat until desired number of samples

from imblearn.over_sampling import SMOTE, BorderlineSMOTE, SVMSMOTE

# Standard SMOTE
smote = SMOTE(random_state=42)

# Borderline-SMOTE (focuses on decision boundary)
borderline_smote = BorderlineSMOTE(random_state=42)

# SVM-SMOTE (uses SVM to find boundary)
svm_smote = SVMSMOTE(random_state=42)

# Compare results
for name, sampler in [
    ("SMOTE", smote),
    ("Borderline-SMOTE", borderline_smote),
    ("SVM-SMOTE", svm_smote)
]:
    X_res, y_res = sampler.fit_resample(X, y)
    print(f"{name}: {np.bincount(y_res)}")

Pros and Cons

Technique 4: Class Weights

Class weights penalize the model more for misclassifying the minority class.

from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

# Without class weights
model_no_weights = RandomForestClassifier(random_state=42)
model_no_weights.fit(X_train, y_train)
y_pred = model_no_weights.predict(X_test)
print("Without class weights:")
print(classification_report(y_test, y_pred))

# With class weights
model_weights = RandomForestClassifier(
    class_weight='balanced',  # Automatically adjusts weights
    random_state=42
)
model_weights.fit(X_train, y_train)
y_pred = model_weights.predict(X_test)
print("With class weights:")
print(classification_report(y_test, y_pred))

Custom Class Weights

from sklearn.utils.class_weight import compute_class_weight

# Compute weights automatically
classes = np.unique(y_train)
weights = compute_class_weight('balanced', classes=classes, y=y_train)
weight_dict = dict(zip(classes, weights))

print(f"Class weights: {weight_dict}")
# {0: 0.526, 1: 10.0}

# Or set manually
model_custom = RandomForestClassifier(
    class_weight={0: 1, 1: 10},  # Minority class weighted 10x
    random_state=42
)

Technique 5: Threshold Adjustment

Instead of using 0.5 as the decision threshold, adjust it to favor the minority class.

from sklearn.metrics import precision_recall_curve
import matplotlib.pyplot as plt

# Get predicted probabilities
y_proba = model_weights.predict_proba(X_test)[:, 1]

# Find optimal threshold
precision, recall, thresholds = precision_recall_curve(y_test, y_proba)

# Calculate F1 for each threshold
f1_scores = 2 * (precision * recall) / (precision + recall + 1e-10)
optimal_idx = np.argmax(f1_scores)
optimal_threshold = thresholds[optimal_idx]

print(f"Optimal threshold: {optimal_threshold:.3f}")
print(f"F1 at optimal threshold: {f1_scores[optimal_idx]:.3f}")

# Apply optimal threshold
y_pred_optimal = (y_proba >= optimal_threshold).astype(int)
print(classification_report(y_test, y_pred_optimal))

Complete Comparison

from sklearn.model_selection import cross_val_score
from imblearn.pipeline import Pipeline as ImbPipeline

# Define pipelines for each technique
pipelines = {
    'Baseline': ImbPipeline([
        ('classifier', RandomForestClassifier(random_state=42))
    ]),
    'Oversampling': ImbPipeline([
        ('oversampler', RandomOverSampler(random_state=42)),
        ('classifier', RandomForestClassifier(random_state=42))
    ]),
    'Undersampling': ImbPipeline([
        ('undersampler', RandomUnderSampler(random_state=42)),
        ('classifier', RandomForestClassifier(random_state=42))
    ]),
    'SMOTE': ImbPipeline([
        ('smote', SMOTE(random_state=42)),
        ('classifier', RandomForestClassifier(random_state=42))
    ]),
    'Class Weights': ImbPipeline([
        ('classifier', RandomForestClassifier(
            class_weight='balanced', random_state=42
        ))
    ])
}

# Evaluate each pipeline
results = {}
for name, pipeline in pipelines.items():
    scores = cross_val_score(pipeline, X_train, y_train, cv=5, scoring='f1')
    results[name] = {
        'mean_f1': scores.mean(),
        'std_f1': scores.std()
    }
    print(f"{name}: F1 = {scores.mean():.3f} ± {scores.std():.3f}")

When to Use Each Technique

Try It Yourself

• Model Evaluation Beyond Accuracy — Precision, recall, F1 explained
• Data Leakage in ML — Avoid contamination in resampling
• Train/Val/Test Sets — Split correctly before resampling
• ML Production Failures — Why models fail after deployment

Further Reading

• ML Mastery: SMOTE for Imbalanced Classification
• Imbalanced-learn Documentation
• Scikit-learn: Threshold Moving
• BestWordz: Model Evaluation Beyond Accuracy

Conclusion

Imbalanced datasets are common in real-world machine learning: fraud detection, disease diagnosis, spam filtering, anomaly detection. The naive approach of training on imbalanced data produces models that ignore the minority class.

The five techniques — oversampling, undersampling, SMOTE, class weights, and threshold adjustment — each have strengths and weaknesses. The right choice depends on your dataset size, model type, and business requirements.

Key principles:

• Never evaluate with accuracy alone — Use F1, PR-AUC, and confusion matrices
• Resample only training data — Keep test data untouched
• Use imblearn.pipeline — Prevent data leakage in cross-validation
• Try multiple techniques — Compare with cross-validation
• Consider the business cost — False positives vs false negatives

The best technique is the one that improves your specific metric on your specific data.