Feature Scaling and Normalization
Feature Scaling and Normalization in Machine Learning
Feature Scaling and Normalization in Machine Learning are essential preprocessing techniques used to standardize the range and distribution of data. In real world datasets, different features often have different scales, for example, age may range from 18 – 60, while salary may range from thousands to lakhs.
Without scaling, machine learning algorithms can become biased toward features with larger values. Feature scaling ensures that all features contribute equally, improving model performance, accuracy, and convergence speed.
What is Feature Scaling in Machine Learning?
Feature scaling is the process of transforming numerical data so that all features are on a similar scale.
Why it is needed:
- Prevents dominance of large value features
- Improves gradient based model performance
- Helps faster convergence
- Ensures fair comparison between variables
What is Normalization in Machine Learning?
Normalization is a type of feature scaling where values are scaled to a fixed range, typically 0 to 1.
Formula (Min-Max Normalization):
This ensures:
- All values lie between 0 and 1
- Data distribution is preserved proportionally
Types of Feature Scaling Techniques
1. Min-Max Scaling (Normalization)
- Scales data between 0 and 1
- Sensitive to outliers
Example:
- If salary ranges from 10,000 to 100,000 → scaled between 0 and 1
2. Standardization (Z-score Normalization)
Transforms data to mean = 0 and standard deviation = 1
Less affected by outliers compared to Min-Max
3. Robust Scaling
- Uses median and interquartile range (IQR)
- Works well with outliers
4. Max Absolute Scaling
- Scales values between -1 and 1
- Useful for sparse datasets
Feature Scaling vs Normalization
| Feature | Feature Scaling | Normalization |
|---|---|---|
| Definition | General scaling method | Specific scaling (0–1 range) |
| Range | Varies | 0 to 1 |
| Use Case | Most ML models | Distance-based models |
| Sensitivity to Outliers | Depends | High |
When to Use Feature Scaling
Feature scaling is required in
- K-Means clustering
- K-Nearest Neighbors (KNN)
- Logistic Regression
- Neural Networks
- Gradient Descent algorithms
Not required in:
- Decision Trees
- Random Forest
Why Feature Scaling is Important
- Improves model accuracy
- Speeds up training
- Prevents bias toward large features
- Helps optimization algorithms converge faster
Practical Implementation in Python:
from sklearn.preprocessing import MinMaxScaler, StandardScaler
import pandas as pd
# Sample data
data = {'Age': [20, 25, 30, 35, 40],
'Salary': [20000, 30000, 40000, 50000, 60000]}
df = pd.DataFrame(data)
# Min-Max Scaling
minmax = MinMaxScaler()
df_minmax = minmax.fit_transform(df)
# Standardization
standard = StandardScaler()
df_standard = standard.fit_transform(df)
print("Min-Max Scaled:\n", df_minmax)
print("\nStandardized:\n", df_standard)
Real world example for feature scaling:
Consider a dataset:
- Age: 20–50
- Salary: 20,000–1,00,000
Without scaling: Salary dominates model
After scaling: Both features contribute equally
Result: Better model performance and accuracy
Common Mistakes to Avoid….
- Applying scaling before splitting data
- Ignoring outliers
- Using wrong scaling method
- Scaling categorical data unnecessarily
Frequently Asked Questions
Answer:
Feature scaling is the process of standardizing data so that all features have a similar range, improving model performance.
Answer:
Normalization scales data between 0 and 1 using Min-Max scaling.
Answer:
Normalization scales data between 0–1, while standardization transforms data to mean 0 and standard deviation 1.
Answer:
It ensures that all features contribute equally and improves model accuracy.
Answer:
It is used in distance based and gradient based algorithms like KNN and K Means.
