Feature Scaling Part 3: Advanced Techniques

Beyond the basics: explore MaxAbs Scaling, Power and Quantile Transformations, Vector Normalization, and custom scaling strategies. Includes Python and R code examples with a focus on sparse data and practical ML use cases.
Fundamentals
Data Preprocessing
Scaling
Sparse Data
Series: Feature Scaling
Author

January Tabaka

Published

August 27, 2025

In the previous post we covered standardization, normalization and robust scaling. In this post we will take a look at other scaling techniques.

MaxAbs Scaling

While similar to Min-Max Scaling, this method is purpose-built for data where the zero point is meaningful. It excels with sparse data (e.g., TF-IDF vectors), as it preserves the crucial zero entries, and with naturally centered data like audio waveforms (where 0 means silence) or financial price changes (where 0 means no change).

Principle: It scales each feature by its maximum absolute value so that the largest absolute value becomes 1 (or −1). The actual range depends on the data’s signs.

  • If your feature values are all non-negative, the scaled values will be in the range [0, 1].
  • If your feature values are all non-positive, the scaled values will be in the range [-1, 0].

Formula: \[X_{scaled} = \frac{X}{max(|X|)}\] Where:

  • \(X\) - is the vector of original values (the transformation is applied element-wise).

When to use it: Its primary advantage is that it does not shift or center the data. This is crucial for sparse datasets (matrices with many zeros), as you wouldn’t want to turn all the zeros into a non-zero number. Centering (subtracting mean/median) breaks sparsity.

Caution

In scikit-learn, use StandardScaler(with_mean=False) or RobustScaler(with_centering=False) to avoid densifying. Centering a sparse matrix is not supported and raises an error.

from sklearn.preprocessing import MaxAbsScaler
import numpy as np

# Data with both positive and negative values, and a zero
data = np.array([[-20], [-10], [0], [0], [0], [10], [100]])

# Instantiate the scaler and fit-transform data
maxabs_scaler = MaxAbsScaler()
scaled_data = maxabs_scaler.fit_transform(data)

print("Scaled Data: ", scaled_data.T)
Scaled Data:  [[-0.2 -0.1  0.   0.   0.   0.1  1. ]]
# Find the maximum absolute value in the data
# np.abs() gets the absolute value of each element first
max_abs_val = np.max(np.abs(data))

# Apply the formula: x / max(|x|)
scaled_data = data / max_abs_val

print("Maximum Absolute Value:", max_abs_val,
      "\nManually Scaled Data (MaxAbsScaler): ", scaled_data.T)
Maximum Absolute Value: 100 
Manually Scaled Data (MaxAbsScaler):  [[-0.2 -0.1  0.   0.   0.   0.1  1. ]]
library(tidymodels)
library(dplyr)

data <- tibble(feature = c(-20, -10, 0, 0, 0, 10, 100))

# For a self-contained example we compute from 'data'.
# In practice: compute max_abs_val on TRAIN split
# and reuse for validation/test
max_abs_val <- max(abs(data$feature))

# Define recipe (scale by max absolute value)
maxabs_recipe <- recipe(~ feature, data = data) %>%
  step_mutate(feature = feature / max_abs_val)

# Prep and bake
prepared_maxabs_recipe <- prep(maxabs_recipe, training = data)
scaled_data <- bake(prepared_maxabs_recipe, new_data = data)

cat("Scaled Data: [", paste(round(scaled_data$feature, 3), collapse = " "), "]")
Scaled Data: [ -0.2 -0.1 0 0 0 0.1 1 ]
  • Deep Learning: Rarely used. Neural nets usually stick with Min-Max (e.g. image pixels → [0,1]) or Standardization (e.g. embeddings, continuous features).
  • Classical ML: Highly relevant when working with sparse features (TF-IDF, user-item matrices). Keeps zeros - crucial for efficiency.

Power Transformation

This is a more advanced, non-linear transformation. Its goal is not just to rescale but to change the very distribution of the variable to make it more Gaussian-like (bell-curve shaped).

Principle: It applies mathematical transformations like Box-Cox (for positive data only) or Yeo-Johnson (for any data) to stabilize variance and minimize skewness.

When to use it: When you want to stabilize variance and reduce skew so that model residuals behave better (common for linear/GLM families). For heavily skewed features (e.g., incomes), power transforms can improve fit and inference.

At the heart of both the Box-Cox and Yeo-Johnson transformations is a parameter called lambda (\(\lambda\)), which controls the “shape” of the power transformation applied to the data. The algorithm’s goal is to automatically find the optimal \(\lambda\) that makes your data’s distribution as close to a Gaussian (bell-curve) shape as possible.

  • Box-Cox is a powerful and well-established method, but has one important limitation: it only works on strictly positive data.
  • Yeo-Johnson is a more modern and flexible version that can handle both positive and negative values, which is why it’s the default in scikit-learn’s PowerTransformer.

The underlying math involves a family of functions that includes logarithmic and power transformations. For a detailed look at the formulas, the Wikipedia article on Power Transforms is an excellent resource.

The key takeaway is that you don’t need to choose \(\lambda\) yourself; the fit() method does all the work of finding the best value for your data.

import numpy as np
from sklearn.preprocessing import PowerTransformer

# Skewed data (like an exponential distribution)
data = np.array([[-2], [-1], [0], [1], [2], [5], [10],[100]])

# method='yeo-johnson' is the default and works with any data
# standardize=True makes the output have a mean of 0 and std of 1
power_scaler = PowerTransformer(standardize=True)
transformed_data = power_scaler.fit_transform(data)

print("Transformed Data:\n", np.round(transformed_data.T, 3))
Transformed Data:
 [[-1.765 -0.882 -0.33  -0.057  0.11   0.41   0.686  1.828]]
Tip

PowerTransformer automatically standardizes the data by default after applying the transformation. This is controlled by the standardize parameter which defaults to True. If you only need de-skewing (not zero-mean/unit-variance), set standardize=False.

library(tidymodels)

data <- tibble(feature = c(-2, -1, 0, 1, 2, 5, 10, 100))

# Yeo-Johnson transformation (works with positive + negative)
power_recipe <- recipe(~ feature, data = data) %>%
  step_YeoJohnson(feature) %>%
  step_normalize(feature)

prepared_power_recipe <- prep(power_recipe, training = data)
transformed_data <- bake(prepared_power_recipe, new_data = data)

cat("Transformed Data:\n[", paste(round(transformed_data$feature, 3), collapse = " "), "]")
Transformed Data:
[ -1.651 -0.825 -0.308 -0.053 0.103 0.383 0.642 1.71 ]

You can see that the R’s recipe for this transformation includes two steps (step_YeoJohnson followed by step_normalize), it is there to ensure that our example produces a standardized result. If you only need de-skewing (not zero-mean/unit-variance) you can eliminate step_normalize step.

If all values were strictly positive, step_BoxCox(feature) could also be used.

Figure 1: On the left, a heavily skewed distribution. On the right, the same data after a power transformation, now resembling a normal distribution.
  • Deep Learning: Almost never used — neural nets don’t need features to be Gaussian; they learn non-linear mappings themselves.
  • Classical ML: Very useful in regression/classical models (linear regression, GLMs) where skewness can harm performance or violate assumptions.

Quantile-Based Transformation

This is another powerful non-linear transformation that can be very effective, especially with outliers.

Principle: It transforms the features to follow a uniform or a normal distribution. It does this based on the quantile (or rank) of each data point. This means that outliers are often squeezed closer to the other data points.

When to use it: When you have outliers and you want to spread out the most frequent values. It can be very effective at transforming a feature’s distribution into something more manageable for the model.

import numpy as np
from sklearn.preprocessing import QuantileTransformer

# Outlier dataset
data = np.array([[10], [20], [30], [40], [500]])

# output_distribution can be 'uniform' (default) or 'normal'
# n_quantiles is how many quantiles to use for the mapping
quantile_scaler = QuantileTransformer(n_quantiles=5, output_distribution='normal')
transformed_data = quantile_scaler.fit_transform(data)

# Values are now mapped to approximately normal quantiles
print("Transformed Data: ", np.round(transformed_data.T, 3))
Transformed Data:  [[-5.199 -0.674  0.     0.674  5.199]]
Caution

QuantileTransformer is rank-based and non-linear; distances and linear relationships are not preserved.

Sparse caution: QuantileTransformer outputs dense arrays, avoid it on high-dimensional sparse data (e.g., BoW/TF-IDF) to prevent memory blowups.

library(tidymodels)
library(bestNormalize)

data <- tibble(feature = c(10, 20, 30, 40, 500))

quantile_recipe <- recipe(~ feature, data = data) %>%
  step_orderNorm(feature)

prepared_quantile_recipe <- prep(quantile_recipe, training = data)
transformed_data <- bake(prepared_quantile_recipe, new_data = data)

cat("Transformed Data: [",
    paste(round(transformed_data$feature, 3), collapse = " "), "]")
Transformed Data: [ -1.282 -0.524 0 0.524 1.282 ]

step_orderNorm always aims for normal distribution (not uniform). It’s the closest R-native equivalent of scikit-learn’s QuantileTransformer.

Figure 2: On the left, data with a dense cluster and extreme outliers. In the middle, a uniform transformation spreads the data evenly. On the right, a normal transformation maps the data to a bell curve, effectively mitigating the outliers.
  • Deep Learning: Rarely used. Quantile-based methods break the relative scale of distances between points, which is not ideal for gradient-based optimizers.
  • Classical ML: Anomaly detection, and any model sensitive to heavy tails/outliers.

Vector Normalization

This one is often confused with the others, but it is fundamentally different.

Key Difference: StandardScaler, MinMaxScaler, etc., all operate column-wise (on each feature independently). Normalizer operates row-wise (on each sample independently).

Principle: It scales each sample (row) so that it has a unit norm (L1 or L2). This is about changing the length of the feature vector for a given sample, not about comparing values across samples.

When to use it: When the direction of the data vector matters more than its magnitude. The most common use case is in text classification or clustering with features like TF-IDF, where you want to know if two documents are about the same topic, regardless of how long the documents are.

import numpy as np
from sklearn.preprocessing import Normalizer

# Each row is a sample. Start with a standard 2D array.
data = np.array([
    [3, 4, 0],  # L2 norm sqrt(3^2+4^2+0^2) = sqrt(25) = 5
    [1, 2, 2],  # L2 norm sqrt(1^2+2^2+2^2) = sqrt(9)  = 3
    [0, 0, 5],  # L2 norm sqrt(0^2+0^2+5^2) = sqrt(25) = 5
    [0, 0, 0]   # L2 norm sqrt(0^2+0^2+0^2) = sqrt(0) = 0
])

# Apply L2 normalization (default)
norm_scaler = Normalizer(norm='l2')
normalized_data = norm_scaler.fit_transform(data)

print("\nNormalized Data:\n", normalized_data)

Normalized Data:
 [[0.6        0.8        0.        ]
 [0.33333333 0.66666667 0.66666667]
 [0.         0.         1.        ]
 [0.         0.         0.        ]]

There’s no direct row-wise Normalizer in recipes. In R, this is usually done manually with base operations, since ML workflows rarely use row normalization outside of text analysis.

library(dplyr)

data <- tibble(
  f1 = c(3, 1, 0, 0),
  f2 = c(4, 2, 0, 0),
  f3 = c(0, 2, 5, 0)
)

# Apply L2 normalization row-wise
normalized_data <- data %>%
  rowwise() %>%
  mutate(norm = sqrt(sum(c_across(everything())^2))) %>%
  mutate(across(everything(), ~ ifelse(norm == 0, 0, .x / norm))) %>%
  select(-norm)

print(normalized_data)
# A tibble: 4 × 3
# Rowwise: 
     f1    f2    f3
  <dbl> <dbl> <dbl>
1 0.6   0.8   0    
2 0.333 0.667 0.667
3 0     0     1    
4 0     0     0

Explanation:
- Each row is scaled independently so its L2 norm = 1.
- Direction of the vector is preserved, but magnitude is ignored.
- This is why it’s useful in text similarity: documents are compared by angle (cosine similarity), not length.

  • Deep Learning: Not a preprocessing step — instead, it’s often used after training on embeddings (e.g. L2-normalizing word embeddings before cosine similarity).
  • Classical ML: Critical in text mining, clustering, search/retrieval — makes cosine similarity comparisons meaningful.

Summary

Scaler/Transformer Principle Works On Primary Use Case Sensitivity to Outliers
MaxAbs Scaling Scales by max absolute value (range depends on sign of data: [-1, 1], [0, 1], or [-1, 0]) Columns Sparse data (preserves zero entries). High
Power Transformation Non-linear transform to make data more Gaussian-like Columns Stabilizing variance and handling skewed data. Varies
Quantile-Based Transformation Maps data distribution to a uniform or normal one Columns Spreading out dense clusters; handling outliers. Low
Vector Normalization Scales each sample to unit norm (length=1) Rows When vector direction matters more than magnitude (e.g., text analysis). N/A (row-based)

All the scalers so far are built-in. But what if you need something custom? That’s where FunctionTransformer (Python) or step_mutate (R) comes in. For example two very practical custom transforms are:

  • Log-scaling (e.g., log1p) for strictly positive, heavily skewed features (counts, income).
  • Clipping / Winsorization to cap extreme values before scaling.
import numpy as np
import pandas as pd
from sklearn.preprocessing import FunctionTransformer
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LinearRegression

# Toy data: skewed "count" feature + a continuous feature
df = pd.DataFrame({
    "count": [0, 1, 2, 3, 10, 50, 200, 1000],  # heavily skewed, many small, a few huge
    "feat":  [2.1, 2.0, 2.3, 2.2, 2.5, 3.0, 3.2, 3.5],
    "y":     [1.2, 1.1, 1.3, 1.2, 1.6, 2.2, 2.6, 3.1]
})

# Example 1: log1p on the skewed "count" feature
log1p_transformer = FunctionTransformer(np.log1p, feature_names_out="one-to-one")

# Example 2: clipping (winsorization) on "feat"
def clip_0_99(X):
    # clip column-wise to [1st, 99th] percentiles
    lo = np.percentile(X, 1, axis=0, keepdims=True)
    hi = np.percentile(X, 99, axis=0, keepdims=True)
    return np.clip(X, lo, hi)

clip_transformer = FunctionTransformer(clip_0_99, feature_names_out="one-to-one")

preprocess = ColumnTransformer(
    transformers=[
        ("log_count", log1p_transformer, ["count"]),
        ("clip_feat", clip_transformer,   ["feat"]),
    ],
    remainder="drop"
)

pipe = Pipeline([
    ("prep", preprocess),
    ("scale", StandardScaler()),
    ("model", LinearRegression())
])

pipe.fit(df[["count","feat"]], df["y"])
Pipeline(steps=[('prep',
                 ColumnTransformer(transformers=[('log_count',
                                                  FunctionTransformer(feature_names_out='one-to-one',
                                                                      func=<ufunc 'log1p'>),
                                                  ['count']),
                                                 ('clip_feat',
                                                  FunctionTransformer(feature_names_out='one-to-one',
                                                                      func=<function clip_0_99 at 0x7f2b1f7ac360>),
                                                  ['feat'])])),
                ('scale', StandardScaler()), ('model', LinearRegression())])
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
print("Transformed sample:\n", pipe.named_steps["prep"].transform(df[["count","feat"]])[:5])
Transformed sample:
 [[0.         2.1       ]
 [0.69314718 2.007     ]
 [1.09861229 2.3       ]
 [1.38629436 2.2       ]
 [2.39789527 2.5       ]]

Sparse note: If you pass a sparse matrix, ensure your function can handle it (e.g., use scipy.sparse - aware ops). FunctionTransformer will pass the matrix through, your function must not densify it unintentionally.

library(tidymodels)
library(dplyr)

df <- tibble(
  count = c(0, 1, 2, 3, 10, 50, 200, 1000),
  feat  = c(2.1, 2.0, 2.3, 2.2, 2.5, 3.0, 3.2, 3.5),
  y     = c(1.2, 1.1, 1.3, 1.2, 1.6, 2.2, 2.6, 3.1)
)

# Helper for winsorization (column-wise clip)
winsor <- function(x, p = c(0.01, 0.99)) {
  qs <- quantile(x, probs = p, na.rm = TRUE, names = FALSE)
  pmin(pmax(x, qs[1]), qs[2])
}

rec <- recipe(y ~ count + feat, data = df) %>%
  # Example 1: log1p for skewed counts
  step_mutate(count = log1p(count)) %>%
  # Example 2: winsorize feat at [1%, 99%]
  step_mutate(feat  = winsor(feat)) %>%
  # Optionally follow with standardization
  step_normalize(all_numeric_predictors())

prep_rec <- prep(rec)
baked <- bake(prep_rec, new_data = NULL)
print(baked)
# A tibble: 8 × 3
   count   feat     y
   <dbl>  <dbl> <dbl>
1 -1.11  -0.898   1.2
2 -0.828 -1.07    1.1
3 -0.662 -0.538   1.3
4 -0.544 -0.718   1.2
5 -0.130 -0.177   1.6
6  0.499  0.724   2.2
7  1.06   1.08    2.6
8  1.72   1.59    3.1

When to reach for custom transforms?

  • You’ve profiled a feature and know the pathology (e.g., extreme skew or a few absurd outliers).
  • Built-ins don’t match your exact need (e.g., clip first, then scale).
  • You want the step to live inside your pipeline/recipe so it’s reproducible and applied consistently to train/validation/test.

Rule of Thumb: Apply log1p for positive, heavy-tailed counts; clip/winsorize before scaling when a handful of wild points distort ranges or standard deviations.

What is Sparse Data?

In the simplest terms, sparse data is data that is mostly made up of zeros.

Its opposite is dense data, where most of the values are non-zero.

There isn’t a strict mathematical rule, but if a very high percentage (e.g., >90%, >99%) of your data’s entries are zero, you are dealing with a sparse dataset.

An Intuitive Analogy: The Movie Rental Store

Imagine a huge movie rental store with 50,000 unique movies. This represents features.

Now, imagine we track the rental history of one customer, “Alice.” Alice has only ever rented 15 different movies. This is the sample.

If we were to represent Alice’s rental history as a row of data with 50,000 columns (one for each movie), it would look something like this:

[0, 0, 2, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, …, 5, 0, 0, …]

This row has 50,000 values, but only 15 of them are non-zero. The other 49,985 values are zero. This is a highly sparse vector.

Why Sparsity is Special and Important

Sparse data is handled differently for two main reasons:

  1. Storage and Memory Efficiency: It would be incredibly wasteful to store all those zeros. Instead of a standard NumPy array, sparse data is stored in special structures (like a scipy.sparse.csr_matrix) that only keep track of the non-zero elements and their locations (row and column index).
  • Dense Storage: [0, 0, 7, 0, 4] (stores all 5 elements)
  • Sparse Storage: (row:0, col:2, value:7), (row:0, col:4, value:4) (only stores 2 items)

This can reduce memory usage from gigabytes down to megabytes.

  1. The Problem with Standard Scalers:

This is why we need special scalers for sparse data. Let’s see what happens if you incorrectly use StandardScaler on sparse data.

  • The formula is \(z = \frac{x - \mu}{\sigma}\).
  • In a sparse dataset, the vast majority of values are 0, so the mean (\(\mu\)) will be a very small number close to zero (but not exactly zero). Let’s say \(\mu = 0.01\).
  • What happens when you scale a value that is 0? \(z = \frac{0 - 0.01}{\sigma}\)
  • The result is a non-zero number!

You have just taken a matrix that was 99% zeros and was stored efficiently, and you’ve turned it into a 100% dense matrix. The memory usage will explode.

How to Handle Sparse Data Correctly

This is where MaxAbsScaler and Normalizer come in.

  • MaxAbsScaler: The formula is \(x_{scaled} = \frac{x_i}{max(|x|)}\).
    • If a value \(x\) is \(0\), the result is \(\frac{0}{max(|x|)} = 0\).
    • It keeps the zeros as zeros! It preserves the sparsity of your data while still scaling the non-zero values.
  • Normalizer: This scaler works row-by-row, so it also inherently handles sparsity correctly by only considering the non-zero elements in its calculations for that row.

Summary: Sparse vs. Dense

Characteristic Dense Data Sparse Data
Value Composition Mostly non-zero values. Vast majority of values are zero.
Common Examples Sensor readings, age, income, physical measurements. Text data (Bag-of-Words), user-item interactions (e.g., who bought what).
Storage Standard NumPy arrays. Specialized sparse matrix formats (scipy.sparse).
Appropriate Scalers StandardScaler, MinMaxScaler, RobustScaler. MaxAbsScaler, Normalizer.

Example 1: Text Data with Bag-of-Words

This is the most common source of sparse data in machine learning. The “Bag-of-Words” (BoW) model represents text by counting the occurrence of each known word, disregarding grammar and word order.

Scenario: We have a small corpus of three simple documents (sentences), and we want to convert them into numerical vectors.

import pandas as pd
from sklearn.feature_extraction.text import CountVectorizer

# Collection of text documents
corpus = [
    'The Ring must go to Mordor',
    'Frodo takes the Ring to Mordor',
    'Sauron rules in Mordor'
]

# Initialize the vectorizer
# This object will learn the vocabulary and transform the text
vectorizer = CountVectorizer()

# Fit the vectorizer to the corpus and transform the data
# .fit_transform() learns the vocabulary
# and converts the corpus into a word-count matrix.
X = vectorizer.fit_transform(corpus)

# Display the results in a human-readable format
# Get the learned vocabulary (features)
feature_names = vectorizer.get_feature_names_out()

# Create a pandas DataFrame to visualize the sparse matrix
df = pd.DataFrame(X.toarray(), columns=feature_names, index=['Doc 1', 'Doc 2', 'Doc 3'])

print("Learned Vocabulary (Features):", feature_names,
      "\nBag-of-Words Matrix (Dense Representation):")
Learned Vocabulary (Features): ['frodo' 'go' 'in' 'mordor' 'must' 'ring' 'rules' 'sauron' 'takes' 'the'
 'to'] 
Bag-of-Words Matrix (Dense Representation):
print(df)
       frodo  go  in  mordor  must  ring  rules  sauron  takes  the  to
Doc 1      0   1   0       1     1     1      0       0      0    1   1
Doc 2      1   0   0       1     0     1      0       0      1    1   1
Doc 3      0   0   1       1     0     0      1       1      0    0   0
print(f"\nShape of the matrix: {X.shape}",
      f"Number of non-zero elements: {X.nnz}")

Shape of the matrix: (3, 11) Number of non-zero elements: 16

recipes does not yet support sparse matrices. In R, sparse workflows typically use the Matrix package, which represents large sparse datasets efficiently (e.g., dgCMatrix).

library(text2vec)
library(Matrix)

corpus <- c("The Ring must go to Mordor",
            "Frodo takes the Ring to Mordor",
            "Sauron rules in Mordor")

# Create an iterator over tokens
tokens <- space_tokenizer(corpus)

# Create a vocabulary
it <- itoken(tokens)
vocab <- create_vocabulary(it)

# Vectorizer
vectorizer <- vocab_vectorizer(vocab)

# Create document-term matrix (sparse)
dtm <- create_dtm(it, vectorizer)

print(dtm)
3 x 12 sparse Matrix of class "dgCMatrix"
                         
1 . . 1 1 . 1 . . . 1 1 1
2 1 . . . . . . 1 1 1 1 1
3 . 1 . . 1 . 1 . . . . 1
cat("Matrix dimensions:", dim(dtm), "\nNon-zero entries:", length(dtm@x))
Matrix dimensions: 3 12 
Non-zero entries: 16

As you can see from the output DataFrame, the result is a matrix where: * Each row is a document. * Each column is a unique word from the vocabulary. * The value in each cell is the count of how many times that word appeared in that document.

Notice the high number of zeros. Even in this tiny example, most cells are zero. Now, imagine a real-world vocabulary of 50,000 words. A single sentence might only use 10-20 of those words, meaning its vector representation would have over 49,980 zero entries. This is a sparse matrix. The scikit-learn CountVectorizer intelligently stores X in a scipy.sparse format to save memory. We call .toarray() only for the purpose of visualization.

Example 2: User-Item Interactions

This is the bedrock of recommendation systems (like those used by Netflix or Amazon). The goal is to represent which users have interacted with (e.g., bought, viewed, or rated) which items.

Scenario: We have a small e-commerce site. We want to create a matrix representing which of our 4 users have bought which of our 6 products.

import pandas as pd

# Raw data of user-item interactions (e.g., from a database)
# A '1' indicates a purchase.
data = {
    'user':  ['Alice', 'Bob', 'Alice', 'Charlie', 'Alice', 'David', 'Bob'],
    'item':  ['Laptop', 'Mouse', 'Keyboard', 'Mouse', 'Monitor', 'Keyboard', 'Webcam']
}
df_interactions = pd.DataFrame(data)
df_interactions['purchased'] = 1

print("Original Interaction Data:")
Original Interaction Data:
print(df_interactions)
      user      item  purchased
0    Alice    Laptop          1
1      Bob     Mouse          1
2    Alice  Keyboard          1
3  Charlie     Mouse          1
4    Alice   Monitor          1
5    David  Keyboard          1
6      Bob    Webcam          1
# Create the user-item interaction matrix using pivot_table
# This transforms the "long" data format into a "wide" matrix format.
# We fill non-interactions with 0.
user_item_matrix = df_interactions.pivot_table(
    index='user',
    columns='item',
    values='purchased',
    fill_value=0
)

print("\nUser-Item Sparse Matrix:")

User-Item Sparse Matrix:
print(user_item_matrix)
item     Keyboard  Laptop  Monitor  Mouse  Webcam
user                                             
Alice         1.0     1.0      1.0    0.0     0.0
Bob           0.0     0.0      0.0    1.0     1.0
Charlie       0.0     0.0      0.0    1.0     0.0
David         1.0     0.0      0.0    0.0     0.0
Note

Sparse-friendly scalers are rare in R; most workflows here rely on packages like Matrix, text2vec, or specialized recommender frameworks rather than recipes.

library(tidyr)
library(dplyr)
library(Matrix)

data <- tibble(
  user = c('Alice', 'Bob', 'Alice', 'Charlie', 'Alice', 'David', 'Bob'),
  item = c('Laptop', 'Mouse', 'Keyboard', 'Mouse', 'Monitor', 'Keyboard', 'Webcam'),
  purchased = 1
)

cat("Original Interaction Data:\n")
Original Interaction Data:
print(data)
# A tibble: 7 × 3
  user    item     purchased
  <chr>   <chr>        <dbl>
1 Alice   Laptop           1
2 Bob     Mouse            1
3 Alice   Keyboard         1
4 Charlie Mouse            1
5 Alice   Monitor          1
6 David   Keyboard         1
7 Bob     Webcam           1
# Dense user-item matrix
user_item_matrix <- data %>%
  pivot_wider(names_from = item, values_from = purchased, values_fill = 0)

cat("\nDense User-Item Matrix:\n")

Dense User-Item Matrix:
print(user_item_matrix)
# A tibble: 4 × 6
  user    Laptop Mouse Keyboard Monitor Webcam
  <chr>    <dbl> <dbl>    <dbl>   <dbl>  <dbl>
1 Alice        1     0        1       1      0
2 Bob          0     1        0       0      1
3 Charlie      0     1        0       0      0
4 David        0     0        1       0      0
# Sparse representation
sparse_matrix <- sparseMatrix(
  i = as.numeric(factor(data$user)),
  j = as.numeric(factor(data$item)),
  x = data$purchased,
  dimnames = list(levels(factor(data$user)), 
                  levels(factor(data$item)))
)

cat("\nUser-Item Sparse Matrix:\n")

User-Item Sparse Matrix:
print(sparse_matrix)
4 x 5 sparse Matrix of class "dgCMatrix"
        Keyboard Laptop Monitor Mouse Webcam
Alice          1      1       1     .      .
Bob            .      .       .     1      1
Charlie        .      .       .     1      .
David          1      .       .     .      .

Look at the resulting “User-Item Sparse Matrix”.

  • Each row is a user.
  • Each column is an item available for sale.
  • A 1 means the user has bought that item. A 0 means they have not.

Again, the matrix is mostly zeros. Alice has bought 3 out of the 6 available items, so her row is 50% sparse. In a real system like Amazon with millions of users and millions of products, any given user will have purchased an infinitesimally small fraction of the total items available. Their corresponding row in the user-item matrix would be incredibly sparse.

This sparsity is fundamental to how collaborative filtering algorithms work. They use this matrix to find users with similar patterns of non-zero entries to make recommendations.

MaxAbs Scaling vs Standardization on sparse data

This first example is dense on purpose to show the numeric effect; the next block shows true CSR sparsity.

import numpy as np
from sklearn.preprocessing import MaxAbsScaler, StandardScaler

# A DENSE array used to illustrate scaling
# Each row is a document, each column is a word count
data = np.array([
    [5, 0, 0, 1, 0],
    [0, 1, 0, 0, 0],
    [0, 0, 0, 0, 3],
    [1, 0, 2, 0, 0]
])

# --- CORRECT SCALING with MaxAbsScaler ---
maxabs_scaler = MaxAbsScaler()
maxabs_scaled = maxabs_scaler.fit_transform(data)
print("Scaled with MaxAbsScaler (zeros are preserved):\n", maxabs_scaled)
Scaled with MaxAbsScaler (zeros are preserved):
 [[1.  0.  0.  1.  0. ]
 [0.  1.  0.  0.  0. ]
 [0.  0.  0.  0.  1. ]
 [0.2 0.  1.  0.  0. ]]
# --- INCORRECT SCALING with StandardScaler ---
std_scaler = StandardScaler()
std_scaled = std_scaler.fit_transform(data)
print("Scaled with StandardScaler (sparsity is destroyed):\n", np.round(std_scaled, 2))
Scaled with StandardScaler (sparsity is destroyed):
 [[ 1.7  -0.58 -0.58  1.73 -0.58]
 [-0.73  1.73 -0.58 -0.58 -0.58]
 [-0.73 -0.58 -0.58 -0.58  1.73]
 [-0.24 -0.58  1.73 -0.58 -0.58]]
import numpy as np
from scipy.sparse import csr_matrix
from sklearn.preprocessing import MaxAbsScaler, StandardScaler

# True sparse matrix (CSR)
X = csr_matrix(np.array([
    [5, 0, 0, 1, 0],
    [0, 1, 0, 0, 0],
    [0, 0, 0, 0, 3],
    [1, 0, 2, 0, 0],
]))

print("number of non-zeros before:", X.nnz)
number of non-zeros before: 6
# --- CORRECT: preserves sparsity
X_maxabs = MaxAbsScaler().fit_transform(X)
print("number of non-zeros after MaxAbs:", X_maxabs.nnz)
number of non-zeros after MaxAbs: 6
# --- CORRECT for sparse: no centering
X_std = StandardScaler(with_mean=False).fit_transform(X)
print("number of non-zeros after StandardScaler(with_mean=False):", X_std.nnz)
number of non-zeros after StandardScaler(with_mean=False): 6
# --- INCORRECT: centering on sparse is not supported
try:
    _ = StandardScaler(with_mean=True).fit_transform(X)
except ValueError as e:
    print("Centering sparse matrices is not supported:", str(e)[:62], "…")
Centering sparse matrices is not supported: Cannot center sparse matrices: pass `with_mean=False` instead. …
library(Matrix)

# A sparse matrix
data <- Matrix(
  c(5, 0, 0, 1, 0,
    0, 1, 0, 0, 0,
    0, 0, 0, 0, 3,
    1, 0, 2, 0, 0),
  nrow = 4, byrow = TRUE, sparse = TRUE
)

print("Original Sparse Matrix:")
[1] "Original Sparse Matrix:"
print(data)
4 x 5 sparse Matrix of class "dgCMatrix"
              
[1,] 5 . . 1 .
[2,] . 1 . . .
[3,] . . . . 3
[4,] 1 . 2 . .
# MaxAbs scaling manually
max_abs_vals <- apply(abs(data), 2, max)
maxabs_scaled <- data
for (j in seq_along(max_abs_vals)) {
  if (max_abs_vals[j] != 0) {
    maxabs_scaled[, j] <- data[, j] / max_abs_vals[j]
  }
}

print("Scaled with MaxAbs (zeros preserved):")
[1] "Scaled with MaxAbs (zeros preserved):"
print(maxabs_scaled)
4 x 5 sparse Matrix of class "dgCMatrix"
                
[1,] 1.0 . . 1 .
[2,] .   1 . . .
[3,] .   . . . 1
[4,] 0.2 . 1 . .
# Standardization manually (destroys sparsity)
col_means <- colMeans(data)
col_sds   <- apply(data, 2, sd)

std_scaled <- as.matrix(scale(data, center = TRUE, scale = TRUE))

print("Scaled with Standardization (dense):")
[1] "Scaled with Standardization (dense):"
print(round(std_scaled, 2))
      [,1] [,2] [,3] [,4] [,5]
[1,]  1.47 -0.5 -0.5  1.5 -0.5
[2,] -0.63  1.5 -0.5 -0.5 -0.5
[3,] -0.63 -0.5 -0.5 -0.5  1.5
[4,] -0.21 -0.5  1.5 -0.5 -0.5
attr(,"scaled:center")
[1] 1.50 0.25 0.50 0.25 0.75
attr(,"scaled:scale")
[1] 2.380476 0.500000 1.000000 0.500000 1.500000

What’s Next: Bringing It All Together

Across this three-part journey, we’ve explored the entire landscape of feature scaling—from the foundational “why” in Part 1, to the everyday workhorses in Part 2, and now to the powerful specialists in Part 3.

But theory is one thing, and seeing the impact is another.

In the final, concluding post of this series, we will bring everything together. We will build a complete summary table of all the scalers we’ve discussed, provide downloadable Python and R notebooks with all the code in one place, and, most importantly, we will walk through a practical example. We’ll train a simple machine learning model on unscaled data, see how it performs, and then apply the right scaling technique to see the tangible improvement. This will be the practical demonstration of why scaling is a crucial preprocessing step when the model demands it.