Pandas factorize() Function

import pandas as pd 

s = pd.Series(['apple', 'banana', 'apple'])
labels, uniques = pd.factorize(s)
print(labels)

Expected Output:

[0 1 0]

1. Comprehensive Overview and Function Signature

The pd.factorize() function encodes 1D categorical structures, text Series, or object arrays into discrete integer codes while simultaneously returning an index of unique categories. In machine learning feature engineering, statistical modeling, and data compression, algorithms require numerical inputs rather than raw string strings. Unlike get_dummies(), which expands categorical features into high-dimensional binary sparse matrices, factorize() preserves 1D array shapes by mapping distinct category strings to sequential integers (e.g., 0, 1, 2...).

2. Practical Usage Scenarios: Where & Why to Use `factorize()`

• Machine Learning Label Encoding: Scikit-Learn models (like XGBoost, Random Forests, or Decision Trees) require target labels (e.g., "Cat", "Dog", "Bird") to be encoded as integers (0, 1, 2).
• Memory-Efficient Indexing: Replacing long repetitive text identifiers across millions of records with compact int8 or int16 codes significantly reduces overall RAM consumption.
• Reconstructing Original Categories: Because factorize() returns both integer codes and the unique category index, decoding integer predictions back to original human-readable strings is achieved effortlessly via uniques[codes].

3. Deep Dive Examples & Parameter Manipulations

Example 3.1: Handling Missing Values (NaN Sentinel Handling)

By default, factorize() assigns -1 to missing values, preventing NaN entries from creating spurious category classes.

import pandas as pd 
import numpy as np

s_nan = pd.Series(['High', np.nan, 'Low', 'High'])
codes, uniques = pd.factorize(s_nan)

print("Codes:", codes)
print("Uniques:", uniques)

Expected Output:

Codes: [ 0 -1  1  0]
Uniques: Index(['High', 'Low'], dtype='object')

Example 3.2: Alphabetical Sorting of Unique Categories

Setting sort=True guarantees consistent integer code assignment across different datasets by sorting categories alphabetically prior to encoding.

import pandas as pd 

s_dept = pd.Series(['Sales', 'Admin', 'Sales', 'IT'])
codes_sorted, uniques_sorted = pd.factorize(s_dept, sort=True)

print("Sorted Codes:", codes_sorted)
print("Sorted Uniques:", uniques_sorted)

Expected Output:

Sorted Codes: [2 0 2 1]
Sorted Uniques: Index(['Admin', 'IT', 'Sales'], dtype='object')

4. Common Errors & Troubleshooting

5. Frequently Asked Questions (FAQs)

Q: How does factorize() differ from get_dummies()?
factorize() returns a single 1D array of integer codes (0, 1, 2...). get_dummies() generates a wide binary matrix of separate 0/1 indicator columns for each category.

Q: Can I decode integer codes back to original string values?
Yes, simply pass the integer code array as an index selector into the returned unique categories: uniques[codes].

Interactive Companion Jupyter Notebook

Explore interactive code cells, test missing value sentinel codes, run category sorting, and execute self-validating assertions.

View on GitHub

6. Related Tutorials

• Pandas get_dummies() Method - One-hot encode categorical variables into indicator matrices.
• Pandas astype() Method - Convert Series directly to Categorical dtypes.
• Pandas DataFrame Methods Directory - Complete searchable index of all DataFrame operations.

Subhendu Mohapatra

Author & Instructor at plus2net

I write and maintain practical tutorials on Python, PHP, SQL, JavaScript, HTML, jQuery, and web development at plus2net. The tutorials focus on clear explanations, working examples, and code that readers can test and adapt while learning.

About the author   Watch on YouTube

Connect: