Python Regular Expressions: Pattern Matching & re Module

Python's standard re module provides powerful regular expression engines for advanced text pattern matching, data extraction, validation, and substitutions that exceed the limits of fixed string methods.

The Raw String Invariant: Regular expression patterns should always be written using raw string literals (r"..."). This prevents Python's standard string parser from consuming backslashes (such as \b for word boundary vs. ASCII backspace 0x08, or \d for digit vs. literal 'd').

Show Table of Contents ↓

1. Core Functions in the re Module 🔝

Python provides specialized functions across the re module tailored to distinct operational needs:

Function	Primary Role	Return Value
`re.search(pattern, s)`	First match anywhere in string	`re.Match` object or `None`
`re.match(pattern, s)`	Matches string start only	`re.Match` object or `None`
`re.fullmatch(pattern, s)`	Exact full string match	`re.Match` object or `None`
`re.findall(pattern, s)`	All non-overlapping matches	`list[str]` or `list[tuple]`
`re.finditer(pattern, s)`	Iterator of match objects	Iterator of `re.Match` objects
`re.sub(pattern, repl, s)`	Replaces pattern occurrences	New string `str`
`re.split(pattern, s)`	Splits by regex pattern	`list[str]`

2. Critical Distinctions: search() vs. match() vs. fullmatch() 🔝

One of the most frequent errors in Python scripting is using re.match() expecting it to search anywhere in the string:

Comparing Scan Anchors

import re

text = "Error code: 404 in server logs"

print("re.match():    ", re.match(r"\d+", text))

match_obj = re.search(r"\d+", text)
print("re.search():   ", match_obj.group() if match_obj else None)

print("re.fullmatch():", re.fullmatch(r"\d+", text))
print("Numeric check: ", bool(re.fullmatch(r"\d+", "404")))

Expected Output:

re.match():     None
re.search():    404
re.fullmatch(): None
Numeric check:  True

3. Capturing Groups & Match Objects 🔝

Parentheses define capturing groups for extracting structured data:

Example: Positional & Named Capturing Groups

import re

log_line = "2026-10-14 [ERROR] Database timeout on host db.local:5432"

pattern = r"(?P<date>\d{4}-\d{2}-\d{2})\s+\[(?P<level>[A-Z]+)\]\s+(?P<msg>.+)"
match = re.search(pattern, log_line)

if match:
    data = match.groupdict()
    print("Date:   ", data["date"])
    print("Level:  ", data["level"])
    print("Message:", data["msg"])

Expected Output:

Date:    2026-10-14
Level:   ERROR
Message: Database timeout on host db.local:5432

4. Mass Extraction & Substitution: findall() & sub() 🔝

For batch extraction and substitution, findall() and sub() provide fast pipelines:

Example: Extracting Dates and Masking Sensitive Data

import re

corpus = "Birthdays: Alice (24-12-2002), Bob (05-09-1998), Charlie (12-04-2005)"

dates = re.findall(r"\b\d{2}-\d{2}-\d{4}\b", corpus)
print("Extracted dates:", dates)

masked = re.sub(r"(\d{2}-\d{2}-)\d{4}", r"\g<1>XXXX", corpus)
print("Masked records: ", masked)

Expected Output:

Extracted dates: ['24-12-2002', '05-09-1998', '12-04-2005']
Masked records:  Birthdays: Alice (24-12-XXXX), Bob (05-09-XXXX), Charlie (12-04-XXXX)

5. Precompilation & Regex Flags 🔝

Precompile patterns with re.compile() when processing loops:

Using re.compile() with Flags

import re

EMAIL_PATTERN = re.compile(r"^[a-zA-Z0-9_.+-]+@[a-zA-Z0-9-]+\.[a-zA-Z0-9-.]+$", re.IGNORECASE)

print("Valid email:  ", bool(EMAIL_PATTERN.match("admin@plus2net.com")))
print("Invalid email:", bool(EMAIL_PATTERN.match("admin@plus2net")))

Expected Output:

Valid email:   True
Invalid email: False

6. Real-World Engineering Recipes 🔝

Recipe 1: Date Format Converter (DD-MM-YYYY to ISO YYYY-MM-DD)

import re

def convert_to_iso_date(text: str) -> str:
        return re.sub(r"\b(\d{2})-(\d{2})-(\d{4})\b", r"\3-\2-\1", text)

raw_doc = "Report generated on 24-12-2026 for fiscal quarter ending 31-03-2027."
print("ISO Converted:\n", convert_to_iso_date(raw_doc))

Expected Output:

ISO Converted:
 Report generated on 2026-12-24 for fiscal quarter ending 2027-03-31.

Recipe 2: HTML Hyperlink Target Extractor

import re

def extract_hyperlinks(html_content: str) -> list[str]:
    pattern = r'''href=['"]?([^'" >]+)'''
    return re.findall(pattern, html_content, re.IGNORECASE)

html_sample = '''
<a href="https://plus2net.com/python">Python Tutorials</a>
<a href='/sql/database.php'>SQL Guide</a>
<a class="btn" href=site_map.php>Sitemap</a>
'''
print("Extracted URLs:", extract_hyperlinks(html_sample))

Expected Output:

Extracted URLs: ['https://plus2net.com/python', '/sql/database.php', 'site_map.php']

7. Common Pitfalls & Edge Cases 🔝

Pitfall 1: Forgetting Raw String Notation r"..."

In standard strings, "\b" is backspace (0x08). In regex, \b is a word boundary. Always prefix regexes with r.

Pitfall 2: Greedy vs. Non-Greedy Quantifiers

By default, quantifiers like .* and .+ are greedy and consume as much text as possible. In one and two, the pattern .* matches everything from the first  to the last . Use non-greedy .*? to stop at the first closing tag.

8. Production Implementation & Self-Checking Assertions 🔝

Below is a production-grade utility class, RegexParser, demonstrating date extraction, URL parsing, sanitization, and automated unit assertions:

Production RegexParser Class with Unit Assertions

import re

class RegexParser:
    DATE_PATTERN = re.compile(r"\b(\d{2})-(\d{2})-(\d{4})\b")
    URL_PATTERN = re.compile(r'''href=['"]?([^'" >]+)''', re.IGNORECASE)

    @classmethod
    def extract_dates(cls, text: str) -> list[str]:
        """Extracts date strings in DD-MM-YYYY format."""
        return [f"{d}-{m}-{y}" for d, m, y in cls.DATE_PATTERN.findall(text)]

    @classmethod
    def to_iso_dates(cls, text: str) -> str:
        """Converts DD-MM-YYYY dates to ISO YYYY-MM-DD format."""
        return cls.DATE_PATTERN.sub(r"\3-\2-\1", text)

    @classmethod
    def extract_hrefs(cls, html_str: str) -> list[str]:
        """Finds all URL targets in HTML markup."""
        return cls.URL_PATTERN.findall(html_str)


# ==========================================
# SELF-CHECKING UNIT ASSERTIONS
# ==========================================
parser = RegexParser()

sample_text = "Event on 24-12-2002 and followup on 15-01-2003"
dates = parser.extract_dates(sample_text)
assert dates == ["24-12-2002", "15-01-2003"]

iso_text = parser.to_iso_dates(sample_text)
assert iso_text == "Event on 2002-12-24 and followup on 2003-01-15"

html_doc = '<a href="https://plus2net.com">Home</a><a href=\'/python/string.php\'>Strings</a>'
urls = parser.extract_hrefs(html_doc)
assert urls == ["https://plus2net.com", "/python/string.php"]

assert re.match(r"\d+", "abc 123") is None
assert re.search(r"\d+", "abc 123").group() == "123"

html_snippet = "<b>first</b> and <b>second</b>"
assert re.findall(r"<b>(.*?)</b>", html_snippet) == ["first", "second"]

print("Extracted dates: ", dates)
print("ISO reformatted: ", iso_text)
print("Extracted links: ", urls)
print("All regular expression unit assertions passed successfully!")

Practice in Google Colab & GitHub 🔝

Open in Google Colab View on GitHub

Run all regex pattern matching examples interactively, contrast search() vs match(), extract dates and URLs, and test non-greedy quantifiers in Google Colab.

str.replace() Function — Exact substring replacement with count limits.
str.split() Function — Delimiter-based splitting into lists.
str.find() Function — Substring position searching returning -1 on failure.
str.startswith() Function — Prefix matching with tuple support.
str.endswith() Function — Suffix matching with tuple support.
Python String Methods Overview — Complete reference of built-in string methods.

String Methods Palindrome Check Practice Questions Python Tutorials

Subhendu Mohapatra

Author & Instructor at plus2net

I write and maintain practical tutorials on Python, PHP, SQL, JavaScript, HTML, jQuery, and web development at plus2net. The tutorials focus on clear explanations, working examples, and code that readers can test and adapt while learning.

About the author Watch on YouTube

Connect: