Pandas read_xml(): XPath, Namespaces and Validated XML Imports
Use read_xml() to read repeating XML records into a DataFrame. Select the correct nodes, preserve identifiers and validate fields before analysis.
Create and read the student XML 🔝
read_xml() returns one DataFrame. Choose repeating record elements with xpath. The default ./* selects direct children of the root. This practice generates XML from the student sample, so no separate student.xml download is needed.
import pandas as pd
import tempfile
import xml.etree.ElementTree as ET
from pathlib import Path
from io import StringIO
student_csv = 'id,name,class,mark,gender\n1,John Deo,Four,75,female\n2,Max Ruin,Three,85,male\n3,Arnold,Three,55,male\n4,Krish Star,Four,60,female\n5,John Mike,Four,60,female\n6,Alex John,Four,55,male\n7,My John Rob,Fifth,78,male\n8,Asruid,Five,85,male\n9,Tes Qry,Six,78,male\n10,Big John,Four,55,female\n11,Ronald,Six,89,female\n12,Recky,Six,94,female\n13,Kty,Seven,88,female\n14,Bigy,Seven,88,female\n15,Tade Row,Four,88,male\n16,Gimmy,Four,88,male\n17,Tumyu,Six,54,male\n18,Honny,Five,75,male\n19,Tinny,Nine,18,male\n20,Jackly,Nine,65,female\n21,Babby John,Four,69,female\n22,Reggid,Seven,55,female\n23,Herod,Eight,79,male\n24,Tiddy Now,Seven,78,male\n25,Giff Tow,Seven,88,male\n26,Crelea,Seven,79,male\n27,Big Nose,Three,81,female\n28,Rojj Base,Seven,86,female\n29,Tess Played,Seven,55,male\n30,Reppy Red,Six,79,female\n31,Marry Toeey,Four,88,male\n32,Binn Rott,Seven,90,female\n33,Kenn Rein,Six,96,female\n34,Gain Toe,Seven,69,male\n35,Rows Noump,Six,88,female\n'
students = pd.read_csv(StringIO(student_csv))
student_xml = students.to_xml(index=False, root_name="Students", row_name="Row", parser="etree")
df = pd.read_xml(StringIO(student_xml), xpath=".//Row", parser="etree")
print(df.head())
print("Rows:", len(df))
assert df.equals(students)Expected output
id name class mark gender
0 1 John Deo Four 75 female
1 2 Max Ruin Three 85 male
2 3 Arnold Three 55 male
3 4 Krish Star Four 60 female
4 5 John Mike Four 60 female
Rows: 35Read an XML file or text buffer 🔝
Pass an actual path for a file, or StringIO for literal XML markup. Relative paths use the current working directory; a Windows raw path avoids escape sequences. A supported URL can also be used when it returns accessible XML.
with tempfile.TemporaryDirectory(dir=Path.cwd()) as folder:
target = Path(folder) / "student.xml"
target.write_text(student_xml, encoding="utf-8")
loaded = pd.read_xml(target, xpath=".//Row", parser="etree")
print("File rows:", len(loaded))
assert loaded.equals(df)
# Windows example:
# pd.read_xml(r"E:\testing\data\student.xml", xpath=".//Row", parser="etree")
# Optional Colab file export:
# Path("student.xml").write_text(student_xml, encoding="utf-8")Expected output
File rows: 35Choose the repeating record level 🔝
xpath selects record elements, rather than a single scalar value. Explicit selection helps when records are deeper than the direct children. Check both field names and row count.
wrapped = "<Report><Data>" + student_xml.split("<Students>", 1)[1].split("</Students>", 1)[0] + "</Data></Report>"
selected = pd.read_xml(StringIO(wrapped), xpath=".//Row", parser="etree")
print(selected.head(2))
print("Columns:", selected.columns.tolist())
assert len(selected) == len(students)Expected output
id name class mark gender
0 1 John Deo Four 75 female
1 2 Max Ruin Three 85 male
Columns: ['id', 'name', 'class', 'mark', 'gender']Choose etree or lxml 🔝
etree uses Python’s standard ElementTree parser and supports simpler XPath expressions. lxml is the default Pandas parser and supports more capable XPath and XSLT 1.0 stylesheets. It requires the lxml package; install it in your environment if needed.
basic = pd.read_xml(StringIO(student_xml), xpath=".//Row", parser="etree")
try:
import lxml
except ImportError:
print("etree example ready; lxml comparison is optional.")
else:
advanced = pd.read_xml(StringIO(student_xml), xpath=".//Row", parser="lxml")
print("Parser results match:", basic.equals(advanced))
assert basic.equals(advanced)Expected output
Parser results match: TrueRead attributes and child elements 🔝
Attributes on the selected record elements and their direct child fields can both become columns. Inspect the result when names collide.
mixed_xml = """<Students>
<Student id="1"><name>John Deo</name><class>Four</class><mark>75</mark></Student>
<Student id="2"><name>Max Ruin</name><class>Three</class><mark>85</mark></Student>
</Students>"""
mixed = pd.read_xml(StringIO(mixed_xml), xpath=".//Student", parser="etree")
print(mixed)
assert mixed.columns.tolist() == ["id", "name", "class", "mark"]Expected output
id name class mark
0 1 John Deo Four 75
1 2 Max Ruin Three 85Extract only attributes 🔝
attrs_only=True extracts record attributes and excludes child fields and record text. Do not enable it when the required values are stored in children.
attribute_xml = """<Students>
<Student id="1" name="John Deo" class="Four">Passed</Student>
<Student id="2" name="Max Ruin" class="Three">Failed</Student>
<Student id="3" name="Arnold" class="Three">Passed</Student></Students>"""
attributes = pd.read_xml(StringIO(attribute_xml), xpath=".//Student", attrs_only=True, parser="etree")
print(attributes)
assert attributes.columns.tolist() == ["id", "name", "class"]Expected output
id name class
0 1 John Deo Four
1 2 Max Ruin Three
2 3 Arnold ThreeExtract only child elements 🔝
elems_only=True excludes record attributes. Choose the option that matches your data rather than enabling attrs_only and elems_only together.
elements = pd.read_xml(StringIO(mixed_xml), xpath=".//Student", elems_only=True, parser="etree")
print(elements)
assert "id" not in elements.columns and elements["mark"].tolist() == [75, 85]Expected output
name class mark
0 John Deo Four 75
1 Max Ruin Three 85Map an explicit namespace prefix 🔝
Map the XPath prefix to the namespace URI. A prefix is an alias for the URI; the spelling of that alias does not have to match the source prefix.
namespace_xml = """<ns:Students xmlns:ns="http://example.com/ns">
<ns:Student id="1" name="John Deo" class="Four">Passed</ns:Student>
<ns:Student id="2" name="Max Ruin" class="Three">Failed</ns:Student></ns:Students>"""
namespaced = pd.read_xml(StringIO(namespace_xml), xpath=".//ns:Student",
namespaces={"ns": "http://example.com/ns"}, parser="etree")
print(namespaced)
assert namespaced["name"].tolist() == ["John Deo", "Max Ruin"]Expected output
id name class Student
0 1 John Deo Four Passed
1 2 Max Ruin Three FailedHandle a default namespace 🔝
Assign a temporary prefix when the document declares xmlns without an explicit prefix. Use that prefix in the XPath.
default_xml = """<Students xmlns="http://example.com/ns">
<Student><id>1</id><name>John Deo</name></Student>
<Student><id>2</id><name>Max Ruin</name></Student></Students>"""
default_frame = pd.read_xml(StringIO(default_xml), xpath=".//doc:Student",
namespaces={"doc": "http://example.com/ns"}, parser="etree")
print(default_frame)
assert len(default_frame) == 2Expected output
id name
0 1 John Deo
1 2 Max RuinRename parsed columns 🔝
names must correspond to the parsed fields in number and order. Inspect the source fields before renaming them. See column inspection.
renamed = pd.read_xml(StringIO(student_xml), xpath=".//Row", parser="etree",
names=["student_id", "student_name", "class_name", "mark", "gender"])
print(renamed.head(2))
assert renamed.columns.tolist() == ["student_id", "student_name", "class_name", "mark", "gender"]Expected output
student_id student_name class_name mark gender
0 1 John Deo Four 75 female
1 2 Max Ruin Three 85 malePreserve types and convert text 🔝
Use string for identifiers with significant leading zeros and nullable Int64 for integer values that may be missing. converters can transform individual fields and takes precedence over dtype for those fields. See dtypes.
typed_xml = """<rows><row><id>001</id><name> Ravi </name><mark>75</mark></row>
<row><id>002</id><name> Raju </name><mark/></row></rows>"""
typed = pd.read_xml(StringIO(typed_xml), parser="etree", dtype={"id": "string", "mark": "Int64"},
converters={"name": lambda value: value.strip() if value is not None else value})
print(typed)
print(typed.dtypes)
assert typed["id"].tolist() == ["001", "002"] and typed["mark"].isna().sum() == 1Expected output
id name mark
0 001 Ravi 75
1 002 Raju <NA>
id string
name str
mark Int64
dtype: objectParse date fields and validate failures 🔝
parse_dates can read valid date columns. For strict format checks and invalid input, parse explicitly after importing with to_datetime(). Retain failed rows for review.
date_xml = """<Students><Student><name>John Deo</name><joined>2026-01-10</joined></Student>
<Student><name>Max Ruin</name><joined>2026-02-15</joined></Student></Students>"""
dated = pd.read_xml(StringIO(date_xml), xpath=".//Student", parser="etree", parse_dates=["joined"])
print(dated.dtypes)
assert dated.iloc[0]["joined"] == pd.Timestamp("2026-01-10")Expected output
name str
joined datetime64[us]
dtype: objectSelect a nested level or flatten it deliberately 🔝
read_xml() does not automatically turn every nested field into a dotted column. Selecting marks reads its children but omits the parent ID. When both levels are required, parse the parent and child fields together with ElementTree, or transform the XML first with lxml/XSLT.
nested_xml = """<Students><Student><id>1</id><name>John Deo</name>
<marks><score1>75</score1><score2>80</score2></marks></Student></Students>"""
parent = pd.read_xml(StringIO(nested_xml), xpath=".//Student", parser="etree")
marks = pd.read_xml(StringIO(nested_xml), xpath=".//marks", parser="etree")
print("Parent columns:", parent.columns.tolist())
print(marks)
rows = []
for node in ET.fromstring(nested_xml).findall(".//Student"):
rows.append({"id": node.findtext("id"), "name": node.findtext("name"),
"score1": int(node.findtext("marks/score1")), "score2": int(node.findtext("marks/score2"))})
flat = pd.DataFrame(rows)
print(flat)
assert flat.iloc[0]["score2"] == 80Expected output
Parent columns: ['id', 'name', 'marks']
score1 score2
0 75 80
id name score1 score2
0 1 John Deo 75 80Use iterparse on a local XML file 🔝
iterparse replaces XPath parsing and selects a repeating element plus descendant fields. It avoids building the full XML tree, but the resulting DataFrame still needs memory; it is not a chunksize iterator. Use a physical local file for this example.
with tempfile.TemporaryDirectory(dir=Path.cwd()) as folder:
target = Path(folder) / "student.xml"
target.write_text(student_xml, encoding="utf-8")
streamed = pd.read_xml(target, parser="etree",
iterparse={"Row": ["id", "name", "class", "mark", "gender"]})
print(streamed.head(2))
print("Rows:", len(streamed))
assert streamed.equals(students)Expected output
id name class mark gender
0 1 John Deo Four 75 female
1 2 Max Ruin Three 85 male
Rows: 35Practical XML import: keep accepted and review records 🔝
This synthetic sales rule requires a non-blank ID, non-negative numeric amount and valid ISO date. Preserve original values and keep rejected records separate. See the Data Cleaning hub.
sales_xml = """<orders>
<order id="001"><date>2026-01-02</date><amount>5.25</amount></order>
<order id="002"><date>2026-01-03</date><amount>20</amount></order>
<order id="003"><date>bad</date><amount>8</amount></order>
<order id="004"><date>2026-01-04</date><amount>pending</amount></order></orders>"""
raw = pd.read_xml(StringIO(sales_xml), parser="etree", dtype="string")
clean = raw.copy()
clean["date"] = pd.to_datetime(raw["date"], format="%Y-%m-%d", errors="coerce")
clean["amount"] = pd.to_numeric(raw["amount"], errors="coerce")
valid = raw["id"].notna() & raw["id"].str.strip().ne("") & clean["date"].notna() & clean["amount"].ge(0).fillna(False)
accepted = clean.loc[valid].copy()
review = raw.loc[~valid].copy()
print(accepted)
print("Review IDs:", review["id"].tolist())
print("Accepted revenue:", accepted["amount"].sum())
assert accepted["id"].tolist() == ["001", "002"]
assert accepted["amount"].sum() == 25.25
assert len(accepted) + len(review) == len(raw)Expected output
id date amount
0 001 2026-01-02 5.25
1 002 2026-01-03 20.0
Review IDs: ['003', '004']
Accepted revenue: 25.25Export checked records 🔝
Write accepted records without disposable index labels. Preserve identifier types when reloading. See to_xml(), CSV export and Python XML.
text = accepted.to_csv(index=False, date_format="%Y-%m-%d")
print(text.rstrip())
checked = pd.read_csv(StringIO(text), dtype={"id": "string"})
assert checked["id"].tolist() == ["001", "002"]
assert checked["amount"].sum() == 25.25
# Optional Colab download:
# accepted.to_csv("accepted_xml_orders.csv", index=False, date_format="%Y-%m-%d")
# from google.colab import files
# files.download("accepted_xml_orders.csv")Expected output
id,date,amount
001,2026-01-02,5.25
002,2026-01-03,20.0Common errors and parameter choices 🔝
Literal XML is mistaken for a path: wrap markup in StringIO. No nodes match: inspect XPath and namespace mappings. A complex XPath fails with etree: use lxml or simplify the expression. Parent values are missing: selecting only a nested level does not attach its parent fields. Required attributes disappeared: check elems_only. Required children disappeared: check attrs_only. Dates or numbers are invalid: retain text first and validate explicitly. Large files still require memory: iterparse returns a DataFrame. What does stylesheet do? With lxml, an XSLT 1.0 stylesheet can flatten the XML; XPath then targets the transformed structure. encoding, compression, storage_options and dtype_backend cover additional source and type requirements. For a desktop import workflow, see Tkinter with Pandas.
Exercise: read and filter student XML 🔝
Read id, name and mark from student_xml, then select students with marks at least 90. Count the selected rows.
Exercise solution 🔝
Select the required fields after importing, then filter using loc.
exercise = pd.read_xml(StringIO(student_xml), xpath=".//Row", parser="etree")
answer = exercise.loc[exercise["mark"].ge(90), ["id", "name", "mark"]]
print(answer)
print("Rows:", len(answer))
assert len(answer) == int(students["mark"].ge(90).sum())Expected output
id name mark
11 12 Recky 94
31 32 Binn Rott 90
32 33 Kenn Rein 96
Rows: 3Practice in Google Colab 🔝
Open in Google Colab View on GitHub
Run the examples in order, change XML structures and sales values, then inspect imported records and review rows. Save your own copy to keep edits. All sample tables are included in the notebook.
Reference: Pandas read_xml documentation.
Subhendu Mohapatra
Author & Instructor at plus2net
I write and maintain practical tutorials on Python, PHP, SQL, JavaScript, HTML, jQuery, and web development at plus2net. The tutorials focus on clear explanations, working examples, and code that readers can test and adapt while learning.
Subscribe to our YouTube Channel here
Python Video Tutorials