This lesson on DataFrame & Series Basics: Selecting Rows and Columns is hands-on and example-driven. You will be able to distinguish between pandas DataFrames and Series objects based on their dimensionality. You can efficiently select single columns, multiple columns, or specific rows using bracket notation and the iloc indexer. This foundational skill is crucial for all subsequent data analysis tasks.
What You'll Be Able To Do
- Identify the structural difference between a DataFrame (2D) and a Series (1D).
- Select a single column from a DataFrame using dictionary-style bracket notation.
- Retrieve multiple, non-contiguous columns by passing a list of column names.
- Access a specific row using its integer position with the
ilocindexer. - Explain why bracket notation is preferred over dot notation for column access.
Detailed Concept Walkthrough
1. DataFrame and Series Structure
A DataFrame is a 2D structure (rows and columns), analogous to a table or a dictionary of lists. A Series is a 1D structure, representing a single column of data.
- Layman's Terms: A DataFrame can be conceptualized as a Python dictionary where keys are column names and values are lists of data (the rows).
- Mechanism: A DataFrame is essentially a container for multiple Series objects, where each Series shares the same index (rows).
- Under the Hood: Accessing a single column from a DataFrame returns a Series object, which has its own index and data type.
import pandas as pd
data = {'first': ['Corey', 'Jane', 'John'], 'last': ['Schafer', 'Doe', 'Doe']}
df = pd.DataFrame(data)
# Check the type of a single column access
print(type(df['first'])) # Output: <class 'pandas.core.series.Series'>
Key Takeaway: A DataFrame is 2D (columns and rows); a Series is 1D (rows of a single column).
2. Selecting Single Columns
Columns are accessed using bracket notation, similar to accessing a value by key in a Python dictionary. This operation returns a pandas Series.
- Syntax Rule: Use
df['column_name']to retrieve the column as a Series object. - Best Practice: Bracket notation is preferred over
df.column_name(dot notation) for reliability. - Pitfall: Dot notation fails if the column name conflicts with a DataFrame attribute or method (e.g., 'count' or 'head').
# Assuming df is defined from the previous snippet
email_series = df['email']
print(email_series)
Key Takeaway: Always use
df['column_name']to reliably access columns and avoid conflicts with DataFrame methods.
3. Selecting Multiple Columns
To select multiple columns, pass a Python list of column names inside the DataFrame's indexing brackets. This operation returns a new DataFrame.
- Syntax Rule: The syntax requires double brackets:
df[['col1', 'col2']]. The outer brackets are for indexing; the inner brackets define the list of columns to retrieve. - Mechanism: Passing a list tells pandas to return a subset of the original DataFrame, preserving the 2D structure.
- Result Type: Since multiple columns are returned, the result is always a DataFrame, not a Series.
# Assuming df has 'first', 'last', and 'email' columns
subset_df = df[['last', 'email']]
print(subset_df)
Key Takeaway: Use double brackets
df[['col1', 'col2']]to return a DataFrame subset containing specific columns.
4. Row Selection with Iloc
The .iloc indexer (integer location) is used to select rows based on their zero-based integer position, ignoring any custom index labels.
- Mechanism:
.iloctreats the DataFrame like a 2D array, relying solely on positional indexing (0, 1, 2, etc.). - Syntax Rule: Use
df.iloc[row_index]to retrieve a single row ordf.iloc[start:end]for a slice of rows. - Result Type: Selecting a single row returns a Series, where the index of the resulting Series is the column names of the DataFrame.
# Get the first row (position 0)
first_row = df.iloc[0]
print(first_row)
Key Takeaway: Use
df.iloc[N]to access the Nth row based on its integer position.
Topics Covered in DataFrame & Series Basics: Selecting Rows and Columns
- DataFrame as a Table (0:00 - 2:00) — DataFrames are the primary pandas data structure, displayed as rows and columns, similar to a spreadsheet table.
- DataFrame Mental Model (2:00 - 5:00) — Conceptually, a DataFrame is similar to a Python dictionary where keys are columns and values are lists representing the rows.
- Creating a DataFrame (5:00 - 7:00) — A DataFrame can be created directly from a dictionary of lists using
pd.DataFrame(). - Accessing Columns (Series) (7:00 - 9:00) — Accessing a single column using bracket notation returns a Series object, which is a 1D array of data.
- Series Definition (9:00 - 11:00) — A Series is a one-dimensional array of data, and a DataFrame is a container for multiple Series objects.
- Bracket vs. Dot Notation (11:00 - 13:00) — Bracket notation is preferred for column access because dot notation can fail if the column name conflicts with a DataFrame method.
- Selecting Multiple Columns (13:00 - 15:00) — Multiple columns are selected by passing a list of column names inside double brackets, which returns a subset DataFrame.
- Row Selection with Iloc (15:00 - 16:00) — Rows can be accessed by their integer position using the
.ilocindexer.
Python Cheat Sheet
-
df['col_name']— Selects a single column by name, returns a Seriesdf['email'] -
df.col_name— Selects a single column using dot notationdf.email -
df[['col1', 'col2']]— Selects multiple columns by name, returns a DataFramedf[['first', 'last']] -
df.columns— Returns an Index object containing all column namesprint(df.columns) -
df.iloc[N]— Selects the row at integer position Ndf.iloc[0]
Comparison Table
| Selection Method | Result Type | Reliability/Use Case |
|---|---|---|
| df['col_name'] | Series | Most reliable single column access. |
| df.col_name | Series | Quick access; fails on name conflicts. |
| df[['col1', 'col2']] | DataFrame | Required for selecting multiple columns. |
Common Pitfalls
- Mistake: Using dot notation when the column name is 'count'.
Avoid: Always use bracket notation
df['count']to access the column. - Mistake: Trying to select multiple columns without double brackets.
Avoid: Pass a list of names:
df[['col1', 'col2']]. - Mistake: Confusing
iloc(position) withloc(label). Avoid: Useiloconly for integer-based row selection (0, 1, 2...). - Mistake: Expecting a DataFrame when selecting a single column. Avoid: Remember single column access returns a Series.
FAQs
- What is the difference between a DataFrame and a Series? A DataFrame is a 2D table (rows and columns); a Series is a 1D array representing a single column of data.
- Why is accessing a single column like accessing a dictionary key? Pandas DataFrames are conceptually built like dictionaries where keys are column names and values are lists of data (the rows).
- Why does selecting multiple columns return a DataFrame instead of a Series? A Series can only hold one column of data. Since multiple columns are selected, the result must be a 2D structure (a DataFrame).
- What is the index on the far left of the DataFrame? The index provides a unique identifier for each row. It is used for efficient data lookup and alignment, and will be covered in detail in the next lesson.