Methods and Technical Analysis for Creating New Columns in Pandas DataFrame

Introduction

In the process of data analysis and processing, there is often a need to create new derived columns based on existing data columns. Pandas, as a powerful data processing library in Python, provides multiple flexible ways to achieve this functionality. This article will systematically introduce the core methods for creating new columns through specific examples.

Basic Column Operation Methods

The simplest and most direct approach is to use arithmetic operations between columns. Assuming we have a DataFrame containing columns A and B:

import pandas as pd

df = pd.DataFrame({
    'A': [1, 2, 3],
    'B': [4, 6, 9]
})

To create a new column C with values equal to the sum of corresponding elements in columns A and B, the following code can be used:

df['C'] = df['A'] + df['B']

The advantage of this method lies in its simplicity and efficiency. Pandas internally uses vectorized operations, avoiding explicit loops and significantly improving computational efficiency. The operation is performed element-wise, resulting in the following new DataFrame:

   A  B   C
0  1  4   5
1  2  6   8
2  3  9  12

Flexible Approach Using Apply Function

For more complex calculation logic, the apply function combined with lambda expressions can be used:

df['C'] = df.apply(lambda row: row['A'] + row['B'], axis=1)

Although this method is syntactically more flexible and can handle complex row-level calculations, its performance is inferior to direct vectorized operations due to the need for row-by-row processing. In practical applications, vectorized methods should be prioritized.

Advanced Applications of Multi-Column Summation

When simultaneous summation of multiple columns is required, the sum method can be employed:

df['sum'] = df[['A', 'B']].sum(axis=1)

Or in a more general form:

df['sum'] = df[list(df.columns)].sum(axis=1)

This method is particularly suitable for situations where the same operation needs to be performed on multiple specified columns, offering high code readability.

Case Study of Practical Applications

Referring to the air quality data processing case, we can observe similar technical applications. For example, converting NO₂ concentration from μg/m³ to mg/m³ for the London station:

air_quality["london_mg_per_cubic"] = air_quality["station_london"] * 1.882

Similarly, calculating the concentration ratio between Paris and Antwerp stations:

air_quality["ratio_paris_antwerp"] = air_quality["station_paris"] / air_quality["station_antwerp"]

Performance Analysis and Best Practices

Through performance testing and analysis of different methods, the following conclusions can be drawn:

• Direct column operation methods exhibit the best performance
• The apply function is suitable for complex custom calculation logic
• The sum method provides good code readability in multi-column operations

In actual projects, it is recommended to choose the appropriate method based on specific requirements, balancing performance, readability, and flexibility.

Conclusion

Pandas offers multiple methods for creating new columns, each with its specific applicable scenarios. Understanding the principles and performance characteristics of these methods helps in making more reasonable technical choices in data science projects, thereby improving code efficiency and quality.