A DataFrame is the primary data structure of the Pandas library in Python and is commonly used for storing and working with tabular data.
A common operation that could be performed on such data is to normalize the columns of the DataFrame to work with the information in a better manner.
To start working with Pandas, we first need to import it:
import pandas as pdLet us understand this operation with the help of an example. Consider the following DataFrame containing 3 students with names A, B, and C and their corresponding marks (out of 10) for two subjects, Mathematics and Physics.
Code snippet for generating the above DataFrame:
import pandas as pd
# Dictionary for our data
data = {'Name' : ['A', 'B', 'C'], 'Mathematics' : [8, 5, 10], 'Physics' : [7, 9, 8]}
# DataFrame for the dictionary
df = pd.DataFrame(data)
# Printing the DataFrame
print(df)Here, data is a dictionary we created to initialize the DataFrame. For this, we use the DataFrame() function of the Pandas library which takes the dictionary as an argument and returns the required DataFrame.
Now, let’s say we need to normalize a column of the DataFrame, say the physics column. The resulting output would look like this:
Let us look at different ways of performing this operation on a given DataFrame :
In this method, we use statistical functions of the series to normalize the desired column, here physics.
We use square brackets to access the column of interest in the form of df[‘Physics’].
Here, we implement mean normalization by subtracting the mean from the data point and dividing it by the standard deviation of the column data.
We use the mean() function to get the mean of the column data and the std() function to get the standard deviation. The updates are not made in place so reassignment is required.
Let us take a look at the corresponding code snippet and generated output for this method:
# Importing required libraries
import pandas as pd
# Dictionary for our data
data = {'Name' : ['A', 'B', 'C'], 'Mathematics' : [8, 5, 10], 'Physics' : [7, 9, 8]}
# DataFrame for the dictionary
df = pd.DataFrame(data=data)
# Performing the operation
df['Physics'] = (df['Physics'] - df['Physics'].mean())/df['Physics'].std()
# PrintingÂ
print(df)Output :
Instead of using df[‘Physics’] to access the physics column, we could also simply use the dot operator in the form of df.Physics to get the same result.
Let us take a look at the corresponding code snippet and generated output for this method:
# Importing required libraries
import pandas as pd
# Dictionary for our data
data = {'Name' : ['A', 'B', 'C'], 'Mathematics' : [8, 5, 10], 'Physics' : [7, 9, 8]}
# DataFrame for the dictionary
df = pd.DataFrame(data=data)
# Performing the operation
df.Physics = (df.Physics - df.Physics.mean())/df.Physics.std()
# PrintingÂ
print(df)Output :
Instead of performing mean normalization, we could also choose to perform min-max normalization which involves subtracting the minimum value of the column data from the data point and dividing by the difference between maximum and minimum values.
This returns values between 0 and 1 for each data entry. Let us take a look at the corresponding code snippet and generated output for this method:
# Importing required libraries
import pandas as pd
# Dictionary for our data
data = {'Name' : ['A', 'B', 'C'], 'Mathematics' : [8, 5, 10], 'Physics' : [7, 9, 8]}
# DataFrame for the dictionary
df = pd.DataFrame(data=data)
# Performing the operation
df.Physics = (df.Physics - df.Physics.min()) / (df.Physics.max() - df.Physics.min())
# PrintingÂ
print(df)Output :
In this method, we use the apply() function to normalize the contents of the desired column, here physics. df.Physics is used to access the column.
We perform mean normalization in this example. Inside the apply() function, we pass the lambda function as a parameter to be applied to each element of the column, using which we specify the manner of normalization.
This function returns the updated series of the column physics. The updates are not made in place so reassignment is required. Let us take a look at the corresponding code snippet and generated output for this method:
# Importing required libraries
import pandas as pd
# Dictionary for our data
data = {'Name' : ['A', 'B', 'C'], 'Mathematics' : [8, 5, 10], 'Physics' : [7, 9, 8]}
# DataFrame for the dictionary
df = pd.DataFrame(data=data)
# Performing the operation
df.Physics = df.Physics.apply(lambda x : (x - df.Physics.mean())/ df.Physics.std())
# PrintingÂ
print(df)Output :
In this topic, we have learned to normalize the columns of an existing Pandas DataFrame, following a running example of test scores of students in different subjects, thus giving us an intuition of how this concept could be applied in real-world situations. Feel free to reach out to info.javaexercise@gmail.com in case of any suggestions.