Convert Nested List of Dictionary into Pandas Dataframe
Below are the methods that will be used
- Using from_dict(orient=’index’)
- Native Method
Nested Dictionary to Pandas DataFrame Using orient=’index’
In this example, the Panda’s library is used to create a DataFrame named df from a dictionary (countries) where each key represents a unique identifier for a country, and the values contain information about the country’s name, capital, and population. The resulting DataFrame is printed, displaying the modified country details with columns for “Country,” “Capital,” and “Population.
Python3
# importing pandas import pandas as pd countries = { "1" : { "Country" : "New Country 1" , "Capital" : "New Capital 1" , "Population" : "123,456,789" }, "2" : { "Country" : "New Country 2" , "Capital" : "New Capital 2" , "Population" : "987,654,321" }, "3" : { "Country" : "New Country 3" , "Capital" : "New Capital 3" , "Population" : "111,222,333" } } df = pd.DataFrame.from_dict(countries, orient = 'index' ) print (df) |
Output:
Country Capital Population
1 New Country 1 New Capital 1 123,456,789
2 New Country 2 New Capital 2 987,654,321
3 New Country 3 New Capital 3 111,222,333
Nested Dictionary to Pandas DataFrame using Native Method
Below are the steps by which we can convert list of nested dictionary into Pandas DataFrame:
Step 1: Create a List of Nested Dictionary
In this step, a list of dictionaries represents students with associated exam scores and grades. The data is structured such that each dictionary includes a “Student” key containing a list of exam details, and a “Name” key indicating the student’s name.
Python3
# importing pandas import pandas as pd # List of nested dictionary initialization list = [{ "Student" : [{ "Exam" : 90 , "Grade" : "a" }, { "Exam" : 99 , "Grade" : "b" }, { "Exam" : 97 , "Grade" : "c" }, ], "Name" : "Paras Jain" }, { "Student" : [{ "Exam" : 89 , "Grade" : "a" }, { "Exam" : 80 , "Grade" : "b" } ], "Name" : "Chunky Pandey" } ] print ( list ) |
Output:
[{'Student': [{'Exam': 90, 'Grade': 'a'}, {'Exam': 99, 'Grade': 'b'}, {'Exam': 97, 'Grade': 'c'}],
'Name': 'Paras Jain'},
{'Student': [{'Exam': 89, 'Grade': 'a'}, {'Exam': 80, 'Grade': 'b'}],
'Name': 'Chunky Pandey'}]
Step 2: Adding Dict Values to Rows
In this step, a new list named “rows” is initialized to store flattened data. The code iterates through the original list of dictionaries, extracting each student’s exam details along with their name and appends the flattened rows to create a Pandas DataFrame, which is then displayed.
Python3
# rows list initialization rows = [] # appending rows for data in list : data_row = data[ 'Student' ] time = data[ 'Name' ] for row in data_row: row[ 'Name' ] = time rows.append(row) # using data frame df = pd.DataFrame(rows) print (df) |
Output:
Exam Grade Name
0 90 a Paras Jain
1 99 b Paras Jain
2 97 c Paras Jain
3 89 a Chunky Pandey
4 80 b Chunky Pandey
Step 3: Pivoting DataFrame and Assigning Column Names
In this step, the Pandas pivot_table function is utilized to reshape the DataFrame. The “Name” column becomes the index, “Grade” values are transformed into separate columns (‘Maths’, ‘Physics’, ‘Chemistry’), and corresponding ‘Exam’ values fill the table. The resulting DataFrame is then displayed with the updated column names.
Python3
# using pivot_table df = df.pivot_table(index = 'Name' , columns = [ 'Grade' ], values = [ 'Exam' ]).reset_index() # Defining columns df.columns = [ 'Name' , 'Maths' , 'Physics' , 'Chemistry' ] # print dataframe print (df) |
Output:
Name Maths Physics Chemistry
0 Chunky Pandey 89.0 80.0 NaN
1 Paras Jain 90.0 99.0 97.0
Below is the complete code:
Python3
# importing pandas import pandas as pd # List of list of dictionary initialization list = [ { "Student" : [{ "Exam" : 90 , "Grade" : "a" }, { "Exam" : 99 , "Grade" : "b" }, { "Exam" : 97 , "Grade" : "c" }, ], "Name" : "Paras Jain" }, { "Student" : [{ "Exam" : 89 , "Grade" : "a" }, { "Exam" : 80 , "Grade" : "b" } ], "Name" : "Chunky Pandey" } ] # rows list initialization rows = [] # appending rows for data in list : data_row = data[ 'Student' ] time = data[ 'Name' ] for row in data_row: row[ 'Name' ] = time rows.append(row) # using data frame df = pd.DataFrame(rows) # using pivot_table df = df.pivot_table(index = 'Name' , columns = [ 'Grade' ], values = [ 'Exam' ]).reset_index() # Defining columns df.columns = [ 'Name' , 'Maths' , 'Physics' , 'Chemistry' ] # print dataframe print (df) |
Output:
Name Maths Physics Chemistry
0 Chunky Pandey 89 80 NaN
1 Paras Jain 90 99 97
Python | Convert list of nested dictionary into Pandas dataframe
Given a list of the nested dictionary, write a Python program to create a Pandas dataframe using it. We can convert list of nested dictionary into Pandas DataFrame. Let’s understand the stepwise procedure to create a Pandas Dataframe using the list of nested dictionary.