Copy Csv Data Into Csv Files in Python
Below are some of the ways by which we can copy data between CSV files in Python:
The CSV file used is StudentPerformance.CSV.Contents of the file are
gender,race/ethnicity,parental level of education,lunch,test preparation course,math score,reading score,writing score
female,group B,bachelor's degree,standard,none,72,72,
female,group C,some college,standard,completed,69,90,88
female,group B,master's degree,standard,none,90,95,93
male,group A,associate's degree,free/reduced,none,47,57,44
male,group C,some college,standard,none,76,78,75
female,group B,associate's degree,standard,none,71,83,78
female,group B,some college,standard,completed,88,95,92
male,group B,some college,free/reduced,none,40,43,39
Copy Csv Data Into Csv Files Using CSV Module
This fundamental approach leverages the built-in csv module, offering fine-grained control over data movement. You’ll open both files, iterate through each row in the source file, and selectively write desired data to the target file.
This code iterates through each row in student_scores2.csv, extracts elements at specific indices (0 and 2), modifies the second element by adding 10, and writes the modified row to target.csv. This showcases basic data transfer and manipulation within the loop.
Python3
import csv with open ( "StudentPerformance.csv" , "r" ) as source, open ( "target.csv" , "w" ) as target: reader = csv.reader(source) writer = csv.writer(target) for row in reader: if row[ 2 ].isdigit(): new_row = [row[ 0 ], str ( int (row[ 2 ]) + 10 )] else : new_row = row # Handle rows with non-numerical data at index 2 # Print the selected or manipulated row to the console print (new_row) # Write the row to the target CSV file writer.writerow(new_row) print ( '\ncsv Data copied to target csv files' ) |
Output:
['gender', 'race/ethnicity', 'parental level of education', 'lunch', 'test preparation course', 'math score', 'reading score', 'writing score']
['female', 'group B', "bachelor's degree", 'standard', 'none', '72', '72', '']
['female', 'group C', 'some college', 'standard', 'completed', '69', '90', '88']
['female', 'group B', "master's degree", 'standard', 'none', '90', '95', '93']
['male', 'group A', "associate's degree", 'free/reduced', 'none', '47', '57', '44']
['male', 'group C', 'some college', 'standard', 'none', '76', '78', '75']
['female', 'group B', "associate's degree", 'standard', 'none', '71', '83', '78']
['female', 'group B', 'some college', 'standard', 'completed', '88', '95', '92']
['male', 'group B', 'some college', 'free/reduced', 'none', '40', '43', '39']
csv Data copied to target csv files
Python Copy Csv Data Into Csv Files Using Pandas
In this example, we are using Pandas to copy CSV data into CSV files. Here reads StudentPerformance.csv into a Pandas dataframe, filters rows based on a condition in a specific column, and writes the filtered data to target.csv without including the dataframe index.
Python3
import pandas as pd # Read the dataset url = "https://media.w3wiki.org/wp-content/uploads/20240206150437/StudentPerformance.csv" df = pd.read_csv(url) # Define the threshold value for "Hours" threshold = 90 # Replace with your desired threshold filtered_data = df[df[ "reading score" ] > threshold] # Print the filtered DataFrame to the console print (filtered_data) # Write the filtered data to the target CSV file filtered_data.to_csv( "target.csv" , index = False ) print ( '\n Filtered csv Data copied to target csv files' ) |
Output:
gender race/ethnicity parental level of education lunch \
2 female group B master's degree standard
6 female group B some college standard
test preparation course math score reading score writing score
2 none 90 95 93.0
6 completed 88 95 92.0
Filtered csv Data copied to target csv files
Merging Multiple Files with glob and Pandas
Need to combine the insights from multiple CSV files? Enter the dynamic duo of glob and Pandas. glob finds files matching a pattern, while Pandas seamlessly merges them into one DataFrame for further analysis or writing to a single CSV.
Python3
import pandas as pd import os directory = 'Data' all_files = os.listdir(directory) # Get the full file paths file_paths = [os.path.join(directory, file ) for file in all_files if file .split( '.' )[ - 1 ] = = 'csv' ] # Print the file paths print ( 'All csv files path:\n' ,file_paths) combined_data = pd.concat([pd.read_csv(f) for f in file_paths]) # Reset the index combined_data.reset_index(inplace = True , drop = True ) # Print the combined DataFrame to the console print (combined_data) # Save the combined data to the CSV file combined_data.to_csv( "master_data.csv" , index = False ) |
Output:
All csv files path:
['Data/StudentPerformance.csv', 'Data/target.csv']
gender race/ethnicity parental level of education lunch \
0 female group B bachelor's degree standard
1 female group C some college standard
2 female group B master's degree standard
3 male group A associate's degree free/reduced
4 male group C some college standard
5 female group B associate's degree standard
6 female group B some college standard
7 male group B some college free/reduced
8 female group B master's degree standard
9 female group B some college standard
test preparation course math score reading score writing score
0 none 72 72 NaN
1 completed 69 90 88.0
2 none 90 95 93.0
3 none 47 57 44.0
4 none 76 78 75.0
5 none 71 83 78.0
6 completed 88 95 92.0
7 none 40 43 39.0
8 none 90 95 93.0
9 completed 88 95 92.0
Copying Csv Data Into Csv Files Using Python
CSV files, the stalwarts of information exchange, can be effortlessly harnessed to extract specific data or merge insights from multiple files. In this article, we unveil five robust approaches, transforming you into a virtuoso of CSV data migration in Python. Empower your data-wrangling endeavors, mastering the art of copying and organizing information with finesse.