Splitting a CSV File using Powershell

powershell May 01, 2019

Scenario

Given a CSV file, suppose you want to split it into more manageable chunks. This is useful if the CSV file is some sort of input that must be processed in batches.

Code Snippet

Source: https://www.codetwo.com/admins-blog/how-to-split-csv-file-into-multiple-files-using-powershell/


# variable used to store the path of the source CSV file
$sourceCSV = <PATH-OF-SOURCE-CSV> ;

# variable used to advance the number of the row from which the export starts
$startrow = 0 ;

# counter used in names of resulting CSV files
$counter = 1 ;

# setting the while loop to continue as long as the value of the $startrow variable is smaller than the number of rows in your source CSV file
while ($startrow -lt <TOTAL-NUMBER-OF-ROWS-IN-SOURCE-CSV>)
{

# import of however many rows you want the resulting CSV to contain starting from the $startrow position and export of the imported content to a new file
Import-CSV $sourceCSV | select-object -skip $startrow -first <NUMBER-OF-ROWS-IN-OUTPUT-CSV> | Export-CSV "<PATH-OF-OUTPUT-CSV>$($counter).csv" -NoClobber;

# advancing the number of the row from which the export starts
$startrow += <NUMBER-OF-ROWS-IN-OUTPUT-CSV> ;

# incrementing the $counter variable
$counter++ ;

}

If you don't want the headers to be included, change the Import-Csv line to the following:


Import-CSV $sourceCSV | select-object -skip $startrow -first <NUMBER-OF-ROWS-IN-OUTPUT-CSV> | ConvertTo-Csv -NoType | Select-Object -Skip 1 | Set-Content "<PATH-OF-OUTPUT-CSV>$($counter).csv";

Save your script as splitcsv.ps1 and run on your Powershell console.


./splitcsv

unsplash-logodavisco

Kristina Alberto

Software engineer from Sydney. I work at Domain Group.