Ways to Split Large File for CSV or TXT on Linux Mint

Ways to Split Large File for CSV or TXT on Linux Mint –The text based file contain many rows of data and you can split them into separated file easily on Linux. How you can split large text file into multiple files on Linux let’s follow these guide.

Split a large file into smaller files on Linux can be done with Terminal, the command promt on Linux and you can also Keep Header Row of the file.

For an example you have a CSV file which contain around 1000 rows; each row having 5 columns. You want to break this CSV file into 10 CSV files of 100 records each. (Each having same CSV header as present in original CSV). How to solve this case ?

This is the result of files that I have tried using these method:

split large file linux split large file linux command divide large file linux split large text file linux split large log file linux split large csv file linux split large sql file linux split large binary file linux split large xml file linux split large file into chunks linux linux split large file into parts split large file into smaller files linux linux split large file in two linux split large file by lines split large file into multiple files linux split large text file into multiple files linux split large text file into smaller files linux split large xml file into smaller files linux linux split large tar file linux split compress large file split a large file linux split a large text file linux split a large file into smaller files linux linux split and compress large file

Ways to Split Large File for CSV or TXT on Linux Mint

Linux has a great little utility called split, which can take a file and split it into chunks of whatever size you want, eg 100 line chunks.

However, for CSV files etc, each chunk generally needs to have the header row in there. Unfortunately the split command doesn’t have an option for that. However with a little bit more code you can achieve it.

If yuo have CSV file called hotel.csv which contain 1000 rows; each row having 5 columns. You want to break this CSV file into 10 CSV files of 100 records each.

Place that file into your home directory and open Terminal (press Ctrl + Alt + T), then type the following command :
tail -n +2 hotel.csv | split -l 100 – split_

Press enter then enter this command :
for file in split_*; do head -n 1 hotel.csv > tmp_file; cat $file >> tmp_file; mv -f tmp_file $file; done

Press enter again, for the result you can see the created new file named split_aa.csv, split_ab.csv, …etc on your home directory. Now you have finished split large text file into smaller files on Linux.

You can also execute the command one by one per line into Terminal as follow:

tail -n +2 hotel.csv | split -l 100 - split_
for file in split_*
do
    head -n 1 hotel.csv > tmp_file
    cat $file >> tmp_file
    mv -f tmp_file $file
done