将CSV文件拆分为两个文件，在两个文件中保留标头
提问于 2021-10-13 16:00:48
我正在尝试将一个大的CSV文件拆分为两个文件。我正在使用下面的代码
import pandas as pd
#csv file name to be read in
in_csv = 'Master_file.csv'
#get the number of lines of the csv file to be read
number_lines = sum(1 for row in (open(in_csv)))
#size of rows of data to write to the csv,
#you can change the row size according to your need
rowsize = 600000
#start looping through data writing it to a new file for each set
for i in range(0,number_lines,rowsize):
df = pd.read_csv(in_csv,
nrows = rowsize,#number of rows to read at each loop
skiprows = i)#skip rows that have been read
#csv to write data to a new file with indexed name. input_1.csv etc.
out_csv = 'File_Number' + str(i) + '.csv'
df.to_csv(out_csv,
index=False,
header=True,
mode='a',#append data to csv file
chunksize=rowsize)#size of data to append for each loop它正在拆分文件，但它在第二个文件中缺少标头。我怎么才能修复它呢？
回答 1
Stack Overflow用户
发布于 2021-10-13 18:14:50
当与chunksize一起使用时，.read_csv()返回一个迭代器，然后跟踪标头。下面是一个示例。这应该要快得多，因为上面的原始代码读取整个文件来计算行数，然后在每次块迭代中重新读取之前的所有行；而下面的代码只读取文件一次：
import pandas as pd
with pd.read_csv('Master_file.csv', chunksize=60000) as reader:
for i,chunk in enumerate(reader):
chunk.to_csv(f'File_Number{i}.csv', index=False, header=True)页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接：
https://stackoverflow.com/questions/69563629
复制相关文章
腾讯云开发者
