如何在使用python pandas连接一组csv文件时删除重复的标题(多行)
如何在使用python pandas连接一组csv文件时删除重复的标题(多行)
提问于 2019-04-29 16:47:29
我使用python pandas库将一组(6-10个文件) .csv文件连接成一个.csv文件。我想删除标题(行)包含7行的所有csv文件，除了第一个文件。我该怎么做呢？
import glob
import pandas as pd
#filenames = glob.glob(path + "/*.csv")
filenames = glob.glob("*.csv")
print(filenames)
count_files = 0 dfs = []
for filename in filenames:
if count_files ==0:
dfs.append(pd.read_csv(filename))
full_df =pd.concat(dfs) count_files += 1
else:
dfs.append(pd.read_csv(filename, sep=";", skiprows=[0])) #dfs.append(pd.read_csv(filename))
full_df =pd.concat(dfs)
count_files +=1
full_df.to_csv( "combined_csv.csv",header = None, index=False, encoding='utf-8-sig')回答 2
Stack Overflow用户
发布于 2019-04-29 17:19:38
#creating dummy csv's for your requirement.
## appending muliple csvs in to one single csv
df=pd.DataFrame({'A':[1,1,1],
'B':[1,2,3],
'C':[3,9,3],
'D':[1,8,9]})
df1=pd.DataFrame({'A':[4,5,5],
'B':[1,1,2],
'C':[2,2,8],
'D':[6,4,3]})
df2=pd.DataFrame({'A':[9,1,1],
'B':[9,2,3],
'C':[3,9,13],
'D':[9,8,9]})
df3=pd.DataFrame({'A':[14,15,5],
'B':[1,11,2],
'C':[12,12,8],
'D':[6,4,3]})
df.to_csv("one.csv")
df1.to_csv("two.csv")
df2.to_csv("three.csv")
df3.to_csv("four.csv")
import os
csv_list = []
for root, dirs,files in os.walk(os.getcwd(), topdown=True):
for name in files:
csv_list.append(os.path.join(root, name))
print(csv_list)
['/home/vikas.rana/stck_flw/two.csv',
'/home/vikas.rana/stck_flw/one.csv',
'/home/vikas.rana/stck_flw/four.csv',
'/home/vikas.rana/stck_flw/three.csv']
names = ['A','B','C','D']
combined_csv = pd.concat([pd.read_csv(f, header=None,skiprows=[0],names = names) for f in csv_list ],ignore_index=True)
print(combined_csv)
# output
A B C D
0 4 1 2 6
1 5 1 2 4
2 5 2 8 3
3 1 1 3 1
4 1 2 9 8
5 1 3 3 9
6 14 1 12 6
7 15 11 12 4
8 5 2 8 3
9 9 9 3 9
10 1 2 9 8
11 1 3 13 9Stack Overflow用户
发布于 2019-04-29 17:24:27
正如每个人都说的那样，省略一些代码将有助于澄清您的意图。
但是，这可以解决您的问题。它包括从rest创建一个辅助CSV文件，然后导入它以将其存储为Pandas DataFrame (以防万一您需要它)。
假设FileName1.csv包含以下内容：
ColumnName_1,ColumnName_2,ColumnName_3
data11,data12,data13
data21,data22,data33和FileName2.csv具有以下内容：
ColumnName_1,ColumnName_2,ColumnName_3
Row to be deleted
Row to be deleted
Row to be deleted
data2_11,data2_12,data2_13
data2_21,data2_22,data2_33让我们假设您想要保留文件1中的头文件，并跳过第二个文件的前4行。
import pandas as pd
# Define a function that gets the file content ignoring n first rows
def get_content(file_path,ignored_rows):
f = open(file_path,'r')
file_data = f.readlines()
for line in file_data[ignored_rows:]:
files_content.append(line.rstrip('\n'))
# Generate empty List to allocate files rows
files_content = []
# Read first file
get_content('Files/FileName1.csv',0)
# Read second file
get_content('Files/FileName2.csv',4)
# Generate Complete CSV File
with open('Files/FullData.csv','w') as f:
for line in files_content:
f.write(line+'\n')
df = pd.read_csv('Files/FullData.csv')这样就可以读取少量文件了。如果需要读取大量文件，可以添加另一个循环来应用相同的代码。
页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接：
https://stackoverflow.com/questions/55899820
复制相关文章
相似问题
腾讯云开发者
