数据存储为json和csv

进行 python 爬虫时,我们需要将数据存储起来。对于小数据量,我们可以存储到json文件或csv文件。

Python对json文件的操作是通过JSON模块实现的,分为编码和解码两个过程。

编码是指将python对象转换为json对象,常用的函数是dump()和dumps(),区别是dump()将python对象转换为json对象,并将json对象通过文件流写入文件,dumps()函数是返回字符串。

dump(obj, fp, skipkeys=False, ensure_ascii=True, check_circular=True,

allow_nan=True, cls=None, indent=None, separators=None,

encoding='utf-8', default=None, sort_keys=False, **kw)

dumps(obj, skipkeys=False, ensure_ascii=True, check_circular=True,

allow_nan=True, cls=None, indent=None, separators=None,

encoding='utf-8', default=None, sort_keys=False, **kw)

解码是指将json对象转换为python对象,常用的函数为load()和loads(),区别是load()是将json对象转换为python对象,并将python对象写入到文件,而loads()是返回字符串。

load(fp, encoding=None, cls=None, object_hook=None, parse_float=None,

parse_int=None, parse_constant=None, object_pairs_hook=None, **kw)

loads(s, encoding=None, cls=None, object_hook=None, parse_float=None,

parse_int=None, parse_constant=None, object_pairs_hook=None, **kw)

CSV(Comma-Separated Values,逗号分隔值,有时也称为字符分隔值),其文件是以纯文本形式存储表格数据。CSV文件由任意数目的记录组成,记录间以换行符分隔,每条记录由字段组成,字段间的分隔符是其他字符或字符串,最常见的是逗号和制表符。CSV文件示例如下:

ID,Username,Password,Age,Country

1001,yhw,123456,24,China

1002,Mary,654321,25,USA

1003,Jack,123567,22,English

Python通过 csv库来读写csv文件。Python写csv文件,需要用到write对象,然后按行写每一个记录;

importcsv

header = ("ID","Username","Password","Age","Country")

rows = [(1001,"yhw","123456",24,"China"),

(1002,"Mary","654321",25,"USA"),

(1003,"Jack","123567",22,"English")]

withopen("test3.csv","w")asfp:

fp_csv = csv.writer(fp)

fp_csv.writerow(header)

fp_csv.writerows(rows)

importcsv

rows = [{"ID":1001,"Username":"yhw","Password":"123456","Age":24,"Country":"China"},

{"ID":1002,"Username":"Mary","Password":"654321","Age":25,"Country":"USA"},

{"ID":1003,"Username":"Jack","Password":"123567","Age":22,"Country":"English"}]

withopen("test3.csv","w")asfp:

fp_csv = csv.DictWriter(fp, header)

fp_csv.writeheader()

fp_csv.writerows(rows)

Python读取 csv 文件需要用到reader对象,然后同样是按行读取每一条记录。要读取特定字段信息,python可以利用索引、命名元祖、字典这三种方式访问。

importcsv

fromcollectionsimportnamedtuple

withopen("test3.csv","r")asfp:

# 遍历记录

fp_csv = csv.reader(fp)

forrowinfp_csv:

printrow

# 索引

fp_csv = csv.reader(fp)

forrowinfp_csv:

printrow[]

# 命名元组

fp_csv = csv.reader(fp)

header = next(fp_csv)

Row = namedtuple("Row", header)

forrinfp_csv:

row = Row(*r)

printrow.Username, row.Password

printrow

# 字典

fp_csv = csv.DictReader(fp)

forrowinfp_csv:

printrow.get("Username")

  • 发表于:
  • 原文链接https://kuaibao.qq.com/s/20190114G0X0JC00?refer=cp_1026
  • 腾讯「腾讯云开发者社区」是腾讯内容开放平台帐号(企鹅号)传播渠道之一,根据《腾讯内容开放平台服务协议》转载发布内容。
  • 如有侵权,请联系 cloudcommunity@tencent.com 删除。

同媒体快讯

扫码关注腾讯云开发者

领取腾讯云代金券