blocks|key|3274796|text|如果你没有任何的互操作性要求(例如，你只需要在Python中使用数据)，二进制格式就可以了，那就用cPickle吧，它能让你真正快速地序列化Python对象。|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|3274797|如果您希望实现互操作性，或者希望使用文本格式来存储数据，请使用JSON+(或其他一些适当的格式，具体取决于您的约束条件)。|3274798|entityMap|0|LINK|mutability|MUTABLE|url|https://docs.python.org/2/library/pickle.html^0|1D|7|0|0|0^^$0|@$1|2|3|4|5|6|7|N|8|@]|9|@$A|O|B|P|1|Q]]|C|$]]|$1|D|3|E|5|6|7|R|8|@]|9|@]|C|$]]|$1|F|3|-4|5|6|7|S|8|@]|9|@]|C|$]]]|G|$H|$5|I|J|K|C|$L|M]]]]

If you do not have any interoperability requirements (e.g. you are just going to use the data with Python) and a binary format is fine, go with <a href="https://docs.python.org/2/library/pickle.html" rel="noreferrer">cPickle</a> which gives you really fast Python object serialization.

If you want interoperability or you want a text format to store your data, go with JSON (or some other appropriate format depending on your constraints).

blocks|key|316277|text|对于我的序列化，我更喜欢JSON而不是pickle。Unpickling可以运行任意代码，并且使用pickle在程序之间传输数据或在会话之间存储数据是一个安全漏洞。JSON没有引入安全漏洞，并且是标准化的，因此如果您需要的话，可以使用不同语言的程序访问数据。|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|316278|entityMap^0|1D|6|0^^$0|@$1|2|3|4|5|6|7|H|8|@$9|I|A|J|B|C]]|D|@]|E|$]]|$1|F|3|-4|5|6|7|K|8|@]|D|@]|E|$]]]|G|$]]

I prefer JSON over pickle for my serialization. Unpickling can run arbitrary code, and using <code>pickle</code> to transfer data between programs or store data between sessions is a security hole. JSON does not introduce a security hole and is standardized, so the data can be accessed by programs in different languages if you ever need to.

blocks|key|3274879|text|您可能还会发现这很有趣，下面是一些可供比较的图表：http://kovshenin.com/archives/pickle-vs-json-which-is-faster/|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|3274880|entityMap|0|LINK|mutability|MUTABLE|url|http://kovshenin.com/archives/pickle-vs-json-which-is-faster/^0|P|1P|0|0^^$0|@$1|2|3|4|5|6|7|L|8|@]|9|@$A|M|B|N|1|O]]|C|$]]|$1|D|3|-4|5|6|7|P|8|@]|9|@]|C|$]]]|E|$F|$5|G|H|I|C|$J|K]]]]

You might also find this interesting, with some charts to compare: <a href="http://kovshenin.com/archives/pickle-vs-json-which-is-faster/" rel="noreferrer">http://kovshenin.com/archives/pickle-vs-json-which-is-faster/</a>

blocks|key|3061832|text|如果您主要关心速度和空间，请使用cPickle，因为cPickle比JSON更快。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|3061833|如果您更关心互操作性、安全性和/或人类可读性，那么可以使用JSON。|3061834|3061835|其他答案中引用的测试结果记录在2010年，2016年使用cPickle+protocol+2更新的测试结果显示：|offset|length|3061836|3061837|cPickle快3.8倍loading|unordered-list-item|style|BOLD|3061838|cPickle快1.5倍reading|3061839|cPickle编码稍小|3061840|3061841|使用this+gist自己重现这一点，它基于其他答案中引用的Konstantin's+benchmark，但使用带有协议2的cPickle而不是pickle，并使用json而不是simplejson+(因为json比simplejson更快)，例如|3061842|wget+https://gist.github.com/jdimatteo/af317ef24ccf1b3fa91f4399902bb534/raw/03e8dbab11b5605bc572bc117c8ac34cfa959a70/pickle_vs_json.py
python+pickle_vs_json.py|code-block|syntax|javascript|3061843|在2015+Xeon处理器上运行python+2.7的结果：|3061844|Dir+Entries+Method++Time++++Length

dump++++10++JSON++++0.017+++1484510
load++++10++JSON++++0.375+++-
dump++++10++Pickle++0.011+++1428790
load++++10++Pickle++0.098+++-
dump++++20++JSON++++0.036+++2969020
load++++20++JSON++++1.498+++-
dump++++20++Pickle++0.022+++2857580
load++++20++Pickle++0.394+++-
dump++++50++JSON++++0.079+++7422550
load++++50++JSON++++9.485+++-
dump++++50++Pickle++0.055+++7143950
load++++50++Pickle++2.518+++-
dump++++100+JSON++++0.165+++14845100
load++++100+JSON++++37.730++-
dump++++100+Pickle++0.107+++14287900
load++++100+Pickle++9.907+++-|3061845|Python+3.4+with+pickle+protocol+3+is+even+faster.|3061846|entityMap|0|LINK|mutability|MUTABLE|url|https://docs.python.org/2/library/pickle.html#data-stream-format|1|https://gist.github.com/jdimatteo/af317ef24ccf1b3fa91f4399902bb534|2|https://konstantin.blog/2010/pickle-vs-json-which-is-faster/|3|https://stackoverflow.com/a/26860404/1007353^0|0|0|0|10|A|0|0|0|0|J|0|0|J|0|0|0|2V|F|2|9|1|U|M|2|0|0|0|0|0|1D|3|0^^$0|@$1|2|3|4|5|6|7|1K|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|1L|8|@]|9|@]|A|$]]|$1|D|3|-4|5|6|7|1M|8|@]|9|@]|A|$]]|$1|E|3|F|5|6|7|1N|8|@]|9|@$G|1O|H|1P|1|1Q]]|A|$]]|$1|I|3|-4|5|6|7|1R|8|@]|9|@]|A|$]]|$1|J|3|K|5|L|7|1S|8|@$G|1T|H|1U|M|N]]|9|@]|A|$]]|$1|O|3|P|5|L|7|1V|8|@$G|1W|H|1X|M|N]]|9|@]|A|$]]|$1|Q|3|R|5|L|7|1Y|8|@]|9|@]|A|$]]|$1|S|3|-4|5|6|7|1Z|8|@]|9|@]|A|$]]|$1|T|3|U|5|6|7|20|8|@$G|21|H|22|M|N]]|9|@$G|23|H|24|1|25]|$G|26|H|27|1|28]]|A|$]]|$1|V|3|W|5|X|7|29|8|@]|9|@]|A|$Y|Z]]|$1|10|3|11|5|6|7|2A|8|@]|9|@]|A|$]]|$1|12|3|13|5|X|7|2B|8|@]|9|@]|A|$Y|Z]]|$1|14|3|15|5|6|7|2C|8|@]|9|@$G|2D|H|2E|1|2F]]|A|$]]|$1|16|3|-4|5|6|7|2G|8|@]|9|@]|A|$]]]|17|$18|$5|19|1A|1B|A|$1C|1D]]|1E|$5|19|1A|1B|A|$1C|1F]]|1G|$5|19|1A|1B|A|$1C|1H]]|1I|$5|19|1A|1B|A|$1C|1J]]]]

If you are primarily concerned with speed and space, use cPickle because cPickle is faster than JSON.

If you are more concerned with interoperability, security, and/or human readability, then use JSON.

<hr>

The tests results referenced in other answers were recorded in 2010, and the updated tests in 2016 with cPickle <a href="https://docs.python.org/2/library/pickle.html#data-stream-format" rel="noreferrer" title="protocol 2">protocol 2</a> show:

<ul>
<li>cPickle 3.8x faster loading</li>
<li>cPickle 1.5x faster reading</li>
<li>cPickle slightly smaller encoding</li>
</ul>

Reproduce this yourself with <a href="https://gist.github.com/jdimatteo/af317ef24ccf1b3fa91f4399902bb534" rel="noreferrer" title="this gist">this gist</a>, which is based on the <a href="https://konstantin.blog/2010/pickle-vs-json-which-is-faster/" rel="noreferrer">Konstantin's benchmark</a> referenced in other answers, but using cPickle with protocol 2 instead of pickle, and using json instead of simplejson (since json is faster than simplejson), e.g.

<pre><code>wget https://gist.github.com/jdimatteo/af317ef24ccf1b3fa91f4399902bb534/raw/03e8dbab11b5605bc572bc117c8ac34cfa959a70/pickle_vs_json.py
python pickle_vs_json.py
</code></pre>

Results with python 2.7 on a decent 2015 Xeon processor:

<pre><code>Dir Entries Method Time Length

dump 10 JSON 0.017 1484510
load 10 JSON 0.375 -
dump 10 Pickle 0.011 1428790
load 10 Pickle 0.098 -
dump 20 JSON 0.036 2969020
load 20 JSON 1.498 -
dump 20 Pickle 0.022 2857580
load 20 Pickle 0.394 -
dump 50 JSON 0.079 7422550
load 50 JSON 9.485 -
dump 50 Pickle 0.055 7143950
load 50 Pickle 2.518 -
dump 100 JSON 0.165 14845100
load 100 JSON 37.730 -
dump 100 Pickle 0.107 14287900
load 100 Pickle 9.907 -
</code></pre>

<a href="https://stackoverflow.com/a/26860404/1007353">Python 3.4 with pickle protocol 3 is even faster.</a>

blocks|key|319207|text|JSON还是pickle？JSON和pickle怎么样！|type|unstyled|depth|inlineStyleRanges|entityRanges|data|319208|您可以使用jsonpickle。它易于使用，并且磁盘上的文件是可读的，因为它是JSON。|offset|length|style|CODE|319209|请参阅jsonpickle+Documentation|319210|entityMap|0|LINK|mutability|MUTABLE|url|https://jsonpickle.readthedocs.io/en/latest/^0|0|5|A|0|3|O|0|0^^$0|@$1|2|3|4|5|6|7|R|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|S|8|@$D|T|E|U|F|G]]|9|@]|A|$]]|$1|H|3|I|5|6|7|V|8|@]|9|@$D|W|E|X|1|Y]]|A|$]]|$1|J|3|-4|5|6|7|Z|8|@]|9|@]|A|$]]]|K|$L|$5|M|N|O|A|$P|Q]]]]

JSON or pickle? How about JSON and pickle!
You can use <code>jsonpickle</code>. It easy to use and the file on disk is readable because it's JSON.
See <a href="https://jsonpickle.readthedocs.io/en/latest/" rel="nofollow noreferrer">jsonpickle Documentation</a>

blocks|key|3274949|text|我尝试了几种方法，发现使用cPickle并将转储方法的协议参数设置为：cPickle.dumps(obj,+protocol=cPickle.HIGHEST_PROTOCOL)是最快的转储方法。|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|3274950|import+msgpack
import+json
import+pickle
import+timeit
import+cPickle
import+numpy+as+np

num_tests+=+10

obj+=+np.random.normal(0.5,+1,+[240,+320,+3])

command+=+'pickle.dumps(obj)'
setup+=+'from+__main__+import+pickle,+obj'
result+=+timeit.timeit(command,+setup=setup,+number=num_tests)
print("pickle:++%25f+seconds"+%25+result)

command+=+'cPickle.dumps(obj)'
setup+=+'from+__main__+import+cPickle,+obj'
result+=+timeit.timeit(command,+setup=setup,+number=num_tests)
print("cPickle:+++%25f+seconds"+%25+result)


command+=+'cPickle.dumps(obj,+protocol=cPickle.HIGHEST_PROTOCOL)'
setup+=+'from+__main__+import+cPickle,+obj'
result+=+timeit.timeit(command,+setup=setup,+number=num_tests)
print("cPickle+highest:+++%25f+seconds"+%25+result)

command+=+'json.dumps(obj.tolist())'
setup+=+'from+__main__+import+json,+obj'
result+=+timeit.timeit(command,+setup=setup,+number=num_tests)
print("json:+++%25f+seconds"+%25+result)


command+=+'msgpack.packb(obj.tolist())'
setup+=+'from+__main__+import+msgpack,+obj'
result+=+timeit.timeit(command,+setup=setup,+number=num_tests)
print("msgpack:+++%25f+seconds"+%25+result)|code-block|syntax|javascript|3274951|输出：|3274952|pickle+++++++++:+++0.847938+seconds
cPickle++++++++:+++0.810384+seconds
cPickle+highest:+++0.004283+seconds
json+++++++++++:+++1.769215+seconds
msgpack++++++++:+++0.270886+seconds|3274953|entityMap^0|Z|1H|0|0|0|0^^$0|@$1|2|3|4|5|6|7|Q|8|@$9|R|A|S|B|C]]|D|@]|E|$]]|$1|F|3|G|5|H|7|T|8|@]|D|@]|E|$I|J]]|$1|K|3|L|5|6|7|U|8|@]|D|@]|E|$]]|$1|M|3|N|5|H|7|V|8|@]|D|@]|E|$I|J]]|$1|O|3|-4|5|6|7|W|8|@]|D|@]|E|$]]]|P|$]]

I have tried several methods and found out that using cPickle with setting the protocol argument of the dumps method as: <code>cPickle.dumps(obj, protocol=cPickle.HIGHEST_PROTOCOL)</code> is the fastest dump method.

<pre><code>import msgpack
import json
import pickle
import timeit
import cPickle
import numpy as np

num_tests = 10

obj = np.random.normal(0.5, 1, [240, 320, 3])

command = 'pickle.dumps(obj)'
setup = 'from __main__ import pickle, obj'
result = timeit.timeit(command, setup=setup, number=num_tests)
print("pickle: %f seconds" % result)

command = 'cPickle.dumps(obj)'
setup = 'from __main__ import cPickle, obj'
result = timeit.timeit(command, setup=setup, number=num_tests)
print("cPickle: %f seconds" % result)


command = 'cPickle.dumps(obj, protocol=cPickle.HIGHEST_PROTOCOL)'
setup = 'from __main__ import cPickle, obj'
result = timeit.timeit(command, setup=setup, number=num_tests)
print("cPickle highest: %f seconds" % result)

command = 'json.dumps(obj.tolist())'
setup = 'from __main__ import json, obj'
result = timeit.timeit(command, setup=setup, number=num_tests)
print("json: %f seconds" % result)


command = 'msgpack.packb(obj.tolist())'
setup = 'from __main__ import msgpack, obj'
result = timeit.timeit(command, setup=setup, number=num_tests)
print("msgpack: %f seconds" % result)
</code></pre>

Output:

<pre><code>pickle : 0.847938 seconds
cPickle : 0.810384 seconds
cPickle highest: 0.004283 seconds
json : 1.769215 seconds
msgpack : 0.270886 seconds
</code></pre>

blocks|key|317320|text|就我个人而言，我通常更喜欢JSON，因为数据是human-readable.的。当然，如果你需要序列化一些JSON不能接受的东西，那就使用pickle。|type|unstyled|depth|inlineStyleRanges|offset|length|style|BOLD|entityRanges|data|317321|但是对于大多数数据存储来说，您不需要序列化任何奇怪的东西，而且JSON要容易得多，并且总是允许您在文本编辑器中将其弹出并自己检出数据。|317322|速度很好，但对于大多数数据集来说，差异可以忽略不计；Python通常不会太快。|317323|entityMap^0|N|E|0|0|0^^$0|@$1|2|3|4|5|6|7|L|8|@$9|M|A|N|B|C]]|D|@]|E|$]]|$1|F|3|G|5|6|7|O|8|@]|D|@]|E|$]]|$1|H|3|I|5|6|7|P|8|@]|D|@]|E|$]]|$1|J|3|-4|5|6|7|Q|8|@]|D|@]|E|$]]]|K|$]]

Personally, I generally prefer JSON because the data is human-readable. Definitely, if you need to serialize something that JSON won't take, than use pickle.

But for most data storage, you won't need to serialize anything weird and JSON is much easier and always allows you to pop it open in a text editor and check out the data yourself.

The speed is nice, but for most datasets the difference is negligible; Python generally isn't too fast anyways.

I need to save to disk a little <code>dict</code> object whose keys are of the type <code>str</code> and values are <code>int</code>s and then recover it. Something like this:

<pre><code>{'juanjo': 2, 'pedro':99, 'other': 333}
</code></pre>

What is the best option and why? Serialize it with <code>pickle</code> or with <code>simplejson</code>?

I am using Python 2.6.

Pickle or json?

数据存储

Python

JSON

我需要将一个小的dict对象保存到磁盘上，该对象的键类型为str，值为int的，然后将其恢复为。如下所示：{'juanjo': 2, 'pedro':99, 'other': 333}最好的选择是什么?为什么？使用pickle或simplejson序列化它我使用的是Python 2.6。

问要泡菜还是要json？
EN

回答 7

Stack Overflow用户

Stack Overflow用户

Stack Overflow用户

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问要泡菜还是要json？EN

回答 7

Stack Overflow用户

Stack Overflow用户

Stack Overflow用户

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问要泡菜还是要json？
EN