blocks|key|1175588|text|我在我的一个项目中使用了它。如果应用程序在运行过程中被终止(它做了一个冗长的任务并处理了大量数据)，我需要保存整个数据结构，并在应用程序再次运行后重新加载它。为此，我使用了cPickle，因为速度是至关重要的，而且数据量非常大。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|1175589|entityMap^0|0^^$0|@$1|2|3|4|5|6|7|D|8|@]|9|@]|A|$]]|$1|B|3|-4|5|6|7|E|8|@]|9|@]|A|$]]]|C|$]]

I have used it in one of my projects. If the app was terminated during it's working (it did a lengthy task and processed lots of data), I needed to save the whole data structure and reload it after the app was run again. I used cPickle for this, as speed was a crucial thing and the size of data was really big.

blocks|key|961367|text|我遇到的一些用法：|type|unstyled|depth|inlineStyleRanges|entityRanges|data|961368|1)将程序的状态数据保存到磁盘上，以便在重启时可以在中断的地方继续(持久化)|961369|2)在多核或分布式系统中通过TCP连接发送python数据(编组)|961370|3)在数据库中存储python对象|961371|4)将任意python对象转换为字符串，以便可以将其用作字典键(例如，用于缓存和记忆)。|961372|最后一种方法有一些问题--两个相同的对象可能会产生不同的字符串--甚至两次相同的对象可能会有不同的表示。这是因为pickle可以包括引用计数信息。|961373|强调@lunaryorn的评论-你永远不应该解开来自不可信来源的字符串，因为精心制作的pickle可能会在你的系统上执行任意代码。例如，请参阅https://blog.nelhage.com/2011/03/exploiting-pickle/|offset|length|961374|entityMap|0|LINK|mutability|MUTABLE|url|https://blog.nelhage.com/2011/03/exploiting-pickle/^0|0|0|0|0|0|0|1Z|1F|0|0^^$0|@$1|2|3|4|5|6|7|X|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|Y|8|@]|9|@]|A|$]]|$1|D|3|E|5|6|7|Z|8|@]|9|@]|A|$]]|$1|F|3|G|5|6|7|10|8|@]|9|@]|A|$]]|$1|H|3|I|5|6|7|11|8|@]|9|@]|A|$]]|$1|J|3|K|5|6|7|12|8|@]|9|@]|A|$]]|$1|L|3|M|5|6|7|13|8|@]|9|@$N|14|O|15|1|16]]|A|$]]|$1|P|3|-4|5|6|7|17|8|@]|9|@]|A|$]]]|Q|$R|$5|S|T|U|A|$V|W]]]]

Some uses that I have come across:

1) saving a program's state data to disk so that it can carry on where it left off when restarted (persistence)

2) sending python data over a TCP connection in a multi-core or distributed system (marshalling)

3) storing python objects in a database

4) converting an arbitrary python object to a string so that it can be used as a dictionary key (e.g. for caching &amp; memoization).

There are some issues with the last one - two identical objects can be pickled and result in different strings - or even the same object pickled twice can have different representations. This is because the pickle can include reference count information.

To emphasise @lunaryorn's comment - you should never unpickle a string from an untrusted source, since a carefully crafted pickle could execute arbitrary code on your system. For example see <a href="https://blog.nelhage.com/2011/03/exploiting-pickle/" rel="noreferrer">https://blog.nelhage.com/2011/03/exploiting-pickle/</a>

blocks|key|961451|text|添加一个真实的示例：Sphinx的Python使用pickle缓存已解析的文档和文档之间的交叉引用，以加快文档的后续构建。|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|961452|entityMap|0|LINK|mutability|MUTABLE|url|http://sphinx.pocoo.org/^0|A|6|0|0^^$0|@$1|2|3|4|5|6|7|L|8|@]|9|@$A|M|B|N|1|O]]|C|$]]|$1|D|3|-4|5|6|7|P|8|@]|9|@]|C|$]]]|E|$F|$5|G|H|I|C|$J|K]]]]

To add a real-world example: The <a href="http://sphinx.pocoo.org/" rel="nofollow">Sphinx</a> documentation tool for Python uses pickle to cache parsed documents and cross-references between documents, to speed up subsequent builds of the documentation.

blocks|key|961519|text|酸洗对于分布式和并行计算来说是绝对必要的。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|961520|假设您想使用multiprocessing+(或使用pyina跨集群节点)执行并行map-reduce，那么您需要确保您希望跨并行资源映射的函数将被pickle。如果它不泡菜，你就不能把它发送给另一个进程、计算机等上的其他资源。另请参阅here以获得一个很好的例子。|offset|length|style|CODE|961521|为此，我使用了dill，它可以序列化python中的几乎任何内容。Dill还提供了some+good+tools，可以帮助您理解在代码失败时是什么原因导致酸洗失败。|961522|是的，人们使用picking来保存计算的状态，或者你的ipython会话，或者其他任何东西。|961523|entityMap|0|LINK|mutability|MUTABLE|url|http://pythonhosted.org/pyina|1|http://nbviewer.ipython.org/5241793|2|http://pythonhosted.org/dill|3|https://github.com/uqfoundation/dill/blob/master/dill/detect.py|4|http://ipython.org/^0|0|6|F|Q|5|0|3A|4|1|0|7|4|2|15|F|3|0|R|7|4|0^^$0|@$1|2|3|4|5|6|7|11|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|12|8|@$D|13|E|14|F|G]]|9|@$D|15|E|16|1|17]|$D|18|E|19|1|1A]]|A|$]]|$1|H|3|I|5|6|7|1B|8|@]|9|@$D|1C|E|1D|1|1E]|$D|1F|E|1G|1|1H]]|A|$]]|$1|J|3|K|5|6|7|1I|8|@]|9|@$D|1J|E|1K|1|1L]]|A|$]]|$1|L|3|-4|5|6|7|1M|8|@]|9|@]|A|$]]]|M|$N|$5|O|P|Q|A|$R|S]]|T|$5|O|P|Q|A|$R|U]]|V|$5|O|P|Q|A|$R|W]]|X|$5|O|P|Q|A|$R|Y]]|Z|$5|O|P|Q|A|$R|10]]]]

Pickling is absolutely necessary for distributed and parallel computing.

Say you wanted to do a parallel map-reduce with <code>multiprocessing</code> (or across cluster nodes with <a href="http://pythonhosted.org/pyina" rel="noreferrer">pyina</a>), then you need to make sure the function you want to have mapped across the parallel resources will pickle. If it doesn't pickle, you can't send it to the other resources on another process, computer, etc. Also see <a href="http://nbviewer.ipython.org/5241793" rel="noreferrer">here</a> for a good example.

To do this, I use <a href="http://pythonhosted.org/dill" rel="noreferrer">dill</a>, which can serialize almost anything in python. Dill also has <a href="https://github.com/uqfoundation/dill/blob/master/dill/detect.py" rel="noreferrer">some good tools</a> for helping you understand what is causing your pickling to fail when your code fails.

And, yes, people use picking to save the state of a calculation, or your <a href="http://ipython.org/" rel="noreferrer">ipython</a> session, or whatever.

blocks|key|1175834|text|对于初学者(我就是这样)来说，很难理解为什么在阅读official+documentation时首先要使用pickle。这可能是因为文档暗示您已经知道序列化的全部目的。只有在阅读了序列化的general+description之后，我才理解了这个模块的原因和它的常见用例。宽泛地解释串行化而不考虑特定的编程语言也会有所帮助：https://stackoverflow.com/a/14482962/4383472、What+is+serialization?、https://stackoverflow.com/a/3984483/4383472|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|1175835|entityMap|0|LINK|mutability|MUTABLE|url|https://docs.python.org/3/library/pickle.html|1|http://en.wikipedia.org/wiki/Serialization|2|https://stackoverflow.com/a/14482962/4383472|3|https://stackoverflow.com/questions/633402/what-is-serialization|4|https://stackoverflow.com/a/3984483/4383472^0|P|M|0|2M|J|1|4I|18|2|5R|M|3|6E|17|4|0^^$0|@$1|2|3|4|5|6|7|T|8|@]|9|@$A|U|B|V|1|W]|$A|X|B|Y|1|Z]|$A|10|B|11|1|12]|$A|13|B|14|1|15]|$A|16|B|17|1|18]]|C|$]]|$1|D|3|-4|5|6|7|19|8|@]|9|@]|C|$]]]|E|$F|$5|G|H|I|C|$J|K]]|L|$5|G|H|I|C|$J|M]]|N|$5|G|H|I|C|$J|O]]|P|$5|G|H|I|C|$J|Q]]|R|$5|G|H|I|C|$J|S]]]]

For the beginner (as is the case with me) it's really hard to understand why use pickle in the first place when reading the <a href="https://docs.python.org/3/library/pickle.html" rel="nofollow noreferrer">official documentation</a>. It's maybe because the docs imply that you already know the whole purpose of serialization. Only after reading the <a href="http://en.wikipedia.org/wiki/Serialization" rel="nofollow noreferrer">general description</a> of serialization have I understood the reason for this module and its common use cases. Also broad explanations of serialization disregarding a particular programming language may help:
<a href="https://stackoverflow.com/a/14482962/4383472">https://stackoverflow.com/a/14482962/4383472</a>, <a href="https://stackoverflow.com/questions/633402/what-is-serialization">What is serialization?</a>,
<a href="https://stackoverflow.com/a/3984483/4383472">https://stackoverflow.com/a/3984483/4383472</a>

blocks|key|961678|text|泡菜就像“另存为..”和“打开..”用于您的数据结构和类。假设我想保存我的数据结构，这样它在程序运行之间是持久的。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|961679|保存：|961680|with+open("save.p",+"wb")+as+f:++++
++++pickle.dump(myStuff,+f)++++++++|code-block|syntax|javascript|961681|正在加载：|961682|try:
++++with+open("save.p",+"rb")+as+f:
++++++++myStuff+=+pickle.load(f)
except:
++++myStuff+=+defaultdict(dict)|961683|现在，我不必从头开始构建myStuff，我只需从我停止的地方重新开始。|961684|entityMap^0|0|0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|Q|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|R|8|@]|9|@]|A|$]]|$1|D|3|E|5|F|7|S|8|@]|9|@]|A|$G|H]]|$1|I|3|J|5|6|7|T|8|@]|9|@]|A|$]]|$1|K|3|L|5|F|7|U|8|@]|9|@]|A|$G|H]]|$1|M|3|N|5|6|7|V|8|@]|9|@]|A|$]]|$1|O|3|-4|5|6|7|W|8|@]|9|@]|A|$]]]|P|$]]

Pickle is like "Save As.." and "Open.." for your data structures and classes. Let's say I want to save my data structures so that it is persistent between program runs.

Saving:

<pre><code>with open("save.p", "wb") as f: 
 pickle.dump(myStuff, f) 
</code></pre>

Loading:

<pre><code>try:
 with open("save.p", "rb") as f:
 myStuff = pickle.load(f)
except:
 myStuff = defaultdict(dict)
</code></pre>

Now I don't have to build myStuff from scratch all over again, and I can just pick(le) up from where I left off.

blocks|key|1175987|text|我使用酸洗在网页抓取其中一个网站的时候，我想存储超过8000k的网址，并希望尽快处理它们，所以我使用酸洗，因为它的输出质量非常高。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|1175988|你可以很容易地访问到url，在你停止的地方，甚至作业目录关键字也可以非常快地获取url的详细信息，以便重新开始这个过程。|1175989|entityMap^0|0|0^^$0|@$1|2|3|4|5|6|7|F|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|G|8|@]|9|@]|A|$]]|$1|D|3|-4|5|6|7|H|8|@]|9|@]|A|$]]]|E|$]]

I use pickling during web scraping one of website at that time I want to store more than 8000k urls and want to process them as fast as possible so I use pickling because its output quality is very high.
you can easily reach to url and where you stop even job directory key word also fetch url details very fast for resuming the process.

I've looked at the <a href="http://en.wikipedia.org/wiki/Pickle_%28Python%29" rel="noreferrer">pickle</a> documentation, but I don't understand where pickle is useful.

What are some common use-cases for pickle?

Common use-cases for pickle in Python

我看过文档，但我不明白pickle在哪里有用。泡菜的一些常见用例是什么？

问Python中酸菜的常见用例
EN

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问Python中酸菜的常见用例EN

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问Python中酸菜的常见用例
EN