blocks|key|4453984|text|这不是最干净的解决方案，但它对程序员来说是完全透明的：|type|unstyled|depth|inlineStyleRanges|entityRanges|data|4453985|import+functools
import+weakref

def+memoized_method(*lru_args,+**lru_kwargs):
++++def+decorator(func):
++++++++@functools.wraps(func)
++++++++def+wrapped_func(self,+*args,+**kwargs):
++++++++++++#+We're+storing+the+wrapped+method+inside+the+instance.+If+we+had
++++++++++++#+a+strong+reference+to+self+the+instance+would+never+die.
++++++++++++self_weak+=+weakref.ref(self)
++++++++++++@functools.wraps(func)
++++++++++++@functools.lru_cache(*lru_args,+**lru_kwargs)
++++++++++++def+cached_method(*args,+**kwargs):
++++++++++++++++return+func(self_weak(),+*args,+**kwargs)
++++++++++++setattr(self,+func.__name__,+cached_method)
++++++++++++return+cached_method(*args,+**kwargs)
++++++++return+wrapped_func
++++return+decorator|code-block|syntax|javascript|4453986|它采用与lru_cache完全相同的参数，并且工作方式完全相同。但是，它从不将self传递给lru_cache，而是使用每个实例的lru_cache。|offset|length|style|CODE|4453987|entityMap^0|0|0|4|9|13|4|1A|9|1T|9|0^^$0|@$1|2|3|4|5|6|7|O|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|P|8|@]|9|@]|A|$E|F]]|$1|G|3|H|5|6|7|Q|8|@$I|R|J|S|K|L]|$I|T|J|U|K|L]|$I|V|J|W|K|L]|$I|X|J|Y|K|L]]|9|@]|A|$]]|$1|M|3|-4|5|6|7|Z|8|@]|9|@]|A|$]]]|N|$]]

This is not the cleanest solution, but it's entirely transparent to the programmer:

<pre><code>import functools
import weakref

def memoized_method(*lru_args, **lru_kwargs):
 def decorator(func):
 @functools.wraps(func)
 def wrapped_func(self, *args, **kwargs):
 # We're storing the wrapped method inside the instance. If we had
 # a strong reference to self the instance would never die.
 self_weak = weakref.ref(self)
 @functools.wraps(func)
 @functools.lru_cache(*lru_args, **lru_kwargs)
 def cached_method(*args, **kwargs):
 return func(self_weak(), *args, **kwargs)
 setattr(self, func.__name__, cached_method)
 return cached_method(*args, **kwargs)
 return wrapped_func
 return decorator
</code></pre>

It takes the exact same parameters as <code>lru_cache</code>, and works exactly the same. However it never passes <code>self</code> to <code>lru_cache</code> and instead uses a per-instance <code>lru_cache</code>.

blocks|key|4454008|text|对于这个用例，我将介绍methodtools。|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|4454009|用于安装https://pypi.org/project/methodtools/的pip+install+methodtools|4454010|然后，您的代码将只需将functools替换为methodtools即可。|4454011|from+methodtools+import+lru_cache
class+Foo:
++++@lru_cache(maxsize=16)
++++def+cached_method(self,+x):
++++++++return+x+%2B+5|code-block|syntax|javascript|4454012|当然，gc测试也会返回0。|4454013|entityMap|0|LINK|mutability|MUTABLE|url|https://pypi.org/project/methodtools/^0|B|B|0|16|N|4|11|0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|Y|8|@$9|Z|A|10|B|C]]|D|@]|E|$]]|$1|F|3|G|5|6|7|11|8|@$9|12|A|13|B|C]]|D|@$9|14|A|15|1|16]]|E|$]]|$1|H|3|I|5|6|7|17|8|@]|D|@]|E|$]]|$1|J|3|K|5|L|7|18|8|@]|D|@]|E|$M|N]]|$1|O|3|P|5|6|7|19|8|@]|D|@]|E|$]]|$1|Q|3|-4|5|6|7|1A|8|@]|D|@]|E|$]]]|R|$S|$5|T|U|V|E|$W|X]]]]

I will introduce <code>methodtools</code> for this use case.

<code>pip install methodtools</code> to install <a href="https://pypi.org/project/methodtools/" rel="noreferrer">https://pypi.org/project/methodtools/</a>

Then your code will work just by replacing functools to methodtools.

<pre class="lang-py prettyprint-override"><code>from methodtools import lru_cache
class Foo:
 @lru_cache(maxsize=16)
 def cached_method(self, x):
 return x + 5
</code></pre>

Of course the gc test also returns 0 too.

blocks|key|4454051|text|Python3.8在functools模块中引入了cached_property装饰器。在测试时，它似乎没有保留实例。|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|4454052|如果你不想更新到Python3.8，你可以使用source+code。您只需导入RLock并创建_NOT_FOUND对象。含义：|4454053|from+threading+import+RLock

_NOT_FOUND+=+object()

class+cached_property:
++++#+https://github.com/python/cpython/blob/v3.8.0/Lib/functools.py#L930
++++...|code-block|syntax|javascript|4454054|entityMap|0|LINK|mutability|MUTABLE|url|https://docs.python.org/3/library/functools.html#functools.cached_property|1|https://github.com/python/cpython/blob/v3.8.0/Lib/functools.py#L930^0|A|9|P|F|P|F|0|0|14|5|1C|A|N|B|1|0|0^^$0|@$1|2|3|4|5|6|7|W|8|@$9|X|A|Y|B|C]|$9|Z|A|10|B|C]]|D|@$9|11|A|12|1|13]]|E|$]]|$1|F|3|G|5|6|7|14|8|@$9|15|A|16|B|C]|$9|17|A|18|B|C]]|D|@$9|19|A|1A|1|1B]]|E|$]]|$1|H|3|I|5|J|7|1C|8|@]|D|@]|E|$K|L]]|$1|M|3|-4|5|6|7|1D|8|@]|D|@]|E|$]]]|N|$O|$5|P|Q|R|E|$S|T]]|U|$5|P|Q|R|E|$S|V]]]]

python 3.8 introduced the <a href="https://docs.python.org/3/library/functools.html#functools.cached_property" rel="noreferrer"><code>cached_property</code></a> decorator in the <code>functools</code> module.
when tested its seems to not retain the instances.
If you don't want to update to python 3.8 you can use the <a href="https://github.com/python/cpython/blob/v3.8.0/Lib/functools.py#L930" rel="noreferrer">source code</a>.
All you need is to import <code>RLock</code> and create the <code>_NOT_FOUND</code> object. meaning:
<pre><code>from threading import RLock

_NOT_FOUND = object()

class cached_property:
 # https://github.com/python/cpython/blob/v3.8.0/Lib/functools.py#L930
 ...
</code></pre>

blocks|key|4454100|text|这个问题的一个更简单的解决方案是在构造函数中而不是在类定义中声明缓存：|type|unstyled|depth|inlineStyleRanges|entityRanges|data|4454101|from+functools+import+lru_cache
import+gc

class+BigClass:
++++pass
class+Foo:
++++def+__init__(self):
++++++++self.big+=+BigClass()
++++++++self.cached_method+=+lru_cache(maxsize=16)(self.cached_method)
++++def+cached_method(self,+x):
++++++++return+x+%2B+5

def+fun():
++++foo+=+Foo()
++++print(foo.cached_method(10))
++++print(foo.cached_method(10))+#+use+cache
++++return+'something'
++++
if+__name__+==+'__main__':
++++fun()
++++gc.collect()++#+collect+garbage
++++print(len([obj+for+obj+in+gc.get_objects()+if+isinstance(obj,+Foo)]))++#+is+0|code-block|syntax|javascript|4454102|entityMap^0|0|0^^$0|@$1|2|3|4|5|6|7|I|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|J|8|@]|9|@]|A|$E|F]]|$1|G|3|-4|5|6|7|K|8|@]|9|@]|A|$]]]|H|$]]

An even simpler solution to this problem is to declare the cache in the constructor and not in the class definition:
<pre><code>from functools import lru_cache
import gc

class BigClass:
 pass
class Foo:
 def __init__(self):
 self.big = BigClass()
 self.cached_method = lru_cache(maxsize=16)(self.cached_method)
 def cached_method(self, x):
 return x + 5

def fun():
 foo = Foo()
 print(foo.cached_method(10))
 print(foo.cached_method(10)) # use cache
 return 'something'
 
if __name__ == '__main__':
 fun()
 gc.collect() # collect garbage
 print(len([obj for obj in gc.get_objects() if isinstance(obj, Foo)])) # is 0
</code></pre>

How can I use functools' lru_cache inside classes without leaking memory?
In the following minimal example the <code>foo</code> instance won't be released although going out of scope and having no referrer (other than the lru_cache).

<pre><code>from functools import lru_cache
class BigClass:
 pass
class Foo:
 def __init__(self):
 self.big = BigClass()
 @lru_cache(maxsize=16)
 def cached_method(self, x):
 return x + 5

def fun():
 foo = Foo()
 print(foo.cached_method(10))
 print(foo.cached_method(10)) # use cache
 return 'something'

fun()
</code></pre>

But <code>foo</code> and hence <code>foo.big</code> (a <code>BigClass</code>) are still alive

<pre><code>import gc; gc.collect() # collect garbage
len([obj for obj in gc.get_objects() if isinstance(obj, Foo)]) # is 1
</code></pre>

That means that Foo/BigClass instances are still residing in memory. Even deleting <code>Foo</code> (del <code>Foo</code>) will not release them.

Why is lru_cache holding on to the instance at all? Doesn't the cache use some hash and not the actual object?

What is the recommended way use lru_caches inside classes?

I know of two workarounds:
<a href="https://stackoverflow.com/a/14946506/2500411">Use per instance caches</a> or <a href="https://stackoverflow.com/a/32655449/2500411">make the cache ignore object</a> (which might lead to wrong results, though)

Python functools lru_cache with class methods: release object

如何在不泄漏内存的情况下在类中使用functools的lru_cache？在下面的最小示例中，尽管超出了作用域并且没有referrer (除了lru_cache之外)，foo实例也不会被释放。from functools import lru_cacheclass BigClass:    passclass Foo:...

问Python functools带有类方法的lru_cache : release对象
EN

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问Python functools带有类方法的lru_cache : release对象EN

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问Python functools带有类方法的lru_cache : release对象
EN