blocks|key|52969|text|我不确定您所说的分配的缓冲区/内存长度是什么意思，但是如果您想要存储在BytesIO对象中的用户数据的长度，您可以这样做|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|52970|>>>+bio+=+io.BytesIO()
>>>+bio.getbuffer().nbytes
0
>>>+bio.write(b'here+is+some+data')
17
>>>+bio.getbuffer().nbytes
17|code-block|syntax|javascript|52971|但这似乎等同于您当前使用的len(buf.getvalue())。|52972|可以使用sys.getsizeof()找到BytesIO对象的实际大小|52973|>>>+bio+=+io.BytesIO()
>>>+sys.getsizeof(bio)
104|52974|或者你可以让人讨厌，直接调用__sizeof__()+(类似于sys.getsizeof()，但是没有垃圾收集器的开销)：|52975|>>>+bio+=+io.BytesIO()
>>>+bio.__sizeof__()
72|52976|BytesIO的内存是根据需要分配的，并且确实会发生一些缓冲：|52977|>>>+bio+=+io.BytesIO()
>>>+for+i+in+range(20):
...+++++_=bio.write(b'a')
...+++++print(bio.getbuffer().nbytes,+sys.getsizeof(bio),+bio.__sizeof__())
...
1+106+74
2+106+74
3+108+76
4+108+76
5+110+78
6+110+78
7+112+80
8+112+80
9+120+88
10+120+88
11+120+88
12+120+88
13+120+88
14+120+88
15+120+88
16+120+88
17+129+97
18+129+97
19+129+97
20+129+97|52978|entityMap|0|LINK|mutability|MUTABLE|url|https://docs.python.org/3/library/sys.html#sys.getsizeof^0|Z|7|0|0|D|J|0|4|F|4|F|0|0|0|E|C|V|F|0|0|0|7|0|0^^$0|@$1|2|3|4|5|6|7|16|8|@$9|17|A|18|B|C]]|D|@]|E|$]]|$1|F|3|G|5|H|7|19|8|@]|D|@]|E|$I|J]]|$1|K|3|L|5|6|7|1A|8|@$9|1B|A|1C|B|C]]|D|@]|E|$]]|$1|M|3|N|5|6|7|1D|8|@$9|1E|A|1F|B|C]]|D|@$9|1G|A|1H|1|1I]]|E|$]]|$1|O|3|P|5|H|7|1J|8|@]|D|@]|E|$I|J]]|$1|Q|3|R|5|6|7|1K|8|@$9|1L|A|1M|B|C]|$9|1N|A|1O|B|C]]|D|@]|E|$]]|$1|S|3|T|5|H|7|1P|8|@]|D|@]|E|$I|J]]|$1|U|3|V|5|6|7|1Q|8|@$9|1R|A|1S|B|C]]|D|@]|E|$]]|$1|W|3|X|5|H|7|1T|8|@]|D|@]|E|$I|J]]|$1|Y|3|-4|5|6|7|1U|8|@]|D|@]|E|$]]]|Z|$10|$5|11|12|13|E|$14|15]]]]

I am not sure what you mean by allocated buffer/memory length, but if you want the length of the user data stored in the <code>BytesIO</code> object you can do

<pre><code>&gt;&gt;&gt; bio = io.BytesIO()
&gt;&gt;&gt; bio.getbuffer().nbytes
0
&gt;&gt;&gt; bio.write(b'here is some data')
17
&gt;&gt;&gt; bio.getbuffer().nbytes
17
</code></pre>

But this seems equivalent to the <code>len(buf.getvalue())</code> that you are currently using.

The actual size of the BytesIO object can be found using <a href="https://docs.python.org/3/library/sys.html#sys.getsizeof" rel="noreferrer"><code>sys.getsizeof()</code></a>:

<pre><code>&gt;&gt;&gt; bio = io.BytesIO()
&gt;&gt;&gt; sys.getsizeof(bio)
104
</code></pre>

Or you could be nasty and call <code>__sizeof__()</code> directly (which is like <code>sys.getsizeof()</code> but without garbage collector overhead applicable to the object):

<pre><code>&gt;&gt;&gt; bio = io.BytesIO()
&gt;&gt;&gt; bio.__sizeof__()
72
</code></pre>

Memory for <code>BytesIO</code> is allocated as required, and some buffering does take place:

<pre><code>&gt;&gt;&gt; bio = io.BytesIO()
&gt;&gt;&gt; for i in range(20):
... _=bio.write(b'a')
... print(bio.getbuffer().nbytes, sys.getsizeof(bio), bio.__sizeof__())
...
1 106 74
2 106 74
3 108 76
4 108 76
5 110 78
6 110 78
7 112 80
8 112 80
9 120 88
10 120 88
11 120 88
12 120 88
13 120 88
14 120 88
15 120 88
16 120 88
17 129 97
18 129 97
19 129 97
20 129 97
</code></pre>

blocks|key|53009|text|io.BytesIO()返回一个具有tell()函数的标准文件对象。它报告当前的描述符位置，并且不复制整个缓冲区来计算总大小作为bio.getbuffer().nbytes的len(bio.getvalue())。这是一种非常快速和简单的方法，可以在buffer对象中获得已用内存的确切大小。|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|53010|但是，如果预设缓冲区，tell()将指向缓冲区的开头并返回0，但缓冲区大小不为零。在这种情况下，您可以将指针移动到buffer+chank的末尾，这将报告总的缓冲区大小，而无需将整个缓冲区复制到内存的另一个seek(0,2)中。|53011|我最近更新了一个示例代码和一个更详细的答案here|53012|entityMap|0|LINK|mutability|MUTABLE|url|https://stackoverflow.com/a/54030779/1887559^0|0|C|I|6|1S|M|2F|J|0|B|6|2V|9|0|L|4|0|0^^$0|@$1|2|3|4|5|6|7|R|8|@$9|S|A|T|B|C]|$9|U|A|V|B|C]|$9|W|A|X|B|C]|$9|Y|A|Z|B|C]]|D|@]|E|$]]|$1|F|3|G|5|6|7|10|8|@$9|11|A|12|B|C]|$9|13|A|14|B|C]]|D|@]|E|$]]|$1|H|3|I|5|6|7|15|8|@]|D|@$9|16|A|17|1|18]]|E|$]]|$1|J|3|-4|5|6|7|19|8|@]|D|@]|E|$]]]|K|$L|$5|M|N|O|E|$P|Q]]]]

<code>io.BytesIO()</code> returns a standard file object which has function <code>tell()</code>. It reports the current descriptor position and does not copy the whole buffer out to compute total size as <code>len(bio.getvalue())</code> of <code>bio.getbuffer().nbytes</code>. It is a very fast and simple method to get the exact size of used memory in the buffer object.
However, if you preset your buffer, <code>tell()</code> will point at the beginning of the buffer and return 0, but the buffer size is not zero. In this case, you can move the pointer to the end of the buffer <code>seek(0,2)</code>, which will report the total buffer size without copying the whole buffer into another chank of the memory.
I posted and recently updated an example code and a more detailed answer <a href="https://stackoverflow.com/a/54030779/1887559">here</a>

blocks|key|4559633|text|还可以通过在tracemalloc.get_traced_memory()中包装内存事件，使用tracemalloc间接获取有关对象大小的信息|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|4559634|请注意，程序的活动线程(如果有的话)和副作用将影响输出，但如果采样较多，它也可能更能代表实际内存成本，如下所示。|4559635|>>>+import+tracemalloc
>>>+from+io+import+BytesIO
>>>+tracemalloc.start()
>>>
>>>+memory_traces+=+[]
>>>
>>>+with+BytesIO()+as+bytes_fh:
...+++++#+returns+(current+memory+usage,+peak+memory+usage)
++++++++#+..but+only+since+calling+.start()
...+++++memory_traces.append(tracemalloc.get_traced_memory())
...+++++bytes_fh.write(b'a'+*+(1024**2))++#+create+1MB+of+'a'
...+++++memory_traces.append(tracemalloc.get_traced_memory())
...
1048576
>>>+print("used_memory+=+{}b".format(memory_traces[1][0]+-+memory_traces[0][0]))
used_memory+=+1048870b
>>>+1048870+-+1024**2++#+show+small+overhead
294++|code-block|syntax|javascript|4559636|entityMap^0|6|V|1B|B|0|0|0^^$0|@$1|2|3|4|5|6|7|O|8|@$9|P|A|Q|B|C]|$9|R|A|S|B|C]]|D|@]|E|$]]|$1|F|3|G|5|6|7|T|8|@]|D|@]|E|$]]|$1|H|3|I|5|J|7|U|8|@]|D|@]|E|$K|L]]|$1|M|3|-4|5|6|7|V|8|@]|D|@]|E|$]]]|N|$]]

You can also use <code>tracemalloc</code> to get indirect information about the size of objects, by wrapping memory events in <code>tracemalloc.get_traced_memory()</code>

Do note that active threads (if any) and side effects of your program will affect the output, but it may also be more representative of the real memory cost if many samples are taken, as shown below.

<pre class="lang-py prettyprint-override"><code>&gt;&gt;&gt; import tracemalloc
&gt;&gt;&gt; from io import BytesIO
&gt;&gt;&gt; tracemalloc.start()
&gt;&gt;&gt;
&gt;&gt;&gt; memory_traces = []
&gt;&gt;&gt;
&gt;&gt;&gt; with BytesIO() as bytes_fh:
... # returns (current memory usage, peak memory usage)
 # ..but only since calling .start()
... memory_traces.append(tracemalloc.get_traced_memory())
... bytes_fh.write(b'a' * (1024**2)) # create 1MB of 'a'
... memory_traces.append(tracemalloc.get_traced_memory())
...
1048576
&gt;&gt;&gt; print("used_memory = {}b".format(memory_traces[1][0] - memory_traces[0][0]))
used_memory = 1048870b
&gt;&gt;&gt; 1048870 - 1024**2 # show small overhead
294 
</code></pre>

This is the code i am using to test the memory allocation 

<pre><code>import pycurl
import io


url = "http://www.stackoverflow.com"
buf = io.BytesIO()


print(len(buf.getvalue())) #here i am getting 0 as length


c = pycurl.Curl()
c.setopt(c.URL, url)
c.setopt(c.CONNECTTIMEOUT, 10)
c.setopt(c.TIMEOUT, 10)
c.setopt(c.ENCODING, 'gzip')
c.setopt(c.FOLLOWLOCATION, True)
c.setopt(c.IPRESOLVE, c.IPRESOLVE_V4)
c.setopt(c.USERAGENT, 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:8.0) Gecko/20100101 Firefox/8.0')
c.setopt(c.WRITEFUNCTION, buf.write)
c.perform()
c.close()

print(len(buf.getvalue())) #here length of the dowloaded file


print(buf.getvalue())
buf.close()
</code></pre>

How to get the allocated buffer/memory length by BytesIO ?
what am i doing wrong here ? python doesn't allocate fixed buffer length ?

python how to get BytesIO allocated memory length?

Windows

这是我用来测试内存分配的代码import pycurlimport iourl = "http://www.stackoverflow.com"buf = io.BytesIO()print(len(buf.getvalue()))   #here i am getting 0 as lengthc = pycurl....

问python如何获取BytesIO分配的内存长度？
EN

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问python如何获取BytesIO分配的内存长度？EN

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问python如何获取BytesIO分配的内存长度？
EN