blocks|key|289346|text|您可以更改数组类型而不进行转换，如下所示：|type|unstyled|depth|inlineStyleRanges|entityRanges|data|289347|a.dtype+=+numpy.float32|code-block|syntax|javascript|289348|但首先，您必须将所有整数更改为将被解释为相应的浮点数的值。一种非常慢的方法是使用python的struct模块，如下所示：|offset|length|style|CODE|289349|def+toi(i):
++++return+struct.unpack('i',struct.pack('f',float(i)))[0]|289350|...applied到数组中的每个成员。|289351|但也许更快的方法是使用numpy的ctypeslib工具(我不熟悉)。|289352|编辑-++-+|BOLD|289353|由于ctypeslib似乎不起作用，那么我将继续使用典型的numpy.astype方法进行转换，但在内存限制内继续进行块大小的转换：|289354|a[0:10000]+=+a[0:10000].astype('float32').view('int32')|289355|...then完成后更改数据类型。|289356|下面是一个函数，它可以完成任何兼容数据类型的任务(只适用于具有相同大小项的数据类型)，并通过用户控制块大小来处理任意形状的数组：|289357|import+numpy

def+astype_inplace(a,+dtype,+blocksize=10000):
++++oldtype+=+a.dtype
++++newtype+=+numpy.dtype(dtype)
++++assert+oldtype.itemsize+is+newtype.itemsize
++++for+idx+in+xrange(0,+a.size,+blocksize):
++++++++a.flat[idx:idx+%2B+blocksize]+=+\
++++++++++++a.flat[idx:idx+%2B+blocksize].astype(newtype).view(oldtype)
++++a.dtype+=+newtype

a+=+numpy.random.randint(100,size=100).reshape((10,10))
print+a
astype_inplace(a,+'float32')
print+a|289358|entityMap^0|0|0|1B|6|0|0|0|0|4|3|0|T|C|0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|17|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|18|8|@]|9|@]|A|$E|F]]|$1|G|3|H|5|6|7|19|8|@$I|1A|J|1B|K|L]]|9|@]|A|$]]|$1|M|3|N|5|D|7|1C|8|@]|9|@]|A|$E|F]]|$1|O|3|P|5|6|7|1D|8|@]|9|@]|A|$]]|$1|Q|3|R|5|6|7|1E|8|@]|9|@]|A|$]]|$1|S|3|T|5|6|7|1F|8|@$I|1G|J|1H|K|U]]|9|@]|A|$]]|$1|V|3|W|5|6|7|1I|8|@$I|1J|J|1K|K|L]]|9|@]|A|$]]|$1|X|3|Y|5|D|7|1L|8|@]|9|@]|A|$E|F]]|$1|Z|3|10|5|6|7|1M|8|@]|9|@]|A|$]]|$1|11|3|12|5|6|7|1N|8|@]|9|@]|A|$]]|$1|13|3|14|5|D|7|1O|8|@]|9|@]|A|$E|F]]|$1|15|3|-4|5|6|7|1P|8|@]|9|@]|A|$]]]|16|$]]

You can change the array type without converting like this:

<pre><code>a.dtype = numpy.float32
</code></pre>

but first you have to change all the integers to something that will be interpreted as the corresponding float. A very slow way to do this would be to use python's <code>struct</code> module like this:

<pre><code>def toi(i):
 return struct.unpack('i',struct.pack('f',float(i)))[0]
</code></pre>

...applied to each member of your array.

But perhaps a faster way would be to utilize numpy's ctypeslib tools (which I am unfamiliar with)

- edit -

Since ctypeslib doesnt seem to work, then I would proceed with the conversion with the typical <code>numpy.astype</code> method, but proceed in block sizes that are within your memory limits:

<pre><code>a[0:10000] = a[0:10000].astype('float32').view('int32')
</code></pre>

...then change the dtype when done.

Here is a function that accomplishes the task for any compatible dtypes (only works for dtypes with same-sized items) and handles arbitrarily-shaped arrays with user-control over block size:

<pre><code>import numpy

def astype_inplace(a, dtype, blocksize=10000):
 oldtype = a.dtype
 newtype = numpy.dtype(dtype)
 assert oldtype.itemsize is newtype.itemsize
 for idx in xrange(0, a.size, blocksize):
 a.flat[idx:idx + blocksize] = \
 a.flat[idx:idx + blocksize].astype(newtype).view(oldtype)
 a.dtype = newtype

a = numpy.random.randint(100,size=100).reshape((10,10))
print a
astype_inplace(a, 'float32')
print a
</code></pre>

blocks|key|289373|text|可以创建具有不同数据类型的视图，然后将其在位复制到视图中：|type|unstyled|depth|inlineStyleRanges|entityRanges|data|289374|import+numpy+as+np
x+=+np.arange(10,+dtype='int32')
y+=+x.view('float32')
y[:]+=+x

print(y)|code-block|syntax|javascript|289375|收益率|289376|array([+0.,++1.,++2.,++3.,++4.,++5.,++6.,++7.,++8.,++9.],+dtype=float32)|289377|要显示转换已就地进行，请注意，从x复制到y会更改x|offset|length|style|CODE|289378|print(x)|289379|打印|289380|array([+++++++++0,+1065353216,+1073741824,+1077936128,+1082130432,
+++++++1084227584,+1086324736,+1088421888,+1090519040,+1091567616])|289381|entityMap^0|0|0|0|0|G|1|K|1|O|1|0|0|0|0^^$0|@$1|2|3|4|5|6|7|Y|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|Z|8|@]|9|@]|A|$E|F]]|$1|G|3|H|5|6|7|10|8|@]|9|@]|A|$]]|$1|I|3|J|5|D|7|11|8|@]|9|@]|A|$E|F]]|$1|K|3|L|5|6|7|12|8|@$M|13|N|14|O|P]|$M|15|N|16|O|P]|$M|17|N|18|O|P]]|9|@]|A|$]]|$1|Q|3|R|5|D|7|19|8|@]|9|@]|A|$E|F]]|$1|S|3|T|5|6|7|1A|8|@]|9|@]|A|$]]|$1|U|3|V|5|D|7|1B|8|@]|9|@]|A|$E|F]]|$1|W|3|-4|5|6|7|1C|8|@]|9|@]|A|$]]]|X|$]]

You can make a view with a different dtype, and then copy in-place into the view:

<pre><code>import numpy as np
x = np.arange(10, dtype='int32')
y = x.view('float32')
y[:] = x

print(y)
</code></pre>

yields

<pre><code>array([ 0., 1., 2., 3., 4., 5., 6., 7., 8., 9.], dtype=float32)
</code></pre>

To show the conversion was in-place, note that copying from <code>x</code> to <code>y</code> altered <code>x</code>:

<pre><code>print(x)
</code></pre>

prints

<pre><code>array([ 0, 1065353216, 1073741824, 1077936128, 1082130432,
 1084227584, 1086324736, 1088421888, 1090519040, 1091567616])
</code></pre>

blocks|key|3005889|text|更新:此函数仅在可能的情况下避免复制，因此这不是此问题的正确答案。unutbu's+answer是正确的选择。|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|3005890|3005891|a+=+a.astype(numpy.float32,+copy=False)|code-block|syntax|javascript|3005892|numpy+astype有一个复制标志。为什么我们不能使用它呢？|3005893|entityMap|0|LINK|mutability|MUTABLE|url|https://stackoverflow.com/a/4396247/3798217^0|X|F|0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|T|8|@]|9|@$A|U|B|V|1|W]]|C|$]]|$1|D|3|-4|5|6|7|X|8|@]|9|@]|C|$]]|$1|E|3|F|5|G|7|Y|8|@]|9|@]|C|$H|I]]|$1|J|3|K|5|6|7|Z|8|@]|9|@]|C|$]]|$1|L|3|-4|5|6|7|10|8|@]|9|@]|C|$]]]|M|$N|$5|O|P|Q|C|$R|S]]]]

Update: This function only avoids copy if it can, hence this is not the correct answer for this question. <a href="https://stackoverflow.com/a/4396247/3798217">unutbu's answer</a> is the right one.

<hr>

<pre><code>a = a.astype(numpy.float32, copy=False)
</code></pre>

numpy astype has a copy flag. Why shouldn't we use it ?

blocks|key|289390|text|使用以下命令：|type|unstyled|depth|inlineStyleRanges|entityRanges|data|289391|In+[105]:+a
Out[105]:+
array([[15,+30,+88,+31,+33],
+++++++[53,+38,+54,+47,+56],
+++++++[67,++2,+74,+10,+16],
+++++++[86,+33,+15,+51,+32],
+++++++[32,+47,+76,+15,+81]],+dtype=int32)

In+[106]:+float32(a)
Out[106]:+
array([[+15.,++30.,++88.,++31.,++33.],
+++++++[+53.,++38.,++54.,++47.,++56.],
+++++++[+67.,+++2.,++74.,++10.,++16.],
+++++++[+86.,++33.,++15.,++51.,++32.],
+++++++[+32.,++47.,++76.,++15.,++81.]],+dtype=float32)|code-block|syntax|javascript|289392|entityMap^0|0|0^^$0|@$1|2|3|4|5|6|7|I|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|J|8|@]|9|@]|A|$E|F]]|$1|G|3|-4|5|6|7|K|8|@]|9|@]|A|$]]]|H|$]]

Use this:

<pre><code>In [105]: a
Out[105]: 
array([[15, 30, 88, 31, 33],
 [53, 38, 54, 47, 56],
 [67, 2, 74, 10, 16],
 [86, 33, 15, 51, 32],
 [32, 47, 76, 15, 81]], dtype=int32)

In [106]: float32(a)
Out[106]: 
array([[ 15., 30., 88., 31., 33.],
 [ 53., 38., 54., 47., 56.],
 [ 67., 2., 74., 10., 16.],
 [ 86., 33., 15., 51., 32.],
 [ 32., 47., 76., 15., 81.]], dtype=float32)
</code></pre>

blocks|key|286637|text|a+=+np.subtract(a,+0.,+dtype=np.float32)|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|286638|entityMap^0|0|14|0^^$0|@$1|2|3|4|5|6|7|H|8|@$9|I|A|J|B|C]]|D|@]|E|$]]|$1|F|3|-4|5|6|7|K|8|@]|D|@]|E|$]]]|G|$]]

<code>a = np.subtract(a, 0., dtype=np.float32)</code>

blocks|key|3005913|text|import+numpy+as+np
arr_float+=+np.arange(10,+dtype=np.float32)
arr_int+=+arr_float.view(np.float32)|type|code-block|depth|inlineStyleRanges|entityRanges|data|syntax|javascript|3005914|使用view()和参数'dtype‘就地更改数组。|unstyled|3005915|entityMap^0|0|0^^$0|@$1|2|3|4|5|6|7|I|8|@]|9|@]|A|$B|C]]|$1|D|3|E|5|F|7|J|8|@]|9|@]|A|$]]|$1|G|3|-4|5|F|7|K|8|@]|9|@]|A|$]]]|H|$]]

<pre class="lang-py prettyprint-override"><code>import numpy as np
arr_float = np.arange(10, dtype=np.float32)
arr_int = arr_float.view(np.float32)
</code></pre>

use view() and parameter 'dtype' to change the array in place.

blocks|key|291749|text|读取data所用的时间|type|unstyled|depth|inlineStyleRanges|offset|length|style|BOLD|entityRanges|data|291750|t1=time.time()+;+V=np.load+('udata.npy');t2=time.time()-t1+;+print(+t2+)|code-block|syntax|javascript|291751|95.7923333644867|291752|V.dtype|291753|dtype('>f8')|291754|V.shape|291755|(3072,1024,4096)|291756|**创建新数组**|291757|t1=time.time()+;+V64=np.array(+V,+dtype=np.double);+t2=time.time()-t1+;+print(+t2+)|291758|1291.669689655304|291759|简单的就地数值转换|291760|t1=time.time()+;+V64=np.array(+V,+dtype=np.double);+t2=time.time()-t1+;+print(+t2+)|291761|205.64322113990784|291762|使用astype的|291763|t1=time.time()+;+V+=+V.astype(np.double)+;+t2=time.time()-t1+;+print(+t2+)|291764|400.6731758117676|291765|使用视图的|291766|t1=time.time()+;+x=V.view(np.double);V[:,:,:]=x+;t2=time.time()-t1+;+print(+t2+)|291767|556.5982494354248|291768|请注意，每次我清除变量时。因此，简单地让python处理转换是最有效的。|291769|entityMap^0|9|2|0|0|0|G|0|0|0|C|0|0|0|G|0|0|9|0|0|0|H|0|0|9|0|0|0|I|0|0|8|0|0|0|H|0|0|4|0|0|0|H|0|0|10|0^^$0|@$1|2|3|4|5|6|7|1M|8|@$9|1N|A|1O|B|C]]|D|@]|E|$]]|$1|F|3|G|5|H|7|1P|8|@]|D|@]|E|$I|J]]|$1|K|3|L|5|6|7|1Q|8|@$9|1R|A|1S|B|C]]|D|@]|E|$]]|$1|M|3|N|5|H|7|1T|8|@]|D|@]|E|$I|J]]|$1|O|3|P|5|6|7|1U|8|@$9|1V|A|1W|B|C]]|D|@]|E|$]]|$1|Q|3|R|5|H|7|1X|8|@]|D|@]|E|$I|J]]|$1|S|3|T|5|6|7|1Y|8|@$9|1Z|A|20|B|C]]|D|@]|E|$]]|$1|U|3|V|5|6|7|21|8|@$9|22|A|23|B|C]]|D|@]|E|$]]|$1|W|3|X|5|H|7|24|8|@]|D|@]|E|$I|J]]|$1|Y|3|Z|5|6|7|25|8|@$9|26|A|27|B|C]]|D|@]|E|$]]|$1|10|3|11|5|6|7|28|8|@$9|29|A|2A|B|C]]|D|@]|E|$]]|$1|12|3|13|5|H|7|2B|8|@]|D|@]|E|$I|J]]|$1|14|3|15|5|6|7|2C|8|@$9|2D|A|2E|B|C]]|D|@]|E|$]]|$1|16|3|17|5|6|7|2F|8|@$9|2G|A|2H|B|C]]|D|@]|E|$]]|$1|18|3|19|5|H|7|2I|8|@]|D|@]|E|$I|J]]|$1|1A|3|1B|5|6|7|2J|8|@$9|2K|A|2L|B|C]]|D|@]|E|$]]|$1|1C|3|1D|5|6|7|2M|8|@$9|2N|A|2O|B|C]]|D|@]|E|$]]|$1|1E|3|1F|5|H|7|2P|8|@]|D|@]|E|$I|J]]|$1|1G|3|1H|5|6|7|2Q|8|@$9|2R|A|2S|B|C]]|D|@]|E|$]]|$1|1I|3|1J|5|6|7|2T|8|@$9|2U|A|2V|B|C]]|D|@]|E|$]]|$1|1K|3|-4|5|6|7|2W|8|@]|D|@]|E|$]]]|1L|$]]

Time spent reading data
<pre><code>t1=time.time() ; V=np.load ('udata.npy');t2=time.time()-t1 ; print( t2 )
</code></pre>
95.7923333644867
<pre><code>V.dtype
</code></pre>
dtype('&gt;f8')
<pre><code>V.shape
</code></pre>
(3072, 1024, 4096)
**Creating new array **
<pre><code>t1=time.time() ; V64=np.array( V, dtype=np.double); t2=time.time()-t1 ; print( t2 )
</code></pre>
1291.669689655304
Simple in-place numpy conversion
<pre><code>t1=time.time() ; V64=np.array( V, dtype=np.double); t2=time.time()-t1 ; print( t2 )
</code></pre>
205.64322113990784
Using astype
<pre><code>t1=time.time() ; V = V.astype(np.double) ; t2=time.time()-t1 ; print( t2 )
</code></pre>
400.6731758117676
Using view
<pre><code>t1=time.time() ; x=V.view(np.double);V[:,:,:]=x ;t2=time.time()-t1 ; print( t2 )
</code></pre>
556.5982494354248
Note that each time I cleared the variables. Thus simply let python handle the conversion is the most efficient.

Given a NumPy array of <code>int32</code>, how do I convert it to <code>float32</code> in place? So basically, I would like to do

<pre><code>a = a.astype(numpy.float32)
</code></pre>

without copying the array. It is big.

The reason for doing this is that I have two algorithms for the computation of <code>a</code>. One of them returns an array of <code>int32</code>, the other returns an array of <code>float32</code> (and this is inherent to the two different algorithms). All further computations assume that <code>a</code> is an array of <code>float32</code>.

Currently I do the conversion in a C function called via <code>ctypes</code>. Is there a way to do this in Python?

In-place type conversion of a NumPy array

Python

给定int32的NumPy数组，如何将其转换为float32 in place？所以基本上，我想做的是a = a.astype(numpy.float32)而不复制数组。它很大。这样做的原因是我有两种算法来计算a。其中一个返回int32数组，另一个返回float32数组(这是两种不同算法所固有的)。所有进一步的计算都假...

问NumPy数组的就地类型转换
EN

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问NumPy数组的就地类型转换EN

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问NumPy数组的就地类型转换
EN