blocks|key|3083623|text|NaN不能存储在整数数组中。这是目前已知的熊猫的局限性；我一直在等待NumPy中的nA值的进展(类似于R中的NAs+)，但NumPy似乎至少需要6个月到一年的时间才能获得这些功能：|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|3083624|http://pandas.pydata.org/pandas-docs/stable/gotchas.html#support-for-integer-na|3083625|(此功能是从pandas的0.24版本开始添加的，但请注意，它需要使用扩展数据类型Int64+(大写)，而不是默认的数据类型int64+(小写)：https://pandas.pydata.org/pandas-docs/version/0.24/whatsnew/v0.24.0.html#optional-integer-na-support+)|3083626|entityMap|0|LINK|mutability|MUTABLE|url|1|https://pandas.pydata.org/pandas-docs/version/0.24/whatsnew/v0.24.0.html#optional-integer-na-support^0|0|3|0|0|27|0|0|21|2S|1|0^^$0|@$1|2|3|4|5|6|7|S|8|@$9|T|A|U|B|C]]|D|@]|E|$]]|$1|F|3|G|5|6|7|V|8|@]|D|@$9|W|A|X|1|Y]]|E|$]]|$1|H|3|I|5|6|7|Z|8|@]|D|@$9|10|A|11|1|12]]|E|$]]|$1|J|3|-4|5|6|7|13|8|@]|D|@]|E|$]]]|K|$L|$5|M|N|O|E|$P|G]]|Q|$5|M|N|O|E|$P|R]]]]

<code>NaN</code> can't be stored in an integer array. This is a known limitation of pandas at the moment; I have been waiting for progress to be made with NA values in NumPy (similar to NAs in R), but it will be at least 6 months to a year before NumPy gets these features, it seems:

<a href="http://pandas.pydata.org/pandas-docs/stable/gotchas.html#support-for-integer-na" rel="noreferrer">http://pandas.pydata.org/pandas-docs/stable/gotchas.html#support-for-integer-na</a>

(This feature has been added beginning with version 0.24 of pandas, but note it requires the use of extension dtype Int64 (capitalized), rather than the default dtype int64 (lower case):
<a href="https://pandas.pydata.org/pandas-docs/version/0.24/whatsnew/v0.24.0.html#optional-integer-na-support" rel="noreferrer">https://pandas.pydata.org/pandas-docs/version/0.24/whatsnew/v0.24.0.html#optional-integer-na-support</a>
)

blocks|key|217530|text|如果性能不是主要问题，您可以改为存储字符串。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|217531|df.col+=+df.col.dropna().apply(lambda+x:+str(int(x))+)|code-block|syntax|javascript|217532|然后你就可以随心所欲地使用NaN了。如果您确实想要整数，可以使用-1、0、1234567890或其他一些专用值来表示NaN，具体取决于应用程序。|offset|length|style|CODE|217533|您还可以临时复制列:一个是浮点型的，另一个是实验性的，是整型或字符串。然后在每个合理的位置插入asserts，检查两者是否同步。经过足够的测试后，你就可以放手了。|217534|entityMap^0|0|0|D|3|W|2|Z|1|11|A|1M|3|0|1B|7|0^^$0|@$1|2|3|4|5|6|7|Q|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|R|8|@]|9|@]|A|$E|F]]|$1|G|3|H|5|6|7|S|8|@$I|T|J|U|K|L]|$I|V|J|W|K|L]|$I|X|J|Y|K|L]|$I|Z|J|10|K|L]|$I|11|J|12|K|L]]|9|@]|A|$]]|$1|M|3|N|5|6|7|13|8|@$I|14|J|15|K|L]]|9|@]|A|$]]|$1|O|3|-4|5|6|7|16|8|@]|9|@]|A|$]]]|P|$]]

If performance is not the main issue, you can store strings instead.

<pre><code>df.col = df.col.dropna().apply(lambda x: str(int(x)) )
</code></pre>

Then you can mix then with <code>NaN</code> as much as you want. If you really want to have integers, depending on your application, you can use <code>-1</code>, or <code>0</code>, or <code>1234567890</code>, or some other dedicated value to represent <code>NaN</code>. 

You can also temporarily duplicate the columns: one as you have, with floats; the other one experimental, with ints or strings. Then inserts <code>asserts</code> in every reasonable place checking that the two are in sync. After enough testing you can let go of the floats.

blocks|key|2869859|text|这并不是适用于所有情况的解决方案，但我的(基因组坐标)我求助于使用0作为NaN|type|unstyled|depth|inlineStyleRanges|entityRanges|data|2869860|a3['MapInfo']+=+a3['MapInfo'].fillna(0).astype(int)|code-block|syntax|javascript|2869861|这至少允许使用适当的“原生”列类型，减法、比较等操作按预期工作|2869862|entityMap^0|0|0|0^^$0|@$1|2|3|4|5|6|7|K|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|L|8|@]|9|@]|A|$E|F]]|$1|G|3|H|5|6|7|M|8|@]|9|@]|A|$]]|$1|I|3|-4|5|6|7|N|8|@]|9|@]|A|$]]]|J|$]]

This is not a solution for all cases, but mine (genomic coordinates) I've resorted to using 0 as NaN

<pre><code>a3['MapInfo'] = a3['MapInfo'].fillna(0).astype(int)
</code></pre>

This at least allows for the proper 'native' column type to be used, operations like subtraction, comparison etc work as expected

blocks|key|220908|text|此功能已添加到pandas+(从0.24版开始)：https://pandas.pydata.org/pandas-docs/version/0.24/whatsnew/v0.24.0.html#optional-integer-na-support|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|220909|此时，它需要使用扩展数据类型Int64+(大写)，而不是默认的数据类型int64+(小写)。|220910|entityMap|0|LINK|mutability|MUTABLE|url|https://pandas.pydata.org/pandas-docs/version/0.24/whatsnew/v0.24.0.html#optional-integer-na-support^0|P|2S|0|0|0^^$0|@$1|2|3|4|5|6|7|N|8|@]|9|@$A|O|B|P|1|Q]]|C|$]]|$1|D|3|E|5|6|7|R|8|@]|9|@]|C|$]]|$1|F|3|-4|5|6|7|S|8|@]|9|@]|C|$]]]|G|$H|$5|I|J|K|C|$L|M]]]]

This capability has been added to pandas (beginning with version 0.24):
<a href="https://pandas.pydata.org/pandas-docs/version/0.24/whatsnew/v0.24.0.html#optional-integer-na-support" rel="noreferrer">https://pandas.pydata.org/pandas-docs/version/0.24/whatsnew/v0.24.0.html#optional-integer-na-support</a>

At this point, it requires the use of extension dtype Int64 (capitalized), rather than the default dtype int64 (lowercase).

blocks|key|2869919|text|熊猫v0.24%2B|type|unstyled|depth|inlineStyleRanges|entityRanges|data|2869920|支持整数系列中的NaN的功能将在v0.24以上版本中可用。在v0.24“新增功能”部分有information+on+this，在Nullable+Integer+Data+Type下有更多详细信息。|offset|length|style|CODE|2869921|Pandas+v0.23及更早版本|2869922|通常，最好在可能的情况下使用float系列，即使该系列由于包含了NaN值而从int向上转换为float。这启用了基于NumPy的矢量化计算，否则将处理Python级别的循环。|2869923|文档做了suggest：“一种可能性是使用dtype=object数组。”例如：|2869924|s+=+pd.Series([1,+2,+3,+np.nan])

print(s.astype(object))

0++++++1
1++++++2
2++++++3
3++++NaN
dtype:+object|code-block|syntax|javascript|2869925|出于表面上的原因，例如输出到文件，这可能更可取。|2869926|Pandas+v0.23及更早版本:背景|2869927|NaN+is+considered+a+float。docs+currently+(as+of+v0.23)指定将整数序列向上转换为float的原因|2869928|2869929|在没有从头构建到NumPy中的高性能NA支持的情况下，主要的损失是在整数阵列中表示NAs的能力。|blockquote|2869930|2869931|这种权衡很大程度上是出于内存和性能的原因，也是为了使结果系列仍然是“数字的”。|2869932|2869933|由于包含了NaN，文档还provide+rules进行了向上转换：|2869934|Typeclass+++Promotion+dtype+for+storing+NAs
floating++++no+change
object++++++no+change
integer+++++cast+to+float64
boolean+++++cast+to+object|2869935|entityMap|0|LINK|mutability|MUTABLE|url|http://pandas-docs.github.io/pandas-docs-travis/whatsnew/v0.24.0.html#optional-integer-na-support|1|http://pandas-docs.github.io/pandas-docs-travis/user_guide/integer_na.html#integer-na|2|http://pandas.pydata.org/pandas-docs/stable/gotchas.html#support-for-integer-na|3|https://stackoverflow.com/questions/48558973/why-is-nan-considered-as-a-float|4|5|http://pandas.pydata.org/pandas-docs/stable/user_guide/gotchas.html#na-type-promotions^0|0|8|3|18|J|0|1T|Q|1|0|0|E|5|W|3|12|3|1A|5|0|L|C|4|7|2|0|0|0|0|0|3|K|5|1U|5|0|P|3|Q|S|4|0|0|0|0|0|0|5|3|C|D|5|0|0^^$0|@$1|2|3|4|5|6|7|1R|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|1S|8|@$D|1T|E|1U|F|G]]|9|@$D|1V|E|1W|1|1X]|$D|1Y|E|1Z|1|20]]|A|$]]|$1|H|3|I|5|6|7|21|8|@]|9|@]|A|$]]|$1|J|3|K|5|6|7|22|8|@$D|23|E|24|F|G]|$D|25|E|26|F|G]|$D|27|E|28|F|G]|$D|29|E|2A|F|G]]|9|@]|A|$]]|$1|L|3|M|5|6|7|2B|8|@$D|2C|E|2D|F|G]]|9|@$D|2E|E|2F|1|2G]]|A|$]]|$1|N|3|O|5|P|7|2H|8|@]|9|@]|A|$Q|R]]|$1|S|3|T|5|6|7|2I|8|@]|9|@]|A|$]]|$1|U|3|V|5|6|7|2J|8|@]|9|@]|A|$]]|$1|W|3|X|5|6|7|2K|8|@$D|2L|E|2M|F|G]|$D|2N|E|2O|F|G]|$D|2P|E|2Q|F|G]]|9|@$D|2R|E|2S|1|2T]|$D|2U|E|2V|1|2W]]|A|$]]|$1|Y|3|-4|5|6|7|2X|8|@]|9|@]|A|$]]|$1|Z|3|10|5|11|7|2Y|8|@]|9|@]|A|$]]|$1|12|3|-4|5|6|7|2Z|8|@]|9|@]|A|$]]|$1|13|3|14|5|6|7|30|8|@]|9|@]|A|$]]|$1|15|3|-4|5|6|7|31|8|@]|9|@]|A|$]]|$1|16|3|17|5|6|7|32|8|@$D|33|E|34|F|G]]|9|@$D|35|E|36|1|37]]|A|$]]|$1|18|3|19|5|P|7|38|8|@]|9|@]|A|$Q|R]]|$1|1A|3|-4|5|6|7|39|8|@]|9|@]|A|$]]]|1B|$1C|$5|1D|1E|1F|A|$1G|1H]]|1I|$5|1D|1E|1F|A|$1G|1J]]|1K|$5|1D|1E|1F|A|$1G|1L]]|1M|$5|1D|1E|1F|A|$1G|1N]]|1O|$5|1D|1E|1F|A|$1G|1L]]|1P|$5|1D|1E|1F|A|$1G|1Q]]]]

<h3>Pandas v0.24+</h3>

Functionality to support <code>NaN</code> in integer series will be available in v0.24 upwards. There's <a href="http://pandas-docs.github.io/pandas-docs-travis/whatsnew/v0.24.0.html#optional-integer-na-support" rel="nofollow noreferrer">information on this</a> in the v0.24 "What's New" section, and more details under <a href="http://pandas-docs.github.io/pandas-docs-travis/user_guide/integer_na.html#integer-na" rel="nofollow noreferrer">Nullable Integer Data Type</a>.

<h3>Pandas v0.23 and earlier</h3>

In general, it's best to work with <code>float</code> series where possible, even when the series is upcast from <code>int</code> to <code>float</code> due to inclusion of <code>NaN</code> values. This enables vectorised NumPy-based calculations where, otherwise, Python-level loops would be processed.

The docs do <a href="http://pandas.pydata.org/pandas-docs/stable/gotchas.html#support-for-integer-na" rel="nofollow noreferrer">suggest</a> : "One possibility is to use <code>dtype=object</code> arrays instead." For example:

<pre><code>s = pd.Series([1, 2, 3, np.nan])

print(s.astype(object))

0 1
1 2
2 3
3 NaN
dtype: object
</code></pre>

For cosmetic reasons, e.g. output to a file, this may be preferable.

<h3>Pandas v0.23 and earlier: background</h3>

<a href="https://stackoverflow.com/questions/48558973/why-is-nan-considered-as-a-float"><code>NaN</code> is considered a <code>float</code></a>. The <a href="http://pandas.pydata.org/pandas-docs/stable/gotchas.html#support-for-integer-na" rel="nofollow noreferrer">docs currently (as of v0.23)</a> specify the reason why integer series are upcasted to <code>float</code>:

<blockquote>
 In the absence of high performance NA support being built into NumPy
 from the ground up, the primary casualty is the ability to represent
 NAs in integer arrays.
 
 This trade-off is made largely for memory and performance reasons, and
 also so that the resulting Series continues to be “numeric”.
</blockquote>

The docs also <a href="http://pandas.pydata.org/pandas-docs/stable/user_guide/gotchas.html#na-type-promotions" rel="nofollow noreferrer">provide rules</a> for upcasting due to <code>NaN</code> inclusion:

<pre><code>Typeclass Promotion dtype for storing NAs
floating no change
object no change
integer cast to float64
boolean cast to object
</code></pre>

blocks|key|220970|text|现在这是可能的，因为pandas+v0.24.0|type|unstyled|depth|inlineStyleRanges|entityRanges|data|220971|pandas+0.24.x+release+notes引用："Pandas已经获得了保存缺少值的整数数据类型的能力。|offset|length|style|BOLD|220972|entityMap|0|LINK|mutability|MUTABLE|url|https://pandas.pydata.org/pandas-docs/version/0.24/whatsnew/v0.24.0.html#optional-integer-na-support^0|0|V|R|0|R|0|0^^$0|@$1|2|3|4|5|6|7|P|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|Q|8|@$D|R|E|S|F|G]]|9|@$D|T|E|U|1|V]]|A|$]]|$1|H|3|-4|5|6|7|W|8|@]|9|@]|A|$]]]|I|$J|$5|K|L|M|A|$N|O]]]]

This is now possible, since pandas v 0.24.0

<a href="https://pandas.pydata.org/pandas-docs/version/0.24/whatsnew/v0.24.0.html#optional-integer-na-support" rel="nofollow noreferrer">pandas 0.24.x release notes</a>
Quote: "Pandas has gained the ability to hold integer dtypes with missing values.

blocks|key|3083810|text|我只是想补充一下，以防你试图将一个浮点(1.143)向量转换为整数(1)，而将NA转换为新的'Int64‘dtype将会给你一个错误。为了解决这个问题，你必须舍入数字，然后执行".astype('Int64')“|type|unstyled|depth|inlineStyleRanges|entityRanges|data|3083811|s1+=+pd.Series([1.434,+2.343,+np.nan])
#without+round()+the+next+line+returns+an+error+
s1.astype('Int64')
#cannot+safely+cast+non-equivalent+float64+to+int64
##with+round()+it+works
s1.round().astype('Int64')
0++++++1
1++++++2
2++++NaN
dtype:+Int64|code-block|syntax|javascript|3083812|我的用例是，我有一个浮点数列，我想将其四舍五入为整数，但是当您执行.round()时，数字末尾的'*.0‘仍然是一个’*.0‘，所以您可以通过转换为整数从末尾去掉那个0。|3083813|entityMap^0|0|0|0^^$0|@$1|2|3|4|5|6|7|K|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|L|8|@]|9|@]|A|$E|F]]|$1|G|3|H|5|6|7|M|8|@]|9|@]|A|$]]|$1|I|3|-4|5|6|7|N|8|@]|9|@]|A|$]]]|J|$]]

Just wanted to add that in case you are trying to convert a float (1.143) vector to integer (1) that has NA converting to the new 'Int64' dtype will give you an error. In order to solve this you have to round the numbers and then do ".astype('Int64')"

<pre><code>s1 = pd.Series([1.434, 2.343, np.nan])
#without round() the next line returns an error 
s1.astype('Int64')
#cannot safely cast non-equivalent float64 to int64
##with round() it works
s1.round().astype('Int64')
0 1
1 2
2 NaN
dtype: Int64
</code></pre>

My use case is that I have a float series that I want to round to int, but when you do .round() a '*.0' at the end of the number remains, so you can drop that 0 from the end by converting to int.

blocks|key|2870065|text|如果文本数据中包含空格，则通常为整数的列将转换为浮点型为float64数据类型，因为int64数据类型不能处理空值。如果您正在加载多个文件，这可能会导致模式不一致，其中一些文件带有空格(它将以float64结尾，而其他文件则以int64结尾|type|unstyled|depth|inlineStyleRanges|entityRanges|data|2870066|此代码将尝试将任何number类型的列转换为Int64+(与Int64相反)，因为Int64可以处理nulls|2870067|import+pandas+as+pd
import+numpy+as+np

#show+datatypes+before+transformation
mydf.dtypes

for+c+in+mydf.select_dtypes(np.number).columns:
++++try:
++++++++mydf[c]+=+mydf[c].astype('Int64')
++++++++print('casted+{}+as+Int64'.format(c))
++++except:
++++++++print('could+not+cast+{}+to+Int64'.format(c))

#show+datatypes+after+transformation
mydf.dtypes|code-block|syntax|javascript|2870068|entityMap^0|0|0|0^^$0|@$1|2|3|4|5|6|7|K|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|L|8|@]|9|@]|A|$]]|$1|D|3|E|5|F|7|M|8|@]|9|@]|A|$G|H]]|$1|I|3|-4|5|6|7|N|8|@]|9|@]|A|$]]]|J|$]]

If there are blanks in the text data, columns that would normally be integers will be cast to floats as float64 dtype because int64 dtype cannot handle nulls. This can cause inconsistent schema if you are loading multiple files some with blanks (which will end up as float64 and others without which will end up as int64

This code will attempt to convert any number type columns to Int64 (as opposed to int64) since Int64 can handle nulls

<pre><code>import pandas as pd
import numpy as np

#show datatypes before transformation
mydf.dtypes

for c in mydf.select_dtypes(np.number).columns:
 try:
 mydf[c] = mydf[c].astype('Int64')
 print('casted {} as Int64'.format(c))
 except:
 print('could not cast {} to Int64'.format(c))

#show datatypes after transformation
mydf.dtypes
</code></pre>

blocks|key|2870129|text|Pandas+v1.00+%2B的新功能|type|unstyled|depth|inlineStyleRanges|entityRanges|data|2870130|您不再(也不能)使用numpy.nan。现在您有了pandas.NA。|offset|length|style|CODE|2870131|请阅读：https://pandas.pydata.org/pandas-docs/stable/user_guide/integer_na.html|2870132|2870133|+IntegerArray目前处于实验阶段。它的API或实现可能会在没有警告的情况下发生变化。|blockquote|2870134|2870135|版本1.0.0中的更改:现在使用pandas.NA而不是numpy.nan作为缺失值。|2870136|在处理丢失的数据时，我们看到pandas主要使用NaN来表示丢失的数据。因为NaN是浮点型的，所以这会强制将缺少任何值的整数数组转换为浮点型。在某些情况下，这可能并不重要。但是如果你的整型列是一个标识符，转换成浮点型可能会有问题。有些整数甚至不能表示为浮点数。|2870137|entityMap|0|LINK|mutability|MUTABLE|url|https://pandas.pydata.org/pandas-docs/stable/user_guide/integer_na.html^0|0|A|9|P|9|0|4|1Z|0|0|0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|10|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|11|8|@$D|12|E|13|F|G]|$D|14|E|15|F|G]]|9|@]|A|$]]|$1|H|3|I|5|6|7|16|8|@]|9|@$D|17|E|18|1|19]]|A|$]]|$1|J|3|-4|5|6|7|1A|8|@]|9|@]|A|$]]|$1|K|3|L|5|M|7|1B|8|@]|9|@]|A|$]]|$1|N|3|-4|5|6|7|1C|8|@]|9|@]|A|$]]|$1|O|3|P|5|6|7|1D|8|@]|9|@]|A|$]]|$1|Q|3|R|5|6|7|1E|8|@]|9|@]|A|$]]|$1|S|3|-4|5|6|7|1F|8|@]|9|@]|A|$]]]|T|$U|$5|V|W|X|A|$Y|Z]]]]

<h2>New for Pandas v1.00 +</h2>
You do not (and can not) use <code>numpy.nan</code> any more.
Now you have <code>pandas.NA</code>.
Please read: <a href="https://pandas.pydata.org/pandas-docs/stable/user_guide/integer_na.html" rel="nofollow noreferrer">https://pandas.pydata.org/pandas-docs/stable/user_guide/integer_na.html</a>
<blockquote>
IntegerArray is currently experimental. Its API or implementation may
change without warning.
Changed in version 1.0.0: Now uses pandas.NA as the missing value
rather than numpy.nan.
In Working with missing data, we saw that pandas primarily uses NaN to
represent missing data. Because NaN is a float, this forces an array
of integers with any missing values to become floating point. In some
cases, this may not matter much. But if your integer column is, say,
an identifier, casting to float can be problematic. Some integers
cannot even be represented as floating point numbers.
</blockquote>

Is there a preferred way to keep the data type of a <code>numpy</code> array fixed as <code>int</code> (or <code>int64</code> or whatever), while still having an element inside listed as <code>numpy.NaN</code>?

In particular, I am converting an in-house data structure to a Pandas DataFrame. In our structure, we have integer-type columns that still have NaN's (but the dtype of the column is int). It seems to recast everything as a float if we make this a DataFrame, but we'd really like to be <code>int</code>.

Thoughts?

Things tried:

I tried using the <code>from_records()</code> function under pandas.DataFrame, with <code>coerce_float=False</code> and this did not help. I also tried using NumPy masked arrays, with NaN fill_value, which also did not work. All of these caused the column data type to become a float.

NumPy or Pandas: Keeping array type as integer while having a NaN value

数据结构

Python

有没有一种更好的方法来保持numpy数组的数据类型固定为int (或int64或其他任何类型)，同时内部仍有一个元素列为numpy.NaN特别是，我正在将内部数据结构转换为Pandas DataFrame。在我们的结构中，我们有整数类型的列，它们仍然有NaN(但是列的dtype是int)。如果我们把它变成一个DataF...

问NumPy或Pandas:在具有NaN值的情况下保持数组类型为整数
EN

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问NumPy或Pandas:在具有NaN值的情况下保持数组类型为整数EN

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问NumPy或Pandas:在具有NaN值的情况下保持数组类型为整数
EN