blocks|key|1878595|text|如果一个值是一个np.nan对象，那么Pandas将把它识别为null，这将在DataFrame中打印为NaN。您缺少的值可能是空字符串，Pandas不会将其识别为null。要解决这个问题，可以使用replace()将空字符串(或空单元格中的任何内容)转换为np.nan对象，然后在DataFrame上调用dropna()来删除具有null租户的行。|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|1878596|为了演示，我们在Tenants列中创建一个包含一些随机值和一些空字符串的DataFrame：|1878597|>>>+import+pandas+as+pd
>>>+import+numpy+as+np
>>>+
>>>+df+=+pd.DataFrame(np.random.randn(10,+2),+columns=list('AB'))
>>>+df['Tenant']+=+np.random.choice(['Babar',+'Rataxes',+''],+10)
>>>+print+df

++++++++++A+++++++++B+++Tenant
0+-0.588412+-1.179306++++Babar
1+-0.008562++0.725239+++++++++
2++0.282146++0.421721++Rataxes
3++0.627611+-0.661126++++Babar
4++0.805304+-0.834214+++++++++
5+-0.514568++1.890647++++Babar
6+-1.188436++0.294792++Rataxes
7++1.471766+-0.267807++++Babar
8+-1.730745++1.358165++Rataxes
9++0.066946++0.375640+++++++++|code-block|syntax|javascript|1878598|现在我们用np.nan对象替换Tenants列中的所有空字符串，如下所示：|1878599|>>>+df['Tenant'].replace('',+np.nan,+inplace=True)
>>>+print+df

++++++++++A+++++++++B+++Tenant
0+-0.588412+-1.179306++++Babar
1+-0.008562++0.725239++++++NaN
2++0.282146++0.421721++Rataxes
3++0.627611+-0.661126++++Babar
4++0.805304+-0.834214++++++NaN
5+-0.514568++1.890647++++Babar
6+-1.188436++0.294792++Rataxes
7++1.471766+-0.267807++++Babar
8+-1.730745++1.358165++Rataxes
9++0.066946++0.375640++++++NaN|1878600|现在我们可以删除空值：|1878601|>>>+df.dropna(subset=['Tenant'],+inplace=True)
>>>+print+df

++++++++++A+++++++++B+++Tenant
0+-0.588412+-1.179306++++Babar
2++0.282146++0.421721++Rataxes
3++0.627611+-0.661126++++Babar
5+-0.514568++1.890647++++Babar
6+-1.188436++0.294792++Rataxes
7++1.471766+-0.267807++++Babar
8+-1.730745++1.358165++Rataxes|1878602|entityMap^0|8|6|1G|3|2R|9|3L|6|49|8|0|8|7|0|0|5|6|F|7|0|0|0|0^^$0|@$1|2|3|4|5|6|7|W|8|@$9|X|A|Y|B|C]|$9|Z|A|10|B|C]|$9|11|A|12|B|C]|$9|13|A|14|B|C]|$9|15|A|16|B|C]]|D|@]|E|$]]|$1|F|3|G|5|6|7|17|8|@$9|18|A|19|B|C]]|D|@]|E|$]]|$1|H|3|I|5|J|7|1A|8|@]|D|@]|E|$K|L]]|$1|M|3|N|5|6|7|1B|8|@$9|1C|A|1D|B|C]|$9|1E|A|1F|B|C]]|D|@]|E|$]]|$1|O|3|P|5|J|7|1G|8|@]|D|@]|E|$K|L]]|$1|Q|3|R|5|6|7|1H|8|@]|D|@]|E|$]]|$1|S|3|T|5|J|7|1I|8|@]|D|@]|E|$K|L]]|$1|U|3|-4|5|6|7|1J|8|@]|D|@]|E|$]]]|V|$]]

Pandas will recognise a value as null if it is a <code>np.nan</code> object, which will print as <code>NaN</code> in the DataFrame. Your missing values are probably empty strings, which Pandas doesn't recognise as null. To fix this, you can convert the empty stings (or whatever is in your empty cells) to <code>np.nan</code> objects using <code>replace()</code>, and then call <code>dropna()</code>on your DataFrame to delete rows with null tenants.

To demonstrate, we create a DataFrame with some random values and some empty strings in a <code>Tenants</code> column:

<pre><code>&gt;&gt;&gt; import pandas as pd
&gt;&gt;&gt; import numpy as np
&gt;&gt;&gt; 
&gt;&gt;&gt; df = pd.DataFrame(np.random.randn(10, 2), columns=list('AB'))
&gt;&gt;&gt; df['Tenant'] = np.random.choice(['Babar', 'Rataxes', ''], 10)
&gt;&gt;&gt; print df

 A B Tenant
0 -0.588412 -1.179306 Babar
1 -0.008562 0.725239 
2 0.282146 0.421721 Rataxes
3 0.627611 -0.661126 Babar
4 0.805304 -0.834214 
5 -0.514568 1.890647 Babar
6 -1.188436 0.294792 Rataxes
7 1.471766 -0.267807 Babar
8 -1.730745 1.358165 Rataxes
9 0.066946 0.375640 
</code></pre>

Now we replace any empty strings in the <code>Tenants</code> column with <code>np.nan</code> objects, like so:

<pre><code>&gt;&gt;&gt; df['Tenant'].replace('', np.nan, inplace=True)
&gt;&gt;&gt; print df

 A B Tenant
0 -0.588412 -1.179306 Babar
1 -0.008562 0.725239 NaN
2 0.282146 0.421721 Rataxes
3 0.627611 -0.661126 Babar
4 0.805304 -0.834214 NaN
5 -0.514568 1.890647 Babar
6 -1.188436 0.294792 Rataxes
7 1.471766 -0.267807 Babar
8 -1.730745 1.358165 Rataxes
9 0.066946 0.375640 NaN
</code></pre>

Now we can drop the null values:

<pre><code>&gt;&gt;&gt; df.dropna(subset=['Tenant'], inplace=True)
&gt;&gt;&gt; print df

 A B Tenant
0 -0.588412 -1.179306 Babar
2 0.282146 0.421721 Rataxes
3 0.627611 -0.661126 Babar
5 -0.514568 1.890647 Babar
6 -1.188436 0.294792 Rataxes
7 1.471766 -0.267807 Babar
8 -1.730745 1.358165 Rataxes
</code></pre>

blocks|key|1664756|text|默认情况下，value_counts省略了NaN，因此您最有可能处理的是"“。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|1664757|所以你可以像这样过滤掉它们|1664758|filter+=+df["Tenant"]+!=+""
dfNew+=+df[filter]|code-block|syntax|javascript|1664759|entityMap^0|0|0|0^^$0|@$1|2|3|4|5|6|7|K|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|L|8|@]|9|@]|A|$]]|$1|D|3|E|5|F|7|M|8|@]|9|@]|A|$G|H]]|$1|I|3|-4|5|6|7|N|8|@]|9|@]|A|$]]]|J|$]]

value_counts omits NaN by default so you're most likely dealing with "".

So you can just filter them out like

<pre><code>filter = df["Tenant"] != ""
dfNew = df[filter]
</code></pre>

blocks|key|1664849|text|您可以使用此变体：|type|unstyled|depth|inlineStyleRanges|entityRanges|data|1664850|import+pandas+as+pd
vals+=+{
++++'name'+:+['n1',+'n2',+'n3',+'n4',+'n5',+'n6',+'n7'],
++++'gender'+:+['m',+'f',+'f',+'f',++'f',+'c',+'c'],
++++'age'+:+[39,+12,+27,+13,+36,+29,+10],
++++'education'+:+['ma',+None,+'school',+None,+'ba',+None,+None]
}
df_vals+=+pd.DataFrame(vals)+#converting+dict+to+dataframe|code-block|syntax|javascript|1664851|这将输出(**-仅突出显示所需的行)：|1664852|+++age+education+gender+name
0+++39++++++++ma++++++m+++n1+**
1+++12++++++None++++++f+++n2++++
2+++27++++school++++++f+++n3+**
3+++13++++++None++++++f+++n4
4+++36++++++++ba++++++f+++n5+**
5+++29++++++None++++++c+++n6
6+++10++++++None++++++c+++n7|1664853|因此，要删除不具有“education”值的所有内容，请使用以下代码：|1664854|df_vals+=+df_vals[~df_vals['education'].isnull()]+|1664855|('~‘表示不)|1664856|结果：|1664857|+++age+education+gender+name
0+++39++++++++ma++++++m+++n1
2+++27++++school++++++f+++n3
4+++36++++++++ba++++++f+++n5|1664858|entityMap^0|0|0|0|0|0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|W|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|X|8|@]|9|@]|A|$E|F]]|$1|G|3|H|5|6|7|Y|8|@]|9|@]|A|$]]|$1|I|3|J|5|D|7|Z|8|@]|9|@]|A|$E|F]]|$1|K|3|L|5|6|7|10|8|@]|9|@]|A|$]]|$1|M|3|N|5|D|7|11|8|@]|9|@]|A|$E|F]]|$1|O|3|P|5|6|7|12|8|@]|9|@]|A|$]]|$1|Q|3|R|5|6|7|13|8|@]|9|@]|A|$]]|$1|S|3|T|5|D|7|14|8|@]|9|@]|A|$E|F]]|$1|U|3|-4|5|6|7|15|8|@]|9|@]|A|$]]]|V|$]]

You can use this variation:

<pre><code>import pandas as pd
vals = {
 'name' : ['n1', 'n2', 'n3', 'n4', 'n5', 'n6', 'n7'],
 'gender' : ['m', 'f', 'f', 'f', 'f', 'c', 'c'],
 'age' : [39, 12, 27, 13, 36, 29, 10],
 'education' : ['ma', None, 'school', None, 'ba', None, None]
}
df_vals = pd.DataFrame(vals) #converting dict to dataframe
</code></pre>

This will output(** - highlighting only desired rows):

<pre><code> age education gender name
0 39 ma m n1 **
1 12 None f n2 
2 27 school f n3 **
3 13 None f n4
4 36 ba f n5 **
5 29 None c n6
6 10 None c n7
</code></pre>

So to drop everything that does not have an 'education' value, use the code below:

<pre><code>df_vals = df_vals[~df_vals['education'].isnull()] 
</code></pre>

('~' indicating NOT)

Result:

<pre><code> age education gender name
0 39 ma m n1
2 27 school f n3
4 36 ba f n5
</code></pre>

blocks|key|1878918|text|有一种情况是单元格有空格，您看不到它，请使用|type|unstyled|depth|inlineStyleRanges|entityRanges|data|1878919|df['col'].replace('++',+np.nan,+inplace=True)|code-block|syntax|javascript|1878920|若要将空白替换为NaN，则|1878921|df=+df.dropna(subset=['col'])|1878922|entityMap^0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|M|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|N|8|@]|9|@]|A|$E|F]]|$1|G|3|H|5|6|7|O|8|@]|9|@]|A|$]]|$1|I|3|J|5|D|7|P|8|@]|9|@]|A|$E|F]]|$1|K|3|-4|5|6|7|Q|8|@]|9|@]|A|$]]]|L|$]]

There's a situation where the cell has white space, you can't see it, use 

<pre><code>df['col'].replace(' ', np.nan, inplace=True)
</code></pre>

to replace white space as NaN, then 

<pre><code>df= df.dropna(subset=['col'])
</code></pre>

blocks|key|1879014|text|Pythonic+%2B+Pandorable：df[df['col'].astype(bool)]|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|1879015|空字符串是假的，这意味着您可以像这样过滤布尔值：|1879016|df+=+pd.DataFrame({
++++'A':+range(5),
++++'B':+['foo',+'',+'bar',+'',+'xyz']
})
df
+++A++++B
0++0++foo
1++1+++++
2++2++bar
3++3+++++
4++4++xyz|code-block|syntax|javascript|1879017|df['B'].astype(bool)++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
0+++++True
1++++False
2+++++True
3++++False
4+++++True
Name:+B,+dtype:+bool

df[df['B'].astype(bool)]++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+++A++++B
0++0++foo
2++2++bar
4++4++xyz|1879018|如果您的目标是不仅要删除空字符串，而且还要删除仅包含空格的字符串，请提前使用str.strip：|1879019|df[df['B'].str.strip().astype(bool)]
+++A++++B
0++0++foo
2++2++bar
4++4++xyz|1879020|比你想象的更快|1879021|.astype是一个矢量化操作，这比到目前为止提出的每个选项都要快。至少从我的测试来看是这样。YMMV.|1879022|这是一个时间上的比较，我提出了一些我能想到的其他方法。|1879023|​|1879024|📷|atomic|1879025|1879026|基准测试代码，供参考：|1879027|import+pandas+as+pd
import+perfplot

df1+=+pd.DataFrame({
++++'A':+range(5),
++++'B':+['foo',+'',+'bar',+'',+'xyz']
})

perfplot.show(
++++setup=lambda+n:+pd.concat([df1]+*+n,+ignore_index=True),
++++kernels=[
++++++++lambda+df:+df[df['B'].astype(bool)],
++++++++lambda+df:+df[df['B']+!=+''],
++++++++lambda+df:+df[df['B'].replace('',+np.nan).notna()],++#+optimized+1-col
++++++++lambda+df:+df.replace({'B':+{'':+np.nan}}).dropna(subset=['B']),++
++++],
++++labels=['astype',+"!=+''",+"replace+%2B+notna",+"replace+%2B+dropna",+],
++++n_range=[2**k+for+k+in+range(1,+15)],
++++xlabel='N',
++++logx=True,
++++logy=True,
++++equality_check=pd.DataFrame.equals)|1879028|entityMap|0|IMAGE|mutability|IMMUTABLE|imageUrl|https://ask.qcloudimg.com/http-save/yehe-900000/aa6af0e5aa88433cd242c59c892f1cc1.png|imageAlt^0|M|Q|0|0|0|0|12|9|0|0|0|0|7|0|0|0|0|1|0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|1H|8|@$9|1I|A|1J|B|C]]|D|@]|E|$]]|$1|F|3|G|5|6|7|1K|8|@]|D|@]|E|$]]|$1|H|3|I|5|J|7|1L|8|@]|D|@]|E|$K|L]]|$1|M|3|N|5|J|7|1M|8|@]|D|@]|E|$K|L]]|$1|O|3|P|5|6|7|1N|8|@$9|1O|A|1P|B|C]]|D|@]|E|$]]|$1|Q|3|R|5|J|7|1Q|8|@]|D|@]|E|$K|L]]|$1|S|3|T|5|6|7|1R|8|@]|D|@]|E|$]]|$1|U|3|V|5|6|7|1S|8|@$9|1T|A|1U|B|C]]|D|@]|E|$]]|$1|W|3|X|5|6|7|1V|8|@]|D|@]|E|$]]|$1|Y|3|Z|5|6|7|1W|8|@]|D|@]|E|$]]|$1|10|3|11|5|12|7|1X|8|@]|D|@$9|1Y|A|1Z|1|20]]|E|$]]|$1|13|3|Z|5|6|7|21|8|@]|D|@]|E|$]]|$1|14|3|15|5|6|7|22|8|@]|D|@]|E|$]]|$1|16|3|17|5|J|7|23|8|@]|D|@]|E|$K|L]]|$1|18|3|-4|5|6|7|24|8|@]|D|@]|E|$]]]|19|$1A|$5|1B|1C|1D|E|$1E|1F|1G|-4]]]]

<h1>Pythonic + Pandorable: <code>df[df['col'].astype(bool)]</code></h1>
Empty strings are falsy, which means you can filter on bool values like this:
<pre><code>df = pd.DataFrame({
 'A': range(5),
 'B': ['foo', '', 'bar', '', 'xyz']
})
df
 A B
0 0 foo
1 1 
2 2 bar
3 3 
4 4 xyz
</code></pre>

<pre><code>df['B'].astype(bool) 
0 True
1 False
2 True
3 False
4 True
Name: B, dtype: bool

df[df['B'].astype(bool)] 
 A B
0 0 foo
2 2 bar
4 4 xyz
</code></pre>
If your goal is to remove not only empty strings, but also strings only containing whitespace, use <code>str.strip</code> beforehand:
<pre><code>df[df['B'].str.strip().astype(bool)]
 A B
0 0 foo
2 2 bar
4 4 xyz
</code></pre>
<h1>Faster than you Think</h1>
<code>.astype</code> is a vectorised operation, this is faster than every option presented thus far. At least, from my tests. YMMV.
Here is a timing comparison, I've thrown in some other methods I could think of.
<a href="https://i.stack.imgur.com/EXnza.png" rel="noreferrer"><img src="https://i.stack.imgur.com/EXnza.png" alt="enter image description here" /></a>
Benchmarking code, for reference:
<pre><code>import pandas as pd
import perfplot

df1 = pd.DataFrame({
 'A': range(5),
 'B': ['foo', '', 'bar', '', 'xyz']
})

perfplot.show(
 setup=lambda n: pd.concat([df1] * n, ignore_index=True),
 kernels=[
 lambda df: df[df['B'].astype(bool)],
 lambda df: df[df['B'] != ''],
 lambda df: df[df['B'].replace('', np.nan).notna()], # optimized 1-col
 lambda df: df.replace({'B': {'': np.nan}}).dropna(subset=['B']), 
 ],
 labels=['astype', &quot;!= ''&quot;, &quot;replace + notna&quot;, &quot;replace + dropna&quot;, ],
 n_range=[2**k for k in range(1, 15)],
 xlabel='N',
 logx=True,
 logy=True,
 equality_check=pd.DataFrame.equals)
</code></pre>

I have a <code>pd.DataFrame</code> that was created by parsing some excel spreadsheets. A column of which has empty cells. For example, below is the output for the frequency of that column, 32320 records have missing values for Tenant.

<pre><code>&gt;&gt;&gt; value_counts(Tenant, normalize=False)
 32320
 Thunderhead 8170
 Big Data Others 5700
 Cloud Cruiser 5700
 Partnerpedia 5700
 Comcast 5700
 SDP 5700
 Agora 5700
 dtype: int64
</code></pre>

I am trying to drop rows where Tenant is missing, however <code>.isnull()</code> option does not recognize the missing values. 

<pre><code>&gt;&gt;&gt; df['Tenant'].isnull().sum()
 0
</code></pre>

The column has data type "Object". What is happening in this case? How can I drop records where Tenant is missing?

Drop rows containing empty cells from a pandas DataFrame

我有一个pd.DataFrame，它是通过解析一些excel电子表格创建的。其中一列具有空单元格。例如，下面是该列频率的输出，有32320条记录缺少租户的值。>>> value_counts(Tenant, normalize=False)                              32320    T...

问从pandas DataFrame中删除包含空单元格的行
EN

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问从pandas DataFrame中删除包含空单元格的行EN

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问从pandas DataFrame中删除包含空单元格的行
EN