blocks|key|2034852|text|在切片数组上使用np.unique，return_index+param优于axis=0，这为我们提供了唯一的索引，将每一行看作一个实体。然后，可以使用这些索引对原始数组进行行索引，以获得所需的输出。|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|2034853|因此，以a作为输入数组，它将是-|2034854|a[np.unique(a[:,[0,1,2,5]],return_index=True,axis=0)[1]]|code-block|syntax|javascript|2034855|试运行来分解步骤，希望能把事情弄清楚-|2034856|In+[29]:+a
Out[29]:+
array([[+-4,+++5,+++9,++30,++50,++80],
+++++++[++2,++-6,+++9,++34,++12,+++7],
+++++++[+-4,+++5,+++9,++98,+-21,++80],
+++++++[++5,++-9,+++0,++32,++18,+++0]])

In+[30]:+a_slice+=+a[:,[0,1,2,5]]

In+[31]:+_,+unq_row_indices+=+np.unique(a_slice,return_index=True,axis=0)

In+[32]:+final_output+=+a[unq_row_indices]

In+[33]:+final_output
Out[33]:+
array([[-4,++5,++9,+30,+50,+80],
+++++++[+2,+-6,++9,+34,+12,++7],
+++++++[+5,+-9,++0,+32,+18,++0]])|2034857|entityMap|0|LINK|mutability|MUTABLE|url|https://docs.scipy.org/doc/numpy/reference/generated/numpy.unique.html^0|8|9|I|C|12|6|8|9|0|0|4|1|0|0|0|0^^$0|@$1|2|3|4|5|6|7|Y|8|@$9|Z|A|10|B|C]|$9|11|A|12|B|C]|$9|13|A|14|B|C]]|D|@$9|15|A|16|1|17]]|E|$]]|$1|F|3|G|5|6|7|18|8|@$9|19|A|1A|B|C]]|D|@]|E|$]]|$1|H|3|I|5|J|7|1B|8|@]|D|@]|E|$K|L]]|$1|M|3|N|5|6|7|1C|8|@]|D|@]|E|$]]|$1|O|3|P|5|J|7|1D|8|@]|D|@]|E|$K|L]]|$1|Q|3|-4|5|6|7|1E|8|@]|D|@]|E|$]]]|R|$S|$5|T|U|V|E|$W|X]]]]

Use <a href="https://docs.scipy.org/doc/numpy/reference/generated/numpy.unique.html" rel="nofollow noreferrer"><code>np.unique</code></a> on the sliced array with <code>return_index</code> param over <code>axis=0</code>, that gives us unique indices, considering each row as one entity. These indices could be then used for row-indexing into the original array for the desired output.

So, with <code>a</code> as the input array, it would be -

<pre><code>a[np.unique(a[:,[0,1,2,5]],return_index=True,axis=0)[1]]
</code></pre>

Sample run to break down the steps and hopefully make things clear -

<pre><code>In [29]: a
Out[29]: 
array([[ -4, 5, 9, 30, 50, 80],
 [ 2, -6, 9, 34, 12, 7],
 [ -4, 5, 9, 98, -21, 80],
 [ 5, -9, 0, 32, 18, 0]])

In [30]: a_slice = a[:,[0,1,2,5]]

In [31]: _, unq_row_indices = np.unique(a_slice,return_index=True,axis=0)

In [32]: final_output = a[unq_row_indices]

In [33]: final_output
Out[33]: 
array([[-4, 5, 9, 30, 50, 80],
 [ 2, -6, 9, 34, 12, 7],
 [ 5, -9, 0, 32, 18, 0]])
</code></pre>

blocks|key|3085687|text|熊猫通过pd.DataFrame.drop_duplicates有这方面的功能。然而，方便的语法是以性能为代价的。|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|3085688|import+pandas+as+pd
import+numpy+as+np

A+=+np.array([[+-4,+++5,+++9,++30,++50,++80],
++++++++++++++[++2,++-6,+++9,++34,++12,+++7],
++++++++++++++[+-4,+++5,+++9,++98,+-21,++80],
++++++++++++++[++5,++-9,+++0,++32,++18,+++0]])

res+=+pd.DataFrame(A)\
++++++++.drop_duplicates(subset=[0,+1,+2,+5])\
++++++++.values

print(res)

array([[-4,++5,++9,+30,+50,+80],
+++++++[+2,+-6,++9,+34,+12,++7],
+++++++[+5,+-9,++0,+32,+18,++0]])|code-block|syntax|javascript|3085689|entityMap|0|LINK|mutability|MUTABLE|url|http://pandas.pydata.org/pandas-docs/version/0.22/generated/pandas.DataFrame.drop_duplicates.html^0|4|S|4|S|0|0|0^^$0|@$1|2|3|4|5|6|7|S|8|@$9|T|A|U|B|C]]|D|@$9|V|A|W|1|X]]|E|$]]|$1|F|3|G|5|H|7|Y|8|@]|D|@]|E|$I|J]]|$1|K|3|-4|5|6|7|Z|8|@]|D|@]|E|$]]]|L|$M|$5|N|O|P|E|$Q|R]]]]

Pandas has functionality for this via <a href="http://pandas.pydata.org/pandas-docs/version/0.22/generated/pandas.DataFrame.drop_duplicates.html" rel="nofollow noreferrer"><code>pd.DataFrame.drop_duplicates</code></a>. However, the convenient syntax comes at the cost of performance.

<pre><code>import pandas as pd
import numpy as np

A = np.array([[ -4, 5, 9, 30, 50, 80],
 [ 2, -6, 9, 34, 12, 7],
 [ -4, 5, 9, 98, -21, 80],
 [ 5, -9, 0, 32, 18, 0]])

res = pd.DataFrame(A)\
 .drop_duplicates(subset=[0, 1, 2, 5])\
 .values

print(res)

array([[-4, 5, 9, 30, 50, 80],
 [ 2, -6, 9, 34, 12, 7],
 [ 5, -9, 0, 32, 18, 0]])
</code></pre>

blocks|key|2034929|text|您可以使用np.take方法(https://docs.scipy.org/doc/numpy-1.14.0/reference/generated/numpy.take.html)从您关心的数组中获取唯一的列，然后在return_index=True中使用唯一的方法。|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|2034930|>>>+arr+=+np.array([[+-4,+++5,+++9,++30,++50,++80],
...++++++++[++2,++-6,+++9,++34,++12,+++7],
...++++++++[+-4,+++5,+++9,++98,+-21,++80],
...++++++++[++5,++-9,+++0,++32,++18,+++0]])
>>>+relevant_columns+=+np.take(arr,+[0,1,2,5],+axis=1)
>>>+np.unique(relevant_columns,+axis=0,+return_index=True)
(array([[+2,+-6,++9,++7],
+++++++[+5,+-9,++0,++0],
+++++++[-4,++5,++9,+80]]),+array([1,+3,+0]))|code-block|syntax|javascript|2034931|然后，您可以在原始的numpy数组中再次使用np.take()。传递array([1,+3,+0])作为索引的参数。|2034932|entityMap|0|LINK|mutability|MUTABLE|url|https://docs.scipy.org/doc/numpy-1.14.0/reference/generated/numpy.take.html^0|5|7|31|H|F|23|0|0|0|M|9|Y|G|0^^$0|@$1|2|3|4|5|6|7|U|8|@$9|V|A|W|B|C]|$9|X|A|Y|B|C]]|D|@$9|Z|A|10|1|11]]|E|$]]|$1|F|3|G|5|H|7|12|8|@]|D|@]|E|$I|J]]|$1|K|3|L|5|6|7|13|8|@$9|14|A|15|B|C]|$9|16|A|17|B|C]]|D|@]|E|$]]|$1|M|3|-4|5|6|7|18|8|@]|D|@]|E|$]]]|N|$O|$5|P|Q|R|E|$S|T]]]]

You can use the <code>np.take</code> method (<a href="https://docs.scipy.org/doc/numpy-1.14.0/reference/generated/numpy.take.html" rel="nofollow noreferrer">https://docs.scipy.org/doc/numpy-1.14.0/reference/generated/numpy.take.html</a>) to get the only the columns from the array that you care about and then use the unique method with <code>return_index=True</code>. 

<pre><code>&gt;&gt;&gt; arr = np.array([[ -4, 5, 9, 30, 50, 80],
... [ 2, -6, 9, 34, 12, 7],
... [ -4, 5, 9, 98, -21, 80],
... [ 5, -9, 0, 32, 18, 0]])
&gt;&gt;&gt; relevant_columns = np.take(arr, [0,1,2,5], axis=1)
&gt;&gt;&gt; np.unique(relevant_columns, axis=0, return_index=True)
(array([[ 2, -6, 9, 7],
 [ 5, -9, 0, 0],
 [-4, 5, 9, 80]]), array([1, 3, 0]))
</code></pre>

You can then use <code>np.take()</code> again with your original numpy array. Pass <code>array([1, 3, 0])</code> as the parameter for the indices.

I have a large dataset in a <code>numpy.ndarray</code> similar to this:

<pre><code>array([[ -4, 5, 9, 30, 50, 80],
 [ 2, -6, 9, 34, 12, 7],
 [ -4, 5, 9, 98, -21, 80],
 [ 5, -9, 0, 32, 18, 0]])
</code></pre>

I would like to remove duplicate rows, where the 0th, 1st, 2nd and 5th columns are equal. I.e. On the above matrix, the response would be:

<pre><code>-4, 5, 9, 30, 50, 80
2, -6, 9, 34, 12, 7
5, -9, 0, 32, 18, 0
</code></pre>

<code>numpy.unique</code> does something very similar but it only finds duplicates over all columns (axis). I only want specific columns. How would one get around to do this with <code>numpy</code>? I could not find any decent <code>numpy</code> algorithm to do this. Is there a better module?

Remove rows by duplicate column(s) values

翻译质量差，导致语言生硬或混乱。

没有提供实际的解决方法或示例。

解答不清晰，无法理解或解决问题。

页面排版不美观，阅读体验差。

文章

问答

视频

学习中心

腾讯云实验室

直播

竞赛

腾讯云代码分析专区

腾讯iOA零信任安全管理系统专区

腾讯云架构师技术同盟交流圈

腾讯云数据库专区

腾讯云顾问专区

腾讯云原生专区

腾讯混元专区

腾讯云TCE专区

腾讯云Lighthouse专区

腾讯云HAI专区

腾讯云Edgeone专区

腾讯云存储专区

腾讯云智能专区

腾讯轻联专区 

腾讯云开发专区

TAPD专区

腾讯轻量云游戏服专区

腾讯云最具价值专家

腾讯云架构师技术同盟

腾讯云创作之星

腾讯云开发者先锋

腾讯云代码助手

云原生构建

TAPD 敏捷项目管理

Cloud Studio

SDK中心

API中心

命令行工具

涵盖代码开发、场景应用、自动测试全流程，助你从零构建专属AI助手

一站式MCP教程库，解锁AI应用新玩法

我在numpy.ndarray中有一个类似于以下内容的大型数据集：array([[ -4,   5,   9,  30,  50,  80],       [  2,  -6,   9,  34,  12,   7],       [ -4,   5,   9,  98, -21,  80],       [  5, ...

问按重复列值删除行
EN

Stack Overflow用户

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问按重复列值删除行EN