blocks|key|859773|text|通过sort发送它(将相邻的项目放在一起)，然后通过uniq+-c提供计数，即：|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|859774|sort+filename+%7C+uniq+-c|code-block|syntax|javascript|859775|要以排序顺序(按频率)获得该列表，您可以|859776|sort+filename+%7C+uniq+-c+%7C+sort+-nr|859777|entityMap^0|2|4|Q|7|0|0|0|0^^$0|@$1|2|3|4|5|6|7|Q|8|@$9|R|A|S|B|C]|$9|T|A|U|B|C]]|D|@]|E|$]]|$1|F|3|G|5|H|7|V|8|@]|D|@]|E|$I|J]]|$1|K|3|L|5|6|7|W|8|@]|D|@]|E|$]]|$1|M|3|N|5|H|7|X|8|@]|D|@]|E|$I|J]]|$1|O|3|-4|5|6|7|Y|8|@]|D|@]|E|$]]]|P|$]]

Send it through <code>sort</code> (to put adjacent items together) then <code>uniq -c</code> to give counts, i.e.:

<pre><code>sort filename | uniq -c
</code></pre>

and to get that list in sorted order (by frequency) you can

<pre><code>sort filename | uniq -c | sort -nr
</code></pre>

blocks|key|642519|text|几乎与uniq相同，但是如果你在borribles中添加d参数，它只会显示重复项。|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|642520|sort+filename+%7C+uniq+-cd+%7C+sort+-nr|code-block|syntax|javascript|642521|entityMap^0|3|4|S|1|0|0^^$0|@$1|2|3|4|5|6|7|M|8|@$9|N|A|O|B|C]|$9|P|A|Q|B|C]]|D|@]|E|$]]|$1|F|3|G|5|H|7|R|8|@]|D|@]|E|$I|J]]|$1|K|3|-4|5|6|7|S|8|@]|D|@]|E|$]]]|L|$]]

Almost the same as borribles' but if you add the <code>d</code> param to <code>uniq</code> it only shows duplicates.

<pre class="lang-bsh prettyprint-override"><code>sort filename | uniq -cd | sort -nr
</code></pre>

blocks|key|642304|text|uniq+-c+file|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|642305|在文件尚未排序的情况下：|642306|sort+file+%7C+uniq+-c|642307|entityMap^0|0|C|0|0|0|J|0^^$0|@$1|2|3|4|5|6|7|L|8|@$9|M|A|N|B|C]]|D|@]|E|$]]|$1|F|3|G|5|6|7|O|8|@]|D|@]|E|$]]|$1|H|3|I|5|6|7|P|8|@$9|Q|A|R|B|C]]|D|@]|E|$]]|$1|J|3|-4|5|6|7|S|8|@]|D|@]|E|$]]]|K|$]]

<code>uniq -c file</code>

and in case the file is not sorted already:

<code>sort file | uniq -c</code>

blocks|key|642330|text|cat+<filename>+%7C+sort+%7C+uniq+-c|type|code-block|depth|inlineStyleRanges|entityRanges|data|syntax|javascript|642331|unstyled|entityMap^0|0^^$0|@$1|2|3|4|5|6|7|G|8|@]|9|@]|A|$B|C]]|$1|D|3|-4|5|E|7|H|8|@]|9|@]|A|$]]]|F|$]]

<pre><code>cat &lt;filename&gt; | sort | uniq -c
</code></pre>

blocks|key|642253|text|你能接受一个按字母顺序排列的列表吗：|type|unstyled|depth|inlineStyleRanges|entityRanges|data|642254|echo+"red+apple
>+green+apple
>+green+apple
>+orange
>+orange
>+orange
>+"+%7C+sort+-u+|code-block|syntax|javascript|642255|642256|green+apple
orange
red+apple|642257|或|642258|sort+-u+FILE|642259|-u代表unique，唯一性需要排序才能达到。|642260|一种保留顺序的解决方案：|642261|echo+"red+apple
green+apple
green+apple
orange
orange
orange
"+%7C+{+old="";+while+read+line+;+do+++if+[[+$line+!=+$old+]];+then++echo+$line;+++old=$line;+fi+;+done+}
red+apple
green+apple
orange|642262|并且，使用一个文件|642263|cat+file+%7C+{+
old=""
while+read+line
do
++if+[[+$line+!=+$old+]]
++then
++++echo+$line
++++old=$line
++fi
done+}|642264|最后两个只删除重复项，紧跟其后的是--这符合您的示例。|642265|echo+"red+apple
green+apple
lila+banana
green+apple
"+...|642266|将打印两个苹果，被一个香蕉分成两半。|642267|entityMap^0|0|0|0|0|0|0|0|0|0|0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|15|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|16|8|@]|9|@]|A|$E|F]]|$1|G|3|-4|5|6|7|17|8|@]|9|@]|A|$]]|$1|H|3|I|5|D|7|18|8|@]|9|@]|A|$E|F]]|$1|J|3|K|5|6|7|19|8|@]|9|@]|A|$]]|$1|L|3|M|5|D|7|1A|8|@]|9|@]|A|$E|F]]|$1|N|3|O|5|6|7|1B|8|@]|9|@]|A|$]]|$1|P|3|Q|5|6|7|1C|8|@]|9|@]|A|$]]|$1|R|3|S|5|D|7|1D|8|@]|9|@]|A|$E|F]]|$1|T|3|U|5|6|7|1E|8|@]|9|@]|A|$]]|$1|V|3|W|5|D|7|1F|8|@]|9|@]|A|$E|F]]|$1|X|3|Y|5|6|7|1G|8|@]|9|@]|A|$]]|$1|Z|3|10|5|D|7|1H|8|@]|9|@]|A|$E|F]]|$1|11|3|12|5|6|7|1I|8|@]|9|@]|A|$]]|$1|13|3|-4|5|6|7|1J|8|@]|9|@]|A|$]]]|14|$]]

Can you live with an alphabetical, ordered list:

<pre><code>echo "red apple
&gt; green apple
&gt; green apple
&gt; orange
&gt; orange
&gt; orange
&gt; " | sort -u 
</code></pre>

?

<pre><code>green apple
orange
red apple
</code></pre>

or 

<pre><code>sort -u FILE
</code></pre>

-u stands for unique, and uniqueness is only reached via sorting.

A solution which preserves the order:

<pre><code>echo "red apple
green apple
green apple
orange
orange
orange
" | { old=""; while read line ; do if [[ $line != $old ]]; then echo $line; old=$line; fi ; done }
red apple
green apple
orange
</code></pre>

and, with a file

<pre><code>cat file | { 
old=""
while read line
do
 if [[ $line != $old ]]
 then
 echo $line
 old=$line
 fi
done }
</code></pre>

The last two only remove duplicates, which follow immediately - which fits to your example.

<pre><code>echo "red apple
green apple
lila banana
green apple
" ...
</code></pre>

Will print two apples, split by a banana.

blocks|key|642343|text|尝尝这个|type|unstyled|depth|inlineStyleRanges|entityRanges|data|642344|cat+myfile.txt%7C+sort%7C+uniq|code-block|syntax|javascript|642345|entityMap^0|0|0^^$0|@$1|2|3|4|5|6|7|I|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|J|8|@]|9|@]|A|$E|F]]|$1|G|3|-4|5|6|7|K|8|@]|9|@]|A|$]]]|H|$]]

Try this

<pre><code>cat myfile.txt| sort| uniq
</code></pre>

blocks|key|859920|text|下面是一个使用Counter类型的简单python脚本。好处是，这不需要对文件进行排序，基本上使用零内存：|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|859921|import+collections
import+fileinput
import+json

print(json.dumps(collections.Counter(map(str.strip,+fileinput.input())),+indent=2))|code-block|syntax|javascript|859922|输出：|859923|$+cat+filename+%7C+python3+script.py
{
++"red+apple":+1,
++"green+apple":+2,
++"orange":+3
}|859924|或者，您可以使用简单的一行代码：|859925|$+cat+filename+%7C+python3+-c+'print(__import__("json").dumps(__import__("collections").Counter(map(str.strip,+__import__("fileinput").input())),+indent=2))'|859926|entityMap|0|LINK|mutability|MUTABLE|url|https://docs.python.org/3/library/collections.html#collections.Counter^0|7|7|0|0|0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|Y|8|@]|9|@$A|Z|B|10|1|11]]|C|$]]|$1|D|3|E|5|F|7|12|8|@]|9|@]|C|$G|H]]|$1|I|3|J|5|6|7|13|8|@]|9|@]|C|$]]|$1|K|3|L|5|F|7|14|8|@]|9|@]|C|$G|H]]|$1|M|3|N|5|6|7|15|8|@]|9|@]|C|$]]|$1|O|3|P|5|F|7|16|8|@]|9|@]|C|$G|H]]|$1|Q|3|-4|5|6|7|17|8|@]|9|@]|C|$]]]|R|$S|$5|T|U|V|C|$W|X]]]]

Here is a simple python script using the <a href="https://docs.python.org/3/library/collections.html#collections.Counter" rel="nofollow noreferrer">Counter</a> type. The benefit is that this does not require sorting the file, essentially using zero memory:
<pre class="lang-py prettyprint-override"><code>import collections
import fileinput
import json

print(json.dumps(collections.Counter(map(str.strip, fileinput.input())), indent=2))
</code></pre>
Output:
<pre><code>$ cat filename | python3 script.py
{
 &quot;red apple&quot;: 1,
 &quot;green apple&quot;: 2,
 &quot;orange&quot;: 3
}
</code></pre>
or you can use a simple one-liner:
<pre><code>$ cat filename | python3 -c 'print(__import__(&quot;json&quot;).dumps(__import__(&quot;collections&quot;).Counter(map(str.strip, __import__(&quot;fileinput&quot;).input())), indent=2))'
</code></pre>

If I have a text file with the following conent

<pre><code>red apple
green apple
green apple
orange
orange
orange
</code></pre>

Is there a Linux command or script that I can use to get the following result?

<pre><code>1 red apple
2 green apple
3 orange
</code></pre>

Linux command or script counting duplicated lines in a text file?

Linux

如果我有一个包含以下内容的文本文件red applegreen applegreen appleorangeorangeorange有没有Linux命令或脚本可以让我得到下面的结果？1 red apple2 green apple3 orange

问Linux命令或脚本计算文本文件中的重复行？
EN

回答 7

Stack Overflow用户

Stack Overflow用户

Stack Overflow用户

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问Linux命令或脚本计算文本文件中的重复行？EN

回答 7

Stack Overflow用户

Stack Overflow用户

Stack Overflow用户

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问Linux命令或脚本计算文本文件中的重复行？
EN