blocks|key|820782|text|我的FOSS+CSV流编辑器CSVfix完全可以做您想做的事情。Windows有一个二进制安装程序，UNIX/Linux有一个可编译版本(通过makefile)。|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|820783|entityMap|0|LINK|mutability|MUTABLE|url|https://bitbucket.org/neilb/csvfix/src^0|E|6|0|0^^$0|@$1|2|3|4|5|6|7|L|8|@]|9|@$A|M|B|N|1|O]]|C|$]]|$1|D|3|-4|5|6|7|P|8|@]|9|@]|C|$]]]|E|$F|$5|G|H|I|C|$J|K]]]]

My FOSS CSV stream editor <a href="https://bitbucket.org/neilb/csvfix/src" rel="nofollow noreferrer">CSVfix</a> does exactly what you want. There is a binary installer for Windows, and a compilable version (via a makefile) for UNIX/Linux.

blocks|key|2347133|text|good工具真的很好。可在Debian+/+Ubuntu+(apt-get+install+csvtool)中获得。示例：|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|2347134|csvtool+namedcol+Account,Cost+input.csv+>+output.csv|code-block|syntax|javascript|2347135|有关使用技巧，请参阅CSVTool手册页。|2347136|entityMap|0|LINK|mutability|MUTABLE|url|http://colin.maudry.com/csvtool-manual-page/^0|U|N|0|0|A|A|0|0^^$0|@$1|2|3|4|5|6|7|U|8|@$9|V|A|W|B|C]]|D|@]|E|$]]|$1|F|3|G|5|H|7|X|8|@]|D|@]|E|$I|J]]|$1|K|3|L|5|6|7|Y|8|@]|D|@$9|Z|A|10|1|11]]|E|$]]|$1|M|3|-4|5|6|7|12|8|@]|D|@]|E|$]]]|N|$O|$5|P|Q|R|E|$S|T]]]]

csvtool is really good. Available in Debian / Ubuntu (<code>apt-get install csvtool</code>). Example:

<pre><code>csvtool namedcol Account,Cost input.csv &gt; output.csv
</code></pre>

See the <a href="http://colin.maudry.com/csvtool-manual-page/" rel="noreferrer">CSVTool manual page</a> for usage tips.

blocks|key|821021|text|正如@Jonathan在注释中所建议的那样，有一个python模块提供命令行工具csvfilter。它像剪切一样工作，但正确地处理CSV列引用：|type|unstyled|depth|inlineStyleRanges|entityRanges|data|821022|csvfilter+-f+1,3,5+in.csv+>+out.csv|code-block|syntax|javascript|821023|如果您有python+(而且您应该这样做)，您可以简单地安装它，如下所示：|821024|pip+install+csvfilter|821025|更多信息请访问https://github.com/codeinthehole/csvfilter/|offset|length|821026|entityMap|0|LINK|mutability|MUTABLE|url|https://github.com/codeinthehole/csvfilter/^0|0|0|0|0|7|17|0|0^^$0|@$1|2|3|4|5|6|7|W|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|X|8|@]|9|@]|A|$E|F]]|$1|G|3|H|5|6|7|Y|8|@]|9|@]|A|$]]|$1|I|3|J|5|D|7|Z|8|@]|9|@]|A|$E|F]]|$1|K|3|L|5|6|7|10|8|@]|9|@$M|11|N|12|1|13]]|A|$]]|$1|O|3|-4|5|6|7|14|8|@]|9|@]|A|$]]]|P|$Q|$5|R|S|T|A|$U|V]]]]

As suggested by @Jonathan in a comment, there is a module for python that provides the command line tool csvfilter. It works like cut, but properly handles CSV column quoting:

<pre><code>csvfilter -f 1,3,5 in.csv &gt; out.csv
</code></pre>

If you have python (and you should), you can install it simply like this:

<pre><code>pip install csvfilter
</code></pre>

More info at <a href="https://github.com/codeinthehole/csvfilter/" rel="noreferrer">https://github.com/codeinthehole/csvfilter/</a>

blocks|key|941590|text|我发现csvkit很有用，它基于python模块，有很多解析复杂csv文件的选项。|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|941591|虽然看起来有点慢。当从带有5列的7GB+csv中提取一个字段时，我得到了4MB/s+(使用100%25+cpu)。|941592|从file.csv中提取第4列|style|CODE|941593|csvcut+-c+4+file.csv|code-block|syntax|javascript|941594|entityMap|0|LINK|mutability|MUTABLE|url|https://csvkit.readthedocs.io/en/1.0.3/^0|3|6|0|0|0|1|8|0|0^^$0|@$1|2|3|4|5|6|7|W|8|@]|9|@$A|X|B|Y|1|Z]]|C|$]]|$1|D|3|E|5|6|7|10|8|@]|9|@]|C|$]]|$1|F|3|G|5|6|7|11|8|@$A|12|B|13|H|I]]|9|@]|C|$]]|$1|J|3|K|5|L|7|14|8|@]|9|@]|C|$M|N]]|$1|O|3|-4|5|6|7|15|8|@]|9|@]|C|$]]]|P|$Q|$5|R|S|T|C|$U|V]]]]

I found <a href="https://csvkit.readthedocs.io/en/1.0.3/" rel="nofollow noreferrer">csvkit</a> to be useful, it is based on python csv module and has quite a lot of options for parsing complex csv files.

Although it seems to be a bit slow. I am getting 4MB/s (with 100% cpu) when extracting one field from a 7GB csv with 5 columns. 

To extract 4th column from <code>file.csv</code>

<pre><code>csvcut -c 4 file.csv
</code></pre>

blocks|key|942100|text|尝试一下压碎工具，它们在处理分隔数据方面非常出色。听起来就像你要找的东西。|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|942101|entityMap|0|LINK|mutability|MUTABLE|url|http://code.google.com/p/crush-tools/^0|4|4|0|0^^$0|@$1|2|3|4|5|6|7|L|8|@]|9|@$A|M|B|N|1|O]]|C|$]]|$1|D|3|-4|5|6|7|P|8|@]|9|@]|C|$]]]|E|$F|$5|G|H|I|C|$J|K]]]]

Try <a href="http://code.google.com/p/crush-tools/" rel="noreferrer">crush-tools</a>, they are great at manipulating delimited data.
It sounds like exactly what you're looking for.

blocks|key|820844|text|我的本能反应是围绕Python的csv模块编写一个脚本包装器(如果还没有这样的东西)。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|820845|entityMap^0|0^^$0|@$1|2|3|4|5|6|7|D|8|@]|9|@]|A|$]]|$1|B|3|-4|5|6|7|E|8|@]|9|@]|A|$]]]|C|$]]

My gut reaction would be to write a script wrapper around Python's csv module (if there isn't already such a thing).

blocks|key|820873|text|我还编写了其中一个工具(仅用于UNIX)，名为csvprintf。它还可以以在线方式转换为XML。|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|820874|entityMap|0|LINK|mutability|MUTABLE|url|http://code.google.com/p/csvprintf/^0|N|9|0|0^^$0|@$1|2|3|4|5|6|7|L|8|@]|9|@$A|M|B|N|1|O]]|C|$]]|$1|D|3|-4|5|6|7|P|8|@]|9|@]|C|$]]]|E|$F|$5|G|H|I|C|$J|K]]]]

I wrote one of these tools too (UNIX only) called <a href="http://code.google.com/p/csvprintf/" rel="nofollow">csvprintf</a>. It can also converts to XML in an online fashion.

blocks|key|2347093|text|Perl脚本(需要Text::CSV_XS)：|type|unstyled|depth|inlineStyleRanges|entityRanges|data|2347094|#!/usr/bin/perl

use+strict;
use+warnings;

use+Getopt::Long;
my+@opt_columns;
GetOptions("column=i@"+=>+\@opt_columns)
++or+die+"Failed+parsing+options\n";
die+"Must+give+at+least+one+--column\n"+if+int(@opt_columns)+==+0;
@opt_columns+=+map+{+$_-1+}+@opt_columns;+#+convert+1-based+to+0-based

use+Text::CSV_XS;
my+$csv+=+Text::CSV_XS->new+(+{+binary+=>+1+}+);

open(my+$stdin,+"<-")+or+die+"Couldn't+open+stdin\n";
open(my+$stdout,+">-")+or+die+"Couldn't+open+stdout\n";
while+(my+$row+=+$csv->getline($stdin))+{
++++my+@nrow+=+@{$row}[@opt_columns];
++++$csv->print($stdout,+\@nrow);
++++print+"\n";
}|code-block|syntax|javascript|2347095|将其放入文件csvcut.pl中。|offset|length|style|CODE|2347096|仅取第3和第4列的示例：|2347097|cat+foo.csv+%7C+./csvcut.pl+--c+3+--c+4|2347098|这只会引用需要引用的列，所以如果输入列有"+Bar“(带引号)，那么就会出现Bar(没有引号)。|2347099|entityMap^0|0|0|6|9|0|0|0|0^^$0|@$1|2|3|4|5|6|7|U|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|V|8|@]|9|@]|A|$E|F]]|$1|G|3|H|5|6|7|W|8|@$I|X|J|Y|K|L]]|9|@]|A|$]]|$1|M|3|N|5|6|7|Z|8|@]|9|@]|A|$]]|$1|O|3|P|5|D|7|10|8|@]|9|@]|A|$E|F]]|$1|Q|3|R|5|6|7|11|8|@]|9|@]|A|$]]|$1|S|3|-4|5|6|7|12|8|@]|9|@]|A|$]]]|T|$]]

Perl script (requires Text::CSV_XS):

<pre><code>#!/usr/bin/perl

use strict;
use warnings;

use Getopt::Long;
my @opt_columns;
GetOptions("column=i@" =&gt; \@opt_columns)
 or die "Failed parsing options\n";
die "Must give at least one --column\n" if int(@opt_columns) == 0;
@opt_columns = map { $_-1 } @opt_columns; # convert 1-based to 0-based

use Text::CSV_XS;
my $csv = Text::CSV_XS-&gt;new ( { binary =&gt; 1 } );

open(my $stdin, "&lt;-") or die "Couldn't open stdin\n";
open(my $stdout, "&gt;-") or die "Couldn't open stdout\n";
while (my $row = $csv-&gt;getline($stdin)) {
 my @nrow = @{$row}[@opt_columns];
 $csv-&gt;print($stdout, \@nrow);
 print "\n";
}
</code></pre>

Put it in a file <code>csvcut.pl</code>.

Example of taking only columns 3 and 4:

<pre><code>cat foo.csv | ./csvcut.pl --c 3 --c 4
</code></pre>

This will only quote columns that need quoting, so if an input column has "Bar" (with quotes) it will come out Bar (without quotes).

blocks|key|820982|text|对于Python的csv模块的超级轻量级包装器，您可以查看毛绒。|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|820983|entityMap|0|LINK|mutability|MUTABLE|url|https://pypi.python.org/pypi/pluckr^0|9|3|T|2|0|0^^$0|@$1|2|3|4|5|6|7|N|8|@$9|O|A|P|B|C]]|D|@$9|Q|A|R|1|S]]|E|$]]|$1|F|3|-4|5|6|7|T|8|@]|D|@]|E|$]]]|G|$H|$5|I|J|K|E|$L|M]]]]

For a super lightweight wrapper around Python's <code>csv</code> module, you could look at <a href="https://pypi.python.org/pypi/pluckr" rel="nofollow">pluckr</a>.

blocks|key|821045|text|ffe是另一个很好的工具。它要求您为大多数重要任务创建一个配置文件。好处是它非常灵活，能够处理其他工具无法处理的各种结构、逻辑和格式。|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|821046|我喜欢将csvtool用于快速作业，而ffe用于复杂作业或需要频繁重复的作业。|821047|entityMap|0|LINK|mutability|MUTABLE|url|http://ff-extractor.sourceforge.net/^0|0|3|0|3|0|0|4|7|J|3|0^^$0|@$1|2|3|4|5|6|7|P|8|@$9|Q|A|R|B|C]]|D|@$9|S|A|T|1|U]]|E|$]]|$1|F|3|G|5|6|7|V|8|@$9|W|A|X|B|C]|$9|Y|A|Z|B|C]]|D|@]|E|$]]|$1|H|3|-4|5|6|7|10|8|@]|D|@]|E|$]]]|I|$J|$5|K|L|M|E|$N|O]]]]

<a href="http://ff-extractor.sourceforge.net/" rel="nofollow"><code>ffe</code></a> is another great tool. It requires you to create a configuration file for most non-trivial tasks. The upside is that it's very flexible and can handle all sorts of structure, logic, and formatting that other tools can't.

I like to use <code>csvtool</code> for quick jobs and use <code>ffe</code> for complex jobs or jobs that require frequent repeating.

blocks|key|820726|text|一个快速的谷歌显示，一个awk脚本似乎处理csv文件。|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|820727|entityMap|0|LINK|mutability|MUTABLE|url|http://lorance.freeshell.org/csv/^0|C|5|0|0^^$0|@$1|2|3|4|5|6|7|L|8|@]|9|@$A|M|B|N|1|O]]|C|$]]|$1|D|3|-4|5|6|7|P|8|@]|9|@]|C|$]]]|E|$F|$5|G|H|I|C|$J|K]]]]

A quick google reveals an <a href="http://lorance.freeshell.org/csv/" rel="nofollow noreferrer">awk script</a> that seems to handle csv files.

blocks|key|820652|text|这听起来像是为awk做的工作。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|820653|您很可能需要为您的特定需求编写自己的脚本，但是这站点有一些关于如何这样做的对话。|offset|length|820654|您还可以使用cut实用程序将字段去掉。|820655|类似于：|820656|cut+-f+2,5,6+-d+,+filename|code-block|syntax|javascript|820657|其中，-f参数是您想要的字段，而-d是您想要的参数。然后，您可以对这些结果进行排序，找到唯一的结果，或者使用任何其他bash实用程序。关于使用命令行中的CSV文件，有一个很酷的视频这里。只要一分钟，我就会看一看。|820658|但是，我想您可以使用awk对剪切实用程序进行分组，而不想使用它。不过，我真的不知道您所说的本机bash命令到底是什么意思，所以我仍然建议您这样做。|820659|entityMap|0|LINK|mutability|MUTABLE|url|http://www.nzgames.com/forums/archive/index.php/t-73094.html|1|http://www.linuxjournal.com/video/working-csv-files-command-line^0|0|N|1|0|0|0|0|0|2I|2|1|0|0^^$0|@$1|2|3|4|5|6|7|12|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|13|8|@]|9|@$D|14|E|15|1|16]]|A|$]]|$1|F|3|G|5|6|7|17|8|@]|9|@]|A|$]]|$1|H|3|I|5|6|7|18|8|@]|9|@]|A|$]]|$1|J|3|K|5|L|7|19|8|@]|9|@]|A|$M|N]]|$1|O|3|P|5|6|7|1A|8|@]|9|@$D|1B|E|1C|1|1D]]|A|$]]|$1|Q|3|R|5|6|7|1E|8|@]|9|@]|A|$]]|$1|S|3|-4|5|6|7|1F|8|@]|9|@]|A|$]]]|T|$U|$5|V|W|X|A|$Y|Z]]|10|$5|V|W|X|A|$Y|11]]]]

This sounds like a job for awk. 

You will most likely need to write your own script for your specific needs, but <a href="http://www.nzgames.com/forums/archive/index.php/t-73094.html" rel="nofollow noreferrer">this</a> site has some dialogue about how to go about doing this.

You could also use the cut utility to strip the fields out.

Something like:

<pre><code>cut -f 2,5,6 -d , filename
</code></pre>

where the -f argument is the field you want and -d is the delimeter you want. You could then sort these results, find the unique ones, or use any other bash utility. There is a cool video <a href="http://www.linuxjournal.com/video/working-csv-files-command-line" rel="nofollow noreferrer">here</a> about working with CSV files from the command line. Only about a minute, I'd take a look.

However, I guess you could group the cut utility with awk and not want to use it. I don't really know what exactly you mean by native bash command though, so I'll still suggest it.

How can I parse CSV files on the Linux command line?
To do things like:
<pre><code>csvparse -c 2,5,6 filename
</code></pre>
to extract fields from columns 2, 5 and 6 from all rows.
It should be able to handle the csv file format: <a href="https://www.rfc-editor.org/rfc/rfc4180" rel="nofollow noreferrer">https://www.rfc-editor.org/rfc/rfc4180</a> which means quoting fields and escaping inner quotes as appropriate, so for an example row with 3 fields:
<pre><code>field1,&quot;field, number &quot;&quot;2&quot;&quot;, has inner quotes and a comma&quot;,field3
</code></pre>
so that if I request field 2 for the row above I get:
<pre><code>field, number &quot;2&quot;, has inner quotes and a comma
</code></pre>
I appreciate that there are numerous solutions, Perl, Awk (etc.) to this problem but I would like a native bash command line tool that does not require me to invoke some other scripting environment or write any additional code(!).

How can I parse CSV files on the Linux command line?

翻译质量差，导致语言生硬或混乱。

没有提供实际的解决方法或示例。

解答不清晰，无法理解或解决问题。

页面排版不美观，阅读体验差。

文章

问答

视频

学习中心

腾讯云实验室

直播

竞赛

腾讯云架构师技术同盟交流圈

腾讯云数据库专区

腾讯云顾问专区

腾讯云原生专区

腾讯混元专区

腾讯云TCE专区

腾讯云Lighthouse专区

腾讯云HAI专区

腾讯云Edgeone专区

腾讯云存储专区

腾讯云智能专区

腾讯轻联专区 

腾讯云开发专区

TAPD专区

腾讯轻量云游戏服专区

腾讯云最具价值专家

腾讯云架构师技术同盟

腾讯云创作之星

腾讯云开发者先锋 

腾讯云代码助手

CODING DevOps

Cloud Studio

SDK中心

API中心

命令行工具

如何在Linux命令行上解析CSV文件？做这样的事情：csvparse -c 2,5,6 filename从所有行中提取列2、5和6中的字段。它应该能够处理csv文件格式：，这意味着引用字段和根据适当的转义内部引号，因此对于包含3个字段的示例行：field1,"field, number ""2"", has inner quotes and a comma",field3因此，如果我为上面的行请

问如何在Linux命令行上解析CSV文件？
EN

回答 12

Stack Overflow用户

Stack Overflow用户

Stack Overflow用户

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问如何在Linux命令行上解析CSV文件？EN

回答 12

Stack Overflow用户

Stack Overflow用户

Stack Overflow用户

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问如何在Linux命令行上解析CSV文件？
EN