blocks|key|136677|text|使用sed|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|136678|$+sed+-e+'s/[[:punct:]]*//g;s/[[:space:]]\%2B/\n/g'+<+inputfile|code-block|syntax|javascript|136679|基本上，这会删除所有标点符号，并将所有空格替换为换行符。这还假设您的sed风格理解\n。有些则不是--在这种情况下，您可以只使用文字换行符(即，通过将其嵌入到引号中)。|136680|entityMap^0|2|3|0|0|Y|3|15|2|0^^$0|@$1|2|3|4|5|6|7|O|8|@$9|P|A|Q|B|C]]|D|@]|E|$]]|$1|F|3|G|5|H|7|R|8|@]|D|@]|E|$I|J]]|$1|K|3|L|5|6|7|S|8|@$9|T|A|U|B|C]|$9|V|A|W|B|C]]|D|@]|E|$]]|$1|M|3|-4|5|6|7|X|8|@]|D|@]|E|$]]]|N|$]]

Using <code>sed</code>:

<pre><code>$ sed -e 's/[[:punct:]]*//g;s/[[:space:]]\+/\n/g' &lt; inputfile
</code></pre>

basically this deletes all punctuation and replaces any spaces with newlines. This also assumes your flavor of <code>sed</code> understands <code>\n</code>. Some do not -- in which case you can just use a literal newline instead (i.e. by embedding it inside your quotes).

blocks|key|136699|text|首先，一个非常简单的选择是，|type|unstyled|depth|inlineStyleRanges|entityRanges|data|136700|sed+'s,$\w*$,\1\n,g'+file|code-block|syntax|javascript|136701|注意它既不能处理撇号也不能处理标点符号|136702|entityMap^0|0|0|0^^$0|@$1|2|3|4|5|6|7|K|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|L|8|@]|9|@]|A|$E|F]]|$1|G|3|H|5|6|7|M|8|@]|9|@]|A|$]]|$1|I|3|-4|5|6|7|N|8|@]|9|@]|A|$]]]|J|$]]

A very simple option would first be,

<pre><code>sed 's,$\w*$,\1\n,g' file
</code></pre>

beware it doens't handle neither apostrophes nor punctuation

blocks|key|354500|text|使用perl|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|354501|perl+-ne+'print+join("\n",+split)'+<+file|354502|entityMap^0|2|4|0|0|15|0^^$0|@$1|2|3|4|5|6|7|J|8|@$9|K|A|L|B|C]]|D|@]|E|$]]|$1|F|3|G|5|6|7|M|8|@$9|N|A|O|B|C]]|D|@]|E|$]]|$1|H|3|-4|5|6|7|P|8|@]|D|@]|E|$]]]|I|$]]

Using <code>perl</code>:

<code>perl -ne 'print join("\n", split)' &lt; file</code>

blocks|key|136734|text|cat+input.txt+%7C+tr+-d+",."+%7C+tr+"+\t"+"\n"+%7C+grep+-e+"%5E$"+-v|type|code-block|depth|inlineStyleRanges|entityRanges|data|syntax|javascript|136735|tr+-d+",."删除,和.|unstyled|offset|length|style|CODE|136736|tr+"+\t"+"\n"将空格和制表符更改为换行符|136737|grep+-e+"%5E$"+-v删除空行(如果有两个或更多空格)|136738|entityMap^0|0|0|A|C|1|E|1|0|0|D|0|0|F|0^^$0|@$1|2|3|4|5|6|7|Q|8|@]|9|@]|A|$B|C]]|$1|D|3|E|5|F|7|R|8|@$G|S|H|T|I|J]|$G|U|H|V|I|J]|$G|W|H|X|I|J]]|9|@]|A|$]]|$1|K|3|L|5|F|7|Y|8|@$G|Z|H|10|I|J]]|9|@]|A|$]]|$1|M|3|N|5|F|7|11|8|@$G|12|H|13|I|J]]|9|@]|A|$]]|$1|O|3|-4|5|F|7|14|8|@]|9|@]|A|$]]]|P|$]]

<pre><code>cat input.txt | tr -d ",." | tr " \t" "\n" | grep -e "^$" -v
</code></pre>

<code>tr -d ",."</code> deletes <code>,</code> and <code>.</code>

<code>tr " \t" "\n"</code> changes spaces and tabs to newlines

<code>grep -e "^$" -v</code> deletes empty lines (in case of two or more spaces)

blocks|key|354552|text|使用tr：|type|unstyled|depth|inlineStyleRanges|entityRanges|data|354553|tr+-s+'[[:punct:][:space:]]'+'\n'+<+file|code-block|syntax|javascript|354554|entityMap^0|0|0^^$0|@$1|2|3|4|5|6|7|I|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|J|8|@]|9|@]|A|$E|F]]|$1|G|3|-4|5|6|7|K|8|@]|9|@]|A|$]]]|H|$]]

Using tr:

<pre><code>tr -s '[[:punct:][:space:]]' '\n' &lt; file
</code></pre>

blocks|key|136779|text|使用perl：|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|136780|perl+-pe+'s/(?:\p{Punct}%7C\s%2B)%2B/\n/g'+file|code-block|syntax|javascript|136781|输出|136782|Hola
mundo
hablo
español
y
no
sé
si
escribí
bien
la
pregunta
ojal�
me
puedan
entender
y
ayudar
Adiós|136783|entityMap|0|LINK|mutability|MUTABLE|url|/questions/tagged/perl^0|2|4|0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|U|8|@]|9|@$A|V|B|W|1|X]]|C|$]]|$1|D|3|E|5|F|7|Y|8|@]|9|@]|C|$G|H]]|$1|I|3|J|5|6|7|Z|8|@]|9|@]|C|$]]|$1|K|3|L|5|F|7|10|8|@]|9|@]|C|$G|H]]|$1|M|3|-4|5|6|7|11|8|@]|9|@]|C|$]]]|N|$O|$5|P|Q|R|C|$S|T]]]]

Using <a href="/questions/tagged/perl" class="post-tag" title="show questions tagged 'perl'" rel="tag">perl</a> :

<pre><code>perl -pe 's/(?:\p{Punct}|\s+)+/\n/g' file
</code></pre>

<h2>Output</h2>

<pre><code>Hola
mundo
hablo
español
y
no
sé
si
escribí
bien
la
pregunta
ojal�
me
puedan
entender
y
ayudar
Adiós
</code></pre>

blocks|key|354582|text|这条awk线路也可以工作吗？|type|unstyled|depth|inlineStyleRanges|entityRanges|data|354583|awk+'BEGIN{FS="[[:punct:]+]*";OFS="\n"}{$1=$1}1'++inputfile|code-block|syntax|javascript|354584|entityMap^0|0|0^^$0|@$1|2|3|4|5|6|7|I|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|J|8|@]|9|@]|A|$E|F]]|$1|G|3|-4|5|6|7|K|8|@]|9|@]|A|$]]]|H|$]]

this awk line may work too?

<pre><code>awk 'BEGIN{FS="[[:punct:] ]*";OFS="\n"}{$1=$1}1' inputfile
</code></pre>

blocks|key|136817|text|grep+-o仅打印与模式匹配的匹配行的部分|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|136818|grep+-o+'[[:alpha:]]*'+file|code-block|syntax|javascript|136819|entityMap^0|0|7|0|0^^$0|@$1|2|3|4|5|6|7|M|8|@$9|N|A|O|B|C]]|D|@]|E|$]]|$1|F|3|G|5|H|7|P|8|@]|D|@]|E|$I|J]]|$1|K|3|-4|5|6|7|Q|8|@]|D|@]|E|$]]]|L|$]]

<code>grep -o</code> prints only the parts of matching line that matches pattern

<pre><code>grep -o '[[:alpha:]]*' file
</code></pre>

blocks|key|136850|text|根据您目前的回答，我认为您可能正在寻找的是将单词视为由空格、逗号、句子结尾字符(即".“"!”)分隔的字符序列。或者"?“英文)和其他通常不会与字母数字字符组合在一起的字符(例如，"<“和";”，但不是'+-+#+$+%25)。现在，“。是一个句子结束字符，但是您说$27.00应该被视为一个“单词”，所以需要根据上下文对.进行不同的处理。我认为对于"-“和其他一些字符可能也是如此。|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|136851|因此，您需要一个解决方案来将以下内容转换为：|136852|I+have+$27.00.+We're+20%25+under-budget,+right?+This+is+#2+-+mail+me+at+"foo@bar.com".|code-block|syntax|javascript|136853|如下所示：|136854|I
have
$27.00
We're
20%25
under-budget
right
This
is
#2
mail
me
at+
foo@bar.com|136855|对吗？|136856|尝试使用GNU+awk，这样我们就可以将RS设置为多个字符：|136857|$+cat+file
I+have+$27.00.+We're+20%25+under-budget,+right?+This+is+#2+-+mail+me+at+"foo@bar.com".

$+gawk+-v+RS="[[:space:]?!]%2B"+'{gsub(/%5E[%5E[:alnum:]$#]%2B%7C[%5E[:alnum:]%25]%2B$/,"")}+$0!=""'+file
I
have
$27.00
We're
20%25
under-budget
right
This
is
#2
mail
me
at
foo@bar.com|136858|试着想出一些其他的测试用例，看看这是否总是能达到你想要的效果。|136859|entityMap^0|2T|1|2V|1|2X|1|2Z|1|31|1|3N|6|4F|1|0|0|0|0|0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|10|8|@$9|11|A|12|B|C]|$9|13|A|14|B|C]|$9|15|A|16|B|C]|$9|17|A|18|B|C]|$9|19|A|1A|B|C]|$9|1B|A|1C|B|C]|$9|1D|A|1E|B|C]]|D|@]|E|$]]|$1|F|3|G|5|6|7|1F|8|@]|D|@]|E|$]]|$1|H|3|I|5|J|7|1G|8|@]|D|@]|E|$K|L]]|$1|M|3|N|5|6|7|1H|8|@]|D|@]|E|$]]|$1|O|3|P|5|J|7|1I|8|@]|D|@]|E|$K|L]]|$1|Q|3|R|5|6|7|1J|8|@]|D|@]|E|$]]|$1|S|3|T|5|6|7|1K|8|@]|D|@]|E|$]]|$1|U|3|V|5|J|7|1L|8|@]|D|@]|E|$K|L]]|$1|W|3|X|5|6|7|1M|8|@]|D|@]|E|$]]|$1|Y|3|-4|5|6|7|1N|8|@]|D|@]|E|$]]]|Z|$]]

Based on your responses so far, I THINK what you probably are looking for is to treat words as sequences of characters separated by spaces, commas, sentence-ending characters (i.e. "." "!" or "?" in English) and other characters that you would NOT normally find in combination with alpha-numeric characters (e.g. "&lt;" and ";" but not <code>'</code> <code>-</code> <code>#</code> <code>$</code> <code>%</code>). Now, "." is a sentence ending character but you said that <code>$27.00</code> should be considered a "word" so <code>.</code> needs to be treated differently depending on context. I think the same is probably true for "-" and maybe some other characters.

So you need a solution that will convert this:

<pre><code>I have $27.00. We're 20% under-budget, right? This is #2 - mail me at "foo@bar.com".
</code></pre>

into this:

<pre><code>I
have
$27.00
We're
20%
under-budget
right
This
is
#2
mail
me
at 
foo@bar.com
</code></pre>

Is that correct?

Try this using GNU awk so we can set RS to more than one character:

<pre><code>$ cat file
I have $27.00. We're 20% under-budget, right? This is #2 - mail me at "foo@bar.com".

$ gawk -v RS="[[:space:]?!]+" '{gsub(/^[^[:alnum:]$#]+|[^[:alnum:]%]+$/,"")} $0!=""' file
I
have
$27.00
We're
20%
under-budget
right
This
is
#2
mail
me
at
foo@bar.com
</code></pre>

Try to come up with some other test cases to see if this always does what you want.

blocks|key|136876|text|最简单的工具是fmt：|type|unstyled|depth|inlineStyleRanges|entityRanges|data|136877|fmt+-1+<your-file|code-block|syntax|javascript|136878|fmt设计用于换行以适合指定的宽度，如果您提供-1，它会紧跟在单词后面换行。有关文档，请参阅man+fmt。灵感来自http://everythingsysadmin.com/2012/09/unorthodoxunix.html|offset|length|style|CODE|136879|entityMap|0|LINK|mutability|MUTABLE|url|http://everythingsysadmin.com/2012/09/unorthodoxunix.html^0|0|0|N|2|1A|7|1M|1L|0|0^^$0|@$1|2|3|4|5|6|7|U|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|V|8|@]|9|@]|A|$E|F]]|$1|G|3|H|5|6|7|W|8|@$I|X|J|Y|K|L]|$I|Z|J|10|K|L]]|9|@$I|11|J|12|1|13]]|A|$]]|$1|M|3|-4|5|6|7|14|8|@]|9|@]|A|$]]]|N|$O|$5|P|Q|R|A|$S|T]]]]

The simplest tool is fmt:

<pre><code>fmt -1 &lt;your-file
</code></pre>

fmt designed to break lines to fit the specified width and if you provide <code>-1</code> it breaks immediately after the word. See <code>man fmt</code> for documentation. Inspired by <a href="http://everythingsysadmin.com/2012/09/unorthodoxunix.html" rel="noreferrer">http://everythingsysadmin.com/2012/09/unorthodoxunix.html</a>

I'm doing a faster tests for a naive boolean information retrival system, and I would like use awk, grep, egrep, sed or thing similiar and pipes for split a text file into words and save them into other file with a word per line. Example my file cotains:
<pre><code>Hola mundo, hablo español y no sé si escribí bien la
pregunta, ojalá me puedan entender y ayudar
Adiós.
</code></pre>
The output file should contain:
<pre><code>Hola
mundo
hablo
español
...
</code></pre>
Thank!

How split a file in words in unix command line?

我正在为一个朴素的布尔信息检索系统做一个更快的测试，我想使用awk，grep，egrep，sed或类似的东西和管道将文本文件分割成单词，并将它们保存到其他文件中，每行一个单词。示例我的文件包含：Hola mundo, hablo español y no sé si escribí bien lapregunta, o...

问如何在unix命令行中拆分文件？
EN

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问如何在unix命令行中拆分文件？EN

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问如何在unix命令行中拆分文件？
EN