blocks|key|615378|text|你可以使用标准的gnu+utils文件和awk获取你需要的东西。示例：|type|unstyled|depth|inlineStyleRanges|entityRanges|data|615379|file+-bi+.xsession-errors给我：“文本/纯文本；字符集=us-ascii”|offset|length|style|CODE|615380|所以file+-bi+.xsession-errors+%7Cawk+-F+"="+'{print+$2}'给了我"us-ascii“|615381|我在脚本中使用它，像这样：|615382|CHARSET="$(file+-bi+"$i"%7Cawk+-F+"="+'{print+$2}')"

if+[+"$CHARSET"+!=+utf-8+];+then
++iconv+-f+"$CHARSET"+-t+utf8+"$i"+-o+outfile
fi|code-block|syntax|javascript|615383|entityMap^0|0|0|P|0|2|1E|0|0|0^^$0|@$1|2|3|4|5|6|7|S|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|T|8|@$D|U|E|V|F|G]]|9|@]|A|$]]|$1|H|3|I|5|6|7|W|8|@$D|X|E|Y|F|G]]|9|@]|A|$]]|$1|J|3|K|5|6|7|Z|8|@]|9|@]|A|$]]|$1|L|3|M|5|N|7|10|8|@]|9|@]|A|$O|P]]|$1|Q|3|-4|5|6|7|11|8|@]|9|@]|A|$]]]|R|$]]

You can get what you need using standard gnu utils file and awk. Example:

<code>file -bi .xsession-errors</code>
gives me:
"text/plain; charset=us-ascii"

so <code>file -bi .xsession-errors |awk -F "=" '{print $2}'</code>
gives me
"us-ascii"

I use it in scripts like so:

<pre class="lang-sh prettyprint-override"><code>CHARSET="$(file -bi "$i"|awk -F "=" '{print $2}')"

if [ "$CHARSET" != utf-8 ]; then
 iconv -f "$CHARSET" -t utf8 "$i" -o outfile
fi
</code></pre>

blocks|key|399930|text|把它们都编译起来。转到目录，创建dir2utf8.sh|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|399931|#!/bin/bash
#+converting+all+files+in+a+dir+to+utf8

for+f+in+*
do
++if+test+-f+$f+then
++++echo+-e+"\nConverting+$f"
++++CHARSET="$(file+-bi+"$f"%7Cawk+-F+"="+'{print+$2}')"
++++if+[+"$CHARSET"+!=+utf-8+];+then
++++++iconv+-f+"$CHARSET"+-t+utf8+"$f"+-o+"$f"
++++fi
++else
++++echo+-e+"\nSkipping+$f+-+it's+a+regular+file";
++fi
done|code-block|syntax|javascript|399932|entityMap^0|G|B|0|0^^$0|@$1|2|3|4|5|6|7|M|8|@$9|N|A|O|B|C]]|D|@]|E|$]]|$1|F|3|G|5|H|7|P|8|@]|D|@]|E|$I|J]]|$1|K|3|-4|5|6|7|Q|8|@]|D|@]|E|$]]]|L|$]]

Compiling all them. Go to dir, create <code>dir2utf8.sh</code>:

<pre class="lang-sh prettyprint-override"><code>#!/bin/bash
# converting all files in a dir to utf8

for f in *
do
 if test -f $f then
 echo -e "\nConverting $f"
 CHARSET="$(file -bi "$f"|awk -F "=" '{print $2}')"
 if [ "$CHARSET" != utf-8 ]; then
 iconv -f "$CHARSET" -t utf8 "$f" -o "$f"
 fi
 else
 echo -e "\nSkipping $f - it's a regular file";
 fi
done
</code></pre>

blocks|key|399994|text|下面是使用recode和uchardet放置所有文件的my+solution|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|399995|#!/bin/bash

apt-get+-y+install+recode+uchardet+>+/dev/null
find+"$1"+-type+f+%7C+while+read+FFN+#+'dir'+should+be+changed...
do
++encoding=$(uchardet+"$FFN")
++echo+"$FFN:+$encoding"
++enc=`echo+$encoding+%7C+sed+'s#%5Ex-mac-#mac#'`
++set+%2Bx
++recode+$enc..UTF-8+"$FFN"
done|code-block|syntax|javascript|399996|将其放入convert-dir-to-utf8.sh并运行：|style|CODE|399997|bash+convert-dir-to-utf8.sh+/pat/to/my/trash/dir|399998|请注意，sed是这里针对mac编码的一种变通方法。许多不常见的编码都需要这样的变通方法。|399999|entityMap|0|LINK|mutability|MUTABLE|url|https://github.com/rrthomas/recode/|1|https://www.freedesktop.org/wiki/Software/uchardet/|2|https://gist.github.com/demofly/25f856a96c29b89baa32^0|5|6|0|C|8|1|R|B|2|0|0|4|M|0|0|4|3|0^^$0|@$1|2|3|4|5|6|7|12|8|@]|9|@$A|13|B|14|1|15]|$A|16|B|17|1|18]|$A|19|B|1A|1|1B]]|C|$]]|$1|D|3|E|5|F|7|1C|8|@]|9|@]|C|$G|H]]|$1|I|3|J|5|6|7|1D|8|@$A|1E|B|1F|K|L]]|9|@]|C|$]]|$1|M|3|N|5|F|7|1G|8|@]|9|@]|C|$G|H]]|$1|O|3|P|5|6|7|1H|8|@$A|1I|B|1J|K|L]]|9|@]|C|$]]|$1|Q|3|-4|5|6|7|1K|8|@]|9|@]|C|$]]]|R|$S|$5|T|U|V|C|$W|X]]|Y|$5|T|U|V|C|$W|Z]]|10|$5|T|U|V|C|$W|11]]]]

Here is <a href="https://gist.github.com/demofly/25f856a96c29b89baa32" rel="nofollow noreferrer">my solution</a> to in place all files using <a href="https://github.com/rrthomas/recode/" rel="nofollow noreferrer">recode</a> and <a href="https://www.freedesktop.org/wiki/Software/uchardet/" rel="nofollow noreferrer">uchardet</a>:
<pre class="lang-sh prettyprint-override"><code>#!/bin/bash

apt-get -y install recode uchardet &gt; /dev/null
find &quot;$1&quot; -type f | while read FFN # 'dir' should be changed...
do
 encoding=$(uchardet &quot;$FFN&quot;)
 echo &quot;$FFN: $encoding&quot;
 enc=`echo $encoding | sed 's#^x-mac-#mac#'`
 set +x
 recode $enc..UTF-8 &quot;$FFN&quot;
done
</code></pre>
put it into <code>convert-dir-to-utf8.sh</code> and run:
<pre><code>bash convert-dir-to-utf8.sh /pat/to/my/trash/dir
</code></pre>
Note that <code>sed</code> is a workaround for mac encodings here.
Many uncommon encodings need workarounds like this.

blocks|key|615503|text|查看可用于linux+cli中的数据转换的工具：https://www.debian.org/doc/manuals/debian-reference/ch11.en.html|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|615504|此外，还有一项任务是找到iconv中可用的完整编码列表。只需运行iconv+--list并发现编码名称与uchardet工具返回的名称不同(例如：uchardet中的x-mac-cyrillic与iconv中的mac-cyrillic+)|style|CODE|615505|entityMap|0|LINK|mutability|MUTABLE|url|https://www.debian.org/doc/manuals/debian-reference/ch11.en.html^0|O|1S|0|0|C|5|W|C|1G|8|21|8|2Q|5|0^^$0|@$1|2|3|4|5|6|7|P|8|@]|9|@$A|Q|B|R|1|S]]|C|$]]|$1|D|3|E|5|6|7|T|8|@$A|U|B|V|F|G]|$A|W|B|X|F|G]|$A|Y|B|Z|F|G]|$A|10|B|11|F|G]|$A|12|B|13|F|G]]|9|@]|C|$]]|$1|H|3|-4|5|6|7|14|8|@]|9|@]|C|$]]]|I|$J|$5|K|L|M|C|$N|O]]]]

Check out tools available for a data convertation in a linux cli: <a href="https://www.debian.org/doc/manuals/debian-reference/ch11.en.html" rel="nofollow">https://www.debian.org/doc/manuals/debian-reference/ch11.en.html</a>

Also, there is a quest to find out a full list of encodings which are available in <code>iconv</code>. Just run <code>iconv --list</code> and find out that encoding names differs from names returned by <code>uchardet</code> tool (for example: x-mac-cyrillic in <code>uchardet</code> vs. mac-cyrillic in <code>iconv</code>)

blocks|key|615544|text|使用iconv和uchardet+(thx+farseerfc)|type|unstyled|depth|inlineStyleRanges|entityRanges|data|615545|鱼壳|615546|cat+your_file++%7C+iconv+-f+(uchardet+your_file+)+-t+UTF-8|code-block|syntax|javascript|615547|bash+shell|615548|cat+your_file++%7C+iconv+-f+$(uchardet+your_file+)+-t+UTF-8|615549|如果使用bash脚本|615550|#!/usr/bin/bash
for+fn+in+"$@"
do
++++iconv+<+"$fn"+-f+$(uchardet+"$fn")+-t+utf8
done|615551|来自ubuntu组的@flowinglight。|615552|entityMap^0|0|0|0|0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|U|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|V|8|@]|9|@]|A|$]]|$1|D|3|E|5|F|7|W|8|@]|9|@]|A|$G|H]]|$1|I|3|J|5|6|7|X|8|@]|9|@]|A|$]]|$1|K|3|L|5|F|7|Y|8|@]|9|@]|A|$G|H]]|$1|M|3|N|5|6|7|Z|8|@]|9|@]|A|$]]|$1|O|3|P|5|F|7|10|8|@]|9|@]|A|$G|H]]|$1|Q|3|R|5|6|7|11|8|@]|9|@]|A|$]]|$1|S|3|-4|5|6|7|12|8|@]|9|@]|A|$]]]|T|$]]

use iconv and uchardet (thx farseerfc)
fish shell
<pre><code>cat your_file | iconv -f (uchardet your_file ) -t UTF-8
</code></pre>
bash shell
<pre class="lang-sh prettyprint-override"><code>cat your_file | iconv -f $(uchardet your_file ) -t UTF-8
</code></pre>
if use bash script
<pre class="lang-sh prettyprint-override"><code>#!/usr/bin/bash
for fn in &quot;$@&quot;
do
 iconv &lt; &quot;$fn&quot; -f $(uchardet &quot;$fn&quot;) -t utf8
done
</code></pre>
by @flowinglight at ubuntu group.

I am trying to point iconv to a directory and all files will be converted UTF-8 regardless of the current encoding

I am using this script but you have to specify what encoding you are going FROM. How can I make it autdetect the current encoding?

dir_iconv.sh

<pre class="lang-sh prettyprint-override"><code>#!/bin/bash

ICONVBIN='/usr/bin/iconv' # path to iconv binary

if [ $# -lt 3 ]
then
 echo "$0 dir from_charset to_charset"
 exit
fi

for f in $1/*
do
 if test -f $f
 then
 echo -e "\nConverting $f"
 /bin/mv $f $f.old
 $ICONVBIN -f $2 -t $3 $f.old &gt; $f
 else
 echo -e "\nSkipping $f - not a regular file";
 fi
done
</code></pre>

terminal line

<pre><code>sudo convert/dir_iconv.sh convert/books CURRENT_ENCODING utf8
</code></pre>

iconv any encoding to UTF-8

数据转换

我正在尝试将iconv指向一个目录，并且无论当前编码如何，所有文件都将转换为UTF-8我正在使用这个脚本，但您必须指定您要使用的编码。如何让它自动检测当前的编码？dir_iconv.sh#!/bin/bashICONVBIN='/usr/bin/iconv' # path to iconv binaryif [ $# ...

问iconv UTF-8的任何编码
EN

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问iconv UTF-8的任何编码EN

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问iconv UTF-8的任何编码
EN