blocks|key|2749349|text|查看jchardet|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|2749350|entityMap|0|LINK|mutability|MUTABLE|url|http://jchardet.sourceforge.net/^0|2|8|0|0^^$0|@$1|2|3|4|5|6|7|L|8|@]|9|@$A|M|B|N|1|O]]|C|$]]|$1|D|3|-4|5|6|7|P|8|@]|9|@]|C|$]]]|E|$F|$5|G|H|I|C|$J|K]]]]

Check out <a href="http://jchardet.sourceforge.net/" rel="nofollow noreferrer">jchardet</a>

blocks|key|2539351|text|应该已经有东西可用了|type|unstyled|depth|inlineStyleRanges|entityRanges|data|2539352|谷歌搜索找到了icu4j|2539353|或|2539354|http://jchardet.sourceforge.net/|offset|length|2539355|entityMap|0|LINK|mutability|MUTABLE|url^0|0|0|0|0|W|0|0^^$0|@$1|2|3|4|5|6|7|Q|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|R|8|@]|9|@]|A|$]]|$1|D|3|E|5|6|7|S|8|@]|9|@]|A|$]]|$1|F|3|G|5|6|7|T|8|@]|9|@$H|U|I|V|1|W]]|A|$]]|$1|J|3|-4|5|6|7|X|8|@]|9|@]|A|$]]]|K|$L|$5|M|N|O|A|$P|G]]]]

Should be stuff already available

google search turned up icu4j

or

<a href="http://jchardet.sourceforge.net/" rel="nofollow noreferrer">http://jchardet.sourceforge.net/</a>

blocks|key|54978|text|迟浩田的答案似乎是最有希望真正使用的。我只想补充一点，根据Joel+Spolsky的说法，Internet+Explorer在它的时代使用了基于频率的猜测算法：|type|unstyled|depth|inlineStyleRanges|entityRanges|data|54979|http://www.joelonsoftware.com/articles/Unicode.html|offset|length|54980|粗略地说，所有假定为文本的内容都会被复制，并以任何可以想象到的编码方式进行解析。哪种解析适合一种语言的平均单词(和字母？)频率分布最好，胜出。我不能很快看出jchardet是否使用了同样的方法，所以我想我应该提一下，以防万一。|54981|entityMap|0|LINK|mutability|MUTABLE|url^0|0|0|1F|0|0|0^^$0|@$1|2|3|4|5|6|7|O|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|P|8|@]|9|@$D|Q|E|R|1|S]]|A|$]]|$1|F|3|G|5|6|7|T|8|@]|9|@]|A|$]]|$1|H|3|-4|5|6|7|U|8|@]|9|@]|A|$]]]|I|$J|$5|K|L|M|A|$N|C]]]]

Chi's answer seems most promising for real use. I just want to add that, according to Joel Spolsky, Internet Explorer used a frequency-based guessing algorithm in its day:

<a href="http://www.joelonsoftware.com/articles/Unicode.html" rel="nofollow noreferrer">http://www.joelonsoftware.com/articles/Unicode.html</a>

Roughly speaking, all the assumed-to-be-text is copied, and parsed in every encoding imaginable. Whichever parse fits a language's average word (and letter?) frequency profile best, wins. I can not quickly see if jchardet uses the same kind of approach, so I thought I'd mention this just in case.

blocks|key|2749439|text|如果没有编码指示符，您将永远无法确定。但是，您可以进行一些智能猜测。请看我对这个问题的回答，|type|unstyled|depth|inlineStyleRanges|entityRanges|data|2749440|How+to+determine+if+a+String+contains+invalid+encoded+characters|offset|length|2749441|使用validUTF8()方法。如果返回true，则将其视为UTF8，否则视为拉丁-1。|2749442|entityMap|0|LINK|mutability|MUTABLE|url|https://stackoverflow.com/questions/887148/how-to-determine-if-a-string-contains-invalid-encoded-characters^0|0|0|1S|0|0|0^^$0|@$1|2|3|4|5|6|7|P|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|Q|8|@]|9|@$D|R|E|S|1|T]]|A|$]]|$1|F|3|G|5|6|7|U|8|@]|9|@]|A|$]]|$1|H|3|-4|5|6|7|V|8|@]|9|@]|A|$]]]|I|$J|$5|K|L|M|A|$N|O]]]]

Without encoding indicator, you will never know for sure. However, you can make some intelligent guesses. See my answer to this question,

<a href="https://stackoverflow.com/questions/887148/how-to-determine-if-a-string-contains-invalid-encoded-characters">How to determine if a String contains invalid encoded characters</a>

Use the validUTF8() methods. If it returns true, treat it as UTF8, otherwise as Latin-1.

blocks|key|2749458|text|下面的方法使用juniversalchardet解决了这个问题，它是Mozilla的编码检测库的一个Java端口。|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|2749459|public+static+String+guessEncoding(byte[]+bytes)+{
++++String+DEFAULT_ENCODING+=+"UTF-8";
++++org.mozilla.universalchardet.UniversalDetector+detector+=
++++++++new+org.mozilla.universalchardet.UniversalDetector(null);
++++detector.handleData(bytes,+0,+bytes.length);
++++detector.dataEnd();
++++String+encoding+=+detector.getDetectedCharset();
++++detector.reset();
++++if+(encoding+==+null)+{
++++++++encoding+=+DEFAULT_ENCODING;
++++}
++++return+encoding;
}|code-block|syntax|javascript|2749460|上面的代码已经过测试，并按预期工作。只需将juniversalchardet-1.0.3.jar添加到类路径即可。|2749461|我已经测试了juniversalchardet和jchardet。我的总体印象是，juniversalchardet提供了这两个库更好的检测准确性和更好的API。|2749462|entityMap|0|LINK|mutability|MUTABLE|url|http://code.google.com/p/juniversalchardet/|1|https://code.google.com/archive/p/juniversalchardet/downloads|2|3|http://jchardet.sourceforge.net/^0|7|H|0|0|0|L|R|1|0|6|H|2|O|8|3|0^^$0|@$1|2|3|4|5|6|7|Z|8|@]|9|@$A|10|B|11|1|12]]|C|$]]|$1|D|3|E|5|F|7|13|8|@]|9|@]|C|$G|H]]|$1|I|3|J|5|6|7|14|8|@]|9|@$A|15|B|16|1|17]]|C|$]]|$1|K|3|L|5|6|7|18|8|@]|9|@$A|19|B|1A|1|1B]|$A|1C|B|1D|1|1E]]|C|$]]|$1|M|3|-4|5|6|7|1F|8|@]|9|@]|C|$]]]|N|$O|$5|P|Q|R|C|$S|T]]|U|$5|P|Q|R|C|$S|V]]|W|$5|P|Q|R|C|$S|T]]|X|$5|P|Q|R|C|$S|Y]]]]

The following method solves the problem using <a href="http://code.google.com/p/juniversalchardet/" rel="noreferrer">juniversalchardet</a>, which is a Java port of Mozilla's encoding detection library.

<pre><code>public static String guessEncoding(byte[] bytes) {
 String DEFAULT_ENCODING = "UTF-8";
 org.mozilla.universalchardet.UniversalDetector detector =
 new org.mozilla.universalchardet.UniversalDetector(null);
 detector.handleData(bytes, 0, bytes.length);
 detector.dataEnd();
 String encoding = detector.getDetectedCharset();
 detector.reset();
 if (encoding == null) {
 encoding = DEFAULT_ENCODING;
 }
 return encoding;
}
</code></pre>

The code above has been tested and works as intented. Simply add <a href="https://code.google.com/archive/p/juniversalchardet/downloads" rel="noreferrer">juniversalchardet-1.0.3.jar</a> to the classpath.

I've tested both <a href="http://code.google.com/p/juniversalchardet/" rel="noreferrer">juniversalchardet</a> and <a href="http://jchardet.sourceforge.net/" rel="noreferrer">jchardet</a>. My general impression is that juniversalchardet provides the better detection accuracy and the nicer API of the two libraries.

blocks|key|2749506|text|这是我最喜欢的：https://github.com/codehaus/guessencoding|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|2749507|它是这样工作的：|2749508|2749509|如果存在UTF-8或UTF-16BOM，则返回该编码。|unordered-list-item|2749510|如果没有设置高位的字节，则返回ASCII码(或者您可以强制它返回默认的8位编码)。|2749511|如果存在设置了高位的字节，但它们的排列方式符合UTF-8的正确模式，则返回UTF-8.|2749512|Otherwise，返回平台默认编码(例如，在英语语言环境的Windows系统上为windows-1252+)。|2749513|2749514|这听起来可能过于简单化了，但在我的日常工作中，它的准确率远远超过90%25。|2749515|entityMap|0|LINK|mutability|MUTABLE|url|https://github.com/codehaus/guessencoding^0|8|15|0|0|0|0|0|0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|10|8|@]|9|@$A|11|B|12|1|13]]|C|$]]|$1|D|3|E|5|6|7|14|8|@]|9|@]|C|$]]|$1|F|3|-4|5|6|7|15|8|@]|9|@]|C|$]]|$1|G|3|H|5|I|7|16|8|@]|9|@]|C|$]]|$1|J|3|K|5|I|7|17|8|@]|9|@]|C|$]]|$1|L|3|M|5|I|7|18|8|@]|9|@]|C|$]]|$1|N|3|O|5|I|7|19|8|@]|9|@]|C|$]]|$1|P|3|-4|5|6|7|1A|8|@]|9|@]|C|$]]|$1|Q|3|R|5|6|7|1B|8|@]|9|@]|C|$]]|$1|S|3|-4|5|6|7|1C|8|@]|9|@]|C|$]]]|T|$U|$5|V|W|X|C|$Y|Z]]]]

Here's my favorite: <a href="https://github.com/codehaus/guessencoding" rel="nofollow noreferrer">https://github.com/codehaus/guessencoding</a>

It works like this:

<ul>
<li>If there's a UTF-8 or UTF-16 BOM, return that encoding.</li>
<li>If none of the bytes have the high-order bit set, return ASCII (or you can force it to return a default 8-bit encoding instead).</li>
<li>If there are bytes with the high bit set but they're arranged in the correct patterns for UTF-8, return UTF-8.</li>
<li>Otherwise, return the platform default encoding (e.g., windows-1252 on an English-locale Windows system).</li>
</ul>

It may sound overly simplistic, but in my day-to-day work it's well over 90% accurate.

blocks|key|2539526|text|还有Apache+Tika+-+a+content+analysis+toolkit。它可以猜测mime类型，也可以猜测编码。通常情况下，猜测是正确的，概率很高。|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|2539527|entityMap|0|LINK|mutability|MUTABLE|url|http://tika.apache.org/^0|2|14|0|0^^$0|@$1|2|3|4|5|6|7|L|8|@]|9|@$A|M|B|N|1|O]]|C|$]]|$1|D|3|-4|5|6|7|P|8|@]|9|@]|C|$]]]|E|$F|$5|G|H|I|C|$J|K]]]]

There is also <a href="http://tika.apache.org/" rel="noreferrer">Apache Tika - a content analysis toolkit</a>. It can guess the mime type, and it can guess the encoding. Usually the guess is correct with a very high probability.

Given an array of bytes representing text in some unknown encoding (usually UTF-8 or ISO-8859-1, but not necessarily so), what is the best way to obtain a guess for the most likely encoding used (in Java)?

Worth noting:

<ul>
<li>No additional meta-data is available. The byte array is literally the only available input.</li>
<li>The detection algorithm will obviously not be 100 % correct. If the algorithm is correct in more than say 80 % of the cases that is good enough.</li>
</ul>

Guessing the encoding of text represented as byte[] in Java

翻译质量差，导致语言生硬或混乱。

没有提供实际的解决方法或示例。

解答不清晰，无法理解或解决问题。

页面排版不美观，阅读体验差。

文章

问答

视频

教程

学习中心

腾讯云实验室

直播

竞赛

腾讯云代码分析专区

腾讯iOA零信任安全管理系统专区

腾讯云架构师技术同盟交流圈

腾讯云数据库专区

腾讯云智能顾问专区

腾讯云原生专区

腾讯混元专区

腾讯云TCE专区

腾讯云Lighthouse专区

腾讯云HAI专区

腾讯云Edgeone专区

腾讯云存储专区

腾讯云智能专区

腾讯轻联专区 

腾讯云开发专区

TAPD专区

腾讯轻量云游戏服专区

EdgeOne AI 安全实战专区

腾讯云最具价值专家

腾讯云架构师技术同盟

腾讯云创作之星

腾讯云开发者先锋

腾讯云代码助手

云原生构建

TAPD 敏捷项目管理

Cloud Studio

SDK中心

API中心

命令行工具

涵盖代码开发、场景应用、自动测试全流程，助你从零构建专属AI助手

一站式MCP教程库，解锁AI应用新玩法

聚焦“写作效率、视觉美观与运行性能”三方面进行全面升级，为您提供更高效、稳定的创作环境

社区富文本&Markdown编辑器全新改版上线，欢迎大家体验!

诚挚邀请您参与本次调研，分享您的真实使用感受与建议。您的反馈至关重要，感谢您的支持与参与！

社区新版编辑器体验调研

给定一个表示某种未知编码(通常是UTF-8或ISO-8859-1，但不一定是)的文本的字节数组，猜测最可能使用的编码(在Java中)的最佳方法是什么？值得注意的是：没有其他元数据可用。字面上，字节数组是唯一可用的输入。，检测算法显然不会100 %正确。如果算法在超过80 %的情况下是正确的，那就足够好了。

问猜测Java语言中表示为byte[]的文本的编码
EN

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问猜测Java语言中表示为byte[]的文本的编码EN

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问猜测Java语言中表示为byte[]的文本的编码
EN