blocks|key|726566|text|它是无效的UTF-8。该字符是ISO-Latin1中的e-急性字符，这就是它使用该代码集成功的原因。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|726567|如果您不知道正在接收字符串的代码集，那么您就有麻烦了。最好是为您的协议/应用程序选择一个单独的代码集(最好是UTF-8)，然后您就可以拒绝那些无法解码的代码集。|726568|如果你不能做到这一点，你需要启发式。|726569|entityMap^0|0|0|0^^$0|@$1|2|3|4|5|6|7|H|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|I|8|@]|9|@]|A|$]]|$1|D|3|E|5|6|7|J|8|@]|9|@]|A|$]]|$1|F|3|-4|5|6|7|K|8|@]|9|@]|A|$]]]|G|$]]

It is invalid UTF-8. That character is the e-acute character in ISO-Latin1, which is why it succeeds with that codeset.

If you don't know the codeset you're receiving strings in, you're in a bit of trouble. It would be best if a single codeset (hopefully UTF-8) would be chosen for your protocol/application and then you'd just reject ones that didn't decode.

If you can't do that, you'll need heuristics.

blocks|key|726658|text|在二进制中，0xE9看起来像1110+1001。如果您阅读过UTF-8+on+Wikipedia，就会发现这样的字节后面必须跟两个10xx+xxxx形式的字节。所以，举个例子：|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|726659|>>>+b'\xe9\x80\x80'.decode('utf-8')
u'\u9000'|code-block|syntax|javascript|726660|但这只是异常的机械原因。在本例中，您的字符串几乎肯定是用拉丁语1编码的。您可以看到UTF-8和拉丁语1的不同之处：|726661|>>>+u'\xe9'.encode('utf-8')
b'\xc3\xa9'
>>>+u'\xe9'.encode('latin-1')
b'\xe9'|726662|(请注意，我在这里使用的是Python+2和3的混合表示。输入在任何版本的Python中都是有效的，但是Python解释器不太可能以这种方式同时显示unicode和字节字符串。)|726663|entityMap|0|LINK|mutability|MUTABLE|url|http://en.wikipedia.org/wiki/UTF-8#Design^0|E|9|1T|9|U|I|0|0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|Y|8|@$9|Z|A|10|B|C]|$9|11|A|12|B|C]]|D|@$9|13|A|14|1|15]]|E|$]]|$1|F|3|G|5|H|7|16|8|@]|D|@]|E|$I|J]]|$1|K|3|L|5|6|7|17|8|@]|D|@]|E|$]]|$1|M|3|N|5|H|7|18|8|@]|D|@]|E|$I|J]]|$1|O|3|P|5|6|7|19|8|@]|D|@]|E|$]]|$1|Q|3|-4|5|6|7|1A|8|@]|D|@]|E|$]]]|R|$S|$5|T|U|V|E|$W|X]]]]

In binary, 0xE9 looks like <code>1110 1001</code>. If you read about <a href="http://en.wikipedia.org/wiki/UTF-8#Design" rel="noreferrer">UTF-8 on Wikipedia</a>, you’ll see that such a byte must be followed by two of the form <code>10xx xxxx</code>. So, for example:

<pre><code>&gt;&gt;&gt; b'\xe9\x80\x80'.decode('utf-8')
u'\u9000'
</code></pre>

But that’s just the mechanical cause of the exception. In this case, you have a string that is almost certainly encoded in latin 1. You can see how UTF-8 and latin 1 look different:

<pre><code>&gt;&gt;&gt; u'\xe9'.encode('utf-8')
b'\xc3\xa9'
&gt;&gt;&gt; u'\xe9'.encode('latin-1')
b'\xe9'
</code></pre>

(Note, I'm using a mix of Python 2 and 3 representation here. The input is valid in any version of Python, but your Python interpreter is unlikely to actually show both unicode and byte strings in this way.)

blocks|key|726712|text|如果在操作刚打开的文件时出现此错误，请检查是否在'rb'模式下打开了该文件|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|726713|entityMap^0|O|4|0^^$0|@$1|2|3|4|5|6|7|H|8|@$9|I|A|J|B|C]]|D|@]|E|$]]|$1|F|3|-4|5|6|7|K|8|@]|D|@]|E|$]]]|G|$]]

If this error arises when manipulating a file that was just opened, check to see if you opened it in <code>'rb'</code> mode

blocks|key|942891|text|当我从.txt文件中读取包含希伯来语的文本时，这种情况也发生在我身上。|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|942892|我点击了：file+->+save+as，我将这个文件保存为UTF-8编码|942893|entityMap^0|3|4|0|5|F|U|5|0^^$0|@$1|2|3|4|5|6|7|J|8|@$9|K|A|L|B|C]]|D|@]|E|$]]|$1|F|3|G|5|6|7|M|8|@$9|N|A|O|B|C]|$9|P|A|Q|B|C]]|D|@]|E|$]]|$1|H|3|-4|5|6|7|R|8|@]|D|@]|E|$]]]|I|$]]

This happened to me also, while i was reading text containing Hebrew from a <code>.txt</code> file. 

I clicked: <code>file -&gt; save as</code> and I saved this file as a <code>UTF-8</code> encoding

blocks|key|942939|text|当数值范围超过0到127时，通常会出现utf-8代码错误。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|942940|引发此异常的原因是：|942941|1)如果码位<+128，则每个字节与码位的值相同。2)如果码位为128或更大，则Unicode字符串不能用此编码表示。(在这种情况下，Python会引发UnicodeEncodeError异常。)|942942|为了克服这一点，我们有一套编码，使用最广泛的是“拉丁文-1，也称为ISO-8859-1”|942943|因此，ISO-8859-1Unicode点0-+255与拉丁-1值相同，因此转换为这种编码只需要将代码点转换为字节值；如果遇到大于255的代码点，则无法将字符串编码为拉丁-1|942944|当您尝试加载数据集时发生此异常时，请尝试使用以下格式|942945|df=pd.read_csv("top50.csv",encoding='ISO-8859-1')|code-block|syntax|javascript|942946|在语法末尾添加编码技术，然后接受加载数据集。|942947|entityMap^0|0|0|0|0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|U|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|V|8|@]|9|@]|A|$]]|$1|D|3|E|5|6|7|W|8|@]|9|@]|A|$]]|$1|F|3|G|5|6|7|X|8|@]|9|@]|A|$]]|$1|H|3|I|5|6|7|Y|8|@]|9|@]|A|$]]|$1|J|3|K|5|6|7|Z|8|@]|9|@]|A|$]]|$1|L|3|M|5|N|7|10|8|@]|9|@]|A|$O|P]]|$1|Q|3|R|5|6|7|11|8|@]|9|@]|A|$]]|$1|S|3|-4|5|6|7|12|8|@]|9|@]|A|$]]]|T|$]]

utf-8 code error usually comes when the range of numeric values exceeding 0 to 127.

the reason to raise this exception is:

1)If the code point is &lt; 128, each byte is the same as the value of the code point.
2)If the code point is 128 or greater, the Unicode string can’t be represented in this encoding. (Python raises a UnicodeEncodeError exception in this case.)

In order to to overcome this we have a set of encodings, the most widely used is "Latin-1, also known as ISO-8859-1" 

So ISO-8859-1 Unicode points 0–255 are identical to the Latin-1 values, so converting to this encoding simply requires converting code points to byte values; if a code point larger than 255 is encountered, the string can’t be encoded into Latin-1

when this exception occurs when you are trying to load a data set ,try using this format

<pre class="lang-py prettyprint-override"><code>df=pd.read_csv("top50.csv",encoding='ISO-8859-1')
</code></pre>

Add encoding technique at the end of the syntax which then accepts to load the data set.

blocks|key|942972|text|如果显示UTF-8错误，请使用此命令|type|unstyled|depth|inlineStyleRanges|entityRanges|data|942973|pd.read_csv('File_name.csv',encoding='latin-1')|code-block|syntax|javascript|942974|entityMap^0|0|0^^$0|@$1|2|3|4|5|6|7|I|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|J|8|@]|9|@]|A|$E|F]]|$1|G|3|-4|5|6|7|K|8|@]|9|@]|A|$]]]|H|$]]

Use this, If it shows the error of UTF-8 

<pre><code>pd.read_csv('File_name.csv',encoding='latin-1')
</code></pre>

blocks|key|943043|text|当你在pandas中输入特定的文件或数据时，就会出现这种类型的错误，例如:-|type|unstyled|depth|inlineStyleRanges|entityRanges|data|943044|data=pd.read_csv('/kaggle/input/fertilizers-by-product-fao/FertilizersProduct.csv)|code-block|syntax|javascript|943045|则错误显示如下:-+UnicodeDecodeError：'utf-8‘编解码器无法解码位置1中的字节0xf4+:无效的继续字节|943046|因此，为了避免这种类型的错误，可以通过添加一个参数来删除|943047|data=pd.read_csv('/kaggle/input/fertilizers-by-product-fao/FertilizersProduct.csv',+encoding='ISO-8859-1')|943048|entityMap^0|0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|O|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|P|8|@]|9|@]|A|$E|F]]|$1|G|3|H|5|6|7|Q|8|@]|9|@]|A|$]]|$1|I|3|J|5|6|7|R|8|@]|9|@]|A|$]]|$1|K|3|L|5|D|7|S|8|@]|9|@]|A|$E|F]]|$1|M|3|-4|5|6|7|T|8|@]|9|@]|A|$]]]|N|$]]

Well this type of error comes when u are taking input a particular file or data in pandas such as :-
<pre><code>data=pd.read_csv('/kaggle/input/fertilizers-by-product-fao/FertilizersProduct.csv)
</code></pre>
Then the error is displaying like this :-
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf4 in position 1: invalid continuation byte
So to avoid this type of error can be removed by adding an argument
<pre><code>data=pd.read_csv('/kaggle/input/fertilizers-by-product-fao/FertilizersProduct.csv', encoding='ISO-8859-1')
</code></pre>

blocks|key|726925|text|解决方案被更改为"UTF-8+sin“|type|unstyled|depth|inlineStyleRanges|entityRanges|data|726926|entityMap^0|0^^$0|@$1|2|3|4|5|6|7|D|8|@]|9|@]|A|$]]|$1|B|3|-4|5|6|7|E|8|@]|9|@]|A|$]]]|C|$]]

The solution was change to &quot;UTF-8 sin BOM&quot;

Why is the below item failing? Why does it succeed with &quot;latin-1&quot; codec?
<pre><code>o = &quot;a test of \xe9 char&quot; #I want this to remain a string as this is what I am receiving
v = o.decode(&quot;utf-8&quot;)
</code></pre>
Which results in:
<pre><code> Traceback (most recent call last): 
 File &quot;&lt;stdin&gt;&quot;, line 1, in &lt;module&gt; 
 File &quot;C:\Python27\lib\encodings\utf_8.py&quot;,
 line 16, in decode
 return codecs.utf_8_decode(input, errors, True) UnicodeDecodeError:
 'utf8' codec can't decode byte 0xe9 in position 10: invalid continuation byte
</code></pre>

UnicodeDecodeError, invalid continuation byte

解释器

Python

为什么下面的项目失败了？为什么使用“拉丁语-1”编解码器会成功？o = "a test of \xe9 char" #I want this to remain a string as this is what I am receivingv = o.decode("utf-8")这会导致： Traceback (mo...

问UnicodeDecodeError，无效的继续字节
EN

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问UnicodeDecodeError，无效的继续字节EN

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问UnicodeDecodeError，无效的继续字节
EN