blocks|key|244468|text|type|unstyled|depth|inlineStyleRanges|entityRanges|data|244469|当你通过管道把Python程序放到Windows的输出文件中时，它总是使用这个字符集吗？|blockquote|244470|244471|244472|用于输出到管道的默认编码。在我的机器上：|244473|In+[5]:+sys.getdefaultencoding()
Out[5]:+'ascii'|code-block|syntax|javascript|244474|244475|如果没有，是否有解决方法？|244476|244477|244478|import+sys
try:
++++sys.setappdefaultencoding('utf-8')
except:
++++sys+=+reload(sys)
++++sys.setdefaultencoding('utf-8')|244479|现在，所有输出都被编码为'utf-8‘。|244480|我认为处理这种情况的正确方法是|244481|244482|重做了一大堆逻辑|244483|244484|244485|就是将来自internet源的所有数据从服务器或页面编码解码为unicode，然后使用上面所示的变通方法将默认编码设置为utf-8。|offset|length|style|CODE|244486|entityMap^0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|V|7|1O|5|0^^$0|@$1|2|3|-4|4|5|6|1A|7|@]|8|@]|9|$]]|$1|A|3|B|4|C|6|1B|7|@]|8|@]|9|$]]|$1|D|3|-4|4|5|6|1C|7|@]|8|@]|9|$]]|$1|E|3|-4|4|5|6|1D|7|@]|8|@]|9|$]]|$1|F|3|G|4|5|6|1E|7|@]|8|@]|9|$]]|$1|H|3|I|4|J|6|1F|7|@]|8|@]|9|$K|L]]|$1|M|3|-4|4|5|6|1G|7|@]|8|@]|9|$]]|$1|N|3|O|4|C|6|1H|7|@]|8|@]|9|$]]|$1|P|3|-4|4|5|6|1I|7|@]|8|@]|9|$]]|$1|Q|3|-4|4|5|6|1J|7|@]|8|@]|9|$]]|$1|R|3|S|4|J|6|1K|7|@]|8|@]|9|$K|L]]|$1|T|3|U|4|5|6|1L|7|@]|8|@]|9|$]]|$1|V|3|W|4|5|6|1M|7|@]|8|@]|9|$]]|$1|X|3|-4|4|5|6|1N|7|@]|8|@]|9|$]]|$1|Y|3|Z|4|C|6|1O|7|@]|8|@]|9|$]]|$1|10|3|-4|4|5|6|1P|7|@]|8|@]|9|$]]|$1|11|3|-4|4|5|6|1Q|7|@]|8|@]|9|$]]|$1|12|3|13|4|5|6|1R|7|@$14|1S|15|1T|16|17]|$14|1U|15|1V|16|17]]|8|@]|9|$]]|$1|18|3|-4|4|5|6|1W|7|@]|8|@]|9|$]]]|19|$]]

<blockquote>
 When you pipe a Python program to an output file on Windows, does it always use this character set?
</blockquote>

Default encoding used to output to pipe. On my machine:

<pre><code>In [5]: sys.getdefaultencoding()
Out[5]: 'ascii'
</code></pre>

<blockquote>
 If not, is there a workaround? 
</blockquote>

<pre><code>import sys
try:
 sys.setappdefaultencoding('utf-8')
except:
 sys = reload(sys)
 sys.setdefaultencoding('utf-8')
</code></pre>

Now all output is encoded to 'utf-8'.

I think correct way to handle this situation without

<blockquote>
 redo a whole bunch of logic
</blockquote>

is to decode all data from your internet source from server or page encoding to <code>unicode</code>, and then to use workaround shown above to set default encoding to <code>utf-8</code>.

blocks|key|2915336|text|Windows下的大多数程序都假定您使用的是默认的Windows编码，对于英文安装，编码为ISO-8859-1。这也适用于命令窗口输出。遗憾的是，没有办法将默认编码设置为UTF-8+-有为其定义的代码页，但它没有得到很好的支持。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|2915337|有些编辑器会识别文件开头的任何BOM字符，并切换到UTF-8，但这不能保证。|2915338|如果要生成超文本标记语言，则应该包含适当的charset标记；这样浏览器就会正确地解释它。|offset|length|style|CODE|2915339|entityMap^0|0|0|L|7|0^^$0|@$1|2|3|4|5|6|7|L|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|M|8|@]|9|@]|A|$]]|$1|D|3|E|5|6|7|N|8|@$F|O|G|P|H|I]]|9|@]|A|$]]|$1|J|3|-4|5|6|7|Q|8|@]|9|@]|A|$]]]|K|$]]

Most programs under Windows will assume that you're using the default Windows encoding, which will be ISO-8859-1 for an English installation. This goes for the command window output as well. There's no way to set the default encoding to UTF-8 unfortunately - there's a code page defined for it, but it's not well supported.

Some editors will recognize any BOM characters at the start of the file and switch to UTF-8, but that's not guaranteed.

If you're generating HTML you should include the proper <code>charset</code> tag; then the browser will interpret it properly.

blocks|key|2915348|text|从你的评论和问题更新来看，数据似乎是用UTF-8正确编码的。这意味着您只需要告诉浏览器它是UTF-8，或者通过使用BOM，或者更好的方式，通过向HTML文档添加编码信息：|type|unstyled|depth|inlineStyleRanges|entityRanges|data|2915349|<head>
++<meta+http-equiv="Content-Type"+content="text/html;+charset=utf-8">
</head>|code-block|syntax|javascript|2915350|如果文档不是有效的XML，就不应该使用XML声明。|2915351|最好和最可靠的方法是通过超文本传输协议提供文件并适当地设置Content-Type:头。|offset|length|style|CODE|2915352|entityMap^0|0|0|0|T|D|0^^$0|@$1|2|3|4|5|6|7|Q|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|R|8|@]|9|@]|A|$E|F]]|$1|G|3|H|5|6|7|S|8|@]|9|@]|A|$]]|$1|I|3|J|5|6|7|T|8|@$K|U|L|V|M|N]]|9|@]|A|$]]|$1|O|3|-4|5|6|7|W|8|@]|9|@]|A|$]]]|P|$]]

From your comments and question update it seems that the data is correctly encoded in UTF-8. This means you just need to tell your browser it's UTF-8, either by using a BOM, or better, by adding encoding information to your HTML document:

<pre><code>&lt;head&gt;
 &lt;meta http-equiv="Content-Type" content="text/html; charset=utf-8"&gt;
&lt;/head&gt;
</code></pre>

You really shouldn't use an XML declaration if the document is no valid XML.

The best and most reliable way would be to serve the file via HTTP and set the <code>Content-Type:</code> header appropriately.

I have a Python script that loads a web page using <code>urllib2.urlopen</code>, does some various magic, and spits out the results using <code>print</code>. We then run the program on Windows like so:

<pre><code>python program.py &gt; output.htm
</code></pre>

Here's the problem:

The <code>urlopen</code> reads data from an IIS web server which outputs UTF8. It spits out this same data to the output, however certain characters (such as the long hyphen that Word always inserts for you against your will because it's smarter than you) get garbled and end up like <code>â€“</code> instead.

Upon further investigation, I noticed even though the web server spits out UTF8 data, the <code>output.htm</code> file is encoded with the ISO-8859-1 character set.

My questions:

<ol>
<li>When you redirect a Python program to an output file on Windows, does it always use this character set?</li>
<li>If so, is there any way to change that behavior?</li>
<li>If not, is there a workaround? I suppose I could just pass in <code>output.htm</code> as a command line parameter and write to that file instead of the screen, but I'd have to redo a whole bunch of logic in my program.</li>
</ol>

Thanks for any help!

UPDATE:

At the top of <code>output.htm</code> I added:

<pre><code>&lt;?xml version="1.0" encoding="UTF-8"?&gt;
&lt;!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd"&gt;
</code></pre>

However, it makes no difference. The characters are still garbled. If I manually switch over to UTF-8 in Firefox, the file displays correctly. Both IE and FF think this file is Western ISO even though it is clearly not.

Peter Piper piped a Python program - and lost all his unicode characters

服务器

Windows

Python

我有一个Python脚本，它使用urllib2.urlopen加载网页，执行一些不同的魔术，然后使用print输出结果。然后我们在Windows上运行该程序，如下所示：python program.py > output.htm问题来了：urlopen从输出UTF8的IIS服务器读取数据。它会在输出中显示相同的数据，但...

问Peter Piper编写了一个Python程序，结果丢失了所有的unicode字符
EN

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问Peter Piper编写了一个Python程序，结果丢失了所有的unicode字符EN

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问Peter Piper编写了一个Python程序，结果丢失了所有的unicode字符
EN