blocks|key|179950|text|问题:+DOM需要<tbody/>标记|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|179951|Chrome的开发工具Firebug、JavaScript中的XPath函数和其他工具都可以在DOM上工作，而不是在基本的HTML源代码上工作。|BOLD|179952|HTML的DOM要求所有未包含在页脚(<thead/>，<tfoot/>)表头中的表行都包含在表体标记<tbody/>中。因此，如果在解析(X)HTML时缺少此标记，浏览器会添加此标记。例如，Microsoft's+DOM+documentation说|179953|179954|即使表没有显式定义tbody元素，也会为所有表公开tbody元素。|blockquote|179955|179956|179957|有一个in-depth+explanation+in+another+answer+on+stackoverflow。|179958|另一方面，HTML+does+not+necessarily+require+that+tag+to+be+used|179959|179960|+TBODY开始标记始终是必需的，除非表格只包含一个表体，并且没有表头或表尾部分。|179961|179962|179963|大多数XPath处理器都在原始XML上工作。|179964|除JavaScript外，大多数XPath处理程序都处理原始的XML，而不是DOM，因此不会添加<tbody/>标记。此外，像tag-soup和htmltidy这样的HTML解析器库只输出XHTML，而不输出"DOM-HTML“。|179965|这是Stackoverflow上发布的一个常见问题，适用于PHP、Ruby、Python、Java、C#、Google+Docs+(电子表格)和许多其他语言。Selenium在浏览器内部运行，并在DOM上工作--因此它不会受到影响！|179966|重现问题|179967|将Firebug+(或Chrome的开发工具)显示的源代码与右键单击并选择“显示页面源代码”(或浏览器中的其他名称)+--或通过在命令行上使用curl+http://your.example.org+--获得的源代码进行比较。后者可能不包含任何<tbody/>元素(它们很少使用)，Firebug将始终显示它们。|179968|179969|解决方案1:删除/tbody轴步骤|179970|检查您当前所在的表是否真的不包含<tbody/>元素(参见最后一段)。如果是这样的话，您可能遇到了另一种问题。|179971|现在删除/tbody轴步骤，这样您的查询将如下所示|179972|//table[@id="example"]/tr[2]/td[1]|code-block|syntax|javascript|179973|解决方案2:跳过<tbody/>标记|179974|这是一个相当糟糕的解决方案，对于嵌套表可能会失败(可以跳到内部表)。我只会在非常罕见的情况下推荐这样做。|179975|将/tbody轴步长替换为后代或自身步长：|179976|//table[@id="example"]//tr[2]/td[1]|179977|解决方案3:允许使用和不使用<tbody/>标记的输入|179978|如果您事先不确定您的表或在"HTML“和DOM上下文中都使用了查询，并且不想/不能使用解决方案2中的技巧，那么可以提供另一个查询(对于XPath+1.0)或使用”可选的“axis步骤(XPath+2.0及更高版本)。|179979|179980|XPath+1.0//table[@id="example"]/tr[2]/td[1]+%7C+//table[@id="example"]/tbody/tr[2]/td[1]|unordered-list-item|179981|XPath+2.0：//table[@id="example"]/(tbody,+.)/tr[2]/td[1]|179982|179983|entityMap|0|LINK|mutability|MUTABLE|url|http://msdn.microsoft.com/en-us/library/ie/ms535902(v=vs.85).aspx|1|https://stackoverflow.com/a/1681427/695343|2|http://www.w3.org/TR/html4/struct/tables.html#edef-TBODY|3|/questions/tagged/tag-soup|4|/questions/tagged/htmltidy^0|9|8|0|1B|3|1P|7|0|J|8|S|8|1F|8|2O|T|0|0|0|9|5|P|5|0|0|0|3|1J|1|0|5|1H|2|0|0|1|5|0|0|0|0|1C|8|1R|8|3|20|8|4|0|27|11|0|0|1Z|S|3E|8|0|0|8|6|0|G|8|0|4|6|0|0|8|8|0|0|1|6|0|0|E|8|0|0|0|0|9|9|25|0|0|9|A|19|0|0^^$0|@$1|2|3|4|5|6|7|2K|8|@$9|2L|A|2M|B|C]]|D|@]|E|$]]|$1|F|3|G|5|6|7|2N|8|@$9|2O|A|2P|B|H]|$9|2Q|A|2R|B|H]]|D|@]|E|$]]|$1|I|3|J|5|6|7|2S|8|@$9|2T|A|2U|B|C]|$9|2V|A|2W|B|C]|$9|2X|A|2Y|B|C]]|D|@$9|2Z|A|30|1|31]]|E|$]]|$1|K|3|-4|5|6|7|32|8|@]|D|@]|E|$]]|$1|L|3|M|5|N|7|33|8|@$9|34|A|35|B|C]|$9|36|A|37|B|C]]|D|@]|E|$]]|$1|O|3|-4|5|6|7|38|8|@]|D|@]|E|$]]|$1|P|3|-4|5|6|7|39|8|@]|D|@]|E|$]]|$1|Q|3|R|5|6|7|3A|8|@]|D|@$9|3B|A|3C|1|3D]]|E|$]]|$1|S|3|T|5|6|7|3E|8|@]|D|@$9|3F|A|3G|1|3H]]|E|$]]|$1|U|3|-4|5|6|7|3I|8|@]|D|@]|E|$]]|$1|V|3|W|5|N|7|3J|8|@$9|3K|A|3L|B|C]]|D|@]|E|$]]|$1|X|3|-4|5|6|7|3M|8|@]|D|@]|E|$]]|$1|Y|3|-4|5|6|7|3N|8|@]|D|@]|E|$]]|$1|Z|3|10|5|6|7|3O|8|@]|D|@]|E|$]]|$1|11|3|12|5|6|7|3P|8|@$9|3Q|A|3R|B|C]]|D|@$9|3S|A|3T|1|3U]|$9|3V|A|3W|1|3X]]|E|$]]|$1|13|3|14|5|6|7|3Y|8|@$9|3Z|A|40|B|H]]|D|@]|E|$]]|$1|15|3|16|5|6|7|41|8|@]|D|@]|E|$]]|$1|17|3|18|5|6|7|42|8|@$9|43|A|44|B|C]|$9|45|A|46|B|C]]|D|@]|E|$]]|$1|19|3|-4|5|6|7|47|8|@]|D|@]|E|$]]|$1|1A|3|1B|5|6|7|48|8|@$9|49|A|4A|B|C]]|D|@]|E|$]]|$1|1C|3|1D|5|6|7|4B|8|@$9|4C|A|4D|B|C]]|D|@]|E|$]]|$1|1E|3|1F|5|6|7|4E|8|@$9|4F|A|4G|B|C]]|D|@]|E|$]]|$1|1G|3|1H|5|1I|7|4H|8|@]|D|@]|E|$1J|1K]]|$1|1L|3|1M|5|6|7|4I|8|@$9|4J|A|4K|B|C]]|D|@]|E|$]]|$1|1N|3|1O|5|6|7|4L|8|@]|D|@]|E|$]]|$1|1P|3|1Q|5|6|7|4M|8|@$9|4N|A|4O|B|C]]|D|@]|E|$]]|$1|1R|3|1S|5|1I|7|4P|8|@]|D|@]|E|$1J|1K]]|$1|1T|3|1U|5|6|7|4Q|8|@$9|4R|A|4S|B|C]]|D|@]|E|$]]|$1|1V|3|1W|5|6|7|4T|8|@]|D|@]|E|$]]|$1|1X|3|-4|5|6|7|4U|8|@]|D|@]|E|$]]|$1|1Y|3|1Z|5|20|7|4V|8|@$9|4W|A|4X|B|H]|$9|4Y|A|4Z|B|C]]|D|@]|E|$]]|$1|21|3|22|5|20|7|50|8|@$9|51|A|52|B|H]|$9|53|A|54|B|C]]|D|@]|E|$]]|$1|23|3|-4|5|6|7|55|8|@]|D|@]|E|$]]|$1|24|3|-4|5|6|7|56|8|@]|D|@]|E|$]]]|25|$26|$5|27|28|29|E|$2A|2B]]|2C|$5|27|28|29|E|$2A|2D]]|2E|$5|27|28|29|E|$2A|2F]]|2G|$5|27|28|29|E|$2A|2H]]|2I|$5|27|28|29|E|$2A|2J]]]]



<h2>The Problem: DOM Requires <code>&lt;tbody/&gt;</code> Tags</h2>

<p>Firebug, Chrome's Developer Tool, XPath functions in JavaScript and others work on the <strong>DOM</strong>, not the basic <strong>HTML source code</strong>.</p>

<p>The DOM for HTML requires that all table rows not contained in a table header of footer (<code>&lt;thead/&gt;</code>, <code>&lt;tfoot/&gt;</code>) are included in table body tags <code>&lt;tbody/&gt;</code>. Thus, browsers add this tag if it's missing while parsing (X)HTML. For example, <a href="http://msdn.microsoft.com/en-us/library/ie/ms535902(v=vs.85).aspx" rel="noreferrer">Microsoft's DOM documentation</a> says</p>

<blockquote>
  <p>The <code>tbody</code> element is exposed for all tables, even if the table does not explicitly define a <code>tbody</code> element.</p>
</blockquote>

<p>There is an <a href="https://stackoverflow.com/a/1681427/695343">in-depth explanation in another answer on stackoverflow</a>.</p>

<p>On the other hand, <a href="http://www.w3.org/TR/html4/struct/tables.html#edef-TBODY" rel="noreferrer">HTML does not necessarily require that tag to be used</a>:</p>

<blockquote>
  <p>The <code>TBODY</code> start tag is always required except when the table contains only one table body and no table head or foot sections.</p>
</blockquote>

<h3>Most XPath Processors Work on raw XML</h3>

<p>Excluding JavaScript, most XPath processors work on raw XML, not the DOM, thus do not add <code>&lt;tbody/&gt;</code> tags. Also HTML parser libraries like <a href="/questions/tagged/tag-soup" class="post-tag" title="show questions tagged &#39;tag-soup&#39;" rel="tag">tag-soup</a> and <a href="/questions/tagged/htmltidy" class="post-tag" title="show questions tagged &#39;htmltidy&#39;" rel="tag">htmltidy</a> only output XHTML, not "DOM-HTML".</p>

<p>This is a common problem posted on Stackoverflow for PHP, Ruby, Python, Java, C#, Google Docs (Spreadsheets) and lots of others. <strong>Selenium runs inside the browser and works on the DOM -- so it is not affected!</strong></p>

<h3>Reproducing the Issue</h3>

<p>Compare the source shown by Firebug (or Chrome's Dev Tools) with the one you get by right-clicking and selecting "Show Page Source" (or whatever it's called in your browsers) -- or by using <code>curl http://your.example.org</code> on the command line. Latter will probably not contain any <code>&lt;tbody/&gt;</code> elements (they're rarely used), Firebug will always show them.</p>

<hr>

<h2>Solution 1: Remove <code>/tbody</code> Axis Step</h2>

<p>Check if the table you're stuck at really does not contain a <code>&lt;tbody/&gt;</code> element (see last paragraph). If it does, you've probably got another kind of problem.</p>

<p>Now remove the <code>/tbody</code> axis step, so your query will look like</p>

<pre class="lang-none prettyprint-override"><code>//table[@id="example"]/tr[2]/td[1]
</code></pre>

<h2>Solution 2: Skip <code>&lt;tbody/&gt;</code> Tags</h2>

<p>This is a rather dirty solution and likely to fail for nested tables (can jump into inner tables). I would only recommend to to this in very rare cases.</p>

<p>Replace the <code>/tbody</code> axis step by a descendant-or-self step:</p>

<pre class="lang-none prettyprint-override"><code>//table[@id="example"]//tr[2]/td[1]
</code></pre>

<h2>Solution 3: Allow Both Input With and Without <code>&lt;tbody/&gt;</code> Tags</h2>

<p>If you're not sure in advance that your table or use the query in both "HTML source" and DOM context; and don't want/cannot use the hack from solution 2, provide an alternative query (for XPath 1.0) or use an "optional" axis step (XPath 2.0 and higher).</p>

<ul>
<li><strong>XPath 1.0</strong>:<br>
<code>//table[@id="example"]/tr[2]/td[1] | //table[@id="example"]/tbody/tr[2]/td[1]</code></li>
<li><strong>XPath 2.0</strong>: <code>//table[@id="example"]/(tbody, .)/tr[2]/td[1]</code></li>
</ul>


blocks|key|179153|text|只是遇到了同样的问题。我几乎写了一个递归函数来检查每个tbody标记是否存在，并以这种方式遍历dom，然后我记起我知道regex。:)|type|unstyled|depth|inlineStyleRanges|entityRanges|data|179154|在解析之前，获取字符串形式的html。使用正则表达式插入缺少的<tbody>和</tbody>标记，然后将其加载回DOMDocument对象。|offset|length|style|CODE|179155|延斯·埃拉特给出了一个很好的解释，但这里是|179156|解决方案4:确保HTML源始终具有带正则表达式的<tbody>标记|179157|JavaScript
++++var+html+=+'<html><table><tr><td>foo</td><td>bar</td></tr></table></html>';
++++html.replace(/(<table([%5E>]%2B)?>([%5E<>]%2B)?)(?!<tbody([%5E>]%2B)?>)/g,"$1<tbody>").replace(/(<(?!(\/tbody))([%5E>]%2B)?>)(<\/table([%5E>]%2B)?>)/g,"$1</tbody>$4");

PHP
++++$html+=+$dom->saveHTML();
++++$html+=+preg_replace(array('/(<table([%5E>]%2B)?>([%5E<>]%2B)?)(?!<tbody([%5E>]%2B)?>)/','/(<(?!(\/tbody))([%5E>]%2B)?>)(<\/table([%5E>]%2B)?>)/'),array('$1<tbody>','$1</tbody>$4'),$html);
++++$dom->loadHTML($html);|code-block|syntax|javascript|179158|只有正则表达式：|179159|matches+`<table>`+tag+with+whatever+else+junk+inside+the+tag+and+between+this+and+the+next+tag+if+the+next+tag+is+NOT+`<tbody>`+also+with+stuff+inside+the+tag

++++/(<table([%5E>]%2B)?>([%5E<>]%2B)?)(?!<tbody([%5E>]%2B)?>)/

replace+with

++++$1<tbody>

the+$1+referencing+the+captured+`<table>`+tag+with+contents.
Do+the+same+for+the+closing+tag+like+this:

++++/(<(?!(\/tbody))([%5E>]%2B)?>)(<\/table([%5E>]%2B)?>)/

replace+with

++++$1</tbody>$4|179160|这样，dom将在必要的地方始终具有<tbody>标记。|179161|entityMap^0|0|V|7|13|8|0|0|O|7|0|0|0|0|H|7|0^^$0|@$1|2|3|4|5|6|7|Y|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|Z|8|@$D|10|E|11|F|G]|$D|12|E|13|F|G]]|9|@]|A|$]]|$1|H|3|I|5|6|7|14|8|@]|9|@]|A|$]]|$1|J|3|K|5|6|7|15|8|@$D|16|E|17|F|G]]|9|@]|A|$]]|$1|L|3|M|5|N|7|18|8|@]|9|@]|A|$O|P]]|$1|Q|3|R|5|6|7|19|8|@]|9|@]|A|$]]|$1|S|3|T|5|N|7|1A|8|@]|9|@]|A|$O|P]]|$1|U|3|V|5|6|7|1B|8|@$D|1C|E|1D|F|G]]|9|@]|A|$]]|$1|W|3|-4|5|6|7|1E|8|@]|9|@]|A|$]]]|X|$]]

<p>Just came across the same problem. I almost wrote a recursive funtion to check for every tbody tag if it exists and traverse the dom that way, then I remembered I know regex. :)</p>

<p>Before parsing, get the html as a string. Insert missing <code>&lt;tbody&gt;</code> and <code>&lt;/tbody&gt;</code> tags with regex, then load it back into your DOMDocument object.</p>

<p>Jens Erat gives a good explanation, but here is</p>

<h1>Solution 4: Make sure the HTML source always has the <code>&lt;tbody&gt;</code> tags with regex</h1>

<pre><code>JavaScript
    var html = '&lt;html&gt;&lt;table&gt;&lt;tr&gt;&lt;td&gt;foo&lt;/td&gt;&lt;td&gt;bar&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;&lt;/html&gt;';
    html.replace(/(&lt;table([^&gt;]+)?&gt;([^&lt;&gt;]+)?)(?!&lt;tbody([^&gt;]+)?&gt;)/g,"$1&lt;tbody&gt;").replace(/(&lt;(?!(\/tbody))([^&gt;]+)?&gt;)(&lt;\/table([^&gt;]+)?&gt;)/g,"$1&lt;/tbody&gt;$4");

PHP
    $html = $dom-&gt;saveHTML();
    $html = preg_replace(array('/(&lt;table([^&gt;]+)?&gt;([^&lt;&gt;]+)?)(?!&lt;tbody([^&gt;]+)?&gt;)/','/(&lt;(?!(\/tbody))([^&gt;]+)?&gt;)(&lt;\/table([^&gt;]+)?&gt;)/'),array('$1&lt;tbody&gt;','$1&lt;/tbody&gt;$4'),$html);
    $dom-&gt;loadHTML($html);
</code></pre>

<p>Just the regex:</p>

<pre><code>matches `&lt;table&gt;` tag with whatever else junk inside the tag and between this and the next tag if the next tag is NOT `&lt;tbody&gt;` also with stuff inside the tag

    /(&lt;table([^&gt;]+)?&gt;([^&lt;&gt;]+)?)(?!&lt;tbody([^&gt;]+)?&gt;)/

replace with

    $1&lt;tbody&gt;

the $1 referencing the captured `&lt;table&gt;` tag with contents.
Do the same for the closing tag like this:

    /(&lt;(?!(\/tbody))([^&gt;]+)?&gt;)(&lt;\/table([^&gt;]+)?&gt;)/

replace with

    $1&lt;/tbody&gt;$4
</code></pre>

<p>This way the dom will ALWAYS have the <code>&lt;tbody&gt;</code> tags where necessary.</p>




<p><em>This is meant to provide a canonical Q&amp;A to all that similar (but much too specific questions to be a close target candidate) popping up once or twice a week.</em></p>

<p>I'm developing an application that needs to parse a website with tables in it. As deriving XPath expression for scraping web pages is boring and error-prone work, I'd like to use the <strong>XPath extractor feature of Firebug</strong> (or similar tools in other browsers) for this.</p>

<p>Example input looks like this:</p>

<pre class="lang-html prettyprint-override"><code>&lt;!-- snip --&gt;
&lt;table id="example"&gt;
  &lt;tr&gt;
    &lt;th&gt;Example Cell&lt;/th&gt;
    &lt;th&gt;Another one&lt;/th&gt;
  &lt;/tr&gt;
  &lt;tr&gt;
    &lt;td&gt;foobar&lt;/td&gt;
    &lt;td&gt;42&lt;/td&gt;
  &lt;/tr&gt;
&lt;/table&gt;
&lt;!-- snip --&gt;
</code></pre>

<p>I want to extract the first data cell ("foobar"). Firebug proposes the XPath expression</p>

<pre class="lang-none prettyprint-override"><code>//table[@id="example"]/tbody/tr[2]/td[1]
</code></pre>

<p>which <strong>works fine in any XPath tester plugins, but not my own application (no results found)</strong>. If I cut down the query to <code>//table[@id]</code>, it works again.</p>

<p>What's going wrong?</p>


Why does my XPath query (scraping HTML tables) only work in Firebug, but not the application I'm developing?

Python

Java

这意味着为每周出现一到两次的所有类似(但过于具体的问题，而不是目标候选人)提供一个规范的问答。我正在开发一个需要解析包含表格的网站的应用程序。由于派生XPath表达式用于抓取网页是一项枯燥且容易出错的工作，我想使用Firebug的XPath提取器功能(或其他浏览器中的类似工具)来实现这一点。示例输入如下所示：<!-- ...

问为什么我的XPath查询(抓取HTML表)只能在Firebug中工作，而不能在我正在开发的应用程序中工作？
EN

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问为什么我的XPath查询(抓取HTML表)只能在Firebug中工作，而不能在我正在开发的应用程序中工作？EN

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问为什么我的XPath查询(抓取HTML表)只能在Firebug中工作，而不能在我正在开发的应用程序中工作？
EN