blocks|key|707831|text|在第一页上部分填充您的项目，然后将其放入请求的meta中。当下一页的回调被调用时，它可以接受部分填满的请求，在其中放入更多的数据，然后返回它。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|707832|entityMap^0|0^^$0|@$1|2|3|4|5|6|7|D|8|@]|9|@]|A|$]]|$1|B|3|-4|5|6|7|E|8|@]|9|@]|A|$]]]|C|$]]

Partially fill your item on the first page, and the put it in your request's meta. When the callback for the next page is called, it can take the partially filled request, put more data into it, and then return it.

blocks|key|707875|text|有关传递meta数据和请求对象的更多信息，请参阅文档的这一部分：|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|707876|http://readthedocs.org/docs/scrapy/en/latest/topics/request-response.html#passing-additional-data-to-callback-functions|707877|这个问题还涉及到：Scrapy:+Follow+link+to+get+additional+Item+data?|707878|entityMap|0|LINK|mutability|MUTABLE|url|1|https://stackoverflow.com/questions/9334522/scrapy-follow-link-to-get-additional-item-data^0|4|4|0|0|3B|0|0|9|1C|1|0^^$0|@$1|2|3|4|5|6|7|S|8|@$9|T|A|U|B|C]]|D|@]|E|$]]|$1|F|3|G|5|6|7|V|8|@]|D|@$9|W|A|X|1|Y]]|E|$]]|$1|H|3|I|5|6|7|Z|8|@]|D|@$9|10|A|11|1|12]]|E|$]]|$1|J|3|-4|5|6|7|13|8|@]|D|@]|E|$]]]|K|$L|$5|M|N|O|E|$P|G]]|Q|$5|M|N|O|E|$P|R]]]]

More information on passing the <code>meta</code> data and request objects is specifically described in this part of the documentation:

<a href="http://readthedocs.org/docs/scrapy/en/latest/topics/request-response.html#passing-additional-data-to-callback-functions" rel="nofollow noreferrer">http://readthedocs.org/docs/scrapy/en/latest/topics/request-response.html#passing-additional-data-to-callback-functions</a>

This question is also related to: <a href="https://stackoverflow.com/questions/9334522/scrapy-follow-link-to-get-additional-item-data">Scrapy: Follow link to get additional Item data?</a>

blocks|key|492211|text|来自scrapy+documntation的示例|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|492212|def+parse_page1(self,+response):
++++item+=+MyItem()
++++item['main_url']+=+response.url
++++request+=+scrapy.Request("http://www.example.com/some_page.html",
+++++++++++++++++++++++++callback=self.parse_page2)
++++request.meta['item']+=+item
++++return+request

def+parse_page2(self,+response):
++++item+=+response.meta['item']
++++item['other_url']+=+response.url
++++return+item|code-block|syntax|javascript|492213|entityMap|0|LINK|mutability|MUTABLE|url|http://doc.scrapy.org/en/latest/topics/request-response.html^0|2|J|0|0|0^^$0|@$1|2|3|4|5|6|7|Q|8|@]|9|@$A|R|B|S|1|T]]|C|$]]|$1|D|3|E|5|F|7|U|8|@]|9|@]|C|$G|H]]|$1|I|3|-4|5|6|7|V|8|@]|9|@]|C|$]]]|J|$K|$5|L|M|N|C|$O|P]]]]

An example from <a href="http://doc.scrapy.org/en/latest/topics/request-response.html" rel="noreferrer">scrapy documntation</a>

<pre><code>def parse_page1(self, response):
 item = MyItem()
 item['main_url'] = response.url
 request = scrapy.Request("http://www.example.com/some_page.html",
 callback=self.parse_page2)
 request.meta['item'] = item
 return request

def parse_page2(self, response):
 item = response.meta['item']
 item['other_url'] = response.url
 return item
</code></pre>

blocks|key|492254|text|Scrapy文档代码的一个小插图|type|unstyled|depth|inlineStyleRanges|entityRanges|data|492255|def+start_requests(self):
++++++++yield+scrapy.Request("http://www.example.com/main_page.html",callback=parse_page1)
def+parse_page1(self,+response):
++++item+=+MyItem()
++++item['main_url']+=+response.url+##extracts+http://www.example.com/main_page.html
++++request+=+scrapy.Request("http://www.example.com/some_page.html",callback=self.parse_page2)
++++request.meta['my_meta_item']+=+item+##+passing+item+in+the+meta+dictionary
++++##alternatively+you+can+follow+as+below
++++##request+=+scrapy.Request("http://www.example.com/some_page.html",meta={'my_meta_item':item},callback=self.parse_page2)
++++return+request

def+parse_page2(self,+response):
++++item+=+response.meta['my_meta_item']
++++item['other_url']+=+response.url+##extracts+http://www.example.com/some_page.html
++++return+item|code-block|syntax|javascript|492256|entityMap^0|0|0^^$0|@$1|2|3|4|5|6|7|I|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|J|8|@]|9|@]|A|$E|F]]|$1|G|3|-4|5|6|7|K|8|@]|9|@]|A|$]]]|H|$]]

A bit illustration of Scrapy documentation code

<pre><code>def start_requests(self):
 yield scrapy.Request("http://www.example.com/main_page.html",callback=parse_page1)
def parse_page1(self, response):
 item = MyItem()
 item['main_url'] = response.url ##extracts http://www.example.com/main_page.html
 request = scrapy.Request("http://www.example.com/some_page.html",callback=self.parse_page2)
 request.meta['my_meta_item'] = item ## passing item in the meta dictionary
 ##alternatively you can follow as below
 ##request = scrapy.Request("http://www.example.com/some_page.html",meta={'my_meta_item':item},callback=self.parse_page2)
 return request

def parse_page2(self, response):
 item = response.meta['my_meta_item']
 item['other_url'] = response.url ##extracts http://www.example.com/some_page.html
 return item
</code></pre>

Need example in scrapy on how to get a link from one page, then follow this link, get more info from the linked page, and merge back with some data from first page.

How do I merge results from target page to current page in scrapy?

需要在scrapy如何从一个页面获得链接的例子，然后沿着这个链接，从链接的页面获得更多的信息，并合并回来与第一页的一些数据。

问如何在scrapy中将结果从目标页面合并到当前页面？
EN

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问如何在scrapy中将结果从目标页面合并到当前页面？EN

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问如何在scrapy中将结果从目标页面合并到当前页面？
EN