前往小程序,Get更优阅读体验!
立即前往
首页
学习
活动
专区
工具
TVP
发布
社区首页 >专栏 >pyquery 库详解安装pyquery 介绍数据初始化基本css选择器查找节点遍历节点获取属性获取文本节点操作伪类选择器

pyquery 库详解安装pyquery 介绍数据初始化基本css选择器查找节点遍历节点获取属性获取文本节点操作伪类选择器

作者头像
章鱼喵
发布2018-09-26 18:03:46
1.5K0
发布2018-09-26 18:03:46
举报
文章被收录于专栏:coding

安装

pyquery官网地址

代码语言:javascript
复制
pip install pyquery

验证安装

代码语言:javascript
复制
In [1]: import pyquery

没报错即表示安装成功

pyquery

pyquery 介绍

虽然 xpath 与 Beautiful Soup 已经很强大了,不过语法还是过于啰嗦,pyquery提供了更加简洁优雅的语法,你可以像写jquery一般提取数据

数据初始化

代码语言:javascript
复制
In [1]: html = '''
   ...: <div>
   ...:     <ul>
   ...:         <li class="item-0">first item</li>
   ...:         <li class="item-1"><a href="link2.html">second item</a></li>
   ...:         <li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>
   ...:         <li class="item-1"><a href="link4.html">fourth item</a></li>
   ...:         <li class="item-0"><a href="link5.html">fifth item</a></li>
   ...:     </ul>
   ...: </div>
   ...: '''

In [2]: from pyquery import PyQuery as pq

In [3]: doc = pq(html)

In [4]: print(doc('li'))
<li class="item-0">first item</li>
        <li class="item-1"><a href="link2.html">second item</a></li>
        <li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>
        <li class="item-1"><a href="link4.html">fourth item</a></li>
        <li class="item-0"><a href="link5.html">fifth item</a></li>

初始化的数据还可以从url中获取

代码语言:javascript
复制
In [11]: doc_url = pq(url='https://www.python.org/')

In [12]: print(doc_url('title'))
<title>Welcome to Python.org</title>

除此,也可以传入文件初始化

代码语言:javascript
复制
doc_file = pq(filename='demo.html')

基本css选择器

代码语言:javascript
复制
In [2]: html = '''
   ...: <div id="container">
   ...:     <ul class="list">
   ...:         <li class="item-0">first item</li>
   ...:         <li class="item-1"><a href="link2.html">second item</a></li>
   ...:         <li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>
   ...:         <li class="item-1"><a href="link4.html">fourth item</a></li>
   ...:         <li class="item-0"><a href="link5.html">fifth item</a></li>
   ...:     </ul>
   ...: </div>
   ...: '''

In [4]: from pyquery import PyQuery as pq

In [5]: doc = pq(html)

In [6]: print(doc('#container .list li'))
<li class="item-0">first item</li>
        <li class="item-1"><a href="link2.html">second item</a></li>
        <li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>
        <li class="item-1"><a href="link4.html">fourth item</a></li>
        <li class="item-0"><a href="link5.html">fifth item</a></li>


In [7]: print(type(doc('#container .list li')))
<class 'pyquery.pyquery.PyQuery'>

查找节点

find()

代码语言:javascript
复制
In [8]: items = doc('.list')

In [10]: print(type(items))
<class 'pyquery.pyquery.PyQuery'>

In [11]: print(items)
<ul class="list">
        <li class="item-0">first item</li>
        <li class="item-1"><a href="link2.html">second item</a></li>
        <li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>
        <li class="item-1"><a href="link4.html">fourth item</a></li>
        <li class="item-0"><a href="link5.html">fifth item</a></li>
    </ul>


In [12]: lis = items.find('li')

In [13]: print(type(lis))
<class 'pyquery.pyquery.PyQuery'>

In [14]: print(lis)
<li class="item-0">first item</li>
        <li class="item-1"><a href="link2.html">second item</a></li>
        <li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>
        <li class="item-1"><a href="link4.html">fourth item</a></li>
        <li class="item-0"><a href="link5.html">fifth item</a></li>

find()的查找范围是节点的所有后代节点

children()

查找直接子节点

代码语言:javascript
复制
In [15]: lis = items.children()

In [16]: print(type(lis))
<class 'pyquery.pyquery.PyQuery'>

In [17]: print(lis)
<li class="item-0">first item</li>
        <li class="item-1"><a href="link2.html">second item</a></li>
        <li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>
        <li class="item-1"><a href="link4.html">fourth item</a></li>
        <li class="item-0"><a href="link5.html">fifth item</a></li>


In [18]: items.children('.active')
Out[18]: [<li.item-0.active>]

In [19]: print(items.children('.active'))
<li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>

parent()

查找父节点

代码语言:javascript
复制
In [20]: container = items.parent()

In [21]: print(type(container))
<class 'pyquery.pyquery.PyQuery'>

In [22]: print(container)
<div id="container">
    <ul class="list">
        <li class="item-0">first item</li>
        <li class="item-1"><a href="link2.html">second item</a></li>
        <li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>
        <li class="item-1"><a href="link4.html">fourth item</a></li>
        <li class="item-0"><a href="link5.html">fifth item</a></li>
    </ul>
</div>

parents()

查找祖先节点

siblings()

兄弟节点

遍历节点

代码语言:javascript
复制
In [26]: lis = doc('li').items()

In [27]: print(type(lis))
<class 'generator'>

In [28]: for li in lis:
    ...:     print(li, type(li))
    ...:
<li class="item-0">first item</li>
         <class 'pyquery.pyquery.PyQuery'>
<li class="item-1"><a href="link2.html">second item</a></li>
         <class 'pyquery.pyquery.PyQuery'>
<li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>
         <class 'pyquery.pyquery.PyQuery'>
<li class="item-1"><a href="link4.html">fourth item</a></li>
         <class 'pyquery.pyquery.PyQuery'>
<li class="item-0"><a href="link5.html">fifth item</a></li>
     <class 'pyquery.pyquery.PyQuery'>

获取属性

代码语言:javascript
复制
# 选中class为item-0及active的a节点
In [29]: a = doc('.item-0.active a')

In [30]: print(a, type(a))
<a href="link3.html"><span class="bold">third item</span></a> <class 'pyquery.pyquery.PyQuery'>

In [31]: print(a.attr('href'))
link3.html

如果是多个节点的属性,就需要遍历节点获取

代码语言:javascript
复制
In [32]: a = doc('a')

In [33]: for item in a.items():
    ...:     print(item.attr('href'))
    ...:
link2.html
link3.html
link4.html
link5.html

获取文本

text()

代码语言:javascript
复制
In [35]: a = doc('.item-0.active a')

In [36]: print(a)
<a href="link3.html"><span class="bold">third item</span></a>

In [37]: print(a.text())
third item

html()

代码语言:javascript
复制
In [38]: li = doc('.item-0.active')

In [39]: print(li)
<li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>

In [41]: print(li.html())
<a href="link3.html"><span class="bold">third item</span></a>

在多个节点的情况下,如果要获取每个节点的内部html,需要遍历调用html()。而text()不需要遍历,会将所有节点取出文本后合并成一个字符串

节点操作

addClass 和 removeClass

代码语言:javascript
复制
In [42]: li = doc('.item-0.active')

In [43]: print(li)
<li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>


In [44]: li.removeClass('active')
Out[44]: [<li.item-0>]

In [45]: print(li)
<li class="item-0"><a href="link3.html"><span class="bold">third item</span></a></li>


In [46]: li.addClass('active')
Out[46]: [<li.item-0.active>]

In [47]: print(li)
<li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>

attr, text 和 html

代码语言:javascript
复制
In [48]: li = doc('.item-0.active')

In [49]: print(li)
<li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>


In [50]: li.attr('name', 'link')
Out[50]: [<li.item-0.active>]

In [51]: print(li)
<li class="item-0 active" name="link"><a href="link3.html"><span class="bold">third item</span></a></li>


In [52]: li.text('changed item')
Out[52]: [<li.item-0.active>]

In [53]: print(li)
<li class="item-0 active" name="link">changed item</li>


In [54]: li.html('<span>changed item</span>')
Out[54]: [<li.item-0.active>]

In [55]: print(li)
<li class="item-0 active" name="link"><span>changed item</span></li>

remove

代码语言:javascript
复制
In [56]: text = '''
    ...: <div class="wrap">
    ...:     Hello, World
    ...:     <p>This is a paragraph</p>
    ...: </div>
    ...: '''

In [57]: doc = pq(text)

In [58]: wrap = doc('.wrap')

In [59]: print(wrap.text())
Hello, World
This is a paragraph

如果我们不想要p节点内的text,就可以用remove将其移除

代码语言:javascript
复制
In [60]: wrap.find('p').remove()
Out[60]: [<p>]

In [61]: print(wrap.text())
Hello, World

伪类选择器

代码语言:javascript
复制
In [62]: doc = pq(html)

In [63]: li = doc('li:first-child')

In [64]: print(li)
<li class="item-0">first item</li>


In [65]: li = doc('li:last-child')

In [66]: print(li)
<li class="item-0"><a href="link5.html">fifth item</a></li>


In [67]: li = doc('li:nth-child(2)')

In [68]: print(li)
<li class="item-1"><a href="link2.html">second item</a></li>


In [69]: li = doc('li:gt(2)')

In [70]: print(li)
<li class="item-1"><a href="link4.html">fourth item</a></li>
        <li class="item-0"><a href="link5.html">fifth item</a></li>


In [71]: li = doc('li:nth-child(2n)')

In [72]: print(li)
<li class="item-1"><a href="link2.html">second item</a></li>
        <li class="item-1"><a href="link4.html">fourth item</a></li>


In [73]: li = doc('li:contains(second)')

In [74]: print(li)
<li class="item-1"><a href="link2.html">second item</a></li>
本文参与 腾讯云自媒体同步曝光计划,分享自作者个人站点/博客。
原始发表:2018.09.20 ,如有侵权请联系 cloudcommunity@tencent.com 删除

本文分享自 作者个人站点/博客 前往查看

如有侵权,请联系 cloudcommunity@tencent.com 删除。

本文参与 腾讯云自媒体同步曝光计划  ,欢迎热爱写作的你一起参与!

评论
登录后参与评论
0 条评论
热度
最新
推荐阅读
目录
  • 安装
  • pyquery 介绍
  • 数据初始化
  • 基本css选择器
  • 查找节点
    • find()
      • children()
        • parent()
          • parents()
            • siblings()
            • 遍历节点
            • 获取属性
            • 获取文本
            • 节点操作
              • addClass 和 removeClass
                • attr, text 和 html
                  • remove
                  • 伪类选择器
                  领券
                  问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档