python爬虫练手,爬取名言,实现英语词典

要爬取的网站 http://quotes.toscrape.com/

image.png

爬取名言,作者,标签。*

她们的Html为,通过beautiful库的html.parser解析,通过id,class选择器,提取我们需要的东西。

<span class="text" itemprop="text">“The world as we have created it is a process of our thinking. It cannot be changed without changing our thinking.”</span>

<span class="text" itemprop="text">“It is our choices, Harry, that show what we truly are, far more than our abilities.”</span>



<div class="tags">
            Tags:
            <meta class="keywords" itemprop="keywords" content="change,deep-thoughts,thinking,world"> 
            
            <a class="tag" href="/tag/change/page/1/">change</a>
            
            <a class="tag" href="/tag/deep-thoughts/page/1/">deep-thoughts</a>
            
            <a class="tag" href="/tag/thinking/page/1/">thinking</a>
            
            <a class="tag" href="/tag/world/page/1/">world</a>
            
        </div>


<div class="tags">
            Tags:
            <meta class="keywords" itemprop="keywords" content="abilities,choices"> 
            
            <a class="tag" href="/tag/abilities/page/1/">abilities</a>
            
            <a class="tag" href="/tag/choices/page/1/">choices</a>
            
        </div>

<small class="author" itemprop="author">J.K. Rowling</small>

相关的code如下

import requests
import os
from bs4 import BeautifulSoup

def get_html(url):
    try:
        header={'User-Agent':'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.90 Safari/537.36 2345Explorer/9.3.2.17331', }
        ht=requests.get(url,headers=header,verify=True)
        ht.raise_for_status
        ht.encoding=ht.apparent_encoding
        return ht
    except Exception as e:
        print("error:",e)
        
       
    
        
def writeHtml(url):
    text=get_html(url).content
    path="C:\\Users\\Administrator\\Desktop\\python\\baidu.html"
    with open(path,"wb") as f:
        f.write(text)
print("success")



def getInfor(url):
    finalSay=[]
    html=get_html(url).text
    soup=BeautifulSoup(html,"html.parser")
    print(" the saying is:")
    say_list=soup.select("span.text")
    print("-------名言--------------")
    print("the length=",len(say_list))
   # print(say_list)
    for elem in say_list:
        te=elem.text
        finalSay.append(te);
    print(finalSay)
    print("-------标签--------------")
    finalTag=[]
    tag_list=soup.select("div.tags")
    print("the length of tag=",len(tag_list))
    #print(tag_list)
    for tag in tag_list:
        a_tag=tag.select("a.tag")
        #print(a_tag)
        
       # for elem in a_tag:
       #    print(elem.text)
        tag_list=[elem.text for elem in a_tag]
        #print(tag_list)
        finalTag.append(tag_list)
    print(finalTag)
    print("-------作者--------------")
    finalAuthor=[]
    author_list=soup.select("small.author")
    print("the length of authoer=",len(author_list))
    for elem in author_list:
        finalAuthor.append(elem.text)
    print(finalAuthor)
    for i in range(len(author_list)):
        print("the saying:",finalSay[i],"\t","the author:",finalAuthor[i],"\t","the tag:",finalTag[i])

结果如下

the saying: “The world as we have created it is a process of our thinking. It cannot be changed without changing our thinking.”          the author: Albert Einstein     the tag: ['change', 'deep-thoughts', 'thinking', 'world']
the saying: “It is our choices, Harry, that show what we truly are, far more than our abilities.”        the author: J.K. Rowling        the tag: ['abilities', 'choices']
the saying: “There are only two ways to live your life. One is as though nothing is a miracle. The other is as though everything is a miracle.”          the author: Albert Einstein     the tag: ['inspirational', 'life', 'live', 'miracle', 'miracles']
the saying: “The person, be it gentleman or lady, who has not pleasure in a good novel, must be intolerably stupid.”     the author: Jane Austen         the tag: ['aliteracy', 'books', 'classic', 'humor']
the saying: “Imperfection is beauty, madness is genius and it's better to be absolutely ridiculous than absolutely boring.”      the author: Marilyn Monroe      the tag: ['be-yourself', 'inspirational']
the saying: “Try not to become a man of success. Rather become a man of value.”          the author: Albert Einstein     the tag: ['adulthood', 'success', 'value']
the saying: “It is better to be hated for what you are than to be loved for what you are not.”   the author: André Gide          the tag: ['life', 'love']
the saying: “I have not failed. I've just found 10,000 ways that won't work.”    the author: Thomas A. Edison    the tag: ['edison', 'failure', 'inspirational', 'paraphrased']
the saying: “A woman is like a tea bag; you never know how strong it is until it's in hot water.”        the author: Eleanor Roosevelt   the tag: ['misattributed-eleanor-roosevelt']
the saying: “A day without sunshine is like, you know, night.”   the author: Steve Martin        the tag: ['humor', 'obvious', 'simile']

借助bing的翻译,实现英语翻译中文

完整 https://cn.bing.com/dict/search?q=snow

其翻译的信息主要在

<span class="pos">n.</span>
<span class="pos">v.</span>
<span class="pos">adj.</span>
<span class="pos web">网络</span>

<span class="def"><span>书;书籍;著作;部</span></span>
<span class="def"><span>书籍的;书本上的;账面上的</span></span>

对应的正则表达式为

 re0=r'<span class="(pos|pos web)">(.*?)</span>'
 re1=r'<span class="def"><span>(.*?)</span></span>'

相关code 主要是正则表达式的使用,已经写文件操作。

def getDictionary():
    word=input("亲输入要翻译的词语:")
    url="https://cn.bing.com/dict/search?q="
    hurl=url+word
    trant=[]
    html=get_html(hurl).text
    re0=r'<span class="(pos|pos web)">(.*?)</span>'
    flag_list=[]
    nav_list=re.findall(re0,html)
    #print(hurl)
    #print(html)
    if len(nav_list)==0 and len(nav_list[0])<1:
        return 
    for elem in nav_list:
        flag_list.append(elem[1])  
    print(flag_list)
    re1=r'<span class="def"><span>(.*?)</span></span>'
    tran_list=re.findall(re1,html)
    #print(tran_list)
    for i in range(len(nav_list)):
        tra="\t".join([flag_list[i],tran_list[i]])
        print(tra)
        trant.append(tra)
        #trant.append("\n")
    return trant
        
         
        
def writeTet(t):
    path="E:\\infor.txt"
    with open(path,"a+") as f:
        for txt in t:
             f.write(txt)
             f.write("\n")
    print("success")

结果为

亲输入要翻译的词语:Python
['n.', '网络']
n.      蟒;蚺蛇
网络      蟒蛇;巨蟒;派森
success

image.png

参考文章 beautifulsoup之CSS选择器 Beautiful Soup Documentation Python 文件I/O python对文件的操作读写追加等演示

本文参与腾讯云自媒体分享计划,欢迎正在阅读的你也加入,一起分享。

发表于

我来说两句

0 条评论
登录 后参与评论

相关文章

来自专栏菩提树下的杨过

Flash/Flex学习笔记(15):FMS 3.5之远程共享对象(Remote Shared Object)

FMS中的“远程共享对象”可以让多个Client端的flash应用共享同一个全局对象,并且当客户端中的任何一个改变该对象时,系统会自动将该对象回发到FMS服务器...

22680
来自专栏章鱼的慢慢技术路

CodeCombat森林关卡Python代码

23330
来自专栏GIS讲堂

OpenLayers3基础教程——OL3基本概念

从本节开始,我会陆陆续续的更新有关OL3的相关文章——OpenLayers3基础教程,欢迎大家关注我的博客,同时也希望我的博客能够给大家带来一点帮助。

17230
来自专栏CDA数据分析师

Python 爬取北京二手房数据,分析北漂族买得起房吗? | 附完整源码

本文主要分为两部分:Python爬取赶集网北京二手房数据,R对爬取的二手房房价做线性回归分析,适合刚刚接触Python&R的同学们学习参考。

15720
来自专栏写写代码吃吃瓜

OpenGL初学--环境配置和视景体初步接触

21450
来自专栏小灰灰

zxing二维码生成服务之深度定制

二维码生成服务之深度定制 之前写了一篇二维码服务定制的博文,现在则在之前的基础上,再进一步,花样的实现深度定制的需求,我们的目标是二维码上的一切都是可以由用户...

57460
来自专栏菩提树下的杨过

ExtJs学习笔记(2)_Basic GridPanel[基本网格]

这一节,将学习如何使用网络上最常见的UI控件_Grid 1.静态示例: 静态示例其实官方下载包里,就有sample,这里只贴出代码,后面的如何跟WCF结合,做出...

23290
来自专栏非典型技术宅

Quartz2D进行渲染1. 渲染模式2. even-odd rule:奇偶填充规则3. nonzero winding number rule:非零绕数规则4. 其他会用到的渲染模式5. 混合模式

15830
来自专栏AI研习社

如何使用 OpenCV 编写基于 Node.js 命令行界面和神经网络模型的图像分类

如何使用 OpenCV 编写基于 Node.js 命令行界面和神经网络模型的图像分类

25950
来自专栏程序员互动联盟

【专业技术】Win32 SDK编程:我们如何输出文本

在使用Win32编程时,我们常常需要输出文本到窗口上,Windows所有的文本字符或者图形输出都是通过图形设备接口(GDI)进行的,Windows的三大组件之一...

39850

扫码关注云+社区

领取腾讯云代金券