python采集豆瓣网top250前10和后10电影短评并进行语义分析

import requests
import csv
import time
from lxml import etree
headers = {'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36',}

url = "https://movie.douban.com/subject/1291824/comments?start={}&limit=20&status=P&sort=time".format(i)
response = requests.get(url=url,headers=headers).text

html_datas = etree.HTML(response)
comments = html_datas.xpath('//span[@class="short"]/text()')
for comment in comments:
    #print(comment)
    comment = "".join(comment).strip("").replace('\n','，').replace(',','，')
    print(comment)

with open('moviecomment.csv', mode='a', newline="") as csvfile:
    csv_writer = csv.writer(csvfile, delimiter=',')
    csv_writer.writerow([comment])
二.情感语义分析
使用snowNLP进行评论的语义分析1.打开爬取的csv文件
f = open("moviecomment.csv","r")
datalist = f.readlines()2.进行语义分析for data in datalist:
    s = SnowNLP(data)
    a = s.sentiments
    print(a)#a的值为语义值3.对每部电影的100条影评取平均值得出结果为0.8310106064935262, 0.8044251181047334, 0.8108712143092854,0.858670540134834, 0.8589124243025005, 0.8762928658423270.8486057545234993, 0.8051373657653026, 0.777180409449240.823043718672946, 0.8363996333655149, 0.8584828917206430.8721060226962409, 0.7198451239815864, 0.879365569139330.8183443743080219, 0.9269355936086786, 0.772749500162290.8122440996032988, 0.8656266024493757三.使用matlibplot对数据进行简单的可视化import matplotlib.pyplot as plt
x_data = [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20]
y_data = [0.8310106064935262, 0.8044251181047334, 0.8108712143092854, 0.858670540134834, 0.8589124243025005, 0.8762928658423276, 0.8486057545234993, 0.8051373657653026, 0.7771804094449548, 0.823043718672946, 0.8363996333655149, 0.8584828917206438, 0.8721060226962409, 0.7198451239815864, 0.8793655691393392, 0.8183443743080219, 0.9269355936086786, 0.7727495001622988, 0.8122440996032988, 0.8656266024493757]
plt.xlabel('name')
plt.ylabel('score')
plt.xticks(x_data,x_data[::1])
plt.title('movie score line bar chart')
#折线图
plt.plot(x_data,y_data,"--o")
#柱状图
plt.bar(x_data,y_data)
plt.show()最后得到结果图：
   最后得出的图比较简单，但是也能说明一些问题，其中第14部电影《猜火车》语义值最低，第17部电影《宝莱坞之钢铁奶爸》的语义值最高，说明目前大众比较喜欢看一些亲情片。今年《我的姐姐》和《你好，李焕英》的成功可能也说明了这个问题。大家需要源码请私聊我，这次的教程就到这里了，非常感谢大家。