首页
学习
活动
专区
工具
TVP
发布
精选内容/技术社群/优惠产品,尽在小程序
立即前往
您找到你想要的搜索结果了吗?
是的
没有找到

python3 selenium + f

一、 分析: 抓取动态页面js加载的人民日报里面的新闻详情 https://wap.peopleapp.com/news/1 先打开,然后查看网页源码,发现是一堆js,并没有具体的每个新闻的url详情,于是第一反应,肯定是js动态加载拼接的url。然后接着按f12 查看,就看url,发现出来了好多url。 然后点击具体的某一个新闻详情页面,查看url,把这个url的 后面两个数字其中一个拿到访问主页的时候,f12 抓包结果里面去查找,发现一个url,点击这个url,发现preview里面有好多数据,我第一反应,肯定是每个新闻数据了。看到这些数据里面有两个ID,联想到刚刚访问具体新闻详情页面也有两个数字,肯定,具体新闻页面肯定是 https://wap.peopleapp.com/article 加上两个ID形成的。于是试了一下拼接一个url访问,果然是。于是乎只要抓到这个url,就能获取到每个新闻的详情页了。

03

Response.ContentType 所有类型

大家好,又见面了,我是你们的朋友全栈君。‘ez’ => ‘application/andrew-inset’, ‘hqx’ => ‘application/mac-binhex40’, ‘cpt’ => ‘application/mac-compactpro’, ‘doc’ => ‘application/msword’, ‘bin’ => ‘application/octet-stream’, ‘dms’ => ‘application/octet-stream’, ‘lha’ => ‘application/octet-stream’, ‘lzh’ => ‘application/octet-stream’, ‘exe’ => ‘application/octet-stream’, ‘class’ => ‘application/octet-stream’, ‘so’ => ‘application/octet-stream’, ‘dll’ => ‘application/octet-stream’, ‘oda’ => ‘application/oda’, ‘pdf’ => ‘application/pdf’, ‘ai’ => ‘application/postscript’, ‘eps’ => ‘application/postscript’, ‘ps’ => ‘application/postscript’, ‘smi’ => ‘application/smil’, ‘smil’ => ‘application/smil’, ‘mif’ => ‘application/vnd.mif’, ‘xls’ => ‘application/vnd.ms-excel’, ‘ppt’ => ‘application/vnd.ms-powerpoint’, ‘wbxml’ => ‘application/vnd.wap.wbxml’, ‘wmlc’ => ‘application/vnd.wap.wmlc’, ‘wmlsc’ => ‘application/vnd.wap.wmlscriptc’, ‘bcpio’ => ‘application/x-bcpio’, ‘vcd’ => ‘application/x-cdlink’, ‘pgn’ => ‘application/x-chess-pgn’, ‘cpio’ => ‘application/x-cpio’, ‘csh’ => ‘application/x-csh’, ‘dcr’ => ‘application/x-director’, ‘dir’ => ‘application/x-director’, ‘dxr’ => ‘application/x-director’, ‘dvi’ => ‘application/x-dvi’, ‘spl’ => ‘application/x-futuresplash’, ‘gtar’ => ‘application/x-gtar’, ‘hdf’ => ‘application/x-hdf’, ‘js’ => ‘application/x-javascript’, ‘skp’ => ‘application/x-koan’, ‘skd’ => ‘application/x-koan’, ‘skt’ => ‘application/x-koan’, ‘skm’ => ‘application/x-koan’, ‘latex’ => ‘application/x-latex’, ‘nc’ => ‘application/x-netcdf’, ‘cdf’ => ‘application/x-netcdf’, ‘sh’ => ‘application/x-sh’, ‘shar’ => ‘application/x-shar’, ‘swf’ => ‘application/x-shockwave-flash’, ‘sit’ => ‘application/x-stuffit’, ‘sv4cpio’ => ‘application/x-sv4cpio’, ‘sv4crc’ => ‘application/x-sv4crc’, ‘tar’ => ‘application/x-tar’, ‘tcl’ => ‘application/x-tcl’, ‘tex’ => ‘application/x-tex’, ‘texinfo’ => ‘application/x-texinfo’, ‘texi’ => ‘application/x-texinfo’, ‘t’ => ‘application/x-troff’, ‘tr’ => ‘applica

05

content-type 的所有可选项

‘ez’        => ‘application/andrew-inset’,     ‘hqx’        => ‘application/mac-binhex40’,     ‘cpt’        => ‘application/mac-compactpro’,     ‘doc’        => ‘application/msWord’,     ‘bin’        => ‘application/octet-stream’,     ‘dms’        => ‘application/octet-stream’,     ‘lha’        => ‘application/octet-stream’,     ‘lzh’        => ‘application/octet-stream’,     ‘exe’        => ‘application/octet-stream’,     ‘class’        => ‘application/octet-stream’,     ‘so’        => ‘application/octet-stream’,     ‘dll’        => ‘application/octet-stream’,     ‘oda’        => ‘application/oda’,     ‘pdf’        => ‘application/pdf’,     ‘ai’        => ‘application/postscript’,     ‘eps’        => ‘application/postscript’,     ‘ps’        => ‘application/postscript’,     ‘smi’        => ‘application/smil’,     ‘smil’        => ‘application/smil’,     ‘mif’        => ‘application/vnd.mif’,     ‘xls’        => ‘application/vnd.ms-Excel’,     ‘ppt’        => ‘application/vnd.ms-Powerpoint’,     ‘wbxml’        => ‘application/vnd.wap.wbxml’,     ‘wmlc’        => ‘application/vnd.wap.wmlc’,     ‘wmlsc’        => ‘application/vnd.wap.wmlscriptc’,     ‘bcpio’        => ‘application/x-bcpio’,     ‘vcd’        => ‘application/x-cdlink’,     ‘pgn’        => ‘application/x-chess-pgn’,     ‘cpio’        => ‘application/x-cpio’,     ‘csh’        => ‘application/x-csh’,     ‘dcr’        => ‘application/x-Director’,     ‘dir’        => ‘application/x-director’,     ‘dxr’        => ‘application/x-director’,     ‘dvi’        => ‘application/x-dvi’,     ‘spl’        => ‘application/x-futuresplash’,     ‘gtar’        => ‘application/x-gtar’,     ‘hdf’        => ‘application/x-hdf’,     ‘js’        => ‘application/x-javascript’,     ‘skp’        => ‘application/x-koan’,     ‘skd’        => ‘application/x-koan’,     ‘skt’        => ‘application/x-koan’,     ‘skm’        => ‘application/x-koan’,     ‘latex’        => ‘application/x-latex’,     ‘nc’        => ‘application/x-netcdf’,     ‘

02

扫码

添加站长 进交流群

领取专属 10元无门槛券

手把手带您无忧上云

扫码加入开发者社群

相关资讯

热门标签

活动推荐

    运营活动

    活动名称
    广告关闭
    领券