文章/答案/技术大牛

发布

社区首页 >问答首页 >无法在傀儡页面加载中获得所有ajax调用响应

问无法在傀儡页面加载中获得所有ajax调用响应
EN

Stack Overflow用户

提问于 2020-10-07 17:16:47

回答 1查看 509关注 0票数 0

在加载url 单击此处时，我在网络选项卡上有大约四个ajax调用

但是，当使用puppetter刮取数据时，只记录了一个ajax调用响应url，即https://chartink.com/screener/trend-trading-11。

这是我尝试过的代码:-

async function main() {
  try {
    const browser = await puppeteer.launch({ headless: false });
    const page = await browser.newPage();

    page.waitForResponse(async (response) => {
      console.log(response.url());
    });

    await page.goto("https://chartink.com/screener/trend-trading-11", {
      waitUntil: "networkidle0",
      timeout: 0,
    });

    await browser.close();
  } catch (err) {
    console.error(err);
  }
})();

有人能帮上忙吗？

javascript

node.js

puppeteer

回答 1

Stack Overflow用户

回答已采纳

发布于 2020-10-07 17:35:05

您的问题可能是因为您正在使用async(response)，这对我来说是有效的：

...

const page = await browser.newPage();    
page.on('response', response => {
    response.text().then(function(textBody){ //this returns promise that ajax request body was received
        console.log(response.url());
        console.log(textBody);
    })
})    
await page.goto("https://chartink.com/screener/trend-trading-11", {
  waitUntil: "networkidle0",
  timeout: 0,
});
await browser.close();

我得到的输出体来自urls (不会粘贴主体内容，因为它大约是5MB)：

https://chartink.com/screener/trend-trading-11
https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.8.2/css/all.min.css
https://chartink.com/screener/trend-trading-11
https://chartink.com/js/lives_v2.js?id=439b7e42ac6b39972d7f
https://chartink.com/css/custom.css
https://chartink.com/css/font-awesome-4.7.0/css/font-awesome.min.css
https://cdnjs.cloudflare.com/ajax/libs/Chart.js/2.7.3/Chart.bundle.js
https://chartink.com/css/app.css?id=061148b114879e02aa12
https://chartink.com/css/bootstrap-tour.min.css
https://chartink.com/css/datatables.min.css
https://chartink.com/js/jquery.color.min.js
https://chartink.com/js/scan_filter.js?id=06e909b94080867f8291
https://chartink.com/js/bootstrap-tour-standalone.min.js
https://fonts.googleapis.com/css?family=Raleway:300,400,600
https://chartink.com/js/app.js?id=9d30bc56db0d96b0b0b1
https://chartink.com/css/core.css
https://chartink.com/js/atlas.js?id=d575177318b1044bb512
https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js
https://chartink.com/images/bg.gif
https://chartink.com/images/new_header.png
https://chartink.com/images/search2.gif
https://chartink.com/images/white_line.png
https://www.google-analytics.com/analytics.js
https://chartink.com/css/jquery.timepicker.css
https://chartink.com/images/stats/add-circle.png
https://chartink.com/images/stats/operation.svg
https://pagead2.googlesyndication.com/pagead/js/r20201001/r20190131/show_ads_impl.js
https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.8.2/webfonts/fa-solid-900.woff2
https://www.google-analytics.com/j/collect?v=1&_v=j86&a=1793252394&t=pageview&_s=1&dl=https%3A%2F%2Fchartink.com%2Fscreener%2Ftrend-trading-11&ul=pl&de=UTF-8&dt=Trend%20trading%2C%20Technical%20Analysis%20Scanner&sd=24-bit&sr=1920x1080&vp=1903x927&je=0&_u=IEBAAEABAAAAAC~&jid=1026560667&gjid=1096229421&cid=1806703606.1602091906&tid=UA-19900447-1&_gid=339192360.1602091906&_r=1&_slc=1&z=1222043144
https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.8.2/webfonts/fa-brands-400.woff2
https://chartink.com/fonts/glyphicons-halflings-regular.woff2
https://adservice.google.pl/adsid/integrator.js?domain=chartink.com
https://adservice.google.com/adsid/integrator.js?domain=chartink.com
https://www.googletagservices.com/activeview/js/current/osd.js?cb=%2Fr20100101
https://chartink.com/screener/process
https://pagead2.googlesyndication.com/getconfig/sodar?sv=200&tid=gda&tv=r20201001&st=env
https://tpc.googlesyndication.com/sodar/sodar2.js
https://js-agent.newrelic.com/nr-1184.min.js
https://chartink.com/backtest/process
https://chartink.com/favicon.ico
https://chartink.com/manifest.json
https://chartink.com/favicon.ico
https://pagead2.googlesyndication.com/pagead/gen_204?id=sodar2&v=217&t=2&li=gda_r20201001&jk=3868359323948220&bg=!7O-l7_dYuruaC6okFhkCAAAAyVIAAAAXCgB9K_NEuJ7yrGCy1lnOElRw5ctqJzVBPSfadY4e8WtxSMlAGizrMCBaK9b91_dCyeo9z0ksbtgGc2Dq6CAmznGnx9GpH7uehFc9h9mkiioPkpj8C4DSqq4AGPAmrpK6qTGaA55qMHXHa-PUMr-Q9iaXySa2BC1qGyefZ1em3EqZAeFac00sJYEhXkeg5Lf1Uvppw8eowhESkD63DFM10L46d72OnG2pdHC2fm6OjB8PvX-EpzJ2lbpbjpzVSzTfkayLP1Y1a7AfLkqE5kaREFCoa4w_VIwALi4CUXz5EuHD658HGZSv4ab6qiHf5c737JiKwyRASks71aAXAwOv8TNWjnEua25Mjqzk94-UPmzyW53PnMBJCsPx6ty1S9EU4pwa8EGwkXsyDV-uI26K4FHKZk-GOY4M5KQ9Wm5r1w13zd2JR4LfOfF146XC445aI6am77zwRBgKMJdWxgN6HHoYwGkX-EAPfl3_AHow0FPJoFQScpQmOxONhJf9L2pyCBZ8WwysF3gi28x_t3nMy8V38ld9IaQRp5k5I1yxGeUMGwdYT4IZ7lFGFh4s1Yr8xXYAAm5lF9zGP7VX-RXQd_LTsQZFKdjW8Wk4hdJUvl-rCy6qYJDQ8eKCnvESNSmDcywIucYYRaD_EvRBAoRevNp7lfVIQ9s5DMXUxWMrK576TLnymyK2hlzuJOBBUx2VqagT0TZhCAJcJXOhtyLXTuD1Ectp9j75IicdM9aYeOU0AScKay64n_wVCpificMRuLWyrQA4OnGF9WD6LLEIhWpjaQ5iXOaRuAVXvwGaosbukQaq
https://bam.nr-data.net/1/4f831ebbc0?a=375583170&v=1184.ab39b52&to=ZwYGbERVXkEAUEZQCV5MJVtCXV9cTkBRSwNVDQFK&rst=6423&ck=1&ref=https://chartink.com/screener/trend-trading-11&ap=348&be=1009&fe=5489&dc=3288&perf=%7B%22timing%22:%7B%22of%22:1602091902183,%22n%22:0,%22f%22:1,%22dn%22:26,%22dne%22:36,%22c%22:36,%22s%22:220,%22ce%22:432,%22rq%22:432,%22rp%22:968,%22rpe%22:1148,%22dl%22:987,%22di%22:3287,%22ds%22:3287,%22de%22:3295,%22dc%22:5488,%22l%22:5488,%22le%22:5498%7D,%22navigation%22:%7B%7D%7D&fp=2108&fcp=2108&at=S0ERGgxPTR5DUhADHU0e&jsonp=NREUM.setToken

但是要注意，与原始响应相比，解析输出HTML更容易。这就是我获取数据的方式。

票数 0

页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持

原文链接：

https://stackoverflow.com/questions/64249101

复制

相似问题

问无法在傀儡页面加载中获得所有ajax调用响应
EN

回答 1

Stack Overflow用户

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问无法在傀儡页面加载中获得所有ajax调用响应EN

回答 1

Stack Overflow用户

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问无法在傀儡页面加载中获得所有ajax调用响应
EN