R网络抓取RCurl和httr内容
我正在学习一些关于网络抓取的知识，我对两个包(httr和RCurl)有一点怀疑，我试图从researchgate网站上的一本杂志(ISSN)中获取代码，然后我遇到了一个情况。当通过httr和RCurl从网站提取内容时，我在RCurl包中得到ISSN，而在httr中我的函数返回NULL，有人能告诉我为什么会这样吗？在我看来，这两个功能都可以工作。遵循下面的代码。
library(rvest)
library(httr)
library(RCurl)
url <- "https://www.researchgate.net/journal/0730-0301_Acm_Transactions_On_Graphics"
########
# httr #
########
conexao <- GET(url)
conexao_status <- http_status(conexao)
conexao_status
content(conexao, as = "text", encoding = "utf-8") %>% read_html() -> webpage1
ISSN <- webpage1 %>%
html_nodes(xpath = '//*/div/div[2]/div[1]/div[1]/table[2]/tbody/tr[7]/td') %>%
html_text %>%
str_to_title() %>%
str_split(" ") %>%
unlist
ISSN
########
# RCurl #
########
options(RCurlOptions = list(verbose = FALSE,
capath = system.file("CurlSSL", "cacert.pem", package = "RCurl"),
ssl.verifypeer = FALSE))
webpage <- getURLContent(url) %>% read_html()
ISSN <- webpage %>%
html_nodes(xpath = '//*/div/div[2]/div[1]/div[1]/table[2]/tbody/tr[7]/td') %>%
html_text %>%
str_to_title() %>%
str_split(" ") %>%
unlist
ISSNsessionInfo() R版本3.5.0 (2018-04-23)平台: x86_64-w64-mingw32/x64 (64位)运行环境: Windows >= 8 x64 (build 9200)矩阵产品:默认
区域设置:1 LC_COLLATE=Portuguese_Brazil.1252 LC_CTYPE=Portuguese_Brazil.1252 LC_MONETARY=Portuguese_Brazil.1252 4 LC_NUMERIC=C LC_TIME=Portuguese_Brazil.1252
附加的基本软件包:1统计图形grDevices实用程序
datasets方法基于其他附加的包:1 testit_0.7 dplyr_0.7.4
progress_1.1.2 readxl_1.1.0 stringr_1.3.0 RCurl_1.95-4.10 bitops_1.0-6 8 httr_1.3.1 rvest_0.3.2 xml2_1.2.0
通过命名空间加载(且未附加)的jsonlite_1.5 :1 Rcpp_0.12.16
bindr_0.1.1 magrittr_1.5 R6_2.2.2 rlang_0.2.0
tools_3.5.0 7 yaml_2.1.19 assertthat_0.2.0 tibble_1.4.2 bindrcpp_0.2.2 curl_3.2 glue_1.2.0
13 stringi_1.1.7 pillar_1.2.2 compiler_3.5.0
cellranger_1.1.0 prettyunits_1.0.2 pkgconfig_2.0.1
回答 1
Stack Overflow用户
发布于 2018-06-02 11:20:07
因为内容类型是JSON而不是HTML，所以不能对其使用read_html()：
> conexao
Response [https://www.researchgate.net/journal/0730-0301_Acm_Transactions_On_Graphics]
Date: 2018-06-02 03:15
Status: 200
Content-Type: application/json; charset=utf-8
Size: 328 kB取而代之的是使用fromJSON()提取issn：
library(jsonlite)
result <- fromJSON(content(conexao, as = "text", encoding = "utf-8") )
result$result$data$journalFullInfo$data$issn结果：
> result$result$data$journalFullInfo$data$issn
[1] "0730-0301"https://stackoverflow.com/questions/50650515
复制相似问题