时间戳

首页 > 解决方案 > 使用 modify_url 构建多个 URL

r - 使用 modify_url 构建多个 URL

问题描述

我创建了一个页面抓取功能来抓取一些数据。我希望能够创建一个 URL 列表，以便我可以在函数调用中传递多个参数以构建不同的 URL。有没有办法做到这一点httr::modify_url？

我创建单个 URL 的代码如下：

library(tidyverse)
#> Registered S3 methods overwritten by 'ggplot2':
#>   method         from 
#>   [.quosures     rlang
#>   c.quosures     rlang
#>   print.quosures rlang
library(httr)

# Arguments for Function
hand = NULL
prp = "P"
month = NULL
year = 2019
pitch_type = "FA"
report_type = "pfx"
lim = 0

url <- httr::modify_url("https://legacy.baseballprospectus.com/pitchfx/leaderboards/index.php",
                        query = list(
                          hand = hand,
                          reportType = report_type,
                          prp = prp,
                          month = month,
                          year = year,
                          pitch = pitch_type,
                          ds = "velo",
                          lim = lim
                        ))

# Single Query Result
url
#> [1] "https://legacy.baseballprospectus.com/pitchfx/leaderboards/index.php?reportType=pfx&prp=P&year=2019&pitch=FA&ds=velo&lim=0"

我想知道是否可以使用httr::modify_url上面的查询和一些组合purrr::reduce(paste0)来创建 URL 的附加参数：

# Requested Query
pitch_type = c("FA", "SI")
report_type = c("pfx", "outcome")

# URL Generating Function for User inputs
generate_urls <- function(hand = NULL, report_type = c("pfx", "outcome"), prp = "P", month = NULL, year = NULL, pitch_type = c("FA", "SI"), lim = 0) {
# Not sure of what to put in function for modify_url call
}





# Result 
"https://legacy.baseballprospectus.com/pitchfx/leaderboards/index.php?reportType=pfx&prp=P&year=2019&pitch=FA&ds=velo&lim=0"
#> [1] "https://legacy.baseballprospectus.com/pitchfx/leaderboards/index.php?reportType=pfx&prp=P&year=2019&pitch=FA&ds=velo&lim=0"
"https://legacy.baseballprospectus.com/pitchfx/leaderboards/index.php?reportType=pfx&prp=P&year=2019&pitch=SI&ds=velo&lim=0"
#> [1] "https://legacy.baseballprospectus.com/pitchfx/leaderboards/index.php?reportType=pfx&prp=P&year=2019&pitch=SI&ds=velo&lim=0"
"https://legacy.baseballprospectus.com/pitchfx/leaderboards/index.php?reportType=outcome&prp=P&year=2019&pitch=FA&ds=velo&lim=0"
#> [1] "https://legacy.baseballprospectus.com/pitchfx/leaderboards/index.php?reportType=outcome&prp=P&year=2019&pitch=FA&ds=velo&lim=0"
"https://legacy.baseballprospectus.com/pitchfx/leaderboards/index.php?reportType=outcome&prp=P&year=2019&pitch=SI&ds=velo&lim=0"
#> [1] "https://legacy.baseballprospectus.com/pitchfx/leaderboards/index.php?reportType=outcome&prp=P&year=2019&pitch=SI&ds=velo&lim=0"

标签： rhttr

解决方案

这是使用 tidyverse 功能的选项。首先，我们可以定义我们想要遍历的参数空间

params <- list(
  hand = NULL,
  prp = "P",
  year = 2019,
  month = NULL,
  pitch_type = c("FA", "SI"),
  report_type = c("pfx", "outcome"),
  lim = 0
)

然后我们可以得到所有的URL

library(tidyverse) # tidyr for crossing(); purrr for pmap(), map_chr()
library(httr)  
baseurl <- "https://legacy.baseballprospectus.com/pitchfx/leaderboards/index.php"
crossing(!!!params) %>% 
  pmap(list) %>% 
  map_chr( ~modify_url(baseurl, query=.x) )

负责crossing()获取所有可能的参数组合。然后pmap(list)将 tibble 的每一行变成自己的列表（这是我们需要传递给 的query=参数modify_url。最后我们为每组参数调用 url 生成函数并返回一个字符串。

推荐阅读