Web scraping in R for CSV link on apple mobility webpage

General
,
1

Hi, I am trying to get the csv link from the following webpage

I tried the following

library(rvest)
library(stringr)
page <- read_html("https://www.apple.com/covid19/mobility")

url <- page %>%
  html_nodes("a") %>%       # find all links
  html_attr("href") %>%     # get the url
  str_subset("\\.csv")

But this is returning the character(0). This page does have a csv file with download option.

Any help would be greatly appreciated.

Thanks.

2

Hi. If you check the page source html, in fact there's no link with "csv" in the hrefs, but using Firefox inspection tool (F12) I found out the link to download the dataset to be

https://covid19-static.cdn-apple.com/covid19-mobility-data/2008HotfixDev26/v2/en-us/applemobilitytrends-2020-05-13.csv

Remember to parametrize the date at the end. It's probably stable, but I know nothing about industry best practices for such things.

EDIT: You can download the csv with
read.csv(url("https://...")) or readr::read_csv("https://...").

3

Thank you @ableb0rges. Yes I have seen the csv link in chrome and downloaded the data using read.csv function. Hopefully the CSV is link is stable because its difficult to download if the link changes everytime. Someone suggested me to use RSelenium to scrape the webpage to get the link. Let's see if I can get that working.

Thanks again :slight_smile: