Download multiple files using “download.file” function while skipping links that do not exist (with walk2)

Hello I am trying to perform a task similar to this one but instead of downloading csv I want to download tif files.

I do not know what I am doing wrong but I tried with both passying otherwise=NULL to safely and with a print message ... in every case when it comes to files that do not exist (e.g. 29 February wbgtmax.2015.02.29.tif ) I got an error message

Caused by error in `download.file()`:
! cannot open URL ''
Run `rlang::last_trace()` to see where the error occurred.
Warning messages:
1: In download.file(url = x, destfile = y, mode = "wb") :
  downloaded length 0 != reported length 0
2: In download.file(url = x, destfile = y, mode = "wb") :
  cannot open URL '': HTTP status was '404 Not Found'

This is my code

getOption('timeout') # Look at the timeout option and increase it 



# Creates a String of the URL Addresses
urls <- 
  tidyr::expand_grid(months, days) |> 

head(urls, 5)  

# Creates Names for the Files 
tif_names <- 
  tidyr::expand_grid(months, days) %>%


  safely(download.file(url=x,destfile = y,mode = "wb"),
         otherwise = print(glue("{x} file not found")))}


Can anyone help me understand what I am doing wrong? I have already used safely in the past but never come to such issue


Hi @angela_italy
Your code couldn't find a file because 29-Feb-2015 did not exist (2015 was not a leap year).

@DavoWW I know that .. Indeed I would like to make safely skip the files that do not exist

Hi @angela_italy
This code may do the job. I tested it on some of the URLs you supplied but none were available.

# Subset of URLs for Testing
urls <- urls[1:5]

# See:

# Helper function
url_exists <- function(url){
  hd <- httr::HEAD(url)
  status <- hd$all_headers[[1]]$status
  result <- list(exists = status == HTTP_STATUS_OK, status = status)

for (ii in 1:length(urls)) {
  inurl <- urls[ii]
  Sys.sleep(2)  # slight delay to manage loop speed
  if (result$exists[1] == TRUE & result$status[1] == 200L) {
    destfile <- basename(inurl)
    download.file(url=inurl, destfile = destfile, mode = "wb")
    print(paste(destfile, ": download complete")) } else {
      print(paste(basename(inurl), ": file not found. Continuing to next."))

Thank you @DavoWW
So my understanding is that the problem is not the safely function but the download function and should be prevented before running it (e.g. checking that the file exists)

replace the above with

safe_download <- 
  purrr::safely(.f =function(x,y){download.file(url=x,destfile = y,mode = "wb")},
                quiet = FALSE)

and though it wouldnt print exactly your {x} file not found message, it would work otherwise as intended.

Thanks @nirgrahamuk. So, the problem was that I specified 'safely' within the function rather than outside it... wasn't it?

I would phrase it that safely is a wrapper for altering a functions behaviour so it should receive a function definition. Your formulation was in some way wrapping around it, and I think the download.file() would be evaluated with x and y, and the result passed to purrr::safely , the result not being a function but its evaluation.

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.

If you have a query related to it or one of the replies, start a new topic and refer back with a link.