Webscraping with RSelenium: File Download with remote driver not working

I am attempting to download files from SmartRow using RSelenium. I am stuck on getting the actual file download. Below, I've included my code including everything except the credentials.

As you can surely tell, I'm new to web scraping and RSelenium. I would greatly appreciate any suggestions you may have.

I've included a screenshot of just before I click on the last element. I feel like I'm so close, but I'm officially stumped.

I originally posted this to Stack Overflow, but it does not appear to be getting attention. I'll cross post answers if there are any on either site.

library(RSelenium)
eCaps <- list(chromeOptions = list(
  # driver = "/bin/chromedriver",
  args = c("--window-size=1920,1080",
           "--disable-extensions",
           "--proxy-server='direct://'",
           "--proxy-bypass-list=*",
           "--start-maximized",
           "--headless",
           "--desable-gpu",
           "--disable-dev-shm-usage",
           "--no-sandbox",
           "--ignore-certificate-errors"
           ),
  prefs = list("profile.default_content_settings.popups" = 0L,
               "download.prompt_for_download" = FALSE,
               "download.default_directory" = "/home/ua21849/downloads")
))

remDr <- remoteDriver(
  remoteServerAddr = "********",
  port = 4444L,
  browserName = "chrome",
  extraCapabilities = eCaps
)


remDr$open()
remDr$navigate("https://smartrow.fit/#/profile")
username <- remDr$findElement(using = "id", value = "mat-input-0")
remDr$findElement(using = "id" , value = "mat-input-0")$sendKeysToElement(list("******"))
remDr$findElement(using = "id", value = "mat-input-1")$sendKeysToElement(list("*******"))
remDr$findElement(using = "css selector", value = ".mat-stroked-button")$clickElement()
remDr$navigate("https://smartrow.fit/#/log")
remDr$findElement(using = "css selector", value = ".mat-icon-no-color")$clickElement()

remDr$screenshot(file = "test.png")

b <- remDr$findElement(using = "css selector",
                       value = "a.mat-list-item.mat-2-line.ng-star-inserted")

b$clickElement()

so close!!!

Session info:

> sessionInfo()
R version 4.1.1 (2021-08-10)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: CentOS Linux 7 (Core)

Matrix products: default
BLAS:   /share/apps/R/openblas/OpenBLAS-0.2.19/libopenblas.so
LAPACK: /share/apps/R/R-4.1.1-openblas/lib64/R/lib/libRlapack.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8   
 [6] LC_MESSAGES=en_US.UTF-8    LC_PAPER=en_US.UTF-8       LC_NAME=C                  LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] rvest_1.0.1     magrittr_2.0.1  RSelenium_1.7.6

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.7       XML_3.99-0.7     binman_0.1.2     assertthat_0.2.1 bitops_1.0-7     R6_2.5.1         lifecycle_1.0.0  jsonlite_1.7.2  
 [9] semver_0.2.0     httr_1.4.2       stringi_1.7.4    rlang_0.4.11     curl_4.3.2       xml2_1.3.2       tools_4.1.1      stringr_1.4.0   
[17] selectr_0.4-2    wdman_0.2.5      compiler_4.1.1   askpass_1.1      caTools_1.18.2   openssl_1.4.4  

The remote driver is a singularity container that is a clone of the docker container recommended here.

So the last click but one produces a popup window (?)
And with the next click you try to select one of the export formats (?)
I am also a newbie concerning Selenium but I googled on
'selenium popup window button click' and I think you should at least browse
https://www.toolsqa.com/selenium-webdriver/alerts-in-selenium/
I hope this helps.

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.

If you have a query related to it or one of the replies, start a new topic and refer back with a link.