Use RSelenium within a Shiny App

Posit Professional & Hosted Products shinyapps.io
,
1

I have an Rscript that uses RSelenium to collect data (I cannot use rvest because the webpage uses javascript). I included this script as a function in my shinyApp (I perform the scraping interactively after some user input in the shinyApp). When I run it from my machine (locally) everything works as expected. The trouble occurs when it is published on the RStudio server (shinyapps.io). Basically, it cannot connect to the local port.

[1] "Connecting to remote server"
2020-03-14T23:14:25.090285+00:00 shinyapps[1626004]: Warning: Error in checkError: Undefined error in httr call. httr output: Failed to connect to localhost port 4445: Connection refused

I am not the most familiar with RSelenium, Docker, etc. but in general, is it possible to run RSelenium based scripts within shinyApp from RStudio server (assumning it works in shinyApp locally)? If so, how do I do so (I think I have to set up Docker to allow for other port access maybe)? Any help will be greatly appreciated.

2

Here is some example code to provide more information. After downloading Docker and running docker run -d -p 4445:4444 selenium/standalone-chrome inside terminal, the following should work on your local machine (it does on mine):

library(shiny)
library(RSelenium)

ui <- fluidPage(
  actionButton("go", "Scrape")
)
server <- function(input, output, session) {
  observeEvent(input$go, {
    remDr <- RSelenium::remoteDriver(remoteServerAddr = "localhost",
                                                             port = 4445L,
                                                             browserName = "chrome")
    # Open browser session
    remDr$open()

    remDr$navigate("https://www.google.com/") # or any website
    # Click on links and scrape some stuff
    # Close browser session
    remDr$close()
  })
}

shinyApp(ui, server)

The problem is that this does not work when you publish to shinyapps.io. It must have something to do with the way I set up Docker, remoteServerAddr = "localhost", the port, etc., but I do not know how to solve this so that users can interact with my shinyApp to scrape websites that use javascript. Thanks for any help!

3

I'm having the same problem. I hope someone can help.