Only one step left to a full web-scrapping and table display with shiny and rvest. Please help.

[ronzenith]

I'd like to display a line graph using R shiny, with data from web-scrapping. I kind of succeed in scrapping with one day, but fail with a date range.

The following is my code for one day. I select the date by hard-coding the digits in R console (i.e. 20221018) since I fail to do so in ui:

rows <-
  read_html("https://www.immd.gov.hk/eng/stat_20221018.html") %>% html_elements(".table-passengerTrafficStat tbody tr")
prefixes <- c("arr", "dep")
cols <-
  c("Hong Kong Residents",
    "Mainland Visitors",
    "Other Visitors",
    "Total")
headers <-
  c(
    "Control_Point",
    crossing(prefixes, cols) %>% unite("headers", 1:2, remove = T) %>% unlist() %>% unname()
  )

df <- map_dfr(rows,
              function(x) {
                x %>%
                  html_elements("td[headers]") %>%
                  set_names(headers) %>%
                  html_text()
              }) %>%
  filter(Control_Point %in% c("Airport")) %>%
  mutate(across(c(-1), ~ str_replace(.x, ",", "") %>% as.integer())) %>%
  mutate(date = "2022-10-18")

ui <- fluidPage(dataTableOutput("T"))

server <- function(input, output) {
  output$T <- renderDataTable({
    df
  })
}

shinyApp(ui = ui, server = server)

Most Importantly, I fail to scrape the population data in a date range, and the code appears clumsy:

library(rvest)
library(dplyr)
library(tidyverse)
library(purrr)
library(shiny)

ui <- fluidPage(
  textInput("choice_company", "Enter name of a company"),
  dateRangeInput(
    "daterange",
    "Date range:",
    start  = "2022-10-01",
    end    = Sys.Date() - 1,
    min    = "2022-10-01",
    max    = Sys.Date() - 1,
    format = "yyyymmdd",
    separator = "/"
  ),
  textOutput("ShowUrl"),
  hr(),
  textOutput("ShowHtml"),
  dataTableOutput("T")
)

server <- function(input, output) {
  prefixes <- c("arr", "dep")
  
  cols <-
    c("Hong Kong Residents",
      "Mainland Visitors",
      "Other Visitors",
      "Total")
  headers <-
    c(
      "Control_Point",
      crossing(prefixes, cols) %>% unite("headers", 1:2, remove = T) %>% unlist() %>% unname()
    )
  
  theDate <- input$daterange[1]
  
  answer <- list() #empty list
  
  while (input$theDate <= end) {
    URL <- reactive({
      paste0("https://www.immd.gov.hk/eng/stat_",
             input$theDate,
             ".html")
    })
    
    rows <-
      read_html(url_data) %>% html_elements(".table-passengerTrafficStat tbody tr")
    
    df <- map_dfr(rows,
                  function(x) {
                    x %>%
                      html_elements("td[headers]") %>%
                      set_names(headers) %>%
                      html_text()
                  }) %>%
      filter(Control_Point %in% c("Airport")) %>%
      mutate(across(c(-1), ~ str_replace(.x, ",", "") %>% as.integer())) %>%
      mutate(date =  input$daterange[1])
    answer[[input$daterange[1]]] <- df
    input$daterange[1] <- input$daterange[1] + 1
    Sys.sleep(1)
    
    output$T <- renderDataTable({
      URL
    })
  }
}


shinyApp(ui = ui, server = server)

A complaint message is returned:

Warning: Error in $: Can't access reactive value 'daterange' outside of reactive consumer. i Do you need to wrap inside reactive() or observer()? 53: Error in input$daterange : Can't access reactive value 'daterange' outside of reactive consumer. i Do you need to wrap inside reactive() or observer()?

1. May I know what the complaint means?

2. How to fix the error?

3. If possible, how to translate the data to a line graph ?

Thank you so much in advance.