tidyverse and GDAL based packages doen't work in Shiny with Docker

[Leprechault1]

I create a Dockerfile:

# Base image https://hub.docker.com/u/rocker/
FROM rocker/shiny:latest

# system libraries of general use
## install debian packages
RUN apt-get update -qq && apt-get -y --no-install-recommends install \
    libxml2-dev \
    libcairo2-dev \
    libsqlite3-dev \
    libmariadbd-dev \
    libpq-dev \
    libssh2-1-dev \
    unixodbc-dev \
    libcurl4-openssl-dev \
    libssl-dev \
    libproj-dev \
    gdal-bin 

## update system libraries
RUN apt-get update && \
    apt-get upgrade -y && \
    apt-get clean

RUN install2.r --error \
    methods \
    jsonlite \
    tseries
    

RUN R -e "install.packages(pkgs=c('shiny','leaflet','leaflet.providers','ggplot2','dplyr','shinythemes','ggspatial','googleCloudStorageR','sf','lubridate','googleAuthR','blastula','shinymanager','tiff','raster','scales','sp','RColorBrewer','anytime','rmarkdown','knitr','zip','evaluate','highr','methods','markdown','stringr','yaml','xfun','formatR','testit','digest','rgl','codetools','htmlwidgets','webshot','tikzDevice','tinytex','reticulate','JuliaCall','magick','png','jpeg','gifski','xml2','showtext','tibble','sass','bslib','tidyverse','bigrquery','DBI','glue','geojsonR','fst'), repos='https://cran.rstudio.com/',force=TRUE,dependencies=TRUE)" 

RUN mkdir /root/app

COPY R /root/shiny_save

EXPOSE 8080

# RUN dos2unix /usr/bin/shiny-server.sh && apt-get --purge remove -y dos2unix && rm -rf /var/lib/apt/lists/*
CMD ["R", "-e", "shiny::runApp('/root/shiny_save', host='0.0.0.0', port=8080)"]unApp('/root/shiny_save', host='0.0.0.0', port=8080)"]

But when I run inside Docker, several packages are not installed. Like:

Please, why did this occur? Despite the install package specifications, and all the packages are found in cran:

RUN R -e "install.packages(pkgs=c('shiny','leaflet','leaflet.providers','ggplot2','dplyr','shinythemes','ggspatial','googleCloudStorageR','sf','lubridate','googleAuthR','blastula','shinymanager','tiff','raster','scales','sp','RColorBrewer','anytime','rmarkdown','knitr','zip','evaluate','highr','methods','markdown','stringr','yaml','xfun','formatR','testit','digest','rgl','codetools','htmlwidgets','webshot','tikzDevice','tinytex','reticulate','JuliaCall','magick','png','jpeg','gifski','xml2','showtext','tibble','sass','bslib','tidyverse','bigrquery','DBI','glue','geojsonR','fst'), repos='https://cran.rstudio.com/', force=TRUE)"

Thanks in advance!

[keithn]

Those look like the logs from once the R application has started to run where it's discovering that some packages aren't available. What we really need to see are the logs from when the docker container attempted to install those R packages, i.e. logs from the stage where it ran

@Leprechault1

RUN R -e "install.packages(pkgs=c('shiny','leaflet','leaflet.providers','ggplot2','dplyr','shinythemes','ggspatial','googleCloudStorageR','sf','lubridate','googleAuthR','blastula','shinymanager','tiff','raster','scales','sp','RColorBrewer','anytime','rmarkdown','knitr','zip','evaluate','highr','methods','markdown','stringr','yaml','xfun','formatR','testit','digest','rgl','codetools','htmlwidgets','webshot','tikzDevice','tinytex','reticulate','JuliaCall','magick','png','jpeg','gifski','xml2','showtext','tibble','sass','bslib','tidyverse','bigrquery','DBI','glue','geojsonR','fst'), repos='https://cran.rstudio.com/',force=TRUE,dependencies=TRUE)" 

and work backwards from there in diagnosing this issue.

[michaelmayer]

In addition to what @keithn already mentioned, I would recommend to use pak for the package installation.

RUN R -e "install.packages('pak')"
RUN R -e "options(repos=c(CRAN="https://packagemanager.posit.co/cran/latest"));pak::pkg_install(c('shiny','leaflet','leaflet.providers','ggplot2','dplyr','shinythemes','ggspatial','googleCloudStorageR','sf','lubridate','googleAuthR','blastula','shinymanager','tiff','raster','scales','sp','RColorBrewer','anytime','rmarkdown','knitr','zip','evaluate','highr','methods','markdown','stringr','yaml','xfun','formatR','testit','digest','rgl','codetools','htmlwidgets','webshot','tikzDevice','tinytex','reticulate','JuliaCall','magick','png','jpeg','gifski','xml2','showtext','tibble','sass','bslib','tidyverse','bigrquery','DBI','glue','geojsonR','fst')"

The above will make your whole section "install debian packages" obsolete. pak will not only install all of the requested R packages, it will also install all the missing system dependencies, for example, without you ever needing to think about them again. Additionally, pak will also install packages in parallel, leading to further speedup in docker build time.

PS: On a little side node: One of the many reasons why package sf is not found/installed with your current approach is the fact that you have added gdal-bin as a dependency but you have missed out on libgdal-dev. The -dev packages are in particular needed for packages that need compilation from source.