R Shiny Flexdashboard and Docker

[ciarandeceol]

Hi all, I am hoping to access the hive mind to get some advice on working with Docker to deploy and R Shiny Flexdashboard. Everything is working fine locally, but containerising it has been giving me headaches for weeks. I'm all out of ideas so I'm coming here to try get some help!

Notes:

• I've made sure to download the base image openanalytics/r-shiny and also installed base R.
• As part of my workflow, I ensured I am using the correct ARM64 architecture for my system
• My Docker-Compose.yaml has no issues:

version: '3.9'
services:
shiny:
    build: 
      context: ./
    container_name: forecast_dashboard
    hostname: forecast_dashboard
    restart: always
        # the default port for Shiny apps
    ports:
      - 3838:3838
    networks:
        network
            driver: bridge

• Which leads me to believe there is an error with my Dockerfile. I just cannot see what is the issue. The Docker container builds without problems via docker-compose build. However, when I try run it via docker-compose up, it tries to run app.R but tells me it cannot find Tidyverse. This is despite Tidyverse being successfully installed during the container build. Also despite Tidyverse being the 2nd library I am loading in my app.R file, which tells me the first library I am loading, tidymodels, is being installed and loaded correctly.

# Use an official R base image
FROM r-base:4.3.0

# Set the working directory
WORKDIR /app

# Set the DOCKER_DEFAULT_PLATFORM environment variable
ENV DOCKER_DEFAULT_PLATFORM linux/arm64

# Install system dependencies
RUN apt-get update && apt-get install -y \
    libcurl4-openssl-dev \
    libssl-dev \
    libxml2-dev

# Install required R packages and their dependencies
RUN Rscript -e 'install.packages(c('devtools','shiny', 'shinyjs', 'plotly', 'reactable', 'bslib', 'modeltime', 'tidymodels', 'tidyverse', 'timetk', 'bigrquery'), repos = "https://cloud.r-project.org")'

# Copy the R.app file into the container
COPY app.R /app/app.R
COPY ui.Rmd /app/ui.Rmd

# Set the library path for R packages
RUN echo '.libPaths(c("/usr/local/lib/R/site-library", .libPaths()))' > /app/.Rprofile

# Set the entry point to run Rscript with the R.app file
ENTRYPOINT ["Rscript", "/app/app.R"]

Could anybody help me out? I have run out of ideas!

[Gabor]

Are your files in a public repo? It would be much easier to debug them if we could run them.

[ciarandeceol]

I appreciate the response. Unfortunately it is not possible to make the files public. However, everything runs without issue locally however. All packages have installed, and load. My app.R file runs without issue which in turn triggers my ui.RMD file which runs without issue.

I would be curious to hear if there are any known issues with Tidyverse and Docker? I even entered the Docker container with a bash terminal and saw that all the packages I installed are in fact installed correctly in /usr/local/lib/R/site-library. Tidyverse however, just will not load from here for some reason.

edit: a reproducible example. The app.R file, ui.RMD file, and data can be found below. This is the code which I found online originally and modified for my use case.
https://github.com/obrienciaran/r_forecasting

@Gabor

[Gabor]

Any reason why you cannot add the Dockerfile and the docker compose file to that repository, so we can run docker build, etc?

[ciarandeceol]

Its the exact same as above. I dropped it into the repo so its available when cloning to save you from copy and pasting. Thanks. Any insight would be invaluable.

[Gabor]

The tidyverse package does not load because it was not installed properly. install.packages() does not stop if an installation fails, but continues happily and you end up with a broken container. (See https://pak.r-lib.org/ if you want to try a better alternative.)

tidyverse does not install properly because some of its dependencies need system packages. These ones:

+ libcurl4-openssl-dev  - curl
+ libfontconfig1-dev    - systemfonts
+ libfreetype6-dev      - ragg, systemfonts, textshaping
+ libfribidi-dev        - textshaping
+ libharfbuzz-dev       - textshaping
+ libjpeg-dev           - ragg
+ libpng-dev            - ragg
+ libssl-dev            - curl, openssl
+ libtiff-dev           - ragg
+ libxml2-dev           - xml2
+ pandoc                - knitr, reprex, rmarkdown

Left is the system package, right is which R package needs it. You need to install these system packages first. Or you can use the nightly development version of pak (see All about installing pak. — Installing pak • pak), that will install system packages automatically.

Btw.

@ciarandeceol

RUN Rscript -e 'install.packages(c('devtools','shiny', 'shinyjs', 'plotly', 'reactable', 'bslib', 'modeltime', 'tidymodels', 'tidyverse', 'timetk', 'bigrquery'), repos = "https://cloud.r-project.org")'

is a syntax error, so you'll need to change some single quotes to double quotes.

Btw. 2.

@ciarandeceol

ENV DOCKER_DEFAULT_PLATFORM linux/arm64

this does not do anything within the Dockerfile, you need to set it outside, e.g. in docker-compose.yml you can add

    platform: linux/arm64

to the service.

[ciarandeceol]

Thanks for the detailed response. I would have never gotten to the end of this alone. Let me try this when i get back to my laptop and off mobile.

Btw1:
Well spotted. I copy/pasted old code.

Btw2
I was not aware of that so thank you, I will make the change.

[ciarandeceol]

@Gabor

tidyverse does not install properly because some of its dependencies need system packages. These ones:

+ libcurl4-openssl-dev  - curl
+ libfontconfig1-dev    - systemfonts
+ libfreetype6-dev      - ragg, systemfonts, textshaping
+ libfribidi-dev        - textshaping
+ libharfbuzz-dev       - textshaping
+ libjpeg-dev           - ragg
+ libpng-dev            - ragg
+ libssl-dev            - curl, openssl
+ libtiff-dev           - ragg
+ libxml2-dev           - xml2
+ pandoc                - knitr, reprex, rmarkdown

Can I ask where you found the info on these? I can't spot it in any logs from Docker or R.

I added all of the above packages to my Dockerfile it can now find and load Tidyverse, which is amazing, thank you! However, I am now getting the same error for another R library Error in library(modeltime) : there is no package called ‘modeltime’. I am assuming there are other system packages I need. I'd love to know how you found these.

[Gabor]

@ciarandeceol

Can I ask where you found the info on these? I can't spot it in any logs from Docker or R.

pak prints them before installing system requirements. Here are the ones for modeltime, tidymodels and timetk:

+ libcurl4-openssl-dev  - curl
+ libfontconfig1-dev    - systemfonts
+ libfreetype6-dev      - ragg, systemfonts, textshaping
+ libfribidi-dev        - textshaping
+ libharfbuzz-dev       - textshaping
+ libjpeg-dev           - ragg
+ libnode-dev           - V8
+ libpng-dev            - ragg
+ libssl-dev            - curl, openssl
+ libtiff-dev           - ragg
+ libxml2-dev           - xml2
+ pandoc                - knitr, loo, reprex, rmarkdown, rstan, rstantools, StanHeaders
+ pandoc-citeproc       - loo

This seems to be a superset of the ones for tidyverse, as modeltime depends on tidyverse.

[ciarandeceol]

Gotcha, thanks for that info. I manually installed the list earlier but will take a look at using Pak.

[ciarandeceol]

@Gabor

Here are the ones for modeltime, tidymodels and timetk:

I'm unfamiliar with pak. It's my first time to use it. The documentation isn't clear. I guess I have to either 1) manually install the system packages, or 2) use pak for this. But not both?

For example, the below is incorrect

...

# Install system dependencies
RUN apt-get update && apt-get install -y \
    libcurl4-openssl-dev \
    libssl-dev \
    libxml2-dev \
    libfontconfig1-dev \
    libfreetype6-dev \
    libfribidi-dev \
    libharfbuzz-dev \
    libjpeg-dev \
    libpng-dev \
    libtiff-dev \
    libnode-dev  \
    libjq-dev \
    pandoc \

RUN Rscript -e 'install.packages("pak", repos = "https://cloud.r-project.org")'

# Install remaining R packages using 'pak'
RUN Rscript -e 'library(pak); pak::pkg_install(c("shiny", "shinyjs", "plotly", "reactable", "bslib", "modeltime", "tidymodels", "tidyverse", "timetk", "bigrquery"))'

...

However, I'm sure pak itself has some system dependencies. My confusion lies in which ones I should remove and which ones I should keep from the apt-get install.

Something similar to this?:

...
# Install system dependencies
RUN apt-get update && apt-get install -y \
    libcurl4-openssl-dev \
    libssl-dev \
    libxml2-dev

# Install 'devtools' package
RUN Rscript -e 'install.packages("pak", repos = "https://cloud.r-project.org")'

# Need to set this environment variable for liv8-dev enginer
ENV DOWNLOAD_STATIC_LIBV8=1

# Install remaining R packages
RUN Rscript -e 'library(pak); pak::pkg_install(c("shiny", "shinyjs", "plotly", "reactable", "bslib", "modeltime", "tidymodels", "tidyverse", "timetk", "bigrquery"))'
...

Ultimately, I am unsure how to alter to RUN apt-get update && apt-get install -y portion and the install.packages portion of the Dockerfile.

Sorry for the persistent questions. Pak seems like the solution I need, I'm just unfamiliar with its usage!

[Gabor]

Use the installation instructions I linked to above. E.g.

RUN R -q -e 'install.packages("pak", repos = sprintf("https://r-lib.github.io/p/pak/%s/%s/%s/%s", "devel", .Platform$pkgType, R.Version()$os, R.Version()$arch))'

And then

RUN R -q -e 'pak::pkg_install(c("shiny", "shinyjs", "plotly", "reactable", "bslib", "modeltime", "tidymodels", "tidyverse", "timetk", "bigrquery"))'

[ciarandeceol]

I kept getting various errors with the installation instructions. I just kept adding various system dependencies as per the errors I was getting; and also manually installed rstan. It seems to work fine now.

Thank you for the support, you've been a huge help today and I really appreciate it.

...
# Install system dependencies
RUN apt-get update && apt-get install -y \
    libcurl4-openssl-dev \
    libssl-dev \
    libxml2-dev \
    libfontconfig1-dev \
    libfreetype6-dev \
    libpng-dev \
    libtiff5-dev \
    libjpeg-dev \
    libharfbuzz-dev \
    libfribidi-dev

# Need to set this environment variable for the modeltime package to run
ENV DOWNLOAD_STATIC_LIBV8=1

# Install 'pak' library
RUN Rscript -e 'install.packages("pak", repos = "https://cloud.r-project.org")'

# Manually install rstan first as installing them later with pak causes issues
RUN Rscript  -e 'install.packages("StanHeaders",repos = c("https://mc-stan.org/r-packages/",getOption("repos")))'
RUN Rscript  -e 'install.packages("rstan",repos = c("https://mc-stan.org/r-packages/",getOption("repos")))'

# Install additional R packages using 'pak'
RUN Rscript -e 'library(pak); pak::pkg_install(c("shiny", "shinyjs", "plotly", "reactable", "bslib", "modeltime", "tidymodels", "tidyverse", "timetk", "bigrquery"))'

...

[Gabor]

Great! For the record, pak has no system requirements if you install it the way I linked to in my previous comment.

[ciarandeceol]

@Gabor

For the record, pak has no system requirements if you install it the way I linked to in my previous comment.

I will check this nightly build version out in future. It sounds cleaner. Thanks again!