Updates to dplyr/vctrs crash shiny app

[avelarius]

Hello dear RStudio Community!

I have developed a web app for scientists in my research field to help parse, visualise, inspect and clean (very specific) data. It is hosted on shinyapps.io (no advertisement intended) and it works well enough (source code on github).
I haven't worked on the source code in a while and wanted to publish my web app as part of my research project. To my astonishment and irritation, the web app is completely broken due to updates - in tidyverse packages, as I found out after a while.

I am faced with this ominous message that I just can't trace back (I have tried using browser(), checked every place where I use bind_rows() and printed intermediate results to the console):

Warning: Error in dplyr::bind_rows: Can't combine `MI18-L8-A$value` <character> and `MI18-L8-B$value` <logical>.                                                                                  
  3: runApp
  2: print.shiny.appobj
  1: <Anonymous>
Warning: Error in dplyr::bind_rows: Can't combine `MI18-L8-A$value` <character> and `MI18-L8-B$value` <logical>.
  3: runApp
  2: print.shiny.appobj
  1: <Anonymous>

MI18-L8-A and MI18-L8-B are the names/identifiers of the two sediment cores/datasets in the sample data that are merged upon parsing the raw data.

What I know so far:

• When I print/inspect intermediary results, I can't find the step that throws the error.
• The main parsing is done in an encapsulated function. The result seems to be fine when using browser()
• I am not sure if this error could be caused by unwanted reactivity, but...
• The above error is very clearly a consequence of the stricter type checking dplyr 1.0 is now doing
• I do not understand why it says MI18-L8-A$value when MI18-L8-A is not a dataframe.
• It doesn't matter which files I use, there is always one "character" and one "logical".

Long story short: I have no idea where this error comes from and before I'm rewriting my whole app not to use the tidyverse, I hope I can profit from the experience of this community. Has anyone experienced something similar?

[nirgrahamuk]

• It doesn't matter which files I use, there is always one "character" and one "logical".

Have you tried it with the same data twice. i.e. copy one file, and then have your app load those two files.

[cnbrownlie]

Just in case you weren't aware when debugging, purrr::map_dfr uses dplyr::bind_rows under the hood so I think that is probably where the error is originating from (e.g. where you're using enframe and map_dfr in your server.R)

[avelarius]

That does work indeed!...so the problem only arises with different files or actually only if the name is different...

[avelarius]

I was aware that purrr::map_dfr uses bind_rows but didn't think of enframe...that might be the culprit. I'll check those parts a bit more closer...Thanks for the suggestion!

[nirgrahamuk]

I would start with the easiest things to 'fix'
for example if you have requirements for the files that are loaded in, then you can fully instruct read_csv how to interpret the data, by defining an explicit column type spec ( col_types) rather than having it guess per file.
i.e.

makeTidyData <- function(tmppath, counter, n) {
  tmp <- read_csv(tmppath, quote = "") %>%
    distinct()

[avelarius]

I would do this but in this case, the structure of the files is...messy. There is a certain number of columns that are fixed/tidy, but then there are many columns following containing the actual data: for every element that is measured, there are multiple columns (number can vary due to program settings) that record associated values - Those columns then have to by pivoted/gathered and cleaned up.

[nirgrahamuk]

you may need to add additional code to tidy your data further then, particularly in managing column that end up as different types across different files... it seems from what you are saying that this hasn't been directly addressed.

[avelarius]

I am very grateful for the inputs. I have found the culprit for this issue (on github, it's in server.R on lines 150, 157, 598, 603).

I'm not exactly sure how the problem arises...my best guess is that enframe() guesses column types and them map_dfr complains about differing types? In my (not so reproducible) example further below, the problem does not arise.

Problematic original code, where excludeddata$exclelem is a reactiveValue holding deselected elements on the diagnostics page. excludeddata$exclelem holds a list for every core name (with the core name as list item name, e.g. MI18-L8-A) and the list content is initially NULL but then populated with deselected elements :

    elemdf <-
      map_dfr(excludeddata$exclelem, ~ enframe(., name = NULL), .id = "CoreID") %>%
      mutate(elem_removed = TRUE) %>%
      rename(Element = value)

The problem was solved by simply encasing the data in enframe in as.character():

    elemdf <-
      map_dfr(excludeddata$exclelem, ~ enframe(as.character(.), name = NULL), .id = "CoreID") %>%
      mutate(elem_removed = TRUE) %>%
      rename(Element = value)

I tried to make a reprex but I do not run into an error when trying to combine the rows:

library(tidyverse)

nulled_list_names <- list("A" = NULL, "B" = NULL, "C" = NULL)

# This causes the list name column to be a character but the value of the list, i.e. Element is logical.
rowbound_df_names <-
  map_dfr(nulled_list, ~ enframe(., name = NULL), .id = "CoreID") %>%
  mutate(elem_removed = TRUE) %>%
  rename(Element = value)

# This causes all initally created columns to be of type character.
rowbound_df_typecast_names <-
  map_dfr(nulled_list, ~ enframe(as.character(.), name = NULL), .id = "CoreID") %>%
  mutate(elem_removed = TRUE) %>%
  rename(Element = value)