rowSums crashing when dttm is first column

tidyverse
1

Not sure which update caused my code to begin crashing, but rowSums doesn't seem to work anymore when the first column is dttm.

I have a tibble of timeseries data. I want to sum across all the numeric columns where the number of numeric columns varies in each loop. My original code looked like this:

  sub_data <- sub_data %>%% 
    replace(is.na(.), 0) %>%
    mutate(sum = rowSums(.[-1])) %>%   
    select(DateTime, sum)

Now I get an error that says:
Error in [<-:
! Assigned data values must be compatible with existing data.
??? Error occurred for column DateTime.
Caused by error in vec_assign():
! Can't convert to <datetime>.

My solution was to drop the DateTime column and sum across all the rows and then cbind the columns back together, which is not very elegant.

  sub_data <- sub_data %>%
    select(-DateTime) %>% 
    replace(is.na(.), 0) %>%
    mutate("sum" = rowSums((.))) %>%  
    select(sum)

Silly I know, but was curious as to why I couldn't get rowSums to work without dropping the "DateTime" column. I even tried various versions of the mutate statement that specifically calls out "DateTime" with a setdiff

2

I suspect it has something to do with sub_data pre-processing. Can you confirm if you get the same error on a very simple data frame, such as:

library(tidyverse)

set.seed(1)

sub_data <- tibble(
  DateTime = ymd_hms("2010-08-03 00:50:50", "2012-08-03 15:50:50"),
  other = rnorm(2)
)

sub_data %>%
  replace(is.na(.), 0) %>%
  mutate(sum = rowSums(.[-1])) %>%   
  select(DateTime, sum) 
#> # A tibble: 2 × 2
#>   DateTime               sum
#>   <dttm>               <dbl>
#> 1 2010-08-03 00:50:50 -0.626
#> 2 2012-08-03 15:50:50  0.184

sub_data %>%
  select(-DateTime) %>% 
  replace(is.na(.), 0) %>%
  mutate("sum" = rowSums((.))) %>%  
  select(sum) 
#> # A tibble: 2 × 1
#>      sum
#>    <dbl>
#> 1 -0.626
#> 2  0.184

Created on 2024-12-10 with reprex v2.1.0

3

I don't believe it's a pre-processing issue. An example dataset looks more like this:

library(tidyverse)

set.seed(1)

sub_data <- tibble(
  DateTime = ymd_hms("2010-08-03 00:50:50", "2012-08-03 15:50:50"),
  other = rnorm(4)
)

colnames(sub_data) <- c("DateTime" , `TD02-102`, `TD02-116`, `TD02-128`, `TD02-132`)

head(sub_data)

  sub_data <- sub_data %>%
    replace(is.na(.), 0) %>%
    mutate(sum = rowSums(.[-1])) %>% 
    select(DateTime, sum) 

rlang::last_trace()
<error/tibble_error_assign_incompatible_type>
Error in `[<-`:
! Assigned data `values` must be compatible with existing data.
ℹ Error occurred for column `DateTime`.
Caused by error in `vec_assign()`:
! Can't convert <double> to <datetime<UTC>>.
---
Backtrace:
     ▆
  1. ├─... %>% select(DateTime, sum)
  2. ├─dplyr::select(., DateTime, sum)
  3. ├─dplyr::mutate(., sum = rowSums(.[-1]))
  4. └─base::replace(., is.na(.), 0)
  5.   ├─base::`[<-`(`*tmp*`, list, value = `<dbl>`)
  6.   └─tibble:::`[<-.tbl_df`(`*tmp*`, list, value = `<dbl>`)
  7.     └─tibble:::tbl_subassign_matrix(x, j, value, j_arg, substitute(value))
  8.       ├─base::withCallingHandlers(...)
  9.       └─tibble:::vectbl_assign(x[[j]], cells[[j]], value)
 10.         └─vctrs::vec_assign(x, i, value)
4

Well this is not possible: in a tibble (or data frame), the columns must all have the same number of rows. If I try running this example code, I get the expected error:

library(tidyverse)

set.seed(1)

sub_data <- tibble(
  DateTime = ymd_hms("2010-08-03 00:50:50", "2012-08-03 15:50:50"),
  other = rnorm(4)
)
#> Error in `tibble()`:
#> ! Tibble columns must have compatible sizes.
#> • Size 2: Existing data.
#> • Size 4: Column `other`.
#> ℹ Only values of size one are recycled.

Created on 2024-12-10 with reprex v2.1.0

Can you create a reproducible example that gives this error?
If your data is not confidential, maybe you can share the result of dput(head(sub_data)) so we can work with your actual data.

5

Upon further investigation, it seem you are right and the error is a pre-processing one. There is an NA in the DateTime field and it couldn't coerce the datetime object to a double.

Thanks for looking into this though. Much appreciated!

1 Like