Broom in sparklyr

I have a data frame of about 20 GB. I need to read it into Sparklyr since my RAM is 16 GB. I am not sure if I could use the functionality of broom in sparklyr. For example, I want to know if the code below works in sparklyr, because I need to run a lot of regression for each my group and then combine the estimated fitted value and residual with my data.

regressions <- mtcars %>%
  nest(-am) %>% 
    fit = map(data, ~ lm(wt ~ mpg + qsec + gear, data = .x)),
    augmented = map(fit, augment)

regressions %>% 
  unnest(augmented) might helps.

1 Like