If I run the following loop a as a job, everything works fine:
library(tidyverse)
initial_date <- '2020-01-01'
loop_df <- mtcars[0, ]
dates <- seq(as.Date(initial_date), as.Date("2020-01-3"), by = 1)
for(d in seq_along(dates)) {
loop_date = dates[d]
loop_row = mtcars[d, ] %>% mutate(loop_date = loop_date)
loop_df = loop_df %>% bind_rows(loop_row)
}
loop_df
If I try to source part of the loop, I can make it work using local = T Here I just replace the loop_row part to come from a sourced script e.g.
initial_date <- '2020-01-01'
loop_df <- mtcars[0, ]
dates <- seq(as.Date(initial_date), as.Date("2020-01-3"), by = 1)
for(d in seq_along(dates)) {
loop_date = dates[d]
source("example.R", local = T) # contains a single line of code "loop_row <- mtcars[d, ] %>% mutate(loop_date = loop_date)"
loop_df = loop_df %>% bind_rows(loop_row)
}
loop_df
So far everything works fine when running as a job in rstudio. If I save the results to an environment object I can see variable loop_df as the first 3 rows of mtcars along with a loop date field:
mpg cyl disp hp drat wt qsec vs am gear carb loop_date
1 21.0 6 160 110 3.90 2.620 16.46 0 1 4 4 2020-01-01
2 21.0 6 160 110 3.90 2.875 17.02 0 1 4 4 2020-01-02
3 22.8 4 108 93 3.85 2.320 18.61 1 1 4 1 2020-01-03
However, I would like to call that last line, loop_df <- loop_df %>% bind_rows(loop_row) by sourcing a file within example.R source("example2_nested.R", local = T) where source("example2_nested.R", local = T) contains one line of code:
loop_df <- loop_df %>% bind_rows(loop_row)
This works when I run directly in the console. But it does not when I try to run as a job. I get error:
Error in eval(lhs, parent, parent) : object 'loop_df' not found
Calls: sourceWithProgress ... source -> withVisible -> eval -> eval -> %>% -> eval -> eval
Execution halted
Full code is below. If I run main.R in the console I get the desired result of a 3 row df. If I run as a job it fails with the above error message:
# main.R
initial_date <- '2020-01-01'
loop_df <- mtcars[0, ]
dates <- seq(as.Date(initial_date), as.Date("2020-01-3"), by = 1)
for(d in seq_along(dates)) {
loop_date = dates[d]
source("example.R", local = T)
}
loop_df
# example.R
loop_row <- mtcars[d, ] %>% mutate(loop_date = loop_date)
source("example2_nested.R")
# example2_nested.R
loop_df <- loop_df %>% bind_rows(loop_row)
How can I run this as a job?