I have a new article on speeding up R by parallelization that I want to share: Speed up your R Work. I show how to speed up code from any of the packages data.table
, dplyr
, or rqdatatable
(and it also works with base-R
). In particular I think the rquery
/rqdatatable
packages are really going to be something big, if given the chance. Please try them out!
3 Likes
Have you considered setDTthreads(1)
for the data.table
code to avoid nested parallelism and to separate the data.table
and parallel components?
Good point for timings. For actual execution I think just leaving the threads "as is" is fine.