code parallelisation in RStudio

[vtosevski]

Dear Community,

I have been using R (mostly through RStudio) for many years now and I am by no means a software developer, rather a user (analyzing large quantities of biomedical data).

How does one begin to troubleshoot issues with parallelization through RStudio?

Most of the time I am interactively building analysis pipelines. For parallel computation I use either future or parallel packages. Future is not supported from within RStudio, this is clearly stated when one tries. In the past, parallel package used to work (on-off, sometimes it would, sometimes it wouldn't) but recently I cannot get a simple mclapply to run from RStudio. Running the same code from R session launched from Terminal works just fine (in parallel, but also future). So, I am guessing either my system is somehow different now (currently on Ubuntu 20.04) or RStudio is different (1.3.1073).

Can someone point me in the right direction, to start troubleshooting? Right now I am switching between Terminal and RStudio and that is very frustrating and cumbersome.

Appreciate any input I can get on this.

Thanks,
Vinko

[nirgrahamuk]

@vtosevski

Future is not supported from within RStudio, this is clearly stated when one tries.

I think that's an oversimplification. Multicore mode based on forking is not supported, but multisession is, which will cover a lot of use cases. Perhaps look into that?

[vtosevski]

Thanks for the sound comment Nir! You're absolutely right, it was an oversimplification, and multisession plan might indeed be a solution for me. I've mistaken the RStudio error message to be about the whole future framework, when instead, it was about the multicore plan. Then, I completely lost sight of the fact that all the *apply function have been "futurised" by now. This covers a lot of ground for me, thanks for pointing that out. I have to tweak some of my code and learn more about random number generation in future framework, but I am optimistic.
Thanks a lot!
Vinko