parLapply in nested function

[ace]

I have a script like the one below, with a function called within its own definition - using a cluster object cl. The problem is that each iteration of the first for loop will run in parallel, whereas the same computation within the if statement will use only 1 core. There's no error or warning, it just uses one core.

Unfortunately this MWE doesn't reproduce the problem, its too tricky for me to reproduce at this stage. I'll try to fix it myself, I was just wondering whether anyone had encountered this before or knows whether there's a rule against parallelism in nested functions?

cl <- parallel::makeCluster(detectCores())

prod_exp <- function(x){prod(log((exp(x)))}

MWE <- function(cl, y, threshold){
  
  lfunc <- function(...){parallel::parLapply(cl, ...)}
  
  for(i in 1:10){
    y <- y * as.numeric(lfunc(X = rnorm(1e6), fun = prod_exp))
  }
  if(y < threshold){
    print("resampling")
    y <- MWE(cl, y, -Inf)
  }
  return(y)
}

MWE(cl, 3, 5)

Thanks for your help.

[ace]

Nevermind, I sorted it out. The input differed slightly, slowing down the nested function.

[mara]

If your question's been answered (even if by you), would you mind choosing a solution? (See FAQ below for how).

Having questions checked as resolved makes it a bit easier to navigate the site visually and see which threads still need help.

FAQ: How do I mark a solution?

If your question has been answered, don't forget to mark the solution! How do I mark a solution? Find the reply you want to mark as the solution and look for the row of small gray icons at the bottom of that reply. Click the one that looks like a box with a checkmark in it: [image] Hovering over the mark solution button shows the label, "Select if this reply solves the problem". If you don't see the mark solution button, try clicking the three dots button ( ••• ) to expand the full set of options. When a solution is chosen, the icon turns green and the hover label changes to: "Unselect if this reply no longer solves the problem". Success! [solution_reply_author] …

Thanks