Problems when I try to invoke a system function using package {processx}

[HugoSantarem]

I'm trying to run an exe file by invoking a system command through R's IDE RStudio. My OS is Windows.

I have the following hierarchy for directories:

|_scripts
|_files
  |_prog.exe
|_data
  |_executed_data
    |_case1
    |_case2

My exe file is stored at the files directory. The wrangled, ready-to-use data are stored in specific case# directories, where # represents the respective number for that case.

The code is modular. I write functions and then use purrr::map to iterate that function over a vector of the same length as the number of cases.

Here's how the code looks:

#Library(processx)
#Library(purrr)

run_exe = function(directory) {
   
  # Copy .exe file to folder to run
  from_dir = c("../files/prog.exe")
  to_dir = paste0("../data/executed_data/", directory)

  file.copy(from = from_dir,  to = to_dir, overwrite = T)
  
  # Run .exe file
  processx::run( command = paste0("../data/executed_data/",directory,"/prog.exe" )
}

# Run function over all cases
all_cases = list.files("../data/executed_data")
purrr::map(all_cases, run_exe)

The attempt to run with map fails and a message of error is printed to the console. The printed message says one of the subroutines of the exe file can't find a specific file, hence, the error.

I noticed that the supposed missing file has its name printed in lower case so I renamed the file to make sure all letters were small. Still, it didn't work.

The message also says an error.err file has been printed. This file does get printed when an error happens, however, if I go directly to one of the cases# directory and double click on the exe file, it runs with no problem. Since it runs with no problem, I know there's no missing file.

Given this question deals with running an exe file, I didn't think a reproducible example would be possible. If, however, you have had similar issues when trying to invoke a system function, please share the solution you found.

P.S.: Before using processx::run, I toyed with the system, system2, and sys functions but, again, no success.

[HanOostdijk]

It is not clear to me what you would like to achieve.
Questions:

• Why is it necessary to copy prog.exe?
• Is it because prog.exe works only on the directory it sits in?
• Do you realize that while you loop over the case directories the working directory does not change?

[HugoSantarem]

Hi, thanks for your input.

1. It may not be necessary to copy prog.exe. I just thought it'd be easier at first. I actually tried the file.copy command with a setwd(). After copying it, I'd set the working directory to the current directory and then I'd run run(command = "./prog.exe") but it didn't work.

2. No.

3. I did, thanks for pointing it out. The problem is that I tried both

• using setwd() and then run(command = "./prog.exe") - as mentioned above
• using run( command = paste0("cd ../data/executed_data/",directory,"/prog.exe")
run( command = "./prog.exe")

But none worked.

I think the solution would be to find a way to set directories for each case as the code loops over all the possible directories. Then call run( command = "../files/prog.exe).

[HanOostdijk]

Questions:

• How does prog.exe knows on which dataset(s) to operate?
• Is it possible for you to replace prog.exe by a simple executable to see if that does work?
• I understand prog.exe works when do it manually: switch to the directory and execute it?

[HugoSantarem]

1. It understands on which datasets to operate when I "tell" it where to run, hence, my desire to find a way to set the directory. The ways I tried didn't work.

2. I don't understand your point. prog.exe is a simple executable. If I go to the console, set the directory to my current case, and run run(command = "/prog.exe"), prog.exe will run with no errors.

3. No, as I've done this before using shell, once the dataset directory is set as the working directory, it'll run even if given a relative path.

[HanOostdijk]

Then please include the code in the setwd version.

[HugoSantarem]

One (inelegant) possible solution consists of:

1. Copy prog.exe to all cases directories.
2. Run a for-loop that will automatically change the directory with setwd().
3. Inside that same loop, call run(command = "./prog.exe")
4. As the loop finishes, setwd() back to Rproj directory.

This is, however, a very non-efficient, non-reproducible way of getting things done. Also, it's my experience that trying to use setwd() inside a long loop is a recipe for disaster.

[HanOostdijk]

Another disaster is of course the last thing we want.
Regarding the temporarily setting of the working directory you probably are familiar with withr::with_dir .
To summarize the discussion above:

• you want to execute prog.exe for each of a number of directories
• the prog.exe needs to run in such a directory i.e. that directory has to be the working directory
• you are hesitant to change the working directory

I must admit that given this setting, I have no solution for you. Maybe someone else ... ?

[HugoSantarem]

You may not have a solution, HanOostdijk, but I have to thank you for you sharing your knowledge and taking time for this exchange of ideas. I'll add that I did not know withr::with_dir. Thank you again for letting me know of another tool.

People like you are the ones who make the R community so welcoming to everyone. =)

[HugoSantarem]

This will do the trick:

# List all directory paths. These are the directories you have your data in.
all_dir = dir(here::here('data', 'executed_data'), full.names = TRUE)

# Function to process all your data. Note the arg "wd"
process_data = function (data_dir) {
    processx::run(here::here('files', 'prog.exe'), wd = data_dir)
}

# Iterate over all directories
purrr::map(.x = all_dir, .f = process_data)

It's important to have the .Rproj at the same level as directories scripts, files, data.

[HanOostdijk]

Nice!

And thanks for closing the issue in this way!