Deep learning in R Memory allocation


After extracting features of 539 images from training set and 244 images from test set using application_xception() model in Keras for Rstudio, then built a randomForest model and evaluated the model performance.

The whole process took around 1.35 hours,during the execution, got a message saying a certain object took more than 10% of memory, just want to know the limitations of deep learning in R.

After referring documentations in R and github, below points I have considered as the limitation

•In order to work with Keras in Rstudio, R matrices and arrays are marshaled to python to convert them to NumPy arrays.

•Implies that the NumPy array will be Fortran-ordered(default in R), rather than C-ordered which is default for NumPy library.

•In the case of Keras training, between shuffling, drawing batches, copying to the GPU/CPU, and 10 epochs there are a ton of accesses to the Fortran-ordered array which add up to a ~ 40% performance penalty.

•The performance penalty causes R to consume more time than Python.

•Deep learning models can be implemented in R as well, with the cost of time.

1 Like