Hi,
I am getting the following error - I have a new PC. So, memory shouldn't be a problem. Any help will be appreciated.
Thanks,
Kamlesh
Hi, and welcome!
This problem doesn't require it, but keep in mind this FAQ: What's a reproducible example (`reprex`) and how do I create one? . Often answers without them go unanswered.
The best explanation is in the docs.
Error messages beginning cannot allocate vector of size indicate a failure to obtain memory, either because the size exceeded the address-space limit for a process or, more likely, because the system was unable to provide the memory. Note that on a 32-bit build there may well be enough free memory available, but not a large enough contiguous block of address space into which to map it.
I have been blessedly WIN-free for 10-years and can't offer specific tweaks. In the *nix world it is -ulimit.
If you are routinely going to be working with largish datasets that create this problem, you should consider renting EC2 instances with enough RAM to avoid this problem and do your pre-testing on a smaller sample.
New PCs can have low memory too. Can you tell us how much memory your system has and the size of your dataset?
Thanks for your response!
The size of my dataset is 61.7 MB and My CPU has the following configuration:
8GB RAM + 1TB HDD + 256 SSD
Regards,
Kamlesh
8GB is not an enormous amount of RAM, have in mind that there are other programs running in your computer that also consume memory so not all is available for R at any given moment and dplyr doesn't make in-place modifications so it is not very memory efficient.
You are going to have to manage your memory allocation more consciously, or look for other solutions like using data.table, using "on-disk" solutions like disk.frame, or using a sql server.
1 Like
it might also be helpful to use
sample_n()
from the dplyr package mentioned above, this will allow you to randomly sample 'n' number of rows from your data to be able to get your analysis running, and then come back and load more of the data into the program when more resource may come available
1 Like
Perfect! Thank you so much! I will try sample_n()
Thank you! I will see what can be done!
Thank you! I will look into it!
This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.