Memory Issue in R

[yash_yp]

Hello,

I am facing an issue in RStudio while using FuzzyJoin package.

Error: "Cannot allocate vector of size 1005.9 Mb"

I also tried checking for memory.limit(), it gives an output "4014"

Please help me resolve this issue.
Thanks in advance

[mkulkarni]

can you please share the chunks of code?

[yash_yp]

Hi,

This is the piece of code -

joined_file <- fuzzy_left_join(
file1, file2,
by = c(
"Type" = "Description",
"Date" = "EffectiveDate",
"Date" = "ExpiryDate"
),
match_fun = list(==, >=, <=)
) %>%
select(Key,Date, Type, Num_Rate)

[mkulkarni]

First Divide the code
and add print statements and look what happens in background on Console

joined_file <- fuzzy_left_join(
file1, file2,
by = c(
"Type" = "Description",
"Date" = "EffectiveDate",
"Date" = "ExpiryDate"
),
match_fun = c("==" ,  ">=" , "<=")
) 

print("joined_file")
print(head(joined_file, 1000)

new_db<- dplyr::select(Key,Date, Type, Num_Rate)

print("new_db")
print(head(new_db, 1000)

In this way you can understand where the error occurs

[nirgrahamuk]

It seems like a straightforward case of biting off more computation than you can chew.
Be careful to remove all other objects from memory than those that are key for your present calculation.
You might need to iterate over subsets of your file1 and stitch the results together. I'd look at purrr for that.