If you need some of the new functionalities or you are experiencing some bug with your current setup, then it would be a good idea.
RStudio is an IDE for the R programming language but it has not direct influence on your R package library. This will only be the case if you update your R version.
There is no need to delete any file, unless you specify a custom location for your new installation, the new one is going to overwrite the old one.
There are some befit to upgrade if your using 4.1.1(1) or R 4.1.3, and I will be updating the other threads involving the Rstudio saving bug to reflect that.
If there are issues with your Windows build(2) or for some reason your Business One Drive accidently has R 4.2.0 or any R file in %App-Data% and Documents, then in the latest version of R-studio you will get a consistent error message about being unable to locate the file or file permissions. I believe that getting meaningful error codes now try to save/open R files in older versions of R is a success even if a small one
Footnotes (as the details are not directly on topic):
(1)This is a complete aside: My organization found that R 4.2.0 does not function as expected on using Window 10 update build prior to the build published before December 2021. The error codes we got from testing the latest release help us discover that R 4.2.0 does not work on a re-imaged Windows device as is done when we re-issue computer here in county government.