R version 4.1.0 Patched (2021-05-23 r80364) is blazing fast on my M1 iMac, but it slows to a crawl with RStudio version 1.4.1717 "Juliet Rose" (df86b69e, 2021-05-24). Any trouble shooting suggestions?
Since you are using a daily build superior to the preview version that is supposed to be R 4.1.0 compatible, I recommend you to file an issue on the GitHub repository (if there isn't a related one already).