Yes, but you only need to load the packages whose functions you are actually using in that session. You don’t need to load all the packages you have installed.
R loads a small set of core packages by default (as described in one of @EconomiCurtis’s links). For the rest, it’s up to the user. Most people start out just giving R commands at the console, but eventually you should move toward writing scripts. The first few lines of most scripts are library() statements loading all the packages necessary for that script.
Since you’re just getting started, here’s another tip to help you navigate the landscape better: R and RStudio are different things (see: Differentiating R from RStudio). What we’re discussing in this thread is actually how R works (regardless of whether you’re using RStudio or not).