I believe in learning Statistics and a programming language together as you need a computer language or at least Excel to try out the ideas. And I'm not sure if Tidyverse is suitable/enough.

There are many Statistics in R books but nothing for Statistics in Tidyverse. What's the difference and what's missing from Tidyverse for this propose.

I know Tidyverse is not everything but so far I haven't encountered a clear overview/explanation what Tidyverse can't deliver and how to connect whatever from the rest of R language to enhance the approach learning Statistics (primarily) in Tidyverse.

The tidyverse is a set of tools for interactive data analysis. The goal isn't to be self-contained, but, rather, to make working with other packages and data in R easier. tidymodels might give you a better sense of the ways in which tidyverse tools complement other statistical packages:

Preaching to the choir! My favourite is when you're trying to describe a type of visualization. If you can't point to something, or doodle it out, it can be nigh impossible!