---
title: title
author:
- Noone
output:
bookdown::pdf_document2:
keep_tex: yes
latex_engine: xelatex
fig_caption: yes
number_sections: yes
toc: yes
toc_depth: 3
---
This is a $\mathbb{R}$.
This is mathcal : $\mathcal{ABC}$

when knitted it looked like what I have mentioned.

Besides, I use a full Texlive installation. And I use no pandoc templates.

I think I got it. The symbol you want is in the LaTeX package (amsmath} so you need to load it.

I suspect this is very inelegant but try

---
title: The Case of the mysterious *R*
date: today
author:
- Noone
output:
bookdown::pdf_document2:
keep_tex: yes
latex_engine: xelatex
fig_caption: yes
number_sections: yes
toc: yes
toc_depth: 3
header-includes:
\usepackage{amsmath}
---
This is a $\mathbb{R}$.