I copied and pasted the whole greek UTF-8 table from here into my book just for testing. All the greek characters render fine in HTML but with XeLeTeX only some of them do.
You can see in this subset (colored boxes added for emphasis) that the core uppercase greeks work fine but not the lowercase:
Interestingly, this is the case whether I surround the text with code ticks ``` or not. Even though code text uses a different font than body text.
So I decided that this must be a font issue. After re-reading a lot of the Bookdown book I realized that I can change the font by changing a few settings in my index.Rmd file. So as an experiment I added the following to my YAML:
mainfont: Arial
monofont: "Courier New"
monofontoptions: "Scale=0.7"
Future travelers should note that the above font options are ONLY applicable when using XeLaTeX... other LaTeX engines use different options to change fonts.
Then when I rendered my book to PDF all worked as expected:
So the issue was the default font used by XeLeTeX.