Actually solved this one myself by using the Conda environment ("r-reticulate") that the R-package "reticulate" made.
Code:
install.packages("remotes")
remotes::install_github("rstudio/tensorflow")
library(tensorflow)
install_tensorflow(version = "2.0.0b1", method = "conda", envname = "r-reticulate")
library(keras)