You can use plotmath expressions in the labels:
library(pheatmap)
rmt <- matrix(rnorm(9), ncol = 3)
pheatmap(rmt,
labels_row = c(expression(italic(Aaaa)), expression(B^b), expression(C[c])),
labels_col = c("E","F","G"))
Created on 2024-02-01 with reprex v2.0.2