You are mapping the x-axis for geom_col()only so geom_text() doesn't inherit it, if you want to map it globally for your plot you have to do it inside the ggplot() function.
If you need more specific help, please provide a proper REPRoducible EXample (reprex) illustrating your issue.