Why can't ggplot2 use %>%?

The way that I have thought of it is that both of these operators, %>% and +, are each doing different types of operations. For %>% we are passing values from the left into a function on the right. For + in ggplot we are adding elements to a plot. Conceptually this is not the same as passing previous elements of a plot into a geom function.