Here is one way to do this using some functions from the {dplyr} package. It is worth noting that you can't combine the string "all" with a numeric value like 300, since Rdata.frames constrain a column vector to be the same data type. So I changed the strength variable in df1 to be a character vector, instead.
library(dplyr)
df1 <- data.frame(Drugs = c("Drug A","Drug A","Drug B"), strength = c("50", "100", "300"))
df2 <- data.frame(Drugs = c("Drug A","Drug X","Drug Z"))
df1 %>%
mutate(strength = if_else(Drugs %in% df2$Drugs, "all", strength))
#> Drugs strength
#> 1 Drug A all
#> 2 Drug A all
#> 3 Drug B 300