This is not so much about data masking as it is the difference between a string (sequence of characters within quotes) and an expression (something that R can evaluate).
I believe your first attempt would have said this in the error message :
# ..1 (left)` must be a logical vector, not the string "col1|col2|col3".
you can get true/false ( a logical) from evaluating an expression that would return true or false, but not really from a string per say.
Here is a version that would work using base functions and then a modification using rlang