tilde operator is often used to denote a statistical model, where the thing on the left of the ~ is the response and the things on the right of the ~ are the explanatory variables.

The ~ operator can be defined differently in different packages. sp::coordinates simply choose it as convenient without explanation

value
spatial coordinates; either a matrix, list, or data frame with numeric data, or column names, column number or a reference: a formula (in the form of e.g. ~x+y), column numbers (e.g. c(1,2)) or column names (e.g. c("x","y")) specifying which columns in object are the spatial coordinates. If the coordinates are part of object, giving the reference does not duplicate them, giving their value does duplicate them in the resulting structure.

x and y are two objects to which the user wishes to apply a function and wishes to distinguish x from y unambiguously, to avoid naming or order. ~ serves in that case to place x as a particular argument to the function and + indicates one or more additional arguments. The function must coerce the binary operator + into a method.

The detailed procedures to do this come up only in package writing. Advanced R by Hadley Wickham will give you an idea of the principles involve.

Note that sp generally produces objects of the S4 class.