Here is an example with mtcars (pinched from http://varianceexplained.org/r/broom-intro/)
library(dplyr)
library(broom)
mtcars %>% group_by(am) %>% do(tidy(lm(mpg ~ wt, .)))
so in your case
library(dplyr)
library(broom)
abalone %>%
group_by(Sex) %>%
do(tidy(lm(Diameter~Whole.weight, .)))
should work.