Thanks for the suggestion Pieter. I managed to get some decent looking results with the stat_ellipse() function. Do you know if there are any critical differences between these two?
I experimented with the stat_ellipse() too, but in that case you have to fiddle with the level parameter to have the ellipse enclose all data points as it is using statistical boundaries. But the result should be very similar.