This is an interesting problem, as there are multiple ways to approach it.
The first is to work with the way your markers are displayed - circles, icons, or something in between. A while back I wrote a blog post describing alternative approaches to the problem, you can find it here Leaflet Map Markers in R · Jindra Lacko
The second is to incorporate generic HTML - which of course can handle images - in your popups. This would look like the code below; note how the code to construct the popup uses single quotes to deliminate the string, so that double quotes can be used inside as part of the HTML syntax.