How to make the 'leaflet' full-screen button work after switching the url?

Currently, we are using the Professional Plan (Monthly) which give us the ability to deploy the website to our own url. I follow the following documentation provided on the shiny official website

After following this tutorial, we met one problem. The 'full-screen' button for the leaflet map will not longer work because we get a error like this:

Mixed Content: The page at '<URL>' was loaded over HTTPS, but requested an insecure image '<URL>'. This content should also be served over HTTPS.

Are there some possible solutions for this? Note: We are setting the CNAME using the [blue host] and deploy the application to the shiny web server.