If you haven't already seen it, Using renv with Docker • renv may be helpful.
You likely need to install the requisite system library (libicu) on your container. Or, if you did install that library, then the issue is most likely that the version used in your container differs from your host machine.
renv::restore()
doesn't change the library paths; it only installs packages from a lockfile into the requested library paths. Changing the active library paths for the session is a separate step.