The openssl R package is built against the OpenSSL RPM that the linux distribution (e.g. Amazon Linux 2 in your case provides). You can check that by running ldd /opt/R/3.6.3/lib/R/library/openssl/libs/openssl.so in a terminal and you will get an output similar to (ldd is listing all the dynamically loaded libraries that a certain file is linked against).
ldd /opt/R/3.6.3/lib/R/library/openssl/libs/openssl.so
linux-vdso.so.1 => (0x00007ffc13ff8000)
libssl.so.10 => /lib64/libssl.so.10 (0x00007f830445f000)
libcrypto.so.10 => /lib64/libcrypto.so.10 (0x00007f8303ffc000)
libR.so => /opt/R/4.3.2/lib/R/lib/libR.so (0x00007f8303994000)
libc.so.6 => /lib64/libc.so.6 (0x00007f83035c6000)
libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007f8303379000)
libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007f8303090000)
libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007f8302e8c000)
libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007f8302c59000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f8302a55000)
libz.so.1 => /lib64/libz.so.1 (0x00007f830283f000)
libRblas.so => /opt/R/3.6.3/lib/R/lib/libRblas.so (0x00007f830029f000)
libgfortran.so.3 => /lib64/libgfortran.so.3 (0x00007f82fff7d000)
libm.so.6 => /lib64/libm.so.6 (0x00007f82ffc7b000)
libquadmath.so.0 => /lib64/libquadmath.so.0 (0x00007f82ffa3f000)
libreadline.so.6 => /lib64/libreadline.so.6 (0x00007f82ff7f9000)
libpcre2-8.so.0 => /lib64/libpcre2-8.so.0 (0x00007f82ff582000)
liblzma.so.5 => /lib64/liblzma.so.5 (0x00007f82ff35c000)
libbz2.so.1 => /lib64/libbz2.so.1 (0x00007f82ff14c000)
librt.so.1 => /lib64/librt.so.1 (0x00007f82fef44000)
libicuuc.so.50 => /lib64/libicuuc.so.50 (0x00007f82febcb000)
libicui18n.so.50 => /lib64/libicui18n.so.50 (0x00007f82fe7cc000)
libgomp.so.1 => /lib64/libgomp.so.1 (0x00007f82fe5a6000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f82fe38a000)
/lib64/ld-linux-x86-64.so.2 (0x00007f83048e6000)
libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007f82fe17a000)
libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007f82fdf76000)
libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f82fdd5c000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f82fdb46000)
libtinfo.so.5 => /lib64/libtinfo.so.5 (0x00007f82fd91c000)
libicudata.so.50 => /lib64/libicudata.so.50 (0x00007f82fc349000)
libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f82fc041000)
libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f82fbe1a000)
libpcre.so.1 => /lib64/libpcre.so.1 (0x00007f82fbbb8000)
The two most important lines are the ones containing libssl.so.10 and libcrypto.so.10 which you both can trace back to the OpenSSL installation provided by the linux distribution (e.g. via rpm -qfi /lib64/libssl.so.10).
I guess if you show your IT security team the above combination of ldd and rpm -qfi you should be able to convince them that the openssl R package is using the patched version of the openssl linux software. Finally, as a proof that the installed openssl software is the patched one, get the first few lines of the changelog via rpm -q openssl-libs --changelog | head -2 which will lead to
# rpm -q openssl-libs --changelog | head -2
* Mon Jan 29 2024 Keerthana Purushotham <keepur@amazon.com> - 1.0.2k-24.amzn2.0.12
- Fix for CVE-2024-0727