-
in order to add certs you would need a certificate authority, capable of setting up a proper certificate chain. There are several, ranging from the super simple like mkcert to full scale management tools like openCA. The second step is harder and require your consuming apps (the devices and apps that connects to your jellyfin) to trust your CA. An alternative would be to have a proper domain and use LetsEncrypt to get a cert, then trust will work out of the box, but it might also leak information about your private infrastructure.
-
traffic will go through your proxy
Yes, I personally have a valid domain name and then use a wildcart cert so that anything.mydomain.com is valid.