First of all, you have to add the correct settings in Cloudflare under the SSL/TLS tab. Make sure it is set to Full if using self-generated/certbot generated certificate or else Full Strict if using Cloudflare provided certificate. Once the settings have been done from Cloudflare, move to the Nginx configuration of the website and set them like this:
The first server block ensures redirection to HTTPS and 2nd makes sure non-www is redirected to www. You can modify them at your convenience. Avoid automatic NGINX redirections settings done by Certbot if using Cloudflare.