acme.sh Wildcard Certs And Debugging With openssl and curl

Use acme.sh to issue wildcard certificates on domain hosted with Cloudflare. Use openssl to inspect the certificate and curl to verify the certificate is actually being used.

acme.sh Wildcard Certs And Debugging With openssl and curl
wildcard certificate

Around 20% of websites use Cloudflare as their CDN provider [1]. One advantage of Cloudflare is its ability to secure the communication between the browser, CDN, and web server through encryption. This ensures that all information transmitted between the browser and CDN is protected. Depending on the SSL settings, a certificate may or may not be present on the web server.

It is recommended to generate a certificate on the web server using a reputable CA. Using wildcard certificates is recommended for better operational security.

Certificate transparency websites, like crt.sh, record all certificates issued by reputable Certificate Authorities to prevent abuse[2][3].

However, this also means that anyone who knows a domain can see all the certificates issued under that domain. This can be problematic if a user wants to keep their subdomains private.

We will use acme.sh to obtain and install wildcard certificates. Then, we will use them in nginx and inspect the certificate using openssl. Lastly, we will verify that the certificate is being used by curl.


  1. https://w3techs.com/technologies/details/cn-cloudflare ↩︎

  2. https://developer.mozilla.org/en-US/docs/Web/Security/Certificate_Transparency ↩︎

  3. https://certificate.transparency.dev/ ↩︎


Cloudflare

API Key and Account/Zone ID

I will use Cloudflare for this post because most of my domains are there. However, acme.sh supports many DNS providers [1].

API Key

For this we will be generating an inital restricted api key.

Go to your profile and click on "API Token," then select "Create Token."
cloudflare-token-page.png

Since this token will be used by acme.sh, we only need to set up the "Zone.DNS" permissions.

cloudflare-token-generate.png

At first, acme.sh needs the "Zone Resources" to contain "All Zones." But after the certificate is issued, you can modify it to contain a certain zone if necessary.

This will become CF_Token=<VALUE>

Zone ID

If your web server will host only one domain and you need a wildcard for it, then using a single DNS zone should suffice. You can find the zone ID for that domain on its "Overview" page.
This value will be CF_Zone_ID=<VALUE>

cloudflare-zone-id.png

Account ID

If the web server will host many domains and we need multiple wildcard certificates, we must utilize the account ID [2].
This value will be CF_Account_ID=<VALUE>.

cloudflare-account-id.png

Security Considerations

Protect the API key at all costs. The key was designed to cover all DNS zones, so anyone possessing the API key and account ID could easily destroy or redirect DNS records.

Some suggested precautions are:

  • Set expiry dates on the api key
  • Set Client IP Address filtering to include the IP address of the server issuing and updating the certificate
  • Upon initial issue of certificates change the "Zone Resource" to include specific zones

  1. https://github.com/acmesh-official/acme.sh/wiki/dnsapi ↩︎

  2. https://github.com/acmesh-official/acme.sh/wiki/dnsapi#ii-multiple-dns-zones ↩︎

acme.sh

Issue and install wilcard certificate

Installation

To download acme.sh, visit the installation section on the github project to get the latest instructions.

While not mandatory, it is suggested that you use root while executing the acme.sh commands.

To install directly from the website:

curl https://get.acme.sh | sh -s [email protected]

Exporting Cloudflare Details

Before we can issue the certificate we need to export two variables

  • CF_Token
  • CF_Account_ID (multiple domains) or CF_Zone_ID (single domain)
export CF_Token=<API KEY>
export CF_{Account|Zone}_ID=<VALUE>

Issue and install a wildcard certificate

~/.acme.sh/acme.sh --issue -d example.com -d '*.example.com' --dns dns_cf --key-file /etc/acme/live/example.com.privkey.pem --fullchain-file /etc/acme/live/example.com.fullchain.pem

Replace example.com with your domain

After successfully completing the task, you should be able to locate two files in either /etc/acme/live or the directory of your choice.

Nginx

example.com.conf

In our nginx config file under the server heading we specify our ssl_certificate and ssl_certificate_key location

server {
    server_name example.com;
    listen 443 ssl http2;
    
    ssl_certificate /etc/acme/live/example.com.fullchain.pem;
    ssl_certificate_key /etc/acme/live/example.com.privkey.pem;
    ...
    ..
    .
}

Validate your nginx config by running nginx -t. If there are no errors, restart nginx systemctl restart nginx

openssl & curl

Is the certificate correctly issued? and are we actually using it?

At the beginning of the post, I said that one-fifth of the site uses Cloudflare, including all the publicly-facing parts. Nonetheless, it's tough to tell if our certificates are issued and used properly because Cloudflare utilizes its own edge certificate for browser connections.

It can be hard because if you check the certificate of this website, you won't see the webserver certificate. Instead, you'll see the certificate given by Cloudflare.

cloudflare-certificate.png

Our webserver will have a certificate issued either by letsencrypt or Zerossl not from Google.

Viewing The Issued Certificate

Lets start by inspecting the certificate issued

root at 20:38:01 in /etc/acme/live [I] ➜ openssl x509 -in unhackable.lol.fullchain.pem -text                                                                                                                                                                                                                                                                         
Certificate:                                                                                                                                                                                                                                                                                                                                                         
    Data:                                                                                                                                                                                                                                                                                                                                                            
        Version: 3 (0x2)                                                  
        .....
        ....
        ...
        ..
        .
        Issuer: C = AT, O = ZeroSSL, CN = ZeroSSL ECC Domain Secure Site CA                           
        Validity                                                                                      
            Not Before: Oct 31 00:00:00 2023 GMT                                                      
            Not After : Jan 29 23:59:59 2024 GMT                                                      
        Subject: CN = unhackable.lol    
        .....
        ....
        ...
        ..
        .
            X509v3 Subject Alternative Name:                                             
                DNS:unhackable.lol, DNS:*.unhackable.lol    

We can see that the certificate includes our wildcard certificate.

Are We Actually Using The Certificate?

The error below is an example of a certificate and website mismatch.

user at 15:19:58 in ~ [I] ➜ curl -v --resolve unhackble.lol:443:1.1.1.1 https://unhackable.lol                                                                                            
* Added unhackable.lol:443:1.1.1.1 to DNS cache                                                                                                                                                  
* Hostname unhackable.lol was found in DNS cache                                                                                                                                                      
*   Trying 1.1.1.1:443...                                                                                                                                                                              
* Connected to unhackable.lol (1.1.1.1) port 443 (#0)                                                                                                                                            
* ALPN, offering h2                                                                                                                                                                                         
* ALPN, offering http/1.1                                                                                                                                                                                   
*  CAfile: /etc/ssl/certs/ca-certificates.crt                                                                                                                                                               
*  CApath: /etc/ssl/certs                                                                                                                                                                                   
* TLSv1.0 (OUT), TLS header, Certificate Status (22):                                                                                                                                                       
* TLSv1.3 (OUT), TLS handshake, Client hello (1):                                                                                                                                                           
* TLSv1.2 (IN), TLS header, Certificate Status (22):                                                                                                                                                        
* TLSv1.3 (IN), TLS handshake, Server hello (2):                                                                                                                                                            
* TLSv1.2 (IN), TLS header, Finished (20):                                                                                                                                                                  
* TLSv1.2 (IN), TLS header, Supplemental data (23):                                                                                                                                                         
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):                                                                                                                                                    
* TLSv1.2 (IN), TLS header, Supplemental data (23):                                                                                                                                                         
* TLSv1.3 (IN), TLS handshake, Certificate (11):                                                                                                                                                            
* TLSv1.2 (IN), TLS header, Supplemental data (23): 
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.2 (IN), TLS header, Supplemental data (23): 
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.2 (OUT), TLS header, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=blog.unhackable.lol
*  start date: Oct 31 00:00:00 2023 GMT
*  expire date: Jan 29 23:59:59 2024 GMT
*  subjectAltName does not match unhackable.lol
* SSL: no alternative certificate subject name matches target host name 'element.chilled.chat'
* Closing connection 0
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.3 (OUT), TLS alert, close notify (256):
curl: (60) SSL: no alternative certificate subject name matches target host name 'unhackable.lol'
More details here: https://curl.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

Replace 1.1.1.1 with the public ip of the webserver and unhackable.lol with your own domain

However if everything is working correctly you should find the html in the response.

user at 20:42:54 in ~ took 14.8s [I] ➜ curl -v --resolve unhackable.lol:443:1.1.1.1 https://unhackable.lol                                                                                                                                                                                                                                                      
* Added unhackable.lol:443:1.1.1.1 to DNS cache                                                                                                                                                                                                                                                                                                                 
* Hostname unhackable.lol was found in DNS cache                                                                                                                                                                                                                                                                                                                     
*   Trying 1.1.1.1:443...                                                                                                                                                                                                                                                                                                                                       
* Connected to unhackable.lol (1.1.1.1) port 443 (#0)                                                                                            ...
..
.
* ALPN, server accepted to use h2                                                                                                                                                                                                                                                                                                                                    
* Server certificate:                                                                                                                                                                                                                                                                                                                                                
*  subject: CN=unhackable.lol                                                                                                                                                                                                                                                                                                                                        
*  start date: Sep  5 00:00:00 2023 GMT                                                                                                                                                                                                                                                                                                                              
*  expire date: Dec  4 23:59:59 2023 GMT                                                                                                                                                                                                                                                                                                                             
*  subjectAltName: host "unhackable.lol" matched cert's "unhackable.lol"                                                                                                                                                                                                                                                                                             
*  issuer: C=AT; O=ZeroSSL; CN=ZeroSSL RSA Domain Secure Site CA                                                                                                                                                                                                                                                                                                     
*  SSL certificate verify ok.                                                                                                                         



* Connection state changed (HTTP/2 confirmed)                                                                                                                                                                                                                                                           
> GET / HTTP/2                                                                                                                                                                                                                                                                                                                                                       
> Host: unhackable.lol                                                                                                                                                                                                                                                                                                                                               
> user-agent: curl/7.81.0                                                                                                                                                                                                                                                                                                                                            
> accept: */*                                                                                                                                                                                   

< HTTP/2 200                                                                                                                                                                                                
< server: nginx                                                                                                                                                                                             
< date: Wed, 01 Nov 2023 03:50:23 GMT                                                                                                                                                                       
< content-type: text/html; charset=utf-8                                                                                                                                                                    
< content-length: 7493                                                                                                                                                                                      
< vary: Accept-Encoding                                                                                                                                                                                     
< x-powered-by: Express                                                                                                                                                                                     
< cache-control: public, max-age=0                                                                                                                                                                          
< etag: W/"1d45-+33rtP9GG+52NSODLDFh3doOq4w"                                                                                                                                                                
< vary: Accept-Encoding                                                                                                                                                                                     
< x-frame-options: SAMEORIGIN                                                                                                                                                                               
< x-xss-protection: 1; mode=block                                                                                                                                                                           
< x-content-type-options: nosniff                                                                                                                                                                           
< referrer-policy: no-referrer-when-downgrade                                                                                                                                                               
< strict-transport-security: max-age=31536000; includeSubDomains; preload                                                                                                                                   
<                                                                                                                                                                                                           
<!DOCTYPE html>                                                                                                                                                                                             
<html lang="en">                                                                                                                                                                                            
                                                                                                                                                                                                            
<head>                                                                                                                                                                                                      
    <meta charset="utf-8">                                                                                                                                                                                  
    <meta name="viewport" content="width=device-width, initial-scale=1">                                                                                                                                    
    <title>Unhackable</title>                                                                                                                                                                               
    <link rel="stylesheet" href="/assets/built/screen.css?v=aca6e18aaa">                                                                                                                                    
                                                                                                                                                                                                            
    <meta name="description" content="Thoughts, stories and ideas.">                                                                                                                                        
    <link rel="canonical" href="https://unhackable.lol/">                                                                                                                                                   
    <meta name="referrer" content="no-referrer-when-downgrade">    

https://serverfault.com/questions/1023906/how-do-i-check-ssl-certificate-when-using-cloudflare-dns


edits

Nov 3rd: Various grammar fixes