This is my last post related to HTTP Public Key Pinning (HPKP). This is a post in response to Scott Helme’s latest post about him giving up on HPKP and how my blog is a perfect example of his concerns.
In the past I’ve written three articles about the HPKP header:
The point of each of these articles are pretty well summed up in their titles. For work, I was tasked with learning about HPKP and of course I made blog posts as I tired out the new header.
Starting with HPKP
While learning about HPKP, Helme’s posts were a great resource for me. Thanks to him, I was able to understand the process well enough to get some tests up online using this domain.
Everything worked fine until I had to renew my LetsEncrypt SSL certificate. Renewing caused the public key pin to not match the certificate and blocked all users who visited my blog with a browser that supported HPKP.
Running into issues
In Helme’s post, my issue is part of the “bad hygiene” that is warned against. HPKP requires that the best practices are used to key certificates up to date as well as the pins. Using LetsEncrypt certbot to auto-renew my domain’s certificate is not the best way to do this, since I can’t keep track of the pins, I was unable to properly support the header.
In order to “resolve” the issue, I had to strip the header from my site and ask all concerned clients to forget my domain from their browser’s saved history that remembered my old pin.
It is at this point that I agree with Helme’s conclusion that “One of the biggest concerns I have with HPKP right now is sites trying to use it and getting it wrong”. Especially that my blog is one of the sites that proves how easy it is to get wrong! It’s because of this difficulty and the massive impact a bad certificate/pin combination has on end users that I agree HPKP is not worth implementing.
Please go and read Helme’s post if HPKP is something you deal with or if you’re starting to look into it as well!
Over the last two weeks, I’ve posting a lot about HTTP Public Key Pinning. This will be my last post about it, I want to focus on testing HPKP. If you don’t know what HPKP is, read the first post. To learn how to add those headers, read the second post.
I’ve had to spend a lot of time trying to figure out how to properly test these headers. In theory, this is how it should work. The first time your browser loads a website with HPKP, it saves the pin in it’s memory and then compares the pin on every request after that until the pin expires. If the browser ever finds a request without a pin or a incorrect pin, then the browser will block the request and warn the user.
Failed attempts at testing HPKP
In order to test this, you should just have to change the pin. Easy right? Well so far the execution of that has been tricky. When I tried to change the pin to a random number that doesn’t match the cert, the browser seems to ignore the pin (According to my own results) and doesn’t send an error.
The next idea I attempted was to use OWASP ZAP to man-in-the-middle (MITM) the request, which would change the certificate and the pin could be modified in all the requests. This also didn’t work because HPKP is ignored when a client trusted certificate is used instead of one approved by a CA. “Firefox (and Chrome) disable Pin Validation for Pinned Hosts whose validated certificate chain terminates at a user-defined trust anchor (rather than a built-in trust anchor).” –Mozilla.
Better ways for testing HPKP
What did seem to work was setting Chrome’s expected value to a false pin, and then getting an error when going to the site with a real pin. Here are the steps:
Go to chrome://net-internals/#hsts in chrome
Add your domain and use the example sha256/7HIpactkIAq2Y49orFOOQKurWxmmSFZhBCoQYcRhJ3Y= as the Public key fingerprints
Go to your domain
You should see an error message like the one below.
This works for testing it from one application. What about other browsers like Firefox?
Another idea for testing HPKP is to keep the pin but change the certificate used to one that doesn’t match the pin. This requires buying another certificate because it has to be CA approved. You can either modify your domain’s headers to change the cert or you can try to use MITM and the CA cert instead of the self-signed cert.
Let me know how you test HPKP, it’s tricky business and isn’t easy to do. There are online tools like report-uri’s HPKP analyser that can compare the certificate and the pin, but will not test the browsers usage of the pin. A lot of people think it’s good enough, but that’s not true. If browsers do not properly block other requests without a correct pin, then there is no point in using HPKP.
Before we try to add a HPKP header, let’s review from last week. I made a post about what HTTP public key pinning is. It’s a fingerprint that browsers use to compare certificates can warn the user if the certificate is from a different source, even if it’s trusted or from the same server. If that doesn’t make sense, check out the link to the previous post.
A Public-Key-Pins header looks like this:
Public-Key-Pins: pin-256=”…”; pin-256=”…”; max-age=###; include subdomains; reporting-url=”…”;
The required parts are pin-sha256, and max-age. The pin is where you add the actual pin and the age is the number of seconds the the pin is kept on a browser. It’s good to note that the policy will not work with only one pin in the header, at least one backup pin is required. I’ll go over creating pins further below. For testing keep age short like 10 seconds (I did 500), otherwise standards recommends a two month period (5184000 seconds). Both include subdomains and the reporting url are optional. the include is just a boolean, I set it to true just to be safe. I don’t have reporting set up so I do not include that url in my header.
Getting a pin
In order to get a pin, you first need a certificate. So make sure you have TLS enabled on your site. Basically, you need to make sure https://your-website works and you get the green lock. An easy way to get TLS set up is Let’s Encrypt, I wrote about that in my TLS blog post a few months ago, scroll to the Try it! section. Once you have a certificate, you have two choices. First, you can use a nice tool and grab the pin generated for you (don’t worry, you’ll be sending this hash to everyone anyways, it’s ok if someone else generates it for you… as long as it’s valid). All you have to do is type in your website’s domain / URL into the tool.
The other option is to use openssl on your server to encode and hash the value yourself. First you need to find the certificate. Since I use Let’s Encrypt. Mine was in /etc/le and called private.pem. All you need to do from there is run the command below:
Be sure to change private.pem to your filename, all else should stay the same. This won’t modify anything but will echo your new pin.
Getting a backup pin
Backup pins are a little tricker to create. First off, the reason we need a backup pin is we are limited what’s accepted by the browser. It’s a good idea to give the browser more then a single option just in case the private key from the first pin gets compromised. This method will generate a new CSR and private key for the same TLS certificate. Here are the steps. It’s very similar to generating the first pin, you just need to create the csr and key first.
Step 1: Generate a new private key openssl genrsa -out name.your.backup.key 4096
– 4096 is the bits used (I think), 2048 or 4096 is the only two values you should be using.
Step 2: Generate a CSR openssl req -new -key name.your.backup.key -sha256 -out name.your.backup.csr
– There are fields you will be prompted to fill in, do so to the best of your ability.
Step 3: Generate the second pin openssl x509 -pubkey < name.your.backup.csr | openssl pkey -pubin -outform der | openssl dgst -sha256 -binary | base64
So now you should have both pins*, for future reference, I would recommend writing a bash script to run these commands and then echo the pin values in the format of the header so it’s possible to copy and paste the header into a configuration file. Can you think of a better way? *This is also a good time to mention that you will have to redo this EVERY time you change your TLS certificates, if you’re being super-cyber-secure, that means every three months. That means the more automated, the better!
Adding a HPKP header
Now, depending on your sever, there are a lot of different ways to add a header to your requests. If you’re using apache, add a line to the vhost file. If you’re using nginx, there’s a similar line you can add to the configuration. If you’re using a fancy load balancer that’s too expensive for a guy running a blog off a VM, then you can use a rule designed for that load balancer.
I’d be worried to tell you too much about my architecture but the response header already straight up tells you I run on an apache server… [note to self: remove that next time I’m digging in header configs and then update this blog post]. Since we are all caught up on my server’s inner workings. I’ll go through the steps to add this to your apache config.
sudo a2enmod headers
– You need to have permission to modify headers from the vhost file
sudo nano/vim/vi [your favorite text editor] server.conf – Edit your vhost file and add the line in step 3
Step 3: Header always set Public-Key-Pins “pin-sha256=\”base64+primary==\”; pin-sha256=\”base64+backup==\”; max-age=5184000; includeSubDomains” – Save the vhost file
sudo service apache2 restart
Check your HPKP header is there
Your header should be there, if everything works well, your browser won’t yell at you about your website. Nothing will change, yay you’re done! But wait, how do we know? Well.. seeing that the pin is there is easy, check your request headers and you can see the pin. Or use another report-uri tool that tells if the hash is valid and that you have your backup pin available.
In a future blog post, I’ll be discussing a way to actually test the pins and see if the browser will respond well. Right now, I’ve only read about a method that requires purchasing a second certificate. To avoid that, I’m looking into any other possibilities such as using a MITM technique. The issue with that is modern browsers are doing a similar check even without the pins, so at this point in time I’m unsure if it’s the default browsers functionality or the new HPKP header.
If you have any issues with the steps, please comment below. Let me know if you want to see more about securing HTTPS protocols!
On a project I’m involved with, a scanner has picked up a low issue where the HTTPS is missing HTTP Public Key Pins (HPKPs). If you’re like me, you’re probably thinking what the heck is HPKP? Well, I did a little bit of research and got it working on my personal website, I’ll share my struggles below so you don’t have to follow my footsteps.
Our browser stores a list of places that are accepted TLS/SSL certificate providers. If any website used a certificate that has one of those certificate authorities (CAs) as the root, then the browser will trust the certificate and not flag any errors. If only we had a way to check which CA was being used by a website, and if that CA ever changed, browsers would notify the end user.
Public Key Pins are a “fingerprint” that match a server’s TLS certificate. More specifically it’s a base 64 encoded SHA256 hash of the certificate. The browser uses this HPKP to validate the certificate, since a different cert wouldn’t be able to have the same pin. Browsers will store the HPKP on the first visit to a website, and will compare that stored pin to the pin attached to all future requests and make sure that the TLS certificate in the same. If they were different, it’d mean that either the certificate changed or (more likely) there is a man in the middle attack that is routing all the traffic and modifying the certificate chain.
Pretty cool, right? Adding one header and your client’s browsers will start complaining anytime someone changes the TLS certificate. That’s a pretty nice security feature for a small amount of work. Well, it would be, if it wasn’t redundant. From what I’ve seen while trying to test HPKP headers, modern browsers like the latest versions of Chrome and Firefox do a similar check with the TLS certificate alone, I think they compare the certs to see if they’re different or if the certificate’s CA is trusted by the browser. Why do we need a hash if the entire TLS certificate will be compared by the browser?
Why bother with HPKP
Honestly, I’d say to add HPKP because of a few reasons. I mentioned earlier that the latest versions of two browsers do this check, what if it’s older? It’s also better to be redundant and have two security controls then to have one and say “eh, it’ll work, well, it should”. At the very least, add the pin to reduce the issues found by scanners so you can be one step closer to having ZERO ISSUES! Which we all know is application security’s wet dream. If you don’t want to do it for the “compliance” aspect, you can always do it so Qualys’s SSL Lab will cheer for you.
When I was doing my research on pinning, I went to a few different sources. All were great and very helpful. If after this you still don’t understand htp public key pinning, feel free to leave a comment below with questions (I’ll add more, I promise!) and check out the references below.