identity & security

The TLS Handshake Explained

Transport Layer Security (TLS) is the cryptographic protocol behind pretty much any computer network used today: from web browsing to email, APIs, and VoIP. But how exactly does it work?

Arthur Bellore

Cybersec Writer

Mar 7, 2023 • 12 min read

SSL vs. TLS

SSL, or Secure Sockets Layer, is the predecessor to TLS. It was a cryptographic protocol developed by Netscape in 1994, but since it had serious flaws, it was never released to the public. SSL 2.0 was released publicly one year later, in 1995, but it, too, was found to have serious flaws. Finally, SSL 3.0 was rebuilt from the ground up and released in 1996.

Microsoft, who had its own version of the protocol, and Netscape agreed to hand over the project to the Internet Engineering Task Force (IETF) and make it an open standard. As part of the deal, the name was changed to TLS, Transport Layer Security, to signify that it no longer had ties with Netscape.

The first upgrade to SSL 3.0 was published in 1999 in RFC 2246 as TLS 1.0. Tim Dierks, the editor of RFC 2246, has said in his blog that TLS 1.0 was really SSL 3.1, but they changed the name “so it wouldn’t look the IETF was just rubberstamping Netscape’s protocol.” Indeed, if you look at RFC 2246, it says:

This document describes TLS Version 1.0, which uses version { 3, 1 }. Version value 3.1 is historical: TLS version 1.0 is a minor modification to the SSL 3.0 protocol, which bears the version value 3.0.

TLS became the standard of encryption and authentication over computer networks, and it’s still being used today. The most recent version of the protocol is TLS 1.3, which was released in August 2018.

What is the difference between SSL and TLS?

SSL hasn’t been updated since its release in 1996, and it was deprecated in 2015 in RFC 7568. Most modern browsers disable it by default or don’t support it outright anymore.

Despite this, because of the association between the two, many people still use the term SSL when they really mean to say TLS. Vendors typically use the same terms their target customers use to facilitate messaging, so their marketing collateral generally contains terms such as “SSL/TLS” or “SSL,” even though SSL is deprecated.

We now know that SSL and TLS are different things, but for the remainder of this article, SSL will be used interchangeably with TLS to match its everyday usage.

Why SSL/TLS Was Created

For two computers to communicate with each other, they must perform a set of agreed-upon operations in a specific order. A protocol is what dictates those operations. The HTTP protocol is the protocol that web servers and clients use to establish a connection and communicate with each other through the Internet.

It works roughly as follows:

  1. The server listens for new connections on port 80
  2. The client connects to the server, and this starts a chain of operations as set forth by the HTTP protocol
  3. The client sends a GET request to the server; the server receives the request, processes it, then sends back a response: headers and index.html, plus other content it might need to serve (CSS files, JavaScript files, cookies, etc.) for the client to render the content correctly.

This works to distribute content between two remote computers, but there is a problem: HTTP was intended to expand the local knowledge management tools throughout the multiple computers at CERN in the late 1980s, so it was designed as a clear text protocol.

Because it’s clear text, a malicious actor could insert themselves between client and server and potentially read sensitive information, such as credit card numbers or passwords, or they could intercept the client’s requests and serve back malicious files pretending to be server.

As the Internet evolved and the number of sensitive information flying around increased exponentially, an encryption layer sitting below HTTP was necessary. However, this is a bit tricky: if a bad actor can “listen in” the conversation between client and server while the connection is unencrypted, how can they exchange encryption keys securely?

One option would be for the two parties to create a public-private key pair and send over the public key pair to the other party, so they can use that to encrypt their packet before sending them over the wire.

The problem with this is that asymmetric (public-key) cryptography is much slower than symmetric cryptography.

The "strength" of asymmetrical cryptography relies on modular arithmetic, mathematical operations that are very hard to do in the opposite direction. The numbers used in these operations need to be big enough to provide the necessary security, but performing arithmetic with very large numbers can be taxing on a CPU.

Symmetric cryptography, on the other hand, works differently and does not suffer from this problem. On top of that, most devices have AES instruction sets integrated into their CPUs to speed up AES operations, making AES encryption have almost zero overhead on modern devices.

The TLS protocol gets around this issue by using asymmetric cryptography to exchange keys at the beginning and then switch to symmetric keys (“sessions keys”) from then on in a process known as the TLS handshake.

TLS handshakes

The exact steps of the TLS handshake will depend on which TLS version is supported by the client and server, as well as which parameters they are set to use.

Since most modern browsers still support TLS 1.2 and a lot of websites still use TLS 1.2 (according to a paper published by Syed-Winkler, as of February 2021, TLS 1.3 is supported by 42.9% of surveyed websites, while TLS 1.2 is supported by 99.3%), we will talk about TLS 1.2 first and then discuss how TLS 1.3 differs from 1.2.

The TLS 1.2 handshake

TLS 1.2 can be configured to use many key exchange algorithms, and among them, the most well-known and widely used is the RSA key exchange algorithm. Let’s go over how the RSA algorithm works in TLS 1.2, then talk about how the process differs in TLS 1.3 with the Diffie-Hellman (DH) key exchange algorithm.

The RSA key exchange works roughly as follows in TLS 1.2:

  1. Server listens for new connections on port 443.
  2. Client connects to port 443 and initiates the handshake process with a ClientHello message to the server. This message contains the TLS versions and cipher suites the client supports and a 32-byte random number known as Client Random.
  3. Server says “hello” back with a ServerHello message. The server selects a cipher and the TLS version from the list provided by the client, then generates a different 32-byte random number known as Server Random and sends it all back along with the server’s own SSL certificate.
  4. Client uses the SSL certificate to verify the server’s identity. SSL certificates contain a public key generated by the server and a digital signature signed with the private key of a trusted third party known as a certificate authority (CA). Most web browsers and operating systems come bundled with public keys from trusted CAs, which are used to verify that the CA issued the certificate.
  5. The client verifies the server has the matching private key to the certificate through ClientKeyExchange. The client generates the pre-master secret (a 48-byte random number) and encrypts it with the server’s public key obtained from the certificate. If the server is the true owner of the certificate, it must be able to decrypt the message and obtain the original pre-master secret.
  6. Server decrypts the pre-master secret. Now both the server and client have the pre-master secret, the Server Random, and the Client Random. They can use that to derive the master secret using the previously agreed cipher and arrive at the same (symmetric) key, which will become the session key once both parties verify their keys match.
  7. Client sends a ChangeCipherSpec message. This indicates that the client is ready to create the session key and start communicating through an encrypted tunnel using that key.
  8. Client proves to server it has the correct keys. The client hashes all handshake records up to that point, encrypts them with the session key, and sends them to the server. If the server generates the correct key, it will be able to decrypt that message and verify the record hashes (which the server can independently generate).
  9. Server proves to the client that it has the correct keys. The server does the same thing and sends it to the client.
  10. A secure connection is established. If both keys match, the TLS handshake is complete.

Problems with TLS 1.2

TLS 1.2 left a lot of its configuration parameters up to the users, which often resulted in bad choices (the old security adage of “among too many options, the worst one will be chosen”).

Additionally, in the name of backward compatibility, TLS 1.2 also allowed old, insecure ciphers to be used. (TLS 1.2 itself was released in 2008.) As time passed, it became more prone to attacks as vulnerabilities in older cipher or versions of TLS were discovered.

On top of all that, the RSA key exchange has a few weaknesses:

  • The same public-private key pair is used both to authenticate the server and to encrypt the pre-master secret (item #5 above). If an attacker obtains the private key, they will be able to re-generate the session keys and be to decrypt the entire session.
  • There is no Perfect Forward Secrecy (PFS). Since the server’s private key doesn’t change, an attacker might record and store encrypted data for years, hoping to decrypt it years down the line if the server’s private key is ever leaked or compromised.
  • Performance. It’s necessary to make two round trips between the client and server until the session key is established.

To address these problems, the IETF began developing TLS 1.3 in 2013 and finally released it to the public in August 2018.

What changed in TLS 1.3

In practice, the main differences between TLS 1.3 and 1.2 are:

  • Shorter handshake
  • Only supports key exchange algorithms with perfect forward secrecy (PFS), such as ephemeral Diffie-Hellman
  • Support for 0-RTT (Zero Round Trip Time) resumption

This represents a bit of a paradigm change in TLS 1.3: shifting the focus from user configurability and backward compatibility to security by being more opinionated.

This allowed the designers to create a shorter handshake (by restraining DH as the only key exchange algorithm) but also prevented developers from choosing insecure options by restricting the number of options available in the first place.

For example, while it was technically possible to use DH as the key exchange method in TLS 1.2, the participants were responsible for setting the DH parameters. Picking the right parameters is crucial to make sure the DH exchange is secure, as some parameters can be trivially broken.

In 2015, security researchers published a paper describing how the WeakDH attack, which affected 8.4% of the top 1 million domains at the time, could trick servers into using small (insecure) DH parameters, enabling adversaries to decrypt data transmitted between client and server.

TLS 1.3 makes it impossible to use anything but a set of DH parameters that are believed to be secure.

The TLS 1.3 handshake

Because there are significantly fewer options for the client and server to agree on, the TLS 1.3 handshake is much simpler:

  1. Server listens for new connections on port 443.
  2. Client connects to port 443 and initiates the handshake process with a ClientHello message to the server. Since the list of cipher suites was vastly reduced in TLS 1.3 (from 37 to 5), the client assumes the server is going to use one of the five. It proactively calculates a key pair for each of the cipher suites and sends it to the server along with the protocol version.
  3. Server calculates the key session. With the client’s key share, the server is able to generate the session key.
  4. Server says “hello” back with a ServerHello message. The server generates its own key share and sends it over to the client, so it also can generate the session key, along with the server’s encrypted SSL certificate (using the session key created on #3).
  5. Client generates master secret and a secure connection is established. The client receives the server’s key share and calculates the session key. It decrypts and verifies the server’s certificates, and if everything is good, the handshake is complete.

Note that there are two different public-private key pairs on an ephemeral DH exchange: the first is the ephemeral pair, created every time client and server establish a connection, which guarantees forward secrecy. The second pair, embedded on the SSL certificate, is used for authentication and doesn’t change.

Given that a new key is generated for every session, this feature guarantees PFS. If the encryption keys are compromised at a later date, the leaked key will only be able to decrypt the specific session to which it corresponds; all the prior sessions are still safe. As a result, large-scale storage of data transmitted between server and client is rendered worthless.

How Auth0 Can Help

The temptation to intercept, undermine, and circumvent encryption has never been stronger. Cybercriminals are working to find ways to get around the issues that strong encryption presents due to the volume of valuable information that is sent between computers today.

While this rarely results in direct assaults against cryptographic algorithms or protocols, it frequently prompts attackers to devise novel methods of intercepting or capturing information. With these threats ever present, it is more crucial than ever to focus on maintaining strong and up-to-date implementations of effective encryption protocols.

Some applications are still running TLS versions older than 1.2, and others are still running the original SSL protocol. If yours is one of them, you should consider upgrading it immediately. Auth0 supports both TLS 1.2 and TLS 1.3 and can help you keep your applications secure.