Internet Backend 06 ๐ TLS/SSL Handshake and Encryption
TLS is not a single encryption algorithm. It is a protocol that negotiates the algorithms, authenticates the server, and establishes a shared session key, and then uses that key to encrypt the application data. The handshake is the negotiation and the key establishment, and the record layer is the encrypted data transfer. The two are distinct phases, and the security of the connection depends on both. The handshake must authenticate the server and establish a key that the attacker cannot derive, and the record layer must encrypt the data with a key that is unique to the session.
The version of the handshake determines the number of round trips and the security properties. TLS 1.2 uses a two-round-trip handshake with the RSA key exchange or the Diffie-Hellman key exchange. TLS 1.3 reduces the handshake to one round trip, removes the static RSA and Diffie-Hellman key exchanges, and requires the forward secrecy for all key exchanges . The forward secrecy is the property that the session key cannot be derived from the server’s long-term private key, so a compromised server key does not decrypt the past sessions. The TLS 1.3 makes the forward secrecy mandatory, and the TLS 1.2 makes it optional.
Key point: The TLS handshake authenticates the server with the certificate, negotiates the cipher suite, and establishes a shared session key with the Diffie-Hellman key exchange. The session key encrypts the application data with the symmetric cipher. TLS 1.3 uses one round trip, requires the forward secrecy, and encrypts the handshake after the ServerHello. TLS 1.2 uses two round trips and allows the static RSA key exchange without the forward secrecy.
Why the handshake is the security foundation
The authentication problem. The client connects to a server, and the server presents a certificate. The certificate contains the server’s public key and the identity information, and it is signed by a certificate authority. The client verifies the signature chain up to a trusted root, and it checks that the certificate is valid for the hostname it is connecting to. The authentication is the mechanism that prevents the man-in-the-middle attack, where an attacker intercepts the connection and presents its own certificate. Without the certificate verification, the Diffie-Hellman key exchange would establish a key with the attacker, not the server .
The key exchange problem. The client and the server need a shared key that the attacker cannot derive. The Diffie-Hellman key exchange is the mechanism: both sides generate an ephemeral private value, exchange the public value, and compute the shared secret. The attacker who observes the exchange cannot compute the shared secret without solving the discrete logarithm problem. The ephemeral keys are the forward secrecy mechanism: the private values are discarded after the session, so a compromised server key does not reveal the past session keys .
The cipher suite problem. The client and the server must agree on the algorithms: the key exchange algorithm, the authentication algorithm, the bulk encryption algorithm, and the MAC algorithm. The cipher suite is the combination, and the negotiation is the mechanism that selects the strongest suite that both sides support. The TLS 1.3 simplifies the cipher suite to the AEAD algorithms and the hash, and it removes the separate MAC and the separate key exchange algorithm from the suite .
The trade-off. The security has a cost. The TLS 1.2 handshake adds two round trips before the application data can be sent. The TLS 1.3 reduces it to one round trip, and the 0-RTT mode reduces it to zero for the resumed connections, but the 0-RTT has the replay risk . The certificate verification adds the latency of the OCSP or the CRL check, and the forward secrecy adds the cost of the ephemeral key generation. The trade-off is the security against the latency, and the protocol version and the cipher suite are the choices that set the balance.
a. The TLS 1.2 handshake and the two round trips
The TLS 1.2 handshake starts with the ClientHello, which contains the client’s random value, the supported cipher suites, and the supported extensions. The server responds with the ServerHello, which contains the server’s random value and the selected cipher suite. The server then sends the Certificate, the ServerKeyExchange (if the key exchange requires it), and the ServerHelloDone .
Client Server
ClientHello -------->
<-------- ServerHello
Certificate
ServerKeyExchange
ServerHelloDone
Finished -------->
<-------- Finished
Application Data <-------> Application Data
The client verifies the certificate, generates the pre-master secret, and encrypts it with the server’s public key (in the RSA key exchange) or performs the Diffie-Hellman exchange. The client sends the ClientKeyExchange, the ChangeCipherSpec, and the Finished. The server responds with the ChangeCipherSpec and the Finished. The application data can then be sent .
The RSA key exchange is the simpler one, but it does not provide the forward secrecy. The pre-master secret is encrypted with the server’s public key, and the server decrypts it with its private key. An attacker who obtains the server’s private key can decrypt the past sessions. The Diffie-Hellman key exchange is the alternative, and the ephemeral Diffie-Hellman provides the forward secrecy because the private values are not stored .
b. The TLS 1.3 handshake and the one round trip
The TLS 1.3 handshake starts with the ClientHello, which contains the client’s random value, the supported cipher suites, the supported groups, and the key share. The key share is the client’s ephemeral Diffie-Hellman public value, and it is sent in the first message. The server responds with the ServerHello, which contains the server’s key share, and the handshake keys are derived immediately. The server then sends the EncryptedExtensions, the Certificate, the CertificateVerify, and the Finished, and the client sends the Finished .
Client Server
ClientHello -------->
+ key_share
<-------- ServerHello
+ key_share
{EncryptedExtensions}
{Certificate}
{CertificateVerify}
{Finished}
Finished -------->
Application Data <-------> Application Data
The key difference is the early key share. The client sends its ephemeral public value in the ClientHello, and the server can compute the shared secret as soon as it receives the ClientHello. The server sends its own key share in the ServerHello, and both sides can derive the handshake keys. The certificate and the other handshake messages are encrypted with the handshake keys, which is the “all handshake messages after ServerHello are encrypted” property . The one round trip is the result of the early key share.
The TLS 1.3 removes the static RSA and the static Diffie-Hellman key exchanges. The only key exchange is the ephemeral Diffie-Hellman (or the ephemeral elliptic curve Diffie-Hellman), which provides the forward secrecy for every connection . The removal is the reason the TLS 1.3 is more secure than the TLS 1.2 with the RSA key exchange.
Complete Example Session
// ============================================
// PART 1: THE TLS 1.2 HANDSHAKE
// ============================================
Client Server
ClientHello -------->
<-------- ServerHello
Certificate
ServerKeyExchange
ServerHelloDone
ClientKeyExchange -------->
ChangeCipherSpec -------->
Finished -------->
<-------- ChangeCipherSpec
Finished
Application Data <-------> Application Data
// Two round trips before the application data.
// The RSA key exchange does not provide the forward secrecy.
// The ephemeral Diffie-Hellman provides the forward secrecy.
// ============================================
// PART 2: THE TLS 1.3 HANDSHAKE
// ============================================
Client Server
ClientHello -------->
+ key_share
<-------- ServerHello
+ key_share
{EncryptedExtensions}
{Certificate}
{CertificateVerify}
{Finished}
Finished -------->
Application Data <-------> Application Data
// One round trip before the application data.
// The key share is sent in the ClientHello.
// The handshake messages after ServerHello are encrypted.
// The ephemeral key exchange provides the forward secrecy.
// ============================================
// PART 3: THE CERTIFICATE VERIFICATION
// ============================================
// The server sends the certificate chain.
// The client verifies the signature chain up to the trusted root.
// The client checks the hostname and the validity period.
// The client checks the revocation (OCSP or CRL).
// ============================================
// PART 4: THE KEY EXCHANGE
// ============================================
// The client generates the ephemeral private value a.
// The client sends g^a to the server.
// The server generates the ephemeral private value b.
// The server sends g^b to the client.
// Both compute the shared secret g^(ab).
// The attacker cannot compute g^(ab) from g^a and g^b.
// ============================================
// PART 5: THE SESSION KEY
// ============================================
// The shared secret is the input to the key derivation.
// The HKDF derives the session keys from the shared secret.
// The session key encrypts the application data.
// The symmetric encryption is fast, and the asymmetric is slow.
// ============================================
// PART 6: THE FORWARD SECRECY
// ============================================
// The ephemeral private values are discarded after the handshake.
// The server's long-term private key does not reveal the session key.
// A compromised server key does not decrypt the past sessions.
// The TLS 1.3 requires the forward secrecy for all connections.
// ============================================
// PART 7: THE 0-RTT MODE
// ============================================
// The resumed session sends the application data in the first flight.
// The 0-RTT reduces the latency to zero round trips.
// The 0-RTT has the replay risk.
// The 0-RTT should only be used for the idempotent requests.
// ============================================
// PART 8: THE CIPHER SUITE
// ============================================
// The TLS 1.2 cipher suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256.
// The key exchange: ECDHE.
// The authentication: RSA.
// The bulk cipher: AES-128-GCM.
// The MAC: SHA256 (the AEAD combines the encryption and the MAC).
// The TLS 1.3 cipher suite: TLS_AES_128_GCM_SHA256.
// The key exchange is negotiated separately (the key_share extension).
// The cipher suite only names the AEAD and the hash.
// ============================================
// PART 9: THE COMMON HANDSHAKE FAILURES
// ============================================
// The certificate is expired.
// The certificate is self-signed and not trusted.
// The hostname does not match the certificate.
// The cipher suite is not supported by the server.
// The protocol version is not supported by the server.
// ============================================
// PART 10: THE CHECKLIST
// ============================================
// [ ] The server supports TLS 1.3.
// [ ] The server supports TLS 1.2 for the older clients.
// [ ] The server uses the ephemeral key exchange (ECDHE).
// [ ] The server uses the AEAD cipher suites.
// [ ] The certificate chain is complete.
// [ ] The OCSP stapling is enabled.
// [ ] The HSTS header is set.
// [ ] The 0-RTT is enabled only for the idempotent requests.
The ten parts covered the TLS 1.2 handshake, the TLS 1.3 handshake, the certificate verification, the key exchange, the session key, the forward secrecy, the 0-RTT mode, the cipher suite, the common failures, and the checklist. The through-line is that the handshake is the negotiation and the key establishment, and the record layer is the encrypted data transfer.
Quick Reference
The TLS 1.2 vs 1.3
| Aspect | TLS 1.2 | TLS 1.3 |
|---|---|---|
| Round trips | 2 | 1 |
| Key exchange | RSA, DH, ECDHE | ECDHE only (forward secrecy) |
| Handshake encryption | After ChangeCipherSpec | After ServerHello |
| Cipher suite | Key exchange + auth + bulk + MAC | AEAD + hash |
| 0-RTT | Not supported | Supported (with replay risk) |
| Compression | Supported | Removed |
The cipher suite components
| Component | TLS 1.2 | TLS 1.3 |
|---|---|---|
| Key exchange | ECDHE, DHE, RSA | ECDHE (via key_share) |
| Authentication | RSA, ECDSA | RSA, ECDSA |
| Bulk encryption | AES-GCM, ChaCha20-Poly1305 | AES-GCM, ChaCha20-Poly1305 |
| MAC | SHA256, SHA384 | Included in AEAD |
The common handshake failures
| Failure | Cause | Fix |
|---|---|---|
| Certificate expired | The validity period has passed | Renew the certificate |
| Certificate untrusted | The self-signed or the missing intermediate | Complete the chain |
| Hostname mismatch | The certificate is for a different name | Use the correct certificate |
| Cipher mismatch | The server does not support the client’s suite | Enable the modern suites |
| Version mismatch | The server does not support the client’s version | Enable TLS 1.2 and 1.3 |
Best Practices
โ Do This:
// Enable TLS 1.3 on the server.
// Enable TLS 1.2 for the older clients.
// Use the ECDHE key exchange for the forward secrecy.
// Use the AEAD cipher suites (AES-GCM, ChaCha20-Poly1305).
// Complete the certificate chain with the intermediates.
// Enable the OCSP stapling.
โ Don’t Do This:
// Use the RSA key exchange (no forward secrecy).
// Use the CBC-mode cipher suites (the padding oracle risk).
// Use the self-signed certificate in production.
// Enable the 0-RTT for the non-idempotent requests.
Common Pitfalls
| Pitfall | Why It Happens | Fix |
|---|---|---|
| The forward secrecy is missing | The RSA key exchange is used | Use the ECDHE |
| The handshake is slow | The two round trips of TLS 1.2 | Enable TLS 1.3 |
| The certificate is rejected | The chain is incomplete | Add the intermediate certificates |
| The 0-RTT replay | The non-idempotent request is sent in 0-RTT | Disable the 0-RTT for the non-idempotent requests |
| The old client cannot connect | The server only supports TLS 1.3 | Enable TLS 1.2 for the older clients |
Summary
| Item | Value |
|---|---|
| TLS 1.2 handshake | Two round trips, RSA or ECDHE key exchange |
| TLS 1.3 handshake | One round trip, ECDHE only, forward secrecy mandatory |
| Certificate | The server authentication, the chain verification |
| Key exchange | The Diffie-Hellman, the ephemeral for forward secrecy |
| Session key | The symmetric encryption of the application data |
| 0-RTT | The zero round trip, the replay risk |
| Cipher suite | The AEAD + hash in TLS 1.3 |
Key takeaways:
- The handshake authenticates the server and establishes the session key. The certificate verification is the authentication, and the Diffie-Hellman key exchange is the key establishment. The session key encrypts the application data with the symmetric cipher, which is fast enough for the bulk data .
- TLS 1.3 reduces the handshake to one round trip. The client sends its key share in the ClientHello, and the server can derive the handshake keys immediately. The handshake messages after the ServerHello are encrypted, and the forward secrecy is mandatory .
- The forward secrecy is the protection against the server key compromise. The ephemeral key exchange discards the private values after the session, so a compromised server key does not decrypt the past sessions. The TLS 1.2 with the RSA key exchange does not provide the forward secrecy .
- The 0-RTT mode trades the replay protection for the latency. The resumed session sends the application data in the first flight, but the attacker can replay the 0-RTT data. The 0-RTT should only be used for the idempotent requests .
- The certificate chain must be complete. The server sends the leaf certificate and the intermediate certificates, and the client verifies the chain up to the trusted root. The missing intermediate is the most common cause of the handshake failure .
Remember: The TLS handshake is the negotiation and the key establishment, and the record layer is the encrypted data transfer. The certificate authenticates the server, the Diffie-Hellman key exchange establishes the session key, and the symmetric cipher encrypts the data. The TLS 1.3 is one round trip and the forward secrecy is mandatory. The TLS 1.2 is two round trips and the forward secrecy is optional. The 0-RTT is the latency optimization with the replay risk. The handshake is the security foundation, and the record layer is the protection.
Stop using slow, ad-bloated tool sites! ๐คฎ
๐ Search “KandZ Tools” on Google to use many professional utilities for free.
KandZ.me is the ultimate minimalist hub for:
โ
Finance (Mortgage, Interest, Inflation)
โ
Tech (Base64, JSON, Dev Suite, IP)
โ
Health (BMI, BMR, TDEE)
โ
Productivity (Timer, Workspace, QR)
โก๏ธ Fast & Private
๐ No data leaves your device
๐ 100% Free
๐ Use it now: https://tools.kandz.me
๐ Bookmark itโyouโll need it later!