Add TLS Wireshark post

1 files changed, 166 insertions, 0 deletions

diff --git a/_posts/2021-05-14-intercepting-golang-tls-with-wireshark.md b/_posts/2021-05-14-intercepting-golang-tls-with-wireshark.md
new file mode 100644
index 0000000..ee86fe9
--- /dev/null
+++ b/_posts/2021-05-14-intercepting-golang-tls-with-wireshark.md
@@ -0,0 +1,166 @@
+---
+title: Intercepting Go TLS Connections with Wireshark
+---
+
+I wrote previously about how I like to [use mitmproxy for debugging HTTP
+services][mitmproxy-post]. This is a continued exploration of debugging network
+services, in particular focused around inspecting TLS encrypted traffic that
+your application is sending and receiving.
+
+Transport Layer Security is a fundamental building block of secure
+communications on the Internet today, and increasingly the software we write is
+expected to be a fluent speaker of TLS. While this brings security benefits for
+users, it also increases the complexity of understanding what our software is
+doing because when we try to use tools like Wireshark or tcpdump to inspect
+network traffic, all we see is encrypted data. Let's see what a regular HTTP
+request looks like in  Wireshark:
+
+```
+$ curl http://www.benburwell.com
+```
+
+![A Wireshark packet capture showing a plain HTTP request and
+response](https://s.bnbl.io/blog/tls-wireshark/plain.png)
+
+Here, we can see the HTTP request and response.  But what happens when we make
+the request  over TLS?
+
+```
+$ curl https://www.benburwell.com
+```
+
+![A Wireshark packet capture showing encrypted TLS application
+data](https://s.bnbl.io/blog/tls-wireshark/tls.png)
+
+Here all we see are some TLS packets with embedded "encrypted application data."
+We can see that a connection is being made, but we can't inspect the raw HTTP
+request or response as we'd like to.
+
+But all is not lost! There is a way for Wireshark to decrypt TLS connections and
+show you dissected application protocol packets, it just requires a little
+configuration. To understand how this works, we first need to understand a
+little bit about TLS.
+
+## How decrypting TLS in Wireshark works
+
+TLS encrypts data within a session using a "master secret," a symmetric
+encryption key that is established by using a key exchange protocol. So in order
+for Wireshark to be able to decrypt and dissect TLS packets, we need some way to
+tell it the master secret for the session.
+
+The master secret is agreed upon using a cryptographic protocol when the TLS
+connection is established. The exact implementation varies, but in general the
+client and the server use some clever math to derive a value that is known at
+both ends and yet is never directly sent over the wire, such that it is
+computationally expensive for intermediate observers to derive the secret for
+themselves. We won't get into [the specifics][ke], but one important detail for
+later is that this exchange involves the client sending the server a large
+random number in plain text, before the encrypted stream begins.
+
+Conveniently, many TLS client libraries support the use of a key log file, which
+does pretty much exactly what it sounds like: when the `SSLKEYLOGFILE`
+environment variable is set, the library writes the key needed to decrypt the
+traffic each time it establishes a TLS connection. Originally, this was
+implemented in Mozilla's (at the time Netscape's) Network Security Services
+library, so you might also see it referred to as a "[NSS Key Log File][nss]."
+Let's give this a try!
+
+```
+$ SSLKEYLOGFILE=/tmp/keys curl -s https://www.benburwell.com >/dev/null
+$ cat /tmp/keys
+CLIENT_RANDOM 40b1a54e6b38f7accb90e1f5162534b8628389f4257e39f614a3ca28514db2c7 3121d2812c459996b072165c2ece4a1c85687d7073de06be0e1c16bf4a862fbe26a8cba24db1a4a0a9684fb19ad52f97
+```
+
+(Note that `SSLKEYLOGFILE` support was only enabled by default in curl 7.58, so
+if this isn't working for you, check which version of curl you have).
+
+This line in the key log means that for the TLS connection that was initiated
+with the `CLIENT_RANDOM` of `40b1...`, the master secret is `3121...`. So now we
+just need to tell Wireshark about this. Let's start a new capture and make
+another request:
+
+![A Wireshark packet capture showing encrypted TLS application
+data](https://s.bnbl.io/blog/tls-wireshark/tls.png)
+
+Now, we can right-click on the "Transport Layer Security" layer and select
+Protocol Preferences -> (Pre)-Master-Secret log filename... and enter the path
+to our `SSLKEYLOGFILE`, `/tmp/keys`, and something magical happens:
+
+![A Wireshark packet capture showing TLS protocol packets and decrypted HTTP
+traffic](https://s.bnbl.io/blog/tls-wireshark/decrypted.png)
+
+Now, when Wireshark encounters a TLS handshake, it can extract the random value
+sent by the client and consult the key log file to discover a matching
+`CLIENT_RANDOM` line and use the corresponding session master secret to decrypt
+the data sent over the connection. So in addition to seeing the TLS details as
+before, we can also see the decrypted HTTP requests!
+
+## Configuring Go to use a TLS Key File
+
+Go doesn't support the `SSLKEYLOGFILE` environment variable directly, but it
+does have a different mechanism to achieve the same result. The
+`crypto/tls.Config` struct has a `KeyLogWriter` field:
+
+```
+// KeyLogWriter optionally specifies a destination for TLS master secrets
+// in NSS key log format that can be used to allow external programs
+// such as Wireshark to decrypt TLS connections.
+KeyLogWriter io.Writer
+```
+
+In typical Go fashion, I/O has been abstracted to an `io.Writer` interface.
+Since we can use an `*os.File` to satisfy this interface, all we need to do to
+produce a file containing the TLS secrets is to open a file and pass that
+through the `tls.Config.KeyLogWriter`:
+
+```
+package main
+
+import (
+  "crypto/tls"
+  "net/http"
+  "os"
+)
+
+func main() {
+  f, err := os.OpenFile("/tmp/keys", os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0600)
+  if err != nil {
+    panic(err)
+  }
+  defer f.Close()
+
+  client := &http.Client{
+    Transport: &http.Transport{
+      TLSClientConfig: &tls.Config{
+        KeyLogWriter: f,
+      },
+    },
+  }
+
+  client.Get("https://www.benburwell.com")
+}
+```
+
+Building and running our program results in `CLIENT_RANDOM` lines being appended
+to our `/tmp/keys` file and picked up by Wireshark, which in turn is able to
+decrypt the messages being sent by our program:
+
+![A Wireshark packet capture showing decrypted TLS traffic sent by
+Go](https://s.bnbl.io/blog/tls-wireshark/go_decrypted.png)
+
+In practice, for decrypting HTTP traffic for debugging, I find
+[mitmproxy][mitmproxy-post] to be faster and easier, since it doesn't require
+changes to the program. However, sometimes it's preferable to look at the actual
+bytes on the wire, which is where using a key log file with Wireshark might be a
+better approach.
+
+Additionally, there are plenty of protocols other than HTTP that use TLS
+connections, and where proxying isn't an option. For example, I've used a key
+log file with Wireshark to debug a Go program that was making an IMAP connection
+to a mail server. Because of the way Go's libraries tend to be layered, the code
+to do this was very similar to the HTTP example above; I just needed to use my
+custom `tls.Config` when constructing an IMAP client instead of a HTTP client.
+
+[mitmproxy-post]: https://www.benburwell.com/posts/debugging-http-services-with-mitmproxy/
+[ke]: https://en.wikipedia.org/wiki/Transport_Layer_Security#Key_exchange_or_key_agreement
+[nss]: https://developer.mozilla.org/en-US/docs/Mozilla/Projects/NSS/Key_Log_Format