diff options
Diffstat (limited to '_posts')
| -rw-r--r-- | _posts/2021-05-14-intercepting-golang-tls-with-wireshark.md | 166 |
1 files changed, 166 insertions, 0 deletions
diff --git a/_posts/2021-05-14-intercepting-golang-tls-with-wireshark.md b/_posts/2021-05-14-intercepting-golang-tls-with-wireshark.md new file mode 100644 index 0000000..ee86fe9 --- /dev/null +++ b/_posts/2021-05-14-intercepting-golang-tls-with-wireshark.md @@ -0,0 +1,166 @@ +--- +title: Intercepting Go TLS Connections with Wireshark +--- + +I wrote previously about how I like to [use mitmproxy for debugging HTTP +services][mitmproxy-post]. This is a continued exploration of debugging network +services, in particular focused around inspecting TLS encrypted traffic that +your application is sending and receiving. + +Transport Layer Security is a fundamental building block of secure +communications on the Internet today, and increasingly the software we write is +expected to be a fluent speaker of TLS. While this brings security benefits for +users, it also increases the complexity of understanding what our software is +doing because when we try to use tools like Wireshark or tcpdump to inspect +network traffic, all we see is encrypted data. Let's see what a regular HTTP +request looks like in Wireshark: + +``` +$ curl http://www.benburwell.com +``` + + + +Here, we can see the HTTP request and response. But what happens when we make +the request over TLS? + +``` +$ curl https://www.benburwell.com +``` + + + +Here all we see are some TLS packets with embedded "encrypted application data." +We can see that a connection is being made, but we can't inspect the raw HTTP +request or response as we'd like to. + +But all is not lost! There is a way for Wireshark to decrypt TLS connections and +show you dissected application protocol packets, it just requires a little +configuration. To understand how this works, we first need to understand a +little bit about TLS. + +## How decrypting TLS in Wireshark works + +TLS encrypts data within a session using a "master secret," a symmetric +encryption key that is established by using a key exchange protocol. So in order +for Wireshark to be able to decrypt and dissect TLS packets, we need some way to +tell it the master secret for the session. + +The master secret is agreed upon using a cryptographic protocol when the TLS +connection is established. The exact implementation varies, but in general the +client and the server use some clever math to derive a value that is known at +both ends and yet is never directly sent over the wire, such that it is +computationally expensive for intermediate observers to derive the secret for +themselves. We won't get into [the specifics][ke], but one important detail for +later is that this exchange involves the client sending the server a large +random number in plain text, before the encrypted stream begins. + +Conveniently, many TLS client libraries support the use of a key log file, which +does pretty much exactly what it sounds like: when the `SSLKEYLOGFILE` +environment variable is set, the library writes the key needed to decrypt the +traffic each time it establishes a TLS connection. Originally, this was +implemented in Mozilla's (at the time Netscape's) Network Security Services +library, so you might also see it referred to as a "[NSS Key Log File][nss]." +Let's give this a try! + +``` +$ SSLKEYLOGFILE=/tmp/keys curl -s https://www.benburwell.com >/dev/null +$ cat /tmp/keys +CLIENT_RANDOM 40b1a54e6b38f7accb90e1f5162534b8628389f4257e39f614a3ca28514db2c7 3121d2812c459996b072165c2ece4a1c85687d7073de06be0e1c16bf4a862fbe26a8cba24db1a4a0a9684fb19ad52f97 +``` + +(Note that `SSLKEYLOGFILE` support was only enabled by default in curl 7.58, so +if this isn't working for you, check which version of curl you have). + +This line in the key log means that for the TLS connection that was initiated +with the `CLIENT_RANDOM` of `40b1...`, the master secret is `3121...`. So now we +just need to tell Wireshark about this. Let's start a new capture and make +another request: + + + +Now, we can right-click on the "Transport Layer Security" layer and select +Protocol Preferences -> (Pre)-Master-Secret log filename... and enter the path +to our `SSLKEYLOGFILE`, `/tmp/keys`, and something magical happens: + + + +Now, when Wireshark encounters a TLS handshake, it can extract the random value +sent by the client and consult the key log file to discover a matching +`CLIENT_RANDOM` line and use the corresponding session master secret to decrypt +the data sent over the connection. So in addition to seeing the TLS details as +before, we can also see the decrypted HTTP requests! + +## Configuring Go to use a TLS Key File + +Go doesn't support the `SSLKEYLOGFILE` environment variable directly, but it +does have a different mechanism to achieve the same result. The +`crypto/tls.Config` struct has a `KeyLogWriter` field: + +``` +// KeyLogWriter optionally specifies a destination for TLS master secrets +// in NSS key log format that can be used to allow external programs +// such as Wireshark to decrypt TLS connections. +KeyLogWriter io.Writer +``` + +In typical Go fashion, I/O has been abstracted to an `io.Writer` interface. +Since we can use an `*os.File` to satisfy this interface, all we need to do to +produce a file containing the TLS secrets is to open a file and pass that +through the `tls.Config.KeyLogWriter`: + +``` +package main + +import ( + "crypto/tls" + "net/http" + "os" +) + +func main() { + f, err := os.OpenFile("/tmp/keys", os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0600) + if err != nil { + panic(err) + } + defer f.Close() + + client := &http.Client{ + Transport: &http.Transport{ + TLSClientConfig: &tls.Config{ + KeyLogWriter: f, + }, + }, + } + + client.Get("https://www.benburwell.com") +} +``` + +Building and running our program results in `CLIENT_RANDOM` lines being appended +to our `/tmp/keys` file and picked up by Wireshark, which in turn is able to +decrypt the messages being sent by our program: + + + +In practice, for decrypting HTTP traffic for debugging, I find +[mitmproxy][mitmproxy-post] to be faster and easier, since it doesn't require +changes to the program. However, sometimes it's preferable to look at the actual +bytes on the wire, which is where using a key log file with Wireshark might be a +better approach. + +Additionally, there are plenty of protocols other than HTTP that use TLS +connections, and where proxying isn't an option. For example, I've used a key +log file with Wireshark to debug a Go program that was making an IMAP connection +to a mail server. Because of the way Go's libraries tend to be layered, the code +to do this was very similar to the HTTP example above; I just needed to use my +custom `tls.Config` when constructing an IMAP client instead of a HTTP client. + +[mitmproxy-post]: https://www.benburwell.com/posts/debugging-http-services-with-mitmproxy/ +[ke]: https://en.wikipedia.org/wiki/Transport_Layer_Security#Key_exchange_or_key_agreement +[nss]: https://developer.mozilla.org/en-US/docs/Mozilla/Projects/NSS/Key_Log_Format |