1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
|
---
title: Intercepting Go TLS Connections with Wireshark
---
I wrote previously about how I like to [use mitmproxy for debugging HTTP
services][mitmproxy-post]. This is a continued exploration of debugging network
services, in particular focused around inspecting TLS encrypted traffic that
your application is sending and receiving.
Transport Layer Security is a fundamental building block of modern secure
communications on the Internet, and increasingly the software we write is
expected to be a fluent speaker of TLS. While this brings security benefits for
users, it also increases the complexity of understanding what our software is
doing because when we try to use tools like Wireshark or tcpdump to inspect
network traffic, all we see is encrypted data. Let's see what a regular HTTP
request looks like in Wireshark:
```
$ curl http://www.benburwell.com
```
Here, we can see the HTTP request and response. But what happens when we make
the request over TLS?
```
$ curl https://www.benburwell.com
```
Here all we see are some TLS packets with embedded "encrypted application data."
We can see that a connection is being made, but we can't inspect the raw HTTP
request or response as we'd like to.
But all is not lost! There is a way for Wireshark to decrypt TLS connections and
show you dissected application protocol packets, it just requires a little
configuration. To understand how this works, we first need to understand a
little bit about TLS.
## How decrypting TLS in Wireshark works
TLS encrypts data within a session using a "master secret," a symmetric
encryption key that is established by using a key exchange protocol. So in order
for Wireshark to be able to decrypt and dissect TLS packets, we need some way to
tell it the master secret for the session.
The master secret is agreed upon using a cryptographic protocol when the TLS
connection is established. The exact implementation varies, but in general the
client and the server use some clever math to derive a value that is known at
both ends and yet is never directly sent over the wire, such that it is
computationally expensive for intermediate observers to derive the secret for
themselves. We won't get into [the specifics][ke], but one important detail for
later is that this exchange involves the client sending the server a large
random number in plain text, before the encrypted stream begins.
Conveniently, many TLS client libraries support the use of a key log file, which
does pretty much exactly what it sounds like: when the `SSLKEYLOGFILE`
environment variable is set, the library writes the key needed to decrypt the
traffic each time it establishes a TLS connection. Originally, this was
implemented in Mozilla's (at the time Netscape's) Network Security Services
library, so you might also see it referred to as a "[NSS Key Log File][nss]."
Let's give this a try!
```
$ SSLKEYLOGFILE=/tmp/keys curl -s https://www.benburwell.com >/dev/null
$ cat /tmp/keys
CLIENT_RANDOM 40b1a54e6b38f7accb90e1f5162534b8628389f4257e39f614a3ca28514db2c7 3121d2812c459996b072165c2ece4a1c85687d7073de06be0e1c16bf4a862fbe26a8cba24db1a4a0a9684fb19ad52f97
```
(Note that `SSLKEYLOGFILE` support was only enabled by default in curl 7.58, so
if this isn't working for you, check which version of curl you have).
This line in the key log means that for the TLS connection that was initiated
with the `CLIENT_RANDOM` of `40b1...`, the master secret is `3121...`. So now we
just need to tell Wireshark about this. Let's start a new capture and make
another request:
Now, we can right-click on the "Transport Layer Security" layer and select
Protocol Preferences -> (Pre)-Master-Secret log filename... and enter the path
to our `SSLKEYLOGFILE`, `/tmp/keys`, and something magical happens:
Now, when Wireshark encounters a TLS handshake, it can extract the random value
sent by the client and consult the key log file to discover a matching
`CLIENT_RANDOM` line and use the corresponding session master secret to decrypt
the data sent over the connection. So in addition to seeing the TLS details as
before, we can also see the decrypted HTTP requests!
## Configuring Go to use a TLS Key File
Go doesn't support the `SSLKEYLOGFILE` environment variable directly, but it
does have a different mechanism to achieve the same result. The
`crypto/tls.Config` struct has a `KeyLogWriter` field:
```
// KeyLogWriter optionally specifies a destination for TLS master secrets
// in NSS key log format that can be used to allow external programs
// such as Wireshark to decrypt TLS connections.
KeyLogWriter io.Writer
```
In typical Go fashion, I/O has been abstracted to an `io.Writer` interface.
Since we can use an `*os.File` to satisfy this interface, all we need to do to
produce a file containing the TLS secrets is to open a file and pass that
through the `tls.Config.KeyLogWriter`:
```
package main
import (
"crypto/tls"
"net/http"
"os"
)
func main() {
f, err := os.OpenFile("/tmp/keys", os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0600)
if err != nil {
panic(err)
}
defer f.Close()
client := &http.Client{
Transport: &http.Transport{
TLSClientConfig: &tls.Config{
KeyLogWriter: f,
},
},
}
client.Get("https://www.benburwell.com")
}
```
Building and running our program results in `CLIENT_RANDOM` lines being appended
to our `/tmp/keys` file and picked up by Wireshark, which in turn is able to
decrypt the messages being sent by our program:
In practice, for decrypting HTTP traffic for debugging, I find
[mitmproxy][mitmproxy-post] to be faster and easier, since it doesn't require
changes to the program. However, sometimes it's preferable to look at the actual
bytes on the wire, which is where using a key log file with Wireshark might be a
better approach.
Additionally, there are plenty of protocols other than HTTP that use TLS
connections, and where proxying isn't an option. For example, I've used a key
log file with Wireshark to debug a Go program that was making an IMAP connection
to a mail server. Because of the way Go's libraries tend to be layered, the code
to do this was very similar to the HTTP example above; I just needed to use my
custom `tls.Config` when constructing an IMAP client instead of a HTTP client.
[mitmproxy-post]: https://www.benburwell.com/posts/debugging-http-services-with-mitmproxy/
[ke]: https://en.wikipedia.org/wiki/Transport_Layer_Security#Key_exchange_or_key_agreement
[nss]: https://developer.mozilla.org/en-US/docs/Mozilla/Projects/NSS/Key_Log_Format
|
