1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
|
---
title: Debugging HTTP services with mitmproxy
---
I spend a lot of my time at work writing Go services that talk to other Go
services over HTTP. Much of the time, everything works as expected, but every
now and then a situation arises where I'm struggling to understand why my
program is receiving a specific value. Is my request not being built correctly?
Am I not properly deserializing the response? Logging can be helpful, but
sometimes I really just want to look at the HTTP traffic between services.
One tool that I really love in these situations is [mitmproxy][]: "a free and
open source interactive HTTPS proxy," according to its website.
<!--more-->
There are no shortage of features and options for mitmproxy, and when I was
first exploring it they were a little daunting. I'm sure there are still tons of
things that it can do that I don't know about, but the main thing I tend to use
it for is reverse proxying.
Reverse proxying is a pretty simple concept; basically it means that you send
your HTTP requests to a specific proxy endpoint, and the proxy repeats your
request to some specific origin server. Then the same thing just happens
backwards: the origin server sends its reply to the reverse proxy which passes
it along back to you. This means that the proxy can see (and log!) the actual
HTTP request you send, and the response sent by the origin server.
Earlier today, I was working on a program that sent requests to a HTTP server
and my program's output didn't make sense. I wasn't sure if my requests were
being sent incorrectly, or maybe there was a bug in the server I was talking to.
So I fired up mitmproxy to take a look. In my shell, I ran:
```
mitmproxy --mode reverse:https://service.dev.example.com --listen-port 8080
```
This opens up a log window where any requests handled by the reverse proxy will
be displayed. I quickly updated my program to make its requests to
`http://localhost:8080` instead of `https://service.dev.example.com` and re-ran
it. The request and response were logged in the terminal window, and I was
quickly able to identify that a particular dependency needed to be updated.
Of course, you could also use Wireshark or tcpdump to inspect network traffic,
and these are great options that I also use frequently! But the main reason I
tend to turn first to mitmproxy is because it handles TLS like a [honey
badger][] -- it just doesn't give a shit. Basically, you can throw whatever you
want at it and it'll just do the right thing:
```
Client --TLS--> mitmproxy --TLS--> Origin
Client --HTTP--> mitmproxy --TLS--> Origin
Client --TLS--> mitmproxy --HTTP--> Origin
Client --HTTP--> mitmproxy --HTTP--> Origin
```
How does this work? When you first run mitmproxy, it generates a certificate
authority that it uses to generate certificates on-the-fly. All you need to do
is add the CA certificate to your OS trust store (see their [docs about
certificates here][certs]). For example, if I run `mitmproxy --mode
reverse:https://www.benburwell.com --listen-port 8080`, and then connect over
SSL, I can see the certificate that mitmproxy generated:
```
$ openssl s_client -connect localhost:8080
CONNECTED(00000005)
depth=1 CN = mitmproxy, O = mitmproxy
verify error:num=19:self signed certificate in certificate chain
verify return:0
---
Certificate chain
0 s:/CN=www.benburwell.com
i:/CN=mitmproxy/O=mitmproxy
1 s:/CN=mitmproxy/O=mitmproxy
i:/CN=mitmproxy/O=mitmproxy
```
Here, `mitmproxy` CA has generated a certificate with `CN=www.benburwell.com` to
match the hostname I'm reverse proxying to!
Now, there are ways to snoop on TLS encrypted traffic with Wireshark as well
using a TLS key log file, but this usually involves making somewhat non-trivial
modifications to the program you're working with. It's not _very_ complicated or
difficult, and it's a technique I've used a few times, but mitmproxy is usually
quicker and easier for me. I plan to write a post about this topic in the
future, so stay tuned!
[mitmproxy]: https://mitmproxy.org/
[honey badger]: https://youtu.be/4r7wHMg5Yjg
[certs]: https://docs.mitmproxy.org/stable/concepts-certificates/
|
