summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorBen Burwell <ben@benburwell.com>2021-05-06 19:24:31 -0400
committerBen Burwell <ben@benburwell.com>2021-05-06 19:24:45 -0400
commitf7b659631714f47ba9ad1ea266796376a8fe6520 (patch)
treeaa0cf26f41fd73a0f5d32c3e24b36faa32d58177
parentd201c07af90f38e47f55555301ab9452db816db3 (diff)
Add mitmproxy debugging post
-rw-r--r--_posts/2021-05-06-debugging-http-services-with-mitmproxy.md90
1 files changed, 90 insertions, 0 deletions
diff --git a/_posts/2021-05-06-debugging-http-services-with-mitmproxy.md b/_posts/2021-05-06-debugging-http-services-with-mitmproxy.md
new file mode 100644
index 0000000..8c5d7c9
--- /dev/null
+++ b/_posts/2021-05-06-debugging-http-services-with-mitmproxy.md
@@ -0,0 +1,90 @@
+---
+title: Debugging HTTP services with mitmproxy
+---
+
+I spend a lot of my time at work writing Go services that talk to other Go
+services over HTTP. Much of the time, everything works as expected, but every
+now and then a situation arises where I'm struggling to understand why my
+program is receiving a specific value. Is my request not being built correctly?
+Am I not properly deserializing the response? Logging can be helpful, but
+sometimes I really just want to look at the HTTP traffic between services.
+
+One tool that I really love in these situations is [mitmproxy][]: "a free and
+open source interactive HTTPS proxy," according to its website.
+
+<!--more-->
+
+There are no shortage of features and options for mitmproxy, and when I was
+first exploring it they were a little daunting. I'm sure there are still tons of
+things that it can do that I don't know about, but the main thing I tend to use
+it for is reverse proxying.
+
+Reverse proxying is a pretty simple concept; basically it means that you send
+your HTTP requests to a specific proxy endpoint, and the proxy repeats your
+request to some specific origin server. Then the same thing just happens
+backwards: the origin server sends its reply to the reverse proxy which passes
+it along back to you. This means that the proxy can see (and log!) the actual
+HTTP request you send, and the response sent by the origin server.
+
+Earlier today, I was working on a program that sent requests to a HTTP server
+and my program's output didn't make sense. I wasn't sure if my requests were
+being sent incorrectly, or maybe there was a bug in the server I was talking to.
+So I fired up mitmproxy to take a look. In my shell, I ran:
+
+```
+mitmproxy --mode reverse:https://service.dev.example.com --listen-port 8080
+```
+
+This opens up a log window where any requests handled by the reverse proxy will
+be displayed. I quickly updated my program to make its requests to
+`http://localhost:8080` instead of `https://service.dev.example.com` and re-ran
+it. The request and response were logged in the terminal window, and I was
+quickly able to identify that a particular dependency needed to be updated.
+
+Of course, you could also use Wireshark or tcpdump to inspect network traffic,
+and these are great options that I also use frequently! But the main reason I
+tend to turn first to mitmproxy is because it handles TLS like a [honey
+badger][] -- it just doesn't give a shit. Basically, you can throw whatever you
+want at it and it'll just do the right thing:
+
+```
+Client --TLS--> mitmproxy --TLS--> Origin
+Client --HTTP--> mitmproxy --TLS--> Origin
+Client --TLS--> mitmproxy --HTTP--> Origin
+Client --HTTP--> mitmproxy --HTTP--> Origin
+```
+
+How does this work? When you first run mitmproxy, it generates a certificate
+authority that it uses to generate certificates on-the-fly. All you need to do
+is add the CA certificate to your OS trust store (see their [docs about
+certificates here][certs]). For example, if I run `mitmproxy --mode
+reverse:https://www.benburwell.com --listen-port 8080`, and then connect over
+SSL, I can see the certificate that mitmproxy generated:
+
+```
+$ openssl s_client -connect localhost:8080
+CONNECTED(00000005)
+depth=1 CN = mitmproxy, O = mitmproxy
+verify error:num=19:self signed certificate in certificate chain
+verify return:0
+---
+Certificate chain
+ 0 s:/CN=www.benburwell.com
+ i:/CN=mitmproxy/O=mitmproxy
+ 1 s:/CN=mitmproxy/O=mitmproxy
+ i:/CN=mitmproxy/O=mitmproxy
+```
+
+Here, `mitmproxy` CA has generated a certificate with `CN=www.benburwell.com` to
+match the hostname I'm reverse proxying to!
+
+Now, there are ways to snoop on TLS encrypted traffic with Wireshark as well
+using a TLS key log file, but this usually involves making somewhat non-trivial
+modifications to the program you're working with. It's not _very_ complicated or
+difficult, and it's a technique I've used a few times, but mitmproxy is usually
+quicker and easier for me. I plan to write a post about this topic in the
+future, so stay tuned!
+
+[mitmproxy]: https://mitmproxy.org/
+[honey badger]: https://youtu.be/4r7wHMg5Yjg
+[certs]: https://docs.mitmproxy.org/stable/concepts-certificates/