||[Jan. 1st, 2007|10:39 pm]
I've just wrestled my first real Smalltalk program into a demo'able state! It's called PacketStudio and it's for post-processing packet traces from
tcpdump. You can download it from SqueakSource if you're an interested Squeaker.
The main feature today is to take a pair of trace files (one from either endpoint of a TCP connection) and draw a diagram showing the path of packets between sender and receiver. The left and right sides are the endpoints (hosts) and the lines are packets travelling from one host to the other. The angle of the line indicates how long the packet was in transit (each vertical pixel is two milliseconds). Orange lines are SYN packets, blue carry data (thickness is proportional to amount), and green are pure acknowledgements.
The example pictured is an SSH connection setup between one host in Sweden and one in the USA. To reproduce the graph you can download my example traces and run this:
TCPGraph trace: 'fresh-kyle.pcap' with: 'kyle-fresh.pcap'
You can run it on your own traces too but make sure they include the whole TCP header (e.g.
tcpdump -s 0). PacketStudio will automatically find the matching streams in your trace files and ask you which to display. This is not a polished program yet so do be brave! :-)
PacketStudio is a rewrite from memory of an Erlang program that me and my friend Martin Björklund (now at tail-f) wrote a few years ago while we were investigating the crappy performance of TCP over the early European GPRS networks. The great thing about this being my very own weblog is that I can digress to my heart's content about the fun way we ran that project :-)
- Take lots of client/server packet trace pairs (stationary, bus around Stockholm, train to Gothenberg, etc)
- Analyse the traces with standard and homebrew tools to see how the network behaves (variations in latency, patterns of packet loss, etc).
- Write a custom router to reproduce the network conditions in the lab. (Easy in Erlang using the Linux tap/tap driver in Jungerl.)
- Write benchmarking scripts to run traffic loads using various operating systems over the emulated GPRS network and measure file transfer performance. (I had coordinated these using a homebrew distributed "Tuple Space" in about 50 lines of shell and AWK but due to Cygwin hassles I ended up being boring and rewriting it in Erlang.)
- Look hard at the TCP traces to see where TCP could have done things better, i.e. has failed to "keep the pipe full".
- Dream up new TCP algorithms to avoid the mistakes, hack them into the Linux kernel, see if you improve the benchmarks.
- Install your shiny new kernel and get back on the #3 bus...
We did pretty much the whole lot in a couple of months with no previous TCP or kernel hacking experience and we made some big improvements. Truly one of the most fun things I've ever been paid to do :-). (Hi to our colleagues Pierre, Benoit, John, Jerome, Ben, and Mick if you happen to stumble on this page!)
For various reasons we didn't turn this into a real product and consequently I've had the idea itching in the back of my mind ever since. I always keep half an eye out for a new kind of practical network that TCP might suck on, especially one having significant non-congestion packet loss and high variation in latency. Now that I've hacked Squeak into a packet-processor I'm well poised to strike at network-optimization-glory should opportunity come knocking... :-)
(Would it be cruel to hope that the wireless mesh-network intended for millions of adorable children will turn out to have important TCP-related problems that need fixing? :-))