Yesterday's announcement of a détente between Comcast and BitTorrent was great news. Unfortunately, the general problem of ISPs doing strange things to Internet traffic without telling their customers is likely to continue in the future. EFF and many other organizations are working on software to test ISPs for unusual (mis)behavior. In this detailed post, we have a round-up of the tools that are out there right now, and others that are in development...

The Backstory

When you sign up for an Internet connection, you expect it to actually be an Internet connection. You expect that you can run whatever applications and protocols you choose over the link, or indeed that you can write your own software and run that.

There is a disturbing trend, however, of ISPs stepping in to meddle with your communications, deciding that some applications and protocols are more suitable than others. Or deciding that they can inject advertisements into your queries for domain names, or your browser's exchanges with web sites. Or deciding that encrypted traffic should be throttled across the board.

Whatever you may think about the merits of these practices, we think it's obvious that consumers have a right to know what they're paying for. Only then can they exert pressure on an ISP to change its ways, or vote with their wallets and take their business elsewhere. As we argued in a recent submission to the FCC, ISPs should (at a minimum) disclose the nature of their "network management" practices.

But disclosure will never be enough. Internet users need to be able to test networks themselves to make sure that packets and web pages arrive as they were sent, to make sure that DNS queries are correctly answered, and that ISPs comply with the Internet's standards. That needs to happen around the whole planet.

There are lots of approaches to ISP testing

Before we start talking about all the tools that are popping up for ISP testing, it's worth noting that there are a lot of different ways to test a network, with many different pros and cons. For instance, the software may:

• Actively send "synthetic", pre-determined test traffic, or passively observe the way the network treats natural traffic;
• If the traffic is synthetic, the testing software may try to cope with the complex variation in operating systems and network environments, or try to simplify things by creating or insisting on a known test environment;
• Passive testing systems may focus on one or a small number of protocols, or they may try to test for interference in any protocol that is present;
• The software may (1) be unilateral, just trying to detect interference or delay by examining what's happening on a single computer's network connection, or (2) be multi-party, synchronizing and comparing records from computers that are talking to each other, or (3) be in between, only having authoritative records from one end but possessing special knowledge about how the other end will behave;
• Non-unilateral testing systems may rely on a central server, or they may just try to coordinate records in a peer-to-peer fashion;
• Software may operate at the packet level, measuring integrity, latency and reliability on a per-packet basis, or it may operate at a higher level, confirming (for example) that web pages arrive intact or that a link is running at a certain speed, without worrying about any of the individual packets.

It's a good idea for the Internet community to be pursuing most of these different possibilities, because they're all useful in different situations, and we don't yet know which techniques will prove to be the most important.

Existing and soon-to-be-released tools and data

Last year, EFF released a simple utility called pcapdiff (many thanks to the people who've sent us patches and bug reports; we'll be releasing version 0.2 shortly). EFF is also working on a much more elaborate tool for testing ISPs, which will be called Switzerland. More on that below.

An Italian group has developed an ISO CD image that can be used to test an Internet connection (the CD uses pcapdiff, too!). It's called The Gemini Project. Deploying a whole temporary operating system on a CD is a great example of the "simplify the test environment" approach we described above.

Vuze, the company formed around the Azureus BitTorrent client, has released a plugin that counts the number of RST packets sent to your BT client. These statistics are interesting, but remember that there are legitimate RST packets, and the presence of TCP RSTs isn't evidence that they were spoofed by an intermediary.

The problem of ISPs making arrangements with advertisers to inject extra ads and tracking mechanisms into web pages prompted researchers at the University of Washington and Berkeley to implement some high-level server-side testing tools called "Web Tripwires". These are bits of Javascript in a page whose purpose is to calculate an secure checksum of the final contents. Tripwires will quickly tell a user if their ISP is injecting advertisements — unless the advertisers get sneaky and avoid injecting ads into pages that contain javascript.

Lauren Weinstein has set up a very informative mailing list called NNSquad to discuss "network neutrality" and to find ways to test it. Lauren and some other members of that group have been putting together a piece of software, called NNMA, that will be released soon.

A group of researchers at the Max Planck Institute for Software Systems has published a Java applet that generates BitTorrent traffic from within your browser and looks for unusual treatment of it (they haven't posted any source code, so we aren't sure how reliable their tests are — but if you've seen interesting results from this applet, please let us know).

Several other academic research groups and individual researchers are working in this space too. The Networks Group at Northwestern University has a large project, and several network testing apps in the pipeline. Robb Topolski, Juan Carlos de Martin at the Politecnico di Torino, Parminder Chhabra at Boston University and collaborators, and the networking group at Berkeley's ICSI, and Dan Kaminsky have all told us about interesting tools and results that are in the pipeline. We'll update our Test Your ISP page as these are published.

As we mentioned above, EFF and a team of excellent volunteers has been working on its own more sophisticated network testing tool, which will be called Switzerland. We're aiming to publish a draft specification in the next week or two, and to have an alpha release for network engineers and protocol developers to try out by the end of April. In the mean time, we'll say that (unlike most of the above projects), our first architectural objective is to do full multi-party passive monitoring, looking for forged packets in any protocol as well as unusual dropped packet and latency statistics. We'll do that by having each client report secure summaries of the traffic that's being tested to a server running Switzerland code. Full details and discussion will be in the spec!