One of the most debated items in the launch version of our Secure Messaging Scorecard is whether communications via Skype are end-to-end encrypted, so that the provider (which is currently Microsoft) can't access them.

In preparing the scorecard, Skype was a hard case for us. In its early days, Skype was a product with a significant cypherpunk dimension to its design. It was launched in 2003, based on the same P2P codebase that had powered the Kazaa filesharing network, and unlike almost any other communications software of its day, it encrypted users' communications (at least their VOIP communications) by default. Not only was the encryption present by default, but it operated end-to-end so that under normal circumstances, Skype would lack the keys to decrypt calls between its users.1 As we discussed in an analysis last year, this protection was limited by the fact that Skype itself told each client the public key to use for each other user; if Skype collaborated with an eavesdropper by providing a false key to the participants in a call, it would in principle be possible to launch a successful man-in-the-middle or impersonation attack.

In the Scorecard, we try to capture these two questions in different columns: systems which are end-to-end encrypted get a check mark for "encrypted so the provider can't read it"; systems which offer some method of protection against false keys and man-in-the-middle attacks get a check mark for "can you verify your contacts' identities." We know from the leaked Snowden documents that the limitations in the protocol or implementation were such that by 2013, Microsoft was capable of accessing the content of Skype text, video, and voice communications, at least in some circumstances for some users. But we didn't know how that capability worked: was it a break against the RC4 cipher Skype used? Was it a method for compelling Microsoft to issue false keys to selected Skype users? Or was it some other flaw in the traditional Skype client?

Any of these cases would be bad, but their implications for different threat models were different. In the case of a cipher break, we could expect intelligence agencies and many other actors to collect the content of Skype calls wholesale; in the case of compelled false keys, interception might still be limited to a small number of targeted individuals. Furthermore, the details of how Skype works may have changed significantly in the past few months, as Microsoft transitioned Skype to a new protocol that deprecated old clients in August. In principle that could have been a good thing – RC4 is no longer a reasonable cipher choice and other flaws in the encryption might have been discovered. But it's also possible that the change gave Microsoft more visibility into Skype calls and/or messages.

Here's what Microsoft says publicly about encryption in Skype today: "Encryption is the process of encoding a message, using principles of mathematics, in such a way that it is readable only by the intended recipient. . . . For Skype your key is your Skype Name and password, hence the criticality of keeping that safe. Skype uses well-known standards-based encryption algorithms to protect Skype users' communications from falling into the hands of hackers and criminals. In so doing, Skype helps ensure user's privacy as well as the integrity of the data being sent from one user to another." (Emphasis added.)

In an attempt to reconcile what we know from media reporting with what Microsoft says publicly, we gave Skype tentative credit for end-to-end encryption based on an interpretation of Microsoft's statement. We did not give Skype credit in the third criterion – an ability to verify contacts' identity. We hypothesized that Skype may still have end-to-end encryption, though it certainly doesn't protect against man-in-the-middle attacks, and we asked Microsoft whether that analysis was accurate. Microsoft initially told us they would provide a prompt response, asked to schedule a meeting, but failed to do so before our launch deadline.

Since we launched the Scorecard, we reached out again to Microsoft asking for an answer on whether it is – under any circumstances – able to observe the content of Skype users' calls and messages. Microsoft stated that in its view, our scorecard was accurate as originally published, it acknowledged our concern, but to date it has not provided a further response to our question: is Microsoft ever able to access the content of Skype communications?

We've determined that our tentative award of a check mark for Skype in the second column of the Scorecard was premature. In projects such as this, we must sometimes rely on statements from providers where we are not in a position to technically evaluate their veracity. In the case of Skype, we can't tell whether Skype lacks end-to-end encryption, or if it includes an implementation of end-to-end encryption that Microsoft is able to silently compromise in certain circumstances. Given the gulf between Microsoft's public statements, its statements to us, and reports in the media of what Microsoft's capabilities appear to be, we're removing Skype's check mark for end-to-end encryption. We invite Microsoft to publicly clarify the status of Skype's implementation of encryption.

• 1. The encryption protected not just against Skype but against other parts of the network, since in some cases calls were routed through other users' computers that were acting as supernodes.