A Misunderstanding of Data Leads to a Misunderstanding of Patent Law and Policy

Bad patents shouldn’t be used to stifle competition. A process to challenge bad patents when they improperly issue is important to keeping consumer costs down and encouraging new innovation. But according to a recent post on a patent blog, post-grant procedures at the Patent Office regularly get it “wrong,” and improperly invalidate patents. We took a deep dive into the data being relied upon by patent lobbyists to show that contrary to their arguments, the data they rely on undermines their arguments and conflicts with the claims they’re making.

The Patent Office has several procedures to determine whether an issued patent was improperly granted to a party that does not meet the legal standard for patentability of an invention. The most significant of these processes is called inter partes review, and is essential to reining in overly broad and bogus patents. The process helps prevent patent trolling by providing a target with a low-cost avenue for defense, so it is harder for trolls to extract a nuisance-value settlement simply because litigating is expensive. The process is, for many reasons, disliked by some patent owners. Congress is taking a new look at this process right now as a result of patent owners’ latest attempts to insulate their patents from review.

An incorrect claim about the inter partes review (IPR) and other procedures like IPR at the Patent Trial and Appeal Board (PTAB) has been circulating, and was recently repeated in written comments at a congressional hearing by Philip Johnson, former head of intellectual property at Johnson & Johnson. Josh Malone and Steve Brachmann, writing for a patent blog called “IPWatchdog,” are the source of this error. In their article, cited in the comments to Congress, they claim that the PTAB is issuing decisions contrary to district courts at a very high rate.

We took a closer look at the data they use, and found that the rate is disagreement is actually quite small: about 7%, not the 76% claimed by Malone and Brachmann. How did they get it so wrong? To explain, we’ll have to get into the nuts and bolts of how such an analysis can be run.

Malone and Brachmann relied on data provided by a service called “Docket Navigator,” which collects statistics and documents related to patent litigation and enforcement. The search they used was to see how many cases Docket Navigator marked as a finding of “unpatentable” (from the Patent Office) and a finding of “not invalid” (from a district court).

This is a very, very simplistic analysis. For instance, it would consider an unpatentability finding by the PTAB about Claim 1 of a patent to be inconsistent with a district court finding that Claim 54 is not invalid. It would consider a finding of anticipation by the PTAB to be inconsistent with a district court rejecting an argument for invalidity based on a lack of written description. These are entirely different legal issues; different results are hardly inconsistent.

EFF, along with CCIA, ran the same Docket Navigator search Malone and Brachmann ran for patents found “not invalid” and “unpatentable or not unpatentable,” generating 273 results, and a search for patents found “unpatentable” and “not invalid,” generating 208 results (our analysis includes a few results that weren’t yet available when Malone and Brachmann ran their search). We looked into each of 208 results that Docket Navigator returned for patents found unpatentable and not invalid. Our analysis shows that the “200” number, and consequently the rate at which the Patent Office is supposedly “wrong” based on a comparison to times a court supposedly got it “right” is well off the mark.

We reached our conclusions based on the following methodology:

• We considered “inconsistent results” to occur any time the Patent Office reached a determination on any one of the conditions for patentability (namely, any of 35 U.S.C. §§ 101, 102, 103 or 112) and the district court reached a different conclusion based on the same condition for patentability, with some important caveats, as discussed below. For example, if the Patent Office found claims invalid for lack of novelty (35 U.S.C. § 102), we would not treat a district court finding of claims definite (35 U.S.C. § 112(b)) as inconsistent.
• We did not distinguish between a finding of invalidity or lack of invalidity based on lack of novelty (35 U.S.C. § 102) or obviousness (35 U.S.C. § 103), as these bases are highly related. For example, if the Patent Office determined claims unpatentable based on anticipation, we would mark as inconsistent any jury finding that the claims were not obvious.
• We did not consider a decision relating the validity of one set of claims to be inconsistent with a decision relating to the validity of a different, distinct set of claims. For example, if the Patent Office found claims 1-5 of a patent not patentable, we would not consider that inconsistent with a district court finding claims 6-10 not invalid. We would count as inconsistent, however, any two differing decisions that overlapped in terms of claims, even if there was not identity of claims.
• We distinguished between the conditions for patentability of 35 U.S.C. § 112. For example, a district court finding of definiteness under 35 U.S.C. § 112(b) would not treated as inconsistent with a Patent Office finding of lack of written description under 35 U.S.C. § 112(a).
• We did not consider a district court decision to be inconsistent with Patent Office decision if that district court decision was later overturned by the Federal Circuit. However, we did treat a Patent Office decision as inconsistent with a district court decision even if that Patent Office decision were later reversed.1 For example, if the Patent Office found claims to be not patentable, but the Patent Office was later reversed by the Federal Circuit, we would still mark that decision as inconsistent with the district court. We even counted Patent Office decisions as inconsistent in the five cases where they were affirmed by the Federal Circuit and therefore were correct according to a higher authority than a district court. We did this in order to ensure we included results tending to support Malone and Brachmann’s thesis that the Patent Office was reaching the “wrong” results.
• We excluded fourteen results that were not the result of any district court finding. Specifically, several patents were included because of findings by the International Trade Commission, an agency (like the Patent Office) which hears cases in a non-Article III court and that does not have a jury. Those results would not meet Malone and Brachmann’s thesis of being considered “valid in full and fair trials in a court of law.”
• We excluded two results that should not have been included in the set and appear to be a coding error by Docket Navigator.* These results were excluded because there was no final decision from the Patent Office as to unpatentability.

Here’s what we found of the 194 remaining cases:

• A plurality of the results (n=85) were only included because the Patent Office determined claims were unpatentable based on failure to meet one or more requirements for patentability (usually 35 U.S.C. § 102 or 103) and a district court found the claims met other requirements for patentability (usually 35 U.S.C. § 101 or 112). That is, the district court made no finding whatsoever relating to the reasons why the Patent Office determined the claims should be canceled. Thus the Patent Office and the court did not disagree as to a finding on validity.
• For example, the Docket Navigator results include U.S. Patent No. 5,563,883. The Patent Office determined claims 1, 3, and 4 of that patent were unpatentable based on obviousness (35 U.S.C. § 103). A district court determined that those same claims however, met the definiteness requirements (35 U.S.C. § 112(b)). The Federal Circuit affirmed the Patent Office’s decision invalidating the claims, and the district court did not decide whether those claims were obvious at all.
• A further 46 results were situations where either (1) the patent owner requested the Patent Office cancel claims or (2) claims were stipulated to be “valid” as part of a settlement in district court. Thus the Patent Office and the court findings were not inconsistent because at least one of them did not reach any decision on the merits.
  • For example, the Docket Navigator results includes U.S. Patent No. 6,061,551. A jury found claims not invalid, but the Federal Circuit reversed that finding, holding the claims invalid. After that determination, the Patent Owner requested an adverse judgment at the Patent Office.
  • As another example, the Docket Navigator results includes U.S. Patent No. 7,676,411. The Patent Office found claims invalid as abstract (35 U.S.C. § 101) and obvious (35 U.S.C. § 103). Because the parties stipulated that this patent was “valid” as part of settlement, which is generally not considered to be a merits determination, this patent is also tagged as “not invalid” by Docket Navigator.
• A further 15 results were not inconsistent for a variety of reasons.
  • For example, five results were not inconsistent because the Patent Office and the district court considered different patent claims. As another example, U.S. Patent No. 7,135,641 represented an instance where a jury found claims not invalid, but the district court judge reversed that finding post-trial. As another example, in the district court, U.S. patent 5,371,734 was held “not invalid” on summary judgment, but that determination was later reversed by the Federal Circuit.

Under this initial cut, only 48 of the entries arguably could be considered to have inconsistent or disagreeing results between the Patent Office and a district court.

But in the majority of those cases, a judge or jury considered one set of prior art when determining whether the claim was new and nonobvious, but the Patent Office considered a different set (n=28). It is not surprising that the two forums would consider different evidence. The Patent Office proceedings generally only consider certain types of prior art (printed publications). That a district court proceeding may result in a finding of “not invalid” based on, e.g., prior use, is not an inconsistent result.

Eliminating those results where the Patent Office was considering completely different arguments and art means the total number of times the Patent Office arguably reached a different conclusion than a district court is only 20 times out of 273 that  a district court determined a patent "not invalid" for some reason. That means that the Patent Office is “inconsistent” with district courts only 7% of the time, not 76% of the time.

It is also important to keep in mind that there have been over 1,800 final decisions in inter partes review proceedings, covered business method review proceedings, or post grant review proceedings. In all that though, only 20 times did the Patent Office reach a conclusion that may be considered inconsistent with the district court in ways that negatively impact patent owners. That’s a rate of only around 1% of the time. That’s a remarkably low rate. Moreover, inconsistent results happen even within the court system. For example, in Abbott v. Andrx, 452 F.3d 1331, the Federal Circuit found that Abbott’s patent was likely to be held invalid. But only one year later, in Abbott v. Andrx, 473 F.3d 1196, the Federal Circuit found that the same patent was likely to be not invalid. The two different results were explained by the fact that the two defendants had presented different defenses. This is not unusual. Thus the fact that there may be different results doesn’t lead to a conclusion that the whole system is faulty.

An analysis like ours with respect to this data set takes time and a few cases might slip through the cracks or be incorrectly coded, but the overall result demonstrates that the vast majority of patent owners are never subject to inconsistent results between district court and the Patent Office.

It is disappointing that Johnson, Malone, and Brachmann made claims that the data don’t support, but demonstrates a valuable lesson. When using data sets, it is important to understand what, exactly, the data is and how to interpret it. Unfortunately here it looks like an error in understanding the results provided by Docket Navigator by Malone and Brachmann propagated to Johnson’s testimony, and would likely travel further if no one looked harder at it.

We’ve used both Docket Navigator and Lex Machina in our analyses on numerous occasions, and even briefs we submit to the court. Both services provide extremely valuable information about the state of patent litigation and policy. But its usefulness is diminished where the data they present are not understood. As always, the devil is in the details.

• 1. For this reason, our results differ slightly from those of CCIA, reported here. CCIA did not treat decisions as inconsistent if the Patent Office decision was later affirmed on appeal. Five patents we considered inconsistent in our analysis were excluded in CCIA’s analysis. Each approach has merit.