EFF has filed comments on two proposed rules in California regarding the use of pretrial risk assessment tools in courts, urging the Judicial Council to place meaningful restrictions on the use of pretrial risk assessment tools in the state. The California Judicial Council issued the draft rules in accordance with the state’s new bail reform law, S.B. 10, which replaces cash bail with algorithmic pretrial risk assessments. Under the law, each county in California must use some form of pretrial risk assessment to categorize every person arrested as a “low,” “medium,” or “high” risk of failing to appear for court, or committing another crime that poses a risk to public safety in the period before trial. The Judicial Council rules are supposed to guide how courts use pretrial risk assessment information and make detention decisions for individuals assessed as “medium” risk. But just like the statute itself, the Judicial Council’s proposed rules fail to place any meaningful limitations on the use of risk assessment information that would protect against due process violations and unintentional, unfair, biased, or discriminatory outcomes.

We don’t think actuarial tools should ever be the deciding factor in a decision to detain an individual. For one, they boil a person down to just a few characteristics in order to predict risk and are unlikely to ever be able to capture all of the possible edge cases. Pretrial risk assessment tools can replicate the same sort of outcomes as existing systems that rely on human judgment—and even make new, unexpected errors.

That’s why real limitations are needed. Below are some of the things EFF is calling for. Please see our full comments for a complete list.

1.  Risk assessment tools must clearly distinguish between the types of risks being assessed. The risk of failing to appear is separate and distinct from the risk of committing an offense that presents a threat to public safety. And the risk of intentional flight itself may be importantly different from other forms of failure to appear. Thus, risk assessment tools must define when an individual is assigned a risk score in one category and not the other, so that courts can effectively identify appropriate conditions to place on defendants for release.

2.  The Judicial Counsel must clarify how risk scores should be generated and what they purport to predict, and set thresholds for low, medium, and high risk based on specific policy objectives of the state. When determining these policy objectives, California should be asking and answering the following question: “What trade-offs should we make to ensure justice and lower the massive social costs of incarceration?”

3.  Risk assessment scores must be accompanied by detailed descriptions of how they were calculated—or “audit trails”—and all of the information necessary to enable a reproducible calculation of a particular individual’s risk score. This means all of factors that went into an individual’s the risk score, the weight each factor was given, and how each factor is defined, so that judges can fairly and thoroughly evaluate the risk score in the context of the case at hand and understand what specific factors are being relied upon to define the risk of failing to appear or risk of committing a crime that presents a threat to public safety during the period of pretrial release.

4.  Courts should not be allowed to rely on proprietary risk assessment tools if a vendor refuses to disclose information about how the risk assessment tool was developed. To ensure accountability and the just use of pretrial risk assessment tools, vendors must be transparent about the data and assumptions that they used to build their models and that are reflected in individual risk scores.

5.  Courts must take into account the confidence of the risk assessment tool in the assessed risk assessment score. Not all risk scores are created equal. The same risk assessment tool may have a high confidence in the risk score that it assigns one defendant, and a low confidence in the risk score it generates for another defendant—even if they are both assigned to the same risk category. Even when placing a simple bet, it would be foolish not to consider the level of confidence in a prediction. When an individual’s liberty is at stake, it is improper and irresponsible not to do so.

6. The Judicial Council must define processes by which defendants get access to risk assessment information and can challenge both the risk assessment tool and/or their score. To meaningfully challenge their risk scores, defendants must have access to all of the information that a judge has access to, pursuant to our recommendations—i.e., the entire risk assessment report and any and all information related to the results of the risk assessment. This includes all of the information that went into defining and calculating their risk scores and any information regarding the tool’s confidence in their risk scores. They must also be granted access to the training data, validation data, error rates, and the results of any and all fairness and bias evaluations.

7.  The Judicial Council must place limits on the use of the pretrial risk assessment information in further proceedings. The formulas underlying these tools must be tailored to the specific request at hand, and the scores generated must be used only for that specific purpose. Indeed, predicting these specific risks in the pretrial setting is distinct from predicting them in the context of parole, probation, or sentencing, and a tool must be validated for predicting risk in each specific context. The risk score generated in one context should therefore not be reused in another context.

8.  Courts must not use information generated from a pretrial risk assessment tool if it has not been evaluated and found to satisfy criteria that independent scientific research confirms will accomplish the government’s desired public policy goals—and if the results of such review have not been published and shared with defendants. To evaluate a tool, or information generated by any given tool, judges and other court officers should be required to consider: (i) the methodology and results of an evaluation by an independent group (i.e., not the vendor) on how whether or not the tool disparately affects different protected classes, including an assessment of the false positive and false negative rate across groups; (ii) whether the tool satisfied a fairness criteria demonstrated by independent scientific research to mitigate bias on the basis of race, gender, or other protected classes; (iii) results of a validation check conducted on the tool within the last 12 months by an independent group; and (iv) the results of regular tests measuring the disparate impact of tool error rates by race, gender, and other protected classes.

9.  Pretrial risk assessment information must be used only to aid in the assessment of what conditions of release are appropriate. While S.B. 10 requires pre-arraignment detention for those who are assessed “high” risk, S.B. 10 enables court officers to consider whether to detain people who are assessed “medium” risk. That decision to detain must be based entirely on other factors—not the risk score. Because of the inherent limitations and challenges of risk assessment tools, they are not appropriate for recommending detention or for informing decisions about whether to detain an individual. What’s more, a decision to detain an individual who receives a “medium” risk score should be based on an independent evaluation of the circumstances and factors underlying the risk score, not the risk score itself.

10.  Pretrial risk assessment tools, and court officials, should not consider arrests that that did not result in convictions in assessing the risk that someone will commit a dangerous crime if released before their trial. The objective of the risk assessment is not to determine whether the defendant is likely to be convicted of or arrested for any other alleged offense in the indefinite future. It is to determine how likely a defendant is to commit a violent offense in the pretrial period that justifies pretrial detention. An arrest that did not result in a conviction is not a good proxy for future criminality. It is well documented that different demographic groups are stopped, searched, arrested, and charged at very different rates across the United States.

We hope the Judicial Council will seriously consider these recommendations and update its proposed rules to ensure that the state protects the due process rights of all Californians.