Error and bias in forensic behavioral science

A cri­sis is brew­ing in sci­en­tific psy­chol­ogy and foren­sic sci­ence. This con­cerns the over­all reli­a­bil­ity of all psy­cho­log­i­cal sci­ence (more on this later) and the qual­ity of expert evi­dence and tes­ti­mony in courts of law, regard­less of the expert’s dis­ci­pline (yes, even includ­ing the sup­pos­edly hard stuff, like DNA analysis).

The con­sen­sus: alarm­ingly poor. This is the opin­ion of National Academy of Sciences’ National Research Council Report, “Strengthening Forensic Science in the United States: A Path Forward.“
“The focus has shifted to include the eval­u­a­tion of meth­ods and tech­niques rather than sim­ply the expert’s inter­pre­ta­tion of the results” (Christensen et al, 2014, p. 123).

The con­cern here is the qual­ity of expert tes­ti­mony in high stakes sit­u­a­tions where the reli­a­bil­ity of judi­cial decision-making, indeed the very integrity of the legal sys­tem is raised.

If experts are doing due dili­gence in enhanc­ing the qual­ity of their foren­sic work, includ­ing the thor­ough­ness and integrity of their meth­ods and opin­ions, two recent arti­cles, one a major con­tri­bu­tion, address these issues.

First, the arti­cle by Christensen et al (2014), Error and its mean­ing in foren­sic sci­ence. Journal of Forensic Science, 59, 1, 123–126, addresses the fun­da­men­tal ques­tion of error.

What is error? Is error com­mon, inevitable, cor­rectable, or avoid­able? Are you aware that you even have an error rate in your decision-making?

In foren­sic men­tal health assess­ments, research shows that reli­a­bil­ity is poor; hence, height­ened error rates.

What is error? Error means that you made the wrong deci­sion, that your opin­ions are flawed or incor­rect based pro­ce­dural and/or method­olog­i­cal deficiencies…or bias.

What is bias?

A very pow­er­ful arti­cle by Neal & Grisso (2014) pro­vides a con­cep­tual foun­da­tion and call for research on bias in foren­sic men­tal health eval­u­a­tions, address­ing con­scious and uncon­scious biases, includ­ing com­mon cog­ni­tive heuris­tics uti­lized by all human beings, includ­ing sci­en­tists, lawyers, judges, and juries.

They pro­vide clear and con­cise def­i­n­i­tions and con­cep­tual issues for address­ing the con­cep­tual foun­da­tions of bias. This work in rooted in the sem­i­nal, 1974 arti­cle by Tversky & Kahneman, which opened the whole era of cog­ni­tive heuris­tics and the role of bias in the intu­itive judg­ment of prob­a­bil­ity under con­di­tions of uncertainty.

Hard going in the read, but valu­able in the payoff.

References:

Neal, T., & Grisso, T. (2014). The cog­ni­tive under­pin­nings of bias in foren­sic men­tal health eval­u­a­tions. Psychology, Public Policy, and the Law, on line publication.

Tversky, A., & Kahneman, D. (1974). Judgment under uncer­tainty: Heuristics and biases. Science, 185, 27 September 1974, 1124–1131.


Ipse Dixit, Expert witnesses, & Specialty Guidelines for Forensic Psychology

Ipse Dixit in Latin means, “he him­self said it”; It is fre­quently encoun­tered in expert tes­ti­mony. Ipse dixit is an unsup­ported state­ment that rests solely on the author­ity of the speaker. I think a bet­ter trans­la­tion is “because I said it…[it is true].” It is a con­clu­sory opin­ion with­out sup­port. Gutheil & Bursztjan (2003) articulate […]


Bias, cognitive heuristics, and forensic decision-making

The cumu­la­tive weight of 30 years of research orig­i­nat­ing in Tversky and Kaheman’s sem­i­nal 1974 descrip­tion of the men­tal short­cuts humans use when con­fronting cog­ni­tive ambi­gu­ity com­plex­ity, and Kahneman’s recent mas­ter­piece Thinking Fast, and Slow (2011, cf. Gilovich, Griffin, & Kahneman, 2002) has cre­ated a mature cog­ni­tive sci­ence of foren­sic decision-making. The find­ings are not […]


Forensic Psychology is Forensic Science.

There has been a grow­ing aware­ness, some would say cri­sis, con­cern­ing the mediocre qual­ity of expert foren­sic tes­ti­mony sub­mit­ted to the courts in the United States. This appears to be a func­tion of weak meth­ods, and a low thresh­old for judi­cial gate­keep­ing, despite clear man­dates in Rule 702 and Daubert and prog­eny on the foundations […]


Professional Ethics,“partisan allegiance,” and expert witness liability

Daniel Murrie and his col­leagues have pro­duced a series of recent stud­ies on “par­ti­san alle­giance” in foren­sic expert wit­ness work (see cita­tions below). They define par­ti­san alle­giance as the ten­dency for experts to bias their tes­ti­mony in favor of the side that calls them. They demon­strate this through inge­nious exper­i­ments with PCL-R rat­ings. The response […]


Thriving as a forensic psychologist

A nice arti­cle by William Foote PhD in APA’s Good Practice (Winter 2014) is worth a look for stu­dents who think they might be inter­ested in foren­sic psy­chol­ogy. Contrasting clin­i­cal and foren­sic psy­chol­ogy, he writes, “But the field is not for everyone…Forensic psy­chol­ogy is also much more con­fronta­tional. You’re often in an adver­sar­ial posi­tion, with […]


Competency Restoration in the State of Hawaii: An object lesson

Karen Franklin’s blog is the pre­mier site for ground-breaking issues in foren­sic psy­chol­ogy. Always worth­while and read­able, witty and bit­ing, some­times breath­tak­ing and bold. Her March 9, 2014, post addresses the recent scan­dal at Napa State Hospital in California, which serves the Northern California incom­pe­tent defen­dant and NGRI pop­u­la­tion, sim­i­lar to our Hawaii State Hospital. […]


Assessing Homicidal Mental States at the Time of Offense IV: The 2011 Norway Massacre: Criminal responsibility analysis using the R-CRAS.

The use of foren­sic assess­ment instru­ments is a dis­tin­guish­ing fac­tor in the qual­ity of foren­sic reports (Fuger, Acklin, Nguyen, Ignacio, & Gowensmith, 2013). In insan­ity deter­mi­na­tions, the Rogers Criminal Responsibility Assessment Scales (R-CRAS, PAR, Inc.) are with­out peer as an aide to orga­niz­ing and ana­lyz­ing case data. It is a sys­tem­atic cod­ing mea­sure which permits […]


Assessing Homicidal Mental States at the Time of Offense III: The 2011 Norway Massacre: The 2nd forensic psychiatric examination.

Based on pub­lic­ity and pro­fes­sional com­men­tary (“…intense media cov­er­age, with repeated requests for a new eval­u­a­tion by major news­pa­pers and politi­cians”), the court ordered a sec­ond foren­sic psy­chi­atric exam­i­na­tion of Breivik. It was con­ducted 6 months after the first exam­i­na­tion. The two psy­chi­a­trists “con­ducted a con­sid­er­ably longer eval­u­a­tion, and used addi­tional psy­cho­log­i­cal assess­ment instru­ments.” They […]


Assessing Homicidal Mental States at the Time of Offense: The 2011 Norway Massacre II: The 1st forensic psychiatric examination.

The first exam­i­na­tion con­ducted by two psy­chi­a­trists included 13 inter­views (36 hours total), review of police inter­ro­ga­tions, and col­lat­eral inter­view with Breivik’s mother. They coded the mate­r­ial using the Norwegian ver­sion of the Mini-International Neuropsychiatric Interview Plus for ICD-10, and selected mod­ules of the Norwegian ver­sion of the Structured Clinical Interview for DSM-IV (SCID-IV). Based […]