Knowing when someone is telling the truth plays a critical role in law enforcement and national security events, to include criminal investigations, screening new employees before hiring, and interviewing potential sources and witnesses.
The polygraph is one tool that members of the Intelligence Community (IC) and law enforcement look to for help, but there is a long-standing debate among researchers and polygraph practitioners about the accuracy and reliability of this tool.
How can we evaluate how good the polygraph is, and how much better new tools may be?
The Intelligence Advanced Research Projects Activity (IARPA), within the Office of the Director of National Intelligence (ODNI), will be launching the Credibility Assessment Standardized Evaluation (CASE) Challenge to address this critical question, and is holding a Participant’s Day for the CASE —prize challenge.
The prize challenge aims to develop novel procedures to evaluate the accuracy, reliability, and utility of current and future credibility assessment techniques and technologies, such as the polygraph.
“The Intelligence Community uses in-person credibility assessment techniques every day to screen new employees before hire, conduct periodic investigations with existing employees, and interview potential sources and witnesses,” explains IARPA Program Manager Alexis Jeannotte.
“The polygraph is one tool used for credibility assessments, but there are newer techniques and technologies on the market and in development that may prove more useful.
“However, we need standardized, rigorous procedures to evaluate these newer approaches and the current methods have limitations.”
“We hope that challenge participants will have new, unique, and creative ideas to address these limitations.”
Credibility refers to the truthfulness of information and/or to the person providing that information.
Assessments of credibility are often complex and may involve an evaluation of many factors of a person and/or their information, to include, but not limited to, veracity, trustworthiness, motivation, and considerations about what may be withheld or concealed.
(Learn More. Is it possible to beat a lie detector test? National Geographic tries out various methods to outwit a lie detector and keep his “crime” hidden. Courtesy of National Geographic and YouTube.)
To evaluate the credibility of an individual and/or their information the IC and law enforcement often use human judgment and complement this with additional techniques and technologies, such as specific interviewing techniques or devices, like the polygraph, to record behavioral or physiological responses when someone responds to a question.
The polygraph test remains the standard for using technology-assisted assessments of credibility, despite having undergone only modest improvements over roughly the last fifty years.
Other methods and systems have been proposed, but for varying reasons have not achieved widespread use for national security or law enforcement applications.
A major limitation is that current procedures for evaluating these technologies frequently have little applicability to real-world credibility assessment situations, resulting in levels of performance in everyday use that sometimes differ dramatically from those reported by laboratory researchers.
Real applications of credibility assessment tools and techniques might include the following:
-
Issue-Specific:
- An investigator must evaluate the credibility of the information a person may provide on a specific event or topic, for example after a crime has been committed.
-
Screening:
- A manager must determine the credibility of a person and/or their information, such as a potential new employee.
- In such situations the manager may ask more general and prospective questions, as well as retrospective ones (e.g. Do you intend to keep the company’s trade secrets? Have you ever used illegal substances?).
- The truth may be difficult to determine and it may not be completely objective.
The CASE Challenge is the first concerted effort to invite interested individuals to develop credibility assessment evaluation methods that can be used to objectively evaluate both existing and novel credibility assessment techniques/technologies.
In doing this the CASE Challenge strives to incentivize a broad range of new ideas, while still ensuring their utility to real-world applications.
To meet this goal, a panel of experts will evaluate each solution based on:
- Its novelty and innovation
- The potential utility of the protocol in practice
- Its scientific credibility and how likely it will be to generate valid and repeatable outcomes and objective ground truth about whether someone is truly credible or not
- Its applicability to real-world use cases for the IC and law enforcement, and
- Its responsible care and protection of people who would participate in the protocol
In addition to the financial incentives for prize winners, participants will be invited to a workshop where winners will present their solutions and people who use credibility assessment techniques and technologies will mix with researchers and other challenge participants.
It is anticipated that there will be several prizes with a total prize purse of $100,000.
The prizes will focus on ideas that are novel, innovative, safe, have the potential to be useful, and are scientifically credible.
IARPA is conducting this prize challenge to invite a broad community of participants, both domestic and international, to participate in a convenient, efficient, and non-contractual way.
An informational web-based Participant’s Day will be held on Tuesday, July 31, 2018. During that session, further details will be provided about the challenge goals and rules.
Solvers who are eligible to win a prize and with the most accurate and complete solutions will be eligible to win cash prizes from a total prize purse of $100,000.
The challenge is expected to officially launch in September 2018 with the winners being announced in Spring 2019.
To learn more about the CASE prize challenge, including details for the Participant’s Day, rules, criteria, and eligibility requirements, visit https://www.iarpa.gov/index.php/working-with-iarpa/prize-challenges/1119-credibility-assessment-standardized-evaluation-case-prize-challenge.
For updates, follow @IARPAnews on Twitter and join the conversation using #iarpaCASE.
For any questions, please contact us at CASEChallenge@iarpa.gov.
IARPA invests in high-risk, high-payoff research programs to tackle some of the most difficult challenges of the agencies and disciplines in the Intelligence Community. Additional information on IARPA and its research may be found on www.iarpa.gov.
