Science

IARPA sets challenge to fact-check fact-checking methods

Fact-checking is a good thing, but who fact-checks the fact-checkers’ methodologies? How do we know if the fact-checking methods actually yield accurate results?

In order to determine the validity of the methods that go into testing whether a source is telling the truth or not, the Intelligence Advanced Research Projects Activity (IARPA) set forth a challenge.

Read More: IARPA launches Proteos program for human identification using shedded skin

There are many so-called fact-checking organizations on the web. Many people turn to sites like Snopes when they have a “hoax or not” type of question.

They pull up the homepage, ask the question, and the results are given, but hardly anybody thinks about how the truth is actually determined.

Read More: Facebook to rely on unnamed third-party fact checkers for fake news

How does a site like Snopes determine what is true and what is fake? How does a site like Snopes make money? Are there potential external influences that might persuade the fact-checkers to lean a certain way? Are there possible flaws in the way fact-checkers evaluate the truth?

IARPA CASE Challenge

Through IARPA, the US intelligence community is looking for ways to assess and improve upon current methods of testing that go into determining whether a source is truthful or not.

Read More: Intelligence community can’t keep up with FOIA requests: IG report

The IARPA Credibility Assessment Standardized Evaluation (CASE) Challenge launched on January 3, and it is an open competition that offers up to $125,000 in prizes to participants who can develop a novel and creative method for measuring the accuracy of current and future credibility assessment  techniques used across government, academia and industry.

This challenge is focused on the methods used to evaluate credibility assessment techniques or technologies, rather than on the techniques or technologies themselves.

The CASE Challenge about focusing on how to assess a broad range of tools and techniques, but not on developing specific credibility tools and techniques.

Credibility assessment refers to both the assessment of the truthfulness of specific claims and to the assessment of the reliability, honesty, and trustworthiness of a source of a particular claim, whether that be an individual, group, or a broader organization or entity.

In theory, the results of this challenge could be applied accross a wide variety of industries, including journalism.

Now, more than ever, people are focused on verifying who, or what, is honest and trustworthy. We see this reflected everywhere from the national media to our professional and personal conversations – every day we must make decisions about someone’s credibility.

According to IARPA, “For some in-person and virtual interactions there are tools to aid our judgments. These might include listening to the way someone tells a story, looking at a user badge, validating with other people – or in more formal settings, verifying biometrics or recording someone’s physiological responses, i.e. the polygraph.

“Each of these examples uses a very different type of tool to augment our ability to evaluate credibility. Yet there are no standardized and rigorous tests to evaluate how comprehensive or accurate such tools really are.”

Where The Truth Lies

There are many tools to help us determine the truth such as body language, polygraph tests, authority figures, common sense, and past experiences, but none of these tools on their own can be 100% accurate 100% of the time.

The US intelligence community is looking to make fact-checking and truth-telling methodologies more accurate.

However, the search for truth seems to be more in the realm of philosophy, religion, and the arts rather than in a government challenge. Perspectives and paradoxes abound.

Read More: Is there nothing that can’t be faked with vocal, facial manipulation?

Like with the famous double-slit experiment, is it a particle, or is it a wave? Depending on your point of view and manner of measurement in space and time, it could be both simultaneously!

Where does the truth lie?

Tim Hinchliffe

The Sociable editor Tim Hinchliffe covers tech and society, with perspectives on public and private policies proposed by governments, unelected globalists, think tanks, big tech companies, defense departments, and intelligence agencies. Previously, Tim was a reporter for the Ghanaian Chronicle in West Africa and an editor at Colombia Reports in South America. These days, he is only responsible for articles he writes and publishes in his own name. tim@sociable.co

Recent Posts

Club of Rome launches joint taskforce that would restrict your food, travel & ownership choices

The 'Materials and Consumption Taskforce' is an attempt to micro-manage all aspects of your life:…

4 days ago

As the tech talent shortage continues, Ness Digital Engineering nurtures its rising stars

Talent in the tech industry has long been a hot commodity. Yet in today's world,…

4 days ago

Prezent launches My Workspace to ensure AI-powered presentation software works as tool, not a burden

In an ideal scenario, professionals in 2025 should be able to leverage a personal suite…

5 days ago

NTT’s simultaneous Kabuki performance shows photonics connectivity is more than a song and dance

In a fusion of tradition and technology, Japanese tech firm NTT unveiled the capabilities of…

6 days ago

UN finalizes neurotech ethics draft, to be adopted at General Conference

The United Nations Educational, Scientific, and Cultural Organization (UNESCO) finalizes its "Recommendation on the Ethics…

2 weeks ago

Inside the Dead Internet Theory: Profits in a World Run by Bots

The dead internet theory is a conspiracy theory that goes: Most of the content we…

2 weeks ago