The eye of the beholder

Transparent pipelines for assessing online information quality

The amount of online information online and the impact it has on society imply the need for an automated assessment of information quality. As quality assessments can be perceived as subjective or biased, and since commercial social networks often operate in a non-transparent manner, technology supporting such assessments needs to be transparent and tunable to support specific scholarly requirements.

To achieve these goals, we build on the existing QuPiD and NEWSGAC platforms. QuPiD is a proof of concept pipeline for information quality assessment that involves crowdsourcing, machine learning, and symbolic reasoning.

To allow scholars to benefit from the platform, we need to empower the user to tune such pipelines. For example, she may decide to collect training data manually or from a crowdsourcing platform; to either use supervised or unsupervised machine learning methods to analyse the quality of documents, and she should be aware of the implications of her decisions.

We build on the NEWSGAC framework, as it allows domain specialists to investigate and tune their machine learning pipelines. We aim at extending NEWSGAC´s transparency-enabling architecture to fulfil the above requirements, thus considering hybrid pipelines that combine crowdsourcing, symbolic reasoning, and machine learning.

Participating organisations

Netherlands eScience Center
Social Sciences & Humanities
Social Sciences & Humanities
CWI
University of Amsterdam

Impact

Output

Team

DC
Davide Ceolin
Lead Applicant
Centrum Wiskunde & Informatica (CWI)
JN
Julia Noordegraaf
Co-applicant
University of Amsterdam
Ji Qi
Ji Qi
Lead Research Software Engineer
Netherlands eScience Center
Erik Tjong Kim Sang
Lead Research Software Engineer
Netherlands eScience Center
Dafne van Kuppevelt
Dafne van Kuppevelt
Research Software Engineer
Netherlands eScience Center
Ole Mussmann
Ole Mussmann
Research Software Engineer
Netherlands eScience Center
Jisk Attema
Programme Manager
Netherlands eScience Center
Willem van Hage
Willem van Hage
Tech Lead
Netherlands eScience Center

Related projects

RAS

Review Argumentation at Scale

Updated 8 months ago
Finished

NEWSGAC

Advancing media history by transparent automatic genre classification

Updated 21 months ago
Finished

Related software

Orange3 Argument Mining Add-on

OR

This open-source Python package facilitates the processing, analysis, and visualization of argument corpora. Additionally, It offers user-friendly GUIs integrated into the Orange3 scientific workflow platform, enabling non-programmers to utilize its capabilities.

Updated 14 months ago
3

Orange3 Story Navigator

OR

Add-on providing quantitative textual story analysis based on principles in narrative psychology

Updated 6 months ago
4