Using Natural Language processing for faster Data Harmonization and easier Data discoverability
Harmony is a data tool for retrospective harmonisation of questionnaire items, which we developed as part of Wellcome’s Mental Health Data Prize (https://harmonydata.ac.uk/app/#/). Harmony is a free-to-use online tool that helps researchers to harmonize data from different studies. Harmony screens meta-data (wor, csv or pdf format) from different studies, and uses natural language processing to identify variables that are comparable across the datasets based on their semantic content, and assigns an empirical ‘similarity score’ to pairs of variables. Harmony was originally developed with mental health questionnaires in mind, where it is common for different research teams to use different questionnaires, even when the same underlying constructs (e.g. depression) are being assessed. However, Harmony can be used to match any content that is semantically similar, even across multiple languages. You can find out more about Harmony, how to use it, and how it was developed on our website (https://harmonydata.ac.uk/)
Making harmonisation simple. Social scientists often have to compare items from different questionnaires or datasets. Harmony is a tool that uses natural language processing and generative AI models to help researchers harmonise questionnaire items quickly, even in different languages.