Inspired during an Odissei meeting, the lead scientist of the Netherlands Environmental Assessment Agency (PBL) wanted to use active learning to screen large amounts of textual data using the open-source software ASReview. PBL initiated an independent study and implemented ASReview at the PBL (and inspired the RIVM and the Netherlands Institute for Social Research - SCP). However, the use case of such institutes differs from the typical user because questions from the Dutch government are typically less structured than a classical systematic review question. For such purposes, ASReview allows users to choose from a wide range of models. Moreover, ASReview ships with a simulation mode to mimic the AI-aided screening process and test the performance of different models. Running such models, however, takes a long time, especially if testing the performance of neural nets, which will most likely be beneficial for users like PBL.
Therefore, the proposal aims to develop a one-click-deployment option for running large-scale simulation studies in the cloud and to evaluate how much computation time can be saved. This way, users like the PBL, can quickly run a large-scale simulation study to select the best-performing model for answering research questions from the Dutch government.
Therefore, we propose to:
- Test the existing one-click deployment functionality.
- Investigate the run-time differences between locally running a simulation study versus using the deployment options.
- Add the one-click deployment option to the simulation pipeline.
