PhD Position: Understanding Human Biases During AI-aided Systematic Reviewing

Faculty or Service: Faculty of Social SciencesDepartment: Department of Social SciencesHours per week: 36 to 40Deadline for response: June 1, 2024

Do you want to understand how and why humans interact with an AI to accelerate systematic reviews? If so, apply for this PhD position This position is part of the bigger project “Transparent and Reproducible AI-aided Systematic Reviewing for the Social Sciences (TRASS)”.

The rapidly evolving field of AI offers promising solutions to the literature screening challenge using machine learning models, like active learning, and, very recently, large language models (LLMs). However, many of these AI-driven solutions emerge from tech companies that publish new and, hopefully, better models at an unprecedented rate. The rapidity of advancements in the field of AI outpaces meticulous scientific evaluations, leaving many methods unrefined and unproven. The challenge is twofold: keeping pace with these relentless innovations while collaboratively forging a comprehensive understanding of their implications.

In the evolving landscape of AI-aided systematic tools (like ASReview), we need to explore the nuanced complexities and potential biases that arise when human discernment is coupled with the predictive capabilities of AI, aiming to foster a synergistic relationship that enhances the efficacy and reliability of the review process.

Therefore, in the subproject for which we are looking a PhD candidate, we will investigate questions like: - Which biases - such as position bias or authority bias - do humans display when reviewing abstracts? - How does knowledge about these biases generalize to other contexts, such as in doing research and university work generally.- Should we replace human screeners with an AI (as predicted by OpenAI), or should we keep a human in the loop? - How do (or should) humans interact with AI-aided screening models spotlighting potential biases inherent in AI-assisted processes? - How can we reduce biases by humans and AI?

You will be responsible for conducting several high-quality experiments where humans interact with an AI to address such questions. Addressing these questions is crucial in fostering a collaborative synergy between humans and AI models, and to spotlight and reduce potential biases, fostering a more transparent and equitable research process. Moreover, it seeks to delineate the roles and responsibilities of human researchers in an AI-assisted landscape, paving the way for a harmonious and productive human-AI collaboration in academic research.

What do you bring with you?

We are looking for an enthusiastic colleague who meets multiple of the following requirements:- You have (completed or almost completed) a Research Master’s degree in Psychology, Sociology, Methods & Statistics, Organisation Science, Economics, Management, Public Administration, or a similar study.- You have the ambition to conduct excellent scientific research and are motivated to create an impact with your research.- You have an affinity with transdisciplinary research.- You have demonstrable affinity and experience with quantitative research such as (field/lab) experiments and/or questionnaires.- You need an interest in systematic reviewing and AI-aided techniques, or have used software like ASReview, to accelerate the screening process.- You have an affinity with or strong interest in machine learning / AI / NLP.- You have a demonstrable affinity with open science, open-code and/or open-source software.- You have experience with R (Markdown) or Python (Jupyter Notebooks).- You are familiar with Github.- You have excellent verbal and written communication skills in English (or Dutch), which you can prove with for instance your Bachelor's or Master's thesis.- You are social, well-organised, communicative, and collaborative.- You are interested in contributing to high-quality teaching, including supervising students, teaching lectures in data science and statistics.

What do we offer?

We offer:- a collaborative, enthusiastic, and dedicated team that guids you in your research, teaching, and other tasks;- high-quality PhD education, being part of a research school, following courses, presenting at international conferences, etc.;- a position for 4-4,5 years (depending on the amount of teaching);- a working week of 38 hours and a gross monthly salary between €2,770 and €3,539 in the case of full-time employment (salary scale P under the Collective Labour Agreement for Dutch Universities (CAO NU)); - 8% holiday pay and 8.3% year-end bonus; - a pension scheme, partially paid parental leave and flexible terms of employment based on the CAO NU.

In addition to the laid down in the CAO NU, Utrecht University has a number of schemes and facilities of its own for employees. This includes schemes facilitating , leave schemes and schemes for , as well as discounts on software and other IT products. We also offer access to additional employee benefits through our Terms of Employment Options Model. In this way, we encourage our employees to continue to invest in their growth. For more information, please visit .

About us

A better future for everyone. This ambition motivates our scientists in executing their leading research and inspiring teaching. At , the various disciplines collaborate intensively towards major . Our focus is on Dynamics of Youth, Institutions for Open Societies, Life Sciences and Pathways to Sustainability. .

The is one of the leading faculties in Europe providing research and academic teaching in cultural anthropology, educational sciences, interdisciplinary social science, pedagogical sciences, psychology, and sociology. Almost 7,000 students are enrolled in a broad range of and programmes. The Faculty of Social and Behavioural Sciences has some 1,100 faculty and staff members, all providing their individual contribution to the training and education of young talent and to the research into and finding solutions for scientific and societal issues. The faculty is located at near the historical city centre of Utrecht.

The current project will be part of the DISC-AI-lab (), one of the official at Utrecht University. You will be officially appointed at the Methodology and Statistics Department of the Faculty of Social and Behavioural Sciences and will work one day per week in the AI-lab with Rens van de Schoot. The other days you work at the Utrecht School of Governance and will be part of the Public Management Research group, where Lars Tummers and Henrico van Roekel are also situated. You can work 1-2 days a week from home.

More information

For more information, please contact at k.vankats@uu.nl.

Please note that international candidates that need a visa/work permit for the Netherlands require at least four months processing time after selection and acceptance. Our (ISD) can answer your questions about living in the Netherlands as . Finding appropriate housing in or near Utrecht is your own responsibility, but the ISD may be able to advise you therewith. In case of general questions about working and living in The Netherlands, please consult the .

Candidates for this vacancy will be recruited by Utrecht University.

Solicitation

As Utrecht University, we want to be a for everyone. We value staff with diverse backgrounds, perspectives and identities, including cultural, religious or ethnic background, gender, sexual orientation, disability or age. We strive to create a safe and inclusive environment in which everyone can flourish and contribute.

To apply, please send your curriculum vitae, including a letter of motivation, via .

The first round of interviews takes place in June. An assessment might be part of the procedure. Our preferred start date is 1 September 2024.

Respond no later than June 1, 2024.