PhD on Developing a User-centric AI evaluation framework for LLMs

2 maanden geleden


Eindhoven, Nederland Eindhoven University of Technology Voltijd

Job description

Do you want to develop methods to better evaluate, understand and improve human-AI interaction in the context of value-aligned Large Language models? Do you want to conduct impactful user research in close collaboration with scientists and engineers in a EU-funded doctoral network, helping to promote fair and transparent generative AI?

We are seeking a highly motivated doctoral candidate to join our research team as part of a prestigious Marie Skłodowska-Curie Actions (MSCA) doctoral network: AlignAI. The AlignAI doctoral network aims to train doctoral candidates to develop, evaluate, and engage with Large Language Models (LLMs). It focuses on aligning these models with human values to ensure their development and deployment are ethically sound and socially beneficial. By integrating expertise from social sciences, humanities, and technical disciplines, the project will address critical issues such as explainability and fairness, thereby ensuring LLMs contribute positively to education, mental health, and news consumption.

The doctoral candidate in this position will develop and test a user-centric AI evaluation framework that triangulates subjective metrics such as perceived transparency, understandability, fairness and cognitive explainability (mental models), with objective measures of explanation and model quality. Quantitative and qualitative user studies will be developed iteratively throughout the project within the three use cases of the AlignAI consortium: Education, positive health and online news consumption. The candidate will support the validation of AI tools developed by other candidates within alignAI consortium and final validation step with user groups towards the end of the project.

The doctoral candidate will be supervised by associate Professor Martijn Willemsen and embedded within the Human-Technology Interaction group, Department of Industrial Engineering and Innovation Sciences, Eindhoven University of Technology (TU/e). Additional supervision will be provided within the AlignAI doctoral network by Assistant Professor Sneha Das (Computer Science, DTU) and Professor Daniel Gatica- Perez (Digital Humanities,EPFL). The candidate will be affiliated with the Eindhoven Artificial Intelligent Systems Institute (EAISI), and will be expected to contribute actively and positively to Human-AI interaction research communities in Eindhoven, as well as to attend workshops, summer schools, and research visits (secondments) at AlignAI partner institutions as part of the MSCA doctoral network.

Key Responsibilities

Develop and test a user-centric AI evaluation framework under supervision of Dr. Martijn Willemsen Collaborate with other doctoral candidates in the consortium on validation of the framework by means of several user studies performed within the three use cases Engage in secondments at DTU and EPFL as part of the project’s training activities. Contribute to academic publications, project reports, and other dissemination activities. Participate in structured training programs and network-wide events organized by the consortium. Collaborate with international partners and other Doctoral Candidates within the network.

This position is one of four PhD positions being offered at TU/e through the AlignAI doctoral network, the others being in Philosophy & Ethics (Carlos Zednik) and Industrial Design (Jesse Benjamin & Stephan Wensween).

Job requirements

A master’s degree (or an equivalent university degree) in Human-computer Interaction, Human-Technology Interaction, Computer Science, Cognitive Science, Artificial Intelligence or related disciplines Candidate is expected to have a multidisciplinary background that covers user aspects / psychology, research methods and engineering skills in the field of AI A research-oriented attitude. Applicants must not own a doctoral degree at the date of recruitment. Applicants may be of any nationality, but must not have resided or carried out their main activity (work, studies, etc.) in the Netherlands for more than 12 months in the 36 months immediately before the recruitment date. Ability to work in an interdisciplinary team and interested in collaborating with industrial partners. Motivated to develop your teaching skills and coach students. Fluent in spoken and written English (C1 level).

Selection Process

Application Submission : Applications must be submitted by December 1, 2024. Selection Criteria : Candidates will be evaluated based on their academic background, research experience, and alignment with the project’s objectives. The selection process will be open, transparent, merit-based, impartial, and equitable. Interview : Shortlisted candidates will be invited for a structured interview with predefined questions and a scoring system. Reserve List : A reserve list of top candidates may be created for future opportunities. Selection Committee : The selection committee will comprise diverse experts ensuring gender balance, nationality diversity, and relevant experience.

Conditions of employment

A meaningful job in a dynamic and ambitious university, in an interdisciplinary setting and within an international network. You will work on a beautiful, green campus within walking distance of the central train station. In addition, we offer you:

Full-time employment for four years, with an intermediate evaluation (go/no-go) after nine months. You will spend 10% of your employment on teaching tasks. Salary and benefits (such as a pension scheme, paid pregnancy and maternity leave, partially paid parental leave) in accordance with the Collective Labour Agreement for Dutch Universities, scale P (min. €2,872 max. €3,670). A year-end bonus of 8.3% and annual vacation pay of 8%. High-quality training programs and other support to grow into a self-aware, autonomous scientific researcher. At TU/e we challenge you to take charge of your own learning process. An excellent technical infrastructure, on-campus children's day care and sports facilities. An allowance for commuting, working from home and internet costs. A Staff Immigration Team and a tax compensation scheme (the 30% facility) for international candidates

  • Eindhoven, Nederland TU Eindhoven Voltijd

    **Job description**: - Do you want to develop methods to better evaluate, understand and improve human-AI interaction in the context of value-aligned Large Language models? Do you want to conduct impactful user research in close collaboration with scientists and engineers in a EU-funded doctoral network, helping to promote fair and transparent generative...


  • Eindhoven, Noord-Brabant, Nederland TU Eindhoven Voltijd

    About the RoleWe are seeking a highly motivated doctoral candidate to join our research team as part of a prestigious Marie Skłodowska-Curie Actions (MSCA) doctoral network: Align AI.The Align AI doctoral network aims to train doctoral candidates to develop, evaluate, and engage with Large Language Models (LLMs). It focuses on aligning these models with...

  • AI Research Scientist

    4 weken geleden


    Eindhoven, Noord-Brabant, Nederland Eindhoven University of Technology Voltijd

    We are seeking a highly motivated PhD candidate to join our research team at Eindhoven University of Technology, as part of the prestigious Marie Sk ldowska-Curie Actions (MSCA) doctoral network: AlignAI.The AlignAI doctoral network aims to train doctoral candidates to develop, evaluate, and engage with Large Language Models (LLMs), focusing on aligning...


  • Eindhoven, Nederland Eindhoven University of Technology Voltijd

    Job description Do you want to conduct philosophical research on the explainability and fairness of state-of-the-art large language models? Do you have an interdisciplinary background that spans philosophy, artificial intelligence, and cognitive science? Are you interested in conducting impactful philosophical research in close collaboration with...


  • Eindhoven, Noord-Brabant, Nederland Eindhoven University of Technology Voltijd

    Designing the Future of Education with AIWe are seeking a highly motivated PhD candidate to join our research team at Eindhoven University of Technology. As part of the AlignAI doctoral network, you will have the opportunity to conduct design-led research on the application of generative AI in design education.About the ProjectThe AlignAI doctoral network...


  • Eindhoven, Nederland Eindhoven University of Technology Voltijd

    Job description Do you want to understand and design how cutting-edge AI technologies are used in educational settings? Do you have an affinity for getting hands-on with your research topics, from field research to participatory methods to co-creating prototypes? Apply for this position for the opportunity to conduct design-led research on a major...


  • Eindhoven, Nederland Eindhoven University of Technology Voltijd

    Job description Are you an engineer who wants to develop an office environment that effortlessly provides the right lighting for the user in the most sustainable way? Are you eager to develop models that can facilitate the user’s need by a smart lighting control system? Do you want to work in an exciting multidisciplinary team with members from...


  • Eindhoven, Nederland Eindhoven University of Technology Voltijd

    Job description Do you want to understand and shape how professionals of all backgrounds can contribute to emergent futures of generative AI? Do you thrive on connecting and collaborating with a wide range of partners, putting participation and co-creation at the heart of your research work? Apply for this position for the opportunity to conduct...


  • Eindhoven, Noord-Brabant, Nederland Eindhoven University of Technology Voltijd

    About the Job">The Eindhoven University of Technology invites applications for a PhD position in the field of electromagnetic interference evaluation. As a researcher, you will be part of a team focused on developing innovative solutions to tackle the complex electromagnetic environment.">Job Description">The successful candidate will work on advancing the...


  • Eindhoven, Nederland Eindhoven University of Technology Voltijd

    Job description We are seeking an enthusiastic PhD candidate to develop novel ideas to establish trust in deep learning models. Trustworthy AI is a major topic in machine learning, which is illustrated by the increasing number of initiatives to enforce AI systems to be more trustworthy. Although machine learning models typically perform well on input...


  • Eindhoven, Nederland Eindhoven University of Technology Voltijd

    Job description Are you eager to contribute to innovative research at the forefront of AI in Power Electronics? Are you fascinated by the potential of AI to support Power Electronics design for reliability, and willing to take on the challenge of multidisciplinary research to advance AI methods for reliability modelling of PECs? The overarching goal of...


  • Eindhoven, Nederland Eindhoven University of Technology Voltijd

    Job description We are seeking to fill a PhD position in the Data and Artificial Intelligence Cluster, within the Faculty of Mathematics and Computer Science at Eindhoven University of Technology (TU/e). You will join a vibrant and diverse team of 15 faculty members and over 40 PhD students and postdocs who are advancing the science and engineering of...


  • Eindhoven, Noord-Brabant, Nederland Eindhoven University of Technology Voltijd

    Job Description:We are seeking an enthusiastic PhD candidate to develop novel ideas to establish trust in deep learning models. Trustworthy AI is a major topic in machine learning, which is illustrated by the increasing number of initiatives to enforce AI systems to be more trustworthy.Machine learning models typically perform well on input data they are...


  • Eindhoven, Nederland TU Eindhoven Voltijd

    **Job description**: **Research challenges** You will be working within the newly granted NWA ORC (Research along Routes by Consortia) “NL-ECO” project. See here for more information: Big consortium starts research into energy-efficient information technology (utwente.nl). Within this extensive project framework, we will focus on combining mapping...


  • Eindhoven, Nederland TU Eindhoven Voltijd

    **Job description**: Are you passionate about exploring the intersection of human cognition and technology? Are you interested in gaining expertise in AI, particularly in the field of _Deep Learning_? Do you have the drive to tackle real-world challenges in assistive technology (e.g., products, equipment, and systems aimed at enhancing learning, working, and...


  • Eindhoven, Noord-Brabant, Nederland Snap Voltijd

    Snap, a pioneering technology company, revolutionizes the way people live and communicate through its innovative products. By empowering users to express themselves, learn about the world, and have fun together, Snap contributes to human progress.The company's core products are , a visual messaging app that enhances relationships, , an augmented reality...


  • Eindhoven, Nederland TU Eindhoven Voltijd

    **Job description**: We challenge you to develop an adaptive lighting system that provides the right light at the right time according to the individual user’s need in the context of an office environment. We envision a simulation framework that accurately predicts the light (daylight and electric light) distribution on relevant surfaces within a space,...


  • Eindhoven, Nederland TU Eindhoven Voltijd

    **Job description**: Are you eager to contribute to innovative research at the forefront of AI in Power Electronics? Are you fascinated by the potential of AI to support Power Electronics design for reliability, and willing to take on the challenge of multidisciplinary research to advance AI methods for reliability modelling of PECs? The overarching goal of...


  • Eindhoven, Noord-Brabant, Nederland Thermo Fisher Scientific Voltijd

    Job Title: Senior AI Research Engineer for Generative ModelsAbout the Role:We are seeking an experienced Senior AI Research Engineer to join our team at Thermo Fisher Scientific. As a key member of our AI research group, you will be responsible for developing and proposing innovative proof-of-concepts for application of generative AI in customer products or...


  • Eindhoven, Nederland TU Eindhoven Voltijd

    **Job description**: Are you fascinated by signal processing and artificial intelligence (AI) for healthcare? Are you interested in developing smart algorithms for new sleep monitoring technologies and exploring how AI can be used to go from sensor-specific to sensor-agnostic models? Join us to work on reliable and sensor-agnostic algorithms for long-term...