The Trustworthy AI team at the Centre for Frontier AI Research (CFAR), IAIC, is seeking motivated scientists to conduct research on the safety of generative AI (GenAI) and multi-agent systems. Working with regulatory and clinical partners, the team develops the scientific evidence, testing tools and guidance needed for the safe and responsible deployment of AI solutions in real-world.
GenAI and multi-agent systems, such as clinical chatbots built on large language models (LLMs) and vision-language models (VLMs) that interpret medical images, introduce risks that existing regulatory frameworks were not designed to address. These include open-ended outputs, hallucinations, the lack of accepted safety metrics, and vulnerability to jailbreaks, prompt injection and data leakage. You will conduct cutting-edge R&D to assess, test and mitigate these risks, and translate your findings into regulatory-grade toolkits and guidance with real-world meaningful impact.
Key responsibilities
- Design and conduct systematic red-teaming and risk assessments of GenAI and multi-agent systems, covering adversarial attacks, hallucinations, privacy leakage and performance across diverse scenarios
- Develop efficient guardrails and risk-mitigation methods, such as knowledge validation, hallucination and uncertainty detection, adversarial input detection, and automated flagging of high-risk outputs for human review
- Develop methods for continuous post-deployment monitoring, including near real-time performance tracking, drift detection and anomaly alerting
Competencies
1. Integrity
2. Critical Thinking
3. Innovativeness
4. Effective Communication
5. Working with Others
6. Achieving Outcomes
IDARE Behaviours - 2. Dream & Act Boldly
IDARE Behaviours - 4. Reach Outward, Tackle Real-World Challenges
Research areas (one or more of the following)
- Red-teaming and adversarial robustness of LLMs, VLMs and LLM-based agents
- Hallucination detection and mitigation, uncertainty quantification and calibration for medical GenAI
- Guardrails and runtime safety mechanisms for clinical conversational agents and multimodal assistants
- Robustness to distribution shift, and fairness across patient populations, clinical sites and imaging devices
- Safety evaluation metrics and benchmarks, post-deployment performance monitoring and drift detection
Requirements
- PhD degree in Computer Science, Electrical & Electronic Engineering, Computer Engineering, Biomedical Engineering or a related field
- Solid programming skills in Python and hands-on experience with deep learning frameworks such as PyTorch
- Research experience in one or more of the following: AI safety and security, adversarial machine learning, trustworthy machine learning, LLMs or VLMs, and multi-agent systems
- Good communication and teamwork skills, and the ability to work with clinicians, regulators and engineers in a multidisciplinary setting
You apply on MyCareersFuture, where this job was posted. Peter Nowak isn’t the employer and doesn’t check listings.