Salary: $123,000 - 163,000 per year
Requirements:
- Bachelors degree; education or certifications in Interaction Design, HCI, Psychology, Computer Science, prompt engineering, or a related field are preferred.
- 8 years of design experience, including at least 4 years in conversation design and 1 year working on AI agentic platforms.
- Expertise in data analysis, prompt engineering, and AI tools for producing deliverables such as golden datasets and trends in user responses.
- Knowledge of language modeling and machine learning frameworks, such as NLUs, RAG databases, TensorFlow, PyTorch, or similar technologies.
- Working knowledge of code, including HTML, Python, XML, and regex, to create dynamic dialogue flows using Boolean logic and decision frameworks.
- Exceptional written and verbal communication skills.
- Strong leadership abilities, with a demonstrated record of managing stakeholders, mentoring team members, and developing talent.
- Experience conducting user research and usability testing, designing experiments, and analyzing data.
Responsibilities:
- Identify and prioritize ways to improve customer experiences and achieve business goals in conversational products such as chatbots and voice assistants.
- Create end-to-end conversation principles, design strategies, and playbooks, and align stakeholders across engineering, data science, and other teams.
- Build AI skills using Claude Code or similar tools to generate golden datasets for AI evaluations.
- Evaluate AI answers at scale through offline and online testing, using LLM-as-a-judge models and human review.
- Lead multiple projects, meeting deadlines and maintaining quality and business metrics.
Technologies:
- AI
- Claude Code
- LLM
- Machine Learning
- PyTorch
- Python
- RAG
- TensorFlow
- XML
- LESS
- UX UI Design
More:
We are seeking an Information Designer II to join the AI Evaluations team in Mountain View, California, for a Kforce client. In this conversation design role, you will help build golden datasets, analyze AI responses at scale, and provide insights into quality and accuracy. Compensation depends on factors such as education, qualifications, certifications, experience, skills, seniority, location, performance, union contract, and business needs; the stated pay range may change, and this position is not eligible for bonuses, incentives, or commissions. We offer eligible employees medical, dental, and vision insurance, HSA and FSA options, a 401(k), and life, disability, and accidental death and dismemberment insurance. Salaried employees receive paid time off; hourly employees generally do not, except where required by law, and hourly employees on Service Contract Act projects are eligible for paid sick leave. We are an Equal Opportunity/Affirmative Action Employer.
last updated 46 week of 2026
You apply on DevITjobs, where this job was posted. Peter Nowak isn’t the employer and doesn’t check listings.
