CB

AI Developer - Boston

  • Cobalt Benefits Group LLC
  • Boston, United States
  • On-site
  • Full-Time
  • $120,000 - 155,000 per year
  • Posted yesterday
Data & AI

Salary: $120,000 - 155,000 per year

Requirements:

  • Bachelors degree in Computer Science, Information Systems, or a related field, or equivalent practical experience, plus at least 3 years of professional software development experience.
  • Hands-on generative AI development and evaluation experience, including RAG, context engineering, tool or function calling, and at least one production or production-pilot solution.
  • Strong Python and/or C#/.NET skills, with experience building production APIs, REST integrations, OAuth or similar authorization, and secure secrets and configuration management.
  • Ability to review, test, debug, and maintain AI-generated code; working knowledge of SQL, data modeling, and data pipelines.
  • Experience handling HIPAA-regulated information, PHI masking, and access controls.
  • Experience with AI regression testing, monitoring, model version management, release controls, cost optimization, and guarded multi-step agents.
  • Ability to document technical decisions, follow governance and change control, conduct business requirements sessions, and recommend delaying or stopping solutions that do not meet evaluation thresholds.
  • Preferred: Experience with Microsoft Copilot Studio, Power Automate, Power Platform, Azure OpenAI, and Azure AI Search.
  • Preferred: Demonstrated improvement of retrieval or extraction systems using before-and-after evaluation results.
  • Preferred: Experience with AI coding agents and Model Context Protocol (MCP).
  • Preferred: Familiarity with healthcare, third-party administration, employee benefits, claims, eligibility, plan documents, or EDI.
  • Collaborative communicator able to translate business processes into technical requirements and work across business, engineering, security, and compliance teams.
  • Analytical, detail-oriented problem solver who values measurable quality, documentation, and continuous improvement.
  • Sound judgment and accountability regarding privacy, fairness, human oversight, and responsible AI delivery.
  • Adaptable, hands-on builder comfortable choosing between low-code and custom engineering approaches.

Responsibilities:

  • Design, prototype, build, test, and deploy AI applications and agents into production, selecting suitable Copilot Studio, workflow, agent, custom-service, vendor, or non-AI approaches in line with Build-vs-Buy guidance.
  • Develop RAG solutions, including document ingestion, chunking, indexing, metadata, retrieval tuning, and grounded responses with source citations.
  • Integrate internal and third-party systems through authenticated REST and GraphQL APIs, and collaborate on pipelines for documents and structured data.
  • Work with IT Security to apply least-privilege permissions, single sign-on, secrets management, PHI/PII protections, output validation, and appropriate logging and retention controls.
  • Build evaluation harnesses using golden test sets, subject-matter expert rubrics, and production sampling; monitor accuracy, citation support and completeness, latency, and transaction costs, including results by plan type.
  • Implement telemetry for prompts, retrieval, tool calls, and outputs; maintain runbooks, hand-off documentation, cost controls, and operational monitoring.
  • Advance solutions through our AI Solution Delivery Playbook decision gates and required artifacts, including risk assessments, requirements, evaluation plans, and transition documentation.
  • Partner as appropriate with our AI Risk Manager, Compliance, Data Protection Officer, Security, and AI Ethics Committee; share reusable components, templates, and approved prompt patterns.
  • During the first 90 days, complete AI governance onboarding, ship an enhancement through Production Readiness Review, and establish a golden-test-set evaluation harness.
  • Within 6 months, release a new approved AI solution, publish reusable components adopted by another project, and enable monitoring dashboards.
  • Within 12 months, meet approved business KPIs, demonstrate reusable delivery patterns, and keep inventory, risk classification, evaluation, and monitoring records current.

Technologies:

  • AI
  • API
  • Azure
  • C#
  • CI/CD
  • Copilot
  • DevOps
  • GitHub
  • GraphQL
  • Support
  • MCP
  • Microsoft 365
  • OWASP
  • OAuth
  • Python
  • RAG
  • REST
  • SQL
  • Security
  • Office 365
  • ASP.NET
  • Cloud
  • Cosmos DB

More:

We are Cobalt Benefits Group (CBG), a healthcare third-party administrator specializing in self-funded benefit plans. With more than 30 years of experience and a team of over 300 employees, we administer benefits through four lines of business and provide customized health plans and programs including FSAs, HSAs, COBRA, and retiree billing. We are hiring a hands-on AI Developer to turn approved use cases into production-ready solutions that improve operational efficiency and member experience, working with business owners, DevOps, IT Security, Compliance, and AI governance stakeholders. Our technology environment includes Microsoft 365 Copilot, Copilot Studio, Power Automate, Power Apps, Teams, Azure AI services, Azure API Management, Entra ID, Key Vault, data storage and database services, GitHub, and CI/CD pipelines; technologies vary by project and risk tier. Our solutions follow CBG governance and technical security requirements, with privacy, security, and human oversight proportionate to risk. Automated claim denials without human review and use of PHI in public generative AI models are prohibited; work aligns with HIPAA, HITRUST CSF, the NIST AI Risk Management Framework, and applicable OWASP guidance. Eligible full-time employees can access benefits after the applicable waiting period, including medical, dental, and vision insurance; HSA contributions; life, AD&D, and disability coverage; flexible spending accounts; a 401(k) with up to a 6% employer match; paid holidays, vacation, and sick time; Summer Fridays; volunteer, tuition, and wellness reimbursements; and company events. We are an Equal Employment Opportunity employer and participate in E-Verify to confirm new hires employment eligibility.

last updated 47 week of 2026

Apply on DevITjobs

You apply on DevITjobs, where this job was posted. Peter Nowak isn’t the employer and doesn’t check listings.

Get new jobs like this by email

Data & AI · United States

Free. Confirm by email first; unsubscribe anytime in one click.

Similar jobs

AI Developer - Boston at Cobalt Benefits Group LLC — Boston, United States | Rivet Commerce job board