U

Senior AI Ops Engineer - Application

  • USAA
  • San Antonio, United States
  • On-site
  • Full-Time
  • $114,080 - 218,030 per year
  • Posted 3 days ago
Data & AI

Salary: $114,080 - 218,030 per year

Requirements:

  • We require a bachelors degree, or four years of relevant education and/or experience.
  • We require at least five years of hands-on software or data engineering experience, strong Python skills, and proficiency in a second production programming language. Your code should be tested, reviewed, and deployed—not limited to notebooks.
  • You should have practical experience building and productionizing machine-learning models for anomaly detection, classification, or forecasting using frameworks such as TensorFlow, PyTorch, or scikit-learn, including managing the full model lifecycle through retraining.
  • You should be familiar with LLM techniques such as prompt design, retrieval-augmented generation, and structured outputs, and understand their failure modes.
  • We require experience creating and testing operational automation, such as workflow orchestration or runbook-as-code, with rollback design and approval-gated or policy-controlled execution.
  • You should be comfortable working in a regulated environment and able to diagnose and reason through live production incidents under time pressure.
  • Experience deploying production model-driven root-cause analysis or incident-triage solutions in application operations is an advantage.
  • Experience building production automation with tested guardrails, safety controls, and kill-switch capabilities is an advantage.
  • Experience developing AI evaluation and monitoring frameworks—including regression testing, model and prompt evaluation, and drift detection—is an advantage.
  • Experience with event correlation, alert grouping, and high-volume application telemetry, including enterprise AIOps platforms such as Moogsoft, BigPanda, or Dynatrace, is an advantage.
  • Experience working with model governance or model risk teams and supporting model validation with documentation such as model cards is an advantage.
  • US military experience, including experience gained through military service or as a military spouse or domestic partner, is an advantage.
  • We do not provide visa sponsorship for this role. Applicants who will need immigration support now or in the future, including H-1B, TN, or STEM OPT training plans, are not eligible to apply.

Responsibilities:

  • Build and operate high-throughput ingestion and feature pipelines that bring application logs, traces, APM data, and business-transaction events collected by the Observability & Telemetry team into the AIOps feature store.
  • Train, validate, and tune anomaly-detection and forecasting models using application and transaction signals; develop LLM-based log parsing, summarization, and root-cause-hypothesis components with explicit safeguards for hallucination risk.
  • Design and implement event-correlation and probable-cause ranking logic to reduce the volume of raw application alerts that require human attention.
  • Write, test, and version remediation automation for application-layer failures, including a tested rollback for every action and adherence to approval gates, blast-radius limits, and kill-switch controls.
  • Instrument and monitor models and automations in production, retrain models on a defined schedule, and join the on-call rotation for live actions at autonomy level 3 or higher.

Technologies:

  • AI
  • Dynatrace
  • Support
  • LLM
  • Machine Learning
  • PyTorch
  • Python
  • Security
  • TensorFlow
  • API

More:

At USAA, we work to help our members achieve financial security through competitive products, exceptional service, and trusted advice, with a focus on serving the military community and their families. Our values include honesty, integrity, loyalty, and service. We support active-duty military spouses, and some USAA roles may offer remote or hybrid flexibility in line with policy and business needs; this position, however, requires onsite work five days per week in San Antonio, Texas, and relocation assistance is unavailable. As a Senior AI Ops Engineer on our Application AI Ops team, youll build systems that connect machine learning with the services, APIs, and transaction flows customers rely on. This is a from-scratch effort in a regulated financial institution, where production systems handle real transactions and are subject to audit scrutiny. The salary range is $114,080–$218,030; actual pay may vary based on experience, market data, and location. Eligible employees may receive performance-based incentives. Our benefits include medical, dental, and vision coverage, a 401(k), pension, life insurance, parental benefits, adoption assistance, paid time off and holidays, 16 paid volunteer hours, wellness programs, career planning, and continuing education. Applications are accepted on an ongoing basis until the position is filled.

last updated 46 week of 2026

Apply on DevITjobs

You apply on DevITjobs, where this job was posted. Peter Nowak isn’t the employer and doesn’t check listings.

Get new jobs like this by email

Data & AI · United States

Free. Confirm by email first; unsubscribe anytime in one click.

Similar jobs

Senior AI Ops Engineer - Application at USAA — San Antonio, United States | Peter Nowak