QA Engineer – AI Automation Validation

Description

• Develop comprehensive test strategies covering functional, performance, and regression testing for AI agents and automation workflows.
• Build and maintain automated test frameworks and scripts to ensure consistent bot responses.
• Create and curate test assets such as golden conversation sets, synthetic data, evaluation rubrics, and regression suites.
• Validate AI agent safety by testing tool‑calling, parameter mapping, error handling, retries, and edge cases.
• Test integration integrity, including API contracts, data transformations, idempotency, duplicate prevention, partial failures, and latency.
• Identify, log, prioritize, and track defects while coordinating reproductions and fixes with developers.
• Deliver final release certification by confirming acceptance criteria and quality standards are met.
• Collaborate with Business Analysts to define clear, testable acceptance criteria and ensure test coverage maps to requirements.
• Maintain test documentation, results, and quality metrics, communicating outcomes and risks to stakeholders.
• Participate in Agile ceremonies (planning, grooming, stand‑ups, retrospectives) to embed quality throughout delivery.
• Drive continuous improvement of automation coverage, testing effectiveness, and overall quality practices.

Requirements

IEM is looking for a detail‑oriented QA Engineer / Automation Specialist to ensure the quality, reliability and safety of its AI agents and automation workflows. The role involves creating test strategies, building automated frameworks, and validating AI outputs before they are released to production. Collaboration with developers, business analysts and project managers is essential to identify edge cases, track defects and provide final quality sign‑off.

The ideal candidate combines strong analytical and critical‑thinking skills with hands‑on experience designing test scenarios and diagnosing complex issues. They act as a proactive quality advocate, owning end‑to‑end validation of AI agents and automation workflows while producing clear documentation such as test plans, cases and quality reports. Effective communication with both technical and non‑technical partners and a continuous‑learning mindset are also required.

Joining IEM means working with engineers, makers and problem‑solvers on some of the world’s most ambitious projects. The culture values collaboration, impact‑driven work and opportunities for personal growth. Comprehensive benefits and a supportive environment are offered to promote long‑term success.

• Minimum 3 years of experience in QA, test automation, or software testing with proven test strategy execution.
• Hands‑on expertise building automated frameworks using tools such as Selenium, Playwright, Cypress, Jest, Pytest, or similar.
• Proficiency in at least one programming or scripting language (e.g., Python, JavaScript, Java, C#) for test development.
• Direct experience working in Agile/Scrum environments with active sprint and CI involvement.
• Familiarity with defect and test management tools like Jira, Azure DevOps, TestRail, or equivalents.
• Bachelor’s degree preferred (not mandatory).

Benefits

The position is fully remote, with up to 15 % travel for visits to IEM facilities, supplier sites, compliance audits or conferences. Future business needs may require the employee to work from the Austin office.

Training + Development

Interview process

Visa Sponsorship

Security clearance