Senior AQA Engineer (with AI experience) (46)

Senior QA Automation Engineer

N-iX is looking for a Senior QA Automation Engineer to lead the validation and verification strategies for EDGE Group's AI transformation. You will be responsible for defining "what good looks like" for non-deterministic AI systems, ensuring that Large Language Models (LLMs) and predictive engines meet the strict reliability standards of the defense sector.

Qualifications

• 5+ years in QA Automation
• Experience testing data-driven systems, AI, or ML models
• Strong analytical skills for defining test criteria in non-deterministic systems
• Ability to collaborate with engineers, data scientists, and product teams
• Experience in regulated industries (defense, aerospace) will be a plus
• English level - at least Upper-Intermediate

Technical Requirements

• Strong experience in Python and test automation frameworks (Pytest, Selenium/Playwright, API testing)
• Experience with performance testing tools (JMeter, Locust, K6)
• Familiarity with LLM evaluation tools (e.g., DeepEval, TruLens)
• Knowledge of SQL and data validation tools
• Experience with CI/CD pipelines (GitLab) and version control (Git)
• Exposure to test reporting tools (Allure, TestRail)

Key Responsibilities

1. AI & LLM Testing

• Build automated tests to evaluate AI outputs for accuracy, consistency, and reliability
• Validate Retrieval-Augmented Generation (RAG) systems using metrics like relevance and correctness
• Create regression tests to detect changes in AI behavior (prompt drift)

2. System & Integration Testing

• Test integrations between AI systems and enterprise platforms (e.g., SAP)
• Perform API testing, including security and access control validation
• Design performance tests for latency, scalability, and system throughput

3. Quality & Governance

• Align test cases with system requirements and user needs
• Support release readiness through structured testing reviews
• Track and manage defects, ensuring traceability to data, models, or requirements

We offer*:

• Flexible working format - remote, office-based or flexible
• A competitive salary and good compensation package
• Personalized career growth
• Professional development tools (mentorship program, tech talks and trainings, centers of excellence, and more)
• Active tech communities with regular knowledge sharing
• Education reimbursement
• Memorable anniversary presents
• Corporate events and team buildings
• Other location-specific benefits

*not applicable for freelancers