🚀 Key Responsibilities

• Lead AI model quality assurance, evaluation, and benchmarking strategy
• Design frameworks to evaluate accuracy, drift, bias, and performance under stress
• Define and track AI KPIs (precision, recall, LLM evaluation metrics)
• Build automated validation pipelines across lower environments and path-to-live
• Identify and mitigate bias, rogue outputs, and model disparities
• Develop synthetic datasets & test scenarios (including corrupt data handling)
• Establish data quality standards for fair AI outcomes
• Implement MLOps practices within CI/CD pipelines
• Collaborate closely with engineering and DevOps teams

✅ Required Experience

• 5+ years in QA / Software Engineering / Automation Testing
• 2–3 years specifically in AI/ML model testing
• Strong expertise in Python / Java and statistical analysis
• Deep understanding of AI/ML lifecycle and evaluation frameworks
• Experience with tools like LangSmith, Fiddler, InspectAI, OpenAI Evals
• Strong SQL skills and experience with large-scale or unstructured datasets / vector DBs