Skip to Jobs

AI/ML QA Engineer | LLM Testing, Python & Automation – Tarryrise Home & Staff Services – Maharashtra, India

Active
Tarryrise Home & Staff Services

At a glance

Maharashtra, India
Full-time
3–6 Years (1–2 years in AI/ML Quality Assurance preferred)
₹8–40 LPA
Posted 1 days ago

Editorial summary

Tarryrise Home & Staff Services is hiring an AI/ML QA Engineer focused on quality assurance for machine learning systems, large language model (LLM) applications, retrieval-augmented generation (RAG) pipelines, recommendation systems, and AI-powered products. Unlike traditional QA positions focused primarily on UI regression testing, this role requires evaluating model quality, response accuracy, hallucination risk, retrieval effectiveness, data pipeline reliability, and AI system behavior. Candidates are expected to combine Python-based automation skills with modern AI evaluation techniques using tools such as RAGAS, DeepEval, Promptfoo, or LangSmith. The position may appeal to QA professionals interested in transitioning from traditional automation testing into the rapidly growing AI Quality Engineering domain. Employer verification is recommended before proceeding with the application process.

Automation-heavy

Key skills

Tools

pytestSeleniumPostman

Languages

Python

Testing focus

API TestingUI Testing

Also listed

Ai testingLlm testingRag evaluationPrompt engineeringDeepevalPromptfooLangsmithRagasData quality testingGreat expectationsDbtMachine learning lifecycleAutomation testingCi/cd

This job is posted by Tarryrise Home & Staff Services. To apply, visit their official application page.

Apply on Tarryrise Home & Staff Services Career Page

Company snapshot

Public information regarding Tarryrise Home & Staff Services remains limited beyond available job-board references and employer profile listings. Candidates are encouraged to verify company registration details, official website information, office location, and hiring process legitimacy before sharing sensitive information or proceeding with interview stages. From an editorial perspective, the listing should be treated cautiously until additional company verification becomes available.

Role overview

The AI/ML QA Engineer will design, execute, and automate quality validation processes for AI-powered applications, machine learning systems, and LLM-driven products. Responsibilities include testing model outputs, evaluating retrieval quality, building automated validation frameworks, monitoring production model behavior, validating data pipelines, and collaborating closely with data scientists and machine learning engineers. The role combines automation engineering, data quality validation, AI evaluation, and quality governance within modern AI product environments.

Why apply for this role?

Tarryrise Home & Staff Services is recruiting for AI/ML QA Engineer | LLM Testing, Python & Automation in Maharashtra, India. You would work with Python, Pytest, AI Testing and related tools—skills that stay in demand across product teams. It can be a strong fit if you want clearer scope and visibility while staying close to shipping work. ITJobNotify links to the employer’s official application—you are not applying through us.

Key Responsibilities

  • •Design quality assurance strategies for AI and ML systems
  • •Develop automated validation frameworks for model evaluation
  • •Test LLM applications, chatbots, and AI-powered workflows
  • •Evaluate model outputs for accuracy, consistency, and relevance
  • •Perform hallucination and bias validation testing
  • •Validate RAG pipelines and retrieval quality
  • •Build and maintain evaluation datasets and benchmark suites
  • •Monitor model drift and production quality degradation
  • •Validate data pipelines and feature-store quality
  • •Collaborate with data scientists and ML engineers
  • •Perform API testing and automation using Python
  • •Document AI-specific defects and quality findings
  • •Support responsible AI and compliance initiatives

Required Skills

Normalized from the listing (synonyms merged, duplicates removed):

PythonpytestAi testingLlm testingRag evaluationPrompt engineeringDeepevalPromptfooLangsmithRagasSeleniumPostmanAPI TestingData quality testingGreat expectationsDbtMachine learning lifecycleAutomation testingCi/cd

Insider interview tips & role FAQ

Role-specific questions and preparation ideas—always verify details with the employer. You apply on their official career page, not through ITJobNotify.

Candidates should prepare practical examples involving AI testing, model validation, LLM evaluation, and Python automation. Be ready to discuss hallucination detection, retrieval quality assessment, benchmark dataset creation, API testing, prompt evaluation, and data quality validation techniques. Understanding AI evaluation tools such as LangSmith, DeepEval, Promptfoo, and RAGAS may provide a significant advantage during technical discussions.

Common interview questions for this role:

  • How do you test an LLM-powered chatbot?

    By validating response relevance, factual accuracy, consistency, latency, safety behavior, retrieval quality, and regression performance using benchmark datasets and automated evaluation frameworks.

  • What is hallucination testing?

    Hallucination testing evaluates whether an AI model generates incorrect, fabricated, or unsupported information and measures how often such outputs occur under controlled scenarios.

  • Why is data quality important in AI systems?

    Poor-quality data directly impacts model behavior, predictions, recommendations, and output accuracy. Reliable data validation is essential for trustworthy AI systems.

Job Description

Tarryrise Home & Staff Services is looking for an AI/ML QA Engineer to support quality assurance initiatives for machine learning systems, large language model applications, and AI-powered product experiences. The role focuses on testing modern AI workflows including LLM-powered chatbots, retrieval-augmented generation systems, recommendation engines, and machine learning pipelines. Unlike traditional QA positions that primarily validate UI behavior, this role requires evaluating output quality, response consistency, factual accuracy, hallucination rates, retrieval effectiveness, and model performance. Candidates will work closely with data scientists, machine learning engineers, product teams, and software developers to define evaluation criteria, create benchmark datasets, automate quality checks, and improve AI system reliability. The position requires strong Python automation skills and familiarity with AI evaluation frameworks such as DeepEval, Promptfoo, LangSmith, or RAGAS. Additional responsibilities include testing data pipelines, validating feature quality, monitoring production model behavior, identifying bias risks, and supporting responsible AI practices where applicable. This role is ideal for automation engineers, SDETs, and QA professionals seeking to specialize in AI Quality Engineering, one of the fastest-growing areas within software testing and quality assurance.

Career tools

Salary insight

Public job-board imports referenced compensation starting around ₹25 lakh per year; however, this information could not be independently verified. Candidates should confirm compensation, benefits, variable pay structure, and employment terms directly with the employer before making decisions.

Application & Interview Guidance

  • •Before applying, candidates should verify company details, hiring legitimacy, office location, and employment terms. Apply through the official Indeed listing and carefully review screening questions related to AI/ML testing experience. Important resume keywords include Python, Pytest, LLM Testing, RAG Evaluation, AI Quality Assurance, Prompt Engineering, Data Validation, and Machine Learning Testing. Candidates should also highlight projects involving AI products, automation frameworks, or data-quality initiatives.

Career insight

AI Quality Engineering is emerging as one of the most promising specialization paths for experienced QA professionals. Engineers who develop expertise in LLM testing, model evaluation, data quality validation, and AI system reliability can differentiate themselves significantly from traditional automation-only profiles. The demand for AI-focused testing skills continues to grow across product companies, SaaS organizations, and enterprise technology teams.

Common rejection reasons

No Python automation experience No AI or ML testing exposure Weak understanding of LLM evaluation concepts No API testing background Limited analytical problem-solving examples No experience with AI validation workflows Resume focused only on manual testing Weak understanding of data quality concepts Inability to explain hallucination testing Lack of automation framework experience

How to crack this role — preparation roadmap

Week 1 — AI & LLM Fundamentals LLM basics RAG architecture Prompt engineering Hallucination concepts AI testing fundamentals Week 2 — Python & Automation Pytest framework API testing Test automation patterns Data validation Regression testing Week 3 — AI Evaluation Frameworks DeepEval Promptfoo LangSmith RAGAS Benchmark datasets Week 4 — Interview Preparation AI testing case studies Data-quality scenarios Model validation examples Communication practice Real-world AI QA discussions

ITJobNotify Hiring Insight

This role appears to target QA professionals interested in moving beyond traditional automation testing into AI system validation and model-quality evaluation. The inclusion of LLM evaluation frameworks, RAG testing, and machine learning lifecycle concepts suggests the employer is looking for candidates with emerging AI quality engineering capabilities rather than only Selenium-based automation experience.

Related Roles

AI Test Engineer Machine Learning QA Engineer LLM Evaluation Engineer Prompt Engineer AI Automation Engineer QA Automation Engineer SDET (AI Testing) Data Quality Engineer RAG Testing Specialist AI Quality Analyst

Related jobs you may like

SDET jobs (India focus)More AI/ML QA Engineer | LLM Testing, Python & Automation jobs

How to Apply

To apply for this position, visit Tarryrise Home & Staff Services's official career page using the button below.

Apply on Tarryrise Home & Staff Services Career Page

Eligibility & Requirements

  • •Bachelor's or Master's degree in Computer Science or related field
  • •3–6 years of QA experience
  • •1–2 years of AI/ML testing exposure preferred
  • •Strong Python programming skills
  • •Experience with Pytest automation
  • •Knowledge of machine learning lifecycle concepts
  • •Understanding of LLM evaluation frameworks
  • •Experience with API testing
  • •Strong analytical and problem-solving skills
  • •Ability to collaborate with data science teams
  • •Excellent communication and documentation skills

Estimated salary range

₹8–40 LPA

Indicative range for India. Verify with the employer.

Prepare for this role

Practice interview questions and coding problems, and explore career guides.

Page SEO

Slug
ai-ml-qa-engineer-llm-testing-maharashtra-india
Meta title
AI/ML QA Engineer | LLM Testing, Python & RAG Evaluation Jobs
Meta description
Explore an AI/ML QA Engineer opportunity focused on LLM testing, RAG evaluation, Python automation, data quality validation, and AI system reliability in India.

Job Summary

Company

Tarryrise Home & Staff Services

Location

Maharashtra, India

Experience

3–6 Years (1–2 years in AI/ML Quality Assurance preferred)

Job Type

Full-time

Expires

Jul 2026

Apply on Tarryrise Home & Staff Services Career Page

This job is posted by Tarryrise Home & Staff Services. To apply, visit their official application page. ITJobNotify is not responsible for the hiring process or employment decisions.

Important Disclaimer:

This job listing has been sourced from Tarryrise Home & Staff Services's public career page or posted by a verified employer. ITJobNotify is not responsible for the hiring process, application review, or employment decisions.

Always verify: Job details directly with the hiring company. Research employers independently before applying. Contact the employer directly for questions about this position.