About

N/A

Company social profiles

Jobs at Nasugroupcom

Automation and AI product testing

at Nasugroupcom

Posted by Prathibha M

Bengaluru (Bangalore)

3 - 6 yrs

₹9L - ₹10L / yr

Playwright

evals

golden

Large Language Models (LLM)

AI product testing

Job Summary

We are looking for an experienced AI Automation Tester with hands-on experience in testing AI-powered products and applications. The ideal candidate should have strong expertise in Playwright automation, along with practical experience in LLM evaluations (Evals), Golden Datasets (Goldens), and AI Observability.

Key Responsibilities

Design, develop and execute automated test scripts using Playwright.
Test AI-powered applications, LLM-based products and Generative AI solutions.
Work on Evals to assess AI response accuracy, relevance, quality and consistency.
Create, maintain and validate Golden Datasets (Goldens) for benchmarking AI outputs.
Implement and work with AI Observability to monitor model performance, response quality, latency and failures.
Perform functional, regression, API and end-to-end testing.
Identify, report and track defects to ensure product quality and reliability.

Job Summary

Key Responsibilities

Design, develop and execute automated test scripts using Playwright.
Test AI-powered applications, LLM-based products and Generative AI solutions.
Work on Evals to assess AI response accuracy, relevance, quality and consistency.
Create, maintain and validate Golden Datasets (Goldens) for benchmarking AI outputs.
Implement and work with AI Observability to monitor model performance, response quality, latency and failures.
Perform functional, regression, API and end-to-end testing.
Identify, report and track defects to ensure product quality and reliability.