Retail SME - AI Evaluation Expert

About the job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Retail Specialist Type: Contract Compensation: $60–$80/hour Location: Remote Role Responsibilities Guide research and engineering teams to close knowledge gaps in retail merchandising , category management , and operations reasoning . Design challenging, domain-relevant retail tasks and write accurate, well-reasoned solutions grounded in real retail practice. Evaluate AI model outputs against structured rubrics and provide clear, written feedback on correctness, judgment, and reasoning quality. Develop and refine evaluation guidelines and scoring rubrics specific to retail tasks. Collaborate with other subject matter experts to ensure consistency and accuracy in training data . Qualifications Must-Have 8+ years of dedicated professional experience in retail (e.g., merchandising, category management, retail operations, buying/planning) at a recognized, top-tier organization (e.g., Amazon, Walmart, Target, Nike, Costco, Home Depot, or equivalent). Prior hands-on experience evaluating LLM/AI model outputs against rubrics or structured scoring criteria. Demonstrable career progression (e.g., Category Manager → Senior Manager → Director of Merchandising). Ability to engage reliably for at least 35 hours/week during weekdays. Verbal and written communication skills, problem-solving skills, and interpersonal skills. Application Process (Takes 20–30 mins to complete) Upload resume AI interview based on your resume Submit form Resources & Support For details about the interview process and platform information, please check: https://talent.docs.mercor.com/welcome For any help or support, reach out to: support@mercor.com PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.