Labeling & Evaluation Agent (German)

David Kennedy Recruitment is working with a leading technology company specialising in artificial intelligence that is seeking to onboard a Labeling & Evaluation Agent to support the development and continuous improvement of its AI models. The role focuses on evaluating AI-generated outputs, including chatbot responses, and providing high-quality annotations and feedback. You will assess response quality, factual accuracy, relevance, safety and policy compliance, helping improve the performance and reliability of large language models (LLMs). Position: Labeling & Evaluation Agent (German) Location: Lisbon, Portugal Work model: On-site Employment type: Full-time DUTIES AND RESPONSIBILITIES: Evaluate AI-generated responses using detailed quality assessment and annotation guidelines Label and annotate language model outputs accurately and consistently Assess generated content for quality, relevance, factual accuracy, safety and policy compliance Apply complex annotation guidelines across different scenarios and content types Identify inconsistencies, edge cases and potential issues, escalating them when necessary Analyse structured text, language data and AI-generated content Maintain strict confidentiality and comply with project standards and data security requirements Provide accurate evaluations, annotations and actionable feedback to support ongoing AI model improvement REQUIREMENTS: Native-level proficiency in German English proficiency at B2 level or above Strong understanding of Generative AI, large language models (LLMs) and AI annotation or evaluation platforms Excellent analytical and critical thinking skills, with strong attention to detail Ability to make objective, evidence-based decisions Ability to interpret and consistently apply complex written guidelines and policies Strong reading comprehension and written language skills Ability to evaluate structured text, datasets and AI-generated content Ability to meet productivity, accuracy and quality targets Professionalism, integrity and discretion when handling confidential information BENEFITS : Modern office in city center with open spaces, easy to access with public transportation Comprehensive initial training and guidance to make sure you're well prepared and comfortable at a new workplace Full-time employment in Lisbon 24/7 rotational working schedule 2 rotational days off Opportunity to work in the rapidly developing field of Generative AI and Large Language Models International and multicultural working environment Support from colleagues with coaching and mentoring opportunities Health and Wellbeing initiatives Monthly meal allowance And much more!!!