Labeling & Evaluation Agent (German)

About the role

A leading artificial intelligence technology company is hiring a Labeling & Evaluation Agent (German) through David Kennedy Recruitment. The role supports the development and continuous improvement of AI models by evaluating AI-generated outputs such as chatbot responses, and providing high-quality annotations and feedback. It is based in Lisbon, Portugal, on an onsite, full-time basis.

Key responsibilities

• Evaluate AI-generated responses using detailed quality assessment and annotation guidelines.
• Label and annotate language model outputs accurately and consistently.
• Assess content for quality, relevance, factual accuracy, safety and policy compliance.
• Apply complex annotation guidelines across different scenarios and content types.
• Identify inconsistencies, edge cases and potential issues, escalating them when necessary.
• Analyse structured text, language data and AI-generated content.
• Maintain strict confidentiality and comply with project standards and data security requirements.
• Provide accurate evaluations, annotations and actionable feedback to support ongoing AI model improvement.

Required profile

• Native-level proficiency in German.
• English at B2 level or above.
• Strong understanding of Generative AI, large language models (LLMs) and AI annotation or evaluation platforms.
• Excellent analytical and critical thinking skills, with strong attention to detail.
• Ability to make objective, evidence-based decisions and to interpret and apply complex written guidelines and policies.
• Ability to meet productivity, accuracy and quality targets.
• Professionalism, integrity and discretion when handling confidential information.

What we offer

• Modern city-center office with open spaces, easily reached by public transport.
• Comprehensive initial training and guidance.
• 24/7 rotational working schedule with two rotational days off.
• Work in the field of Generative AI and LLMs in an international, multicultural environment.
• Coaching and mentoring support, health and wellbeing initiatives, and monthly meal allowance.