Labeling & Evaluation Agent (French)

About the role

A leading artificial intelligence company, recruited through David Kennedy Recruitment, is looking for a Labeling & Evaluation Agent (French) to support the development and continuous improvement of its AI models. The role evaluates AI-generated outputs such as chatbot responses, annotates them and gives feedback on quality, factual accuracy, relevance, safety and policy compliance, helping improve large language models. It is a full-time, on-site position in Lisbon.

Key responsibilities

• Evaluate AI-generated responses using detailed quality assessment and annotation guidelines.
• Label and annotate language model outputs accurately and consistently.
• Assess content for quality, relevance, factual accuracy, safety and policy compliance.
• Apply complex annotation guidelines across different scenarios and content types.
• Identify inconsistencies, edge cases and potential issues, escalating them when necessary.
• Analyse structured text, language data and AI-generated content.
• Maintain strict confidentiality and comply with project standards and data security requirements.
• Provide accurate evaluations, annotations and actionable feedback to support ongoing AI model improvement.

Required profile

• Native-level proficiency in French.
• English at B2 level or above.
• Strong understanding of Generative AI, large language models (LLMs) and AI annotation or evaluation platforms.
• Excellent analytical and critical thinking skills, with strong attention to detail.
• Ability to make objective, evidence-based decisions and to interpret and apply complex written guidelines and policies.
• Ability to meet productivity, accuracy and quality targets.
• Professionalism, integrity and discretion when handling confidential information.

What we offer

• Modern city-center office with open spaces, easily reached by public transport.
• Comprehensive initial training and guidance.
• 24/7 rotational working schedule with two rotational days off.
• Work in the growing field of Generative AI and LLMs, in an international, multicultural environment.
• Coaching and mentoring support, health and wellbeing initiatives, and a monthly meal allowance.