LLM Evaluation Specialist (Multiple Languages)

About the role

We are seeking multilingual LLM Evaluation Specialists to help improve the next generation of artificial intelligence. You will work with a diverse team in Lisbon, reviewing, annotating and providing feedback on AI‑generated content across a range of languages.

Key responsibilities

• Evaluate AI‑generated responses according to detailed guidelines.
• Label and annotate outputs from large language models.
• Assess responses for quality, accuracy, relevance and safety.
• Identify inconsistencies, edge cases and provide actionable feedback.
• Meet productivity and quality targets while handling confidential information.

Required profile

• Fluent proficiency in at least one of the listed languages (German, Turkish, Russian, Italian, Arabic, French, English, Polish, Hebrew, Portuguese, Spanish, Hungarian) with English at minimum B2 level.
• Strong understanding of generative AI and large language models.
• Experience in AI data annotation, LLM evaluation, RLHF, supervised fine‑tuning, content labeling, trust & safety, or quality assurance.
• Excellent analytical skills, attention to detail and ability to follow complex written guidelines.
• Comfortable working in a fast‑paced, quality‑driven environment.

Required skills

• Generative AI
• Large Language Models
• AI Data Annotation
• LLM Evaluation
• RLHF (Reinforcement Learning from Human Feedback)
• Supervised Fine‑Tuning
• Content Labeling
• Trust & Safety
• Quality Assurance

What we offer

• Opportunity to work on a cutting‑edge AI project.
• International and multicultural work environment.
• Full training and ongoing support.
• Valuable experience in the rapidly growing AI industry.