AI Data Engineer – Distributed Cloud

About the role

Design and maintain scalable data solutions that support AI applications and advanced analytics. Build and optimize semantic data models, ensuring high-quality, trusted metrics for business and AI consumption.

Key responsibilities

• Design and maintain scalable data solutions for AI and advanced analytics.
• Build and optimize semantic data models ensuring high-quality metrics.
• Develop integrations between data platforms, semantic layers, and AI‑powered solutions.
• Collaborate with cross‑functional teams to translate business needs into reliable data products.
• Identify opportunities to improve data quality, governance, and semantic consistency.
• Apply causal inference and advanced analytical techniques to generate business insights.
• Support AI initiatives through LLM integration, prompt engineering, and Retrieval‑Augmented Generation (RAG).
• Provide technical leadership via mentoring, code reviews, architectural guidance, and best practices.
• Partner with Data Engineers, Analytics Engineers, Product Managers, and AI teams to deliver scalable solutions.
• Drive innovation by promoting modern data engineering, AI, and analytics best practices.

Required profile

• Bachelor’s or Master’s degree in Computer Science, Software Engineering, Industrial Engineering, Mathematics, or a related technical field.
• 5+ years of experience in senior Data Engineering, AI Engineering, Analytics Engineering, or related data‑focused roles.
• Strong hands‑on experience with modern data platforms such as Databricks (Delta Lake, Unity Catalog, MLflow, Databricks Apps) and Google Cloud Platform services (BigQuery, Cloud Run, Pub/Sub, Vertex AI).
• Solid expertise in Python for advanced data engineering, API development (FastAPI), and integration with LLM frameworks (LangChain, Anthropic).
• Experience building and leveraging semantic layers and metric definitions for advanced analytics and AI applications.
• Strong understanding of causal inference techniques (DiD, RDD, IV, Causal ML frameworks).
• Experience with Generative AI solutions, prompt engineering, Retrieval‑Augmented Generation, and evaluating LLM outputs.
• Good understanding of data governance, identity and access management (IAM), and secure data architecture principles.
• Proven ability to lead technical initiatives, mentor engineering teams, and influence architectural decisions.
• Excellent communication, problem‑solving, and collaboration skills.

Required skills

• Python
• FastAPI
• Databricks (Delta Lake, Unity Catalog, MLflow)
• Google Cloud Platform (BigQuery, Cloud Run, Pub/Sub, Vertex AI)
• SQL
• LangChain
• Anthropic
• Causal inference techniques (DiD, RDD, IV, Causal ML)
• Generative AI / LLM integration
• Data governance & IAM