Lead Cloud Engineer – Build Scalable Cloud Data Solutions

About the role

Interested in building the next generation cloud data ecosystem that processes petabytes of data and powers AI‑driven insights for some of the world’s largest brands? The Marketing Services Technology team is transforming its analytics and data platform from traditional on‑premises Hadoop environments to a modern cloud‑native architecture powered by AWS, Databricks, Apache Spark, Iceberg, and AI technologies.

Key responsibilities

• Design, develop, and support large‑scale cloud‑native data solutions on AWS and Databricks.
• Build and maintain batch, streaming, and real‑time data pipelines for analytics, reporting, machine learning, and AI workloads.
• Engineer scalable distributed data processing systems using Apache Spark and related Big Data technologies.
• Implement modern Lakehouse architectures leveraging Databricks, Delta Lake, and Apache Iceberg.
• Lead migration of data workloads from Hadoop‑based platforms to cloud‑native architectures.
• Develop reusable frameworks and engineering patterns to improve scalability, reliability, and operational efficiency.

Required profile

• Strong hands‑on experience with Hadoop, HDFS, Hive, Spark, and related distributed computing frameworks.
• Deep expertise in cloud data engineering on AWS, including storage, compute, networking, security, and scalable data architectures.
• Proficiency in Python, Java, or Scala for building large‑scale distributed data processing applications.
• Experience building robust batch and streaming pipelines supporting high‑volume and high‑velocity workloads.
• Understanding of modern Lakehouse principles and open‑table formats such as Apache Iceberg and Delta Lake.
• Familiarity with enterprise‑scale data governance, security, privacy, and compliance.
• Experience with Infrastructure‑as‑Code (e.g., Terraform), CI/CD pipelines, and cloud‑native monitoring and observability.

Required skills

• AWS
• Databricks
• Apache Spark
• Hadoop ecosystem (HDFS, Hive)
• Python
• Java
• Scala
• Terraform
• CI/CD
• Apache Iceberg
• Delta Lake