Data Platform Engineer

About the role

Block Labs is building a new data platform that will serve as the backbone for real‑time decisioning and predictive modelling across its products. As a Data Platform Engineer you will own end‑to‑end pipelines, a unified analytical warehouse and a low‑latency feature‑serving layer for AI agents.

Key responsibilities

• Design, build and maintain scalable data pipelines using AWS Glue (PySpark) or equivalent orchestration tools.
• Engineer and optimise the ClickHouse warehouse for sub‑second query performance.
• Implement data contracts that allow onboarding new operators via configuration changes only.
• Build a feature‑serving layer delivering pre‑computed features to AI agents at millisecond latency.
• Integrate with third‑party databases, back‑office APIs and external systems such as CRM and acquisition platforms.
• Establish monitoring, alerting and maintenance procedures, including pipeline health checks and data contract SLA enforcement.
• Own CI/CD and infrastructure‑as‑code for data workloads.

Required profile

• 3+ years of experience building and operating production data pipelines at scale, both streaming and batch.
• Strong expertise with Apache Kafka (or Amazon MSK) including topic design, consumer groups and schema registry.
• Deep SQL and columnar warehouse skills; ClickHouse preferred.
• Proficiency in PySpark / Spark Streaming for data transformation jobs.
• Experience with AWS services (Glue, S3, DynamoDB, ECS, IAM) and infrastructure‑as‑code tools (CloudFormation, Terraform, CDK).
• Data modelling discipline for multi‑tenant schemas and data contracts.
• Mindset for data quality and observability using tools like Great Expectations.

Required skills

• AWS Glue
• PySpark / Spark Streaming
• Apache Kafka / Amazon MSK
• ClickHouse (or similar columnar DB)
• SQL
• Apache Airflow or Apache NiFi
• CloudFormation, Terraform, CDK
• ECS Fargate, Kubernetes
• Great Expectations (or equivalent)