Data engineer for ML products

You will own and evolve our data pipelines on GCP — building new ones, hardening existing ones, improving data quality, and making clean, trustworthy data available across the organisation. You'll work end-to-end on streaming and batch pipelines, from CDC and event ingestion through transformation, serving, and the feature layer that powers our ML and AI products.

• Design and build batch and streaming pipelines on Dataflow, Pub/Sub, and Kafka feeding BigQuery, Bigtable, and ClickHouse
• Help drive the migration off Snowflake onto our GCP-native stack — and retire shadow pipelines along the way
• Own the orchestration layer in Airflow, including SLAs, retries, and data quality gates
• Model data for analytics and for ML — including feature pipelines that serve both training and low-latency online inference
• Partner with ML engineers on feature stores, drift monitoring, and retraining workflows
• Capture requirements from stakeholders and translate them into pragmatic, well-scoped data products
• Continuously improve data quality, reliability, observability, and cost efficiency
• Identify new data sources worth acquiring and integrate them cleanly

• Strong data modelling and warehouse architecture skills (dimensional modelling, event-driven, lakehouse patterns)
• Hands-on experience with GCP data services — BigQuery is a must; Pub/Sub, Dataflow, Bigtable, Cloud Composer are strong pluses
• Production experience with streaming pipelines on Dataflow/Beam, Flink, or Spark Structured Streaming, ingesting from Kafka and/or Pub/Sub
• Solid SQL and strong Python — you write production-quality code, not just notebooks
• Experience with ClickHouse or another columnar OLAP engine in production
• Workflow orchestration experience with Airflow (or Prefect/Dagster)
• Comfortable with dbt or equivalent transformation frameworks
• Working knowledge of ML in production — feature engineering, feature stores, model deployment, drift monitoring, retraining
• Docker & Kubernetes experience
• CI/CD mindset, infrastructure-as-code sensibility, and a bias for simple, observable systems
• Будет плюсом: Experience migrating off legacy warehouses (Snowflake, Redshift, Synapse) onto cloud-native stacks is a plus; Bonus: CDC tooling (Datastream, Debezium), Vertex AI / Feature Store

• 🎯You'll have the exhilarating opportunity to contribute to a product that is highly appreciated by users and the cybersecurity community at large
• 🎯 You'll experience a highly supportive and caring environment, fostering growth, flexibility, and autonomy
• 🎯 You'll embark on an exciting journey of continuous learning and problem-solving, leveling up as our organization grows
• 🎯 Most importantly, you'll have a blast at HTB 🥳 because fun is an essential ingredient in our recipe for success! Just wait until you see our global meet-ups!
• Private health care
• Paid paternity leave
• 25 annual leave days
• Free lunch & snacks at the office
• 120€ Ticket Restaurant by Edenred
• Dedicated budget for training and professional development, participation in conferences
• Full access to the Hack The Box lab offerings; so you can learn how to hack 😉
• State-of-the-art equipment (mac, iPhone, and mobile plan)