описание
We’re looking for someone who loves to build, learn, and guide others. You’ll be working closely with our leadership (CEO/COO) and our clients, so clear communication and a collaborative spirit are key.
задачи
- Build and operate robust ingestion pipelines (both batch and streaming) from databases, event streams, and third-party APIs.
- Own storage architecture across the lake, lakehouse, and warehouse, optimizing for partitioning, cost efficiency, and query speed.
- Keep pipelines observable and restartable by implementing orchestration, automated testing, alerting, and ensuring clean recovery when something breaks at 08:00.
- Work like a true software engineer by implementing infrastructure as code, CI/CD for data workloads, code reviews, and strict version control.
- Agree on data contracts with data-producing teams to keep schema drift under control, while managing access, retention, and lineage.
- Produce clear and comprehensive documentation that ensures your work outlives the engagement.
требования
- You have built and ran data pipelines in production—and, more importantly, you can articulate what you learned when they broke.
- You seamlessly integrate AI tooling into your daily workflow, but you maintain the strict review discipline to verify what it generates before it ships.
- You possess fluent English and are completely comfortable working directly with customer teams.
- Advanced, production-grade SQL and Python.
- Production depth in one major cloud provider (AWS, GCP, or Azure), with a solid grasp of principles to move between them.
- Production experience with an orchestrator from a main vendor (e.g., Airflow, Dagster, Azure Data Factory, or equivalent).
- Warehouse depth in Snowflake or BigQuery (dimensional or Data Vault modelling), and experience with dbt or equivalent transformation frameworks.
- Vast experience with Snowflake & Terraform.
- Будет плюсом: A second cloud platform is a plus; You know how to make a pipeline cheaper and can quantify the savings; You have exposure to regulated environments, including GDPR obligations, access control, and auditable lineage; Spark or PySpark at scale, Databricks, and streaming/pub-sub brokers like Kafka, Kinesis, Pub/Sub, or MQTT-class brokers; Embedding pipelines, vector stores, retrieval feeds, feature stores, and open table/file formats (Parquet, Iceberg, Delta); Docker, Kubernetes, GitHub Actions, and data quality frameworks in production (Great Expectations, Soda, Monte Carlo); NoSQL stores (like MongoDB) and marketing/ad-platform data (Google Ads, Meta, CRM platform APIs)
условия
- Условий в вакансии нет