Data engineer

грейд
senior
формат
гибрид
язык
Английский B2
опубликована
вчера

описание

Hybrid, 2 days per week

Hey there, think you stumbled upon this job posting by chance? We don't believe in chance at Nimber. Caught your eye? Great. Keep going... We are Nimber, and we are not just filling positions, we are building a team that is ready to shake things up. If you are ready to rewrite the rules and make a real impact, this is your moment. Join us and let's put the future where it belongs. In the hands of passionate people. But what does Nimber mean? In mathematics and game theory, they are numbers with special properties of addition and multiplication. And what we really want is to add value to customers' projects. Join a team migrating data and ETL processes from SQL Server to Google Cloud Platform, building the new cloud data platform.

задачи

  • Design and execute the migration of data and ETL processes from SQL Server to GCP;
  • Develop and optimize scripts and pipelines to transfer and transform data into BigQuery;
  • Design, develop and maintain robust ETL/ELT pipelines on GCP, replacing or rebuilding existing flows;
  • Orchestrate and schedule pipelines with Cloud Composer;
  • Develop and optimize complex SQL queries and data schemas for BigQuery (partitioning, clustering);
  • Automate migration, transformation and integration tasks with Python;
  • Ensure data quality and integrity during and after migration;
  • Collaborate with SQL Server administrators, analysts and end-users (business lines, data scientists) to understand needs and ease the transition;
  • Take part in code reviews, best practices and testing strategies;
  • Monitor, debug and resolve incidents on data pipelines, during the transition and in production;
  • Stay up to date with GCP and migration best practices, proposing improvements to the platform.

требования

  • Significant experience with GCP BigQuery: schema design, complex queries, performance optimization and cost management;
  • Solid experience with Cloud Composer: creating, scheduling, monitoring and debugging DAGs;
  • Excellent SQL, with the ability to adapt and optimize it for BigQuery;
  • In-depth Python for scripting (migration, automation, transformation) and GCP APIs;
  • Experience with on-premise to cloud data migration (SQL Server to GCP);
  • Analytical and problem-solving skills;
  • Good communication skills in English (B2 required);
  • Будет плюсом: SQL Server (architecture, T-SQL, basic administration) (highly valued); Other GCP services such as Cloud Storage and Dataflow, and migration tools such as Database Migration Service (highly valued); Power BI (highly valued); Data Warehousing concepts and data modeling, e.g. Data Vault (highly valued); Code versioning (highly valued)

условия

  • Условий в вакансии нет

навыки

компания

похожие вакансии

статьи для разработчиков