Инженер данных

Andersen is a software development company that delivers scalable software solutions and digital transformation services for organizations across various industries.

• Monitor and troubleshoot the batch pipelines day to day, checking every run, debugging failures, and validating in a test environment before production;
• Trace failures through the medallion layers, distinguish source-side changes from pipeline bugs, fix what is understandable, and validate in a test environment;
• Set up failure alerting such as sensor notifications into a teams channel showing whether a batch succeeded or failed;
• Work toward making the refresh cadence of the data more frequent through incremental-load strategies;
• Handle requests to ingest new data and add new sources independently by setting up connections and credentials, landing data in S3, deciding on the Redshift schema and target tables, and granting IAM access to downstream consumers;
• Extract data from the platform sources via Glue, load into Redshift, and perform transformations in dbt inside the warehouse;
• Provision data to ML engineers and analysts on request by creating views and materialized views and managing access;
• Review PRs and mentor junior engineers, checking naming, readability, reuse, and that added functionality does not exceed the ETL runtime budget;
• Onboard well enough to review PRs and troubleshoot in a way that matches the platform overall architecture and modern philosophy.

• Commercial experience in Data Engineering, owning end-to-end ELT/ETL pipelines and cloud data-warehouse architecture for 4+ years;
• Expert-level SQL skills;
• Production experience with Amazon Redshift, including schema design, distribution/sort keys, WLM, query tuning, and reducing refresh latency on a live warehouse;
• Strong MPP data warehouse experience, with Snowflake and Azure Synapse SQL Pool experience valued;
• Experience with incremental loads and CDC, including watermarks, SCD2, change-data-capture, and moving from full rebuilds toward near-real-time processing;
• Experience re-architecting legacy ETL pipelines and data warehouses, including migration from script-based or on-prem legacy systems to scalable cloud platforms;
• Experience using MySQL as a source system, including incremental extraction from relational OLTP sources, and MariaDB familiarity;
• Experience with AWS S3 and IAM for data landing and access management;
• Production experience with Airflow or equivalent orchestration tools and dbt;
• Ability to work fully independently as the sole owner of a platform, including identifying and prioritizing technical debt;
• Strong monitoring and failure recovery practices, including checking pipeline runs, debugging failures, and maintaining a healthy batch platform;
• English from Upper-Intermediate and above;
• Nice to have: CDC tooling in production, mentoring junior engineers, PR-review rigor.

• Experience in teamwork with leaders in FinTech, Healthcare, Retail, Telecom, and others;
• Opportunity to change the project and develop expertise in an interesting business domain;
• Guarantee of professional, financial, and career growth with mentoring and adaptation systems;
• Opportunity to earn up to an additional 1,000 EUR per month, depending on the level of expertise, included in the annual bonus;
• Access to the corporate training portal with an updated knowledge base;
• Bright corporate life including parties, pizza days, PlayStation, fruits, coffee, snacks, and movies;
• Certification compensation for AWS, PMP, and others;
• Referral program;
• Private health insurance and sports compensation, depending on the type of employment.