Data engineer

Описания нет

• Builds, optimizes, and maintains data ingestion, transformation, and loading pipelines (ETL/ELT).
• Defines pipeline architecture, standards, layers (raw, bronze, silver, gold), and governance.
• Coordinates the technical implementation of pipelines, ensuring good practices and standards defined by the architecture.
• Focuses on technical execution, performance, and the reliability of code and jobs.
• Focuses on scalability, interoperability between systems, and data mesh design.
• Balances technical and product perspectives; acts as a bridge between the technical team and stakeholders.
• Chooses tools and technologies in the ecosystem — Azure Fabric.
• Ensures the team is trained in the chosen technologies and helps resolve technical bottlenecks.
• Implements tests and validations, and handles error management and corrupted data.
• Defines standards for data quality, cataloging, security, and access policies.
• Ensures these guidelines are followed in the team's day-to-day work.
• Automates and monitors to continuously improve performance and cost.
• Evaluates and proposes new architectural approaches as the business grows.
• Encourages innovation within the team, promotes POCs, and drives the adoption of new solutions aligned with the strategy.

• Specialist in Spark, SQL, Python, Delta Lake, and data architecture.

• Условий в вакансии нет