data engineer vehicle telemetry

The project collects, processes, and analyzes vehicle telemetry data for a major European automotive manufacturer. Its Azure Databricks platform operates at significant scale across multiple regions, including Europe, North America, and China.

• Monitor the health, stability, and performance of the production data platform;
• Investigate and resolve performance issues and production incidents;
• Analyze Databricks workloads and identify performance bottlenecks;
• Optimize PySpark workloads and Databricks jobs for performance, reliability, and cost efficiency;
• Support and improve approximately 200 existing Databricks jobs and associated data pipelines;
• Prepare the platform for increasing data volumes and workloads as additional vehicle platforms are onboarded;
• Design and implement scaling strategies with a strong focus on predictable infrastructure and operational costs;
• Track and optimize Databricks and Azure resource consumption;
• Improve platform observability, monitoring, alerting, and operational processes;
• Ensure consistency across multiple regional deployments in Europe, North America, and China;
• Identify opportunities for technical improvements, automation, and reduction of operational overhead.

• 3+ Years of hands-on experience in data engineering;
• Expertise in Azure Databricks;
• Proficiency in PySpark and distributed data processing;
• Advanced Python development skills;
• Understanding of performance optimization for large-scale Spark workloads;
• Experience operating and troubleshooting production data platforms;
• Knowledge of monitoring, observability, and capacity planning combined with cloud cost optimization;
• Ability to analyze existing systems, identify bottlenecks, and propose pragmatic improvements;
• English proficiency at an Upper-Intermediate level (B2) or higher;
• Nice to have: familiarity with Microsoft Azure services and cloud infrastructure, experience building and maintaining CI/CD pipelines using GitHub Actions, exposure to large-scale telemetry, IoT, or automotive data, experience managing Databricks environments with a large number of scheduled jobs and pipelines, multi-region or geographically distributed cloud deployments.

• Remote work is available in Georgia and four other locations.