Azure data engineer for machine learning

Описания нет

• Collaborate with business and technical teams to gather requirements and define data solutions.
• Develop and maintain ETL/ELT pipelines with Azure Databricks and PySpark, using Azure Data Factory where applicable.
• Implement data ingestion, transformation, and modeling processes in lakehouse architectures.
• Prepare data for model training, including cleaning, handling missing values, and creating features (feature engineering).
• Collaborate in training, validating, and evaluating classical Machine Learning models, using metrics appropriate to the business problem.
• Implement batch inference pipelines and integrate model predictions into data workflows.
• Monitor data quality and model performance in production, applying best practices for versioning, testing, and documentation.

• 2+ years of experience in Data Engineering, including hands-on participation in classical Machine Learning projects.
• Solid experience with Azure Databricks and good knowledge of Python, SQL, and PySpark.
• Experience with Azure Data Lake Storage, Parquet, and Delta Lake.
• Experience with the complete development lifecycle of data solutions, from ingestion to delivery.
• Experience with Machine Learning libraries and knowledge of data preparation, model training, validation, and evaluation.
• Understanding of concepts such as overfitting, data leakage, separation of training and test data, and cross-validation.
• Experience integrating models into processing pipelines and batch inference.
• Experience with Git and the ability to develop scalable solutions, taking performance and execution costs into account.

• Условий в вакансии нет