Инженер данных

Vantage Point Global is a talent consultancy established in 2014 that supports top-tier clients in building talent pipelines and delivering a future-ready workforce. The organization provides career development, formal training, and mentoring to create a skilled workforce capable of adding value to clients across various industries globally.

• Architect, develop, and maintain high-throughput data pipelines in Databricks and AWS;
• Ingest, normalize, and enrich large volumes of structured and unstructured data;
• Collaborate with AI Engineers, ML scientists, and software teams to translate requirements into scalable data architectures, schemas, and APIs;
• Optimize pipeline performance and cost using distributed processing techniques and AWS best practices;
• Enforce data governance, privacy, and lineage standards, cataloguing assets in Unity Catalog and managing PII/PCI classification;
• Build automated validation, testing, and monitoring frameworks to ensure data quality and freshness;
• Support onboarding and integration of new external data vendors;
• Evaluate emerging GenAI tooling and drive proof-of-concepts;
• Own the creation of tools and workflows for web crawling and scraping;
• Test and validate scraped data for accuracy, quality, and compliance;
• Identify and resolve issues with scrapes and scale processes.

• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field;
• 5+ Years of experience in data engineering or a related role;
• Strong experience in Python and SQL;
• Experience with Spark or Scala and distributed data processing;
• Proficiency in building scalable, distributed data pipelines in a cloud environment;
• Familiarity with Linux/UNIX, HTTP, HTML, JavaScript, and networking concepts;
• Knowledge of web scraping tools and libraries;
• Working knowledge of version control systems and open-source practices;
• Solid understanding of data architecture principles, data modeling, and data warehousing;
• Excellent analytical and problem-solving skills;
• English (written and spoken);
• Nice to have: Experience extracting text from PDFs, images, and applications, familiarity with system monitoring/administration tools, knowledge of graph databases, prior experience analysing big data sets.

• No conditions specified