data engineer

BSC-CNS is Spain’s leading supercomputing center. It houses MareNostrum and conducts research, development, and management of information technologies to facilitate scientific progress. It combines high-performance computing service provision with research and development in computer and computational sciences.

• Curate and assess regional analyses, forecasts, and reanalyses for training and evaluating the regional AI forecasting system
• Design, build, and maintain reproducible ML data pipelines that transform raw simulation and observational data into Anemoi-compatible datasets
• Implement automated testing and CI/CD pipelines for data-processing workflows
• Support deployment and validation of data pipelines on HPC and AI Factory infrastructure, including scalability and performance testing
• Collaborate with the international consortium and document pipelines to ensure reproducibility and traceability

• Bachelor’s degree in Computer Science, Physics, Environmental/Atmospheric Science, or a related discipline
• Strong Python programming skills and experience building data-processing/ETL pipelines
• Experience handling scientific data formats such as NetCDF, GRIB, and Zarr, and tools such as Xarray and CDO/NCO
• Experience preparing datasets for machine-learning training pipelines
• Familiarity with UNIX/Linux environments and scripting languages
• Experience with HPC environments, including job schedulers such as Slurm and GPU-based workflows
• Fluency in English
• Ability to work independently and within an international, multi-partner consortium
• Strong problem-solving skills and attention to detail
• Excellent communication skills for collaborating with interdisciplinary teams
• Будет плюсом: a Master’s degree or PhD in a relevant field, experience in Earth, weather, or climate sciences, familiarity with the Anemoi ecosystem or similar ML-ready NWP dataset frameworks, knowledge of NWP/reanalysis datasets and data-quality assessment, experience with CI/CD tools, Docker/Podman, automated testing, and Git, proficiency in Spanish and other European languages

• Full-time contract, 35 hours per week
• Open-ended contract linked to the technical and scientific activities of the project and its budget duration
• Flexible working hours
• Extensive training plan
• Restaurant tickets
• Private health insurance
• Relocation support is available
• 22 Days of holidays, 6 personal days, and December 24 and 31 off under the collective agreement
• The position is based at BSC within the Earth Sciences Department