Вакансия Big Data Engineer

• Scala
• Apache Spark
• ETL
• Hadoop
• Hive
• Kafka
• SQL
• API

Описание вакансии

Brainence is currently seeking a Big Data Engineer to join our team. You’ll work on the only journey advertising SaaS platform that integrates media planning and buying. Its journey automation technology streamlines planning and buying processes and provides real-time insights, creating a unified solution for digital ads and optimizing advertising budgets.
The project is a long-term commitment lasting at least one year, with a three-month probationary period. You will work full-time with a flexible schedule, either remotely or onsite in our Lviv office.

Responsibilities:

• Design, develop, and maintain large-scale data processing solutions using Apache Spark and Scala.
• Develop ETL pipelines to process, clean, and transform data from various sources.
• Optimize Spark jobs for performance and scalability.
• Develop and maintain APIs to facilitate data access and integration.
• Ensure data accuracy, consistency, and completeness throughout the entire data pipeline.
• Collaborate with data scientists, analysts, and other cross-functional teams to understand data requirements and deliver high-quality solutions.

Requirements:

• 3+ years of experience in Scala programming and Spark development;
• Strong understanding of Spark architecture and internals;
• Experience with distributed computing technologies (e.g., Hadoop, Hive, Kafka);
• Experience with database technologies and SQL;
• Experience with developing and consuming APIs;
• Solid understanding of data monitoring, alerting, and logging systems.