Вакансии Data Engineer / Big Data Architect

We are seeking a Senior Data Engineer to join our growing team and help design, build, and maintain robust data infrastructure for our client — a leader in digital banking solutions. This role is ideal for someone who is passionate about building scalable and efficient data pipelines and enjoys working with modern data tools in a cloud environment.

This is the job

As a Senior Data Engineer, you will play a key role in the development of a modern data platform. You will collaborate closely with data scientists, analysts, and other engineers to ensure high data availability, quality, and performance.

This is you

• You have proven experience with Airflow, Python programming, and SQL / dbt.
• You’ve worked with Snowflake or similar cloud-based data warehouses.
• You have a solid understanding of AWS services and Docker.
• You hold a Bachelor’s degree in Mathematics, Computer Science, or another relevant quantitative field.
• You approach problems with strong analytical and problem-solving skills.
• You’re familiar with Data Engineering best practices, including Data Quality and Monitoring/Observability.
• You’re comfortable working in a dynamic, fast-paced environment, and take ownership of your work.
• You have a growth mindset and are eager to learn and grow through hands-on experience.

This is your role

• Design and develop scalable data pipelines to efficiently process and analyze large volumes of data, utilizing Snowflake, Looker, Airflow, and dbt.
• Collaborate with stakeholders to translate their requirements into technical steps and coordinate the projects you drive with them.
• Monitor and improve the health of our data pipelines.
• Promote knowledge sharing within the team to foster collaboration and continuous learning, and mentor junior colleagues.
• Stay updated on emerging technologies and best practices in data engineering, and bring new ideas to enhance the technical setup.

Our client is a leading SaaS company offering pricing optimization solutions for e-commerce businesses. Its advanced technology utilizes big data, machine learning, and AI to assist customers in optimizing their pricing strategies and maximizing their profits.

About the Role:

As a data engineer you’ll have end-to-end ownership – from system architecture and software development to operational excellence.

Key Responsibilities:

• Design and implement scalable machine learning pipelines with Airflow, enabling efficient parallel execution.
• Enhance our data infrastructure by refining database schemas, developing and improving APIs for internal systems, overseeing schema migrations, managing data lifecycles, optimizing query performance, and maintaining large-scale data pipelines.
• Implement monitoring and observability, using AWS Athena and QuickSight to track performance, model accuracy, operational KPIs and alerts.
• Build and maintain data validation pipelines to ensure incoming data quality and proactively detect anomalies or drift.
• Collaborate closely with software architects, DevOps engineers, and product teams to deliver resilient, scalable, production-grade machine learning pipelines.

Required Competence and Skills:

To excel in this role, candidates should possess the following qualifications and experiences:

• A Bachelor’s or higher in Computer Science, Software Engineering or a closely related technical field, demonstrating strong analytical and coding skills.
• At least 5 years of experience as a data engineer, software engineer, or similar role and using data to drive business results.
• At least 5 years of experience with Python, building modular, testable, and production-ready code.
• Solid understanding of SQL, including indexing best practices, and hands-on experience working with large-scale data systems (e.g., Spark, Glue, Athena).
• Practical experience with Airflow or similar orchestration frameworks, including designing, scheduling, maintaining, troubleshooting, and optimizing data workflows (DAGs).
• A solid understanding of data engineering principles: ETL/ELT design, data integrity, schema evolution, and performance optimization.
• Familiarity with AWS cloud services, including S3, Lambda, Glue, RDS, and API Gateway.

Nice-to-Have:

• Experience with MLOps practices such as CI/CD, model and data versioning, observability, and deployment.
• Familiarity with API development frameworks (e.g., FastAPI).
• Knowledge of data validation techniques and tools (e.g., Great Expectations, data drift detection).
• Exposure to AI/ML system design, including pipelines, model evaluation metrics, and production deployment.

We are seeking a skilled and adaptable Data Engineer who is passionate about data infrastructure and long-term career growth. This role offers an opportunity to build and maintain scalable data solutions while developing expertise in Palantir Foundry and other modern data tools. We value individuals who are excited to expand their technical capabilities over time, work on multiple accounts, and contribute to a dynamic and growing team.
You will play a pivotal role in transforming raw data from various sources into structured, high-quality data products that drive business decisions. The ideal candidate should be motivated to learn and grow within the organization, actively collaborating with experienced engineers to strengthen our data capabilities over time.

About the project

This project focuses on building a centralized data platform for a leading investment firm that supports data-driven decision-making for high-growth companies. Currently, data is sourced from multiple locations, including Excel files, third-party tools, and custom applications, managed within separate systems. This decentralized approach creates inefficiencies and introduces the potential for data inaccuracies.
The objective is to integrate these data sources into a single, unified platform that streamlines access and reduces manual errors. By transforming financial, legal, and operational data into structured data marts, the platform will enable advanced analytics and real-time visualization through BI tools on both web and mobile interfaces.

Skills & Experience

• Bachelor’s degree in Computer Science, Software Engineering, or equivalent experience.
• Minimum 3 years of experience in Python, SQL, and data engineering processes.
• Experience with Palantir Foundry or a strong willingness to learn and develop expertise in it.
• Proficiency in multiple database systems, such as PostgreSQL, Redis, and a data warehouse like Snowflake, including query optimization.
• Hands-on experience with Microsoft Azure services.
• Strong problem-solving skills and experience with data pipeline development.
• Familiarity with testing methodologies (unit and integration testing).
• Docker experience for containerized data applications.
• Collaborative mindset, capable of working across multiple teams and adapting to new projects over time.
• Fluent in English (written & verbal communication).
• Curiosity and enthusiasm for finance-related domains (personal & corporate finance, investment concepts).

Nice to have

• Experience with Databricks.
• Experience with Snowflake.
• Background in wealth management, investment analytics, or financial modeling.
• Contributions to open-source projects or personal projects showcasing data engineering skills.

Responsibilities

• Design and maintain scalable data pipelines to ingest, transform, and optimize data.
• Collaborate with cross-functional teams (engineering, product, and business) to develop solutions that address key data challenges.
• Support data governance, data quality, and security best practices.
• Optimize data querying and processing for efficiency and cost-effectiveness.
• Work with evolving technologies to ensure our data architecture remains modern and adaptable.
• Contribute to a culture of learning and knowledge sharing, supporting newer team members in building their skills.
• Grow into new roles within the company by expanding your technical expertise and working on diverse projects over time.

EPAM у пошуку Senior/Lead Big Data Software Engineer для розробки та підтримки внутрішніх систем сховищ даних для продуктів та додатків нашого клієнта.

Чим ви будете займатися у цій ролі

• Розробка та підтримка складних систем сховищ даних для обробки великих обсягів даних
• Проєктування і оптимізація архітектури баз даних та схем даних
• Розробка та впровадження ефективних запитів SQL для аналізу та обробки даних
• Розробка і впровадження програмних рішень за допомогою екосистеми Hadoop (Yarn, MapReduce, Hive, HDFS)
• Використання Spark та Kafka для обробки та передачі даних у реальному часі
• Налаштування та керування хмарними сервісами AWS (EMR, S3, Lambda)
• Оптимізація існуючих рішень для зменшення витрат часу та ресурсів
• Співпраця з іншими інженерами та командами для створення масштабованих і ефективних рішень
• Виконання аналізу даних, створення звітів та рекомендацій для покращення систем
• Визначення й реалізація кращих практик для забезпечення безпеки та цілісності даних
• Участь у підготовці технічної документації та підтримка документації для користувачів

Навички

• Більше 4 років досвіду в розробці сховищ даних та проєктуванні баз даних
• Експерт в SQL з досвідом роботи не менше 2+ років
• Хороші навички розробки на Python
• Розробка програмних рішень з використанням компонент екосистеми Hadoop (Yarn, MapReduce, Hive, HDFS)
• Практичний досвід з Spark та Kafka
• Досвід роботи з хмарними сервісами AWS (EMR, S3, Lambda)
• Відмінні аналітичні навички та навички комплексного вирішення проблем
• Хороші комунікаційні та презентаційні навички
• Уважність до деталей
• Здатність дотримуватися стислих термінів та вміння розставляти пріоритети

Буде перевагою

• Досвід роботи в адмініструванні баз даних
• Досвід роботи з AWS Redshift
• Практичний досвід з pyspark
• Розуміння підходів масштабування сервісів/аплікацій

We are building scalable data pipelines and infrastructure to generate reports based on massive datasets. Our team is responsible for designing, developing, and validating jobs using the latest versions of Scala and Apache Spark, ensuring accurate statistical results for our stakeholders.
This is a distributed team environment, offering an exciting opportunity to collaborate with top Big Data engineers across Europe and overseas.
While this role primarily focuses on Big Data engineering, experience with CI/CD and DevOps practices is a strong advantage, as infrastructure-related tasks will also be part of the job.

Essential functions

• Design and Develop Scalable Data Pipelines
• Implement and Validate Big Data Solutions
• Integrate and Manage Infrastructure
• Collaborate in a Distributed Environment

Qualifications

• Strong expertise in Spark and Scala
• Hands-on experience with Hadoop
• Proficiency in processing and computation frameworks: Kafka, Spark
• Experience with database engines: Oracle, PostgreSQL, Teradata, Cassandra
• Understanding of distributed computing technologies, approaches, and patterns

Would be a plus

• Experience with Data Lakes, Data Warehousing, or analytics systems

We are seeking an experienced Senior Data Engineer (MySQL, PostgreSQL) to play an essential role in implementing and maintaining a data warehouse. A career in Exadel means you will work alongside exceptional colleagues and be empowered to reach your professional and personal goals.

About the Customer

The world’s largest publisher of investment research. For over two decades it connects the world’s leading asset and wealth managers with nearly 1,000 research firms in more than 50 countries and serves internal teams across multi-national corporations from its offices located in Durham (HQ), New York, London, Edinburgh, and Timisoara.
The client facilitates the equitable exchange of critical investment insights by improving efficiency, collaboration, and security across the complete information lifecycle. The ecosystem is designed to meet users’ bespoke needs, from compliance tracking to interactive publishing, by removing friction from the publication, dissemination, consumption, and application of investment research content.

Requirements

• 5+ years of background in working with MySQL and PostgreSQL in production environments
• Strong expertise in query optimization, indexing strategies, execution plan analysis, and performance tuning
• Competency in database replication, failover strategies, and high-availability architectures
• Hands-on experience with AWS database services, including RDS, Aurora, and DynamoDB
• Proficiency in troubleshooting database performance issues and implementing tuning strategies
• Familiarity with database integration challenges and experience working with Indigo Data Service or similar solutions
• Skills in monitoring and alerting tools like Prometheus, Grafana, or CloudWatch for real-time database performance tracking
• Practice in SQL, Python, or Bash for automation of database operations
• Familiarity with schema design, normalization, and data modeling best practices
• English level – Upper-Intermediate

Nice to Have

• Skills in working with Snowflake and cloud-based data warehouses
• Knowledge of NoSQL databases such as MongoDB, ClickHouse, or BigQuery
• Exposure to containerized database deployments using Kubernetes and Docker
• Experience with CI/CD pipelines for database schema changes and migrations

Responsibilities

• Optimize query performance and fine-tune database configurations for MySQL to enhance system efficiency
• Resolve database integration issues by decoupling dependencies and leveraging the Indigo Data Service for improved failover strategies
• Analyze and mitigate single points of failure within the current database architecture, ensuring high availability and fault tolerance
• Improve indexing strategies, caching mechanisms, and partitioning techniques to reduce query execution times
• Monitor and troubleshoot performance bottlenecks, identifying and addressing slow queries, deadlocks, and resource contention
• Implement database scaling strategies, including read replicas, sharding, and horizontal scaling in AWS environments
• Enhance database security and compliance by implementing best practices for access control, encryption, and backup strategies
• Work closely with software engineers, DevOps, and data teams to ensure seamless database integration and high-performance data access
• Deploy and manage database solutions on AWS, optimizing for cost efficiency and scalability
• Implement automation for database maintenance tasks, including schema migrations, backups, and failover management

SE Ranking is the company standing behind this powerful and intuitive SEO platform, which has been trusted by over a million businesses, agencies, and SEO professionals since 2013 and we are looking for a highly motivated and proactive Senior Big Data / Java Engineer , who is excited to take on new challenges.

Responsibilities:

• Design and develop a large distributed system with multiple nodes.
• Create complex MapReduce and Spark pipelines for processing data volumes reaching hundreds of terabytes.
• Optimize and refine existing Spark pipelines.
• Develop ETL pipelines for OLAP databases, write, and optimize SQL queries.Maintain system stability and respond promptly to emerging issues.

Requirements:

• Experience: At least 7 years of development experience in Java/Scala.
• Expert proficiency in Java, including a deep understanding of multi-threaded and concurrent development.
• Knowledge of JMM (Java Memory Model) and its capabilities in multi-threading.
• Database experience: Schema design, understanding of key concepts (views, joins, transactions, transaction isolation levels, locks, etc.).
• Performance optimization skills for Java applications.
• Experience with distributed systems, including independent design of libraries and subsystems.
• Familiarity with UML and the ability to document architectural decisions.
• Strong OOP understanding and practical application of design patterns.
• Task management skills: Ability to assess complexity, decompose tasks, and prioritize execution.
• Technical documentation skills: Clear and structured documentation of solutions and recommendations.

Personal Qualities:

• Ability to make decisions and take responsibility for them.
• Capability to foresee potential issues at the design stage.
• Willingness to acknowledge mistakes, analyze, and correct them.
• Attention to detail and commitment to delivering high-quality solutions.

Would be an advantage:

• Experience with large-scale data storage and processing systems (HBase, Cassandra, MR Jobs, Flow computations, Kafka, etc.).
• Proficiency in Scala (if Java is the primary language).

Pwrteams are seeking a Senior Data Engineer to join the TUI Musement Data Science & Analytics Team. As a Data & ML Engineer, you will be providing support to the rest of Data profiles in TUI Musement building the tooling used on analytical and operational Data and ML/AI Products helping them at the task of delivering value to the business, and managing the platform used by all Data and ML/AI products on the Data area.

Responsibilities:

• Develop and maintain cutting-edge Data and ML/AI solutions for TUI Musement that meet our Service Level Agreements (SLAs) and fit within our budget.
• Build and optimize Data and ML/AI tools to empower your fellow data professionals, helping them unlock the full potential of our platform.
• Design and manage ETL pipelines and workflows using technologies like SQL and AWS Big Data services, ensuring that data flows seamlessly from source to insight.
• Create robust Machine Learning pipelines that handle the lifecycle of ML models from development to deployment, monitoring, and optimization.
• Implement and manage APIs for model inference, supporting Data Scientists in deploying their solutions at scale.
• Monitor and enhance ML Data products, fine-tuning them for performance, accuracy, and scalability.
• Lead efforts to automate and optimize internal processes – whether it’s reducing manual work, improving data delivery, or re-designing infrastructure for greater efficiency.
• Collaborate with your team in an agile environment, following operational and control procedures to deliver top-tier results.
• Contribute to the design and modeling of a scalable Enterprise Data Platform within the cloud (AWS), ensuring that our data architecture remains future-proof.

Qualifications:

• Proficient in Python, with a commitment to writing clean, maintainable, and testable code.
• Experience with API frameworks like FastAPI, Flask, or Django.
• Demonstrated expertise in Data and ML pipelines, with hands-on experience in CI/CD frameworks.
• Comfortable building containerized solutions with Docker and using orchestration tools like Airflow.
• Proficient in working with both SQL and NoSQL databases: Our current stack includes technologies like Snowflake and DynamoDB, among others.
• A strong understanding of data transformation, metadata management, and workflow orchestration.
• Familiarity with Machine Learning frameworks and libraries is highly valued.
• Experience with AWS cloud services and stream-processing systems like Apache Spark, Apache Flink, or Kinesis is a plus.
• You have an analytical mindset and a passion for continuous learning, embracing challenges as opportunities.
• Fluent in English, you’re able to communicate both technical details and big-picture concepts with ease, whether you’re speaking to senior stakeholders or peers in the technical team.
• You’re passionate about data, machine learning, and finding solutions to complex challenges, but you also thrive in a collaborative environment.
• You see data not just as numbers but as the foundation for meaningful change, and you’re excited about the opportunity to help revolutionize the travel experience.

NBS LVIV is seeking a highly motivated and experienced Twilio Segment CDP and Braze Developer to join our Digital Marketing and Data Integration team. This role will focus on maintaining and implementing integrations for Twilio Segment CDP and Braze platforms to support our marketing efforts across Europe. The ideal candidate will work closely with campaign managers and other team members to ensure seamless data flow and efficient campaign execution.

Key Responsibilities:

• Design, develop, and maintain integrations between Twilio Segment CDP, Braze, and other internal and external systems.
• Collaborate with campaign managers to configure and optimize data flow for marketing campaigns.
• Monitor and troubleshoot data integration issues, ensuring data accuracy and timely delivery.
• Develop and maintain documentation for data flows, system integrations, and processes.
• Participate in code reviews and provide constructive feedback to team members.
• Stay current on industry trends, best practices, and new features of Twilio Segment CDP and Braze.
• Collaborate with cross-functional teams to identify opportunities for improvement and drive continuous integration and deployment practices.

Required Skills and Qualifications:

• Bachelor's degree in Computer Science, Information Technology, or a related field.
• 3+ years of experience in software development, with a focus on data integration and API development.
• Strong proficiency in programming languages such as JavaScript.
• Experience working with RESTful APIs and web services.
• Knowledge of data modeling, data warehousing, and ETL processes.
• Familiarity with Agile/Scrum methodologies and project management tools such as JIRA.
• Excellent problem-solving skills and attention to detail.
• Proficient in English, both written and spoken.

Desired Skills:

• Experience with other marketing automation and customer data platforms (e.g., Salesforce, Adobe, etc.).
• Knowledge of GDPR and other data privacy regulations.
• Familiarity with data visualization tools (e.g., Tableau, Power BI).
• Experience working in a distributed team or remote work environment.
• Hands-on experience with Twilio Segment CDP and Braze platform integrations.

Soft Skills:

• Strong communication and interpersonal skills.
• Ability to work effectively in a team environment.
• Adaptability and willingness to learn new technologies.
• Self-motivated and able to work independently with minimal supervision.
• Strong time management and prioritization skills.