Вакансии Data Engineer / Big Data Architect

Our customer is the leading school transportation provider in North America, being the owner of more than a half of all yellow school buses in the United States. Every day, the company completes 5 million student journeys, moving more passengers than all U.S. airlines combined and delivers reliable, quality services for 1,100 school districts.
N-iX has built a successful cooperation with the client delivering a range of complex initiatives. As a result, N-iX has been selected as a strategic long-term partner to drive the digital transformation on an enterprise level, fully remodeling the technology landscape for 55,000 employees and millions of people across North America.

Responsibilities:

• Design complex ETL processes of various data sources in the data warehouse
• Build new and maintain existing data pipelines using Python to improve efficiency and latency
• Improve data quality through anomaly detection by building and working with internal tools to measure data and automatically detect changes
• Identify, design, and implement internal process improvements, including re-designing infrastructure for greater scalability, optimizing data delivery, and automating manual processes
• Perform data modeling and improve our existing data models for analytics
• Collaborate with SMEs, architects, analysts, and others to build solutions that integrate data from many of our enterprise data sources
• Partner with stakeholders, including data, design, product, and executive teams, and assist them with data-related technical issues

Requirements:

• Proficiency in Python 7+ years
• 3-5 years of commercial experience in building and maintaining a Data Lake
• Experience leading a Data Lake team of 3-5 Engineers (2 years)
• Good knowledge of AWS cloud services, including the Glue framework with integration type of projects (2 years)
• Experience maintaining Apache Kafka
• Steady expertise in data processing tools, including Redis, Apache Spark, Apache Iceberg, Athena.
• Knowledge of job scheduling and orchestration using Airflow
• Experience in events streaming
• Well-versed in the optimization of ETL processes
• Experience of developing high-load backend services on Python.
• Good understanding of algorithms and data structures
• Excellent communication skills, both written and verbal

Nice to have:

• Experience in the schema and dimensional data design
• Collaboration within a scaled team, using Agile methodology
• Decent knowledge of CI/CD (Docker, Cloud formation, Git)

Our mission at Geniusee is to help businesses thrive through tech partnership and strengthen the engineering community by sharing knowledge and creating opportunities. Our values are Continuous Growth, Team Synergy, Taking Responsibility, Conscious Openness and Result Driven. We offer a safe, inclusive and productive environment for all team members, and we’re always open to feedback. If you want to work from home or work in the city center of Kyiv, great – apply right now.

Requirements

• 8+ years of experience in software engineering, with a focus on AI/ML systems or distributed systems;
• Hands-on experience building and deploying retrieval-augmented generation (RAG) systems;
• Deep knowledge of OpenSearch, Elasticsearch, or similar search engines;
• Strong coding skills in Python;
• Experience with frameworks like LlamaIndex or LangChain;
• Familiarity with vector databases such as Pinecone, Qdrant, or FAISS;
• Exposure to LLM fine-tuning, semantic search, embeddings, and prompt engineering;
• Previous work on systems handling millions of users or queries per day;
• Familiarity with cloud infrastructure (AWS, GCP, or Azure) and containerization tools (Docker, Kubernetes);
• Experience with vector search, embedding pipelines, and dense retrieval techniques;
• Proven ability to optimize inference stacks for latency, reliability, and scalability;
• Excellent problem-solving, analytical, and debugging skills;
• Strong sense of ownership, ability to work independently, and a self-starter mindset in fast-paced environments;
• Passion for building impactful technology aligned with our mission;
• Bachelor’s degree in Computer Science or related field, or equivalent practical experience.

What you will do

• Design, build and scale a production-grade inference stack for RAG-based applications;
• Develop efficient retrieval pipelines using OpenSearch or similar vector databases, with a focus on high recall and response relevance;
• Optimize performance and latency for both real-time and batch queries;
• Identify and address bottlenecks in the inference stack to improve response times and system efficiency;
• Ensure high reliability, observability, and monitoring of deployed systems;
• Collaborate with cross-functional teams to integrate LLMs and retrieval components into user-facing applications;
• Evaluate and integrate modern RAG frameworks and tools to accelerate development;
• Guide architectural decisions, mentor team members, and uphold engineering excellence.

ELEKS Software Engineering and Development Office is looking for a Senior Data Engineer in Ukraine, Poland, or Croatia.

About client

The customer is a British company producing electricity with zero carbon emissions.

Requirements

• Experience in Data Engineering, SQL, ETL(data validation + data mapping + exception handling) 5+ years
• Hands-on experience with Databricks 2+ years
• Experience with Python
• Experience with AWS (e.g. S3, Redshift, Athena, Glue, Lambda, etc.)
• Knowledge of the Energy industry (e.g. energy trading, utilities, power systems etc.) would be a plus
• Experience with Geospatial data would be a plus
• At least an Upper-Intermediate level of English

Responsibilities

• Building Databases and Pipelines: Developing databases, data lakes, and data ingestion pipelines to deliver datasets for various projects
• End-to-End Solutions: Designing, developing, and deploying comprehensive solutions for data and data science models, ensuring usability for both data scientists and non-technical users. This includes following best engineering and data science practices
• Scalable Solutions: Developing and maintaining scalable data and machine learning solutions throughout the data lifecycle, supporting the code and infrastructure for databases, data pipelines, metadata, and code management
• Stakeholder Engagement: Collaborating with stakeholders across various departments, including data platforms, architecture, development, and operational teams, as well as addressing data security, privacy, and third-party coordination

We are looking for a proactive Data Integration Tech Lead to lead and collaborate with a strong, high-performing team of engineers. The role involves owning integration strategy, leading complex data pipeline development, and guiding engineers while ensuring scalable, secure, and high-quality solutions. We seek an active team player with strong leadership skills, deep expertise in integrations, and the ability to align technical solutions with business needs.
Our client is a leading provider of software and technology solutions for higher education. From student information systems and enrollment management to finance, HR, and analytics, their products empower universities and colleges to operate more efficiently and deliver a better experience for students.

Requirements

• Bachelor’s degree in Computer Science, Information Technology, or related field.
• 7+ years of experience in integration development, backend/API engineering, or data engineering roles.
• 2+ years in a lead or senior capacity, with proven ability to guide engineers and projects.
• Demonstrated ability to mentor engineers, conduct code/design reviews, and set technical direction.
• Strong communication and presentation skills; able to interact with business stakeholders and technical teams.
• Experience leading cross-functional integration initiatives.
• Experience building and optimizing data pipelines for both real-time and batch processes.
• Deep SQL and PL/SQL knowledge (Oracle, PostgreSQL).
• Understanding of stored procedures, views, DB packages, and performance tuning.
• Familiarity with API testing and debugging tools (Postman, curl, Bruno).
• Hands-on experience with cloud platforms (AWS, Azure, or GCP).
• Experience migrating on-prem integrations to SaaS/cloud platforms.
• Expertise in iPaaS or ETL platforms (Boomi, Talend, SnapLogic, NiFi, Azure Data Factory, etc.).
• Understanding of GraphQL and modern API access patterns.
• Familiarity with BI/reporting tools (Cognos, Argos, Tableau, Metabase).
• Proficiency in one or more languages: JavaScript (Node.js), Python, Java.

Responsibilities

• Own the integration strategy and architecture for enterprise data flows and APIs.
• Lead design, development, and optimization of complex data integration pipelines.
• Define and enforce standards for API usage, data mappings, error handling, and performance.
• Evaluate new technologies and recommend adoption where appropriate.
• Ensure integrations are scalable, secure, and compliant with enterprise requirements.
• Lead and mentor a team of integration engineers, providing technical guidance and career development.
• Conduct design and code reviews, ensuring high-quality deliverables.
• Champion best practices in integration, testing, and DevOps.
• Coordinate workload distribution and ensure alignment with project milestones.
• Work closely with software engineers, architects, business analysts, and external vendors.
• Translate business requirements into technical solutions and present integration designs to stakeholders.
• Act as primary escalation point for integration-related issues.
• Document integration architectures, workflows, and governance frameworks for transparency and knowledge sharing.

What you will do

• Design, build, and maintain large-scale data pipelines and data processing systems in AWS;
• Develop and optimize distributed data workflows using Hadoop, Spark, and related technologies;
• Collaborate with data scientists, analysts, and product teams to deliver reliable and efficient data solutions;
• Implement best practices for data governance, security, and compliance;
• Monitor, troubleshoot, and improve the performance of data systems and pipelines;
• Mentor junior engineers and contribute to building a culture of technical excellence;
• Evaluate and recommend new tools, frameworks, and approaches for data engineering.

Must haves

• Bachelor’s or Master’s degree in Computer Science, Engineering, or related field;
• 5+ years of experience in data engineering, software engineering, or related roles;
• Strong hands-on expertise with AWS services (S3, EMR, Glue, Lambda, Redshift, etc.);
• Deep knowledge of big data ecosystems, including Hadoop (HDFS, Hive, MapReduce) and Apache Spark (PySpark, Spark SQL, streaming);
• Strong SQL skills and experience with relational and NoSQL databases;
• Proficiency in Python, Java, or Scala for data processing and automation;
• Experience with workflow orchestration tools (Airflow, Step Functions, etc.);
• Solid understanding of data modeling, ETL/ELT processes, and data warehousing concepts;
• Excellent problem-solving skills and ability to work in fast-paced environments;
• Ability to work German TimeZone (~ 6-7 am to ~ 2-3 pm Brazil/ ART time);
• Upper-Intermediate English level.

Nice to haves

• Experience with real-time data streaming platforms (Kafka, Kinesis, Flink);
• Knowledge of containerization and orchestration (Docker, Kubernetes);
• Familiarity with data governance, lineage, and catalog tools;
• Previous leadership or mentoring experience.

The primary goal of the project is the modernization, maintenance and development of an eCommerce platform for a big US-based retail company, serving millions of omnichannel customers each week.
Solutions are delivered by several Product Teams focused on different domains - Customer, Loyalty, Search and Browse, Data Integration, Cart.
Current overriding priorities are new brands onboarding, re-architecture, database migrations, migration of microservices to a unified cloud-native solution without any disruption to business.

Responsibilities

We are looking for Data Engineer who will be responsible for designing a solution for a big retail company. The main focus is to support processing of big data volumes and integrate solution to current architecture.

Must have skills

• Strong, recent hands-on expertise with Azure Data Factory and Synapse is a must (3+ years).
• Experience in leading a distributed team.
• Strong expertise in designing and implementing data models, including conceptual, logical, and physical data models, to support efficient data storage and retrieval.
• Strong knowledge of Microsoft Azure, including Azure Data Lake Storage, Azure Synapse Analytics, Azure Data Factory, and Azure Databricks, pySpark for building scalable and reliable data solutions.
• Extensive experience with building robust and scalable ETL/ELT pipelines to extract, transform, and load data from various sources into data lakes or data warehouses.
• Ability to integrate data from disparate sources, including databases, APIs, and external data providers, using appropriate techniques such as API integration or message queuing.
• Proficiency in designing and implementing data warehousing solutions (dimensional modeling, star schemas, Data Mesh, Data/Delta Lakehouse, Data Vault)
• Proficiency in SQL to perform complex queries, data transformations, and performance tuning on cloud-based data storages.
• Experience integrating metadata and governance processes into cloud-based data platforms
• Certification in Azure, Databricks, or other relevant technologies is an added advantage
• Experience with cloud-based analytical databases.
• Experience with Azure MI, Azure Database for Postgres, Azure Cosmos DB, Azure Analysis Services, and Informix.
• Experience with Python and Python-based ETL tools.
• Experience with shell scripting in Bash, Unix or windows shell is preferable.

Nice to have

• Experience with Elasticsearch
• Familiarity with containerization and orchestration technologies (Docker, Kubernetes).
• Troubleshooting and Performance Tuning: Ability to identify and resolve performance bottlenecks in data processing workflows and optimize data pipelines for efficient data ingestion and analysis.
• Collaboration and Communication: Strong interpersonal skills to collaborate effectively with stakeholders, data engineers, data scientists, and other cross-functional teams.

On behalf of our Client from Japan, Mobilunity is looking for a Data Engineer for a long-term engagement.

About project:

Our client is a fast-growing fintech company based in Tokyo. The company offers its real-time monthly consolidated credit service all across Japan. Our client started Japan’s first instant post-pay credit service for e-commerce customers in October 2014. The main product is an online payment platform which requires no pre-registration or credit card to use; via this service the customers can purchase products online using only a mobile phone number and email address and settle a single monthly bill for all their purchases, either at a convenience store, by bank transfer or auto debit. Customers can use credit funds during a month and then return the balance to zero without paying credit interests.
This service also supports multi-pay installments and subscriptions. There are currently over 4 million accounts in use. This service got the largest investment to date in the Japanese financial tech industry, including PayPal Ventures investment.
We are looking for a Data Engineer, reporting to the Head of Data Platform Engineering, who will work closely with product engineers and business stakeholders to turn data into information. From which insights can be provided back to product and business leaders to inform our product design and features. Your responsibilities will include conducting full lifecycle analysis to include requirements, task and design(s). You will develop both analysis on source data systems as well as support reporting activities and capabilities. Our ambition is to be a Data Driven firm and you will assist in this endeavor in part by monitoring performance of source domain data systems, cleansing and identifying improvements.

Key Role and Responsibilities:

• Interpret data, analyze results using statistical techniques and provide ongoing reports.
• Develop and implement systems to analyze, report, and present data for use by other systems and human stakeholders.
• Review, understand, and improve existing reporting systems to optimize performance, add new functionality, and verify correctness.
• Work with the Data Integration team and stakeholders to understand reporting requirements, identify necessary data, and determine additional options for improvement.
• Identify, analyze, and interpret trends or patterns in product domains in source data systems.
• Ensure data quality and correctness, to meet government regulatory standards.

Requirements:

• Over 3 years of experience leveraging data to drive business outcomes, with additional expertise in reporting related to user credit, collections, and finance considered a strong plus.
• Deep domain knowledge in data, with hands-on experience in data layering, data modeling, data mining, and other end-to-end data implementation practices.
• Skilled in designing and developing efficient data processing workflows to extract valuable insights from both structured and unstructured data, enabling data-driven decision-making.
• Familiar with widely used data warehouse tools, with practical experience in working with both relational and non-relational database systems.
• Strong programming skills in Python, Scala, or other languages, combined with solid SQL optimization capabilities; experience with AWS services is a plus.
• Possesses strong data intuition, analytical thinking, and excellent communication and cross-functional collaboration skills.
• Level of English – Upper-Intermediate and higher.

We’re the Wix Payments team.
We provide Wix users with the best way to collect payments from their customers and manage their Wix income online, in person, and on-the-go. We’re passionate about crafting the best experience for our users, and empowering any business on Wix to realize its full financial potential. We have developed our own custom payment processing solution that blends many integrations into one clean and intuitive user interface. We also build innovative products that help our users manage their cash and grow their business. The Payments AI team is instrumental in promoting AI based capabilities within the payments domain and is responsible for ensuring the company is always at the forefront of the AI revolution.
As a Data Engineer on the Wix Payments AI Team, you’ll play a crucial role in leveraging data to uncover insights, refine user experiences, and drive strategy within the payments domain. You’ll develop & maintain infrastructure for both generative AI and classical data science applications and while pushing the limits of what can be built with emerging AI technologies, often requiring research into new technology stacks and identifying optimal combinations to create solutions.

In your day-to-day you will:

• Develop scalable solutions to address data gaps and process design needs
• Monitor data pipeline performance, troubleshoot issues and guarantee data infrastructure availability
• Engage with various stakeholders, including data engineers, analysts, data scientists and product managers, to support data-driven decision-making
• Take part in designing, developing and maintaining the generative AI infrastructure of the Payments product
• Ensure timely and seamless exchange of information between AI applications, users and our data warehouse
• Maintain existing classical data science infrastructure, involving updating and bug fixing multiple distributed Python libraries

Requirements

• Proficient in Trino SQL, with the ability to craft complex queries
• Highly skilled in Python for developing data pipelines, with expertise in Python frameworks (e.g., Streamlit, Airflow, Pyless, etc.)
• Ability to write, test and deploy production-ready code
• Fluent in English with strong communication abilities
• An independent and quick learner
• Open-headed, capable of coming up with creative solutions and adapting to frequently changing circumstances and technological advances
• Experience with Generative AI concepts, including hands-on experience or knowledge of Prompt Engineering, Fine-Tuning, Automatic Evaluation and RAG – an advantage
• Experience serving and/or fine-tuning open-source LLMs – an advantage

As a Data & Application Engineer, you are responsible for the engineering team and the technology that the team owns. You will not only work as a coach for your team but also as a technical leader, ensuring that the right technical decisions are made when building our data and reporting product(s).
As a data and analytics team, we are responsible for building a cloud-based Data Platform for our client and its stakeholders across brands. We aim to provide our end users from different Finance departments, e.g.., Risk, FPA, Tax, Order to Cash, the best possible platform for all of their Analytics, Reporting & Data needs.
Collaborating closely with a talented team of engineers and product managers, you'll lead the delivery of features that meet the evolving needs of our business on time. You will be responsible to conceptualize, design, build and maintain data services through data platforms for the assigned business units. Together, we'll tackle complex engineering challenges to ensure seamless operations at scale and in (near) real-time.
If you're passionate about owning the end-to-end solution delivery, thinking for future and driving innovation and thrive in a fast-paced environment, join us in shaping the future of the Data and Analytics team.

Responsibilities

• Strategy and Project Delivery
• Together with the business Subject Matter Experts and Product Manager, conceptualize, define, shape and deliver the roadmap to achieving the company priority and objectives
• Lead business requirement gathering sessions to translate into actionable delivery solutions backlog for the team to build
• Lead technical decisions in the process to achieve excellence and contribute to organizational goals.
• Lead the D&A teams in planning and scheduling the delivery process, including defining project scope, milestones, risk mitigation and timelines management including allocating tasks to team members and ensuring that the project stays on track.
• Have the full responsibility in ensuring successful delivery by the D&A teams to deliver new products on time, set up processes and operational plans from end to end, e.g., collecting user requirement, design, build & test solution and Ops Maintenance
• Technical leader with a strategic thinking for the team and the organization. Visionary who can deliver strategic projects and products for the organization.
• Own the data engineering processes, architecture across the teams
• Technology, Craft & Delivery
• Experience in designing and architecting data engineering frameworks, dealing with high volume of data
• Experience in large scale data processing and workflow management
• Mastery in technology leadership
• Engineering delivery, quality and practices within own team
• Participating in defining, shaping and delivering the wider engineering strategic objectives
• Ability to get into the technical detail (where required) to provide technical coach, support and mentor the team
• Drive a culture of ownership and technical excellence, including reactive work such as incident escalations
• Learn new technologies and keep abreast of existing technologies to be able to share learnings and apply these to a variety of projects when needed

Role Qualifications and Requirements:

• Bachelor degree
• At least 5 years of experience leading and managing one or multiple teams of engineers in a fast-paced and complex environment to deliver complex projects or products on time and with demonstrable positive results.
• 7+ years' experience with data at scale, using Kafka, Spark, Hadoop/YARN, MySQL (CDC), Airflow, Snowflake, S3 and Kubernetes
• Solid working experience working with Data engineering platforms involving languages like PySpark, Python or other equivalent scripting languages
• Experience working with public cloud providers such as Snowflake, AWS
• Experience to work in a complex stakeholders' organizations
• A deep understanding of software or big data solution development in a team, and a track record of leading an engineering team in developing and shipping data products and solutions.
• Strong technical skills (Coding & System design) with ability to get hands-on with your team when needed
• Excellent communicator with strong stakeholder management experience, good commercial awareness and technical vision
• You have driven successful technical, business and people related initiatives that improved productivity, performance and quality
• You are a humble and thoughtful technology leader, you lead by example and gain your teammates' respect through actions, not the title
• Exceptional and demonstrable leadership capabilities in creating unified and motivated engineering teams