Вакансии Data Engineer / Big Data Architect

Altinity is looking for a great server internals engineer to work on contributions to ClickHouse. As a ClickHouse Open Source Developer, you’ll be responsible for designing, implementing, and supporting features of ClickHouse ranging from encryption to storage to query processing. We’re looking for imaginative engineers with a background in database internals and in high-performance languages like C++.
We have lots of exciting projects underway as we help the community adapt ClickHouse to the cloud and Kubernetes.

Our ideal candidate has:

• Proven experience in design, implementation, and testing high-performance DBMS features in a complex C++ codebase.
• Excellent background in database internals including query languages, access methods, storage, and/or connectivity.
• Demonstrated ability to read and write good C++.
• Good understanding of networking and I/O on Linux.
• Familiar with performance optimization techniques and tools.
• History of getting pull requests vetted and merged in rapidly evolving open-source projects.
• Sound knowledge of database testing, debugging, and low-level performance optimization.
• Enthusiasm to learn more about database technology and data-related applications.
• Good English language reading and writing skills.
• Eager to work with a friendly, distributed team following open-source dev practices.
• MAJOR PLUS: previous development experience on ClickHouse.

A day in your life as a ClickHouse server engineer may include any or all of the following:

• Write good task-specific C++ code and solidify it with tests.
• Debug issues reported by users, fix them and add tests to make sure they won’t happen again.
• Profile existing code and make it faster (either by applying clever algorithms, adding vectorized intrinsics, or by implementing cool tricks), add performance tests.
• Submit your own pull requests and review pull requests from others.
• Help the Support Team investigate customer problems running ClickHouse.
• Help new community members contribute to ClickHouse.
• Attend meetups and make presentations on open-source development.
• Write blog articles and share information about ClickHouse.

EY Belgium Modern Data Platform team supports our clients in defining and rolling out the right data architecture, data platform and infrastructure that support their needs, implement and maintain automated data pipelines, and infuse data through business intelligence and analytics. We are currently looking for a Data Engineer to lead the team and performing business development with other team leaders.

Key Responsibilities

As Data Engineering lead, you will have the following responsibilities:

• Lead clients discussions going from defining the problem, Advising on the solution (Data Architecture, Best practices guidance) and supporting implementation.
• Lead the design, development, and maintenance of efficient data pipelines.
• Create robust data products that leverage cloud Data platforms such Azure Synapse, Snowflake and Databricks
• Use cloud services to build scalable, reliable, and efficient data solutions.
• Help translate data and analytics requirements into data solutions based on the approved technical designs.
• Define the technology stack, best practices, and standards for data engineering and analytics.
• Enable test automation and ensure CI/CD pipelines are in good health.
• Implement monitoring of data applications and track product quality, performance, and stability.
• Carry out effective technical design reviews to ensure that the right architecture patterns are used by the team.
• Optimize and fine-tune data processing workflows for performance and reliability.
• Identify and resolve issues within organizations and processes.
• Educate clients and partners about the data analytics landscape.
• Provide guidance on industry trends, emerging technologies, and best practices.
• Lead a team of Data Engineers to thrive by promoting teamwork, providing guidance, and encouraging skill enhancement.

Skills and Attributes for Success

• Master’s degree in computer science, engineering, mathematics or another relevant subject
• Minimum 5+ years’ experience in Data Engineering
• Experience with batch and real-time processing frameworks.
• Expertise in proactive performance monitoring, conducting data quality testing, and troubleshooting performance issues of complex warehouse data tables.
• Experience in assembling data from multiple sources and analyzing and modeling complex datasets.
• Hands-on Experience with production Cloud / DevOps environments and Data Lake, Data transformation, ETL/ELT, and other data concepts
• Practical knowledge of cloud possibilities and limitations in areas like distributed systems, load balancing, networking, massive data storage, and security.
• Solid Experience of leveraging Microsoft Azure ecosystem to manage the development and maintenance of cloud platform operations.
• Understanding of data architecture concepts such as data modeling, Big Data storage, Lambda architecture, data vault, and dimensional modeling, Data Fabric and Data Mesh is nice to have.
• Excellent analytical skills and problem solver
• Strong communicator who understands team dynamics, able to support where needed and lead when asked.
• Experience in leading and coaching technical teams.
• Fluent in Dutch and/or French, proficient business English

Для розширення команди, ми в пошуку Big Data Engineer.

Основні завдання на цій позиції:

• Визначення зовнішніх та внутрішніх джерел великих даних для проведення аналітичних робіт
• Побудова ефективних процесів отримання та конвеєрної обробки даних великих об’ємів із гетерогенних джерел даних, розробка методів та регламентів необхідних ETL-процесів
• Розробка моделей даних, адаптованих до технологій великих даних
• Розробка необхідних аналітичних звітів, розрізів і агрегатів великих даних
• Оцінка відповідності наборів даних предметній галузі та задачам аналітичних робіт
• Керування життєвим циклом даних (процеси отримання, розміщення, зберігання, розподілення, міграції, архівування та видалення великих даних)

Необхідні навички:

• Досвід в галузі будування сховищ даних та процесів видобутку, обробки та трансформування даних на основі MPP архітектури
• Досвід у database schema design та dimensional data modeling
• Знання та великий досвід використання SQL, Oracle
• Знання Java або Python як стеки для автоматизації процесів
• Досвід у роботі з NoSQL базами на кшталт HBase, Elasticsearch
• Знання Redis DB, MongoDB може бути корисним;
• Розуміння побудови розподілених кластерів на кшталт Spark, Hadoop, та інших
• Досвід у роботі з СУБД для аналізу великих даних типу Vertica, Exadata буде доречним;
• Розуміння та наявність досвіду у роботі з системами брокерів повідомлень на кшталт Kafka
• Знання та розуміння технологій OLAP
• Розуміння побудови та досвід роботи з інструментами data governance
• Досвід побудови та використання data analytics та data science wokbench
• Знання UNIX з досвідом використання Shell для задач автоматизаії