Opdrachten

Cegeka Consulting B.V. (YF) Data Engineer

Data Engineer

Info

Functie

Data Engineer

Locatie

Amsterdam

Uren per week

40 uren per week

Looptijd

16.10.2022 - 30.12.2022

Opdrachtnummer

120281

Sluitingsdatum

date-icon30.09.2022 clock-icon13:09

Rolomschrijving en taakafspraken

 About the Data Engineer
The Data Engineer is a generalist, not only a master of his own tools, but flexible to work with environments used by other partners and organizations. This makes him both curious and studious to learn new techniques. He is mainly focused on how a desired solution can be made and is deeply immersed in the contents of his work, having as few business meetings as he possibly can. You can find the him by following the data in an organization. It is usually the Data Engineer that can show you the data. Working closely with the IT department, he can often be found on the business side of an organization, mainly in the innovation department.

About DSH
DSH is a Kafka based, managed platform intended to facilitate sharing of events among multiple parties at massive scale, with minimal latency, maximal security and being more efficient than what proprietary cloud-vendor services can offer. This facilitates collaboration between ecosystems either within or between organizations in event-driven usecases.
Many of the technical challenges resulting from this type of collaboration have been solved by DSH. DSH currently facilitates an ever growing array of uscases as many organizations start to discover the benefits of working event-based

About the team
The DSH Acceleration Team is at the start of any new usecase on DSH. Together with a team of Business Developers, the Acelerators work in projects to help organizations identify and sharpen their business cases, to design the solutions required and to jumpstart their engineering efforts by training them, designing with them, supporting them or doing the initial work for them. Anything to accelerate the generation of value for our customers. We run projects that last for months and projects that run for only a couple of weeks. Every two weeks we reassess our priorities together with the Business Development team. Working as an accelerator means working in a dynamic, innovative environment. Work to deliver value at scale today using with tomorrow’s technologies.

Skillset
The Data Engineer is someone who can build and maintain production-grade solutions with robustness, scalability, fault tolerance and security in mind. He has advanced software engineering skills and knowledge of distributed computing, event driven processes, logging and monitoring.
Because the Data Engineer will integrate various systems with the KPN Data Services Hub, advanced level API and database skills are required. Besides building streaming processes, he is also often required to translate batch processes to streams and vice versa.

Other notable skills are:
Microservices development
Two high performing languages (e.g. Scala, Go)
Defensive programming mindset
CICD Pipelines
Data Mesh Arcitecture
Apache Kafka
MQTT
Docker

Role and responsibilities
Knowledge of and experience with IoT devices, PLC’s and device management in general is an advantage for a Data Engineer working with the KPN Data Services Hub, as well as knowledge and production experience with underlying technologies and protocols.
The Data Engineer builds data connections between various systems and serves the other roles: Data Solution Designer, Data Scientist and UI/UX Designer. Meaning he is responsible to build the solution designed by the Solution Designer making sure that the functional and non-functional requirements are met, particularly those of low-latency scalability of information processing components created by the Data Scientist, and data transfer that is GDPR compliant and meets the KPN Security Policies (KSP). Finally, he is responsible to make data accessible in the format that is desired by the Data Scientist and the UI/UX Designer.


Bedrijfsgegevens

Bedrijfs gegevens

Cegeka Consulting B.V. (YF)

Rolomschrijving en taakafspraken

 About the Data Engineer
The Data Engineer is a generalist, not only a master of his own tools, but flexible to work with environments used by other partners and organizations. This makes him both curious and studious to learn new techniques. He is mainly focused on how a desired solution can be made and is deeply immersed in the contents of his work, having as few business meetings as he possibly can. You can find the him by following the data in an organization. It is usually the Data Engineer that can show you the data. Working closely with the IT department, he can often be found on the business side of an organization, mainly in the innovation department.

About DSH
DSH is a Kafka based, managed platform intended to facilitate sharing of events among multiple parties at massive scale, with minimal latency, maximal security and being more efficient than what proprietary cloud-vendor services can offer. This facilitates collaboration between ecosystems either within or between organizations in event-driven usecases.
Many of the technical challenges resulting from this type of collaboration have been solved by DSH. DSH currently facilitates an ever growing array of uscases as many organizations start to discover the benefits of working event-based

About the team
The DSH Acceleration Team is at the start of any new usecase on DSH. Together with a team of Business Developers, the Acelerators work in projects to help organizations identify and sharpen their business cases, to design the solutions required and to jumpstart their engineering efforts by training them, designing with them, supporting them or doing the initial work for them. Anything to accelerate the generation of value for our customers. We run projects that last for months and projects that run for only a couple of weeks. Every two weeks we reassess our priorities together with the Business Development team. Working as an accelerator means working in a dynamic, innovative environment. Work to deliver value at scale today using with tomorrow’s technologies.

Skillset
The Data Engineer is someone who can build and maintain production-grade solutions with robustness, scalability, fault tolerance and security in mind. He has advanced software engineering skills and knowledge of distributed computing, event driven processes, logging and monitoring.
Because the Data Engineer will integrate various systems with the KPN Data Services Hub, advanced level API and database skills are required. Besides building streaming processes, he is also often required to translate batch processes to streams and vice versa.

Other notable skills are:
Microservices development
Two high performing languages (e.g. Scala, Go)
Defensive programming mindset
CICD Pipelines
Data Mesh Arcitecture
Apache Kafka
MQTT
Docker

Role and responsibilities
Knowledge of and experience with IoT devices, PLC’s and device management in general is an advantage for a Data Engineer working with the KPN Data Services Hub, as well as knowledge and production experience with underlying technologies and protocols.
The Data Engineer builds data connections between various systems and serves the other roles: Data Solution Designer, Data Scientist and UI/UX Designer. Meaning he is responsible to build the solution designed by the Solution Designer making sure that the functional and non-functional requirements are met, particularly those of low-latency scalability of information processing components created by the Data Scientist, and data transfer that is GDPR compliant and meets the KPN Security Policies (KSP). Finally, he is responsible to make data accessible in the format that is desired by the Data Scientist and the UI/UX Designer.


De recruiter