data engineer real-time data

Описание: The independent technology consultancy works across data, cloud and technology transformation, helping clients modernise their platforms and make better use of their data.

Задачи:

- Design, build and maintain scalable ELT/ETL data pipelines within Snowflake.

- Implement data ingestion from multiple sources, including real-time streams using Kafka.

- Develop and deploy KSQL and KStreams pipelines for real-time data processing.

- Develop efficient and scalable data models within Snowflake for analytics and business intelligence.

- Ensure data quality, integrity and reliability throughout the data lifecycle.

- Optimise data pipelines and Snowflake workloads for performance and scalability.

- Collaborate with architects, analysts and data scientists to understand requirements and deliver effective data solutions.

- Contribute to data engineering standards, best practices and continuous improvement across the platform.

Требования:

- Solid commercial experience as a Data Engineer delivering cloud-based data solutions.

- Previous experience within insurance or financial services.

- Strong knowledge of Snowflake, including data modelling and performance optimisation.

- Hands-on experience with Kafka for data ingestion and stream processing.

- Practical experience developing and deploying KStreams and KSQL pipelines.

- Strong SQL skills and experience with Python or another scripting language.

- Experience with data orchestration tools such as Airflow.

- Experience with at least one major cloud platform, such as AWS, Azure or GCP.

- Strong understanding of data ingestion, transformation, modelling and integration.

- Ability to collaborate with technical and non-technical stakeholders.

- Nice to have: Experience building data pipelines for Policy Administration Systems, particularly Guidewire.

- understanding of data schemas and Confluent Schema Registry.

- experience with CI/CD practices and DevOps teams.

- broader experience across AWS, Azure or GCP.

Условия:

- Salary up to €60,000

- Health insurance and mental health support

- Financial support for ongoing professional training and development

- Flexible working

- Opportunities to work on modern data and cloud transformation programmes

- Career development opportunities within a growing technology consultancy

#J-18808-Ljbffr