Senior Data Engineer

Key Responsibilities

Design end-to-end Lakehouse architectures (Bronze/Silver/Gold) using Databricks and Delta Lake.

Define reference architectures for batch, streaming, and near-real-time workloads.

Select and standardize patterns for ingestion, transformation, orchestration, and serving.

Ensure scalability, security, cost optimization, and high availability across environments.

Build and optimize ETL/ELT pipelines using Spark (PySpark / SQL).

Implement ingestion from APIs, event streams, files, databases, and message brokers.

Implement incremental processing, CDC patterns, and schema evolution.

Governance, Quality & Security

Implement Unity Catalog for data governance, lineage, and access control.

Define data quality rules, validation frameworks, and monitoring.

Enforce data contracts, naming conventions, and lifecycle management.

Integrate Databricks with cloud services (Azure, AWS and GCP).

Implement CI/CD for data pipelines and notebooks.

Manage environments (dev/test/prod) and infrastructure as code.

Collaborate on cost governance and FinOps optimization.

Analytics & Consumption Enablement

Model curated datasets for BI, analytics, and ML use cases.

Support ML workflows and feature engineering where relevant.

Act as technical lead for data engineering teams.

Review designs and code, define best practices, and raise engineering standards.

Mentor junior engineers and contribute to technical roadmaps.

Influence data strategy and platform evolution.

Qualifications

Expert knowledge of Microsoft’s BI stack, namely Azure SQL Database/Azure Synapse, Azure Analysis Services, ADF, Databricks, DevOps. Expertise with Delta Lakes, PowerBI, and Python preferred.

Capable of systematically and rationally analyzing the root cause of problems

Experienced with Agile methodology, and the promotion & facilitation of respective technical deliveries. Includes estimation and project plan creation for the timely delivery of milestones, analysis, and presentation of written and/or oral information, prioritization, and planning.

Self-starter, takes responsibility for driving his/her portfolio of tasks and projects

Willingness to challenge the status quo and to drive innovation and change, being authentic, approachable, open and honest, having a high commitment and positive attitude in the face of setbacks and obstacles, and being capable of generating creative ideas/solutions

Comfortable in international business communication, able to communicate openly and confidently, able to listen effectively, and able to influence and convince others in a way that results in acceptance and agreement

Ability to work with different cultures and to manage cross-functional interfaces, high priority for teamwork, capable of building strong and long-term team relationships within and across functions and geographical boundaries, and based on ethics and trust, ready to take personal accountability for achieving individual and shared goals

Bachelor’s or Master’s degree in Computer Sciences

6-10 years of experience in BI projects, preferably in the Logistics industry

Proficient Level of English (spoken and written)

Expert knowledge in Tabular Models development and maintenance, and DAX Language.

Good knowledge of PowerShell Scripting. (Nice to Have)

Excellent communication and collaboration skills in English (written and oral).

Ability to work independently and as part of a team in a fast-paced, dynamic environment.

Strong problem-solving and analytical skills.

Technical skills

Core

Databricks (Workflows, Delta Lake, Unity Catalog)

Apache Spark (PySpark / Spark SQL)

Lakehouse architecture patterns

Data modeling (dimensional, analytical, medallion)

Azure (ADF, ADLS, Synapse) or AWS (S3, Glue, Redshift) or GCP

Infrastructure as Code (Terraform, ARM, CloudFormation) (Nice-to-Have)

Batch & streaming pipelines

CDC and incremental loading

Performance tuning & optimization

Data quality frameworks.

Governance & Security

Role-based access control

Lineage and metadata management

Data privacy and compliance principles

Nice to have skills

Streaming (Kafka, Event Hubs, Kinesis)

ML integration (MLflow, Feature Stores)

Power BI / Tableau / Looker integration

Cost optimization & FinOps practices

Domain experience (Logistics, Finance, Supply Chain, Sustainability) (Preferred)

Our offer

Permanent contract (with basic salary in 14 installments per year)

Private medical care

English courses - financial or co-financed by company

Food allowance (EUR 10.20, per working day)

Hybrid and flexible work (3 days in a week at the office)

Buddy

Good access to public transports

DSV - Global Transport and Logistics

Global transport and logistics - road, air, sea, rail freight and warehousing. Contact us today.

#J-18808-Ljbffr