Data Engineer (Cloudera)

With just over 9 years of experience in the Portuguese market, we share our DNA with more than 200 workers and position our offer according to 3 lines of service:

Strategy (Outsourcing, NeXel, Team as a Service, Tech Academies);

Nearshore.

In ADENTIS, we focus on PEOPLE. This is our emotional salary:

Great Work-Life balance;

Very flexible organizational routine;

Health benefits (for you and your family);

Over 300 protocols to offer you great discounts in different areas;

Continuous professional development sponsored by our Training and Certification Department;

Regular feedback on your performance through a personalized plan;

Comprehensive career plan and progression involving assertive performance reviews.

Responsibilities:

Distributed Data Processing: Develop and optimize distributed data pipelines using Apache Spark, ensuring efficient processing of large datasets

Caching & Computation Optimization: Implement advanced caching techniques and optimize distributed computation workflows to improve data retrieval times and resource utilization

Partition Key Join Optimization: Optimize partition keys joins to speed up query processing and enhance overall system performance in distributed environments

SQL Query Optimization: Optimize SQL queries for better performance and scalability, particularly in large-scale, distributed database systems

Cloudera Ecosystem: If applicable, leverage the Cloudera platform to manage and scale data workflows in big data ecosystems, ensuring reliability and efficiency

Wants to learn more about DevOps to be autonomous, with the following DevOps responsibilities (5-10% of the time)

Requirements:

Expertise in Apache Spark and distributed computation techniques

Experience with advanced SQL optimization and caching strategies

Knowledge of the Cloudera ecosystem is a valuable asset for managing big data platforms

Ability to work independently and take initiative to solve technical problems

Excellent communication skills to effectively collaborate with technical and non-technical teams.

Version Control & Branching

Containerization & Orchestration

Automation & Monitoring

Available to go to the office twice per week

#J-18808-Ljbffr