Senior Data Engineering Specialist

SENIOR DATA ENGINEERING SPECIALIST — GOOGLE CLOUD

ABOUT THE ROLE

We are looking for a professional to act as Tech Lead on a strategic data platform migration project for a major financial institution.

The challenge involves migrating the Cloudera environment on AWS to Google Cloud, with approximately 13,000 Spark and PySpark jobs and 2.5 PB of data. The position will play a central role in technical leadership of teams and delivery quality.

This position is also open to people with disabilities.

ROLE RESPONSIBILITIES

• Provide technical leadership to engineering teams, assign activities, and review deliverables.
• Lead the migration and deployment of Spark and PySpark jobs, Hive and Impala SQL queries, and streaming workflows on Google Cloud.
• Use AI-assisted conversion accelerators, parity testing, and evidence validation.
• Ensure consistency between source and target environments, performing reprocessing when necessary.
• Solve complex technical challenges and support adaptation of client frameworks.
• Define coding standards, peer reviews, and CI/CD pipelines.
• Facilitate communication between business areas and engineering.
• Support planning, estimates, and tracking of migration waves.

REQUIREMENTS

• Completed higher education in Technology, Engineering, or related fields.
• Minimum of 8 years of experience in data engineering.
• At least 3 years working as a Tech Lead or technical leader.
• Minimum of 3 years of experience with Google Cloud.
• Experience in data migration or data platform projects.
• Availability for 100% on-site work in São Paulo.

BEHAVIORAL COMPETENCIES

• Leadership by example and willingness to solve complex challenges.
• Organization and ability to prioritize demands from multiple teams.
• Clear communication with technical teams and business areas.
• Commitment to quality and accountability for production deliverables.
• Balance and focus to work under pressure and during deployment windows.

TECHNICAL COMPETENCIES

• Proficiency in Spark, PySpark, and advanced SQL.
• Experience with Google Cloud: Dataproc, BigQuery, Cloud Storage, and Composer.
• Knowledge of the Hadoop ecosystem: Hive, Impala, and HDFS, preferably in Cloudera.
• Experience with Git and CI/CD pipelines, such as GitLab CI or similar.

Preferred qualifications: Google Cloud Professional Data Engineer certification; experience with generative AI applied to code conversion or generation; Kafka, Flink, CDC, and Apache Iceberg; and experience in financial institutions.

WORK MODEL

• Employment type: CLT + various benefits.
• Salary: negotiable.
• Format: 100% on-site.
• Location: Avenida Paulista, São Paulo/SP.