Tech Lead for Data Engineering (Google Cloud and Accelerators)

JOB OVERVIEW

We are looking for a professional to work as a Tech Lead on a strategic data platform migration project for a major financial institution.

The challenge involves migrating the Cloudera environment on AWS to Google Cloud, with approximately 13,000 Spark and PySpark jobs and 2.5 PB of data. The position will play a central role in the technical leadership of the teams, the use of migration accelerators, and the quality of production deliveries.

The position is also open to people with disabilities.

JOB RESPONSIBILITIES

Provide technical leadership to engineering teams, assign activities, and review deliverables.

Lead the migration and deployment of Spark and PySpark jobs, Hive and Impala SQL queries, and streaming workflows on Google Cloud.

Use AI-assisted conversion accelerators, parity testing, and evidence validation.

Ensure consistency between source and target environments, performing reprocessing when necessary.

Resolve complex technical challenges and support adaptation of the client's frameworks.

Define code standards, peer reviews, and CI/CD pipelines.

Facilitate communication between business areas and engineering.

Support planning, estimates, and tracking of migration waves.

REQUIREMENTS

Completed higher education in Technology, Engineering, or related fields.

Minimum of 8 years of experience in data engineering.

At least 3 years working as a Tech Lead or technical leader.

Minimum of 3 years of experience with Google Cloud.

Experience in data migration or data platform projects.

Availability for 100% on-site work in São Paulo.

BEHAVIORAL COMPETENCIES

Leadership by example and willingness to solve complex challenges.

Organization and ability to prioritize demands from multiple teams.

Clear communication with technical teams and business areas.

Commitment to quality and accountability for production deliveries.

Balance and focus to work under pressure and during deployment windows.

TECHNICAL COMPETENCIES

Proficiency in Spark, PySpark, and advanced SQL.

Experience with Google Cloud: Dataproc, BigQuery, Cloud Storage, and Composer.

Knowledge of the Hadoop ecosystem: Hive, Impala, and HDFS, preferably in Cloudera.

Experience with Git and CI/CD pipelines, such as GitLab CI or similar.

Nice to have: Google Cloud Professional Data Engineer certification; experience with generative AI applied to code conversion or generation; Kafka, Flink, CDC, and Apache Iceberg; and experience in financial institutions.

WORK MODEL

Employment type: CLT + benefits.

Salary: negotiable.

Format: 100% on-site.

Location: financial institution on Avenida Paulista, São Paulo/SP