Senior Cloudera Specialist (Big Data and Google Cloud)

ABOUT THE ROLE

We are looking for a professional to work as a Senior Cloudera Specialist on a strategic data platform migration project for a major financial institution.

The challenge involves migrating the Cloudera environment from AWS to Google Cloud, with approximately 13,000 Spark and PySpark jobs and 2.5 PB of data. The professional will be the technical reference for the legacy environment, helping to preserve component behavior during the migration.

The position is also open to people with disabilities.

RESPONSIBILITIES

Analyze the Cloudera CDP environment, its services, configurations, dependencies, and production jobs.

Support the inventory and dependency mapping between sources, jobs, tables, and consumers.

Map Cloudera resources and identify equivalences or adjustments needed in Google Cloud.

Support the migration of Spark, Hive, Impala, Kudu, HBase, Kafka, and NiFi, including legacy versions of Spark.

Investigate differences between source and target environments and identify their causes.

Support the migration of security and governance policies, including Ranger, Atlas, and Kerberos.

Collaborate with client specialists to understand proprietary frameworks built on Cloudera.

Document legacy environment knowledge for engineering teams.

REQUIREMENTS

Completed higher education in Technology, Engineering, or related fields.

Minimum of 6 years of experience with the Hadoop ecosystem, including at least 4 years with Cloudera CDH or CDP.

Minimum of 1 year of experience with Google Cloud in data projects or environments.

Availability to work 100% on-site in São Paulo and during implementation windows.

BEHAVIORAL COMPETENCIES

Ability to explain the legacy environment to teams and specialists at different levels.

Ability to investigate problems in a structured way, with a focus on root cause.

Close collaboration with client specialists.

Organization in documentation and sharing technical knowledge.

TECHNICAL COMPETENCIES

Experience with Cloudera CDH or CDP in production and with the Hadoop ecosystem.

Proficiency in Spark, PySpark, Hive, and Impala in large-scale production environments.

Experience with Kudu, HBase, Kafka, or NiFi.

Knowledge of security and governance in Cloudera: Ranger, Atlas, and Kerberos.

Experience with Google Cloud: Dataproc, BigQuery, and Cloud Storage.

Nice to have: experience migrating Cloudera or Hadoop to the cloud; Cloudera or Google Cloud certification; experience with Cloudera Data Engineering (CDE), Airflow, Apache Flink, and CDC (Debezium); and experience in financial institutions.

WORK MODEL

Employment type: CLT + benefits.

Salary: negotiable.

Format: 100% on-site.

Location: financial institution on Avenida Paulista, São Paulo/SP.