Data Engineer / DataOps Specialist

About the Role

We are looking for a Data Engineer / DataOps professional to support the implementation and evolution of the DataOps platform within the Analytics area.

You will be responsible for developing, deploying, and maintaining the infrastructure required to execute batch and real-time data pipelines, as well as designing scalable, resilient, and observable data pipelines.

Key Responsibilities

• Support the implementation and evolution of the DataOps platform within the Analytics area;
• Develop, deploy, and maintain the infrastructure for executing batch and real-time data pipelines;
• Design and evolve scalable, resilient, and observable data pipelines;
• Automate infrastructure processes using Infrastructure as Code (IaC);
• Deploy and manage workloads in an AWS environment, preferably using managed services;
• Collaborate with Analytics and Infrastructure teams to define architectures, standards, and best practices;
• Monitor, optimize, and ensure the availability, security, and performance of the data platform;
• Contribute proposals for continuous improvement regarding architecture, tools, and processes.

Requirements

• Bachelor's degree in Computer Science, Engineering, Information Technology, or related fields;
• Robust experience with AWS environments;
• Knowledge of data platform architecture, observability, automation, and DataOps best practices;
• Experience with Infrastructure as Code, using Terraform and/or CloudFormation;
• Solid experience with query engines, such as Trino;
• Relational databases, especially PostgreSQL;
• Streaming and real-time data processing tools, such as Apache Kafka and Apache Flink;
• Data transformation tools, such as Glue and DBT;
• Architectures based on Data Lake/Lakehouse and table formats like Apache Iceberg;
• Pipeline orchestration using Dagster, Apache Airflow, or similar tools;
• Version control with Git and Bitbucket.

Preferred Qualifications

• Experience with distributed computing using Ray;
• NoSQL or specialized databases, such as Redis;
• Experience with graph databases, such as Neo4j, ArcadeDB, or similar;
• Knowledge of identity and access management solutions (Keycloak, Ping Identity, or similar);
• Experience with Disaster Recovery (DR) strategies in multi-cloud environments;
• Experience with Google Cloud Platform (GCP);
• Advanced English proficiency.