Direct message the job poster from Hyqoo Sr. Sourcing Specialist @ Hyqoo (formerly ClikSource) | MBA, Technical Recruiting Job Title: Data Operations Engineer Location: Remote (Colombia) Experience Level: Entry to Mid-level (3-5) Introduction: We are seeking a highly skilled and detail-oriented Data Operations Engineer to join our growing data platform team. In this role, you will play a crucial part in designing, building, and optimizing data infrastructure that supports our next-generation products and data-driven decision-making. You will develop scalable, secure, and high-performance data pipelines while ensuring effective integration with cloud platforms, big data tools, and CI/CD automation frameworks. The ideal candidate is technically strong, has hands-on experience with AWS, big data processing, and automation, and can collaborate with cross-functional teams with ease. Roles and Responsibilities: 1. Data Pipeline Building and Optimization: - Build and maintain scalable data pipeline architecture using Spark and orchestration tools like Airflow or AWS Step Functions. - Manage large, complex datasets to meet business and performance needs. - Identify, design, and implement process improvements to automate data ingestion, transformation, and loading. - Enhance reliability through pipeline monitoring, failure detection, and automated recovery. - Leverage Snowflake and dbt for optimized data warehousing and transformation workflows. 2. Data Platform and Self-Serving Infrastructure: - Design and implement robust infrastructure using AWS services such as EMR, S3, Lambda, Glue, RDS, and VPC. - Create abstractions and internal services that allow partners to work more autonomously with data. - Implement and maintain metadata, dependency management, and lifecycle processes for datasets. - Manage data access and security using IAM and role-based access control across cloud tools. 3. CI/CD and Infrastructure as Code: - Build and maintain CI/CD pipelines using tools such as Jenkins, GitHub Actions, or GitLab CI. - Automate provisioning and configuration of infrastructure using Terraform or AWS CloudFormation. - Ensure seamless and repeatable deployments for data applications and infrastructure components. 4. Monitoring and Observability: - Implement infrastructure and application monitoring using tools like CloudWatch, Datadog, ELK stack, or Prometheus. - Configure dashboards and alerts to support proactive performance tuning and troubleshooting. 5. Scripting & Programming: - Write modular and reusable scripts using Python, Bash, or Shell for data automation and orchestration. - Develop and maintain tools to support internal analytics and platform services. 6. (Nice to Have) Containerization & Orchestration: - Familiarity with Docker and Kubernetes for deploying and managing containerized workloads. - Experience with ECR/ECS or other container orchestration platforms is a plus. Qualifications: - Bachelor’s or Master’s degree in Computer Science, Engineering, Mathematics, or related field, or equivalent hands-on experience. - 3–5 years of experience in data engineering and cloud-based development using Python and AWS services. - Proficiency with AWS services like EMR, S3, Lambda, Glue, RDS, IAM, and VPC. - Strong expertise with Apache Spark and data pipeline orchestration using Airflow or AWS Step Functions. - Experience with Snowflake for warehousing, including performance tuning and user/role management. - Hands-on experience with dbt and cloud-native tools for data transformation and modeling. - Strong understanding of CI/CD pipelines and IaC tools such as Terraform or CloudFormation. - Proficient in scripting with Python, Bash, or Shell and writing automation scripts for data workflows. - Solid experience in monitoring, logging, and alerting frameworks for cloud infrastructure. - (Nice to have) Exposure to containerized environments and orchestration using Docker and Kubernetes. - Excellent communication, analytical, and problem-solving skills with a collaborative mindset. - Strong sense of ownership, with the ability to work independently and manage multiple priorities. Tools and Technologies: - AWS Services: EMR, S3, Lambda, Glue, RDS, IAM, VPC - Data Pipeline Tools: Apache Spark, Airflow, AWS Step Functions - Data Warehousing: Snowflake - Data Transformation: dbt - CI/CD Tools: Jenkins, GitHub Actions, GitLab CI - IaC Tools: Terraform, AWS CloudFormation - Scripting Languages: Python, Bash, Shell - (Nice to Have) Containerization: Docker, Kubernetes, ECR/ECS This position offers the opportunity to make a significant impact on our data-driven initiatives and contribute to the success of our organization. If you are passionate about data operations and eager to work in a dynamic environment, we encourage you to apply. Seniority level - Seniority level Mid-Senior level Employment type - Employment type Contract Job function - Job function Information Technology - Industries IT Services and IT Consulting Referrals increase your chances of interviewing at Hyqoo by 2x Get notified about new Data Engineer jobs in Colombia. Data Engineer (PySpark) - Technology (Latam) Bogota, D.C., Capital District, Colombia 1 week ago Bogota, D.C., Capital District, Colombia $24,000.00-$36,000.00 1 month ago Bogota, D.C., Capital District, Colombia 1 week ago Bogota, D.C., Capital District, Colombia 4 months ago Data Visualization Engineer (Latin America) Bogota, D.C., Capital District, Colombia 1 week ago Bogota, D.C., Capital District, Colombia 1 day ago We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI. #J-18808-Ljbffr