Data Pipeline Engineer
Position Overview
The Data Pipeline Engineer will be responsible for designing, implementing, and managing CI/CD pipelines and cloud-based infrastructure. Working closely with our Data Architect, you’ll focus on deploying reliable, scalable cloud solutions while ensuring high operational standards. Your work will emphasize automation and continuous integration, supporting seamless data and infrastructure deployments that drive business innovation.
Duties and Responsibilities
• Pipeline Development & Management: Design, build, and optimize scalable data pipelines to support data integration and analytics
• Infrastructure as Code (IaC): Use Terraform to manage and automate cloud infrastructure deployments, particularly on Azure and Snowflake
• ETL/ELT Implementation: Utilize Azure Data Factory, DBT, Fivetran and Airflow to facilitate efficient data extraction, transformation, and loading
• DevOps & CI/CD: Support CI/CD pipelines, automated notifications, and version control to accelerate deployment processes
• Collaboration & Documentation: Work with cross-functional teams to ensure data solutions meet business requirements and maintain comprehensive process documentation
Qualifications
• 3+ years in cloud engineering or data pipeline engineering
• Experience with Snowflake or another leading cloud data warehouse
• Solid understanding of DevOps and CI/CD best processes
• Proficient with Terraform for IaC
• Hands-on experience with Azure services, particularly Azure Data Factory, and Azure DevOps
• Strong knowledge of data integration tools (DBT, Fivetran, Airflow)
• The ability to multi-task and be a strong team player in a busy IT environment
Nice to have
• Familiarity with monitoring and logging tools in cloud environments
• Knowledge of data security and compliance standards
Autres détails
- Type de paie Salaire
- 100 Queens Quay E, Toronto, ON M5E 1Z2, Canada