Databricks Engineer

Our client is running a modern data platform built on AWS and actively moving through a major Databricks implementation. The core objective is transforming data into actionable business products for marketing and promotions. The project team needs a seasoned Senior Data Engineer to help lead the charge, stabilize key deliverables, and guide our technical strategy. This is a critical engagement on a high-visibility, fast-paced project where both strong technical hands-on capability and sharp soft skills are essential.

This is a 6 month contract based in North Toronto and requires 5 days onsite.

Key Responsibilities:
• Lead the design, development, and operation of scalable, cloud-based data platforms supporting analytics and machine learning workloads.
• Design and maintain high-quality data pipelines (ETL/ELT) with strong standards for reliability, observability, metadata, and lineage.
• Build and operate Databricks-based data engineering solutions, including Spark, SparkSQL, and Delta Lake pipelines.
• Enable and support MLOps workflows, including data pipelines, feature engineering, and integration with AWS SageMaker for model training, deployment, and monitoring.
• Collaborate closely with Data Science and ML Engineering teams to transition models from experimentation to production.
• Work with Data Governance and Data Management teams to ensure compliance with enterprise standards and audit requirements.
• Implement and maintain data quality frameworks, monitoring, and automated validation checks.
• Define and maintain enterprise data assets, data models, and transformation logic.
• Perform advanced data analysis to troubleshoot complex data issues and production incidents.
• Contribute to CI/CD pipelines for data and analytics workloads, including testing and deployment automation.
• Provide technical mentorship to data engineers, promoting engineering best practices and code quality.
• Evaluate and recommend tools and technologies for data lineage, observability, and platform scalability.
• Partner with architecture and platform teams to define long-term data and ML platform strategy.
• Produce clear documentation and contribute to shared engineering standards and knowledge bases.

Required Skills & Qualifications:
• 5+ years of professional experience in data engineering or data platform development.
• Advanced proficiency in Python and SQL.
• Strong experience building data platforms on AWS, including services such as Lambda, Glue, Redshift, Step Functions, CloudFormation, Athena, and related services.
• Hands-on experience with Databricks, including Spark, Delta Lake, notebook-driven development, and production pipelines.
• Experience enabling or supporting MLOps, including integration with AWS SageMaker for model lifecycle management.
• Experience delivering CI/CD pipelines for data and analytics systems.
• Strong understanding of data modeling (relational, dimensional) and performance optimization.
• Experience implementing data quality, monitoring, and observability solutions.
• Proficient with source control tools such as Git and collaborative development practices.
• Working knowledge of Linux and scripting.
• Experience with file formats (CSV, JSON, XML) and diverse data integration patterns.
• Strong understanding of cloud-native architectures and emerging data technologies.
• Experience managing technical debt, refactoring, and evolving systems over time.
• Ability to balance short-term delivery with long-term architectural considerations.
• Experience as a DBA or with Salesforce (SFDC) is an asset.
• Bachelor’s degree in Computer Science, Engineering, or equivalent practical experience.