Hire Remote Senior
Data Engineering Developers
in Canada

An industrious, astute, and technically-focused data engineer with hands-on experience in building analytics tools that utilize the data pipeline to provide actionable insights. Ability to deal well with ambiguity, prioritize needs, and deliver results in a dynamic environment. Passionate about finding solutions for challenging and complex data problems. Interested in working with Python, SQL, Airflow, dbt, AWS, Databricks, Snowflake.

Data EngineeringData ModelingApache AirflowGoogle Cloud PlatformApache SparkApache KafkaData WarehouseSQLPythonLinuxAWSBash

Over 6+ years of experience in Data Engineering, Data Pipeline Design, Development and Implementation as a Sr. Data Engineer and Data Developer. • Strong experience in Software Development Life Cycle (SDLC) including Requirements Analysis, Design Specification and Testing as per Cycle in both Waterfall and Agile methodologies. • Strong experience in writing scripts using Python API, PySpark API and Spark API for analyzing the data. • Experience with Azure Cloud, Azure Data Factory, Azure Data Lake Storage, Azure Synapse Analytics, Azure Analytical Service, Big Data Technologies (Apache Spark) • Experience with GCP Cloud Storage, Big Query, Composer, Cloud Dataproc, Cloud SQL, Cloud Functions, Cloud Pub/Sub • Worked on ETL Migration services by creating and deploying AWS Lambda functions to provide a serverless data pipeline that can be written to glue catalog and queried from Athena. • Extensively used Python Libraries PySpark, Pytest, Pymongo, PyExcel, Psycopg, embedPy, NumPy and Beautiful Soup. • Migrated an existing on-premises application to GCP. Used GCP services like Cloud Dataflow and Dataproc for small data sets processing and storage. • Hands On experience on Spark Core, Spark SQL, Spark Streaming and creating the Data Frames handle in SPARK with Scala. • Experience in NoSQL databases and worked on table row key design and to load and retrieve data for real time data processing and performance improvements based on data access patterns. • Experience with Unix/Linux systems with scripting experience and building data pipelines. • Extensive experience in Hadoop Architecture and various components such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node, and Map Reduce concepts. • Performed complex data analysis and provided critical reports to support various departments. • Expertise in development of various reports, dashboards using various PowerBI and Tableau Visualizations • Experience in building large scale highly available Web Applications. Working knowledge of web services and other integration patterns. • Experienced with version control systems like Git, GitHub, CVS, and SVN to keep the versions and configurations of the code organized. Efficiently facilitating task tracking and issue management using JIRA. • Good communication skills, work ethics and the ability to work in a team efficiently with good leadership skills. • Experience with continuous integration and automation using Jenkins. • Experience building docker image using accelerator and scanning images using various scanning techniques • Over 6+ years of experience in Data Engineering, Data Pipeline Design, Development and Implementation as a Sr. Data Engineer and Data Developer. • Strong experience in Software Development Life Cycle (SDLC) including Requirements Analysis, Design Specification and Testing as per Cycle in both Waterfall and Agile methodologies. • Strong experience in writing scripts using Python API, PySpark API and Spark API for analyzing the data. • Experience with Azure Cloud, Azure Data Factory, Azure Data Lake Storage, Azure Synapse Analytics, Azure Analytical Service, Big Data Technologies (Apache Spark) • Experience with GCP Cloud Storage, Big Query, Composer, Cloud Dataproc, Cloud SQL, Cloud Functions, Cloud Pub/Sub • Worked on ETL Migration services by creating and deploying AWS Lambda functions to provide a serverless data pipeline that can be written to glue catalog and queried from Athena. • Extensively used Python Libraries PySpark, Pytest, Pymongo, PyExcel, Psycopg, embedPy, NumPy and Beautiful Soup. • Migrated an existing on-premises application to GCP. Used GCP services like Cloud Dataflow and Dataproc for small data sets processing and storage. • Hands On experience on Spark Core, Spark SQL, Spark Streaming and creating the Data Frames handle in SPARK with Scala. • Experience in NoSQL databases and worked on table row key design and to load and retrieve data for real time data processing and performance improvements based on data access patterns. • Experience with Unix/Linux systems with scripting experience and building data pipelines. • Extensive experience in Hadoop Architecture and various components such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node, and Map Reduce concepts. • Performed complex data analysis and provided critical reports to support various departments. • Expertise in development of various reports, dashboards using various PowerBI and Tableau Visualizations • Experience in building large scale highly available Web Applications. Working knowledge of web services and other integration patterns. • Experienced with version control systems like Git, GitHub, CVS, and SVN to keep the versions and configurations of the code organized. Efficiently facilitating task tracking and issue management using JIRA. • Good communication skills, work ethics and the ability to work in a team efficiently with good leadership skills. • Experience with continuous integration and automation using Jenkins. • Experience building docker image using accelerator and scanning images using various scanning techniques

Data EngineeringETLData PipelinesApache AirflowApache SparkCloudSQLPythonScalaPower BIAzureGoogle BigQueryKubernetesSnowflake

As a skilled data engineer and passionate data science enthusiast, I take a practical approach to problem-solving and have a natural curiosity for discovering solutions. With over 15 years of experience in data analysis, visualization, and automation development, I possess a strong foundation in the field. My career initially started in the electronics and embedded/firmware programming industry, specifically within the cellular sector. After gaining valuable experience in startups and large corporations, I transitioned towards data roles. Throughout my professional journey, I have consistently focused on acquiring high-quality data, both in terms of quantity and quality. To achieve this, I have frequently leveraged my expertise in Excel VBA and Python to develop custom automation solutions. Additionally, I have honed my managerial skills by leading a team of five engineers, defining products, providing expert advice to internal teams, and supporting customers on current and legacy products. My recent experience includes working with cloud services, databases, Snowflake, Tableau, Tableau Server, and Python.

Data EngineeringData warehouseApache SparkGoogle Cloud PlatformPythonConfiguration managementAutomationExcel VBATableauSQLAmazon S3SnowflakeManagementWeb ScrapingGoogle BigQueryPython 3DatabricksOracleSAPServiceNowOracle APEXPostgreSQL