Hire Remote Senior
Data Engineering Developers
in Canada

An industrious, astute, and technically-focused data engineer with hands-on experience in building analytics tools that utilize the data pipeline to provide actionable insights. Ability to deal well with ambiguity, prioritize needs, and deliver results in a dynamic environment. Passionate about finding solutions for challenging and complex data problems. Interested in working with Python, SQL, Airflow, dbt, AWS, Databricks, Snowflake.

Data EngineeringData ModelingApache AirflowGoogle Cloud PlatformApache SparkApache KafkaData WarehouseSQLPythonLinuxAWSBash

Over 6+ years of experience in Data Engineering, Data Pipeline Design, Development and Implementation as a Sr. Data Engineer and Data Developer. • Strong experience in Software Development Life Cycle (SDLC) including Requirements Analysis, Design Specification and Testing as per Cycle in both Waterfall and Agile methodologies. • Strong experience in writing scripts using Python API, PySpark API and Spark API for analyzing the data. • Experience with Azure Cloud, Azure Data Factory, Azure Data Lake Storage, Azure Synapse Analytics, Azure Analytical Service, Big Data Technologies (Apache Spark) • Experience with GCP Cloud Storage, Big Query, Composer, Cloud Dataproc, Cloud SQL, Cloud Functions, Cloud Pub/Sub • Worked on ETL Migration services by creating and deploying AWS Lambda functions to provide a serverless data pipeline that can be written to glue catalog and queried from Athena. • Extensively used Python Libraries PySpark, Pytest, Pymongo, PyExcel, Psycopg, embedPy, NumPy and Beautiful Soup. • Migrated an existing on-premises application to GCP. Used GCP services like Cloud Dataflow and Dataproc for small data sets processing and storage. • Hands On experience on Spark Core, Spark SQL, Spark Streaming and creating the Data Frames handle in SPARK with Scala. • Experience in NoSQL databases and worked on table row key design and to load and retrieve data for real time data processing and performance improvements based on data access patterns. • Experience with Unix/Linux systems with scripting experience and building data pipelines. • Extensive experience in Hadoop Architecture and various components such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node, and Map Reduce concepts. • Performed complex data analysis and provided critical reports to support various departments. • Expertise in development of various reports, dashboards using various PowerBI and Tableau Visualizations • Experience in building large scale highly available Web Applications. Working knowledge of web services and other integration patterns. • Experienced with version control systems like Git, GitHub, CVS, and SVN to keep the versions and configurations of the code organized. Efficiently facilitating task tracking and issue management using JIRA. • Good communication skills, work ethics and the ability to work in a team efficiently with good leadership skills. • Experience with continuous integration and automation using Jenkins. • Experience building docker image using accelerator and scanning images using various scanning techniques • Over 6+ years of experience in Data Engineering, Data Pipeline Design, Development and Implementation as a Sr. Data Engineer and Data Developer. • Strong experience in Software Development Life Cycle (SDLC) including Requirements Analysis, Design Specification and Testing as per Cycle in both Waterfall and Agile methodologies. • Strong experience in writing scripts using Python API, PySpark API and Spark API for analyzing the data. • Experience with Azure Cloud, Azure Data Factory, Azure Data Lake Storage, Azure Synapse Analytics, Azure Analytical Service, Big Data Technologies (Apache Spark) • Experience with GCP Cloud Storage, Big Query, Composer, Cloud Dataproc, Cloud SQL, Cloud Functions, Cloud Pub/Sub • Worked on ETL Migration services by creating and deploying AWS Lambda functions to provide a serverless data pipeline that can be written to glue catalog and queried from Athena. • Extensively used Python Libraries PySpark, Pytest, Pymongo, PyExcel, Psycopg, embedPy, NumPy and Beautiful Soup. • Migrated an existing on-premises application to GCP. Used GCP services like Cloud Dataflow and Dataproc for small data sets processing and storage. • Hands On experience on Spark Core, Spark SQL, Spark Streaming and creating the Data Frames handle in SPARK with Scala. • Experience in NoSQL databases and worked on table row key design and to load and retrieve data for real time data processing and performance improvements based on data access patterns. • Experience with Unix/Linux systems with scripting experience and building data pipelines. • Extensive experience in Hadoop Architecture and various components such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node, and Map Reduce concepts. • Performed complex data analysis and provided critical reports to support various departments. • Expertise in development of various reports, dashboards using various PowerBI and Tableau Visualizations • Experience in building large scale highly available Web Applications. Working knowledge of web services and other integration patterns. • Experienced with version control systems like Git, GitHub, CVS, and SVN to keep the versions and configurations of the code organized. Efficiently facilitating task tracking and issue management using JIRA. • Good communication skills, work ethics and the ability to work in a team efficiently with good leadership skills. • Experience with continuous integration and automation using Jenkins. • Experience building docker image using accelerator and scanning images using various scanning techniques

Data EngineeringETLData PipelinesApache AirflowApache SparkCloudSQLPythonScalaPower BIAzureGoogle BigQueryKubernetesSnowflake

I am a software engineer with 9 years of experience in translating client requirements into end-to-end machine learning systems. Skilled in problem-solving, data modeling, and various technologies including Python, Scala, and Git.

Data EngineeringData ScienceData analysisApache SparkPrompt EngineeringAirflowAWS Cloud InfrastructurePythonMachine learningPython 3NLPGenerative AILinuxPandasCLINltkOpenAI APIsLLMMongoDBAngularFlaskSpringPostgreSQLNginxRedisRabbitMQTypeScriptKibanaKubernetesSpacyScalaAmazon S3KotlinJupyterTerraformAmazon athenaSkySpark