Personal details

Bharath - Remote data engineer

Bharath

Based in: 🇮🇳 India
Timezone: Chennai (UTC+5.5)

About

A Databricks certified Associate Data engineer, having proficiency in building end to end data platform. Expert in data integration from various sources, architecting data pipelines, real time streaming & batch ETL using medallion architecture, data wrangling, designing Python/Spark & Hadoop applications, carving REST API’s, GraphQL’s. Experience in data modeling, participated in dimensional design discussions & product features, rendering technical inputs & driving product development. Exposure to ELK stack. Have improved performance of several Logstash DB queries by 40%. Worked on multiple clouds AWS & Azure and have exposure to Gitlab CI/CD, Terraform. Hands On in Databricks data Lakehouse, Unity catalog, management of resources and warehouses Snowflake & RDS.

Work Experience

Software Engineer III (Lead)
Arcadia | Apr 2023 - Jul 2026
SQL
Kubernetes
Datadog
AWS
  • Leveraged structured streaming/Autoloader for ingesting power generation/consumption data from utilities. Reduced wait time for business reports by a day. ï‚·
  • Orchestrated data ingestion via Kubernetes by assigning dedicated pods & priority queues. Reduced wait times by more than 60%. ï‚·
  • Enabled snowflake/Datadog dashboards for pipeline monitoring.
  • Authored number of PL’s using Synapse.
Sr Data Engineer
A5 Labs | Dec 2021 - Apr 2023
Python
SQL
MongoDB
Big Data
Docker
Apache Spark
Databricks
Big data technologies
AWS
Pyspark mllib
  • Architected number of data pipelines batch & streaming from a variety of sources using DLT, Autoloader in AWS Databricks (Both Spark SQL & PySpark) ï‚·
  • Developed GraphQL (AWS Appsync & Ariadne) resolvers. ï‚·
  • Stepped up to own backend infra involving Docker, Kubernetes, Argo, Kubectl for more than 6 months. Created Argo DAG’s, Gitlab CICD pipelines, learnt & implemented Docker in Docker services.

Education

K.S.Rangasamy College of Technology
Bachelor's degree・Electrical, Electronics and Communications Engineering
Jan 2003 - Jan 2007
Vairams Matric Higher Secondary School
・Higher Secondary, Biology/Biological Sciences, General
Jan 1998 - Jan 2003

Certifications & Awards

Databricks Certified Data Engineer Associate
Databricks | Nov 2023
SQL
HackerRank | Sep 2020