Personal details

Deepa S. - Remote data engineer

Deepa S.

Based in: 🇮🇳 India
Timezone: New Delhi (UTC+5.5)

Summary

As a Software Engineer II at Microsoft, I work on data engineering projects that leverage cloud and machine learning technologies. I have developed a generic ingestion framework in Azure Data Factory for ingesting data from various sources, saving time and compute resources. I have also built an application to convert natural language to SQL, and a near real-time ingestion framework. I have experience in deploying Flask and Django web apps on AWS, and creating CICD pipelines for Azure Data Factory.

Before joining Microsoft, I was a Consultant at Deloitte India, where I worked on ETL development and data migration projects using SAP BODS, Datastage, and databases like Teradata, SAP S4 HANA, and SQL Server.

Work Experience

Software Engineer II
Microsoft | Jun 2021 - Present
Python
SQL
Azure
ETL
Data Science
Apache Spark
Data analytics
SAP BODS
Apache Hive
Azure Data Factory
Data Engineer in Supply Chain 1. Developed Generic Ingestion Framework in Azure Data Factory for ingesting data from various sources (API,SQL sever,Blob,Fileshare) which saves a lot of future development effort. Since the pipelines are metadata driven there is no development required to onboard new sources in future. These pipelines only extract data from source if there is an update in the source . Hence it saves a lot compute. 2. Build an application to convert Natural Language to SQL. 3. Working on building a Near Real Time Ingestion Framework. 4. Developed CICD pipeline for Azure DataFactory. 5. Enhanced the generic scala logic to handle scd logic / transformation. 6. As part of DML retirement (which was a financial and tech debt),developed SAP BODS job to move data from SAP to Azure Containers for multiple datasets and then merged the files. 7. Have contributed in DRI and Livesite activities to maintain reliability and availability of Systems. 8. Have been a Scrum Master and Created ADO Dashboard to Track all the Features, User Stories, Task and other relevant metrics for Leadership. Data Science: 1. Ingested data related to Surface device (Microsoft laptops) and Xbox from Twitter and did sentiment analysis on it . 2. SCE Hackathon Winner 2022: Worked on converting text in books to audio which would be integrated with Spectacles to help people with Vision issues. 3. Suggested best practices in Machine Learning through Experiments (IDEAS team). 4. Highlighted Features of Responsible AI Dashboard (raiwidgets library) And Fairlearn. 5. Built an application which converts Natural Language to SQL. Python/Automation 1. Automated Mapping sheet creation of ETL Job which saves 50 hours of effort per project. 2. Automated creation of BODS Job Schedules. 3. Wrote a code to analyze Job logs for File transfer errors which saved a lot of manual effort. 4. Automated ICM incident creation Process using Power Automate
Software Engineer III
Walmart | Mar 2021 - Jul 2023
SAP BODS
Databricks
1. Worked on migrating Data from ECC to S4 HANA. 2. Tested an Internal Data Profiling tool

Education

SRM University
Bachelor's degree・B.Tech
Jul 2010 - Jul 2014