Abhishek A.

ETL Pipeline for End to End Specials Use Case Squad Spearheading the migration process from Hadoop cluster to Azure Databricks spark cluster. Ensuring optimal code performance in the spark cluster and constantly seeking out ways to improve it. Industrializing the code base to facilitate seamless scaling. Taking the lead in forming the CI/CD process for both Azure Data Factory and Azure Databricks. Collaborating closely with Data Scientists to provide them with accurate data to derive actionable business insights. RETAIL HUB DASHBOARD Developed a dynamic spark notebook to pivot the web crawling data efficiently. Integrated Azure Synapse Warehouse and Data Lake with ADF and created parameterised pipeline in Data Factory for an end-to end ETL process. Designing and developing Warehouse Objects and Stored Procedures for streamlined data processing. Developing a Logic app to send daily email

EPM ONYX BUSINESS REPORTING 1)Gathered requirements from stakeholders for Business Reporting. 2)Prepared Technical & Scope Analysis document for Data Models which includes Facts & Dimensions mapping. 3)Fetched Data from a Multidimensional source system(EPM) using API through Azure Data Factory to Built ETL Pipeline. 4)Built Full load and Delta load Pipeline in Azure Data Factory. 5)tored the raw data into Azure Data Lake in the Date Time folder structure. 6)Written transformation logics using Pyspark and SparkSQL in Databricks to convert the raw data into Facts and Dimensions. 7)Done Erwin Modelling using Erwin Studio and Implemented Semantic Data Model using Azure Analysis Services for various Analytics Power BI reports/dashboards. 8)Created a technical incident/challenge Document for effective communication across the team. MLT O2C DASHBOARD 1)Added a new report in existing order to the cash dashboard to track Market Lead Time(MLT). 2)Read the uploaded data through Logic App into Databricks from Azure Blob Storage. 3)Made required transformation in Databricks using SQL and Python and written the transformed data into curated layer of Azure Data Lake Gen2. 4)Read the Data into Azure Synapse and done Erwin Modelling using ER Studio and implemented Data Models in Azure Analysis Service for Power BI Visualization. Disaster Recovery and CI/CD Pipeline 1)Taken backups for Azure Databricks, Azure Data Factory, Azure Analysis Service using Databricks Command-Line(CLI), ARM templates, SSMS, and PowerShell. 2)Designed a CI/CD pipeline using Azure DevOps to automate the build and release processes across various environments for Azure Data Factory (ADF), Azure Synapse (Data Warehouse) & Azure Analysis Services.