Data engineer with significant experience designing end-to-end solutions from ETL pipelines, APIs, PySpark notebooks, databases, and other software products. I am highly dedicated and driven to exceed management expectations. I am seeking an opportunity to showcase my skills and solve data challenges.
· Work with key stakeholders to design a data strategy that aligns with business objectives.
· Create data pipelines in Airflow to consolidate data from multiple sources into a centralized cloud DW.
· Create ETL processes to transform and clean data into denormalized tables in Power BI.
· Use data to provide actionable insights and KPIs for management to optimize business operations.
· Successfully migrated to an Azure Synapse Analytics DW to improve performance, functionality, and costs.
· Researched and designed a new data model based on Kimbell best-practices that combined multiple data sources, eliminated data silos across the organization, and simplified the report creation process.
· Developed data pipelines in Azure Data Factory to orchestrate the ETL process, conduct data quality testing via SQL scripts, and log all metadata for trouble-shooting and compliance purposes.
· Used CI/CD tools & Git in Azure DevOps to publish code following agile software development principles.