\- Highly skilled and motivated senior data engineer/data analyst with a proven track record in designing, building, and maintaining large-scale data systems.
\- Expertise spans the entire data lifecycle: modeling, designing, ingestion, storage, processing, integration, visualization, and quality testing.
\- Proficient in architecting robust, scalable data pipelines using technologies like Azure Databricks, Data Lake Storage, Data Factory, Blob Storage, Python, PySpark, PyTest, SQL, and Great Expectations.
\- Experienced in data ingestion from diverse sources: structured databases (Oracle, SQL Server), semi-structured formats (JSON, XML), unstructured data (text, multimedia files), and REST APIs.
\- Strong skills in data transformation and processing, efficiently manipulating, cleansing, and transforming large volumes of data with PySpark.
\- Deep understanding of data warehousing concepts, data modeling techniques, and query optimization strategies, delivering optimized solutions for data storage and retrieval.
\- Well-versed in data quality assurance and implementing robust data governance practices to ensure data integrity, compliance, and security.
\- Excel at extracting valuable insights and patterns from complex datasets using advanced statistical techniques and data mining algorithms.
\- Skilled in creating visually compelling dashboards and reports with visualization tools and frameworks, effectively communicating data-driven insights to stakeholders for informed decision-making.
\- Ability to bridge the gap between data engineering and data analysis, bringing a comprehensive approach to projects and enabling organizations to derive actionable insights from their data.