Personal details
Mohammed K.
Summary
Hi,
I am a Senior Data Engineer with over 6 years of experience working on Python. I have worked on Knowledge Graphs (Neo4j, Spark, NetworkX, d3.js), OS programming, creating data pipelines (Airflow, PySpark, SQL, AWS), data ingestion APIs (Dask, Azure, FastAPI, Postgres, AsyncIO) and data analysis (Pandas, Seaborn, Matplotlib). I have also created multiple courses on Udemy on High Performance Computing in Python, Exploratory Data Anaysis, Functional Programming and Scalable Data Analysis. I also answer questions regularly on StackOverflow. I am currently in Top 5% of people who answer on StackOverflow.
Work Experience
- Responsible for designing and maintaining APIs/Data Pipelines for ingesting data into our Machine Learning models (FastAPI, Python, AsyncIO, Dask, Azure, MLFlow, Docker, PostgresSQL).
- Created tools and dashboards to monitor data ingestion pipelines for our customers. • Used MLFlow fortracking and serving models for different customers.
- Streamlined end-to-end Operations forimproved customer experience.
- Migrated our services from on-prem to Azure, which involved re-writing the deployment/monitoring scripts.
- Migrated CI/CD framework from Jenkins to Github Actions using pants.
- Mentorjunior engineers towards implementing efficient and clean code.
• Worked as a Data Engineer, and helped in setting up Apache Airflow DAGs for data ingestion and processing (using Pyspark).
• Responsible for end-to-end design and optimization of Airflow DAGs.
• Creating Dashboards for visualizing Streaming data using Apache Kafka
• Migrated cloud infrastructure from AWS to Microsoft Azure