Personal details

Nikhil B. - Remote data engineer

Nikhil B.

Timezone: Kolkata (UTC+5.5)

Summary

Resourceful and analytical individual with hands-on experience in and comprehensive knowledge of building data processing pipelines and applications. Able to manage coding/programming of big data applications. Capacity to design, build, install, configure, and support Hadoop development and implementation. Ability to create scalable and high‐performance web services for data tracking.

Work Experience

Senior Software Engineer
QBurst Technologies Private Limited | Dec 2018 - Present
SQL
Apache Spark
Provide exceptional services within data engineering team as a developer, producing IoT and streaming applications in line with service level agreement. Steer data engineering projects from conception to delivery with a focus on completing all tasks on time.
Software Engineer
Menerva Software Private Limited | Jul 2017 - Oct 2018
SQL
Apache Spark
Developer in the Data Analytics team of Menerva Software for various projects. Collaborate with marketing, UI Designers and other developers to develop data analytics applications without jeopardising data privacy or security.

Personal Projects

Azure Migration and ETL
2021
Python
Azure
Apache Spark
Azure Data Factory
Migrated the on-premises services and data pipelines into Azure. Designed and implemented various frameworks for data quality  and data movements.  Technologies: PySpark, Azure Data Factory, Azure Data Lake, Python, Azure Databricks Implemented transformation frameworks that can move data from various sources into Azure Data Lake. Designed and developed rule based Data Quality Framework. 
Document Indexing and Searching
2020
Python
Nltk
Data pipeline to collect and refresh data from various sources using AWS Services.  Technologies: Python, AWS Glue, AWS Comprehend, AWS Textract, AWS CloudSearch Retrieved data from various sources and saved them into a data warehouse.  Indexed the documents using NLP techniques and expose the data for fast lookup.