Data Engineer

Job Description

Must Have

• Python programming (Advance)

• Various Data Matching Algorithms (String Similarity based, distance-based, phonetics algorithm)

• Basic Graph DB / Vector DB knowledge

• ML-based solution approach to define/ refine data matching criteria

• Azure platform ( Databricks / Fabric / OpenAI etc.)

Good To Have

• API programming (FastAPI or Flask)

• We have a need for two Azure data engineers for a master data project. One person should be a senior lead engineer almost at architect level and second could be mid to senior engineer.

• The project requires data acquisition into Azure, creating data pipelines, normalize JSON data into relational format.

• Model and design table structures

• Analyze data and identify relationships,

• Match the data with another dataset to find patterns and relationship

• Create combined datasets by using various matching and merging techniques and logics

• Build APIs to search the data

• Build lightweight UI to enable search functionality for users

• Main idea is we need self starter who can work independently and figure out the solution on their own, work with client team, discuss and modify accordingly.