Senior Data Engineer

Senior Data Software Engineer – ML & AI Data Platform

Experience: 7 – 12 Years

Shift Timing: 7:00 AM – 4:00 PM IST

Position Overview

We are looking for a Senior Data Software Engineer with strong expertise in Machine Learning and modern data architectures to join an AI-driven data platform initiative. The role involves building scalable data pipelines, intelligent services powered by large language models, and advanced data retrieval solutions.

Technology Stack

• Python
• Azure Infrastructure
• Azure AI Services
• ETL Pipelines
• Serverless Microservices
• Vector Databases
• Knowledge Graphs
• Retrieval Augmented Generation (RAG)
• Large Language Models (LLMs)
• Prompt Engineering

Responsibilities

• Develop and maintain scalable ETL pipelines for structured and unstructured data.
• Design and implement serverless microservices using cloud-native patterns.
• Build and optimize data solutions using vector databases and knowledge graphs.
• Collaborate with ML engineers to integrate large language model capabilities into production systems.
• Implement Retrieval Augmented Generation (RAG) pipelines for advanced AI use cases.
• Work with Azure AI Services to deliver reliable and scalable solutions.
• Ensure code quality through testing, code reviews, and adherence to engineering standards.
• Collaborate with distributed teams and participate in technical discussions across time zones.

Requirements

• 7+ years of experience in Data Engineering.
• 7+ years of experience building ETL pipelines.
• 7+ years of experience with Python development.
• 3+ years of experience with Azure AI Services.
• 2+ years of experience with Retrieval Augmented Generation (RAG).
• Hands-on experience with ETL pipeline design and implementation.
• Familiarity with Azure infrastructure and cloud-native services.
• Understanding of microservices architecture and serverless computing.
• Experience working with vector databases and data indexing solutions.
• Knowledge of knowledge graphs and Retrieval Augmented Generation concepts.
• Experience integrating or working with large language models.
• Practical experience with prompt engineering.
• English proficiency sufficient for daily communication in an international team.

Nice to Have

• Experience with Azure AI Services in production environments.
• Exposure to distributed system design and scalability patterns.
• Knowledge of data governance and data quality practices.
• Experience working in globally distributed teams.