Tekonika - Lead Data Engineer - Apache Spark

Job Title : Lead Data Engineer

Experience Required : 9 to 15 Years

Mode of Work : Remote

Job Description

We are seeking a seasoned Lead Data Engineer with deep technical expertise and leadership experience to join our team. The ideal candidate should possess a strong foundation in data engineering, a solid understanding of machine learning workflows, and hands-on experience with modern data platforms such as Databricks. This role will require close collaboration with client-side ML engineers, Product Owners, and other stakeholders to ensure seamless delivery and optimization of data and AI-driven solutions.

Key Responsibilities

• Provide technical leadership and guidance to clients on the current data and analytics landscape.
• Optimize and manage large-scale data pipelines, particularly using Apache Spark.
• Collaborate with client ML engineers to define and implement MLOps and LLMOps processes.
• Assist in productionizing Generative AI (GenAI) workflows; prior experience in GenAI is a plus, but strong knowledge is essential.
• Work extensively with Databricks for data engineering and analytics tasks.
• Partner with the clients Product Owner to identify resource requirements, manage team allocations, and plan project milestones and release schedules.
• Lead and mentor a team of data engineers, ensuring high-quality delivery and adherence to best practices.

Required Skills & Qualifications

• 9- 15 years of experience in Data Engineering, with strong hands-on technical skills.
• Proven experience with Apache Spark and performance optimization of data pipelines.
• Working knowledge or experience in MLOps, LLMOps, and GenAI workflows.
• Must have extensive hands-on experience with Databricks.
• Strong communication and stakeholder management skills.
• Experience in leading technical teams and handling project delivery in a collaborative environment.

Nice To Have

• Exposure to cloud platforms like Azure, AWS, or GCP.
• Familiarity with data governance and security best practices.
• Experience working in Agile environments.

(ref:hirist.tech)