Principal Software Engineer, Catalog & Real-Time Serving Systems

We're transforming the grocery industry

At Instacart, we invite the world to share love through food because we believe everyone should have access to the food they love and more time to enjoy it together. Where others see a simple need for grocery delivery, we see exciting complexity and endless opportunity to serve the varied needs of our community. We work to deliver an essential service that customers rely on to get their groceries and household goods, while also offering safe and flexible earnings opportunities to Instacart Personal Shoppers.

Instacart has become a lifeline for millions of people, and we’re building the team to help push our shopping cart forward. If you’re ready to do the best work of your life, come join our table.

Instacart is a Flex First team

There’s no one-size fits all approach to how we do our best work. Our employees have the flexibility to choose where they do their best work—whether it’s from home, an office, or your favorite coffee shop—while staying connected and building community through regular in-person events. Learn more about our flexible approach to where we work.

Overview

About the Role

We are seeking a highly experienced and visionary Sr. Staff or Principal Engineer to join our Customers organization. This role is crucial for the evolution and scalability of our core Catalog and data intensive systems, while also playing a pivotal role in advancing our Machine Learning serving and serving infrastructure capabilities. This position will not only impact core business functions and drive significant revenue but also shape the future of our personalized, real-time ML-driven experiences. The ideal candidate will possess deep expertise in distributed systems, streaming processing, data intensive applications, and particularly, the deployment, scaling, and optimization of Machine Learning models in production.

This is a unique opportunity to join a dynamic and innovative team, and to make a significant impact on the future of our platform by advancing both our core data infrastructure and our Machine Learning capabilities. If you are a highly motivated and experienced engineer with a passion for solving complex technical challenges across distributed systems, data engineering, and ML serving, we encourage you to apply.

About The Team

Join a dynamic team at the heart of Instacart's success, leading the core shopping experience that millions of users rely on. We are obsessive about perfecting every aspect of the customer shopping journey on the app, encompassing UX formats, feeds, algorithms, personalization, recommender engines, and ranking systems to deliver an exceptional experience. Our team thrives on collaborative problem-solving in a fast-paced environment.

About The Job

• Provide architectural leadership for Catalog, streaming, and data-intensive systems, emphasizing ML serving infrastructure and best practices, and drive the technical roadmap.
• Design, build, and scale reliable, efficient, and adaptable solutions to address changing business and ML needs.
• Lead the development and optimization of ML serving endpoints, ensuring high availability, low latency, robust performance, and implement fail-fast input validations and track metrics using Datadog.
• Centralize ML serving logic and decouple it from product applications to improve debugging, manageability, and system performance.
• Drive and contribute to company-wide transformational initiatives, impacting key business metrics like revenue, personalization, and operational efficiency, and influence the direction of ML infrastructure including real-time inferencing.
• Serve as a subject matter expert for Catalog, streaming, data-intensive, and ML serving technologies, providing guidance and mentorship to engineering and data science teams.
• Identify and implement innovative solutions to optimize system performance, reduce costs, and improve data processing and ML serving latency.
• Collaborate with cross-functional teams, including Product, Retailer, IC App, Ads, ML Infrastructure, and Data Science, to deliver integrated ML-driven solutions, and lead incident response and resolution for high-severity issues.

About You

_Minimum Qualifications

_

• Extensive experience in software engineering, with a focus on distributed systems, streaming processing (e.g., Flink), data intensive applications, and particularly, Machine Learning serving and deployment.
• Proven track record of designing, implementing, and scaling large-scale, high-performance systems, including ML serving infrastructure.
• Deep understanding of database technologies, data modeling, data pipelines, and ML model deployment patterns.
• Strong architectural skills and the ability to design and evaluate complex technical solutions across diverse technology domains, including Catalog, Streaming, and Machine Learning.
• Excellent problem-solving and debugging skills, with specific experience in addressing issues related to ML model serving, data quality, and infrastructure stability.
• Strong communication and collaboration skills, with the ability to effectively work across teams, influence stakeholders, and mentor junior engineers.
• Experience with cloud platforms and related technologies, including ML serving platforms (e.g., Sagemaker).
• Ability to quantify and demonstrate the impact of technical contributions on business results (e.g., revenue, efficiency, cost savings, and ML model performance).
• Familiarity with challenges related to ML lifecycle, data flow, and best practices

Preferred Qualifications

• Experience working with large-scale catalog systems or similar data-intensive platforms.
• Significant experience in designing and implementing high-throughput, low-latency ML serving systems.
• Contributions to open-source projects or technical publications related to distributed systems, data engineering, or Machine Learning serving.
• Experience in a high-growth, fast-paced environment, particularly in the context of scaling ML initiatives.

Instacart provides highly market-competitive compensation and benefits in each location where our employees work. This role is remote and the base pay range for a successful candidate is dependent on their permanent work location. Please review our Flex First remote work policy here.

Offers may vary based on many factors, such as candidate experience and skills required for the role. Additionally, this role is eligible for a new hire equity grant as well as annual refresh grants. Please read more about our benefits offerings here.

For US based candidates, the base pay ranges for a successful candidate are listed below.

CA, NY, CT, NJ

$335,000—$372,000 USD

WA

$321,000—$357,000 USD

OR, DE, ME, MA, MD, NH, RI, VT, DC, PA, VA, CO, TX, IL, HI

$308,000—$342,000 USD

All other states

$278,000—$309,000 USD