Principal Software Engineer - OpenSearch

Business Area:

Engineering

Seniority Level:

Director

Job Description:

At Cloudera, we empower people to transform complex data into clear and actionable insights. With as much data under management as the hyperscalers, we're the preferred data partner for the top companies in almost every industry. Powered by the relentless innovation of the open source community, Cloudera advances digital transformation for the world’s largest enterprises.

As a Principal OpenSearch Engineer, you will be the foremost technical authority and visionary for the search heartbeat of the Cloudera Data Platform (CDP). You will not only drive the technical vision but also ensure the strategic evolution of a high-performance, scalable, and secure search infrastructure that powers mission-critical data discovery and analytics for global enterprises. You will solve the most complex, high-impact technical problems that span multiple organizational boundaries.

Key Responsibilities

• Strategic Technical Leadership: Drive the multi-year technical roadmap and architectural vision for search services, ensuring they are secure, cost-efficient, and highly scalable for petabyte-scale enterprise workloads.
• Architectural Oversight: Design and oversee the implementation of large-scale OpenSearch clusters, ensuring low-latency indexing and query performance across multi-cloud environments.
• Deep Platform Integration: Lead the deep integration of OpenSearch with core CDP components like Apache Iceberg, SDX, and Ozone to provide a unified, secure search experience.
• Engineering Excellence: Define and champion organization-wide best practices for security (RBAC, TLS), observability, and enterprise-grade reliability.
• Complex Problem Solving: Design solutions for extreme scale, multi-tenancy isolation, and complex hybrid/multi-cloud deployment models.
• Mentorship & Influence: Mentor staff and senior engineers on complex technical designs and foster a culture of continuous improvement and innovation across the entire engineering organization.
• Open Source Stewardship: Act as a primary influencer and liaison within the upstream OpenSearch community, aligning their roadmap with Cloudera's product strategy.

Technical Requirements

• Search Expertise: 10+ years of experience (scaling from the 8+ years required for Staff level) working with OpenSearch or Elasticsearch in large-scale production environments.
• Distributed Systems Mastery: Deep understanding of consensus algorithms, CAP theorem, replication, sharding, and fault-tolerant design patterns.
• Mastery in Programming: Mastery of Java for core development and proficiency in Go or Python for advanced automation and tooling.
• Infrastructure & Cloud: Extensive experience with Kubernetes (K8s) and container orchestration, including deploying search workloads on AWS (EKS/AOSS), Azure (AKS), or Google Cloud (GKE).
• Systems Thinking: Demonstrated ability to drive architectural decisions with a focus on non-functional requirements such as high availability and security.

Preferred Qualifications

• Deep expertise in Lucene internals (e.g., segment merging, bitsets, and codecs)
• Advanced knowledge of Vector Database capabilities for Generative AI (RAG) and retrieval-augmented analytics.
• Significant history of contributions to open-source projects like the Apache Software Foundation or the OpenSearch Project

Why Cloudera?

At Cloudera, we believe data can make what is impossible today, possible tomorrow. We empower people to transform complex data into clear and actionable insights. You’ll work on a platform that handles more data than almost anyone else on the planet, surrounded by a team that values candor, innovation, and open-source integrity.

What you can expect from us:

• Generous PTO Policy
• Support work life balance with Unplugged Days
• Flexible WFH Policy
• Mental & Physical Wellness programs
• Phone and Internet Reimbursement program
• Access to Continued Career Development
• Comprehensive Benefits and Competitive Packages
• Paid Volunteer Time
• Employee Resource Groups

EEO/VEVRAA