Principal Software Engineer (Ruby/Java - Production Engineering)

Role: Principal Software Engineer (Ruby/Java - Production Engineering)

Location: Remote

Employment Type: 12 Months Contract

Rate: $80/hr on W2

Must-Have Skills

• AWS
• Ruby (Ruby on Rails) and Java
• Strong experience debugging production issues end-to-end (code → infrastructure → data → dependencies)

Job Summary

We are seeking an experienced Principal Software Engineer to join a high-impact production engineering team focused on improving platform reliability, scalability, and operational excellence. This is a hands-on engineering role where you'll work closely with Product, SRE, Platform, and Operations teams to troubleshoot complex production issues and implement long-term solutions.

This is not a traditional support role. The ideal candidate enjoys solving challenging production problems, debugging distributed systems, and driving permanent improvements to platform stability.

Key Responsibilities

• Lead production incident triage across APIs, distributed services, payment workflows, infrastructure, and databases.
• Investigate and resolve complex production issues across application code, cloud infrastructure, data, and third-party integrations.
• Partner with engineering teams to identify root causes and implement permanent fixes.
• Improve system reliability through automation, code enhancements, monitoring, and observability.
• Develop and enhance:
• Monitoring and alerting strategies
• Operational tooling and automation
• Runbooks and diagnostic workflows
• Work across a technology stack that includes Ruby on Rails, Java, AWS, APIs, microservices, and SQL databases.
• Design highly observable, resilient, and maintainable systems.
• Mentor engineers and promote engineering best practices across development, SRE, and operations teams.

Required Qualifications

• 8+ years of experience in Software Engineering, Production Engineering, Site Reliability Engineering (SRE), or distributed systems.
• Strong hands-on experience troubleshooting production issues end-to-end.
• Experience with:
• Ruby on Rails
• Java
• AWS cloud environments
• APIs and microservices
• SQL and database investigations
• Observability tools such as Splunk, Datadog, New Relic, or similar
• Strong understanding of:
• Distributed systems
• Fault isolation
• Performance tuning
• Reliability and resiliency engineering
• Ability to perform effectively during production incidents and critical escalations.
• Excellent communication and collaboration skills.