Personal details

Rich S. - Remote full-stack developer

Rich S.

Based in: 🇹🇩 Canada
Timezone: Pacific Time (US & Canada) (UTC-7)

About

I'm a freelance staff software engineer with a focus on distributed systems. I've developed complex pipelines from scratch, turned around struggling solutions and teams, and delivered complex cross-system cross-team projects.

I'm available for coaching and freelance work! I'm generally available between 9AM and 6PM Pacific Time.

Work Experience

Staff Software Engineer
Lob Inc | Nov 2021 - May 2024
Node.js
TypeScript
Project management
Terraform
AWS Lambda
Golang
AWS

Responsible for the overall stability and direction of the Rendering platform, and the next generation ordering platform.

Managed multiple cross-team, cross-system multi-quarter projects including:

  • Order Failures - led the Lob staff engineer group in designing the ability to automatically fail orders across multiple systems and provide actionable information to the customer. This lead to a ~99% reduction in order defects and helped retain a critical enterprise customer.
  • Shadow Pipeline - key contributor to the design and implementation of a shadow ordering pipeline, allowing Lob to identify potential rendering fidelity issues across a wide variety of input creatives in advance of releasing code changes to production. This generated a >95% reduction in rendering fidelity incident frequency and $200K in annual savings.
  • Order Deduplication - designed the ability to identify orders with duplicate creatives and coordinate system and process changes so that print partners can only download one creative. This reduced S3 order costs by approximately 40% and virtually eliminated partner download time for these campaigns.

Iteratively improved the performance and cost effectiveness of a large distributed PDF rendering pipeline across a combination of AWS Lambdas and containerized services (primarily NodeJS, with some Go), handling highly variable input volumes and sources:

  • Reduced per unit rendering compute costs by 30-40% while maintaining a prompt rendering experience (99% rendered in under 60 seconds).
  • Reduced S3 costs by $250K/year.

Key contributor to the upcoming GA launch of the next generation ordering platform with a seamless API experience for existing customers.

Designed the core structure and patterns of the next generation PDF production engine, enabling other engineers to add new features sustainably and quickly.

Staff Software Consultant
Test Double | Aug 2021 - May 2022
Node.js
Project management
React
Golang

Provide coaching and guidance to client engineers, participated in training and development for Test Double consultants.

Projects

Shadow Pipeline
2022
Node.js
TypeScript
Terraform
Golang
AWS
Following a major production incident, I led the redesign of our PDF rendering pipeline's development process. As staff software engineer, I analyzed our complex distributed system that generates thousands of customer PDFs daily from varied inputs (HTML/PDF). Our existing test suite couldn't catch edge cases in this user-driven system, where failures meant direct business impact. I designed a shadow rendering pipeline that runs live customer orders against pre-production code, automatically comparing outputs to detect regressions. After creating comprehensive system design documentation and flowcharts, I socialized the solution with engineering teams and staff engineers, gaining unanimous approval. Over one month, I guided a team of 5-6 engineers through implementation while transitioning from CI/CD to controlled release windows. The key technical challenge was accounting for non-deterministic text anti-aliasing in our PNG comparisons—I developed tolerance thresholds that allowed for minor rendering differences while still catching meaningful regressions. The successful launch eliminated similar incidents, saving Lob $200K+ annually while restoring team confidence to make substantial changes without fear of production failures.
Order Deduplication
2023
Node.js
TypeScript
Terraform
Golang
AWS
At Lob, our PDF rendering system created a critical bottleneck as we scaled to enterprise print campaigns. Our system was designed to render each order's PDF in isolation, which worked fine for the initial transactional use case, but as Lob acquired customers with print campaigns, this became suboptimal because our print partners would need to download hundreds of thousands of PDFs per campaign, creating bottlenecks that delayed print jobs by 48 hours and threatened our 95% on-time delivery commitment. Engineering Leadership assigned me as technical lead to resolve this critical issue. I conducted discovery through code reviews and team coordination to understand unfamiliar systems. After analysis, I developed two solutions: Campaign Solution: Leverage campaigns service to link identical creatives together. Comprehensive Solution: Implement content-based deduplication using hash identifiers to detect previously processed creatives. Both designs would communicate identities to print partners, allowing them to reference existing files instead of re-downloading. I preferred the comprehensive approach for broader applicability, but we adopted the campaign solution to meet business timelines and reduce complexity. I completed solution design documentation, detailed project planning, and comprehensive testing strategy. The 8-week project required coordinating 8 engineers concurrently. After socializing the plan with wide approval, I led execution through daily standups and cross-team coordination on a system processing millions of orders monthly. Development completed successfully and on-time. Results: Reduced S3 costs by 30%, virtually eliminated download delays, and increased on-time delivery from 95% to 98%.

Education

Bitmaker Labs
Bootcamp・Coding Bootcamp
Sep 2014 - Dec 2014