Senior Data Engineer (Python / Databricks)

ABOUT THE MISSION

Relitix is currently seeking a hands-on Senior Data Engineer to lead a critical transformation of our data platform. We are moving away from legacy structures and re-engineering our ecosystem into a modern, Python-scripted Lakehouse architecture. This is not a maintenance role; it is a builder role for an engineer who values code-first engineering over GUI tools.

If you are energized by owning systems end-to-end and want to build a platform that will scale significantly, we want to meet you. You will own the mission of migrating ADF-based pipelines to Databricks, ensuring internal BI data is accurate, and establishing secure governance via Unity Catalog.

WHO YOU ARE

We don’t just list responsibilities; we define clear outcomes. To achieve them, we look for specific traits in our engineers. These are the principles that will guide your success here:

Code-First Mindset | Deep Ownership | Automation First | Governance Minded | Builder Energy

The purpose of the Senior Data Engineer at Relitix is to modernize core architecture and secure the platform while ensuring reliable downstream BI. You don’t just spot problems; you fix them.

Job Responsibilities

• Modernize Core Architecture: Lead the full migration of priority pipelines into modular Python/Spark jobs running in Databricks, moving away from legacy ADF dependencies.
• Optimize Performance: Re-architect pipelines to significantly reduce runtime and compute costs through partitioning, caching, and optimized orchestration.
• Stabilize Data Ingestion: Architect and deploy robust Python connectors for API integrations (MLS, CRM, operational systems) with consistent logging and error handling.
• Elevate Data Quality: Take ownership of internal BI datasets, implementing automated validation tests to ensure dashboards and operational metrics remain consistent and reliable.
• Secure the Platform: Establish a governance foundation by implementing RBAC roles and Unity Catalog controls across bronze, silver, and gold data layers.

Job Requirements

• Strong Python Engineering: Strong Python engineering skills for ETL/ELT with a preference for writing clean, maintainable code over using drag-and-drop GUI tools.
• Spark/Databricks Expertise: Experience with PySpark and SparkSQL inside Databricks, comfortable designing and building out Delta Lake and medallion architectures.
• Governance Knowledge: Understanding of the importance of governance with exposure to Unity Catalog or similar frameworks for managing data permissions.
• Automation Mindset: Approach problems with an automation-first mindset, utilizing Git-based workflows and CI/CD principles.
• Accountability: High sense of ownership and accountability to ensure reliable downstream BI.

Preferred Qualifications

• Experience with data quality tools (e.g., Great Expectations).
• Familiarity with MLflow or AI-ready pipeline structures.
• Prior experience in multi-tenant SaaS data environments.
• Madison, WI, Milwaukee, WI or Chicago IL area preferred

Job Types: Full-time, Permanent

Pay: $90,000.00 - $115,000.00 per year

Benefits:

• 401(k)
• Dental insurance
• Flexible schedule
• Health insurance
• Health savings account
• Paid time off
• Vision insurance

Work Location: Remote