Would you like to use your data engineering skills to revolutionize retail? Be part of the development and improvement of a dynamic pricing B2B SaaS platform, leading the charge in automating and optimizing pricing strategies with AI-driven insights.
It is a comprehensive solution, embraced by retailers and brands across over 40 markets, that delivers actionable pricing recommendations that drive business growth and profitability.
This is an invitation to join a dynamic and pioneering team where your expertise in data will significantly influence the path forward. Our client's team is passionate about tackling real-world challenges and seeks like-minded individuals eager to make a mark in the retail pricing arena.
Required skills:
- A Bachelor’s or higher in Computer Science, Software Engineering, or a closely related technical field, demonstrating strong analytical and coding skills.
- 3+ years of experience as a data engineer, software engineer, or similar role and using data to drive business results.
- Strong Python skills, with experience building modular, testable, and production-ready code.
- Solid understanding of databases such as SQL, including indexing best practices, and hands-on experience working with large-scale data systems (e.g., Spark, Glue, Athena).
- Practical experience with Airflow or similar orchestration frameworks, including designing, scheduling, maintaining, troubleshooting, and optimizing data workflows (DAGs).
- A solid understanding of data engineering principles: ETL/ELT design, data integrity, schema evolution, and performance optimization.
- Familiarity with AWS cloud services, including S3, Lambda, Glue, RDS, and API Gateway.
Nice-to-Haves:
- Experience with MLOps practices such as CI/CD, model and data versioning, observability, and deployment.
- Familiarity with API development frameworks (e.g., FastAPI).
- Knowledge of data validation techniques and tools (e.g., Great Expectations, data drift detection).
- Exposure to AI/ML system design, including pipelines, model evaluation metrics, and production deployment.
Scope of work:
As a Data Engineer you’ll have end-to-end ownership - from system architecture and software development to operational excellence. Specifically, you will:
- Design and implement scalable machine learning pipelines with Airflow, enabling efficient parallel execution.
- Enhance our data infrastructure by refining database schemas, developing and improving APIs for internal systems, overseeing schema migrations, managing data lifecycles, optimizing query performance, and maintaining large-scale data pipelines.
- Implement monitoring and observability, using AWS Athena and QuickSight to track performance, model accuracy, operational KPIs, and alerts.
- Build and maintain data validation pipelines to ensure incoming data quality and proactively detect anomalies or drift.
- Collaborate closely with software architects, DevOps engineers, and product teams to deliver resilient, scalable, production-grade machine learning pipelines.