Senior Python Software Engineer

Senior Python Software Engineer – Federated Learning & Visualization

We are only considering US citizens per client request.

Location: Remote / Hybrid

Rate: 72/W2.

Type: Contract (12 months with potential for extension)

About the Role

We are seeking an experienced Python Software Engineer to support the continued development of Advanced Privacy-Preserving Federated Learning (APPFL) — an open-source, Python-based framework designed to enable scalable, secure, and privacy-preserving federated learning for scientific and biomedical applications.

APPFL is an actively used, community-driven framework, primarily adopted across Department of Energy national laboratories and academic collaborations. As federated learning (FL) workloads grow in scale and complexity across heterogeneous environments (HPC systems, cloud platforms, and institutional clusters), we are expanding our engineering efforts to improve observability, usability, scalability, and long-term maintainability.

This role will focus on designing and implementing a real-time federated learning visualization toolkit, advancing privacy-preserving capabilities, optimizing performance, and supporting the growth of the APPFL open-source ecosystem.

Key Responsibilities

1. Real-Time Federated Learning Visualization Toolkit

• Design and implement a modular, scalable real-time monitoring and visualization toolkit for federated and distributed learning workflows.
• Architect extensible systems to collect, aggregate, and visualize metrics across distributed clients and servers.
• Enable real-time or near-real-time tracking of:
• Training loss and accuracy
• Round progression
• Client participation and coordination events
• Communication volume and latency
• Queue time and resource utilization
• Client system performance and location
• Ensure compatibility across heterogeneous compute environments (HPC, cloud, hybrid).
• Develop clean APIs, configuration options, and user-facing documentation.
• Deliver a production-ready visualization dashboard suitable for open-source release (e.g., PyPI).

2. Privacy-Preserving Feature Development

• Implement new algorithms and features related to large-scale privacy-preserving federated learning.
• Support additional privacy-enhancing mechanisms for secure FL experimentation.
• Optimize memory usage and communication patterns for scalability across:
• Large model sizes
• High client counts
• Develop distributed client trainers to support foundation model development for scientific use cases.
• Collaborate closely with researchers and domain experts to translate research into production-ready software.

3. Framework Maintenance & Release Support

• Contribute production-quality Python code to the APPFL codebase.
• Investigate and resolve GitHub issues in a timely manner.
• Refactor components to improve robustness, performance, and usability.
• Maintain and expand unit and integration test coverage.
• Review community pull requests as needed.
• Assist with version releases, changelog preparation, and documentation updates.

4. Community & Ecosystem Development

• Improve public documentation, tutorials, and example workflows.
• Develop reproducible demonstration use cases.
• Support GitHub issue triage and community engagement.
• Contribute to best practices in open-source governance and contribution guidelines.
• Support collaborative publications and technical manuscripts as applicable.

Required Qualifications

• Bachelor’s, Master’s, or PhD in Computer Science, Engineering, Applied Mathematics, or related field.
• Strong Python software engineering experience (5+ years preferred).
• Experience building production-quality, well-tested open-source software.
• Experience with distributed systems and/or federated learning.
• Familiarity with HPC, cloud computing, or hybrid compute environments.
• Experience designing modular, scalable system architectures.
• Proficiency with Git, CI/CD workflows, and collaborative development on GitHub.
• Strong written documentation skills.