Mogi I/O : OTT/Podcast/Short Video Apps for you

Work Type: Contractor | Permanent Remote 
 
Compensation: USD 15 – 25/hour 
 
Hours: 40 hours/week (PST overlap required) 
 
Experience Required: 5+ Years 
 
Contract Duration: 3 Months 
 
Notice Period: Max 2 weeks 
 
Note 
 
Contract-based, fully remote role. Payment based on actual hours worked. No paid leaves/benefits. The contractor handles own taxes/compliance. 
 
About The Project 
 
We're building LLM evaluation/training datasets to solve realistic software engineering problems. Our approach involves creating verifiable SWE tasks using public repository histories through synthetic methods with human-in-the-loop validation, expanding coverage across programming languages, difficulty levels, and task types. 
 
About The Role 
 
Seeking a Tech Lead-level software engineer experienced with high-quality public GitHub repositories. You'll drive hands-on engineering work including environment automation, issue triaging, and test coverage/quality evaluation to advance LLM capabilities for real-world coding tasks. 
 
Day-to-Day Responsibilities 
 


* Analyze and triage GitHub issues from trending open-source libraries
* Configure code repositories (Dockerization, environment setup)
* Evaluate unit test coverage and software quality
* Modify/run codebases locally to validate LLM bug-fix performance
* Collaborate with researchers to identify LLM-challenging repositories/issues
* Lead junior engineers on assigned projects 
 
 

Required Skills 
 


* 5+ years overall software engineering experience
* Tech Lead experience with complex codebases
* Proficiency in C++ (primary) or similar systems languages
* Expertise with Git, Docker, and CI/CD pipeline configuration
* Ability to run/debug/test real-world projects locally
* Open-source contribution/evaluation experience

Software Development

Testing

Project management

Automation

Unit Testing

Docker

Continuous Integration

Payment systems

Compliance

CI/CD

Senior LLM Evaluation Engineer (C++ Focus) - 3-Month Contract