Software Engineer – AI Evaluation Analyst

About The Role

What if your years of engineering experience could directly influence how the world's most advanced AI systems write and reason about code? We're looking for experienced software engineers to put frontier AI models through their paces — finding the bugs, hallucinations, and logic failures that only a sharp, seasoned developer would catch.

This is a fully remote, flexible contract role. No background in AI research required — just deep coding expertise, strong analytical thinking, and a genuine instinct for finding what breaks.

• Organization: Alignerr
• Type: Hourly Contract
• Location: Remote
• Commitment: 10–40 hours/week

What You'll Do

• Evaluate the performance of frontier AI language models on complex, real-world software engineering tasks
• Hunt for bugs, logical errors, hallucinations, and reliability failures in AI-generated code
• Design prompts, test cases, and evaluation scenarios that push models to their limits
• Write precise, structured feedback explaining model strengths, weaknesses, and edge case behavior
• Work across multiple programming languages and codebases to assess correctness and generalization
• Think critically about model behavior — not just whether code runs, but whether it's right

Who You Are

• 3+ years of professional software engineering experience
• Strong proficiency in at least one of: TypeScript, Ruby, Java, or C++
• Excellent written and spoken English — you can explain complex technical issues with clarity
• Demonstrated ability to reason about complex systems and debug non-obvious failures
• Comfortable with modern development tooling — Git, CLI workflows, testing frameworks, and IDEs
• A critical thinker who evaluates code quality independently rather than deferring to model outputs

Nice to Have

• Experience across multiple programming languages or paradigms
• Background in code review, software testing, or QA engineering
• Familiarity with large language models or AI-generated code evaluation
• Exposure to prompt engineering or adversarial testing

Why Join Us

• Work on cutting-edge AI projects alongside leading research labs
• Fully remote and flexible — work when and where it suits you
• Freelance autonomy with the structure of meaningful, task-based work
• Make a direct, tangible impact on how AI understands and generates software
• Potential for ongoing work and contract extension as new projects launch