Company Description
We specialize in connecting professionals with flexible work opportunities they can perform from the comfort of their homes. Our mission is to empower individuals by offering job options that provide work-life balance and personal growth. Through our platform, we aim to revolutionize the way work is approached, making quality opportunities more accessible to a global workforce. Our focus is on creating a supportive and dynamic work environment for professionals around the world.
Overview
We're looking for experienced Grafana power users to design expert-level evaluation tasks that test whether AI agents can use Grafana the way a real professional does. Your domain expertise is what makes these tasks authentic.
What You'll Do
- Design realistic, multi-step Grafana workflows - dashboards, alerting rules, data source configuration, panel setup, cross-module operations
- Perform each workflow yourself on a hosted Grafana instance to produce a reference trajectory
- Write clear, specific task prompts with measurable outcomes that can be verified programmatically
- Implement programmatic graders that check whether each instruction was completed correctly
- Review AI agent attempts at your tasks, identify where and why they fail, and tag root causes
- Calibrate task difficulty so tasks are challenging but solvable - iterating on prompts and constraints based on model performance
Requirements
- 2+ years of daily, professional Grafana experience (SRE, Platform Engineering, Observability, or similar)
- Deep familiarity with PromQL, dashboard templating, alerting pipelines, and data source configuration (Prometheus, InfluxDB, etc.)
- Ability to articulate workflows clearly enough for programmatic verification
- Comfort writing basic grading scripts (Python; engineering support provided as needed)
Nice to Have
- Experience with Grafana API automation
- Kubernetes/infrastructure monitoring background
- Familiarity with AI evaluation or benchmarking
Time Commitment
- 10-15 hrs/week minimum during the project
- Fast turnaround expected - responsiveness matters
- We consider all qualified applicants without regard to legally protected characteristics and provide reasonable accommodations upon request.