AI/ML Prompt Scientist/Engineer

STAND 8 provides end to end IT solutions to enterprise partners across the United States and with offices in Los Angeles, New York, New Jersey, Atlanta, and more including internationally in Mexico and India

We're looking for an AI/ML Prompt Scientist/Engineer to design, evaluate, and optimize prompt-based systems that drive real-world impact across cutting-edge AI applications. This position is ideal for a Data Scientist or Machine Learning Engineer with a strong background in prompt optimization and model performance.
In this role, you'll combine your expertise in prompt engineering, orchestration frameworks, retrieval pipelines, and evaluation tooling to build scalable, reliable, and safe AI systems in collaboration with AI Engineers and Data Scientists.
Key Responsibilities

• Design, optimize, and evaluate prompts and orchestration pipelines to enhance LLM performance and scalability, supporting both Data Scientist and engineering workflows.
• Implement programmatic prompting using frameworks such as LangChain, LlamaIndex, DSPy, Guidance, and LangGraph.
• Define and manage evaluation frameworks, including A/B testing, human and automated scoring, and model quality assessments using tools such as Ragas, UpTrain, DeepEval, TruLens, Promptfoo, and OpenAI Evals.
• Engineer and maintain context management systems, including conversation memory, retrieval logic, and context compression strategies (chunking, summarization, prioritization).
• Design and optimize retrieval and RAG pipelines, integrating vector databases (e.g., Weaviate) and rerankers (e.g., Cohere) to enhance relevance and accuracy.
• Establish observability and traceability mechanisms (e.g., Langfuse) to monitor performance metrics, latency, and operational costs.
• Implement guardrails, safety mechanisms, and bias mitigation policies using frameworks such as Guardrails AI, NeMo Guardrails, Outlines, and Rebuff.
• Collaborate with cross-functional engineering and research teams-including Data Scientists, ML, and AI engineering groups-to ensure high-quality, compliant, and well-documented system delivery.

Qualifications

• Proven experience in LLM and NLP model operations, including tokenization, context windows, and safety considerations.
• Strong technical expertise in prompt engineering, orchestration frameworks, and context management.
• Hands-on experience with retrieval-augmented generation, evaluation frameworks, and observability tooling.
• Understanding of key evaluation metrics such as faithfulness, groundedness, coherence, toxicity, latency, and cost.
• Experience with compliance and ethical AI practices, including bias and hallucination mitigation.
• Excellent analytical, documentation, and communication skills, with the ability to collaborate effectively across teams.

Preferred Experience

• Model fine-tuning and alignment (e.g., LoRA, DPO/RLHF, preference tuning).
• Experience developing AI agents and integrating tool-use capabilities.

Benefits

• Medical coverage and Health Savings Account (HSA) through Anthem
• Dental/Vision/Various Ancillary coverages through Unum
• 401(k) retirement savings plan
• Paid-time-off options
• Company-paid Employee Assistance Program (EAP)
• Discount programs through ADP WorkforceNow

Additional Details
The base range for this contract position is $100,000 - 200,000 / per year, depending on experience. Our pay ranges are determined by role, level, and location. The range displayed on each job posting reflects the minimum and maximum target for new hires of this position across all US locations. Within the range, individual pay is determined by work location and additional factors, including job-related skills, experience, and relevant education or training. Qualified applicants with arrest or conviction records will be considered

About Us
STAND 8 provides end-to-end IT solutions to enterprise partners across the United States and globally with offices in Los Angeles, Atlanta, New York, Mexico, Japan, India, and more. STAND 8 focuses on the "bleeding edge" of technology and leverages automation, process, marketing, and over fifteen years of success and growth to provide a world-class experience for our customers, partners, and employees.
Our mission is to impact the world positively by creating success through PEOPLE, PROCESS, and TECHNOLOGY.
Check out more at www.stand8.io; and reach out today to explore opportunities to grow together!
By applying to this position, your data will be processed in accordance with the STAND 8 Privacy Policy.