AI Data Scientist

What You'll Do

• Build evaluation datasets, rubrics, and synthetic test cases for LLM-powered features across patient and internal workflows.
• Analyze production logs to identify model failures, hallucinations, quality issues, and operational bottlenecks.
• Design and run experiments end-to-end, including hypothesis development, dataset creation, evaluation, analysis, and recommendations.
• Track key product and operational metrics including resolution rate, handle time, touches to resolution, latency, and quality and identify opportunities for improvement.
• Partner with engineers to validate improvements and productionize successful experiments.
• Help build or integrate tooling and dashboards that make AI performance easy to understand and monitor.

Who You Are

• 1–4 years in data science, analytics, applied ML, or a closely adjacent role.
• Strong Python and SQL skills.
• Hands-on experience building or evaluating LLM-powered applications through work, research, school, hackathons, or side projects.
• You’re curious about how AI systems work beyond the model itself, including retrieval, prompting, evaluation, and production behavior.
• You’re comfortable communicating analytical findings clearly to engineers, product managers, and operational stakeholders.
• You are excited about learning rapidly in a field where best practices continue to evolve.
• Bonus: Experience with evaluation tooling, A/B testing frameworks, production model monitoring, healthcare, or other operations-heavy environments.