Key Responsibilities & Operational Scope

In this position at Eucalyptus, your primary deliverables and day-to-day focus areas include:

• Design and build a standardised evaluation framework covering clinical interactions, health coach conversations, AI assistant conversations, PX support, and health insights
• Build and run services that monitor patient-facing conversations and automatically flag risky content for review and escalation
• Develop eval datasets, metrics, and thresholds with clinicians, coaches, and support leads — turning “what good looks like” into something measurable
• Own the technical integration between our AI assistants and our data models, making patient & business context available to assistants safely, and consistently
• Operate your systems in production: monitoring, alerting, and iterating as models and prompts evolve
• Own and automate guardrail updates based on internal and patient feedback from interactions with our third-party support tools
• Share results with the teams who own each workstream, and help them improve

Candidate Profile & Required Qualifications

To succeed in this role, Eucalyptus is seeking candidates who possess the following credentials, competencies, and mindset:

• A degree in computer science, engineering, data science, or a related field — or equivalent practical experience
• Strong Python skills and the fundamentals to build reliable services, not just notebooks
• Hands-on experience with LLMs: prompt engineering, and ideally exposure to eval frameworks or LLM-as-judge approaches (coursework, internships, or personal projects all count)
• A careful, safety-first mindset — you care about what an AI system says to a real patient, and you sweat the edge cases
• Clear communication — you can explain eval results to a clinician as well as an engineer
• Comfortable with ambiguity: this layer doesn’t exist yet, and you’ll help define it
• Experience with eval tooling (e.g. promptfoo, Ragas, LangSmith, or in-house harnesses)
• Exposure to healthcare, regulated industries, or other high-stakes AI applications
• Familiarity with GCP, SQL, or modern data stacks

Workplace Culture, Mentorship & Benefits

Eucalyptus fosters a high-autonomy, patient-first culture backed by the scale of Hims & Hers:

• Euc is also behind a growing family of digital healthcare clinics (Pilot, Kin, Software, Compound) across men’s health and well-being, fertility, skincare, and preventative health.
• What’s next – Our goal for the next three years is to support 1 million patients globally to live better for longer. We’re launching into new conditions, demographics, and geographies as we build a truly preventive healthcare ecosystem.
• Build something world-changing – We’re on the path to becoming the world’s largest international digital healthcare company. It will be challenging, fast-paced, and deeply rewarding.
• Make real impact – You will deliver work that directly shapes patient outcomes and scales evidence-based care across markets.
• Accelerate your growth – You will have high ownership, continuous feedback, and dedicated development support.
• Join a motivated team – You will collaborate with talented peers to solve complex clinical and operational problems at scale.

Remuneration & Transparency

Comprehensive compensation package commensurate with experience, plus statutory superannuation and performance incentives.