Company Overview
Function was founded with a singular focus: empower you to live 100 healthy years. We’re doing that by using the best available technology to make sure people don't suffer or die a preventable death. Function has been recognized as one of Fast Company’s Most Innovative Companies of 2024, and is venture-backed by Andreessen Horowitz (a16z). Hundreds of thousands of members have joined Function to take control of their health. We are growing our team and seeking out world-class talent that deeply believes in our mission to positively impact global health, has a relentless bias toward action and a growth mindset. Function fosters a collaborative and dynamic environment, where every day we are building the future.
Role Overview
You will design, ship, and scale production-grade, stateful multi-agent systems end to end—spanning orchestration graphs, model serving, real-time inference, and observability.
You’ll partner with product, infra, and research to integrate LLMs and multimodal models (voice, vision, structured data) into consumer and internal workflows with strong safety, reliability, and cost controls.
This is a hands-on role for a high-ownership engineer with deep systems expertise and a track record of delivering AI agents at scale.
Impact You'll Drive
- Ship agentic features that move core product KPIs with measurable quality and latency targets.
- Establish evaluation gates and on-call reliability for AI systems that handle real users and revenue.
- Reduce cost-to-serve via model routing, KV cache reuse, and retrieval quality improvements.
Key Responsibilities
- Architect and build stateful, graph-based agent workflows with tool use, planning, and memory.
- Integrate LLMs and multimodal models via structured I/O (JSON Schema, Pydantic validators) and function/tool calling.
- Build high-reliability APIs and streaming services for real-time inference, speech, and vision.
- Own production readiness: tracing, logging, metrics, rate limiting, circuit breakers, and SLOs.
- Stand up eval pipelines: offline golden sets, LLM-as-judge with human rubrics, online A/B, and regression tests in CI.
- Implement retrieval and memory: hybrid search, vector and graph retrieval, semantic caches, and long-horizon context.
- Optimize cost/latency: model routing, prompt and tool selection, quantization, and KV cache/prefill strategies.
- Lead cloud-native deployments on Kubernetes with GPU autoscaling, canary/shadow releases, and feature flags.
- Partner cross-functionally to translate research into robust production systems and iterate quickly behind evaluation gates.
- Mentor engineers through code reviews, design docs, and architecture decisions.
Must-Have Qualifications
- 2+ years building agentic AI systems; 4+ years building production backends or ML systems in Python, Go, or similar.
- Fluency with agentic orchestration (e.g., LangGraph, PydanticAI, DSPy, LlamaIndex) and tool/function calling.
- Experience integrating frontier LLMs and multimodal models via managed APIs or self-hosted serving.
- Deep understanding of model serving and inference optimization (vLLM/Triton/TGI/SGLang, batching, KV cache reuse).
- Strong with API design and backend frameworks (FastAPI, Flask) and event-driven architectures.
- Data systems expertise with PostgreSQL and Redis, including caching, token streaming, and throughput tuning.
- Retrieval and memory: vector databases (pgvector, Pinecone, Weaviate, Milvus), hybrid search, and graph/knowledge storage.
- Production evals: LLM-as-judge, human-in-the-loop, rubric design, and CI-integrated regression tests.
- Observability and SRE: OpenTelemetry traces, metrics, structured logs, SLOs, dashboards, and on-call triage.
- Cloud-native delivery: Kubernetes, Terraform, Docker, GPU scheduling/autoscaling on AWS or GCP.
- CI/CD proficiency with GitHub Actions and test automation for prompts, tools, and agents.
- Clear, concise communication and high ownership in fast-paced environments.
Nice-to-Have Qualifications
- Real-time multimodal systems: streaming ASR, low-latency TTS, WebRTC, and vision pipelines.
- Post-training/fine-tuning: DPO/ORPO, RLHF, preference data generation, and safety alignment.
- RAG expertise beyond basics: Graph RAG, multi-hop retrieval, rerankers, query planning, and freshness policies.
- Safety and governance: policy-as-code, red-teaming, PII handling, audit logs, and role-based tool authorization.
- Regulated data experience (HIPAA, SOC 2, GDPR) and data residency controls.
- Personalization at inference time, long-term memory agents, session state, and episodic memory stores.
- Experience with consumer-scale AI apps, high-traffic systems, or on-device/edge acceleration (WebGPU).
Example Tech You'll Touch
- Orchestration: LangGraph, PydanticAI, DSPy, LlamaIndex
- Serving: vLLM, Triton, TGI, SGLang; OpenAI/Anthropic-compatible APIs
- Backend: Python, Go, FastAPI, gRPC, Kafka/PubSub
- Data: PostgreSQL, Redis, pgvector, Pinecone/Milvus/Weaviate
- Observability: OpenTelemetry, Prometheus, Grafana, Sentry
- Infra: Kubernetes, Terraform, Docker, GPU operators, Karpenter/Cluster Autoscaler
- Evals & QA: RAGAS/DeepEval-style frameworks, golden sets, canary/shadow testing
How We Build
- Evaluation-driven development: every change to prompts, tools, routing, or retrieval passes automated eval gates.
- Structured outputs by default: JSON Schema/Pydantic validation, strict tool contracts, and idempotent handlers.
- Safety-first tooling: guardrails, content and data policies, tool sandboxing with timeouts and scopes.
- Pragmatic iteration: short cycles, feature flags, shadow traffic, and fast rollback.
Success in 90 Days
- Launch a production agentic workflow with clear SLOs, tracing, and dashboards.
- Stand up an eval harness with golden sets and CI gates for the top use case.
- Improve latency and cost with routing and KV cache strategies while maintaining quality.
At Function, we celebrate diversity and are committed to building a diverse and inclusive workforce. As an equal opportunity employer, we do not discriminate on the basis of race, color, gender identity, ancestry, religion, age, sexual orientation, national origin, disability, marital status, Veteran status, or any other occupationally irrelevant criteria.
Join the Function Health team and become a part of our mission to build a healthier future for all. Discover more about us and how we're changing the face of healthcare at Function Health.
Important Notice: Legitimate communication from the Function Health team will always come from an email address ending in @functionhealth.com. Function Health will never request personal information such as banking details or payment during the hiring process. Please be cautious of communications or job offers that come from other email domains, instant messaging platforms, or unsolicited calls. If you ever have doubts about the legitimacy of a communication, please reach out to us directly at talent@functionhealth.com.
