Backend Engineer

Join the team building the foundational infrastructure that allows developers to run machine learning models in the cloud with a single line of code. As a Backend Engineer at Replicate, you will design and scale the distributed systems that orchestrate thousands of GPU containers and handle millions of API requests. You will work on the core engine that manages model deployments, autoscaling logic, and billing systems to make AI accessible to every software engineer.

- Design and implement highly scalable backend services to manage the lifecycle of machine learning models. - Optimize the cold-boot performance and autoscaling latency of our GPU-based inference engine. - Build and maintain the public API and client libraries that empower thousands of developers. - Develop internal tooling and observability systems to monitor the health of our global fleet of compute. - Collaborate with the product team to architect new features like fine-tuning workflows and webhooks. - Participate in an on-call rotation to ensure the reliability and uptime of the Replicate platform. - Write high-quality, well-tested code and perform rigorous peer design and code reviews.

- 5+ years of experience building and maintaining high-traffic distributed systems in production. - Expert proficiency in Go or Python, with a deep understanding of concurrency and performance optimization. - Extensive experience with container orchestration (Kubernetes) and cloud infrastructure providers (GCP, AWS). - Proven track record of designing robust, versioned REST/GRPC APIs used by external developers. - Solid understanding of networking, load balancing, and storage systems (PostgreSQL, Redis, S3). - Comfortable working in a small, fast-paced team with a high degree of autonomy and end-to-end ownership. - Experience with low-level Linux primitives and container runtimes (containerd, Firecracker) is a strong plus.