Model Performance Engineer (Remote — Worldwide)

As a Model Performance Engineer at Replicate, you will be the primary driver behind making open-source models run faster and more efficiently than anywhere else. You will work at the intersection of deep learning and systems engineering, optimizing the performance of Cog containers and our underlying inference infrastructure. Your work will directly impact thousands of developers by reducing latency and cost for cutting-edge models like SDXL, Llama 3, and specialized LoRAs.

- Optimize cold-boot times and inference latency for a wide variety of generative AI models hosted on Replicate. - Develop and integrate customized kernels to accelerate specific model architectures using Triton or CUDA. - Benchmark and profile diverse hardware configurations to determine the most cost-effective path for model deployment. - Collaborate with the infrastructure team to improve how Cog packages and executes models within Docker environments. - Provide technical guidance to the community on best practices for model optimization and efficient weights loading. - Research and implement state-of-the-art quantization techniques (AWQ, GPTQ, FP8) to minimize memory footprints.

- Deep understanding of GPU architectures (NVIDIA Ampere/Hopper) and CUDA programming. - Proven experience with performance profiling tools like Nsight Systems, PyTorch Profiler, or Triton. - Familiarity with deep learning optimization frameworks such as TensorRT, vLLM, DeepSpeed, or TVM. - Proficiency in Python and C++, with a strong grasp of the PyTorch internal API. - Experience tuning low-level kernels or managing memory bottlenecks for Large Language Models (LLMs). - Strong communication skills for working asynchronously across a global, fully remote team. - A track record of contributing to or maintaining high-performance open-source projects.

Equity, benefits, remote.