Distinguished Engineer, Generative AI Systems (Remote Eligible)

About the position The Distinguished Engineer, Generative AI Systems at Capital One is responsible for developing model inference services and infrastructure for AI models at scale. This role involves designing robust, secure infrastructure for deploying Large-Language Models (LLMs) and Foundation Models (FMs) on GPU accelerated instances, supporting real-time applications and cutting-edge AI research. The engineer will work within the Enterprise AI team to architect and implement key API products and services that enhance customer-facing applications, optimize inference performance, and enable new GenAI capabilities. Responsibilities • Develop model inference services and infrastructure for AI models at scale. , • Design and implement robust, secure infrastructure for deploying LLMs on GPU accelerated instances. , • Architect, build, and deploy well-managed platform APIs to access LLMs and proprietary FMs. , • Design AI model serving systems for performance, real-time applications, scale, ease of use, and governance automation. , • Optimize inference performance for LLMs and other FMs for cost, latency, throughput, and resiliency. , • Design and implement benchmarks to measure the performance of AI model serving systems. , • Develop tools and processes to monitor API access patterns and operational health. , • Enable users to build new GenAI capabilities. , • Design and implement capabilities to support MLOps for foundation models. Requirements • Bachelor's degree in Computer Science, Computer Engineering, or a technical field. , • At least 7 years of experience designing and building distributed computing HPC and large-scale ML systems. , • At least 5 years of experience developing AI and ML systems using Python or Golang. , • At least 3 years of experience with the full ML development lifecycle using AI and ML frameworks and public cloud. Nice-to-haves • Master's degree or PhD in Engineering, Computer Science, or a related technical field. , • Experience designing large-scale distributed platforms and/or systems in cloud environments such as AWS, Azure, or GCP. , • Experience developing applications that leverage LLMs and FMs. , • Experience architecting cloud systems for security, availability, performance, scalability, and cost. , • Experience with delivering very large models through the MLOps life cycle from exploration to serving. , • Experience with building GPU clusters in the public cloud with tightly-coupled storage and networking. , • Experience with one or multiple areas of AI technology stack including prompt engineering, guardrails, vector databases/knowledge bases, LLM hosting and fine-tuning. , • Authored research publications in top peer-reviewed conferences or industry-recognized contributions in the space of neural networks, distributed training, and SysML. Benefits • Comprehensive health benefits , • Financial benefits including performance-based incentives , • Inclusive workplace policies , • Support for total well-being Apply Job!