C

Capital One

Issues credit cards and runs a consumer bank

Staff Applied Researcher

Natural Language Processing & LLMsConfirmed live in the last 24 hours
Full-time$306.3k - $381.3k /yrIn-person

About the Job

Requirements

  • PhD in Electrical Engineering, Computer Engineering, Computer Science, AI, Mathematics, or a related field plus four years of experience in applied research, or an M.S. in one of those fields plus six years of experience in applied research.
  • Have hands-on experience developing AI foundation models and solutions using open-source tools and cloud computing platforms.
  • Have a deep understanding of the foundations of AI methodologies.
  • Have experience building large deep learning models involving language, images, events, or graphs, and expertise in at least one of training optimization, self-supervised learning, robustness, explainability, or reinforcement learning from human feedback.
  • Have a track record of delivering models at scale in terms of training data and inference volumes.
  • Have experience delivering libraries, platform-level code, or solution-level code to existing products.
  • Have a track record of producing high-quality ideas or improving existing ideas in machine learning, demonstrated by accomplishments such as first-author publications or projects.
  • Be able to own and pursue a research agenda, including selecting impactful research problems and autonomously carrying out long-running projects.
  • For the LLM specialization: PhD focus on natural language processing, or a master's degree with 10 years of industrial natural language processing research experience; have been a core contributor to a team that trained a large language model from scratch with 10 billion or more parameters and 500 billion or more tokens; have numerous publications at ACL, NAACL, EMNLP, NeurIPS, ICML, or ICLR on large language model pre-training topics; have worked on a currently available open-source or commercial LLM; be able to guide the technical direction of a large-scale model training team; have experience with clusters of more than 500 GPU nodes and an LLM scaled to 70 billion parameters and more than 1 trillion tokens; and have experience with training optimization frameworks such as DeepSpeed or NeMo.
  • For the behavioral models specialization: PhD focus on geometric deep learning topics such as graph neural networks, sequential models, or multivariate time series; technical leadership experience for deployment of a very large user behavior model; multiple papers relevant to training models on graph and sequential data structures at KDD, ICML, NeurIPS, or ICLR; experience scaling graph models to more than 50 million nodes and with large-scale deep-learning-based recommender systems; experience with production real-time and streaming environments; contributions to open-source frameworks such as PyTorch Geometric or DGL; proposed methods for inference or representation learning on graphs or sequences; and experience working with datasets of more than 100 million users.
  • For the training and inference optimization specialization: PhD focus on optimizing training of very large language models; and five or more years of experience and/or publications in model sparsification, quantization, training parallelism/partitioning design, gradient checkpointing, or model compression.
  • For the finetuning specialization: PhD focus on guiding LLMs with further tasks such as supervised finetuning, instruction-tuning, dialogue-finetuning, or parameter tuning; demonstrated knowledge of transfer learning, model adaptation, and model guidance; and experience deploying a fine-tuned large language model.
  • For the data preparation specialization: numerous publications studying tokenization, data quality, dataset curation, or labeling; leading contributions to one or more large open-source corpora of more than 1 trillion tokens; and core contributions to open-source libraries for data quality, dataset curation, or labeling.
  • Have a demonstrated history of initiating and leading multi-year research programs that shaped the direction of AI or machine learning subfields, such as representation learning, foundation models, or multimodal reasoning.

Responsibilities

  • Partner with data scientists, software engineers, machine learning engineers, and product managers to deliver AI-powered products that change how customers interact with their money.
  • Use technologies including PyTorch, AWS Ultraclusters, Hugging Face, and Lightning to identify insights in large volumes of numeric and textual data.
  • Build AI foundation models through design, training, evaluation, validation, and implementation.
  • Conduct applied research to bring recent AI developments into new customer experiences.
  • Translate technical complexity into tangible business goals.
  • Define and steward new research directions that expand the organization's long-term scientific agenda and translate foundational discoveries into deployable capabilities.
  • Guide and mentor a team of applied scientists and their managers without direct people-management responsibility.
  • Represent Capital One externally in the research community and collaborate with faculty members in relevant AI research communities.

Desired Qualifications

  • Preferred qualifications include a PhD in Computer Science, Machine Learning, Computer Engineering, Applied Mathematics, Electrical Engineering, or a related field.
  • PhD focus on natural language processing, or a master's degree with 10 years of industrial natural language processing research experience.
  • Core contribution to training a large language model from scratch with 10 billion or more parameters and 500 billion or more tokens.
  • Numerous publications at ACL, NAACL, EMNLP, NeurIPS, ICML, or ICLR on large language model pre-training topics.
  • Experience working on a currently available open-source or commercial LLM.
  • Demonstrated ability to guide the technical direction of a large-scale model training team.
  • Experience with clusters of more than 500 GPU nodes and an LLM scaled to 70 billion parameters and more than 1 trillion tokens.
  • Experience with training optimization frameworks such as DeepSpeed or NeMo.
  • PhD focus on geometric deep learning topics such as graph neural networks, sequential models, or multivariate time series.
  • Membership in technical leadership for deployment of a very large user behavior model.
  • Multiple papers on training models on graph and sequential data structures at KDD, ICML, NeurIPS, or ICLR.
  • Experience scaling graph models to more than 50 million nodes and with large-scale deep-learning-based recommender systems.
  • Experience with production real-time and streaming environments.
  • Contributions to open-source frameworks such as PyTorch Geometric or DGL.
  • Proposed methods for inference or representation learning on graphs or sequences.
  • Experience working with datasets of more than 100 million users.
  • PhD focus on optimizing training of very large language models.
  • Five or more years of experience and/or publications in model sparsification, quantization, training parallelism/partitioning design, gradient checkpointing, or model compression.
  • PhD focus on guiding LLMs with further tasks such as supervised finetuning, instruction-tuning, dialogue-finetuning, or parameter tuning.
  • Demonstrated knowledge of transfer learning, model adaptation, and model guidance.
  • Experience deploying a fine-tuned large language model.
  • Numerous publications studying tokenization, data quality, dataset curation, or labeling.
  • Leading contributions to one or more large open-source corpora of more than 1 trillion tokens.
  • Core contributions to open-source libraries for data quality, dataset curation, or labeling.
  • Demonstrated history of initiating and leading multi-year research programs that shaped AI or machine-learning subfields such as representation learning, foundation models, or multimodal reasoning.

About the Company

cb

Capital One

Issues credit cards and runs a consumer bank

Capital One runs a consumer and commercial bank that issues credit cards, takes deposits, and finances cars and businesses. Its Venture travel cards and Quicksilver cash-back cards let customers earn rewards on everyday spending, while its online banking tools manage payments and savings. Customers use checking and savings accounts through an app, branches, and cash machines, and commercial clients use treasury tools to manage company cash. The bank earns interest on loans and fees from banking and card transactions, with customer deposits helping fund its lending operations. Capital One was founded in 1994 and has its headquarters in McLean, Virginia.

Company size
10,001+
Company stage
IPO
Headquarters
McLean, VA
Founded
1994

Recently Posted Jobs

Explore more opportunities at Capital One

See more jobs