Research Scientist Intern - Multimodal Sensing & On-Device Perception, Global Frontier Tech Recruitment Program

Team Introduction: The eye tracking system architecture team works on the vertical stack of eye tracking systems architecture and related key components. Our goal is enabling an eye tracking system with high population coverage, high performance, and low power consumption. Eye tracking is a highly multidiscipline area. You will have the opportunity to work on state of art technologies across refractive/diffractive optics, image sensor, human factor and CV algorithm. We are looking for talented individuals to join us for an internship in 2027. PhD Internships at our Company aim to provide students with the opportunity to actively contribute to our products and research, and to the organization's future plans and emerging technologies. PhD internships at Our Company provides students with the opportunity to actively contribute to our products and research, and to the organization's future plans and emerging technologies. Our dynamic internship experience blends hands-on learning, enriching community-building and development events, and collaboration with industry experts. Applications will be reviewed on a rolling basis - we encourage you to apply early. Please state your availability clearly in your resume (Start date, End date). Topic Content: The rapid advancement of AI Agents is set to have a far-reaching and transformative impact on daily life. As the hardware bridge linking AI Agents to the physical world and end users, intelligent hardware acts as a vital gateway for agents to perceive their environment and recognize user intent. Enabling AI Agents to maintain round‑the‑clock environmental awareness and efficiently capture real‑time user intentions is essential to improving their everyday service experience. This topic seeks to overcome the limitations of conventional visual perception systems. By deeply integrating sensing and computing, it explores full‑stack innovation spanning from low‑level hardware to high‑level algorithms. Key efforts include developing next‑generation sensors with real‑time understanding capabilities, investigating non‑traditional visual capture and compression techniques, and designing novel hardware architectures to run advanced algorithms—enabling highly efficient coordination across sensing, processing, and communication. Topic Challenge: Break free from the framework of conventional sensing systems, explore novel sensors, signal processing and compression schemes to achieve highly energy-efficient sensing tasks, while enabling seamless integration with large models. Topic Value: Breakthroughs in this research direction will enable intelligent hardware to better connect AI, users, and daily life. Against the backdrop of the AI Agent era, this will unlock the gateway to the next generation of intelligent hardware terminals and open up broader technological possibilities. Project Objective: Rethink visual perception from the sensor up. Instead of treating sensing and computation as separate stages, we're exploring architectures that fuse them — sensors that compress, select, and partially "understand" the scene before data ever leaves the chip. This requires deeply understanding both the hardware constraints and what modern vision-language and world models actually need from the sensing front-end. The goal: order-of-magnitude gains in perception efficiency by co-designing hardware and algorithms across the full pipeline. Responsibilities - Design and prototype novel sensor or imaging architectures that move computation closer to the sensing front-end (e.g., near-sensor processing, event-driven capture, learned compression at the pixel level) - Build and characterize imaging pipelines end-to-end: from optical/sensor physics through ISP to downstream perception models — identify where bits are wasted and where intelligence should be injected - Leverage understanding of VLM/LLM and world models to inform what information the sensing front-end must preserve, discard, or transform — closing the loop between foundation model requirements and hardware design - Develop or adapt machine vision models that are co-optimized with hardware constraints (power, bandwidth, latency) Minimum Qualifications - Currently pursuing a PhD in Computer Science, Electrical Engineering, Optical Engineering, Applied Mathematics, Physics, or a related technical field. - Strong research background in computer vision and machine learning, with hands-on model training experience. - Experience with at least one of: sequence modeling, language modeling, efficient neural network design, or signal processing. Preferred Qualifications - Proven track record of high-impact research, demonstrated by publications in CVPR, ICCV, ECCV, NeurIPS, ICLR, SIGGRAPH, or similar. - Hands-on experience with hardware prototyping involving cameras, structured light, or other active sensing systems. - Familiarity with on-device or on-sensor compute, embedded ML deployment, and hardware/software co-design tradeoffs. - Self-motivated, curious, and excited about taking ambitious research ideas from prototype to working system.