Ziyi Kou

Ziyi Kou

Senior Research Scientist - Meta Reality Lab Research

I am a senior research scientist at Meta Reality Lab. My research focuses on multimodal generative AI, foundational robotic learning and 3D body/hand modeling. I obtained my Ph.D. degree from Computer Science and Engineering (CSE) department at University of Notre Dame. My advisor is Dr. Xiangliang Zhang. I received my master's degree of Computer Science from University of Rochester and my bachelor's degree of Software Engineering from Chongqing University.

Education

  • Aug. 2020 - May 2024
    University of Notre Dame
    Ph.D., Computer Science and Engineering Courses: Text Mining, Algorithm and Complexity, Operating System, Computer Graphic
  • Aug. 2018 - May 2020
    University of Rochester
    M.S. Computer Science Courses: Artificial Intelligence, Adv. Topic of Computer Vision, Intro. of Statistics, Machine Learning
  • Sept. 2014 - Jun. 2018
    Chongqing University, China
    B.S. Software Engineering Courses: Java Programming, Data Mining with Python, Web Development, Database Design, Android Programming

Recent Work Experience

  • June. 2024 - Present
    Senior Research Scientist
    Meta Multi-Modal Foundational Computer Vision
  • June. 2023 - Aug. 2023
    Applied Scientist Intern
    Amazon Graph Neural Network Based Customer Insotck Value Estimation. Received return incline offer.
  • Oct. 2022 - Feb. 2023
    Machine Learning Engineer Intern
    Instacart Graph Based Next-Basket Recommendation. Published at CIKM2023.
  • May. 2022 - Aug. 2022
    Machine Learning Engineer Intern
    Godaddy Inc. Multi-relational Graph Based Recommendation. Received return intern offer.

Main Publications

* denotes equal contribution

AVI-HT: Adaptive Vision-IMU Fusion for 3D Hand Tracking NeurIPS 2026
Ziyi Kou, Ankit Kumar, Mia Huang, Taylor Niehues, Vatsal Mehta, Ergys Ristani, Li Guan
Glove2Hand: Synthesizing Natural Hand-Object Interaction from Multi-Modal Sensing Gloves CVPR 2026
Xinyu Zhang, Ziyi Kou, Chuan Qin, Mia Huang, Ergys Ristani, Ankit Kumar, Lele Chen, Kun He, Abdeslam Boularias, Li Guan
AirGlove: Exploring Egocentric 3D Hand Tracking and Appearance Generalization for Sensing Gloves ICASSP 2026
Wenhui Cui, Ziyi Kou, Chuan Qin, Ergys Ristani, Li Guan
Adaptive and Context-rich Generative Self-supervised Learning on Graphs AAAI 2026
Yijun Tian, Chuxu Zhang, Ziyi Kou, Zheyuan Liu, Xiangliang Zhang, Nitesh V Chawla
Flowing from reasoning to motion: Learning 3d hand trajectory prediction from egocentric human interaction videos ECCV 2026
Mingfei Chen, Yifan Wang, Zhengqin Li, Homanga Bharadhwaj, Yujin Chen, Chuan Qin, Ziyi Kou, Yuan Tian, Eric Whitmire, Rajinder Sodhi, Hrvoje Benko, Eli Shlizerman, Yue Liu
RAt: Injecting Implicit Bias for Text-To-Image Prompt Refinement Models EMNLP 2024
Ziyi Kou, Shichao Pei, Meng Jiang, Xiangliang Zhang
LeMon: Automating Portrait Generation for Zero-Shot Story Visualization with Multi-Character Interactions KDD 2024
Ziyi Kou, Shichao Pei, Xiangliang Zhang
A Controllable Prompt Adversarial Attacking Framework for Black-Box Text Guided Image Generation Models IJCAI 2023
Ziyi Kou, Shichao Pei, Yijun Tian, Xiangliang Zhang
A Crowd-AI Duo Relational Graph Learning Framework Towards Social Impact Aware Photo Classification AAAI 2023
Yang Zhang, Ziyi Kou, Lanyu Shang, Huimin Zeng, Zhenrui Yue, Dong Wang
A Web Crowdsourcing Based Face Partition Approach Towards Privacy-Aware Face Recognition TheWebConf 2022
Ziyi Kou, Lanyu Shang, Yang Zhang, Siyu Duan, Dong Wang
A Duo-Generative Approach to Explainable Multimodal COVID-19 Misinformation Detection TheWebConf 2022
Lanyu Shang, Ziyi Kou, Yang Zhang, Dong Wang
Human-AI Interaction Towards Natural Language Explanation based COVID-19 Misinformation Detection IJCAI 2022
Ziyi Kou, Lanyu Shang, Yang Zhang, Zhenrui Yue, Huimin Zeng, Dong Wang
Improve CAM with Auto-adapted Segmentation and Co-supervised Augmentation WACV 2021
Ziyi Kou, Guofeng Cui, Shaojie Wang, Wentian Zhao, Chenliang Xu
How to make a BLT sandwich? learning to reason towards understanding web instructional videos WACV 2021
Shaojie Wang, Wentian Zhao, Ziyi Kou, Chenliang Xu
Talking-head generation with rhythmic head motion ECCV 2020
Lele Chen, Guofeng Cui, Celong Liu, Zhong Li, Ziyi Kou, Yi Xu, Chenliang Xu

Awards