Ziyi Kou

Ziyi Kou

Senior Research Scientist - Meta Reality Lab Research

I am a senior research scientist at Meta Reality Lab. My research focuses on visual-language-action (VLA) modeling, multimodal representation learning and 3D pose tracking. I obtained my Ph.D. degree from Computer Science and Engineering (CSE) department at University of Notre Dame. My advisor is Dr. Xiangliang Zhang. I received my master's degree of Computer Science from University of Rochester and my bachelor's degree of Software Engineering from Chongqing University.

Education

  • Aug. 2020 - May 2024
    University of Notre Dame
    Ph.D., Computer Science and Engineering Courses: Text Mining, Algorithm and Complexity, Operating System, Computer Graphic
  • Aug. 2018 - May 2020
    University of Rochester
    M.S. Computer Science Courses: Artificial Intelligence, Adv. Topic of Computer Vision, Intro. of Statistics, Machine Learning
  • Sept. 2014 - Jun. 2018
    Chongqing University, China
    B.S. Software Engineering Courses: Java Programming, Data Mining with Python, Web Development, Database Design, Android Programming

Recent Work Experience

  • June. 2024 - Present
    Research Scientist
    Meta Multi-Modal Foundational Computer Vision
  • Dec. 2023 - Mar. 2024
    Research Scientist Intern
    Meta Visual Knowledge Transfer for 3D Glove Tracking in VR
  • June. 2023 - Aug. 2023
    Applied Scientist Intern
    Amazon Graph Neural Network Based Customer Insotck Value Estimation. Received return incline offer.
  • Oct. 2022 - Feb. 2023
    Machine Learning Engineer Intern
    Instacart Graph Based Next-Basket Recommendation. Published at CIKM2023.
  • May. 2022 - Aug. 2022
    Machine Learning Engineer Intern
    Godaddy Inc. Multi-relational Graph Based Recommendation. Received return intern offer.

My Skills

Programming

Machine Learning

Technical App.

Awards

Main Publications

* denotes equal contribution

Glove2Hand: Synthesizing Natural Hand-Object Interaction from Multi-Modal Sensing Gloves
Xinyu Zhang, Ziyi Kou, Chuan Qin, Mia Huang, Ergys Ristani, Ankit Kumar, Lele Chen, Kun He, Abdeslam Boularias, Li Guan
CVPR 2026
AirGlove: Exploring Egocentric 3D Hand Tracking and Appearance Generalization for Sensing Gloves
Wenhui Cui, Ziyi Kou, Chuan Qin, Ergys Ristani, Li Guan
ICASSP 2026
Adaptive and Context-rich Generative Self-supervised Learning on Graphs
Yijun Tian, Chuxu Zhang, Ziyi Kou, Zheyuan Liu, Xiangliang Zhang, Nitesh V Chawla
AAAI 2026
Flowing from reasoning to motion: Learning 3d hand trajectory prediction from egocentric human interaction videos
Mingfei Chen, Yifan Wang, Zhengqin Li, Homanga Bharadhwaj, Yujin Chen, Chuan Qin, Ziyi Kou, Yuan Tian, Eric Whitmire, Rajinder Sodhi, Hrvoje Benko, Eli Shlizerman, Yue Liu
Arxiv (2025)
RAt: Injecting Implicit Bias for Text-To-Image Prompt Refinement Models
Ziyi Kou, Shichao Pei, Meng Jiang, Xiangliang Zhang
EMNLP 2024
LeMon: Automating Portrait Generation for Zero-Shot Story Visualization with Multi-Character Interactions
Ziyi Kou, Shichao Pei, Xiangliang Zhang
KDD 2024
A Controllable Prompt Adversarial Attacking Framework for Black-Box Text Guided Image Generation Models
Ziyi Kou, Shichao Pei, Yijun Tian, Xiangliang Zhang
IJCAI 2023
A Crowd-AI Duo Relational Graph Learning Framework Towards Social Impact Aware Photo Classification
Yang Zhang, Ziyi Kou, Lanyu Shang, Huimin Zeng, Zhenrui Yue, Dong Wang
AAAI 2023
A Web Crowdsourcing Based Face Partition Approach Towards Privacy-Aware Face Recognition
Ziyi Kou, Lanyu Shang, Yang Zhang, Siyu Duan, Dong Wang
TheWebConf 2022
A Duo-Generative Approach to Explainable Multimodal COVID-19 Misinformation Detection
Lanyu Shang, Ziyi Kou, Yang Zhang, Dong Wang
TheWebConf 2022
Human-AI Interaction Towards Natural Language Explanation based COVID-19 Misinformation Detection
Ziyi Kou, Lanyu Shang, Yang Zhang, Zhenrui Yue, Huimin Zeng, Dong Wang
IJCAI 2022
Improve CAM with Auto-adapted Segmentation and Co-supervised Augmentation
Ziyi Kou, Guofeng Cui, Shaojie Wang, Wentian Zhao, Chenliang Xu
WACV 2021
How to make a BLT sandwich? learning to reason towards understanding web instructional videos
Shaojie Wang, Wentian Zhao, Ziyi Kou, Chenliang Xu
WACV 2021
Talking-head generation with rhythmic head motion
Lele Chen, Guofeng Cui, Celong Liu, Zhong Li, Ziyi Kou, Yi Xu, Chenliang Xu
ECCV 2020