Boyuan Zheng

About Me

Hi, I’m Boyuan, a first-year PhD student at The Ohio State University. I am fortunately advised by Prof. Yu Su. Before that, I received my Bachelor degree in Software Engineering from Northeastern University (China) and Master degree in Computer Science from The Johns Hopkins University where I was fortunate to work with Prof. Benjamin Van Durme.

Research Interests

I am interested in developing language agents that can free human beings from tedious tasks and assist with decision making, especially on web environments. Currently, I am working on the following directions:

Multimodality
Grounding, Planing, and Reasoning
Synthetic data
Agent Safety

News

[Oct. 2024] I will attend EMNLP’24 to present WebOlympus! Looking forward to meet new and old friends at Miami!
[Oct. 2024] I will attend COLM!
[May. 2024] SeeAct accepted to ICML’24! See you at Vienna!
[Apr. 2024] I will attend MSLD to present SeeAct at Ann Arbor! Looking forward to meet new and old friends!
[Feb. 2024] MMMU and multimodal web agents accepted to CVPR’24!
[Jan. 2024] Thrilled to release SeeAct, enabling everyone to use GPT-4V-based web agents with one click. More OSS supports to come!
[Oct. 2023] I will attend NeurIPS’23! Looking forward to meet new and old friends at New Orleans!
[Sept. 2023] PhD journey starts!
[June. 2023] Started summer internship in Amazon Alexa AI

Selected Publications

ArXiv

Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents

Boyu Gou, Ruohan Wang, Boyuan Zheng, Yanan Xie, Cheng Chang, Yiheng Shu, Huan Sun, Yu Su

PDF Code Project Page

EMNLP'24 Demo

WebOlympus: An Open Platform for Web Agents on Live Websites

Boyuan Zheng*, Boyu Gou*, Scott Salisbury*, Zheng Du*, Huan Sun, Yu Su

Code Project Page

ICML'24

GPT-4V(ision) is a Generalist Web Agent,if Grounded

Boyuan Zheng, Boyu Gou, Jihyung Kil, Huan Sun, Yu Su

PDF Code Project Page

CVPR'24

MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI

Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, Cong Wei, Botao Yu, Ruibin Yuan, Renliang Sun, Ming Yin, Boyuan Zheng, Zhenzhu Yang, Yibo Liu, Wenhao Huang, Huan Sun, Yu Su, Wenhu Chen

PDF Code Project Page Oral

NeurIPS'23

Mind2Web: Towards a Generalist Agent for the Web

Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Samuel Stevens, Boshi Wang, Huan Sun, Yu Su

Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS), 2023

PDF Code Project Page Spotlight

Experience

Internship

Applied Scientist Intern, Amazon Alexa AI
- Duration: 2023.06 - 2023.09

Education

Ph.D. of Computer Science, The Ohio State University
- Duration: 2023.09 - Present
- Advisor: Yu Su
M.S. of Computer Science and Engineering, The Johns Hopkins University
- Duration: 2021.09 - 2023.06
- Advisor: Benjamin Van Durme
B.Eng of Software Engineering, Northeastern University (China)
- Duration: 2017.09 - 2021.06