CLAW Lab

Connecting Language to Actions & the World

Our research focuses on

grounding

action taking

small robot giving a gift to another robot

and theory-of-mind

All of our current work assumes the importance of perception and/or world building as crucial to constructing semantic representations for and with language. The categories above correspond roughly to WS3, WS4, and WS5 in Experience Grounds Language

PhD Students

Yingshan Chang 常影杉

@_Yingshanshe/her

Reasoning

LTI PhD

WebQA: Multihop and Multimodal QA

Jared Fernandez

@FernJaredhe/him

Efficiency

LTI & NSF GRFP

(co- Emma Strubell)

Vidhi Jain विधि जैन

@viddivjshe/her

Control

RI & FAIR AI resident

Transformers are Adaptable Task Planners

Tiffany Min 민소연

@SoYeonTiffMinshe/her

Embodiment

MLD

(co- Rus Salakhutdinov)

FILM: Following Instructions in Language with Modular Methods

Hao Zhu 朱昊

@_Hao_Zhuhe/him/他

Theory-of-mind

LTI

(co- Graham Neubig)

Few-shot Language Coordination by Modeling Theory of Mind

MRSD Teleop

(co- Zackory Erickson)

Jashkumar Diyora he/him

Shruti Gangopadhyay she/her

Jigar Patel
he/him

Prakhar Pradeep he/him

Shivani Sivakumar she/her

Alumni, Undergraduates, and Visitors

2022	Josh Zhanson	MLT	Embodied Vision	Data & Applied Sci @ MSFT
2022	Xiaoyu (Erin) Zhang	5th year MS	Robot Collaboration	PhD @ UIUC
2021	Feiyu (Gavin) Zhu	Undergrad	Video Summarization	co-Chris Paxton
2021	Blessing Bassey	Intern	TextVQA
2021	Melanie Sclar	Intern	Theory-of-Mind	PhD @ UWashington