Gail Weiss

Publications (Separate Talks Below)

(Notes: 1. If you don't have access to github or arxiv, contact me for a copy. 2. Google Colaboratory notebooks look messy, but press the open with colab button and they sort themselves out.)

• Transformers as recognizers of formal languages: A survey on expressivity – Preprint 2023
  Lena Strobl, William Merrill, Gail Weiss, David Chiang, Dana Angluin.
  PDF



• Discovering knowledge-critical subnetworks in pretrained language models – Preprint 2023
  Deniz Bayazit, Negar Foroutan, Zeming Chen, Gail Weiss, Antoine Bosselut.
  PDF



• RECKONING: Reasoning through Dynamic Knowledge Encoding – NeurIPS 2023
  Zeming Chen, Gail Weiss, Eric Mitchell, Asli Celikyilmaz, Antoine Bosselut.
  PDF // Github



• Neural Sequence Models: A Formal Lens – Technion 2023 (PhD Thesis)
  Gail Weiss.
  PDF // Seminar slides // Faculty link
  



• Thinking Like Transformers – ICLR Blogposts 2023
  Sasha Rush, Gail Weiss.
  Blog post // RASPy - RASP-like embedded DSL



• Extracting Automata from Recurrent Neural Networks Using Queries and Counterexamples (Extended Version) – Springer Machine Learning 2022
  Gail Weiss, Yoav Goldberg, Eran Yahav.
  Downloadable PDF (pre-Springer formatting) // Springer closed access link // Springer PDF viewer (no downloads) // Github
  Poster and talks linked for the regular version listed below (ICML 2018)
  Published in the Special Issue on Grammatical Inference



• Thinking Like Transformers – ICML 2021
  Gail Weiss, Yoav Goldberg, Eran Yahav.
  PDF // Github (RASP REPL) // Github (Experiments) // Poster
  5 min talk (ICML 2021): Slides, Talk // 1 hour talk (Edinburgh 2021): Slides, Talk



• Synthesizing Context-free Grammars from Recurrent Neural Networks – TACAS 2021
  Daniel Yellin, Gail Weiss.
  PDF // Github // 20 min talk (TACAS 2021): Slides and Talk



• A Formal Hierarchy of RNN Architectures – ACL 2020
  William Merrill, Gail Weiss, Yoav Goldberg, Roy Schwartz, Noah A. Smith, Eran Yahav.
  PDF // Will's Blog Post// 10 min talk (ACL 2020): Talk



• Learning Deterministic Weighted Automata with Queries and Counterexamples – NeurIPS 2019
  Gail Weiss, Yoav Goldberg, Eran Yahav.
  PDF // Github // Poster



• On the Practical Computational Power of Finite Precision RNNs for Language Recognition – ACL 2018
  Gail Weiss, Yoav Goldberg, Eran Yahav.
  PDF // Github // 10 min talk (ACL 2018): Talk // Google Colaboratory
  
  Note: This paper is mostly an observation that LSTMs can implement a counting mechanism and GRUs can't, so the code here is just a demonstration of this.



• Extracting Automata from Recurrent Neural Networks Using Queries and Counterexamples – ICML 2018
  Gail Weiss, Yoav Goldberg, Eran Yahav.
  PDF // Github // Poster (with thanks to Xiaokun Luan for corrections!) // 10 min talk (ICML 2018): Talk // Google Colaboratory
  Extended version listed above (Springer 2022)

Talks (Independent of Publications)

• Formal Abstractions of Neural Sequence Models – ICGI 2021 (Tutorial, 1.5hrs)
  Slides, Talk