Getting Started with Transformers for Language Modeling

This session will present a practical introduction to transformer models and their applications to modern natural language processing. Participants will be given an introduction to the inner workings of the transformer architecture. In addition, they will be shown examples of how to apply these models to their own datasets using PyTorch and the Hugging Face library.

Learning objectives: Extend your deep learning knowledge to be able to work with transformers.

Knowledge prerequisites: Participants should be comfortable training simple deep learning models.

Hardware/software prerequisites: (1) Bring a laptop which can connect to the eduroam wireless network. You will also need to be able to Duo authenticate to use campus resources. (2) Have an SSH client installed on your laptop. (3) Register for an account on Adroit and make sure that you can SSH to Adroit (https://bit.ly/3QER9Sv) before the workshop.

Session format: Demonstration and hands-on

Instructor bio: Dave Turner is a senior research software engineer at the Princeton Neuroscience Institute. Here, he works to help researchers scale their software to HPC resources including GPUs and distributed CPU workloads. Prior to Princeton, Dave worked in the industry as a data scientist building machine learning models for clients in the cable and telecommunications industry. He received his PhD from GA Tech in mechanical engineering where his research concentrated on generative models for 3D reconstruction of material microstructures from limited 2D data.

PICSciE/Research Computing | View More Events
Co-hosted with: GradFUTURES

Contact the organizers