Data Storage & Transfer: Basics and Best Practices

It is important that researchers not spend too much time and effort transferring datasets from one place to another. This session will introduce the basics of data transfer so that participants can identify bottlenecks that slow down a transfer job and learn how to overcome them. The session will also introduce various data transfer tools that researchers can use in their daily workflow. The hands-on part will focus on the Globus transfer tool (https://www.globus.org/)

Learning objectives:

Attendees will be able to select the best data transfer tool for different transfer jobs. Attendees will also learn the best way to transfer large and small datasets from and to the Research Computing clusters (Perseus, Della, Tiger, etc) at Princeton.

Knowledge prerequisites:

The session assumes no prior experience or knowledge about networking or data transfer practices. Basic facility with Linux and the Bash command-line is required.

Hardware/software prerequisites:

If you would like to participate in the hands-on activity, you will need to enable your Princeton Unix account and make sure that you can connect to Princeton’s Nobel machine via SSH. Here are the steps:
1. Enable your Princeton Unix account (if you have not already done so), as described in the PICSciE Workshop Requirements page (https://researchcomputing.princeton.edu/learn/workshops-live-training/hardware-and-software-requirements-picscie-workshops).
2. Make sure you can connect via SSH to Princeton’s Nobel computers. For instructions on how to do so for your operating system, visit the PICSciE Workshop Requirements page (https://researchcomputing.princeton.edu/learn/workshops-live-training/hardware-and-software-requirements-picscie-workshops) and see the section titled “Connecting to Adroit” (you need the instructions under “SSH Connection”, not the shorter ones under “Web Portal”), making the following changes:
-everywhere you see “Adroit”, replace it with “Nobel”
-ignore any references to VPN (you don’t need it to connect to Nobel)
3. Finally, verify that you can connect to Nobel (follow the steps on the PICSciE Workshop Requirements page (https://researchcomputing.princeton.edu/learn/workshops-live-training/hardware-and-software-requirements-picscie-workshops) in the section titled “SSH into Adroit”, but put “Nobel” everywhere you see “Adroit”.

Session format:

Lecture and hands-on

What to expect:
Mini workshop (one-off workshop –" 60 mins total)

Meet the facilitator:
Joon Kim is a Cyber Infrastructure Engineer (CIE) in the Princeton Institute for Computational Science & Engineering (PICSciE) and an Associate Research Scholar in the Computer Science department. He works to improve Princeton University’s network infrastructure to better support scientific research. Joon has a Ph.D. in Computer Science from Georgia Tech, specializing in computer networks.

Rishi Joshi is a Research Data Management and Storage Engineer. His focus is in enabling research by applying supercomputing architecture to new areas of need. Before joining Princeton’s Advanced Systems and Storage team, Rishi was fortunate to work on the largest private and public clusters as a specialty platform engineer, where he found a passion in bringing HPC infrastructure and tools to AI, research, and media platforms previously facing bottlenecks. Now at Princeton, his goal is to strategically architect storage services to handle the next generations of diverse research workloads. Rishi received his MS in Information Assurance and Cybersecurity.

To request accommodations for this event, please contact the workshop or event facilitator at least 3 working days prior to the event.

Wintersession | View More Events
Co-hosted with: PICSciE/Research Computing

Contact the organizers