3  Storage and moving data

The three filesystems, what each one promises, how to get data in and out, and /project/cowy, the storage shared across ASCEND projects.

3.1 Where your data lives

3.1.1 Nothing is backed up

Nothing on MedicineBow is backed up. Not home, not project, not scratch.

There are snapshots (point-in-time copies; within about 20 days you can ask arcc-help@uwyo.edu to restore from one), but they live on the same storage array as your live data — if the array fails, the snapshots fail with it. Snapshots protect against accidental deletion, not hardware loss. Anything you can’t afford to lose needs a copy elsewhere: your institution’s storage, a cloud bucket, an ARCC S3 bucket (separate hardware, but also not backed up), or the separate Alcova service, which does have real nightly backups.

3.1.2 The three filesystems

Path Quota Snapshots Backups Purged
/home/$USER 50 GB yes NO only when you leave UW
/project/<project> 5 TB yes NO 6 months after project ends
/gscratch/$USER ($SCRATCH) 5 TB yes NO after 90 days of inactivity
/project/<project>/software +250 GB on top of project yes NO with the project
/project/cowy shared by all ASCEND projects yes NO —
/lscratch node-local, varies no NO wiped at the end of every job

Check your usage and limits for home, scratch, and project:

arccquota

3.1.2.1 /home — small, personal, for config not data

50 GB fills fast once a conda environment eats 15 GB of it. Home is for dotfiles, scripts, and small configs — not datasets and not software installs. When home fills up, things break (jobs fail to write logs, logins misbehave), so watch it with arccquota and du -sh ~/* | sort -h.

3.1.2.2 /project/<project> — the shared workhorse

5 TB (default), shared by everyone on your project. This is where shared datasets, inputs, and results belong. Agree on a directory convention with your group so you don’t clobber each other. It persists for 6 months after the project ends.

3.1.2.3 /project/cowy — shared across ASCEND projects

Every ASCEND project gets read/write access to /project/cowy, for sharing data between projects. It has no compute hours attached: you can’t submit jobs with --account=cowy, but jobs running under your own project can read and write it. The ASCEND PIs manage this space; keep your files in your own subdirectory (/project/cowy/<you>) or in the shared data/ directory, and ask them before creating new top-level directories.

3.1.2.4 /gscratch/$USER — fast and disposable

Also 5 TB, and the fast filesystem — use it as the working directory for jobs with heavy I/O. Files untouched for 90 days are purged automatically. Never keep the only copy of anything here.

3.1.2.5 /lscratch — node-local, gone when the job ends

Each compute node has its own local disk at /lscratch. It’s the fastest storage available to a job (no network in the way), ideal for scratch files a single job reads and rewrites constantly. It’s wiped when your job ends, so copy anything you want to keep back to /project or /gscratch before the job exits.

3.1.3 Install software in project space, not home

Install software into /project/<project>/software, not /home

Your project has an extra 250 GB quota at /project/<project>/software for installing software. A single conda environment with a few scientific packages can be 10–20 GB — two of those fills a 50 GB home directory.

Point conda, pip, R libraries, and manual builds at project software space from the start. The conda chapter (Section 4.2) shows how.

3.1.4 A sane layout to adopt on day one

/home/<you>/                      # dotfiles, small scripts, SSH keys
/project/<project>/
    software/                     # conda envs, container images, manual builds
    data/                         # shared input datasets
    <you>/                        # your working area within the project
/project/cowy/
    data/                         # datasets shared across ASCEND projects
    <you>/                        # your area in the shared space
/gscratch/<you>/                  # active job working dirs, heavy I/O, big temp output

3.2 Moving data in and out

3.2.1 Pick the right tool for the size of the job

Transfer Best tool
Hundreds of GB / TB, or must-not-corrupt datasets Globus
A few files, a script, a config scp or rsync over SSH
To/from cloud storage (AWS S3, Google Drive, etc.) rclone
Drag-and-drop a handful of files in a browser Portal → Files

There is no dedicated data-transfer node (DTN) — CLI transfers go through the login nodes, and Globus manages its own endpoint. Another reason to prefer Globus for big moves: it won’t tie up (or get killed on) a shared login node.

3.2.3 scp and rsync — fine for small, everyday transfers

From your laptop, push a file up:

scp results.csv <username>@medicinebow.arcc.uwyo.edu:/project/<project>/<you>/

Pull one down:

scp <username>@medicinebow.arcc.uwyo.edu:/gscratch/<you>/output.nc .

rsync is better for directories and for resuming — it only copies what changed, so an interrupted transfer picks up where it left off:

rsync -avP ./localdir/ <username>@medicinebow.arcc.uwyo.edu:/project/<project>/<you>/dir/

The -P flag shows progress and enables resume; -a preserves timestamps and permissions. For more than a few GB, prefer Globus — a login-node scp of a huge file competes with everyone else’s login-node activity and will be slow.

3.2.4 rclone — cloud storage (S3, Google Drive, Box…)

rclone is ARCC’s recommended CLI for moving data to and from cloud services. It’s a module:

module load gcc/14.2.0 rclone/1.63.1   # rclone needs a compiler module first
rclone config          # one-time: set up a "remote" (S3 bucket, Drive, etc.)
rclone copy mydata/ myremote:bucket/path/    # copy up
rclone copy myremote:bucket/path/ ./mydata/  # copy down

rclone verifies and retries, and handles many small files far better than a loop of scps.