I tried running 3 ingests simultaneously, and ran out of space on the disk. Somewhere. Apparently RFS server doesn't throttle when it runs low on staging space. Two rsync's died, leaving behind incomplete files (one each). I copied replacements by hand, which means they're now owned by root in the StorHouse (oh well) and restarted the rsync jobs (sequentially). 1997, 1998, 1999, and 2000 data are now in the StorHouse in the appropriate directories.
Since I needed to run rsync in a mode that follows symbolic links, in order to get my organizing scheme to work, some problems appeared. Where there were broken symlinks rsync complained (748 in one job)--and some of the breaks are apparently quite old.
Speeds are not as rapid as one would like (especially if I have to run rsync jobs sequentially!). Note that the small files are copied very slowly.
Modified 19-May-2011 at 11:54
http://icecube.wisc.edu/~jbellinger/StorHouse/19May2011
| Previous notes | Next notes | Main slide directory |