The 2004 ingest finished last night about about 19:00. At about 23:00 I started a 2005 ingest.
While trying to ingest SPASA data using a single E2005 collector on sam, we ran out of disk space and
2011-06-16 00:27:43.111 E-42FDC940: E2005_COLR: reducing MaxCollectionSpace from 2217132437 to 2160000000 2011-06-16 00:27:43.133 E-433E0940: E2005_COLR: reducing MaxStagingSpace from 150000000000 to 4337385984One file didn't copy. I copied it later. However, after that disk reads seem to have gone to 0.
chuma is receiving the files, and a diff between the /hsm and /data/exp for the first file in the IceCube/2005 set (loaded about 5am--currently 9:30) shows no differences. So it was written OK, and since disk writes to /rfs have been pretty steady at 60MB/sec that should have more than filled the /rfs partition three times over--which means the /hsm file info had to have been retrieved from chuma. So data is getting where it belongs--but how is it doing that without disk reads?
We have a 1GB RAM disk on sam for /rfs/cache--which is too small for the whole collection.
So, why is there a steady accumulation of data on disk that doesn't seem to get cleaned up neatly? And why are we apparently running along smoothly without and disk reads?
Modified 16-June-2011 at 09:35
http://icecube.wisc.edu/~jbellinger/StorHouse/16Jun2011
| Previous notes | Next notes | Main slide directory |