The files in /net/user/aura are pretty small.
The transfer rates vary with file size--with some odd blips:
<
This is from the copy of /data/exp/IceCube/2007
Laying one atop the other:
The E2007 ingest completed, so I ingested files from a local disk. This is far faster:
In a little more detail, compare ingest speeds for /net/user/aura with those for local disk files. There is quite a lot of variation, but it has the same shape for both types of ingest, suggesting that it is more a function of either StorHouse or rsync than of network activity. In all cases the disk access histogram is shifted about 25-30% faster than the /net/user histogram.
The above plot is in seconds, which means that the transfer time should be at most 1 second. This is silly, and implies that what I'm looking at is merely the rate for the last second.
Copying a single file:
sending incremental file list
d.tar
32768 0% 0.00kB/s 0:00:00
39878656 33% 38.00MB/s 0:00:02
53248000 44% 21.21MB/s 0:00:03
76873728 64% 21.60MB/s 0:00:01
119078912 99% 25.84MB/s 0:00:00
119410176 100% 25.90MB/s 0:00:04 (xfer#1, to-check=0/1)
sent 119424826 bytes received 31 bytes 21713610.36 bytes/sec
total size is 119410176 speedup is 1.00
real 0m4.725s
user 0m1.012s
sys 0m1.173s
There are a few inconsistencies here. The asserted final speed, applied to the file size, gives 5.5 seconds--which is more than the actual elapsed time. The final reported speed for the transfer may reflect only the speed for the current chunk and not the average for the file. If I add up the 4 main chunks of transfer the time is 4.2sec and not 4.7
To make matters more amusing, if I copy this to /RFS instead I get the following (after restarting the rfsd, which I hope doesn't matter--though there was a lot of memory allocated):
sending incremental file list
d.tar
32768 0% 0.00kB/s 0:00:00
116555776 97% 111.12MB/s 0:00:00
119410176 100% 111.51MB/s 0:00:01 (xfer#1, to-check=0/1)
sent 119424826 bytes received 31 bytes 34121387.71 bytes/sec
total size is 119410176 speedup is 1.00
real 0m2.676s
user 0m0.971s
sys 0m0.714s
The transfer rate I get from dividing the file size by the clock time is 42.65MB/sec, which doesn't look like much of anything in the output.
Modified 18-August-2011 at 08:37
http://icecube.wisc.edu/~jbellinger/StorHouse/18Aug2011
| Previous notes | Next notes | Main slide directory |