The plan is to create sDST archive files for all IC86 and later runs. For all previous runs the level2 will be deemed to be the archive.
Full PFRaw files arrive on tape. On average it requires 100 ± 10 minutes per file on the farm to translate them into sDST. The reduction factor is about 4.3. The nominal reduction is about 8, but only 44% of events are in PFFilt, so the sDST archive includes almost twice as many events.
By request there is no effort made to avoid duplication: all events are processed whether or not they were beamed North earlier.
Some subruns are much sorter than others, and some runs are bad. I do not attempt to exclude bad runs! "Bad" is somewhat flexible, and since this is to serve as an archive I judge it appropriate to keep everything. If someone changes his mind later there's no need to respin tapes.
I need to (8-July-2013) run full processing on a full run, to compare with the processing done on the PFFilt. There should be no difference for most analyses, but this has not been demonstrated.
This is, so far, processed by scripts run by hand rather than the complete machine.
As jbellinger, on onyx,
cd ~jbellinger/ARCHIVE/dumptape eval `/net/local/software/setup.sh` python TDCspawn.py `pwd`/debugout.log >& shortrun.log & # This loads tapes into drives and does some bookkeeping. vi byhand.com # change the tape numbers (e.g. TAPE_00104 to TAPE_00107, Tape_00104 to Tape_00107, etc) ./byhand.com # Wait about 5-6 hours. MAKE SURE THE DUMP IS DONE BEFORE PROCEEDING. # This creates Tape_0xxxx.log files which should contain all the file # names # When the dump is done, look at (e.g.) /data/exp/stagetape/TAPE_00107/getunique.com # This should exist and not be empty. It's object is to rename files from # tempstore to goodfiles such that duplicate files are left behind. cd /data/exp/stagetape/TAPE_00107 more ./getunique.com # This should have already been executed, so there should be files in # goodfiles corresponding to the entries in getunique.com # Set the group ownership so that sdstarchive can write there! sudo chown -R jbellinger:sdst TAPE_00107
As sdstarchive, on npx4
cd /data/user/sdstarchive/ARCHIVE/CandC # Determine which tape dumps need to be processed. Make sure this # happens AFTER the tape is done dumping! This is a by-hand check. # Execute the job creation file, here for TAPE_00107 ./drivesubmit.sh TAPE_00107 condor_submit job.TAPE_00107.submit # Wait for a day or so until the jobs are done.
Checking for problems in jobs is fairly easy:
cd /data/user/sdstarchive/ARCHIVE/TAPE_00107/log grep 'return value' * | grep -v 'value 0' # anything that turns up (e.g. in log.88) you can do more ../output/log.88 # and one of the first things in the output file is the list of files # processed. The logging is not extensive., though # I have a script in /data/user/sdstarchive/ARCHIVE/CandC that # checks for bad return codes: ./countLogs.sh TAPE_00103
Some files give checksum errors. Curiously enough, most of them seem to process well enough until the end or almost the end.
| Tape ID | Run name | Unique good | Failed runs | Missing | Also at |
| 00101 | tape_2012_300_RAW_copy_1 | 558 | Run00120288_Subrun00000000_00000000 | 299 | |
| 00102 | tape_2012_301_RAW_copy_1 | 591 | Run00120288_Subrun00000000_00000001 | 301,302 Fixed in 305 | |
| 00103 | tape_2012_302_RAW_copy_2 | 395 | Fixed in 306 | ||
| 00104 | tape_2012_304_RAW_copy_2 | 565 | 0 | ||
| 00105 | tape_2012_303_RAW_copy_1 | 4 | 0 | ||
| 00106 | tape_2012_305_RAW_copy_1 | 394 | Fixed from 302 | ||
| 00107 | tape_2012_307_RAW_copy_1 | 28 | 0 | ||
| 00108 | tape_2012_306_RAW_copy_2 | 198 | 0 | ||
| 00109 | tape_2012_308_RAW_copy_2 | 546 | 0 | ||
| 00110 | tape_2012_320_RAW_copy_1 | 572 | Fixed in 324 | ||
| 00111 | tape_2012_309_RAW_copy_1 | 587 | Fixed in 311 | ||
| 00112 | tape_2012_321_RAW_copy_2 | 493 | 0 | ||
| 00113 | tape_2012_323_RAW_copy_2 | 593 | Fixed from 326 | ||
| 00114 | tape_2012_322_RAW_copy_1 | 396 | 0 | ||
| 00115 | tape_2012_324_RAW_copy_1 | 556 | Fixed from 325 | ||
| 00116 | tape_2012_326_RAW_copy_1 | 200 | Fixed from 328 | ||
| 00117 | tape_2012_325_RAW_copy_2 | 135 | Fixed from 329 | ||
| 00118 | tape_2012_327_FILTERED_copy_1 | 0 | 0 | ||
| 00119 | tape_2012_340_RAW_copy_1 | 505 | Fixed from 342 | ||
| 00120 | tape_2012_329_RAW_copy_1 | 417 | 0 | ||
| 00121 | tape_2012_341_RAW_copy_2 | 589 | 0 | ||
| 00122 | tape_2012_342_RAW_copy_2 | 0 | 0 | ||
| 00123 | tape_2012_328_RAW_copy_2 | 2 | 0 | ||
| 00124 | tape_2012_343_RAW_copy_1 | 0 | 0 | ||
| 00125 | tape_2012_342_RAW_copy_2 | 407 | 0 | BEGIN FULL EHE | |
| 00126 | tape_2012_328_RAW_copy_2 | 393 | 0 | ||
| 00127 | tape_2012_343_RAW_copy_1 | 202 | 0 | ||
| 00128 | tape_2012_348_RAW_copy_1 | 589 | 0 | ||
| 00129 | tape_2012_347_RAW_copy_2 | 588 | 0 | ||
| 00130 | tape_2012_349_RAW_copy_1 | 204 | 0 | ||
| 00131 | tape_2012_311_RAW_copy_2 | 400 | 0 | ||
| 00132 | tape_2012_310_FILTERED_copy_1 | 0 | 0 | ||
| 00133 | tape_2012_312_RAW_copy_1 | 576 | 0 | ||
| 00134 | tape_2012_314_RAW_copy_1 | 195 | 0 | ||
| 00135 | tape_2012_313_RAW_copy_2 | 66 | 0 | ||
| 00136 | tape_2012_315_RAW_copy_2 | 387 | 0 | ||
| 00137 | tape_2012_317_RAW_copy_2 | ? | |||
| 00138 | tape_2012_316_RAW_copy_1 | ? | |||
| 00139 | tape_2012_318_RAW_copy_1 | ? | |||
| 00140 | tape_2012_330_RAW_copy_2 | ? | |||
| 00141 | tape_2012_319_RAW_copy_2 | ? | |||
| 00142 | tape_2012_331_RAW_copy_1 | ? |