AMANDA Monte Carlo with Simuperl

John Kelley, UW-Madison, April 2008

This page describes how to install and run the latest version (V02-02-04) of Paolo Desiati's simuperl package for AMANDA MC production.


The most important changes in this version of simuperl are:

Update August 2007: Starting with V02-02-00, simuperl Amasim + Photonics defaults to the AHA ice model. Millennium is still available by tweaking the $PHOTVER and $PHOTPATH variables in master.pl. Furthermore, the electrical files for AMANDA use the old, canonical OM sensitivities again. Otherwise, detector configuration files are as mentioned below.

IMPORTANT: The AHA tables are not availble on the NPX2 nodes (slowing things down a bit) or at all on GLOW (so Amasim is broken there). This should be fixed in a week or two.

 

Update July 2007: Starting with V02-00-00-beta, simuperl is transitioning to the aluminum-opt5 [physto.se] version of Amasim. The binary of amasim in simuperl is compiled with gcc and does not use Intel-specific processor optimizations. It is still ~4.5x faster than the aluminum-fix version (thanks Johan!).

Additionally, we have updated the following detector configuration files:

Please let me know if you have any questions.

To install the latest version of simuperl, do the following:

  1. Set up ssh: if you are planning on running simuperl using the NPX2 cluster in Madison, you will need to set up an SSH keypair so that you can ssh into the npx2 machine without any user input. (Note that you may also have to do the same thing elsewhere if you are planning on modifying simuperl to work on your local cluster system.)
    1. First generate your ssh keys. Accept all the default suggestions when prompted and just press "enter" for an empty passphrase when asked to provide one.
      % ssh-keygen -t rsa
    2. Add your public key to the list of authorized hosts:
      % cd ~/.ssh
      % cat id_rsa.pub >> authorized_keys
    3. Make sure the host key for "npx2.icecube.wisc.edu" is added to your known hosts list (using the fully qualified host name, not just "npx2" here is important). After this step, you may want to double-check that you can ssh into npx2 without any user input.
      % ssh npx2.icecube.wisc.edu
  2. Once logged into npx2, make sure the PBS commands are in your $PATH ("qstat", "qsub" and the like).
    % which qstat
    Currently these are located in /usr/pbs/bin/.
  3. If you are planning on using GLOW, you will be submitting jobs from the machine "skua" or "condor" (same machine) instead of "npx2". Once logged into condor, you will need to make sure the CONDOR commands are also in your path:
    % which condor_q
    Currently these are located in /opt/condor/bin/ (also add /opt/condor/sbin/ for good measure).
  4. Checkout the simuperl package from the CVS repository at Madison (make sure you have your CVS_RSH environment variable set to "ssh"):
    % cvs -d pub.icecube.wisc.edu:/net/local/cvsroot co -r V02-02-04 simuperl
  5. Run the install script to create links in the bin/ directory:
    % cd simuperl
    % ./install.pl
  6. Run the binary install script to create links to the current binary versions:
    % cd bin
    % ./install_links.pl
  7. If you are not using Photonics: edit simuperl/master.pl so that $ENV{ICE} is set to "mamint", not "photonics". Then, you're done with the installation.
  8. If you are using Photonics:
    1. If you're in Madison and are using NPX2 or GLOW, copies of the tables are stored on the nodes or are accessible from them (the path is set by simuperl). See the $PHOTPATH variable in master.pl for details.
    2. Otherwise, download the Photonics tables and table lists from Johan's table page. You will need the AmandaCoord_AmandaSpan.tar table lists package, as well as either the sparse tables or the fully detailed tables for simulation.

      Now edit $PHOTPATH in master.pl to point to your table location, and $PHOTLIST to point to the table list file location.

To run simuperl, use the master.pl script. You can see a few notes on the process by Chihwa Song here. You can also see all the command-line options by running the script without any.

Many options can be changed with command-line switches, but you will need to edit the script if:

As an example, here's how to use master.pl to run 100 nusim jobs for the year 2003 through to Amasim, 20 at a time, on the NPX2 cluster at Madison, with output files going to wherever the link my_results points (you can also use a fully qualified path now for -outdir).

% ssh npx2
% cd <simuperl top dir>
% ./master.pl -jobs 20 -max 100 -start 1 -stop 4 -save ama -outdir my_results -generator nusim -year 2003

Good luck!

HELP! My jobs crashed!

Simuperl has a habit of silently dying when encountering an error. It does log information, but it's not necessarily intuitive how to find it.

Known issues:

Updates from V02-02-03 to V02-02-04:

Updates from V02-01-02 to V02-02-03:

Updates from V02-01-01 to V02-02-02:

Updates from V02-01-00 to V02-02-01:

Updates from V02-01-04-beta to V02-02-00:

Updates from V02-01-02-beta to V02-01-04-beta:

Updates from V02-01-01-beta to V02-01-02-beta:

Updates from V02-01-00-beta to V02-01-01-beta:

Updates from V02-00-04-beta to V02-01-00-beta:

Updates from V02-00-03-beta to V02-00-04-beta:

Updates from V02-00-02-beta to V02-00-03-beta:

Updates from V02-00-00-beta to V02-00-02-beta:

Updates from V01-07-00 to V02-00-00-beta:

Updates from V01-06-04 to V01-07-00:

Updates from V01-06-03 to V01-06-03:

Updates from V01-06-01 to V01-06-02:

Updates from V01-06-00 to V01-06-01:

Updates from V01-05-02 to V01-06-00:

Updates from V01-04-00 to V01-05-02:

Updates from V01-03-03 to V01-04-00:

Updates from V01-03-02 to V01-03-03:

Updates from V01-03-01 to V01-03-02:

Updates from V01-02-03 to V01-03-01:

Updates from V01-02-02 to V01-02-03:

Updates from V01-02-01 to V01-02-02:

Updates from V01-02-00 to V01-02-01:

Updates from V01-01-01 to V01-02-00:

Updates from V01-00-00 to V01-01-01:

Thanks — and please contact me if you have any questions.

Back to home page