Home       Trees       Indices       Help   
Package iceprod :: Package server :: Module grid :: Class iGrid
[hide private]
[frames] | no frames]

Class iGrid

source code

Known Subclasses:

This class represents a generic job a distributed system.

Instance Methods [hide private]
 
__init__(self, size=1) source code
 
QueueJobs(self, db, maxjobs, grid_id, jobs_at_once=20, fifo=True, debug=0) source code
 
SetCopyClass(self, cpobj) source code
 
urlcopy(self, file, url) source code
 
PushJob(self, job)
Add new job to queue		 source code
 
PopJob(self)
remove and return new job from queue		 source code
 
GetJob(self, i)
return job from queue		 source code
 
GetJobList(self)
Get list of jobs		 source code
 
AddEnv(self, var, val)
Add an environment variable.		 source code
 
GetEnv(self) source code
 
format_arg(self, tuple)
format runtime cmdline argument		 source code
 
SetInitialdir(self, path)
Define the directory where jobs will be submitted from.		 source code
 
GetInitialdir(self)
Get the directory where jobs will be submitted from.		 source code
 
SetSteering(self, steering)
Add a reference to the steering configuration		 source code
 
GetSteering(self)
Get the reference to the steering configuration		 source code
 
GetJobIds(self)
Get the cluster AND job id for the submitted jobs.		 source code
 
GetClusterId(self)
Get the cluster AND job id for the submitted jobs.		 source code
 
SetMetadataPath(self, path) source code
 
GetMetadataPath(self) source code
 
UpdateMetadata(self)
Update metadata information		 source code
 
GetRootDir(self)
Get the root directory of installation		 source code
 
SetRootDir(self, path)
Define the root directory of installation		 source code
 
FileName(self)
Get the name of this object's serialized file		 source code
 
SetMonitorDB(self, db) source code
 
GetMonitorDB(self) source code
 
SetProduction(self, production) source code
 
GetProduction(self) source code
 
SetQueue(self, nproc) source code
 
GetQueue(self)
Returns: number of jobs enqueueued		 source code
 
SetSimDBKey(self, key)
Set the unique key for current configuration on production database		 source code
 
GetSimDBKey(self)
Get the unique key for current configuration from production database		 source code
 
SetSubmitHost(self, host)
Set the hostname of the submit node		 source code
 
GetSubmitHost(self)
Get the hostname of the submit node		 source code
 
SetHost(self, host)
Set the hostname of the soaptray server that the job was submitted to		 source code
 
GetHost(self)
Get the hostname of the soaptray server that the job was submitted to		 source code
 
SetPort(self, port)
Set the port number of the submit node		 source code
 
GetPort(self)
Get the port number of the submit node		 source code
 
SetURL(self, url)
Set the hostname of the submit node		 source code
 
GetURL(self)
Get the hostname of the submit node		 source code
 
SetStatus(self, status)
(0=init submission,1=runnig,2=completed,3=failed)		 source code
 
_submit(self, job)
Submit job		 source code
 
Submit(self, cookie)
Submit job/cluster		 source code
 
CheckQ(self, db=None)
Interface: Check status of job/cluster in queuing system.		 source code
 
CleanQ(self, jobs=None)
Interface: Check status of job/cluster in queuing system.		 source code
 
CleanDir(self, dir)
Remove temporary directory where the current job(s) was submitted from.		 source code
 
CheckJobStatus(self, jobs)
Querie status of job on queue		 source code
 
IsUrl(self, path) source code
 
QRemove(self, jobid)
Interface: Remove active job/cluster from queuing system.		 source code
 
PostCopy(self, jobdict, target_url, maxtries=4)
Interface: Remove active job/cluster from queuing system.		 source code
 
Clean(self, jobdict, force=False)
Interface: clean submit directory		 source code
 
wait(self, job) source code
 
jobfinalize(self, dataset_id, job_id, job, status='OK', clear_errors=True)
Update status for job		 source code
 
reset_old_jobs(self, grid_id, maxidletime, maxruntime, maxsubmittime, maxcopytime, maxfailures=10, maxevicttime=10, keepalive=14400)
reset status of jobs that where queued but who's status has not changed in more that maxtime minutes		 source code
 
GetResetJobs(self, grid_id, max_reset=50) source code
 
GetFinishedJobs(self, grid_id, max_reset=50) source code
 
GetActiveJobs(self, grid_id) source code
 
GetProcessingJobs(self, grid_id, max_reset=50) source code
 
GetQueuedJobs(self, grid_id, max_reset=50) source code
 
SelfPickle(self)
Serialize iGrid object and write it to a file		 source code
 
AddParam(self, name, value)
Add a classadd parameter to be included in submit file		 source code
 
GetParam(self, name)
Returns: the value bound to parameter given by name, None if parameter has not been set.		 source code
 
GetParamKeys(self)
Returns: a list of parameter names that have been set.		 source code
 
AddArgOption(self, optname, optval='')
Add a options to be passed to executable.		 source code
 
GetArgOptions(self)
Returns: a list of options passed to the executable		 source code
 
GetArgOpt(self, optname)
Get value of option to be passed to executable.		 source code
 
Suffix(self)
Suffix of submit script		 source code
 
GetLogs(self, job)
Read output logs from job		 source code
Class Variables [hide private]
  CopyStatusEnum = {'FAILED': 0, 'NOTREADY': 0, 'OK': 1}
Method Details [hide private]

format_arg(self, tuple)

source code 

format runtime cmdline argument

Parameters:
  • tuple - tuple in the form (argname,argval)

SetInitialdir(self, path)

source code 

Define the directory where jobs will be submitted from.

Parameters:
  • path - system path to directory

GetInitialdir(self)

source code 

Get the directory where jobs will be submitted from.

Returns:
path to directory

SetSteering(self, steering)

source code 

Add a reference to the steering configuration

Parameters:
  • steering - a Steering object

GetSteering(self)

source code 

Get the reference to the steering configuration

Returns:
the Steering object

GetJobIds(self)

source code 

Get the cluster AND job id for the submitted jobs.

Returns:
a list of jobs with their cluster and job id in the condor format None if no jobs have been submitted or if submission failed.

GetClusterId(self)

source code 

Get the cluster AND job id for the submitted jobs.

Returns:
a list of jobs with their cluster and job id in the condor format None if no jobs have been submitted or if submission failed.

GetRootDir(self)

source code 

Get the root directory of installation

Returns:
path to directory

SetRootDir(self, path)

source code 

Define the root directory of installation

Parameters:
  • path - system path to directory

FileName(self)

source code 

Get the name of this object's serialized file

Returns:
path to directory

SetQueue(self, nproc)

source code 
Parameters:
  • nproc - number of jobs to enqueue

GetQueue(self)

source code 
Returns:
number of jobs enqueueued

_submit(self, job)

source code 

Submit job

Parameters:
  • config_file - path to file were submit file will be written

Submit(self, cookie)

source code 

Submit job/cluster

Parameters:
  • cooke - cookie to store submit info to be return to submitter

jobfinalize(self, dataset_id, job_id, job, status='OK', clear_errors=True)

source code 

Update status for job

Parameters:
  • dataset_id - dataset index
  • job_id - process number within dataset

reset_old_jobs(self, grid_id, maxidletime, maxruntime, maxsubmittime, maxcopytime, maxfailures=10, maxevicttime=10, keepalive=14400)

source code 

reset status of jobs that where queued but who's status has not changed in more that maxtime minutes

Parameters:
  • grid_id - id of current cluster
  • maxruntime - maximum run time for jobs
  • maxsubmittime - maximum submit time for jobs
  • maxcopytime - maximum time for jobs to be in 'copying' state
  • maxfailures - maximum number of time a job is allowd to fail
  • keepalive - how often should server expect to hear from jobs

AddParam(self, name, value)

source code 

Add a classadd parameter to be included in submit file

Parameters:
  • name - of parameter to be set
  • value - to be bound to parameter (this can be a classadd expression)

GetParam(self, name)

source code 
Parameters:
  • name - name of parameter to retrieve
Returns:
the value bound to parameter given by name, None if parameter has not been set.

GetParamKeys(self)

source code 
Returns:
a list of parameter names that have been set.

AddArgOption(self, optname, optval='')

source code 

Add a options to be passed to executable. Similar to 'AddArgument' but options will preceed arguemtens For example: executable <option(s)> <argument(s)> Note: Consecutive calls to this method will append options in the order in which they were added.

Parameters:
  • optname - name of option be passed to executable at runtime
  • optval - value of option be passed to executable at runtime

GetArgOptions(self)

source code 
Returns:
a list of options passed to the executable

GetArgOpt(self, optname)

source code 

Get value of option to be passed to executable.

Parameters:
  • optname - name of argument to be passed to executable at runtime

   Home       Trees       Indices       Help   
Generated by Epydoc 3.0.1 on Fri Jan 4 16:45:11 2013 http://epydoc.sourceforge.net