This HOWTO describes how to pass arguments to your program running on the cluster [[InfolabClusterCompute|Infolab Compute Cluster]]. We assume that you are already familiar with the basic principles of submitting jobs to the cluster. If that is not the case please see this tutorial first: [[InfolabClusterComputeHowtoSingle]]. <> = The program = We are going to use the script from [[InfolabClusterComputeHowtoSingle]] that we modified just so it prints out the arguments passed to it. You can download the script here [[attachment:SingleCoreVariables.py]]. {{{#!highlight python #!/usr/bin/python2.7 import socket, datetime, time, getpass, sys start = datetime.datetime.now() hostname = socket.gethostname().split('.')[0] username = getpass.getuser() time.sleep(10) end = datetime.datetime.now() dfmt = "%Y-%m-%d %H:%M:%S" print "Started: %s Finished: %s Host: %s User: %s" % (start.strftime(dfmt), end.strftime(dfmt), hostname, username) print "My arguments:" print sys.argv }}} The script starts, records the current time, figures out the hostname it is running on and the username it is running as. Then it sleeps for 10 seconds (so we at least have some impact on the cluster), records the time again and prints out a string that may look a little something like this (if we called it with myarg1, myarg2 and myarg3, of course): {{{ Started: 2012-10-16 18:19:47 Finished: 2012-10-16 18:19:57 Host: ilhead1 User: akrevl My arguments: ['./SingleCoreVariables.py', 'myarg1', 'myarg2', 'myarg3'] }}} = The submission script = == Static parameters == In this example we show how to run our program with parameters if these parameters are static and can be coded into the submission script. Perhaps not very useful, but here it goes anyway... {{{#!highlight bash #!/bin/bash #PBS -N SingleCoreVariablesJob #PBS -l nodes=1:ppn=1 #PBS -l walltime=00:01:00 /usr/bin/python2.7 /afs/cs.stanford.edu/u/akrevl/tutorial/SingleCore/SingleCoreVariables.py myarg1 myarg2 myarg3 }}} Once our job completes, the output is: {{{ ~/ $ cat SingleCoreVariablesJob.o4656 Started: 2012-10-16 18:51:31 Finished: 2012-10-16 18:51:41 Host: iln28 User: akrevl My arguments: ['/afs/cs.stanford.edu/u/akrevl/tutorial/SingleCoreVariables/SingleCoreVariables.py', 'myarg1', 'myarg2', 'myarg3'] }}} == -V: using environment variables == What if we would like to pass arguments along with the qsub command? We can try a script like this: {{{#!highlight bash #!/bin/bash #PBS -N SingleCoreVariablesJob #PBS -l nodes=1:ppn=1 #PBS -l walltime=00:01:00 /usr/bin/python2.7 /afs/cs.stanford.edu/u/akrevl/tutorial/SingleCore/SingleCoreVariables.py $1 $2 $3 }}} However if we try to run the ''qsub'' with our three arguments it will complain that we have supplied wrong arguments (actually it just prints out the usage information, but we should take the hint). {{{ ~/ $ qsub -V $HOME/tutorial/SingleCoreVariables/SingleCoreVariables.qsub.sh myarg1 myarg2 myarg3 usage: qsub [-a date_time] [-A account_string] [-b secs] [-c [ none | { enabled | periodic | shutdown | depth= | dir= | interval=}... ] ... }}} Instead of passing arguments by the command line, we can pass them as environment variables. Let's export our arguments first: {{{#!highlight bash ~/ $ export qsubarg1="myarg1" ~/ $ export qsubarg2="myarg2" ~/ $ export qsubarg3="myarg3" }}} Now let's make the adjustments to the submission script. We need to use qsubargX (you can download the script here [[attachment:SingleCoreVariables.qsub.sh]]): {{{#!highlight bash #!/bin/bash #PBS -N SingleCoreVariablesJob #PBS -l nodes=1:ppn=1 #PBS -l walltime=00:01:00 /usr/bin/python2.7 /afs/cs.stanford.edu/u/akrevl/tutorial/SingleCoreVariables/SingleCoreVariables.py $qsubarg1 $qsubarg2 $qsubarg3 }}} Let's try and submit this to the cluster (do not forget the -V switch): {{{ qsub -V /afs/cs.stanford.edu/u/akrevl/tutorial/SingleCoreVariables/SingleCoreVariables.qsub.sh }}} ''qsub'' made sure that all the environment variables were passed to the execution node and our program ran with the provided arguments. {{{ ~/ $ cat SingleCoreVariablesJob.o4657 Started: 2012-10-16 19:21:13 Finished: 2012-10-16 19:21:23 Host: iln28 User: akrevl My arguments: ['/afs/cs.stanford.edu/u/akrevl/tutorial/SingleCoreVariables/SingleCoreVariables.py', 'myarg1', 'myarg2', 'myarg3'] }}} == -v: listing the variables == Another way of passing the arguments to ''qsub'' is to just list them as key=value pairs. Let's keep the submission script we used in the previous example (you can download the script here [[attachment:SingleCoreVariables.qsub.sh]]): {{{#!highlight bash #!/bin/bash #PBS -N SingleCoreVariablesJob #PBS -l nodes=1:ppn=1 #PBS -l walltime=00:01:00 /usr/bin/python2.7 /afs/cs.stanford.edu/u/akrevl/tutorial/SingleCoreVariables/SingleCoreVariables.py $qsubarg1 $qsubarg2 $qsubarg3 }}} We can list the argument values as part of the ''qsub'' command using the -v switch (we could actually omit the -V switch, but we just got used to it, so why bother): {{{ qsub -V -v qsubarg1="myarg1",qsubarg2="myarg2",qsubarg3="myarg3" /afs/cs.stanford.edu/u/akrevl/tutorial/SingleCoreVariables/SingleCoreVariables.qsub.sh }}} ''qsub'' made sure that the specified arguments were available on the execution node and our program used them as you can see in the following listing: {{{ ~/ $ cat SingleCoreVariablesJob.o4658 Started: 2012-10-16 19:28:55 Finished: 2012-10-16 19:29:05 Host: iln28 User: akrevl My arguments: ['/afs/cs.stanford.edu/u/akrevl/tutorial/SingleCoreVariables/SingleCoreVariables.py', 'myarg1', 'myarg2', 'myarg3'] }}}