The jyboss command provides python style access (using Jython ) to the IceCube detector. This allows for simple command line access to the detector's controls as well as providing scripting support.
The aim of this document is to provide an overview to you, a developer, on how you can use this tool to access the IceCube detector. This is not the definitive guide, which can be found elsewhere, rather it is designed to introduce the main concepts that you will need to know in order to get up and running using jyboss.
The jyboss command is installed automatically for the jboss user on the IceCube clusters. To use this command as another user, or on your own mercury installation, you simply need to go to http://glacier.lbl.gov/releases/jyboss/, select the version number you require, and download the jyboss file into your ~/bin directory. While you are there, you should also download the jyboss-scripts.tgz file. You should now be able to run jyboss as a normal command provided that your ~/bin directory is in your shell's PATH and you have the standard IceCube environment variables ICECUBE_TOOLS and JBOSS_ROOT (and tools) set up . (If you do not have the standard IceCube environment variables and tools set up you should follow the instruction at http://wiki.icecube.wisc.edu/index.php/Mercury_Dev_Sandbox.)
The jyboss command itself simply starts jython with the a classpath that enables it to access the jboss RMI mechanism. To use it to access the IceCube detector, the contents of the jyboss-scripts.tgz file need to be made available. You can achieve this by setting the JYBOSSPATH environment variable to point to the directory that will contain this file's contents. For the jboss user on the IceCube clusters this directory is ${HOME}/scripts and so we will use that here, if you chose a different directory name you will have to modify the next command. The following command sets the necessary variable.
[patton@spts-expcont patton]$ export JYBOSSPATH=${HOME}/scripts
To make life easy it is recommended that you add this declaration
into your .bashrc so it will always be set for
you.
You now need to populate that directory which is accomplished with the following command.
[patton@spts-expcont patton]$ tar xzf jyboss-scripts.tgz ${HOME}/scripts
You are now ready to try running jyboss.
The jyboss command is designed to be run on the cluster whose operations you wishes to control. For the rest of this document the SPTS cluster will be used as the example cluster, therefore you'll need to be logged on to spts-expcont node if you want to run these examples unmodified. Remember that you need to make sure that you are not interfering with anyone else working on SPTS. Once logged on, you should then move into (making it if necessary) a suitable directory (here I am using the demo directory in my home area) and execute jyboss. You will immediately notice that the first time the jyboss encounters a new .jar file, as it is now, it builds a synopsis of the file. Thereafter the command will start much more quietly (and quickly!).
To exit jyboss you simply follow the python model and press CTRL D .
Now, assuming DAQ is running, let us retrieve the current run number using jyboss. The following jyboss session show how this can be done.
[patton@spts-expcont demo]$ jyboss >>> import acmectrl >>> a = acmectrl.getInstance() >>> a.getRunNumber() 1129
The following are the features you should note.
The import acmectrl statement loads the experiment control module. This contains the AcmeCtrl class, which allows access to and control of various experiment wide features.
The a = acmectrl.getInstance() statement retrieves the global instance of this class that can then be used to access the experiment.
The a.getRunNumber() statement retrieves the current run number. As the result of the getRunNumber invocation is not assigned to a variable it is simply written to standard output.
As well as the AcmeCtrl class, there are control classes for each subsystem to deal with elements that are only of interest to that subsystem. Shortly, when we will walk through the simple example that starts and stops data-taking, we will use both the DAQ and dispatch-delivery control classes, but first we need to make sure the cluster is in the right state in order to run the examples.
To run the following examples we want the cluster to be in a known state. The easiest way to do this is to stop any JBoss servers that are running and restart them. (At this point it is assumed that all of the necessary deployments have been made.) The following start of a jyboss session show how you can stop all of the JBoss servers.
[patton@spts-expcont demo]$ jyboss >>> import acmectrl >>> a = acmectrl.getInstance() >>> locations = ['expcont', 'evbuilder', 'fpmaster', 'stringproc01', 'ichub21', 'icetop01', 'ithub01' ] >>> a.shutdownJBossSet(locations) [] >>> a.isJBossSetRunning(locations) []
The two return values of [] from the last two functions mean that there where, respectively, no problems with the shutdown command, and no JBosses were ultimately left running.
You can then proceed with the rest of the session and restart the JBoss servers.
>>> a.startupJBossSet(locations) >>> a.isJBossSetRunning(locations) ['expcont', 'evbuilder', 'fpmaster', 'stringproc01', 'ichub21', 'icetop01', 'ithub01']
The cluster should now be in a known state and ready to take data.
The following example walks through the steps you can take to start and stop data-taking by hand.
[patton@spts-expcont demo]$ jyboss
>>> import daqctrl
>>> d=daqctrl.getInstance()
>>> d.signal('DiscoverSig')
>>> d.getSummary()
'Idle'
>>> d.getComponents()
[('domHub_21', 'Idle'), ('domHub_81', 'Idle'), ('stringProcessor_21', 'Idle'), ('iceTopDataHandler_1', 'Idle'), ('inIceTrigger_0', 'Idle'), ('iceTopTrigger_0', 'Idle'), ('globalTrigger_0', 'Idle'), ('eventBuilder_0', 'Idle'), ('tcalBuilder_0', 'Idle'), ('monitorBuilder_0', 'Idle')]
The first two lines set the variable, d, to contain the global instance of the DaqCtrl class. This is then used to send a DiscoverSig to the DAQ subsystem and to see the state of that subsystem. The last statement returns a list of all discovered components and their current states.
You will now need to start the dispatch-delivery subsystem so that it can accept the data taken by DAQ. The following jyboss snippet does this as well as reporting the state of the dispatch-delivery subsystem.
>>> import acmectrl >>> a = acmectrl.getInstance() >>> a.startup(acmectrl.DD) >>> a.getSummary(acmectrl.DD) 'Started'
The next step is to connect the DAQ output to the dispatch-delivery input
>>> a.connect(acmectrl.DAQ, acmectrl.DD)
We are not quite ready to start data-taking. To do this requires DAQ to go through a number of state transitions. Some of these take some time, therefore the getSummary function may need to be executed a number of times until the correct state is reached. For clarity's sake, only the first and last calls to this function are included in the following snippet.
>>> d.signal('ConfigSig')
>>> d.getSummary()
'CheckFinalStates'
>>> d.getSummary()
'Ready'
>>> d.signal('ConfigDOMsSig')
>>> d.getSummary()
'CheckFinalStates'
>>> d.getSummary()
'Ready'
>>> d.signal('StartSig')
>>> d.getSummary()
'CheckFinalStates'
>>> d.getSummary()
'Running'
The DAQ is now up and running. We can check the state of all of its components by using the getComponents function again.
>>> d.getComponents()
[('domHub_21', 'Running'), ('domHub_81', 'Running'), ('stringProcessor_21', 'Running'), ('iceTopDataHandler_1', 'Running'), ('inIceTrigger_0', 'Running'), ('iceTopTrigger_0', 'Running'), ('globalTrigger_0', 'Running'), ('eventBuilder_0', 'Running'), ('tcalBuilder_0', 'Running'), ('monitorBuilder_0', 'Running')]
We can also monitor the number of events being handed to the dispatch-delivery subsystem using the following snippet.
>>> import ddctrl
>>> dd=ddctrl.getInstance()
>>> dd.getEventTotal('physics')
87
This returns the number of events the dispatch-delivery has received in its "physics" dataflow.
The next step is to stop data-taking.
>>> d.signal('StopSig')
>>> d.getSummary()
'CheckStoppedStates'
>>> d.getSummary()
'Ready'
>>> d.signal('IdleSig')
d.getSummary()
>>> 'CheckFinalStates'
>>> d.getSummary()
'Idle'
You have now successfully started and stopped data-taking. It is now time to capture these commands in a script.
We will start our work on scripts by looking at a simple script that just handles data-taking. It is assumed at this point you have worked you way thought the previous examples and so the cluster is in the correct state. If not, you should reset the cluster as described in "Managing the cluster". The first script we are going to run is sample1.py and you will find this in the scripts directory discussed earlier, i.e. ${JYBOSSPATH}. To run this script, which runs data-taking for five minute, you need only execute the follow command.
[patton@spts-expcont demo]$ jyboss ${JYBOSSPATH}/sample1.py
If we now take a look this script, we will see that the core of it is just the commands we ran earlier but with the d.getSummary() statements replaced by d.waitForSummary() statements. The rest of the script is just basic python boilerplate, which is explained below.
import time
import acmectrl
a = acmectrl.getInstance()
import daqctrl
d = daqctrl.getInstance()
def takeData(length):
'Set up and execute a simple data-taking run'
print 'Executing simple data-taking for ' + str(length) + ' seconds.'
d.signal('DiscoverSig')
d.waitForSummary('Idle')
a.startup(acmectrl.DD)
a.connect(acmectrl.DAQ,
acmectrl.DD)
print 'Configuring the DAQ'
d.signal('ConfigSig')
d.waitForSummary('Ready')
d.signal('ConfigDOMsSig')
d.waitForSummary('Ready')
d.signal('StartSig')
d.waitForSummary('Running')
print 'Started taking data'
start = time.time()
while(length > time.time() - start):
time.sleep(60)
print 'Dispatched ' + str(dd.getEventTotal()) + "so far."
print 'Stopping taking data'
d.signal('StopSig')
d.waitForSummary('Ready')
d.signal('IdleSig')
d.waitForSummary('Idle')
print 'DAQ is now idle'
if __name__ == '__main__' :
takeData(300)
The following are the features you should note.
The import time statement loads the time module, which contains the sleep function that is used later in the script to time the run.
The def takeData(length): statement creates a new function, called takeData, that contains the code that does the data-taking (there is a surprise!). The reason to encapsulate the code in a function like this is that it allows it to be called from other scripts, as we shall see in "Using Scripts in Other Scripts".
The if __name__ == '__main__' : statement is a standard python idiom that causes the code in the if block to be executed only when the script is run as the main script. Thus, when the script is given as the argument to the jyboss command as we did above, the takeData(300) statement is executed.
In this section, we will build on the previous script to create a new one that resets the cluster, i.e. shutdown and restarts all of the JBoss servers, before data taking, and the proceeds to take data. The script that resets the cluster can be found in sample2.py and you are encouraged to look through it to see how it works, but we will not consider it in detail here as it does not introduce any new ideas. Instead we will look more closely at sample3.py which uses both the sample1.py and sample2.py files to achieve the aims of this section.
import acmectrl
a = acmectrl.getInstance()
import sample1
import sample2
if __name__ == '__main__' :
locations = a.getLocations()
sample2.resetCluster(locations)
sample1.takeData(300)
As you can see this is a nice short file. You should be familiar with the first couple of statements, it is the next statement where the fun starts.
The import simple1 and import simple2 statements make the contents of the two sample files available for use in this script. You can see how they are used in the simple2.resetCluster(locations) and simple1.takeData(300) statements. Here, the syntax to reference a function defined in an imported module, is module.function and you can now see why we chose to collect the data taking statements into a function in "Creating and Executing a Script".
The other new feature introduced by this example is the getLocations function of the AcmeCtrl class. This function returns a list of locations that are expected to exist based on which cluster the jyboss command is executing.
In case you have not already worked it out you can run this example using the following command.
[patton@spts-expcont demo]$ jyboss ${JYBOSSPATH}/sample3.py
While you can always run the script by preceding it with the jyboss command, it is possible to make the script act as a normal command. The next section will should how.
The conversion of a script into a command is very straight-forward, but before we proceed with that you need to make sure that your shell can find the new command by adding the JYBOSSPATH value to your shell's PATH. You can do this with the following command (again it is recommended that you add this declaration into your .bashrc.)
export PATH=${PATH}:${JYBOSSPATH}
We can now look at ${JYBOSSPATH}/sample4.py which is an command version of sample3.py.
#!/usr/bin/env jyboss
import acmectrl
a = acmectrl.getInstance()
import sample1
import sample2
if __name__ == '__main__' :
locations = a.getLocations()
sample2.resetCluster(locations)
sample1.takeData(300)
As you can see, the only difference is the addition of the #!/usr/bin/env jyboss statement at the beginning of the file. This tells your shell that is a script to be executed using the jyboss command.
There is also one other change, which is not apparent in the listing. The file itself has its execution bit set. You can see this if you execute the following command.
[patton@spts-expcont demo]$ (cd ${JYBOSSPATH}; ls -l sample[34].py)
-rw-r--r-- 1 patton patton 199 Jun 30 12:53 sample3.py
-rwxr-xr-x 1 patton patton 222 Jun 30 12:53 sample4.py
To set the execution bit on a script if it is not already set, you need to run the following command.
[patton@spts-expcont demo]$ chmod +x ${JYBOSSPATH}/sample4.py
This script can now be run using the following command.
[patton@spts-expcont demo]$ sample4.py
You are now able to write you own commands to control the IceCube detector. However, the commands available through the *Ctrl classes are very limited so the final step in this introduction is to show you how the create your own brand new classes to expand access to the IceCube detector.
The *Ctrl classes that you have met so far are facade classes that map high level functionality onto direct MBean calls. This allows the high level code to be uncoupled from the underlying MBeans. This should remain true when you develop access to the MBeans that interest you. High level code should never access MBeans directly, rather a ** class should do the MBean access and the high level code use that class to access the detector. For a full discussion on the design philosophies to use when creating jyboss scripts see the appropriate part of the reference manual.
In this example we shall show the initial development of a AutoTrim class which will facilitate management of the dispatch-delivery auto-trimming service. The code for this class in contained in sample5.py.
import clusterctrl
from clusterctrl import createObjectName
# Maps a stream onto the client used as the trim marker.
CLIENT_MAPPING = {'monitor': 'MonitorSyphon',
'physics': 'PNF',
'sn': 'SNSyphon',
'tcal': 'TCalSyphon'}
AUTOTRIM_NAME_TEMPLATE = 'icecube.control.dd:' + \
'auto-trim=%s,component=dd,controller=auto-trim,' + \
'dd-aspect=%s,dd=controller,stream=%s'
class AutoTrim:
'Manages a dispatch-delivery auto-trim service'
def __init__(self,
stream):
self.configMBean = createObjectName(AUTOTRIM_NAME_TEMPLATE %
(CLIENT_MAPPING[stream], 'configuration', stream) )
self.controlMBean = createObjectName(AUTOTRIM_NAME_TEMPLATE %
(CLIENT_MAPPING[stream], 'control', stream) )
def setDelayDepth(self,
depth):
'Sets new delayDepth value, returning the old one'
ctrl = clusterctrl.getInstance()
result = ctrl.getAttribute(self.configMBean,
'delayDepth')
ctrl.setAttribute(self.configMBean,
'delayDepth',
depth)
return result
def trimNow(self):
'Causes the next pending trim to be executed'
clusterctrl.getInstance().invoke(self.controlMBean,
'trimNow',
None,
None)
As this example is not designed to be a python tutorial we will not go into the python details here, we will only skim those features needed to understand this example. Working our way thought the example we can see the following features.
The import clusterctrl statement loads the IceCube module that manages control of the cluster and, in turn, access to the JMX instances available on the cluster.
The from clusterctrl import createObjectName statement make the createObjectName function available to be used later to construct MBean names.
The CLIENT_MAPPING declaration creates a dictionary which returns the name of the client each stream is using to determine where to trim. This information is used later to construct MBean names.
The AUTOTRIM_NAME_TEMPLATE declaration creates a python string template which is used later in the construction of MBean names.
The class AutoTrim: statement declares the class which will be used by higher level code to manage the auto-trim service for a given stream.
The __init__(self, stream) statement is the constructor function for this class. A python constructor always take the self variable as a parameter (as does any class member function) and this is a reference to the object being created. In this class we also have the stream variable which is a string containing the name of the stream that this object will be managing. Within the constructor all that happens is the creation of two instance variables, configMBean and controlMBean that contain the MBean ObjectName instances for MBean we want ot access. These are built by using the createObjectName from the clusterctrl module and the python string formating feature, which takes the stream name; the CLIENT_MAPPING and the AUTOTRIM_NAME_TEMPLATE and creates a string containing the name for each MBean.
All of this preamble leads up to the definition of the The setDelayDepth function that demonstrates how MBean attributes are accessed. This is done, either for setting or getting, by using the appropriate setAttribute and getAttribute function of the global ClusterCtrl instance, which is returned by the clusterctrl.getInstance call. From this example is should be obvious what are the necessary parameters for each function.
Finally, in this example, the trimNow function definition demonstrates the use of the invoke function of the ClusterCtrl class to execute an MBean operation. This example shows the most basic form of the class. If you need a more complex form, e.g. one that takes parameters, you should review the appropriate section of the reference manual.
This document has provided an introduction to the use of jyboss as a mean for accessing the IceCube detector. You should now be able to write basic scripts and command for the detector. To find out more and to be able to write more complex scripts you should read the reference manual