« Previous - Version 10/33 (diff) - Next » - Current version
Redmine Admin, 08/07/2006 09:57 am

= Python bindings Howto =

Starting on version 0.63.0 DrQueue provides Python bindings so privinding access all functionality of DrQueue to python program.


'''''You will need [http://www.swig.org/ Swig] and [http://www.scons.org/ SCons] to compile besides the rest of dependencies for the main code.'''''

Compiling the python bindings is as easy as:

cd /path/to/drqueue/sources
cd python

After that you will have two very important files '''drqueue.py''' and '''_drqueue.so'''. Those two files are needed to import the '''drqueue python module'''.

Once you have those files you can make use of the whole DrQueue library by importing the module in your python script.
You can do that using the following lines in your python script: {{{
import sys
import drqueue

You have examples of the use of the python bindings in directory ''python/examples'' and ''python/DrKeewee'' of the sources.

To browse those examples online: * [source:trunk/python/examples/] * [source:trunk/python/DrKeewee/cgi-bin/drkeewee.cgi]

Basic usage

=== Object types (data structures for libdrqueue) ===

The two most important object types are the '''job''' and the '''computer'''. They represent a job (either already in the queue or not yet sent) and a computer in the queue, that is a registered slave.

As Python everything is an object ,,(yes, I know... I guess),, , we take advantage of that. The original C library deals with C structures. But thanks to Swig, we can extend those data structures with methods to treat them like objects. As mentioned, they're objects in Python.

==== Job object ====

A job object stores all job related information like the job status, frames left and done, number of active processors... as well as the limits related to that job like the maximum number of cpus, the pool to which this job is assigned, etc. It also has methods to perform actions with it.

We can obtain job instances either requesting the list of jobs to the master or creating a job instance ourselves.

'''Example: Requesting the list of jobs to the master'''


import drqueue
job_list = drqueue.request_job_list(drqueue.CLIENT)
len (job_list)


As you can see the result is a python list. The environment variables DRQUEUE_MASTER and DRQUEUE_MASTER should be set, same way as for any other drqueue command line tool.

'''Example: creating a job instance''' {{{
import drqueue

job = drqueue.job()

Then we can modify it, adjusting values to be sent.

Python 2.4.1 (#2, Jul 21 2005, 15:58:35)
[GCC 4.0.2 20050720 (prerelease) (Debian 4.0.1-2ubuntu3)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

import drqueue
job = drqueue.job()
job.name = "Test python job"
job.frame_start = 1
job.frame_end = 10
job.cmd = "ls -al"
job.priority = 100

1 <----- indicates success


4 <----- the job id number assigned by the master


''The are some exceptions implemented to deal with errors. But do not rely only on them yet, try to check the return codes.''

===== Job Methods =====

(The argument ''who'' will be drqueue.CLIENT)

  • job.request_frame_list(who) : Returns a python list of frame_info object instances
  • job.job_frame_index_to_number (index) : Returns the frame number asociated to a frame_info index
  • job.request_continue (who) : Returns 1 on success
  • job.request_stop (who) : Returns 1 on success
  • job.request_hard_stop (who) : Returns 1 on success
  • job.request_delete (who) : Returns 1 on success
  • job.request_rerun (who) : Returns 1 on success
  • job.send_to_queue () : Returns 1 on success
  • job.update (who) : Will contact the master to update the information about the status of the job. (Not including the frame list). Returns 1 on success.

==== the Computer object ====

The computer object stores all information related to the computer (including limits, pools, tasks...) as well as methods to perform actions on them.

As a difference with jobs, there is no use (atm) in creating a computer instance directly. They're only useful when you receive them from the master with ''drqueue.request_computer_list(drqueue.CLIENT)''

Example: {{{

import drqueue
computer_list = drqueue.request_computer_list(drqueue.CLIENT) <---------- retrieves the list of slaves from the master
len (computer_list) <---------- as a python list


computer = computer_list0 <---------- we get the first instance

'localhost' <---------- it's name


1 <---------- number of cpus


0 <---------- is the slave enabled for rendering ? nopes, it's disabled.

computer.request_enable(drqueue.CLIENT) <---------- let's enable it then.
computer.update(drqueue.CLIENT) <---------- we sent the request but, did it work ? the update method will retrieve the information from the master, so we can check.
computer.limits.enabled <---------- is enabled it now ?

1 <---------- yes, that computer will be active now. :)

In this example we can see how to retrieve some basic information about one of the slaves connected to the master. It's name, number of cpus and enabled status.

With the method request_enable we request the slave to change it's enabled status to 1 (enabled). With method update as well as with the method of the same name in the job, we update the information stored in the object computer from the master.

'''''Note''': When we request the slave to change it's status to enabled, we are doing so directly to the slave, NOT to the master. That means that if we write a program that changes the status of the slaves and right away requests the information to the master, it won't probably be updated. It takes some seconds to the slave to report it's status to the master.''

===== computer methods ===== * computer.request_enable(who) * computer.request_disable(who) * computer.update(who)

=== Other useful info ===

Independently of the object's method that are python specific we have access to all libdrqueue functions. So we can call any function using same type of arguments the C function call would use.

For example: {{{
drlist = drqueue.request_computer_list(drqueue.CLIENT) # retrieves the list from the master. raises exception on error.
drlist = None
print "Could not retrieve list of slaves"

found = 0
for tcomp in drlist:
if tcomp.hwinfo.name == 'slave001': # compare it's name as a string
compid = tcomp.hwinfo.id # get the id as an integer
found = 1

if found: # this function requests the slave named 'slave001' to set it's limit of available cpus to 4
rcode = drqueue.request_slave_limits_nmaxcpus_set ('slave0001',4,drqueue.CLIENT)

In this sample code you can see how a regular library function is called: ''drqueue.request_slave_limits_nmaxcpus_set (<string>,<integer>,drqueue.CLIENT)''

Any other function can be used the same way.

Be careful though with double pointer arguments. Those will probably fail or corrupt your data. I have written code to deal with some, but it's not complete.