DrQueue in cloud computing environments

Cloud computing

This term is somewhat often used these days by marketing people for a lot of stuff. If you want to know more about the basics, have a look here: http://en.wikipedia.org/wiki/Cloud_computing .

It might be interesting to use DrQueue in such environments to provide a commercial renderfarm service to people willing to pay for it. A list of possible components will be described on this page.

Needed features / components

Render nodes (master and slaves)

There are lot of cloud service providers which can give access to virtual machines running different operating systems. One example is EC2 (http://aws.amazon.com/de/ec2) from Amazon.

Shared storage

Storage services can also be used in cloud environments. This makes it possible to have VMs which are able to access the same data. One example is S3 (http://aws.amazon.com/de/s3) from Amazon.

High security

Using cryptography for data storage and data transport between VMs can secure a cloud renderfarm against leakage and manipulation of valueable business data. This can be achived for example through dm-crypt (http://www.saout.de/misc/dm-crypt) and OpenVPN (http://openvpn.net).

Tool for renderfarm communication/control

This would be the role of DrQueue as we know it.

Tool for customers to create/control render jobs

This could be DrQueueOnRails. It provides a web-based access to a DrQueue renderfarm, so there is no need for installing the DrQueue software on the customer's computer. Render jobs are separated on a per-user basis. Every user can only control his own jobs. Authentication is done via LDAP.

Tool for user provisioning

There is no component for this task yet. It would be possible to add provisioning functionality in DrQueueOnRails. Another possibility is to create a new tool, so business stuff would be separated from renderfarm usage. But developing another new tool would also increase complexity and communication overhead between the software components.
As a security feature for the renderfarm provider, customers should be only given render nodes after they paid in advance. This means the renderfarm provider will be always able to pay the cloud environment provider. A customer should have the possibility to decide how many nodes and how much render time he requests. When a payment is done, the administrator of the renderfarm service could create a 'render session' (number of nodes for a certain time span) which can then be consumed by the render jobs of the customer.

Tool for cloud service control

A software for configuring DrQueue slaves and starting/stopping cloud VMs is needed to automate things. It would be very time-consuming to do this manually. This could be done by a daemon which has an overview over all render sessions and supervises all DrQueue slaves. It adds/removes slaves to user pools and starts/stops slave VMs.
DrQueueCloudControl, a daemon written in Ruby is in development. See https://ssl.drqueue.org/gitweb?p=drqueue-cloudcontrol.git;a=summary and http://github.com/kaazoo/DrQueueCloudControl for more information.