Parsing the logs to enrich the state information
This is something that we are currently doing ourselves with our xsibatch jobs. We have a custom script that actively parses xsibatch output to fill a database with precise information on the current frame, current pass, actual render times, the path of the generated images, if an image was rendered or skipped, etc..
It would be great if drqueue could handle all this by itself thus having more state info associated with the jobs.
- The python scripts would have to actively parse every single line of output of the render program (with a pipe) searching for several key log lines (with fast precompiled regular expressions).
- Every line would go to stdout anyway (to have them on the log as always)
- When some key information is found, it is shared with the slave process (through sockets, pipes, shm, whatever)