JobSubmission

Introduction

This page describes and discusses the "job submission" effort in VisTrails, i.e. running job from VisTrails on remote servers, and getting a job's result asynchronously in a later session.

Long-running jobs

VisTrails's supports long-running jobs through the ModuleSuspended mechanism. A Module can suspend itself once a job is running (after submitting it on the first run, or after checking that it's not done yet on subsequent runs) by raising a ModuleSuspended exception, which contains information allowing the JobMonitor to automatically check for the status of the job in the background.

The JobMonitor then notifies the user if a known running job is now completed, so that the user can re-run the workflow, which should now be able to pass the suspending module.

In addition, the JobMonitor serializes this information so that VisTrails knows to check for these jobs if you restart it later on (or on a different machine, if this is written to the vistrail).

This is a very abstract and high-level interface for background jobs that any job-submission mechanism is built on.