Command-line Arguments

Starting VisTrails via the Command Line

VisTrails supports a number of command-line arguments that let you modify certain attributes and behaviors of the program. When invoking VisTrails from the command line, the arguments are placed after the “run.py” filename. For example,

python vistrails/run.py -b

starts VisTrails in batch mode. Table table-batch-cli contains a complete list of the command line switches supported by vistrails. Each command line switch has both a short form and a long form. The two forms are logically equivalent, and which one you use is a matter of personal preference. The short form consists of a single minus sign “-” followed by a single letter. The longer form uses two minus signs “–” followed by a descriptive word. For example, the above command for batch mode could have been written as:

python vistrails/run.py --batch

In addition to the explicit switches listed in Table table-batch-cli, the VisTrails command line also lets you indicate the filename of the vistrail you wish to open. For example, assuming your “examples” directory is one level above your current working directory, this is how you would tell VisTrails to load the “lung.vt” example at startup:

python vistrails/run.py examples/lung.vt

Moreover, if you want VisTrails to start on a specific version of the pipeline within the vistrail, you can indicate that version’s tag name on the command line. The filename and version tag should be separated by a colon. For example, to start VisTrails with the colormap version of the “lung.vt” vistrail, use:

python vistrails/run.py examples/lung.vt:colormap

In the event that the version you want to open contains a space in its tag name, simply surround the entire “filename:tag” pair in double quotes. For example:

python vistrails/run.py "examples/lung.vt:Axial View"

You can also open up multiple vistrails at once by listing more than one vistrail file on the command line. This causes the vistrails to be opened in separate tabs, just as if you had opened them via the GUI. For example:

python vistrails/run.py examples/lung.vt examples/head.vt

You can specify version tags in conjunction with multiple filenames. Here is an example of an elaborate command-line invocation that opens two vistrails and sets each one to a specific version:

python vistrails/run.py "examples/lung.vt:Axial View" examples/head.vt:bone
usage: run.py [-h] [--version] [-S DIR] [--subworkflows-dir DIR]
              [--data-dir DIR] [--package-dir DIR] [--user-package-dir DIR]
              [--file-dir DIR] [--log-dir DIR] [--temporary-dir DIR]
              [--thumbs-auto-save | --no-thumbs-auto-save]
              [--thumbs-mouse-hover | --no-thumbs-mouse-hover]
              [--thumbs-tags-only | --no-thumbs-tags-only]
              [--thumbs-cache-dir DIR] [--thumbs-cache-size THUMBS.CACHESIZE]
              [--host URL] [--port PORT] [--db DB] [--user USER]
              [--rpc-server RPCSERVER] [--rpc-port RPCPORT]
              [--rpc-log-file DIR] [--rpc-instances RPCINSTANCES]
              [--multithread] [--rpc-config DIR] [--web-repository-url URL]
              [--web-repository-user WEBREPOSITORYUSER]
              [--job-check-interval JOBCHECKINTERVAL]
              [--job-autorun JOBAUTORUN] [--job-list] [--job-info] [-E] [-b]
              [-o DIR] [-p OUTPUTSETTINGS] [--parameters PARAMETERS]
              [--parameter-exploration] [--show-window]
              [--output-version-tree] [--output-pipeline-graph]
              [--graphs-as-pdf] [--enable-usage] [--disable-usage]
              [--maximize-windows | --no-maximize-windows]
              [--multi-heads | --no-multi-heads]
              [--show-splash | --hide-splash] [--auto-save | --no-auto-save]
              [--db-default | --no-db-default] [--cache | --no-cache]
              [--stop-on-error | --no-stop-on-error]
              [--execution-log | --no-execution-log]
              [--error-log | --no-error-log]
              [--default-file-type DEFAULTFILETYPE] [-v DEBUGLEVEL]
              [--show-vistrails-news | --hide-vistrails-news]
              [--upgrades | --no-upgrades]
              [--migrate-tags | --no-migrate-tags]
              [--hide-upgrades | --no-hide-upgrades]
              [--upgrade-delay | --no-upgrade-delay]
              [--upgrade-module-fail-prompt | --no-upgrade-module-fail-prompt]
              [--auto-connect | --no-auto-connect]
              [--detach-history-view | --no-detach-history-view]
              [--show-connection-errors | --hide-connection-errors]
              [--show-variant-errors | --hide-variant-errors]
              [--show-debug-popups | --hide-debug-popups]
              [--show-scrollbars | --hide-scrollbars]
              [--show-inline-parameter-widgets | --hide-inline-parameter-widgets]
              [--shell-font-face SHELL.FONTFACE]
              [--shell-font-size SHELL.FONTSIZE]
              [--max-recent-vistrails MAXRECENTVISTRAILS]
              [--view-on-load VIEWONLOAD]
              [--custom-version-colors | --no-custom-version-colors]
              [--fixed-custom-version-color-saturation | --no-fixed-custom-version-color-saturation]
              [--enable-packages-silently | --no-enable-packages-silently]
              [--load-packages | --no-load-packages]
              [--install-bundles | --no-install-bundles]
              [--install-bundles-with-pip | --no-install-bundles-with-pip]
              [--repository-local-path DIR] [--repository-httpurl URL]
              [--single-instance | --no-single-instance]
              [--static-registry DIR]
              [vistrail [vistrail ...]]
Positional arguments:
vistrails Vistrail to open
Options:
--version show program’s version and exit
-S, --dot-vistrails
 The location to look for VisTrails user configurations and storage. Defaults to ~/.vistrails.
--subworkflows-dir
 The location where a user’s local subworkflows are stored.
--data-dir The location that VisTrails uses as a default directory for data.
--package-dir The directory to look for VisTrails core packages (use userPackageDir for user-defined packages).
--user-package-dir
 The location for user-installed packages (defaults to ~/.vistrails/userpackages).
--file-dir The location that VisTrails uses as a default directory for specifying files.
--log-dir The path that indicates where log files should be stored.
--temporary-dir
 The directory to use for temporary files generated by VisTrails.
--thumbs-auto-save
 Whether to save thumbnails of results when executing VisTrails.
--no-thumbs-auto-save
 Inverse of –thumbs-auto-save
--thumbs-mouse-hover
 Whether to show thumbnails when hovering over a version in the version tree.
--no-thumbs-mouse-hover
 Inverse of –thumbs-mouse-hover
--thumbs-tags-only
 If True, only stores thumbnails for tagged versions. Otherwise, stores thumbnails for all versions.
--no-thumbs-tags-only
 Inverse of –thumbs-tags-only
--thumbs-cache-dir
 The directory to be used to cache thumbnails.
--thumbs-cache-size
 The size (in MB) of the thumbnail cache.
--host The hostname for the database to load the vistrail from.
--port The port for the database to load the vistrail from.
--db The name for the database to load the vistrail from.
--user The username for the database to load the vistrail from.
--rpc-server Hostname or ip address where this xml rpc server will work.
--rpc-port Port where this xml rpc server will work.
--rpc-log-file Log file for XML RPC server.
--rpc-instances
 Number of other instances that vistrails should start.
--multithread Server will start a thread for each request.
--rpc-config Config file for server connection options.
--web-repository-url
 The URL of the web repository that should be attached to VisTrails (e.g. www.crowdlabs.org).
--web-repository-user
 The default username for logging into a VisTrails web repository like crowdLabs.
--job-check-interval
 How often to check for jobs (in seconds, default=600).
--job-autorun Run jobs automatically when they finish.
--job-list List running workflows.
--job-info List jobs in running workflow.
-E, --no-execute
 Do not execute specified workflows.
-b, --batch Run vistrails in batch mode instead of interactive mode.
-o, --output-directory
 Directory in which to place output files
-p, --output-settings
 One or more comma-separated key=value parameters.
--parameters List of parameters to use when running workflow.
--parameter-exploration
 Open and execute parameter exploration specified by the version argument after the .vt file.
--show-window Show the main VisTrails window.
--output-version-tree
 Output the version tree as an image.
--output-pipeline-graph
 Output the workflow graph as an image.
--graphs-as-pdf
 Generate graphs in PDF format instead of images
--enable-usage Enable sending anonymous usage statistics
--disable-usage
 Disable sending anonymous usage statistics
--maximize-windows
 Whether the VisTrails windows should take up the entire screen space.
--no-maximize-windows
 Inverse of –maximize-windows
--multi-heads Whether or not to use multiple screens for VisTrails windows.
--no-multi-heads
 Inverse of –multi-heads
--show-splash Whether the VisTrails splash screen should be shown on startup.
--hide-splash Inverse of –show-splash
--auto-save Automatically save vistrails to allow recovery from crashes, etc.
--no-auto-save Inverse of –auto-save
--db-default Use a database as the default storage location for vistrails entities.
--no-db-default
 Inverse of –db-default
--cache Cache previous results so they may be used in future computations.
--no-cache Inverse of –cache
--stop-on-error
 Whether or not VisTrails stops executing the rest of the workflow if it encounters an error in one module.
--no-stop-on-error
 Inverse of –stop-on-error
--execution-log
 Track execution provenance when running workflows.
--no-execution-log
 Inverse of –execution-log
--error-log Write errors to a log file.
--no-error-log Inverse of –error-log
--default-file-type
 Defaults to .vt but could be .xml.
-v, --debug-level
 How much information VisTrails should alert the user about (0: Critical errors only, 1: Critical errors and warnings, 2: Critical errors, warnings, and log messages).
--show-vistrails-news
 Show news from VisTrails on startup. Each message will only be displayed once.
--hide-vistrails-news
 Inverse of –show-vistrails-news
--upgrades Whether to upgrade old workflows so they work with newer packages.
--no-upgrades Inverse of –upgrades
--migrate-tags Whether or not the tag on a workflow that was upgraded should be moved to point to the upgraded version.
--no-migrate-tags
 Inverse of –migrate-tags
--hide-upgrades
 Don’t show the “upgrade” nodes in the version tree.
--no-hide-upgrades
 Inverse of –hide-upgrades
--upgrade-delay
 Persist upgrade only after other changes.
--no-upgrade-delay
 Inverse of –upgrade-delay
--upgrade-module-fail-prompt
 Whether to alert the user when an upgrade may fail when upgrading a subworkflow.
--no-upgrade-module-fail-prompt
 Inverse of –upgrade-module-fail-prompt
--auto-connect Try to automatically connect a newly dragged in module to the rest of the workflow.
--no-auto-connect
 Inverse of –auto-connect
--detach-history-view
 Show the version tree in a separate window.
--no-detach-history-view
 Inverse of –detach-history-view
--show-connection-errors
 Alert the user if the value along a connection doesn’t match connection types.
--hide-connection-errors
 Inverse of –show-connection-errors
--show-variant-errors
 Alert the user if the value along a connection coming from a Variant output doesn’t match the input port.
--hide-variant-errors
 Inverse of –show-variant-errors
--show-debug-popups
 Always show the debug popups or only if there is a modal widget.
--hide-debug-popups
 Inverse of –show-debug-popups
--show-scrollbars
 Whether VisTrails should show scrollbars on the version tree and workflow canvases.
--hide-scrollbars
 Inverse of –show-scrollbars
--show-inline-parameter-widgets
 Show editable parameters inside modules.
--hide-inline-parameter-widgets
 Inverse of –show-inline-parameter-widgets
--shell-font-face
 The font to be used for the VisTrails console.
--shell-font-size
 The font size used for the VisTrails console.
--max-recent-vistrails
 How many recently opened vistrails should be stored for “Open Recent” access.
--view-on-load Whether to show pipeline or history view when opening vistrail. Can be either appropriate/pipeline/history.
--custom-version-colors
 Allow setting custom colors for versions, and display these colors in the version tree.
--no-custom-version-colors
 Inverse of –custom-version-colors
--fixed-custom-version-color-saturation
 Don’t change the saturation according to the age of the version if it has a custom color.
--no-fixed-custom-version-color-saturation
 Inverse of –fixed-custom-version-color-saturation
--enable-packages-silently
 Do not prompt the user to enable packages, just do so automatically.
--no-enable-packages-silently
 Inverse of –enable-packages-silently
--load-packages
 Whether to load the packages enabled in the configuration file.
--no-load-packages
 Inverse of –load-packages
--install-bundles
 Automatically try to install missing Python dependencies.
--no-install-bundles
 Inverse of –install-bundles
--install-bundles-with-pip
 Whether to try to use pip to install Python dependencies or use distribution support.
--no-install-bundles-with-pip
 Inverse of –install-bundles-with-pip
--repository-local-path
 Path used to locate packages available to be installed.
--repository-httpurl
 URL used to locate packages available to be installed.
--single-instance
 Whether or not VisTrails should only allow one instance to be running.
--no-single-instance
 Inverse of –single-instance
--static-registry
 If specified, VisTrails uses an XML file defining the VisTrails module registry to load modules instead of from the packages directly.

Specifying a User Configuration Directory

In addition to the default .vistrails directory, VisTrails allows you to create and use additional configuration directories. First, you will need to create a new directory. This is done by running: python vistrails/run.py -S /path_to_new_directory/new_directory_name.

This will both create a new directory containing default configuration files and directories, and launch VisTrails, which will use the newly created files for configuration. The user is then free to add desired configurations to the new directory. Once a configuration directory exists, subsequent calls using the directory name (python vistrails/run.py -S /path_to_directory/existing_directory) will launch VisTrails using the ‘existing_directory’ for configuration and a new directory will not be created.

Note: If you would like to copy configuration directories, you must change the references in copy_of_directory/startup.xml to point to the new directory instead of the original.

Passing Database Parameters on the Command Line

As discussed in Chapter Connecting to a Database, VisTrails can read and write vistrails stored in a relational database as well as in a filesystem. VisTrails allows you to specify the name of the database server, the database name, the port number, and the username on the command line. This potentially saves you the trouble of filling out the same information on the database connection dialog. Note that, for security reasons, VisTrails does not allow you to include a database password on the command line; you must still type your password into the database connection dialog when VisTrails opens.

The last four rows of Table table-batch-cli show the command-line switches that pertain to database connectivity. Be advised that these switches were designed primarily for use by VTL files (see Section Using “Vistrail Link” Files) and as such, are not necessarily user-friendly. In particular, these switches are ignored unless you also specify the vistrail ID and version name on the command line. For example, to open the contour version of the the “spx” vistrail (whose ID is 5) from the database “vistrails” residing on the host “vistrails.sci.utah.edu” with a username of “vistrails”:

python vistrails/run.py --host=vistrails.sci.utah.edu --db=vistrails --user=vistrails 5:contour

Once VisTrails opens, you will be prompted to enter the password. Upon successful authentication, the vistrail is loaded from the database and opened to the pipeline corresponding to the specified version.

Running VisTrails in Batch Mode

Although VisTrails is primarily intended to be run as an interactive, graphical client application, it also supports non-interactive use. VisTrails can thus be invoked programmatically, eg as part of a shell script. You can tell VisTrails to start in non-interactive mode by using the “-b” or “–batch” command line switch when launching vistrails. [1]

Running VisTrails in non-interactive mode has little effect, however, without an additional command line argument indicating which vistrail to load. Since we are running VisTrails as part of a batch process, it only makes sense to execute vistrails whose output is something tangible, such as a file. A vistrail whose only output is an interactive rendering in a VTKCell, for instance, would not be well-suited for running in batch mode.

Consider the following example. The “offscreen.vt” vistrail (included in the “examples” directory) has a variety of output options, depending on which version you select in the History view (Figure The different versions of the offscreen.vt vistrail...). The version tagged only vtk displays its output as an interactive VTK rendering. The version tagged html creates a simple web page in the Spreadsheet. The offscreen version, however, outputs an image file named “image.png”. Since its output (a file) can be easily accessed outside of VisTrails, this version is an ideal candidate for running in batch mode. To try it, invoke VisTrails as shown, specifying both the name of the vistrail file and the desired version:

python vistrails/run.py -b examples/offscreen.vt:offscreen
_images/offscreen_version_tree.png

The different versions of the “offscreen.vt” vistrail offer various forms of output.

As you would expect, this command runs to completion without opening any windows. Instead, it silently loads the requested pipeline, executes it, and closes. Assuming it ran correctly, this pipeline should have created a file named “image.png” in the current directory. When you view this file, it should resemble the picture in Figure Running the offscreen version of offscreeen.vt in batch mode....

_images/offscreen_output.png

Running the offscreen version of “offscreen.vt” in batch mode produces an image named “image.png”.

Running a Specific Workflow in Batch Mode

To run a specific workflow in batch mode, call VisTrails with the following options:

python vistrails/run.py -b path_to_vistrails_file:pipeline

where pipeline can be a version tag name or version id.

Note

If you downloaded the MacOS X bundle, you can run VisTrails from the command line via the following commands in the terminal. Change the current directory to wherever VisTrails was installed (often /Applications), and then type: Vistrails.app/Contents/MacOS/vistrails [<cmd_line_options>]

Running a Workflow with Specific Parameters

An alias is a name assigned to a parameter that allows you to reference that parameter in batch mode. An alias is created by clicking on the type of an existing parameter in VisTrails, then entering a name for it.

_images/create_alias.png

Example of creating an alias

Users can change workflow parameters that have an alias through the command line.

For example, offscreen pipeline in offscreen.vt always creates the file called image.png. If you want generate it with a different filename:

python vistrails/run.py -b examples/offscreen.vt:offscreen --parameters="filename=other.png"

filename in the example above is the alias name assigned to the parameter in the value method inside the String module. When running a pipeline from the command line, VisTrails will try to start the spreadsheet automatically if the pipeline requires it. For example, this other execution will also start the spreadsheet (attention to how $ characters are escaped when running on bash):

python vistrails/run.py -b examples/head.vt:aliases --parameters="isovalue=30\$&\$diffuse_color=0.8, 0.4, 0.2"

You can also execute more than one pipeline on the command line:

python vistrails/run.py -b examples/head.vt:aliases ../examples/spx.vt:spx \ --parameters="isovalue=30"

Use –parameters only once regardless the number of pipelines.

Accessing a Database in Batch Mode

As discussed in Section Passing Database Parameters on the Command Line, you can specify most of the parameters of your database connection on the command line, but the password must be entered through the GUI. This poses a problem for running VisTrails in non-interactive mode, since no database connection dialog will be opened. If your batch process needs to access vistrails stored on a database, the current workaround is to create a special account on the database (probably one with read-only access) that does not require a password, and use this account for connecting to the database in batch mode.

Using VisTrails as a Server

Using the VisTrails server mode, it is possible to execute workflows and control VisTrails through another application. For example, the CrowdLabs Web portal (http://www.crowdlabs.org) accesses a VisTrails sever to execute workflows, retrieve and display vistrail trees and workflows.

The way you access the server is by doing XML-RPC calls. In the current VisTrails release, we include a set of PHP scripts that can talk to a VisTrails server instance. They are in “extensions/http” folder. The files are reasonably well documented. Also, it should be not difficult to create python scripts to access the server (just use xmlrpclib module).

Note that the VisTrails server requires the provenance and workflows to be in a database. More detailed instructions on how to setup the server and the database are available in VisTrails Server Setup and in Setting up the database.

If what you want is just to execute a series of workflows in batch mode, a simpler solution would be to use the VisTrails client in batch mode (see Section Running VisTrails in Batch Mode).

Executing Workflows in Parallel

The VisTrails server can only execute pipelines in parallel if there’s more than one instance of VisTrails running. The command

self.rpcserver=ThreadedXMLRPCServer((self.temp_xml_rpc_options.server, self.temp_xml_rpc_options.port))

starts a multithreaded version of the XML-RPC server, so it will create a thread for each request received by the server. The problem is that Qt/PyQT doesn’t allow these multiple threads to create GUI objects. Only the main thread can. To overcome this limitation, the multithreaded version can instantiate other single threaded versions of VisTrails and put them in a queue, so workflow executions and other GUI-related requests, such as generating workflow graphs and history trees can be forwarded to this queue, and each instance takes turns in answering the request. If the results are in the cache, the multithreaded version answers the requests directly.

Note that this infrastructure works on Linux only. To make this work on Windows, you have to create a script similar to start_vistrails_xvfb.sh (located in the scripts folder) where you can send the number of other instances via command-line options to VisTrails. The command line options are:

python vistrails_server.py --host=<ADDRESS> --port=<PORT> -O<NUMBER_OF_OTHER_VISTRAILS_INSTANCES> [-M]&

If you want the main vistrails instance to be multithreaded, use the -M at the end.

After creating this script, update function start_other_instances in vistrails/gui/application_server.py lines 1007-1023 and set the script variable to point to your script. You may also have to change the arguments sent to your script (line 1016: for example, you don’t need to set a virtual display). You will need to change the path to the stop_vistrails_server.py script (on line 1026) according to your installation path.

Executing Parameter Explorations from the Command Line

Named parameter explorations can be executed from the command line in different ways using the -P flag. The parameter after the vistrail will then indicate the parameter exploration name in place of the workflow version. To open vistrails and execute a parameter exploration named “final” in terminator.vt run:

python vistrails/run.py -P terminator.vt:final

To only show the spreadsheet run:

python vistrails/run.py -P -i terminator.vt:final

To execute the spreadsheet in batch mode, and to output the spreadsheet as images to a directory, use the -b flag and specify a directory with the -e flag:

python vistrails/run.py -P -b -e ./final_images terminator.vt:final

This will create an image for each cell and also create a composite image for each sheet in the spreadsheet.

Finding Methods Via the Command Line

We have tried to make some methods more accessible in the console via an api. You can import the api via import api in the console and see the available methods with dir(api). To open a vistrail:

import api
api.open_vistrail_from_file('/Applications/VisTrails/examples/terminator.vt')

To execute a version of a workflow, you currently have to go through the controller:

api.select_version('Histogram')
api.get_current_controller().execute_current_workflow()

Currently, only a subset of VisTrails functionality is directly available from the api. However, since VisTrails is written in python, you can dig down starting with the VistrailsApplication or controller object to expose most of our internal methods. If you have suggestions for calls to be added to the api, please let us know.

One other feature that we’re working on, but is still in progress is the ability to construct workflows via the console. For example:

vtk = load_package('org.vistrails.vistrails.vtk')
vtk.vtkDataSetReader() # adds a vtkDataSetReader module to the pipeline
# click on the new module
a = selected_modules()[0] # get the one currently selected module
a.SetFile('/vistrails/examples/data/head120.vtk') # sets the SetFile\
                 parameter for the data set reader
b = vtk.vtkContourFilter() # adds a vtkContourFilter module to the\
                 pipeline and saves to var b
b.SetInputConnection0(a.GetOutputPort0()) # connects a's GetOutputPort0\
                 port to b's SetInputConnection0

Footnotes

[1]The parameter “-b” stands for “batch.” In this chapter, we use the terms “batch mode” and “non-interactive mode” synonymously.