SubworkflowsDesign

VisTrails has groups which are "encapsulate by value" entities and subworkflows which are "encapsulate by reference". A group is stored as a workflow with modules and connections (including special InputPort and OutputPort modules that determine which ports are exposed in the group module) while a subworkflow is stored as a version of a workflow in a vistrail. Thus, a change to a group will generate an entirely new workflow that is fully specified while a change to a subworkflow uses change-based provenance to only store the actions between the previous workflow and the new one. Groups are fairly simple and work well to abstract a piece of a workflow but use storage inefficiently. In addition, in their current implementation in VisTrails, they are impossible to edit in place. A user must expand the group, make changes, possibly add back some Input/OutputPort modules, and then group the modules again. Subworkflows solve the storage issue but introduce significant complexity because they must be indexed by the module registry. Subworkflows can exist locally (in a user's .vistrails directory), as part of a distributed package (can use _subworkflows list or add_subworkflow), or as part of a vt bundle. In addition, any reference to a subworkflow in a workflow must both identify the subworkflow and the version to be used.

Group Edits

One goal is to make editing a group easier. If we could open a window like with editing subworkflows, we could take the editing workflow and use it as the new group. Note, however, that this has a couple issues:

1. We must delete any parameters or connections that were attached to ports that were removed (should be able to look at the upgrade code for these routines).
2. Workflow editing current assumes a vistrail exists which may introduce issues. We have the ability to import workflows from xml so we should be able to create a vistrail from the group's workflow, allow the user to make edits and then pull the currentPipeline back as the group's workflow. Ideally, it would be nice to eventually have a separation between workflow and vistrail such that VisTrails could have a standalone workflow editor with provenance threaded in (this would allow us to use the Provenance SDK).

Another possibility that has been proposed is to make grouping something that is more visual than anything else. In other words, I create a "group" from modules a,b,c by visually collapsing these into a single box but all I write to the vistrail is that certain module ids have been marked as collapsed. The major issue with this is that significant functionality has been built assuming that a group is a tangible entity (e.g. in control flow modules).

Subworkflow versions and upgrades

Because we want subworkflows to