VistrailsWiki - User contributions [en]

Version 2

2011-05-26T01:09:45Z

Ctuttle: /* Claurissa's Notes V2.0 */

This page is setup to address and track issues with the new VisTrails 2.0 construction. Please add comments or new issues to this page.

=== Juliana's notes about new interface ===

* It seems there are too many windows when we start VisTrails. What about only showing the main window? And fire the spreadsheet and inspector only when 'needed'? for the inspector, when a vt is open and a wf is selected, and for the spreadsheet when a workflow is run.
** [DK] What do you mean by windows? The tabbed windows or the fact that there is a builder window, "utility" window, and a spreadsheet window?
** [JF] The latter: 3 windows popup when I start VisTrais: the builder, the inspector and the spreadsheet. 
** [DK] Many of the windows in the "utility" window are not dependent on a workflow (for example, the console and messages tabs). We could however, not display them until a user asks to see the given view.
** [JF] This is what I meant.

* I noticed that the picture-in-picture disappeared. I think that is helpful to give context. At least when we are displaying a workflow, it is useful to see the version tree.
** [DK] I think it was too small to be useful. I can see the argument for being able to see both, but I think we need more real estate than the PIP had; perhaps we allow the History View to be shown as a tab on the left or as a detached view?
** [JF] the only issue here is whether this will be too ''busy''
** [DK] It is, arguably, the same amount of business as the PIP, however.
** [CT] I often used the picture-in-picture to select a different version without having to change to the history view and then change back. I miss it. If it is detachable, the user can just put it where they want it, which makes this a question of where to put it by default and whether or not to allow it to be where it used to be (since detachable objects can not generally be placed there).
** [DK] maybe the correct approach here is to allow views to be detatched?
** [CT] no. they should always have the center. perhaps a detachable PIP? Meaning, it might be nice to have the history view by the pipeline view, but when you click the history button on top, you still want the history view to show up in the center panel. I have some ideas, I'll share a few. I'll still refer to the additional view as the PIP even if it is no longer a picture-in-picture.
*** leave the PIP where it was and make it easy to turn on/off (ie a shortcut key, button in the toolbar, or menu item). optionally allow users to resize it.
*** Have a "combined" view button in the toolbar that switches to a view that has the PIP. Then have a simple shortcut key to swap views (switch the larger with the smaller).
*** let the PIP come up in a new window, users can move or resize it. it should be easy to toggle on/off (ie pressing the spacebar, etc.). the issue here is having too many windows. So, we could think of it as an extension to the existing window, meaning it should always stay on top of the main window (but it doesn't need to overlap it, it could be to the side). For example, selecting the spreadsheet window would put it (the spreadsheet) on top. Then, selecting the main window would give the main window the focus, but the PIP would still be on top of it. Also, when I tab through all windows in VisTrails, the PIP would not get it's own tab, it would be considered part of the main window. Then when people get tired of it, they can turn it off.
*** We could do a pseudo detach with the view, meaning the views stay where they are, but when you go to detach it, it comes up in its separate window. Users are then free to put it where they want it. I think we would want this detached view to show the history view when the pipeline view is selected, or the pipeline view when the history view is selected.

* We need some indication of which workflow is being displayed. Somewhere, we should display the label that identifies the workflow
** [DK] In the v2.0 branch, the tab shows this. I have it display "Pipeline: alias" or "Pipeline: ROOT + 3". [JF] Ok.
** [ES] Dave, it doesn't show anymore. I remember seeing the tab before, but I've just checked out v2.0 and I can't see it either.
** [DK] Right, when there is only a single tab, I hide the tab bar. You can see the titles when if you create a new view (Ctrl/Cmd+T)

* When I click on the workflow tab (in the main window,lower left corner) it always shows an "untitled.vt". This looks like a bug... And if I click on untitled.vt, nothing happens.
** [DK] Yes, Emanuele mentioned this at the meeting. I'm also not happy with the current interface.

=== Dave's List ===

* Reduce size of palette names so they don't take so much space

* Set default palettes for diff, other views

* Diff currently being shown as a pipeline. Should we create a mode for this

* Buttons in Messages window look bad on the Mac, don't need all of them now as well, I think

* Missing callbacks for Merge and Control Flow Assistant menu items

* Move execute button so it doesn't look like a mode

* Aliases support for parameters.
** [DK] I really want to see old-style aliases finally go away with the globals for workflow and vistrails taking their place, but we can add them for now.
** [ES] I just added them in v2.0 branch.

* How do we support functions that have no parameters in the new port configuration?

* Do we need to worry about parameters with default values that won't be saved? For example, a Boolean that we want set to False will not show as a set parameter currently because that is the default. Same for an empty string...
** [DK] mimic 1.6 and create the function immediately?

=== Claurissa's Notes V2.0===

* It is unclear to me how to get group and subworkflow options enabled. If I start VisTrails, open a file, and then try to group some of its modules, the options remain disabled. However, if I create a pipeline on the untitled workspace, group and subworkflow are enabled, and then I can open the other file and the options remain enabled.

* In the Module Configuration window, I can click on the check boxes, but it makes me save after each one and doesn't update the builder window. Configuring ports to be visible works on the Module Information tab.

* Parameter Exploration doesn't work. When I hit the execute button, nothing seems to happen.

* Search doesn't work either. If I double click execute, I need to force quit Vistrails. If I press it only once, nothing seems to happen. If I type something into the box rather than creating a visual query, it does display a tree, but nothing is highlighted.

* When you detach and close a tab from the Configure/Console/Documentation/Latex/Messages/etc. window, there is no way to get it back without restarting VisTrails.

* Is there a way to collapse all lists in the modules tab? That would be nice to have.

* The save file option is disabled even after I make a change. I can only select save as, or save on exit (on either the v2.0 or mashup branches). An asterisk does appear next to the filename in the window title on the mashups branch, but not the v2.0 branch.

* Changing between files using the workspace tab doesn't work on the 2.0 branch (but does on mashups).

* The Provenance browser doesn't contain newly executed workflows unless I exit VisTrails and restart (it doesn't update). Perhaps that is because I can't save the file???

* Are the objects that appear in the bottom half of the workspace tab associated with execution provenance? It is not clear to me what they are for. It seems a new one appears after I save a new file and stays there permanently. Clicking on them doesn't seem to do much. Are they just for information?

=== Juliana's notes about the mashup functionality ===
* Since it is not possible to create aliases with the current 2.0, I used 1.6 to create aliases for brain.vt. But when I loaded brain.vt into the 2.0 version and selected the mashup inspector, no aliases were shown [JF] My bad: it works!

* When I save a mashup, where is it saved? Should it be displayed under the "List" tab in the mashup inspector?
** [ES] The mashups are saved when you save the vistrail. When you tag a mashup it is added to the list.

* <del>I changed a tag for a mashup, and clicked "save". But the new tag was not displayed in the "List".</del>
** [ES] There was a problem with the signals that were not being caught. You don't have to save. Just changing a tag should put it in the list.

* <del>In the "List", will I be able to select a previously created mashup and change it?</del>
** [ES] Yes, this is working in the current mashups branch

* What is the difference between the "Save" and the "Update"?
** [ES] In the mashup preview tab, pressing "Update" will run the pipeline with the values filled in the gui. It doesn't change anything in the mashup. I am still not sure what to do with the "save" button. I just wanted a button where the user could tell that he wants that mashup to be in the list, but for that he can just tag the mashup on inspector. What do you think?
** [JF] From the interface, it is not clear that by just changing the tag, the mashup will be saved. Maybe, when the user hits save, if a tag has not been entered (i.e., the default tag is being used), we could prompt the user to enter a tag name? Also, while editing the mashup using the inspector, it might be better to have the name of the workflow on the top, don't you think? (and the mashup tag right below).

* [JF] Should we have a special widget for entering file names?
* [JF] How do I view the actual mashup interface? And how to I execute the mashup on my laptop?

* I made some changes to the "Alias Details" and then clicked on "Save" and then "Preview", nothing happened. After I clicked on these two buttons a few times, the workflow executed, but the actual mashup preview was not displayed.
** [ES] I am working on fixing this now.

* <del>It would be nice if it were possible to center and resize the workflow to fit in the window (cmd-shift-R)</del>
** [ES] This is supported in the current mashups branch (I haven't built a new binary yet)

===More Mashups Comments===

Version 2

2011-05-25T17:40:20Z

Ctuttle: /* Claurissa's Notes V2.0 */

This page is setup to address and track issues with the new VisTrails 2.0 construction. Please add comments or new issues to this page.

=== Juliana's notes about new interface ===

* It seems there are too many windows when we start VisTrails. What about only showing the main window? And fire the spreadsheet and inspector only when 'needed'? for the inspector, when a vt is open and a wf is selected, and for the spreadsheet when a workflow is run.
** [DK] What do you mean by windows? The tabbed windows or the fact that there is a builder window, "utility" window, and a spreadsheet window?
** [JF] The latter: 3 windows popup when I start VisTrais: the builder, the inspector and the spreadsheet. 
** [DK] Many of the windows in the "utility" window are not dependent on a workflow (for example, the console and messages tabs). We could however, not display them until a user asks to see the given view.
** [JF] This is what I meant.

* I noticed that the picture-in-picture disappeared. I think that is helpful to give context. At least when we are displaying a workflow, it is useful to see the version tree.
** [DK] I think it was too small to be useful. I can see the argument for being able to see both, but I think we need more real estate than the PIP had; perhaps we allow the History View to be shown as a tab on the left or as a detached view?
** [JF] the only issue here is whether this will be too ''busy''
** [DK] It is, arguably, the same amount of business as the PIP, however.
** [CT] I often used the picture-in-picture to select a different version without having to change to the history view and then change back. I miss it. If it is detachable, the user can just put it where they want it, which makes this a question of where to put it by default and whether or not to allow it to be where it used to be (since detachable objects can not generally be placed there).
** [DK] maybe the correct approach here is to allow views to be detatched?
** [CT] no. they should always have the center. perhaps a detachable PIP? Meaning, it might be nice to have the history view by the pipeline view, but when you click the history button on top, you still want the history view to show up in the center panel. I have some ideas, I'll share a few. I'll still refer to the additional view as the PIP even if it is no longer a picture-in-picture.
*** leave the PIP where it was and make it easy to turn on/off (ie a shortcut key, button in the toolbar, or menu item). optionally allow users to resize it.
*** Have a "combined" view button in the toolbar that switches to a view that has the PIP. Then have a simple shortcut key to swap views (switch the larger with the smaller).
*** let the PIP come up in a new window, users can move or resize it. it should be easy to toggle on/off (ie pressing the spacebar, etc.). the issue here is having too many windows. So, we could think of it as an extension to the existing window, meaning it should always stay on top of the main window (but it doesn't need to overlap it, it could be to the side). For example, selecting the spreadsheet window would put it (the spreadsheet) on top. Then, selecting the main window would give the main window the focus, but the PIP would still be on top of it. Also, when I tab through all windows in VisTrails, the PIP would not get it's own tab, it would be considered part of the main window. Then when people get tired of it, they can turn it off.
*** We could do a pseudo detach with the view, meaning the views stay where they are, but when you go to detach it, it comes up in its separate window. Users are then free to put it where they want it. I think we would want this detached view to show the history view when the pipeline view is selected, or the pipeline view when the history view is selected.

* We need some indication of which workflow is being displayed. Somewhere, we should display the label that identifies the workflow
** [DK] In the v2.0 branch, the tab shows this. I have it display "Pipeline: alias" or "Pipeline: ROOT + 3". [JF] Ok.
** [ES] Dave, it doesn't show anymore. I remember seeing the tab before, but I've just checked out v2.0 and I can't see it either.
** [DK] Right, when there is only a single tab, I hide the tab bar. You can see the titles when if you create a new view (Ctrl/Cmd+T)

* When I click on the workflow tab (in the main window,lower left corner) it always shows an "untitled.vt". This looks like a bug... And if I click on untitled.vt, nothing happens.
** [DK] Yes, Emanuele mentioned this at the meeting. I'm also not happy with the current interface.

=== Dave's List ===

* Reduce size of palette names so they don't take so much space

* Set default palettes for diff, other views

* Diff currently being shown as a pipeline. Should we create a mode for this

* Buttons in Messages window look bad on the Mac, don't need all of them now as well, I think

* Missing callbacks for Merge and Control Flow Assistant menu items

* Move execute button so it doesn't look like a mode

* Aliases support for parameters.
** [DK] I really want to see old-style aliases finally go away with the globals for workflow and vistrails taking their place, but we can add them for now.
** [ES] I just added them in v2.0 branch.

* How do we support functions that have no parameters in the new port configuration?

* Do we need to worry about parameters with default values that won't be saved? For example, a Boolean that we want set to False will not show as a set parameter currently because that is the default. Same for an empty string...
** [DK] mimic 1.6 and create the function immediately?

=== Claurissa's Notes V2.0===

* In the Module Configuration window, I can click on the check boxes, but it makes me save after each one and doesn't update the builder window. Configuring ports to be visible works on the Module Information tab.

* Parameter Exploration doesn't work. When I hit the execute button, nothing seems to happen.

* Search doesn't work either. If I double click execute, I need to force quit Vistrails. If I press it only once, nothing seems to happen. If I type something into the box rather than creating a visual query, it does display a tree, but nothing is highlighted.

* When you detach and close a tab from the Configure/Console/Documentation/Latex/Messages/etc. window, there is no way to get it back without restarting VisTrails.

* Is there a way to collapse all lists in the modules tab? That would be nice to have.

* The save file option is disabled even after I make a change. I can only select save as, or save on exit (on either the v2.0 or mashup branches). An asterisk does appear next to the filename in the window title on the mashups branch, but not the v2.0 branch.

* Changing between files using the workspace tab doesn't work on the 2.0 branch (but does on mashups).

* The Provenance browser doesn't contain newly executed workflows unless I exit VisTrails and restart (it doesn't update). Perhaps that is because I can't save the file???

* Are the objects that appear in the bottom half of the workspace tab associated with execution provenance? It is not clear to me what they are for. It seems a new one appears after I save a new file and stays there permanently. Clicking on them doesn't seem to do much. Are they just for information?

=== Juliana's notes about the mashup functionality ===
* Since it is not possible to create aliases with the current 2.0, I used 1.6 to create aliases for brain.vt. But when I loaded brain.vt into the 2.0 version and selected the mashup inspector, no aliases were shown [JF] My bad: it works!

* When I save a mashup, where is it saved? Should it be displayed under the "List" tab in the mashup inspector?
** [ES] The mashups are saved when you save the vistrail. When you tag a mashup it is added to the list.

* <del>I changed a tag for a mashup, and clicked "save". But the new tag was not displayed in the "List".</del>
** [ES] There was a problem with the signals that were not being caught. You don't have to save. Just changing a tag should put it in the list.

* <del>In the "List", will I be able to select a previously created mashup and change it?</del>
** [ES] Yes, this is working in the current mashups branch

* What is the difference between the "Save" and the "Update"?
** [ES] In the mashup preview tab, pressing "Update" will run the pipeline with the values filled in the gui. It doesn't change anything in the mashup. I am still not sure what to do with the "save" button. I just wanted a button where the user could tell that he wants that mashup to be in the list, but for that he can just tag the mashup on inspector. What do you think?
** [JF] From the interface, it is not clear that by just changing the tag, the mashup will be saved. Maybe, when the user hits save, if a tag has not been entered (i.e., the default tag is being used), we could prompt the user to enter a tag name? Also, while editing the mashup using the inspector, it might be better to have the name of the workflow on the top, don't you think? (and the mashup tag right below).

* [JF] Should we have a special widget for entering file names?
* [JF] How do I view the actual mashup interface? And how to I execute the mashup on my laptop?

* I made some changes to the "Alias Details" and then clicked on "Save" and then "Preview", nothing happened. After I clicked on these two buttons a few times, the workflow executed, but the actual mashup preview was not displayed.
** [ES] I am working on fixing this now.

* <del>It would be nice if it were possible to center and resize the workflow to fit in the window (cmd-shift-R)</del>
** [ES] This is supported in the current mashups branch (I haven't built a new binary yet)

===More Mashups Comments===

Version 2

2011-05-25T03:29:55Z

Ctuttle: /* More Mashups Comments */

This page is setup to address and track issues with the new VisTrails 2.0 construction. Please add comments or new issues to this page.

=== Juliana's notes about new interface ===

* It seems there are too many windows when we start VisTrails. What about only showing the main window? And fire the spreadsheet and inspector only when 'needed'? for the inspector, when a vt is open and a wf is selected, and for the spreadsheet when a workflow is run.
** [DK] What do you mean by windows? The tabbed windows or the fact that there is a builder window, "utility" window, and a spreadsheet window?
** [JF] The latter: 3 windows popup when I start VisTrais: the builder, the inspector and the spreadsheet. 
** [DK] Many of the windows in the "utility" window are not dependent on a workflow (for example, the console and messages tabs). We could however, not display them until a user asks to see the given view.
** [JF] This is what I meant.

* I noticed that the picture-in-picture disappeared. I think that is helpful to give context. At least when we are displaying a workflow, it is useful to see the version tree.
** [DK] I think it was too small to be useful. I can see the argument for being able to see both, but I think we need more real estate than the PIP had; perhaps we allow the History View to be shown as a tab on the left or as a detached view?
** [JF] the only issue here is whether this will be too ''busy''
** [DK] It is, arguably, the same amount of business as the PIP, however.
** [CT] I often used the picture-in-picture to select a different version without having to change to the history view and then change back. I miss it. If it is detachable, the user can just put it where they want it, which makes this a question of where to put it by default and whether or not to allow it to be where it used to be (since detachable objects can not generally be placed there).
** [DK] maybe the correct approach here is to allow views to be detatched?
** [CT] no. they should always have the center. perhaps a detachable PIP? Meaning, it might be nice to have the history view by the pipeline view, but when you click the history button on top, you still want the history view to show up in the center panel. I have some ideas, I'll share a few. I'll still refer to the additional view as the PIP even if it is no longer a picture-in-picture.
*** leave the PIP where it was and make it easy to turn on/off (ie a shortcut key, button in the toolbar, or menu item). optionally allow users to resize it.
*** Have a "combined" view button in the toolbar that switches to a view that has the PIP. Then have a simple shortcut key to swap views (switch the larger with the smaller).
*** let the PIP come up in a new window, users can move or resize it. it should be easy to toggle on/off (ie pressing the spacebar, etc.). the issue here is having too many windows. So, we could think of it as an extension to the existing window, meaning it should always stay on top of the main window (but it doesn't need to overlap it, it could be to the side). For example, selecting the spreadsheet window would put it (the spreadsheet) on top. Then, selecting the main window would give the main window the focus, but the PIP would still be on top of it. Also, when I tab through all windows in VisTrails, the PIP would not get it's own tab, it would be considered part of the main window. Then when people get tired of it, they can turn it off.
*** We could do a pseudo detach with the view, meaning the views stay where they are, but when you go to detach it, it comes up in its separate window. Users are then free to put it where they want it. I think we would want this detached view to show the history view when the pipeline view is selected, or the pipeline view when the history view is selected.

* We need some indication of which workflow is being displayed. Somewhere, we should display the label that identifies the workflow
** [DK] In the v2.0 branch, the tab shows this. I have it display "Pipeline: alias" or "Pipeline: ROOT + 3". [JF] Ok.
** [ES] Dave, it doesn't show anymore. I remember seeing the tab before, but I've just checked out v2.0 and I can't see it either.
** [DK] Right, when there is only a single tab, I hide the tab bar. You can see the titles when if you create a new view (Ctrl/Cmd+T)

* When I click on the workflow tab (in the main window,lower left corner) it always shows an "untitled.vt". This looks like a bug... And if I click on untitled.vt, nothing happens.
** [DK] Yes, Emanuele mentioned this at the meeting. I'm also not happy with the current interface.

=== Dave's List ===

* Reduce size of palette names so they don't take so much space

* Set default palettes for diff, other views

* Diff currently being shown as a pipeline. Should we create a mode for this

* Buttons in Messages window look bad on the Mac, don't need all of them now as well, I think

* Missing callbacks for Merge and Control Flow Assistant menu items

* Move execute button so it doesn't look like a mode

* Aliases support for parameters.
** [DK] I really want to see old-style aliases finally go away with the globals for workflow and vistrails taking their place, but we can add them for now.
** [ES] I just added them in v2.0 branch.

* How do we support functions that have no parameters in the new port configuration?

* Do we need to worry about parameters with default values that won't be saved? For example, a Boolean that we want set to False will not show as a set parameter currently because that is the default. Same for an empty string...
** [DK] mimic 1.6 and create the function immediately?

=== Claurissa's Notes V2.0===

* Parameter Exploration doesn't work. When I hit the execute button, nothing seems to happen.

* Search doesn't work either. If I double click execute, I need to force quit Vistrails. If I press it only once, nothing seems to happen. If I type something into the box rather than creating a visual query, it does display a tree, but nothing is highlighted.

* When you detach and close a tab from the Configure/Console/Documentation/Latex/Messages/etc. window, there is no way to get it back without restarting VisTrails.

* Is there a way to collapse all lists in the modules tab? That would be nice to have.

* The save file option is disabled even after I make a change. I can only select save as, or save on exit (on either the v2.0 or mashup branches). An asterisk does appear next to the filename in the window title on the mashups branch, but not the v2.0 branch.

* Changing between files using the workspace tab doesn't work on the 2.0 branch (but does on mashups).

* The Provenance browser doesn't contain newly executed workflows unless I exit VisTrails and restart (it doesn't update). Perhaps that is because I can't save the file???

* Are the objects that appear in the bottom half of the workspace tab associated with execution provenance? It is not clear to me what they are for. It seems a new one appears after I save a new file and stays there permanently. Clicking on them doesn't seem to do much. Are they just for information?

=== Juliana's notes about the mashup functionality ===
* Since it is not possible to create aliases with the current 2.0, I used 1.6 to create aliases for brain.vt. But when I loaded brain.vt into the 2.0 version and selected the mashup inspector, no aliases were shown [JF] My bad: it works!

* When I save a mashup, where is it saved? Should it be displayed under the "List" tab in the mashup inspector?
** [ES] The mashups are saved when you save the vistrail. When you tag a mashup it is added to the list.

* <del>I changed a tag for a mashup, and clicked "save". But the new tag was not displayed in the "List".</del>
** [ES] There was a problem with the signals that were not being caught. You don't have to save. Just changing a tag should put it in the list.

* <del>In the "List", will I be able to select a previously created mashup and change it?</del>
** [ES] Yes, this is working in the current mashups branch

* What is the difference between the "Save" and the "Update"?
** [ES] In the mashup preview tab, pressing "Update" will run the pipeline with the values filled in the gui. It doesn't change anything in the mashup. I am still not sure what to do with the "save" button. I just wanted a button where the user could tell that he wants that mashup to be in the list, but for that he can just tag the mashup on inspector. What do you think?
** [JF] From the interface, it is not clear that by just changing the tag, the mashup will be saved. Maybe, when the user hits save, if a tag has not been entered (i.e., the default tag is being used), we could prompt the user to enter a tag name? Also, while editing the mashup using the inspector, it might be better to have the name of the workflow on the top, don't you think? (and the mashup tag right below).

* [JF] Should we have a special widget for entering file names?
* [JF] How do I view the actual mashup interface? And how to I execute the mashup on my laptop?

* I made some changes to the "Alias Details" and then clicked on "Save" and then "Preview", nothing happened. After I clicked on these two buttons a few times, the workflow executed, but the actual mashup preview was not displayed.
** [ES] I am working on fixing this now.

* <del>It would be nice if it were possible to center and resize the workflow to fit in the window (cmd-shift-R)</del>
** [ES] This is supported in the current mashups branch (I haven't built a new binary yet)

===More Mashups Comments===

Version 2

2011-05-25T03:27:34Z

Ctuttle:

This page is setup to address and track issues with the new VisTrails 2.0 construction. Please add comments or new issues to this page.

=== Juliana's notes about new interface ===

* It seems there are too many windows when we start VisTrails. What about only showing the main window? And fire the spreadsheet and inspector only when 'needed'? for the inspector, when a vt is open and a wf is selected, and for the spreadsheet when a workflow is run.
** [DK] What do you mean by windows? The tabbed windows or the fact that there is a builder window, "utility" window, and a spreadsheet window?
** [JF] The latter: 3 windows popup when I start VisTrais: the builder, the inspector and the spreadsheet. 
** [DK] Many of the windows in the "utility" window are not dependent on a workflow (for example, the console and messages tabs). We could however, not display them until a user asks to see the given view.
** [JF] This is what I meant.

* I noticed that the picture-in-picture disappeared. I think that is helpful to give context. At least when we are displaying a workflow, it is useful to see the version tree.
** [DK] I think it was too small to be useful. I can see the argument for being able to see both, but I think we need more real estate than the PIP had; perhaps we allow the History View to be shown as a tab on the left or as a detached view?
** [JF] the only issue here is whether this will be too ''busy''
** [DK] It is, arguably, the same amount of business as the PIP, however.
** [CT] I often used the picture-in-picture to select a different version without having to change to the history view and then change back. I miss it. If it is detachable, the user can just put it where they want it, which makes this a question of where to put it by default and whether or not to allow it to be where it used to be (since detachable objects can not generally be placed there).
** [DK] maybe the correct approach here is to allow views to be detatched?
** [CT] no. they should always have the center. perhaps a detachable PIP? Meaning, it might be nice to have the history view by the pipeline view, but when you click the history button on top, you still want the history view to show up in the center panel. I have some ideas, I'll share a few. I'll still refer to the additional view as the PIP even if it is no longer a picture-in-picture.
*** leave the PIP where it was and make it easy to turn on/off (ie a shortcut key, button in the toolbar, or menu item). optionally allow users to resize it.
*** Have a "combined" view button in the toolbar that switches to a view that has the PIP. Then have a simple shortcut key to swap views (switch the larger with the smaller).
*** let the PIP come up in a new window, users can move or resize it. it should be easy to toggle on/off (ie pressing the spacebar, etc.). the issue here is having too many windows. So, we could think of it as an extension to the existing window, meaning it should always stay on top of the main window (but it doesn't need to overlap it, it could be to the side). For example, selecting the spreadsheet window would put it (the spreadsheet) on top. Then, selecting the main window would give the main window the focus, but the PIP would still be on top of it. Also, when I tab through all windows in VisTrails, the PIP would not get it's own tab, it would be considered part of the main window. Then when people get tired of it, they can turn it off.
*** We could do a pseudo detach with the view, meaning the views stay where they are, but when you go to detach it, it comes up in its separate window. Users are then free to put it where they want it. I think we would want this detached view to show the history view when the pipeline view is selected, or the pipeline view when the history view is selected.

* We need some indication of which workflow is being displayed. Somewhere, we should display the label that identifies the workflow
** [DK] In the v2.0 branch, the tab shows this. I have it display "Pipeline: alias" or "Pipeline: ROOT + 3". [JF] Ok.
** [ES] Dave, it doesn't show anymore. I remember seeing the tab before, but I've just checked out v2.0 and I can't see it either.
** [DK] Right, when there is only a single tab, I hide the tab bar. You can see the titles when if you create a new view (Ctrl/Cmd+T)

* When I click on the workflow tab (in the main window,lower left corner) it always shows an "untitled.vt". This looks like a bug... And if I click on untitled.vt, nothing happens.
** [DK] Yes, Emanuele mentioned this at the meeting. I'm also not happy with the current interface.

=== Dave's List ===

* Reduce size of palette names so they don't take so much space

* Set default palettes for diff, other views

* Diff currently being shown as a pipeline. Should we create a mode for this

* Buttons in Messages window look bad on the Mac, don't need all of them now as well, I think

* Missing callbacks for Merge and Control Flow Assistant menu items

* Move execute button so it doesn't look like a mode

* Aliases support for parameters.
** [DK] I really want to see old-style aliases finally go away with the globals for workflow and vistrails taking their place, but we can add them for now.
** [ES] I just added them in v2.0 branch.

* How do we support functions that have no parameters in the new port configuration?

* Do we need to worry about parameters with default values that won't be saved? For example, a Boolean that we want set to False will not show as a set parameter currently because that is the default. Same for an empty string...
** [DK] mimic 1.6 and create the function immediately?

=== Claurissa's Notes V2.0===

* Parameter Exploration doesn't work. When I hit the execute button, nothing seems to happen.

* Search doesn't work either. If I double click execute, I need to force quit Vistrails. If I press it only once, nothing seems to happen. If I type something into the box rather than creating a visual query, it does display a tree, but nothing is highlighted.

* When you detach and close a tab from the Configure/Console/Documentation/Latex/Messages/etc. window, there is no way to get it back without restarting VisTrails.

* Is there a way to collapse all lists in the modules tab? That would be nice to have.

* The save file option is disabled even after I make a change. I can only select save as, or save on exit (on either the v2.0 or mashup branches). An asterisk does appear next to the filename in the window title on the mashups branch, but not the v2.0 branch.

* Changing between files using the workspace tab doesn't work on the 2.0 branch (but does on mashups).

* The Provenance browser doesn't contain newly executed workflows unless I exit VisTrails and restart (it doesn't update). Perhaps that is because I can't save the file???

* Are the objects that appear in the bottom half of the workspace tab associated with execution provenance? It is not clear to me what they are for. It seems a new one appears after I save a new file and stays there permanently. Clicking on them doesn't seem to do much. Are they just for information?

=== Juliana's notes about the mashup functionality ===
* Since it is not possible to create aliases with the current 2.0, I used 1.6 to create aliases for brain.vt. But when I loaded brain.vt into the 2.0 version and selected the mashup inspector, no aliases were shown [JF] My bad: it works!

* When I save a mashup, where is it saved? Should it be displayed under the "List" tab in the mashup inspector?
** [ES] The mashups are saved when you save the vistrail. When you tag a mashup it is added to the list.

* <del>I changed a tag for a mashup, and clicked "save". But the new tag was not displayed in the "List".</del>
** [ES] There was a problem with the signals that were not being caught. You don't have to save. Just changing a tag should put it in the list.

* <del>In the "List", will I be able to select a previously created mashup and change it?</del>
** [ES] Yes, this is working in the current mashups branch

* What is the difference between the "Save" and the "Update"?
** [ES] In the mashup preview tab, pressing "Update" will run the pipeline with the values filled in the gui. It doesn't change anything in the mashup. I am still not sure what to do with the "save" button. I just wanted a button where the user could tell that he wants that mashup to be in the list, but for that he can just tag the mashup on inspector. What do you think?
** [JF] From the interface, it is not clear that by just changing the tag, the mashup will be saved. Maybe, when the user hits save, if a tag has not been entered (i.e., the default tag is being used), we could prompt the user to enter a tag name? Also, while editing the mashup using the inspector, it might be better to have the name of the workflow on the top, don't you think? (and the mashup tag right below).

* [JF] Should we have a special widget for entering file names?
* [JF] How do I view the actual mashup interface? And how to I execute the mashup on my laptop?

* I made some changes to the "Alias Details" and then clicked on "Save" and then "Preview", nothing happened. After I clicked on these two buttons a few times, the workflow executed, but the actual mashup preview was not displayed.
** [ES] I am working on fixing this now.

* <del>It would be nice if it were possible to center and resize the workflow to fit in the window (cmd-shift-R)</del>
** [ES] This is supported in the current mashups branch (I haven't built a new binary yet)

===More Mashups Comments===
* [CT] When I open the terminator.vt example and select mashups and preview, the center panel is made really wide to fit in all four images that it want to display. The problem then becomes that I can't shrink the window back down to a normal size.

Version 2

2011-05-25T00:11:33Z

Ctuttle: /* Juliana's notes about new interface */

This page is setup to address and track issues with the new VisTrails 2.0 construction. Please add comments or new issues to this page.

=== Juliana's notes about new interface ===

* It seems there are too many windows when we start VisTrails. What about only showing the main window? And fire the spreadsheet and inspector only when 'needed'? for the inspector, when a vt is open and a wf is selected, and for the spreadsheet when a workflow is run.
** [DK] What do you mean by windows? The tabbed windows or the fact that there is a builder window, "utility" window, and a spreadsheet window?
** [JF] The latter: 3 windows popup when I start VisTrais: the builder, the inspector and the spreadsheet. 
** [DK] Many of the windows in the "utility" window are not dependent on a workflow (for example, the console and messages tabs). We could however, not display them until a user asks to see the given view.
** [JF] This is what I meant.

* I noticed that the picture-in-picture disappeared. I think that is helpful to give context. At least when we are displaying a workflow, it is useful to see the version tree.
** [DK] I think it was too small to be useful. I can see the argument for being able to see both, but I think we need more real estate than the PIP had; perhaps we allow the History View to be shown as a tab on the left or as a detached view?
** [JF] the only issue here is whether this will be too ''busy''
** [DK] It is, arguably, the same amount of business as the PIP, however.
** [CT] I often used the picture-in-picture to select a different version without having to change to the history view and then change back. I miss it. If it is detachable, the user can just put it where they want it, which makes this a question of where to put it by default and whether or not to allow it to be where it used to be (since detachable objects can not generally be placed there).
** [DK] maybe the correct approach here is to allow views to be detatched?
** [CT] no. they should always have the center. perhaps a detachable PIP? Meaning, it might be nice to have the history view by the pipeline view, but when you click the history button on top, you still want the history view to show up in the center panel. I have some ideas, I'll share a few. I'll still refer to the additional view as the PIP even if it is no longer a picture-in-picture.
*** leave the PIP where it was and make it easy to turn on/off (ie a shortcut key, button in the toolbar, or menu item). optionally allow users to resize it.
*** Have a "combined" view button in the toolbar that switches to a view that has the PIP. Then have a simple shortcut key to swap views (switch the larger with the smaller).
*** let the PIP come up in a new window, users can move or resize it. it should be easy to toggle on/off (ie pressing the spacebar, etc.). the issue here is having too many windows. So, we could think of it as an extension to the existing window, meaning it should always stay on top of the main window (but it doesn't need to overlap it, it could be to the side). For example, selecting the spreadsheet window would put it (the spreadsheet) on top. Then, selecting the main window would give the main window the focus, but the PIP would still be on top of it. Also, when I tab through all windows in VisTrails, the PIP would not get it's own tab, it would be considered part of the main window. Then when people get tired of it, they can turn it off.
*** We could do a pseudo detach with the view, meaning the views stay where they are, but when you go to detach it, it comes up in its separate window. Users are then free to put it where they want it. I think we would want this detached view to show the history view when the pipeline view is selected, or the pipeline view when the history view is selected.

* We need some indication of which workflow is being displayed. Somewhere, we should display the label that identifies the workflow
** [DK] In the v2.0 branch, the tab shows this. I have it display "Pipeline: alias" or "Pipeline: ROOT + 3". [JF] Ok.
** [ES] Dave, it doesn't show anymore. I remember seeing the tab before, but I've just checked out v2.0 and I can't see it either.
** [DK] Right, when there is only a single tab, I hide the tab bar. You can see the titles when if you create a new view (Ctrl/Cmd+T)

* When I click on the workflow tab (in the main window,lower left corner) it always shows an "untitled.vt". This looks like a bug... And if I click on untitled.vt, nothing happens.
** [DK] Yes, Emanuele mentioned this at the meeting. I'm also not happy with the current interface.

=== Dave's List ===

* Reduce size of palette names so they don't take so much space

* Set default palettes for diff, other views

* Diff currently being shown as a pipeline. Should we create a mode for this

* Buttons in Messages window look bad on the Mac, don't need all of them now as well, I think

* Missing callbacks for Merge and Control Flow Assistant menu items

* Move execute button so it doesn't look like a mode

* Aliases support for parameters.
** [DK] I really want to see old-style aliases finally go away with the globals for workflow and vistrails taking their place, but we can add them for now.
** [ES] I just added them in v2.0 branch.

* How do we support functions that have no parameters in the new port configuration?

* Do we need to worry about parameters with default values that won't be saved? For example, a Boolean that we want set to False will not show as a set parameter currently because that is the default. Same for an empty string...
** [DK] mimic 1.6 and create the function immediately?

=== Claurissa's Notes V2.0===

* Parameter Exploration doesn't work. When I hit the execute button, nothing seems to happen.

* Search doesn't work either. If I double click execute, I need to force quit Vistrails. If I press it only once, nothing seems to happen. If I type something into the box rather than creating a visual query, it does display a tree, but nothing is highlighted.

* When you detach and close a tab from the Configure/Console/Documentation/Latex/Messages/etc. window, there is no way to get it back without restarting VisTrails.

* Is there a way to collapse all lists in the modules tab? That would be nice to have.

* The save file option is disabled even after I make a change. I can only select save as, or save on exit (on either the v2.0 or mashup branches). An asterisk does appear next to the filename in the window title on the mashups branch, but not the v2.0 branch.

* Changing between files using the workspace tab doesn't work on the 2.0 branch (but does on mashups).

* The Provenance browser doesn't contain newly executed workflows unless I exit VisTrails and restart (it doesn't update). Perhaps that is because I can't save the file???

* Are the objects that appear in the bottom half of the workspace tab associated with execution provenance? It is not clear to me what they are for. It seems a new one appears after I save a new file and stays there permanently. Clicking on them doesn't seem to do much. Are they just for information?

=== Juliana's notes about the mashup functionality ===
* Since it is not possible to create aliases with the current 2.0, I used 1.6 to create aliases for brain.vt. But when I loaded brain.vt into the 2.0 version and selected the mashup inspector, no aliases were shown [JF] My bad: it works!

* When I save a mashup, where is it saved? Should it be displayed under the "List" tab in the mashup inspector?
** [ES] The mashups are saved when you save the vistrail. When you tag a mashup it is added to the list.

* <del>I changed a tag for a mashup, and clicked "save". But the new tag was not displayed in the "List".</del>
** [ES] There was a problem with the signals that were not being caught. You don't have to save. Just changing a tag should put it in the list.

* <del>In the "List", will I be able to select a previously created mashup and change it?</del>
** [ES] Yes, this is working in the current mashups branch

* What is the difference between the "Save" and the "Update"?
** [ES] In the mashup preview tab, pressing "Update" will run the pipeline with the values filled in the gui. It doesn't change anything in the mashup. I am still not sure what to do with the "save" button. I just wanted a button where the user could tell that he wants that mashup to be in the list, but for that he can just tag the mashup on inspector. What do you think?
** [JF] From the interface, it is not clear that by just changing the tag, the mashup will be saved. Maybe, when the user hits save, if a tag has not been entered (i.e., the default tag is being used), we could prompt the user to enter a tag name? Also, while editing the mashup using the inspector, it might be better to have the name of the workflow on the top, don't you think? (and the mashup tag right below).

* [JF] Should we have a special widget for entering file names?
* [JF] How do I view the actual mashup interface? And how to I execute the mashup on my laptop?

* I made some changes to the "Alias Details" and then clicked on "Save" and then "Preview", nothing happened. After I clicked on these two buttons a few times, the workflow executed, but the actual mashup preview was not displayed.
** [ES] I am working on fixing this now.

* <del>It would be nice if it were possible to center and resize the workflow to fit in the window (cmd-shift-R)</del>
** [ES] This is supported in the current mashups branch (I haven't built a new binary yet)

Version 2

2011-05-23T23:45:08Z

Ctuttle: /* Claurissa's Notes V2.0 */

This page is setup to address and track issues with the new VisTrails 2.0 construction. Please add comments or new issues to this page.

=== Juliana's notes about new interface ===

* It seems there are too many windows when we start VisTrails. What about only showing the main window? And fire the spreadsheet and inspector only when 'needed'? for the inspector, when a vt is open and a wf is selected, and for the spreadsheet when a workflow is run.
** [DK] What do you mean by windows? The tabbed windows or the fact that there is a builder window, "utility" window, and a spreadsheet window?
** [JF] The latter: 3 windows popup when I start VisTrais: the builder, the inspector and the spreadsheet. 
** [DK] Many of the windows in the "utility" window are not dependent on a workflow (for example, the console and messages tabs). We could however, not display them until a user asks to see the given view.
** [JF] This is what I meant.

* I noticed that the picture-in-picture disappeared. I think that is helpful to give context. At least when we are displaying a workflow, it is useful to see the version tree.
** [DK] I think it was too small to be useful. I can see the argument for being able to see both, but I think we need more real estate than the PIP had; perhaps we allow the History View to be shown as a tab on the left or as a detached view?
** [JF] the only issue here is whether this will be too ''busy''
** [DK] It is, arguably, the same amount of business as the PIP, however.
** [CT] I often used the picture-in-picture to select a different version without having to change to the history view and then change back. I miss it. If it is detachable, the user can just put it where they want it, which makes this a question of where to put it by default and whether or not to allow it to be where it used to be (since detachable objects can not generally be placed there).
* We need some indication of which workflow is being displayed. Somewhere, we should display the label that identifies the workflow
** [DK] In the v2.0 branch, the tab shows this. I have it display "Pipeline: alias" or "Pipeline: ROOT + 3". [JF] Ok.
** [ES] Dave, it doesn't show anymore. I remember seeing the tab before, but I've just checked out v2.0 and I can't see it either.
** [DK] Right, when there is only a single tab, I hide the tab bar. You can see the titles when if you create a new view (Ctrl/Cmd+T)

* When I click on the workflow tab (in the main window,lower left corner) it always shows an "untitled.vt". This looks like a bug... And if I click on untitled.vt, nothing happens.
** [DK] Yes, Emanuele mentioned this at the meeting. I'm also not happy with the current interface.

=== Dave's List ===

* Reduce size of palette names so they don't take so much space

* Set default palettes for diff, other views

* Diff currently being shown as a pipeline. Should we create a mode for this

* Buttons in Messages window look bad on the Mac, don't need all of them now as well, I think

* Missing callbacks for Merge and Control Flow Assistant menu items

* Move execute button so it doesn't look like a mode

* Aliases support for parameters.
** [DK] I really want to see old-style aliases finally go away with the globals for workflow and vistrails taking their place, but we can add them for now.
** [ES] I just added them in v2.0 branch.

* How do we support functions that have no parameters in the new port configuration?

* Do we need to worry about parameters with default values that won't be saved? For example, a Boolean that we want set to False will not show as a set parameter currently because that is the default. Same for an empty string...

=== Claurissa's Notes V2.0===

* Parameter Exploration doesn't work. When I hit the execute button, nothing seems to happen.

* Search doesn't work either. If I double click execute, I need to force quit Vistrails. If I press it only once, nothing seems to happen. If I type something into the box rather than creating a visual query, it does display a tree, but nothing is highlighted.

* When you detach and close a tab from the Configure/Console/Documentation/Latex/Messages/etc. window, there is no way to get it back without restarting VisTrails.

* Is there a way to collapse all lists in the modules tab? That would be nice to have.

* The save file option is disabled even after I make a change. I can only select save as, or save on exit (on either the v2.0 or mashup branches). An asterisk does appear next to the filename in the window title on the mashups branch, but not the v2.0 branch.

* Changing between files using the workspace tab doesn't work on the 2.0 branch (but does on mashups).

* The Provenance browser doesn't contain newly executed workflows unless I exit VisTrails and restart (it doesn't update). Perhaps that is because I can't save the file???

* Are the objects that appear in the bottom half of the workspace tab associated with execution provenance? It is not clear to me what they are for. It seems a new one appears after I save a new file and stays there permanently. Clicking on them doesn't seem to do much. Are they just for information?

=== Juliana's notes about the mashup functionality ===
* Since it is not possible to create aliases with the current 2.0, I used 1.6 to create aliases for brain.vt. But when I loaded brain.vt into the 2.0 version and selected the mashup inspector, no aliases were shown [JF] My bad: it works!

* When I save a mashup, where is it saved? Should it be displayed under the "List" tab in the mashup inspector?
** [ES] The mashups are saved when you save the vistrail. When you tag a mashup it is added to the list.

* <del>I changed a tag for a mashup, and clicked "save". But the new tag was not displayed in the "List".</del>
** [ES] There was a problem with the signals that were not being caught. You don't have to save. Just changing a tag should put it in the list.

* <del>In the "List", will I be able to select a previously created mashup and change it?</del>
** [ES] Yes, this is working in the current mashups branch

* What is the difference between the "Save" and the "Update"?
** [ES] In the mashup preview tab, pressing "Update" will run the pipeline with the values filled in the gui. It doesn't change anything in the mashup. I am still not sure what to do with the "save" button. I just wanted a button where the user could tell that he wants that mashup to be in the list, but for that he can just tag the mashup on inspector. What do you think?
** [JF] From the interface, it is not clear that by just changing the tag, the mashup will be saved. Maybe, when the user hits save, if a tag has not been entered (i.e., the default tag is being used), we could prompt the user to enter a tag name? Also, while editing the mashup using the inspector, it might be better to have the name of the workflow on the top, don't you think? (and the mashup tag right below).

* [JF] Should we have a special widget for entering file names?
* [JF] How do I view the actual mashup interface? And how to I execute the mashup on my laptop?

* I made some changes to the "Alias Details" and then clicked on "Save" and then "Preview", nothing happened. After I clicked on these two buttons a few times, the workflow executed, but the actual mashup preview was not displayed.
** [ES] I am working on fixing this now.

* <del>It would be nice if it were possible to center and resize the workflow to fit in the window (cmd-shift-R)</del>
** [ES] This is supported in the current mashups branch (I haven't built a new binary yet)

Version 2

2011-05-23T17:21:18Z

Ctuttle: /* Juliana's notes about new interface */

This page is setup to address and track issues with the new VisTrails 2.0 construction. Please add comments or new issues to this page.

=== Juliana's notes about new interface ===

* It seems there are too many windows when we start VisTrails. What about only showing the main window? And fire the spreadsheet and inspector only when 'needed'? for the inspector, when a vt is open and a wf is selected, and for the spreadsheet when a workflow is run.
** [DK] What do you mean by windows? The tabbed windows or the fact that there is a builder window, "utility" window, and a spreadsheet window?
** [JF] The latter: 3 windows popup when I start VisTrais: the builder, the inspector and the spreadsheet. 
** [DK] Many of the windows in the "utility" window are not dependent on a workflow (for example, the console and messages tabs). We could however, not display them until a user asks to see the given view.
** [JF] This is what I meant.

* I noticed that the picture-in-picture disappeared. I think that is helpful to give context. At least when we are displaying a workflow, it is useful to see the version tree.
** [DK] I think it was too small to be useful. I can see the argument for being able to see both, but I think we need more real estate than the PIP had; perhaps we allow the History View to be shown as a tab on the left or as a detached view?
** [JF] the only issue here is whether this will be too ''busy''
** [DK] It is, arguably, the same amount of business as the PIP, however.
** [CT] I often used the picture-in-picture to select a different version without having to change to the history view and then change back. I miss it. If it is detachable, the user can just put it where they want it, which makes this a question of where to put it by default and whether or not to allow it to be where it used to be (since detachable objects can not generally be placed there).
* We need some indication of which workflow is being displayed. Somewhere, we should display the label that identifies the workflow
** [DK] In the v2.0 branch, the tab shows this. I have it display "Pipeline: alias" or "Pipeline: ROOT + 3". [JF] Ok.
** [ES] Dave, it doesn't show anymore. I remember seeing the tab before, but I've just checked out v2.0 and I can't see it either.
** [DK] Right, when there is only a single tab, I hide the tab bar. You can see the titles when if you create a new view (Ctrl/Cmd+T)

* When I click on the workflow tab (in the main window,lower left corner) it always shows an "untitled.vt". This looks like a bug... And if I click on untitled.vt, nothing happens.
** [DK] Yes, Emanuele mentioned this at the meeting. I'm also not happy with the current interface.

=== Dave's List ===

* Reduce size of palette names so they don't take so much space

* Set default palettes for diff, other views

* Diff currently being shown as a pipeline. Should we create a mode for this

* Buttons in Messages window look bad on the Mac, don't need all of them now as well, I think

* Missing callbacks for Merge and Control Flow Assistant menu items

* Move execute button so it doesn't look like a mode

* Aliases support for parameters.
** [DK] I really want to see old-style aliases finally go away with the globals for workflow and vistrails taking their place, but we can add them for now.
** [ES] I just added them in v2.0 branch.

* How do we support functions that have no parameters in the new port configuration?

* Do we need to worry about parameters with default values that won't be saved? For example, a Boolean that we want set to False will not show as a set parameter currently because that is the default. Same for an empty string...

=== Claurissa's Notes V2.0===

* When you detach and close a tab from the Configure/Console/Documentation/Latex/Messages/etc. window, there is no way to get it back without restarting VisTrails.

* Is there a way to collapse all lists in the modules tab? That would be nice to have.

* The save file option is disabled even after I make a change. I can only select save as, or save on exit (on either the v2.0 or mashup branches). An asterisk does appear next to the filename in the window title on the mashups branch, but not the v2.0 branch.

* Changing between files using the workspace tab doesn't work on the 2.0 branch (but does on mashups).

* The Provenance browser doesn't contain newly executed workflows unless I exit VisTrails and restart (it doesn't update). Perhaps that is because I can't save the file???

* Are the objects that appear in the bottom half of the workspace tab associated with execution provenance? It is not clear to me what they are for. It seems a new one appears after I save a new file and stays there permanently. Clicking on them doesn't seem to do much. Are they just for information?

=== Juliana's notes about the mashup functionality ===
* Since it is not possible to create aliases with the current 2.0, I used 1.6 to create aliases for brain.vt. But when I loaded brain.vt into the 2.0 version and selected the mashup inspector, no aliases were shown [JF] My bad: it works!

* When I save a mashup, where is it saved? Should it be displayed under the "List" tab in the mashup inspector?
** [ES] The mashups are saved when you save the vistrail. When you tag a mashup it is added to the list.

* <del>I changed a tag for a mashup, and clicked "save". But the new tag was not displayed in the "List".</del>
** [ES] There was a problem with the signals that were not being caught. You don't have to save. Just changing a tag should put it in the list.

* <del>In the "List", will I be able to select a previously created mashup and change it?</del>
** [ES] Yes, this is working in the current mashups branch

* What is the difference between the "Save" and the "Update"?
** [ES] In the mashup preview tab, pressing "Update" will run the pipeline with the values filled in the gui. It doesn't change anything in the mashup. I am still not sure what to do with the "save" button. I just wanted a button where the user could tell that he wants that mashup to be in the list, but for that he can just tag the mashup on inspector. What do you think?
** [JF] From the interface, it is not clear that by just changing the tag, the mashup will be saved. Maybe, when the user hits save, if a tag has not been entered (i.e., the default tag is being used), we could prompt the user to enter a tag name? Also, while editing the mashup using the inspector, it might be better to have the name of the workflow on the top, don't you think? (and the mashup tag right below).

* [JF] Should we have a special widget for entering file names?
* [JF] How do I view the actual mashup interface? And how to I execute the mashup on my laptop?

* I made some changes to the "Alias Details" and then clicked on "Save" and then "Preview", nothing happened. After I clicked on these two buttons a few times, the workflow executed, but the actual mashup preview was not displayed.
** [ES] I am working on fixing this now.

* <del>It would be nice if it were possible to center and resize the workflow to fit in the window (cmd-shift-R)</del>
** [ES] This is supported in the current mashups branch (I haven't built a new binary yet)

Version 2

2011-05-23T17:20:30Z

Ctuttle: /* Juliana's notes about new interface */

This page is setup to address and track issues with the new VisTrails 2.0 construction. Please add comments or new issues to this page.

=== Juliana's notes about new interface ===

* It seems there are too many windows when we start VisTrails. What about only showing the main window? And fire the spreadsheet and inspector only when 'needed'? for the inspector, when a vt is open and a wf is selected, and for the spreadsheet when a workflow is run.
** [DK] What do you mean by windows? The tabbed windows or the fact that there is a builder window, "utility" window, and a spreadsheet window?
** [JF] The latter: 3 windows popup when I start VisTrais: the builder, the inspector and the spreadsheet. 
** [DK] Many of the windows in the "utility" window are not dependent on a workflow (for example, the console and messages tabs). We could however, not display them until a user asks to see the given view.
** [JF] This is what I meant.

* I noticed that the picture-in-picture disappeared. I think that is helpful to give context. At least when we are displaying a workflow, it is useful to see the version tree.
** [DK] I think it was too small to be useful. I can see the argument for being able to see both, but I think we need more real estate than the PIP had; perhaps we allow the History View to be shown as a tab on the left or as a detached view?
** [JF] the only issue here is whether this will be too ''busy''
** [DK] It is, arguably, the same amount of business as the PIP, however.
** [CT] I often used the picture-in-picture to select a different version without having to change to the history view and then change back. I do miss it. If it is detachable, the user can just put it where they want it, which makes this a question of where to put it by default and whether or not to allow it to be where it used to be (since detachable objects can not generally be placed there).
* We need some indication of which workflow is being displayed. Somewhere, we should display the label that identifies the workflow
** [DK] In the v2.0 branch, the tab shows this. I have it display "Pipeline: alias" or "Pipeline: ROOT + 3". [JF] Ok.
** [ES] Dave, it doesn't show anymore. I remember seeing the tab before, but I've just checked out v2.0 and I can't see it either.
** [DK] Right, when there is only a single tab, I hide the tab bar. You can see the titles when if you create a new view (Ctrl/Cmd+T)

* When I click on the workflow tab (in the main window,lower left corner) it always shows an "untitled.vt". This looks like a bug... And if I click on untitled.vt, nothing happens.
** [DK] Yes, Emanuele mentioned this at the meeting. I'm also not happy with the current interface.

=== Dave's List ===

* Reduce size of palette names so they don't take so much space

* Set default palettes for diff, other views

* Diff currently being shown as a pipeline. Should we create a mode for this

* Buttons in Messages window look bad on the Mac, don't need all of them now as well, I think

* Missing callbacks for Merge and Control Flow Assistant menu items

* Move execute button so it doesn't look like a mode

* Aliases support for parameters.
** [DK] I really want to see old-style aliases finally go away with the globals for workflow and vistrails taking their place, but we can add them for now.
** [ES] I just added them in v2.0 branch.

* How do we support functions that have no parameters in the new port configuration?

* Do we need to worry about parameters with default values that won't be saved? For example, a Boolean that we want set to False will not show as a set parameter currently because that is the default. Same for an empty string...

=== Claurissa's Notes V2.0===

* When you detach and close a tab from the Configure/Console/Documentation/Latex/Messages/etc. window, there is no way to get it back without restarting VisTrails.

* Is there a way to collapse all lists in the modules tab? That would be nice to have.

* The save file option is disabled even after I make a change. I can only select save as, or save on exit (on either the v2.0 or mashup branches). An asterisk does appear next to the filename in the window title on the mashups branch, but not the v2.0 branch.

* Changing between files using the workspace tab doesn't work on the 2.0 branch (but does on mashups).

* The Provenance browser doesn't contain newly executed workflows unless I exit VisTrails and restart (it doesn't update). Perhaps that is because I can't save the file???

* Are the objects that appear in the bottom half of the workspace tab associated with execution provenance? It is not clear to me what they are for. It seems a new one appears after I save a new file and stays there permanently. Clicking on them doesn't seem to do much. Are they just for information?

=== Juliana's notes about the mashup functionality ===
* Since it is not possible to create aliases with the current 2.0, I used 1.6 to create aliases for brain.vt. But when I loaded brain.vt into the 2.0 version and selected the mashup inspector, no aliases were shown [JF] My bad: it works!

* When I save a mashup, where is it saved? Should it be displayed under the "List" tab in the mashup inspector?
** [ES] The mashups are saved when you save the vistrail. When you tag a mashup it is added to the list.

* <del>I changed a tag for a mashup, and clicked "save". But the new tag was not displayed in the "List".</del>
** [ES] There was a problem with the signals that were not being caught. You don't have to save. Just changing a tag should put it in the list.

* <del>In the "List", will I be able to select a previously created mashup and change it?</del>
** [ES] Yes, this is working in the current mashups branch

* What is the difference between the "Save" and the "Update"?
** [ES] In the mashup preview tab, pressing "Update" will run the pipeline with the values filled in the gui. It doesn't change anything in the mashup. I am still not sure what to do with the "save" button. I just wanted a button where the user could tell that he wants that mashup to be in the list, but for that he can just tag the mashup on inspector. What do you think?
** [JF] From the interface, it is not clear that by just changing the tag, the mashup will be saved. Maybe, when the user hits save, if a tag has not been entered (i.e., the default tag is being used), we could prompt the user to enter a tag name? Also, while editing the mashup using the inspector, it might be better to have the name of the workflow on the top, don't you think? (and the mashup tag right below).

* [JF] Should we have a special widget for entering file names?
* [JF] How do I view the actual mashup interface? And how to I execute the mashup on my laptop?

* I made some changes to the "Alias Details" and then clicked on "Save" and then "Preview", nothing happened. After I clicked on these two buttons a few times, the workflow executed, but the actual mashup preview was not displayed.
** [ES] I am working on fixing this now.

* <del>It would be nice if it were possible to center and resize the workflow to fit in the window (cmd-shift-R)</del>
** [ES] This is supported in the current mashups branch (I haven't built a new binary yet)

Version 2

2011-05-23T17:03:17Z

Ctuttle: /* Claurissa's Notes */

Undocumented Features

2011-05-23T14:01:05Z

Ctuttle:

Features that should be added to the VisTrails Users' Guide

* <del>Many items in the [http://www.vistrails.org/index.php/FAQ FAQ] </del>
* <del>New syntax for specifying modules in packages (<code>_input_ports</code>, <code>_output_ports</code>, <code>_modules</code>)</del>
* <del>Shortcuts for specifying ports (default to current package, allow 'edu.utah.sci.vistrails.basic' to shorten to 'basic')</del>
* <del>Manipulate workflows via the shell (<code>load_package</code> and then add modules, connections, and parameters)</del>
* Better docs on groups/subworkflows and some integration with control flow
* Publication tools (<del>embed in LaTeX</del>, PowerPoint)
* crowdLabs publishing
* <del>Merge documentation (Tommy's code to merge vistrails)</del>
* <del>Persistence package documentation and general tips on data management with VisTrails</del>
* Vistrail/workflow browser documentation

* ...

FAQ

2011-05-22T02:00:13Z

Ctuttle: /* Integrating your software into VisTrails */

Also check our [[known_issues|Known Issues]] page for troubleshooting.

----
== Running workflows ==

=== How can I run a workflow using the command line? ===

('''Updated for version 1.2''') Call vistrails using the following options:

<code>python vistrails.py -b path_to_vistrails_file:pipeline </code>

where pipeline can be a version '''tag name''' or version '''id'''

'''NOTE:''' If you downloaded the MacOS X bundle, you can run vistrails from the command line via the following commands in the Terminal. Change the current directory to wherever VisTrails was installed (often <code>/Applications</code>), and then type:

<code>Vistrails.app/Contents/MacOS/vistrails [<cmd_line_options>]</code> [http://www.vistrails.org/usersguide/batch.html#running-a-specific-workflow-in-batch-mode Running a Specific Workflow in Batch Mode]

=== Using the command line, we'd like to execute a workflow multiple times, with slightly different parameters, and create a series of output files. Is this possible?===

('''Updated for version 1.2''') We can change parameters that have an alias through the command line.

For example, offscreen pipeline in offscreen.vt always creates the file called image.png. If you want generate it with a different filename:

<code>python vistrails.py -b ../examples/offscreen.vt:offscreen -a"filename=other.png"</code>

<code>filename</code> in the example above is the alias name assigned to the parameter in the <code>value</code> method inside the String module. When running a pipeline from the command line, VisTrails will try to start the spreadsheet automatically if the pipeline requires it. For example, this other execution will also start the spreadsheet (attention to how $ characters are escaped when running on bash):

<code>python vistrails.py -b ../examples/head.vt:aliases -a"isovalue=30\$&\$diffuse_color=0.8,0.4,0.2"</code>

You can also execute more than one pipeline on the command line:

<code>python vistrails.py -b ../examples/head.vt:aliases ../examples/spx.vt:spx \
-a"isovalue=30"</code>

Use the -a parameter only once regardless the number of pipelines. [http://www.vistrails.org/usersguide/batch.html#running-a-workflow-with-specific-parameters Running a Workflow with Specific Parameters]

=== I can load a vistrail, and the version tree shows up fine. However, no pipelines appear when I click on a version. What gives? ===

The most likely reason is that the vistrail uses a package that is not registered with VisTrails. You need to identify the needed package and add it to your <code>.vistrails/startup.py</code>. A single line like the following should be enough:

addPackage('enter_package_name_here')

Some packages might need more information. For example:

addPackage('afront', executable_path='/path/to/afront')

Refer to the package documentation for details. The one inconvenient step is that currently there's no automated way to describe what is the missing package. We're working on this feature for future releases.

=== I have a workflow that reads a file and then does some processing. The first time it runs, it executes correctly. But in subsequent, nothing happens. ===

VisTrails caches by default, so after a workflow is executed, if none of its parameters change, it won't be executed again.

If a workflow reads a file using the basic module File, VisTrails does check whether the file was modified since the last run. It does so by keeping a signature that is based on the modification time of the file. And if the file was modified, the File module and all downstream modules (the ones which depend on File) will be executed.

''Note: If you would like your input and output data to be versioned, you can use the Persistence package.''

If you do not want VisTrails to cache executions, you can turn off caching: go to Menu Edit -> Preferences and in the General Configuration tab, change Cache execution results to Never. [http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

=== Can VisTrails execute workflows in parallel? ===

The VisTrails server can only execute pipelines in parallel if there's more than one instance of VisTrails running. The command

<code> self.rpcserver = ThreadedXMLRPCServer((self.temp_xml_rpc_options.server, self.temp_xml_rpc_options.port))</code>

starts a multithreaded version of the XML-RPC server, so it will create a thread for each request received by the server. The problem is that Qt/PyQT doesn't allow these multiple threads create GUI objects, only in the main thread. To overcome this limitation, the multithreaded version can instantiate other single threaded versions of VisTrails and put them in a queue, so workflow executions and other GUI-related requests, such as generating workflow graphs and history trees can be forwarded to this queue, and each instance takes turns in answering the request. If the results are in the cache, the multithreaded version answers the requests directly.

Note that this infrastructure works on Linux only. To make this work on Windows, you have to create a script similar to start_vistrails_xvfb.sh (located in the scripts folder) where you can send the number of other instances via command-line options to VisTrails. The command line options are:

python vistrails_server.py -T <ADDRESS> -R <PORT> -O<NUMBER_OF_OTHER_VISTRAILS_INSTANCES> [-M]&

If you want the main vistrails instance to be multithreaded, use the -M at the end.

After creating this script, update function start_other_instances in vistrails/gui/application_server.py lines 1007-1023 and set the script variable to point to your script. You may also have to change the arguments sent to your script (line 1016: for example, you don't need to set a virtual display). You will need to change the path to the stop_vistrails_server.py script (on line 1026) according to your installation path. [http://www.vistrails.org/usersguide/batch.html#executing-workflows-in-parallel Executing Workflows in Parallel]

=== When a workflow is executed, what do the colors mean? ===

- lilac: module was notexecuted

- yellow: module is currently being executed

- green: module was successfully executed

- orange: module was cached

- red: the execution of the module failed

[http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

== Building workflows ==

=== Is there a way to give each widget a "display name" in addition to the module name at the center of the widget? ===

Yes, a "display name" can be assigned to a module by selecting the triangle in its top right corner to open a popup menu and selecting the Set Module Label... menu item. You will then be prompted to enter the "display name". [http://www.vistrails.org/usersguide/creating.html#changing-module-labels Changing Module Labels]

=== Is there a way to re-center the picture-in-picture (PiP) view? ===

Yes. If you click on the PIP window to bring it to focus, you can press Ctrl-R (or Command-R on Mac) to re-center the PiP window. [http://www.vistrails.org/usersguide/getting_started.html#vistrails-interaction Vistrails Interaction]

=== How do I search for a literal "?" (question mark) in the search box in the Property panel? ===

Since we allow regular expressions in our search box, question marks are treated as meta-characters. Thus, searching for "?" returns everything and "abc?" will return everything containing "abc". You need to use "\?" instead to search for "?". So the search for "??" would be "\?\?". [http://www.vistrails.org/usersguide/querying.html#textual-queries Textual Queries]

== Using VisTrails as a server ==

=== What is the VisTrails server-mode? ===

Using the VisTrails server mode, it is possible to ''execute workflows and control VisTrails through another application''. For example, the CrowdLabs Web portal (http://www.crowdlabs.org) accesses a VisTrails sever to execute workflows, retrieve and display vistrail trees and workflows. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

=== How do I execute workflows and control VisTrails through another application? ===

The way you access the server is by doing XML-RPC calls. In the current VisTrails release, we include a set of PHP scripts that can talk to a VisTrails server instance. They are in "extensions/http" folder. The files are reasonably well documented. Also, it should be not difficult to create python scripts to access the server (just use xmlrpclib module).

Note that the VisTrails server requires the provenance and workflows to be in a database. More detailed instructions on how to setup the server and the database
are available here:

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_server_setup.html

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_database_setup.html

If what you want is just to execute a series of workflows in batch mode, a simpler solution would be to ''use the VisTrails client in batch mode''. Chapter 12 of the user's guide contains detailed information and examples on that. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

== Control Flow ==

=== Note: using map ===

When using 'map', the module (or subworkflow) used as function port in the map module MUST be a function, i.e., it can only define 1 output port. [http://www.vistrails.org/usersguide/controlflow.html#the-map-operator The Map Operator]

== Spreadsheet ==

Where ''pipeline'' is a version number or a tag.

=== How can I save an image from the spreadsheet? ===

While having the focus on a spreadsheet cell and select the camera on the toolbar to take a snapshot. The system will prompt you for the location and file name where it should be saved. The other icons can be used for saving multiple images that can be used for generating an animation on demand. A whole sheet can also be saved by selecting Export (either from the menu or from the toolbar). [http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet-image Saving a Spreadsheet Image]

===Is it possible to save the complete state of the spreadsheet?===
[http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet Saving a Spreadsheet]

=== Can I view multiple sheets at the same time? ===

Yes. Each sheet on the spreadsheet can be displayed as a dock widget separated from the main spreadsheet window by dragging its tab name out of the tab bar at the bottom of the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Then, how can I put back a separated sheet? ===

A sheet can be docked back to the main window by dragging it back to the tab bar or double-click on its title bar. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== How can I order sheets on the spreadsheet? ===

This can be done by dragging the sheet name on the bottom top bar and drop it to the right place. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Can I control where a cell will be placed on the spreadsheet window? ===

By default, an unoccupied cell on the active sheet will be chosen to display the result. However, you can specify exactly in the pipeline where a spreadsheet cell will be placed by using CellLocation and SheetReference. CellLocation specifies the location (row and column) of a cell when connecting to a spreadsheet cell (VTKCell, ImageViewerCell, ...). Similarly, a SheetReference module (when connecting to a CellLocation) will specify which sheet the cell will be put on given its name, minimum row size and minimum column size. There is an example of this in examples/vtk.xml (select the version below Double Renderer). [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

=== How do I output results to the spreadsheet? ===

By inspecting the VisTrails Spreadsheet package (in the list of packages, to the left of the pipeline builder), you
can see there are built-in cells for different kinds of data, e.g., RichTextCell to display HTML and plain text. op
You (the user) can also define new cell types to display application-specific data. For example, we have developed
VtkCell, MplFigureCell, and OpenGLCell. It is possible to display pretty much anything on the Spreadsheet! [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

Examples of writing cell modules can be found in:
RichTextCell: packages/spreadsheet/widgets/richtext/richtext.py
VTK: packages/vtk/vtkcell.py

Here is the summary of some requirements on a cell widget:

(1) It must be a Qt widget. It should inherit from spreadsheet_cell.QCellWidget in the spreadsheet package. Although any Qt Widget would work, certain features such as animation will not be available (without rewriting it).

(2) It must re-implement the updateContents() function to take a set of inputs (usually coming from input ports of a wrapper Module) and display on the cells. VisTrails uses this function to update/reuse cells on the spreadsheet when new data comes in.

(3) It needs a wrapper VisTrails Module (inherited from basic_widgets.SpreadsheetCell of the spreadsheet package). Inside the compute() method of this module, it may call self.display(CellWidgetType, (inputs)) to trigger the display event on the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#advanced-cell-options Advanced Cell Options]

=== How do I control the default number of cells in the spreadsheet? ===

You can configure the rowCount and colCount using the preferences dialog. Just go to the Module Packages tab, select spreadsheet in the "Enabled packages" and press the Configure button. Then a list of all the configuration options for the spreadsheet will show up. [http://www.vistrails.org/usersguide/spreadsheet.html#custom-layout-options Custom Layout Options]

=== Is it possible to launch a web browser from the vistrails spreadsheet? We would like to output several urls from a parameter sweep and then have the option to click on each one to view the resulting page. I can view the page within the spreadsheet, but it is really too crowded.===

Currently, there isn't a widget that provides exactly this functionality, but I can think of a few solutions that may work for you:

(1) You can use parameter exploration to generate multiple sheets so you might have an exploration that opens each page in a new sheet. Use the third column/dimension in the exploration interface to have a parameter span sheets.

(2) The spreadsheet is extensible so you can write a custom spreadsheet cell widget that has a button or label with the desired link (a QLabel with openExternalLinks set to True, for example).

(3) You can tweak the existing RichTextCell be adding the line "self.browser.setOpenExternalLinks(True)" at line 63 of the source file "vistrails/packages/spreadsheet/widgets/richtext/richtext.py". Then, if your workflow creates a file with html markup text like "<a href="http://www.vistrails.org">VisTrails</a>" connected to a RichTextCell, clicking on the rendered link in the cell will open it in a web browser. You need to add the aforementioned line to the source to let Qt know that you want the link opened externally; by default, it will just issue an event that isn't processed. [http://www.vistrails.org/usersguide/spreadsheet.html#launching-a-web-browser Launching a Web Browser]

== Integrating your software into VisTrails ==

===How can I integrate my own program into VisTrails?===

The easiest way is to create a package. Writing a package is often very simple, here are instructions on how to do it: [[UsersGuideVisTrailsPackages]]

You can also dynamically generate modules. For an example see:

http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#Packages_that_generate_modules_dynamically
[http://www.vistrails.org/usersguide/packages.html#generating-modules-dynamically Generating Modules Dynamically]

In particular, see the new_module call which uses python's type() function to generate new classes dynamically.

===How do I add a port that is not visible on the module (when it appears on the design canvas)?===

This can be accomplished via the "optional" argument. This is the fourth argument of add_input_port (add_output_port) or can be specified as a kwarg. In your example, this would look like:

reg.add_input_port(MyModule, "MyPort", (core.modules.basic_modules.String, 'MyPort Name'), True)

or with kwargs

reg.add_input_port(MyModule, "MyPort", (core.modules.basic_modules.String, 'MyPort Name'),\
optional=True)

or

_input_ports = [('MyPort', '(core.modules.basic_modules.String)', {"optional": True})]
[http://www.vistrails.org/usersguide/packages.html#configuring-ports Configuring Ports]

=== How do modules deal with multiple inputs in a same port? ===

(And should that even be allowed?)

For compatibility reasons, we do need to allow multiple connections to an input port. However, most package developers should never have to use this, and so we do our best to hide it. the default behavior for getting inputs from a port, then, is to always return a single input.

If on your module you need multiple inputs connected to a single port, use the 'forceGetInputListFromPort' method. It will return a list of all the data items coming through the port. The spreadsheet package uses this feature, so look there for usage examples (vistrails/packages/spreadsheet/basic_widgets.py)
[http://www.vistrails.org/usersguide/packages.html#configuring-ports Configuring Ports]

=== Are there mechanisms for attaching widgets to different modules/parameters? ===

Right now, we have a mechanism for putting a specific widget for an input port. For example, if a port is SetColor(red, green, blue), we can put a color wheel widget there. Or we can also replace the SetFileName port with a File Widget. However, this is not per parameter (only per port). We are currently working on this problem.

=== Can I organize my package so it appears hierarchical in the module palette? ===

Yes. Use the namespace keyword argument when adding the module to the registry. For example,

registry.add_module(MyModule, namespace='MyNamespace')
[http://www.vistrails.org/usersguide/packages.html#configuring-modules Configuring Modules - Hierarchy and Visibility]

=== Can I nest namespaces? ===

Yes. Use the '|' character to separate different the hierarchy. For example,

registry.add_module(MyModule, namespace='ParentNamespace|ChildNamespace')

[http://www.vistrails.org/usersguide/packages.html#configuring-modules Configuring Modules - Hierarchy and Visibility]

=== Are there shortcuts for registry initialization? ===

Yes. If you define _modules as a list of classes in the __init__.py file of your package, VisTrails will attempt to load all classes specified as modules. You can provide add_module options as keyword arguments by specifying a tuple (class, kwargs) in the list. For example:

_modules = [MyModule1, (MyModule2, {'namespace': 'MyNamespace'})]

In addition, you need to identify the ports of your modules as a field in your class by defining _input_ports and _output_ports lists. Here, the items in each list must be tuples of the form (portName, portSignature, optional=False, sort_key=-1). For example:

class MyModule(Module):
def compute(self):
pass

_input_ports = [('firstInput', String), ('secondInput', Integer, True)]
_output_ports = [('firstOutput', String), ('secondOutput', String)]

[http://www.vistrails.org/usersguide/packages.html#customizing-modules-and-ports Customizing Modules and Ports]

=== Can I define ports to be of types that I do not import into my package? ===

Yes. You can pass an identifier string as the portSignature instead. The port_signature string is defined by:

<module_string> := <package_identifier>:[<namespace>|]<module_name>,
<port_signature> := (<module_string>*)

For example,

registry.add_input_port(MyModule, 'myInputPort', '(edu.utah.sci.vistrails.basic:String)')

or

_input_ports = [('myInputPort', '(edu.utah.sci.vistrails.basic:String)')]

[http://www.vistrails.org/usersguide/packages.html#configuring-ports Configuring Ports - Port Types]

=== What do I need to change in my package to make it reloadable (new in v1.4.2)? ===
See [http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#How_to_make_you_package_reloadable UsersGuideVisTrailsPackages] for an explanation.
[http://www.vistrails.org/usersguide/packages.html#creating-reloadable-packages Creating Reloadable Packages]

=== Can I add default values or labels for parameters? ===

Yes. Versions 1.4 and greater support these features. See [[UsersGuideVisTrailsPackages#Adding_default_values_and.2For_labels_for_parameters | UsersGuideVisTrailsPacakges]] for more details.
[http://www.vistrails.org/usersguide/packages.html#configuring-ports Configuring Ports - Default Values and Labels]

=== I want to write a module to load HDF data whose output (e.g., data, string) varies according to the input I give it. Is is possible to do this in VisTrails, and if yes, how can I do that? Ideally, I would like to avoid having to change the connection of my output every time I change the input. ===

There are a few ways to tackle this - each has it's own benefits and pitfalls. Firstly, module connections do respect class hierarchies as we're familiar with in object oriented languages. For instance, A module can output a Constant of which String, Float, Integer, etc are specifications. In this way, you can have a subclass of something like HDFData be passed out of the module and the connections will be established regardless of the sub-type. This is a bit dangerous though. Modules downstream of such a class may not really know how to operate on certain types derived from the super-class. Extreme care must be taken both when creating the modules as well as connecting them to prevent things like this from happening.

A second method that I employ in several different packages is the idea of a container class. For instance, the NumSciPy package uses a relatively generic container "Numpy Array" to encapsulate the data. Of course, these encapsulating objects can store dictionaries that other modules can easily access and understand how to operate on. Although this method is slightly more work, the benefits of a stricter typing of ports is beneficial - particularly upon interfacing with other packages that may depend on strongly typed constants (for example).
[http://www.vistrails.org/usersguide/packages.html#configuring-ports Varying Output According to the Input]

=== I need to determine, at run-time, whether or not a "child" module is attached to the output port of a "parent" module. (I do not specifically need to know which child; just if there is one). ===

The outputPorts dictionary of the base Module stores this information. Thus, you should be able to check

("myPortName" in self.outputPorts)

on the parent module to check if there are any downstream connections from the port "myPortName". This might be used, for example, to only set results for output ports that will be used. ***Note***, however, that the caching algorithm assumes that all outputs are set so adding a new connection to a previously unconnected output port will not work as desired if that module is cached. For this reason, I would currently recommend making such a module not cacheable. Another possibility is overriding the update() method to check the outputPorts and set the upToDate flag if they are not equal. In a single, limited test, this seemed to work, but be warned that it is not fully tested. Here is an example:

class TestModule(Module):
_output_ports = [('a1', '(edu.utah.sci.vistrails.basic:String)'),
('a2', '(edu.utah.sci.vistrails.basic:String)')]
def __init__(self):
Module.__init__(self)
self._cached_output_ports = set()

def update(self):
if len(set(self.outputPorts) - self._cached_output_ports) > 0:
self.upToDate = False
Module.update(self)

def compute(self):
if "a1" in self.outputPorts:
self.setResult("a1", "test")
if "a2" in self.outputPorts:
self.setResult("a2", "test2")
self._cached_output_ports = set(self.outputPorts)

[http://www.vistrails.org/usersguide/packages.html#configuring-ports Configuring Ports - Determining Whether or Not a Module is Attached to an Output Port]

=== How can I make a module not display in the modules list? ===

You should set the abstract parameter to True when adding the module to the registry. Using the original syntax, this looks like:

def initialize():
reg = core.modules.module_registry.get_module_registry()
reg.add_module(InvisibleModule, abstract=True)
# ...

With the _modules dictionary shortcut (for more details, see [http://www.vistrails.org/index.php/FAQ#Are_there_shortucts_for_registry_initialization.3F the FAQ section on this]), you include it in a kwargs dict as part of a module tuple:

_modules = [AnotherModule, (InvisibleModule, {'abstract': True})]

There is also a 'hide_descriptor' parameter that prevents the module from appearing in the module palette without declaring it to be abstract.

The technical difference between the two is that 'abstract' will not add the item to the module palette while 'hide_descriptor' does add the item but immediately hides it. If the module should never be instantiated in a workflow, declare it abstract. If you don't want users to be able to add the module to a pipeline, but you have code that may add it programmatically, declare it with hide_descriptor=True.

[http://www.vistrails.org/usersguide/packages.html#configuring-modules Configuring Modules - Hierarchy and Visibility]

== The Console ==

=== Where should I go to find out what I can call from the console and how to import it? ===

We have tried to make some methods more accessible in the console via an api. You can import the api via <code>import api</code> in the console and see the available methods with <code>dir(api)</code>. To open a vistrail:

import api
api.open_vistrail_from_file('/Applications/VisTrails/examples/terminator.vt')

To execute a version of a workflow, you currently have to go through the controller:

api.select_version('Histogram')
api.get_current_controller().execute_current_workflow()

Currently, only a subset of VisTrails functionality is directly available from the api. However, since VisTrails is written in python, you can dig down starting with the VistrailsApplication or controller object to expose most of our internal methods. If you have suggestions for calls to be added to the api, please let us know.

One other feature that we're working on, but is still in progress is the ability to construct workflows via the console. For example:

vtk = load_package('edu.utah.sci.vistrails.vtk')
vtk.vtkDataSetReader() # adds a vtkDataSetReader module to the pipeline
# click on the new module
a = selected_modules()[0] # get the one currently selected module
a.SetFile('/vistrails/examples/data/head120.vtk') # sets the SetFile parmaeter for the data set reader
b = vtk.vtkContourFilter() # adds a vtkContourFilter module to the pipeline and saves to var b
b.SetInputConnection0(a.GetOutputPort0()) # connects a's GetOutputPort0 port to b's SetInputConnection0
[http://www.vistrails.org/usersguide/batch.html#finding-methods-via-the-command-line Finding Methods Via the Command Line]

== Persistence Package ==

=== How do I use the output of one workflow as the input for another using the persistence package? ===

You need to configure the persistence modules using the module's configuration dialog. After adding a <code>PersistentOutputFile</code> to the workflow, click on the triangle in the upper-right corner of the <code>PersistentOutputFile</code>, and select "Edit Configuration" from the menu that appears. In this dialog, select "Create New Reference" and give the reference a name (and any space-delimited tags). Upon running that workflow, the data will be written to the persistent store. In the second workflow where you wish to use that file, add a <code>PersistentInputFile</code> and go to its configuration dialog in the same manner as with the output file. In that dialog, select "Use Existing Reference" and select the data that you just added in the first workflow in the list of files below. Now, when you run that workflow, it will grab the data from the persistent store.

Here is an example: [[Media:offscreen_persistent.vt | offscreen_persistent.vt]]. Run the "persistent offscreen" workflow first, and then run the "display persistent output" to use the output of the first workflow as the input for the second.

== VTK ==

=== Given a VTK visualization, how can I generate a webpage from it? ===

Check out the html pipeline in offscreen.xml.

=== I'm trying to use VTK, but there doesn't seem to be any output. What is wrong? ===

To use VTK on VisTrails, you need a slightly different way of connecting the renderer modules. Instead of using the standard RenderWindow/RenderWindowInteractor infrastructure, you simply connect the renderer to a VTKCell. The examples directory in the distribution has several VTK examples that illustrate.

=== I am trying to add a module to the workflow via Python, but how can I access vtk modules? ===

Here's an example:

import api

vtvtk = 'edu.utah.sci.vistrails.vtk'

module = api.add_module(0, 0, vtvtk, 'vtkContourFilter', '')

The third argument in add_module is the package identifier. You can find this in the "Module Packages" panel of the Preferences; just click on the package you're interested in and it will appear in the information on the right.

== matplotlib ==

=== I'm experiencing a problem with Latex labels and the matplotlib that comes with VisTrails 1.5. The script below entered to the interpreter that comes with VT is sufficient to reproduce it. ===

import matplotlib.pyplot as plt
plt.plot([1,2,3],[1,2,3])
plt.xlabel("$foo$")

Remove your ~/.matplotlib folder and re-start VisTrails

== VisTrails Development ==

=== I would like to build VisTrails from source. Are there instructions on how to do this? ===

Yes! Take a look at http://www.vistrails.org/index.php/Mac_Intel_Instructions
[http://www.vistrails.org/usersguide/getting_started.html#installing-vistrails-from-source Installing VisTrails from source]

== Accessing Provenance Information ==

=== How do I access the information in the execution log? ===

The code responsible for storing execution information is located in the "core/log" directories, and the code that generates much of that information is in "core/interpreter/cached.py". Modules can add execution-specific annotations to provenance via annotate() calls during execution, but much of the data (like timing and errors) is captured by the LogController and CachedInterpreter (the execution engine) objects. To analyze the log from a vistrail (.vt) file, you might have something like the following:

<code>
import core.log.log
import db.services.io

def run(fname):
# open the .vt bundle specified by the filename "fname"
bundle = db.services.io.open_vistrail_bundle_from_zip_xml(fname)[0]
# get the log filename
log_fname = bundle.vistrail.db_log_filename
if log_fname is not None:
# open the log
log = db.services.io.open_log_from_xml(log_fname, True)
# convert the log from a db object
core.log.log.Log.convert(log)
for workflow_exec in log.workflow_execs:
print 'workflow version:', workflow_exec.parent_version
print 'time started:', workflow_exec.ts_start
print 'time ended:', workflow_exec.ts_end
print 'modules executed:', [i.module_id
for i in workflow_exec.item_execs]
if __name__ == '__main__':
run("some_vistrail.vt")
</code>

You should be able to see what information is available by looking at the "core/log" classes. [http://www.vistrails.org/usersguide/log.html Accessing the Execution Log]

Version 2

2011-05-21T18:59:41Z

Ctuttle:

FAQ

2011-05-21T16:50:10Z

Ctuttle: /* How do I add a port that is not visible on the module (when it appears on the design canvas)? */

Also check our [[known_issues|Known Issues]] page for troubleshooting.

----
== Running workflows ==

=== How can I run a workflow using the command line? ===

('''Updated for version 1.2''') Call vistrails using the following options:

<code>python vistrails.py -b path_to_vistrails_file:pipeline </code>

where pipeline can be a version '''tag name''' or version '''id'''

'''NOTE:''' If you downloaded the MacOS X bundle, you can run vistrails from the command line via the following commands in the Terminal. Change the current directory to wherever VisTrails was installed (often <code>/Applications</code>), and then type:

<code>Vistrails.app/Contents/MacOS/vistrails [<cmd_line_options>]</code> [http://www.vistrails.org/usersguide/batch.html#running-a-specific-workflow-in-batch-mode Running a Specific Workflow in Batch Mode]

=== Using the command line, we'd like to execute a workflow multiple times, with slightly different parameters, and create a series of output files. Is this possible?===

('''Updated for version 1.2''') We can change parameters that have an alias through the command line.

For example, offscreen pipeline in offscreen.vt always creates the file called image.png. If you want generate it with a different filename:

<code>python vistrails.py -b ../examples/offscreen.vt:offscreen -a"filename=other.png"</code>

<code>filename</code> in the example above is the alias name assigned to the parameter in the <code>value</code> method inside the String module. When running a pipeline from the command line, VisTrails will try to start the spreadsheet automatically if the pipeline requires it. For example, this other execution will also start the spreadsheet (attention to how $ characters are escaped when running on bash):

<code>python vistrails.py -b ../examples/head.vt:aliases -a"isovalue=30\$&\$diffuse_color=0.8,0.4,0.2"</code>

You can also execute more than one pipeline on the command line:

<code>python vistrails.py -b ../examples/head.vt:aliases ../examples/spx.vt:spx \
-a"isovalue=30"</code>

Use the -a parameter only once regardless the number of pipelines. [http://www.vistrails.org/usersguide/batch.html#running-a-workflow-with-specific-parameters Running a Workflow with Specific Parameters]

=== I can load a vistrail, and the version tree shows up fine. However, no pipelines appear when I click on a version. What gives? ===

The most likely reason is that the vistrail uses a package that is not registered with VisTrails. You need to identify the needed package and add it to your <code>.vistrails/startup.py</code>. A single line like the following should be enough:

addPackage('enter_package_name_here')

Some packages might need more information. For example:

addPackage('afront', executable_path='/path/to/afront')

Refer to the package documentation for details. The one inconvenient step is that currently there's no automated way to describe what is the missing package. We're working on this feature for future releases.

=== I have a workflow that reads a file and then does some processing. The first time it runs, it executes correctly. But in subsequent, nothing happens. ===

VisTrails caches by default, so after a workflow is executed, if none of its parameters change, it won't be executed again.

If a workflow reads a file using the basic module File, VisTrails does check whether the file was modified since the last run. It does so by keeping a signature that is based on the modification time of the file. And if the file was modified, the File module and all downstream modules (the ones which depend on File) will be executed.

''Note: If you would like your input and output data to be versioned, you can use the Persistence package.''

If you do not want VisTrails to cache executions, you can turn off caching: go to Menu Edit -> Preferences and in the General Configuration tab, change Cache execution results to Never. [http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

=== Can VisTrails execute workflows in parallel? ===

The VisTrails server can only execute pipelines in parallel if there's more than one instance of VisTrails running. The command

<code> self.rpcserver = ThreadedXMLRPCServer((self.temp_xml_rpc_options.server, self.temp_xml_rpc_options.port))</code>

starts a multithreaded version of the XML-RPC server, so it will create a thread for each request received by the server. The problem is that Qt/PyQT doesn't allow these multiple threads create GUI objects, only in the main thread. To overcome this limitation, the multithreaded version can instantiate other single threaded versions of VisTrails and put them in a queue, so workflow executions and other GUI-related requests, such as generating workflow graphs and history trees can be forwarded to this queue, and each instance takes turns in answering the request. If the results are in the cache, the multithreaded version answers the requests directly.

Note that this infrastructure works on Linux only. To make this work on Windows, you have to create a script similar to start_vistrails_xvfb.sh (located in the scripts folder) where you can send the number of other instances via command-line options to VisTrails. The command line options are:

python vistrails_server.py -T <ADDRESS> -R <PORT> -O<NUMBER_OF_OTHER_VISTRAILS_INSTANCES> [-M]&

If you want the main vistrails instance to be multithreaded, use the -M at the end.

After creating this script, update function start_other_instances in vistrails/gui/application_server.py lines 1007-1023 and set the script variable to point to your script. You may also have to change the arguments sent to your script (line 1016: for example, you don't need to set a virtual display). You will need to change the path to the stop_vistrails_server.py script (on line 1026) according to your installation path. [http://www.vistrails.org/usersguide/batch.html#executing-workflows-in-parallel Executing Workflows in Parallel]

=== When a workflow is executed, what do the colors mean? ===

- lilac: module was notexecuted

- yellow: module is currently being executed

- green: module was successfully executed

- orange: module was cached

- red: the execution of the module failed

[http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

== Building workflows ==

=== Is there a way to give each widget a "display name" in addition to the module name at the center of the widget? ===

Yes, a "display name" can be assigned to a module by selecting the triangle in its top right corner to open a popup menu and selecting the Set Module Label... menu item. You will then be prompted to enter the "display name". [http://www.vistrails.org/usersguide/creating.html#changing-module-labels Changing Module Labels]

=== Is there a way to re-center the picture-in-picture (PiP) view? ===

Yes. If you click on the PIP window to bring it to focus, you can press Ctrl-R (or Command-R on Mac) to re-center the PiP window. [http://www.vistrails.org/usersguide/getting_started.html#vistrails-interaction Vistrails Interaction]

=== How do I search for a literal "?" (question mark) in the search box in the Property panel? ===

Since we allow regular expressions in our search box, question marks are treated as meta-characters. Thus, searching for "?" returns everything and "abc?" will return everything containing "abc". You need to use "\?" instead to search for "?". So the search for "??" would be "\?\?". [http://www.vistrails.org/usersguide/querying.html#textual-queries Textual Queries]

== Using VisTrails as a server ==

=== What is the VisTrails server-mode? ===

Using the VisTrails server mode, it is possible to ''execute workflows and control VisTrails through another application''. For example, the CrowdLabs Web portal (http://www.crowdlabs.org) accesses a VisTrails sever to execute workflows, retrieve and display vistrail trees and workflows. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

=== How do I execute workflows and control VisTrails through another application? ===

The way you access the server is by doing XML-RPC calls. In the current VisTrails release, we include a set of PHP scripts that can talk to a VisTrails server instance. They are in "extensions/http" folder. The files are reasonably well documented. Also, it should be not difficult to create python scripts to access the server (just use xmlrpclib module).

Note that the VisTrails server requires the provenance and workflows to be in a database. More detailed instructions on how to setup the server and the database
are available here:

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_server_setup.html

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_database_setup.html

If what you want is just to execute a series of workflows in batch mode, a simpler solution would be to ''use the VisTrails client in batch mode''. Chapter 12 of the user's guide contains detailed information and examples on that. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

== Control Flow ==

=== Note: using map ===

When using 'map', the module (or subworkflow) used as function port in the map module MUST be a function, i.e., it can only define 1 output port. [http://www.vistrails.org/usersguide/controlflow.html#the-map-operator The Map Operator]

== Spreadsheet ==

Where ''pipeline'' is a version number or a tag.

=== How can I save an image from the spreadsheet? ===

While having the focus on a spreadsheet cell and select the camera on the toolbar to take a snapshot. The system will prompt you for the location and file name where it should be saved. The other icons can be used for saving multiple images that can be used for generating an animation on demand. A whole sheet can also be saved by selecting Export (either from the menu or from the toolbar). [http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet-image Saving a Spreadsheet Image]

===Is it possible to save the complete state of the spreadsheet?===
[http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet Saving a Spreadsheet]

=== Can I view multiple sheets at the same time? ===

Yes. Each sheet on the spreadsheet can be displayed as a dock widget separated from the main spreadsheet window by dragging its tab name out of the tab bar at the bottom of the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Then, how can I put back a separated sheet? ===

A sheet can be docked back to the main window by dragging it back to the tab bar or double-click on its title bar. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== How can I order sheets on the spreadsheet? ===

This can be done by dragging the sheet name on the bottom top bar and drop it to the right place. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Can I control where a cell will be placed on the spreadsheet window? ===

By default, an unoccupied cell on the active sheet will be chosen to display the result. However, you can specify exactly in the pipeline where a spreadsheet cell will be placed by using CellLocation and SheetReference. CellLocation specifies the location (row and column) of a cell when connecting to a spreadsheet cell (VTKCell, ImageViewerCell, ...). Similarly, a SheetReference module (when connecting to a CellLocation) will specify which sheet the cell will be put on given its name, minimum row size and minimum column size. There is an example of this in examples/vtk.xml (select the version below Double Renderer). [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

=== How do I output results to the spreadsheet? ===

By inspecting the VisTrails Spreadsheet package (in the list of packages, to the left of the pipeline builder), you
can see there are built-in cells for different kinds of data, e.g., RichTextCell to display HTML and plain text. op
You (the user) can also define new cell types to display application-specific data. For example, we have developed
VtkCell, MplFigureCell, and OpenGLCell. It is possible to display pretty much anything on the Spreadsheet! [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

Examples of writing cell modules can be found in:
RichTextCell: packages/spreadsheet/widgets/richtext/richtext.py
VTK: packages/vtk/vtkcell.py

Here is the summary of some requirements on a cell widget:

(1) It must be a Qt widget. It should inherit from spreadsheet_cell.QCellWidget in the spreadsheet package. Although any Qt Widget would work, certain features such as animation will not be available (without rewriting it).

(2) It must re-implement the updateContents() function to take a set of inputs (usually coming from input ports of a wrapper Module) and display on the cells. VisTrails uses this function to update/reuse cells on the spreadsheet when new data comes in.

(3) It needs a wrapper VisTrails Module (inherited from basic_widgets.SpreadsheetCell of the spreadsheet package). Inside the compute() method of this module, it may call self.display(CellWidgetType, (inputs)) to trigger the display event on the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#advanced-cell-options Advanced Cell Options]

=== How do I control the default number of cells in the spreadsheet? ===

You can configure the rowCount and colCount using the preferences dialog. Just go to the Module Packages tab, select spreadsheet in the "Enabled packages" and press the Configure button. Then a list of all the configuration options for the spreadsheet will show up. [http://www.vistrails.org/usersguide/spreadsheet.html#custom-layout-options Custom Layout Options]

=== Is it possible to launch a web browser from the vistrails spreadsheet? We would like to output several urls from a parameter sweep and then have the option to click on each one to view the resulting page. I can view the page within the spreadsheet, but it is really too crowded.===

Currently, there isn't a widget that provides exactly this functionality, but I can think of a few solutions that may work for you:

(1) You can use parameter exploration to generate multiple sheets so you might have an exploration that opens each page in a new sheet. Use the third column/dimension in the exploration interface to have a parameter span sheets.

(2) The spreadsheet is extensible so you can write a custom spreadsheet cell widget that has a button or label with the desired link (a QLabel with openExternalLinks set to True, for example).

(3) You can tweak the existing RichTextCell be adding the line "self.browser.setOpenExternalLinks(True)" at line 63 of the source file "vistrails/packages/spreadsheet/widgets/richtext/richtext.py". Then, if your workflow creates a file with html markup text like "<a href="http://www.vistrails.org">VisTrails</a>" connected to a RichTextCell, clicking on the rendered link in the cell will open it in a web browser. You need to add the aforementioned line to the source to let Qt know that you want the link opened externally; by default, it will just issue an event that isn't processed. [http://www.vistrails.org/usersguide/spreadsheet.html#launching-a-web-browser Launching a Web Browser]

== Integrating your software into VisTrails ==

===How can I integrate my own program into VisTrails?===

The easiest way is to create a package. Writing a package is often very simple, here are instructions on how to do it: [[UsersGuideVisTrailsPackages]]

You can also dynamically generate modules. For an example see:

http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#Packages_that_generate_modules_dynamically
[http://www.vistrails.org/usersguide/packages.html#generating-modules-dynamically Generating Modules Dynamically]

In particular, see the new_module call which uses python's type() function to generate new classes dynamically.

===How do I add a port that is not visible on the module (when it appears on the design canvas)?===

This can be accomplished via the "optional" argument. This is the fourth argument of add_input_port (add_output_port) or can be specified as a kwarg. In your example, this would look like:

reg.add_input_port(MyModule, "MyPort", (core.modules.basic_modules.String, 'MyPort Name'), True)

or with kwargs

reg.add_input_port(MyModule, "MyPort", (core.modules.basic_modules.String, 'MyPort Name'),\
optional=True)

or

_input_ports = [('MyPort', '(core.modules.basic_modules.String)', {"optional": True})]

=== How do modules deal with multiple inputs in a same port? ===

(And should that even be allowed?)

For compatibility reasons, we do need to allow multiple connections to an input port. However, most package developers should never have to use this, and so we do our best to hide it. the default behavior for getting inputs from a port, then, is to always return a single input.

If on your module you need multiple inputs connected to a single port, use the 'forceGetInputListFromPort' method. It will return a list of all the data items coming through the port. The spreadsheet package uses this feature, so look there for usage examples (vistrails/packages/spreadsheet/basic_widgets.py)
[http://www.vistrails.org/usersguide/packages.html#configuring-ports Configuring Ports]

=== Are there mechanisms for attaching widgets to different modules/parameters? ===

Right now, we have a mechanism for putting a specific widget for an input port. For example, if a port is SetColor(red, green, blue), we can put a color wheel widget there. Or we can also replace the SetFileName port with a File Widget. However, this is not per parameter (only per port). We are currently working on this problem.

=== Can I organize my package so it appears hierarchical in the module palette? ===

Yes. Use the namespace keyword argument when adding the module to the registry. For example,

registry.add_module(MyModule, namespace='MyNamespace')
[http://www.vistrails.org/usersguide/packages.html#configuring-modules Configuring Modules - Hierarchy and Visibility]

=== Can I nest namespaces? ===

Yes. Use the '|' character to separate different the hierarchy. For example,

registry.add_module(MyModule, namespace='ParentNamespace|ChildNamespace')

[http://www.vistrails.org/usersguide/packages.html#configuring-modules Configuring Modules - Hierarchy and Visibility]

=== Are there shortcuts for registry initialization? ===

Yes. If you define _modules as a list of classes in the __init__.py file of your package, VisTrails will attempt to load all classes specified as modules. You can provide add_module options as keyword arguments by specifying a tuple (class, kwargs) in the list. For example:

_modules = [MyModule1, (MyModule2, {'namespace': 'MyNamespace'})]

In addition, you need to identify the ports of your modules as a field in your class by defining _input_ports and _output_ports lists. Here, the items in each list must be tuples of the form (portName, portSignature, optional=False, sort_key=-1). For example:

class MyModule(Module):
def compute(self):
pass

_input_ports = [('firstInput', String), ('secondInput', Integer, True)]
_output_ports = [('firstOutput', String), ('secondOutput', String)]

[http://www.vistrails.org/usersguide/packages.html#customizing-modules-and-ports Customizing Modules and Ports]

=== Can I define ports to be of types that I do not import into my package? ===

Yes. You can pass an identifier string as the portSignature instead. The port_signature string is defined by:

<module_string> := <package_identifier>:[<namespace>|]<module_name>,
<port_signature> := (<module_string>*)

For example,

registry.add_input_port(MyModule, 'myInputPort', '(edu.utah.sci.vistrails.basic:String)')

or

_input_ports = [('myInputPort', '(edu.utah.sci.vistrails.basic:String)')]

[http://www.vistrails.org/usersguide/packages.html#configuring-ports Configuring Ports - Port Types]

=== What do I need to change in my package to make it reloadable (new in v1.4.2)? ===
See [http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#How_to_make_you_package_reloadable UsersGuideVisTrailsPackages] for an explanation.
[http://www.vistrails.org/usersguide/packages.html#creating-reloadable-packages Creating Reloadable Packages]

=== Can I add default values or labels for parameters? ===

Yes. Versions 1.4 and greater support these features. See [[UsersGuideVisTrailsPackages#Adding_default_values_and.2For_labels_for_parameters | UsersGuideVisTrailsPacakges]] for more details.
[http://www.vistrails.org/usersguide/packages.html#configuring-ports Configuring Ports - Default Values and Labels]

=== I want to write a module to load HDF data whose output (e.g., data, string) varies according to the input I give it. Is is possible to do this in VisTrails, and if yes, how can I do that? Ideally, I would like to avoid having to change the connection of my output every time I change the input. ===

There are a few ways to tackle this - each has it's own benefits and pitfalls. Firstly, module connections do respect class hierarchies as we're familiar with in object oriented languages. For instance, A module can output a Constant of which String, Float, Integer, etc are specifications. In this way, you can have a subclass of something like HDFData be passed out of the module and the connections will be established regardless of the sub-type. This is a bit dangerous though. Modules downstream of such a class may not really know how to operate on certain types derived from the super-class. Extreme care must be taken both when creating the modules as well as connecting them to prevent things like this from happening.

A second method that I employ in several different packages is the idea of a container class. For instance, the NumSciPy package uses a relatively generic container "Numpy Array" to encapsulate the data. Of course, these encapsulating objects can store dictionaries that other modules can easily access and understand how to operate on. Although this method is slightly more work, the benefits of a stricter typing of ports is beneficial - particularly upon interfacing with other packages that may depend on strongly typed constants (for example).
[http://www.vistrails.org/usersguide/packages.html#configuring-ports Varying Output According to the Input]

=== I need to determine, at run-time, whether or not a "child" module is attached to the output port of a "parent" module. (I do not specifically need to know which child; just if there is one). ===

The outputPorts dictionary of the base Module stores this information. Thus, you should be able to check

("myPortName" in self.outputPorts)

on the parent module to check if there are any downstream connections from the port "myPortName". This might be used, for example, to only set results for output ports that will be used. ***Note***, however, that the caching algorithm assumes that all outputs are set so adding a new connection to a previously unconnected output port will not work as desired if that module is cached. For this reason, I would currently recommend making such a module not cacheable. Another possibility is overriding the update() method to check the outputPorts and set the upToDate flag if they are not equal. In a single, limited test, this seemed to work, but be warned that it is not fully tested. Here is an example:

class TestModule(Module):
_output_ports = [('a1', '(edu.utah.sci.vistrails.basic:String)'),
('a2', '(edu.utah.sci.vistrails.basic:String)')]
def __init__(self):
Module.__init__(self)
self._cached_output_ports = set()

def update(self):
if len(set(self.outputPorts) - self._cached_output_ports) > 0:
self.upToDate = False
Module.update(self)

def compute(self):
if "a1" in self.outputPorts:
self.setResult("a1", "test")
if "a2" in self.outputPorts:
self.setResult("a2", "test2")
self._cached_output_ports = set(self.outputPorts)

[http://www.vistrails.org/usersguide/packages.html#configuring-ports Configuring Ports - Determining Whether or Not a Module is Attached to an Output Port]

=== How can I make a module not display in the modules list? ===

You should set the abstract parameter to True when adding the module to the registry. Using the original syntax, this looks like:

def initialize():
reg = core.modules.module_registry.get_module_registry()
reg.add_module(InvisibleModule, abstract=True)
# ...

With the _modules dictionary shortcut (for more details, see [http://www.vistrails.org/index.php/FAQ#Are_there_shortucts_for_registry_initialization.3F the FAQ section on this]), you include it in a kwargs dict as part of a module tuple:

_modules = [AnotherModule, (InvisibleModule, {'abstract': True})]

There is also a 'hide_descriptor' parameter that prevents the module from appearing in the module palette without declaring it to be abstract.

The technical difference between the two is that 'abstract' will not add the item to the module palette while 'hide_descriptor' does add the item but immediately hides it. If the module should never be instantiated in a workflow, declare it abstract. If you don't want users to be able to add the module to a pipeline, but you have code that may add it programmatically, declare it with hide_descriptor=True.

[http://www.vistrails.org/usersguide/packages.html#configuring-modules Configuring Modules - Hierarchy and Visibility]

== The Console ==

=== Where should I go to find out what I can call from the console and how to import it? ===

We have tried to make some methods more accessible in the console via an api. You can import the api via <code>import api</code> in the console and see the available methods with <code>dir(api)</code>. To open a vistrail:

import api
api.open_vistrail_from_file('/Applications/VisTrails/examples/terminator.vt')

To execute a version of a workflow, you currently have to go through the controller:

api.select_version('Histogram')
api.get_current_controller().execute_current_workflow()

Currently, only a subset of VisTrails functionality is directly available from the api. However, since VisTrails is written in python, you can dig down starting with the VistrailsApplication or controller object to expose most of our internal methods. If you have suggestions for calls to be added to the api, please let us know.

One other feature that we're working on, but is still in progress is the ability to construct workflows via the console. For example:

vtk = load_package('edu.utah.sci.vistrails.vtk')
vtk.vtkDataSetReader() # adds a vtkDataSetReader module to the pipeline
# click on the new module
a = selected_modules()[0] # get the one currently selected module
a.SetFile('/vistrails/examples/data/head120.vtk') # sets the SetFile parmaeter for the data set reader
b = vtk.vtkContourFilter() # adds a vtkContourFilter module to the pipeline and saves to var b
b.SetInputConnection0(a.GetOutputPort0()) # connects a's GetOutputPort0 port to b's SetInputConnection0
[http://www.vistrails.org/usersguide/batch.html#finding-methods-via-the-command-line Finding Methods Via the Command Line]

== Persistence Package ==

=== How do I use the output of one workflow as the input for another using the persistence package? ===

You need to configure the persistence modules using the module's configuration dialog. After adding a <code>PersistentOutputFile</code> to the workflow, click on the triangle in the upper-right corner of the <code>PersistentOutputFile</code>, and select "Edit Configuration" from the menu that appears. In this dialog, select "Create New Reference" and give the reference a name (and any space-delimited tags). Upon running that workflow, the data will be written to the persistent store. In the second workflow where you wish to use that file, add a <code>PersistentInputFile</code> and go to its configuration dialog in the same manner as with the output file. In that dialog, select "Use Existing Reference" and select the data that you just added in the first workflow in the list of files below. Now, when you run that workflow, it will grab the data from the persistent store.

Here is an example: [[Media:offscreen_persistent.vt | offscreen_persistent.vt]]. Run the "persistent offscreen" workflow first, and then run the "display persistent output" to use the output of the first workflow as the input for the second.

== VTK ==

=== Given a VTK visualization, how can I generate a webpage from it? ===

Check out the html pipeline in offscreen.xml.

=== I'm trying to use VTK, but there doesn't seem to be any output. What is wrong? ===

To use VTK on VisTrails, you need a slightly different way of connecting the renderer modules. Instead of using the standard RenderWindow/RenderWindowInteractor infrastructure, you simply connect the renderer to a VTKCell. The examples directory in the distribution has several VTK examples that illustrate.

=== I am trying to add a module to the workflow via Python, but how can I access vtk modules? ===

Here's an example:

import api

vtvtk = 'edu.utah.sci.vistrails.vtk'

module = api.add_module(0, 0, vtvtk, 'vtkContourFilter', '')

The third argument in add_module is the package identifier. You can find this in the "Module Packages" panel of the Preferences; just click on the package you're interested in and it will appear in the information on the right.

== matplotlib ==

=== I'm experiencing a problem with Latex labels and the matplotlib that comes with VisTrails 1.5. The script below entered to the interpreter that comes with VT is sufficient to reproduce it. ===

import matplotlib.pyplot as plt
plt.plot([1,2,3],[1,2,3])
plt.xlabel("$foo$")

Remove your ~/.matplotlib folder and re-start VisTrails

== VisTrails Development ==

=== I would like to build VisTrails from source. Are there instructions on how to do this? ===

Yes! Take a look at http://www.vistrails.org/index.php/Mac_Intel_Instructions
[http://www.vistrails.org/usersguide/getting_started.html#installing-vistrails-from-source Installing VisTrails from source]

== Accessing Provenance Information ==

=== How do I access the information in the execution log? ===

The code responsible for storing execution information is located in the "core/log" directories, and the code that generates much of that information is in "core/interpreter/cached.py". Modules can add execution-specific annotations to provenance via annotate() calls during execution, but much of the data (like timing and errors) is captured by the LogController and CachedInterpreter (the execution engine) objects. To analyze the log from a vistrail (.vt) file, you might have something like the following:

<code>
import core.log.log
import db.services.io

def run(fname):
# open the .vt bundle specified by the filename "fname"
bundle = db.services.io.open_vistrail_bundle_from_zip_xml(fname)[0]
# get the log filename
log_fname = bundle.vistrail.db_log_filename
if log_fname is not None:
# open the log
log = db.services.io.open_log_from_xml(log_fname, True)
# convert the log from a db object
core.log.log.Log.convert(log)
for workflow_exec in log.workflow_execs:
print 'workflow version:', workflow_exec.parent_version
print 'time started:', workflow_exec.ts_start
print 'time ended:', workflow_exec.ts_end
print 'modules executed:', [i.module_id
for i in workflow_exec.item_execs]
if __name__ == '__main__':
run("some_vistrail.vt")
</code>

You should be able to see what information is available by looking at the "core/log" classes. [http://www.vistrails.org/usersguide/log.html Accessing the Execution Log]

FAQ

2011-05-21T16:48:01Z

Ctuttle: /* Integrating your software into VisTrails */

Also check our [[known_issues|Known Issues]] page for troubleshooting.

----
== Running workflows ==

=== How can I run a workflow using the command line? ===

('''Updated for version 1.2''') Call vistrails using the following options:

<code>python vistrails.py -b path_to_vistrails_file:pipeline </code>

where pipeline can be a version '''tag name''' or version '''id'''

'''NOTE:''' If you downloaded the MacOS X bundle, you can run vistrails from the command line via the following commands in the Terminal. Change the current directory to wherever VisTrails was installed (often <code>/Applications</code>), and then type:

<code>Vistrails.app/Contents/MacOS/vistrails [<cmd_line_options>]</code> [http://www.vistrails.org/usersguide/batch.html#running-a-specific-workflow-in-batch-mode Running a Specific Workflow in Batch Mode]

=== Using the command line, we'd like to execute a workflow multiple times, with slightly different parameters, and create a series of output files. Is this possible?===

('''Updated for version 1.2''') We can change parameters that have an alias through the command line.

For example, offscreen pipeline in offscreen.vt always creates the file called image.png. If you want generate it with a different filename:

<code>python vistrails.py -b ../examples/offscreen.vt:offscreen -a"filename=other.png"</code>

<code>filename</code> in the example above is the alias name assigned to the parameter in the <code>value</code> method inside the String module. When running a pipeline from the command line, VisTrails will try to start the spreadsheet automatically if the pipeline requires it. For example, this other execution will also start the spreadsheet (attention to how $ characters are escaped when running on bash):

<code>python vistrails.py -b ../examples/head.vt:aliases -a"isovalue=30\$&\$diffuse_color=0.8,0.4,0.2"</code>

You can also execute more than one pipeline on the command line:

<code>python vistrails.py -b ../examples/head.vt:aliases ../examples/spx.vt:spx \
-a"isovalue=30"</code>

Use the -a parameter only once regardless the number of pipelines. [http://www.vistrails.org/usersguide/batch.html#running-a-workflow-with-specific-parameters Running a Workflow with Specific Parameters]

=== I can load a vistrail, and the version tree shows up fine. However, no pipelines appear when I click on a version. What gives? ===

The most likely reason is that the vistrail uses a package that is not registered with VisTrails. You need to identify the needed package and add it to your <code>.vistrails/startup.py</code>. A single line like the following should be enough:

addPackage('enter_package_name_here')

Some packages might need more information. For example:

addPackage('afront', executable_path='/path/to/afront')

Refer to the package documentation for details. The one inconvenient step is that currently there's no automated way to describe what is the missing package. We're working on this feature for future releases.

=== I have a workflow that reads a file and then does some processing. The first time it runs, it executes correctly. But in subsequent, nothing happens. ===

VisTrails caches by default, so after a workflow is executed, if none of its parameters change, it won't be executed again.

If a workflow reads a file using the basic module File, VisTrails does check whether the file was modified since the last run. It does so by keeping a signature that is based on the modification time of the file. And if the file was modified, the File module and all downstream modules (the ones which depend on File) will be executed.

''Note: If you would like your input and output data to be versioned, you can use the Persistence package.''

If you do not want VisTrails to cache executions, you can turn off caching: go to Menu Edit -> Preferences and in the General Configuration tab, change Cache execution results to Never. [http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

=== Can VisTrails execute workflows in parallel? ===

The VisTrails server can only execute pipelines in parallel if there's more than one instance of VisTrails running. The command

<code> self.rpcserver = ThreadedXMLRPCServer((self.temp_xml_rpc_options.server, self.temp_xml_rpc_options.port))</code>

starts a multithreaded version of the XML-RPC server, so it will create a thread for each request received by the server. The problem is that Qt/PyQT doesn't allow these multiple threads create GUI objects, only in the main thread. To overcome this limitation, the multithreaded version can instantiate other single threaded versions of VisTrails and put them in a queue, so workflow executions and other GUI-related requests, such as generating workflow graphs and history trees can be forwarded to this queue, and each instance takes turns in answering the request. If the results are in the cache, the multithreaded version answers the requests directly.

Note that this infrastructure works on Linux only. To make this work on Windows, you have to create a script similar to start_vistrails_xvfb.sh (located in the scripts folder) where you can send the number of other instances via command-line options to VisTrails. The command line options are:

python vistrails_server.py -T <ADDRESS> -R <PORT> -O<NUMBER_OF_OTHER_VISTRAILS_INSTANCES> [-M]&

If you want the main vistrails instance to be multithreaded, use the -M at the end.

After creating this script, update function start_other_instances in vistrails/gui/application_server.py lines 1007-1023 and set the script variable to point to your script. You may also have to change the arguments sent to your script (line 1016: for example, you don't need to set a virtual display). You will need to change the path to the stop_vistrails_server.py script (on line 1026) according to your installation path. [http://www.vistrails.org/usersguide/batch.html#executing-workflows-in-parallel Executing Workflows in Parallel]

=== When a workflow is executed, what do the colors mean? ===

- lilac: module was notexecuted

- yellow: module is currently being executed

- green: module was successfully executed

- orange: module was cached

- red: the execution of the module failed

[http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

== Building workflows ==

=== Is there a way to give each widget a "display name" in addition to the module name at the center of the widget? ===

Yes, a "display name" can be assigned to a module by selecting the triangle in its top right corner to open a popup menu and selecting the Set Module Label... menu item. You will then be prompted to enter the "display name". [http://www.vistrails.org/usersguide/creating.html#changing-module-labels Changing Module Labels]

=== Is there a way to re-center the picture-in-picture (PiP) view? ===

Yes. If you click on the PIP window to bring it to focus, you can press Ctrl-R (or Command-R on Mac) to re-center the PiP window. [http://www.vistrails.org/usersguide/getting_started.html#vistrails-interaction Vistrails Interaction]

=== How do I search for a literal "?" (question mark) in the search box in the Property panel? ===

Since we allow regular expressions in our search box, question marks are treated as meta-characters. Thus, searching for "?" returns everything and "abc?" will return everything containing "abc". You need to use "\?" instead to search for "?". So the search for "??" would be "\?\?". [http://www.vistrails.org/usersguide/querying.html#textual-queries Textual Queries]

== Using VisTrails as a server ==

=== What is the VisTrails server-mode? ===

Using the VisTrails server mode, it is possible to ''execute workflows and control VisTrails through another application''. For example, the CrowdLabs Web portal (http://www.crowdlabs.org) accesses a VisTrails sever to execute workflows, retrieve and display vistrail trees and workflows. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

=== How do I execute workflows and control VisTrails through another application? ===

The way you access the server is by doing XML-RPC calls. In the current VisTrails release, we include a set of PHP scripts that can talk to a VisTrails server instance. They are in "extensions/http" folder. The files are reasonably well documented. Also, it should be not difficult to create python scripts to access the server (just use xmlrpclib module).

Note that the VisTrails server requires the provenance and workflows to be in a database. More detailed instructions on how to setup the server and the database
are available here:

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_server_setup.html

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_database_setup.html

If what you want is just to execute a series of workflows in batch mode, a simpler solution would be to ''use the VisTrails client in batch mode''. Chapter 12 of the user's guide contains detailed information and examples on that. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

== Control Flow ==

=== Note: using map ===

When using 'map', the module (or subworkflow) used as function port in the map module MUST be a function, i.e., it can only define 1 output port. [http://www.vistrails.org/usersguide/controlflow.html#the-map-operator The Map Operator]

== Spreadsheet ==

Where ''pipeline'' is a version number or a tag.

=== How can I save an image from the spreadsheet? ===

While having the focus on a spreadsheet cell and select the camera on the toolbar to take a snapshot. The system will prompt you for the location and file name where it should be saved. The other icons can be used for saving multiple images that can be used for generating an animation on demand. A whole sheet can also be saved by selecting Export (either from the menu or from the toolbar). [http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet-image Saving a Spreadsheet Image]

===Is it possible to save the complete state of the spreadsheet?===
[http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet Saving a Spreadsheet]

=== Can I view multiple sheets at the same time? ===

Yes. Each sheet on the spreadsheet can be displayed as a dock widget separated from the main spreadsheet window by dragging its tab name out of the tab bar at the bottom of the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Then, how can I put back a separated sheet? ===

A sheet can be docked back to the main window by dragging it back to the tab bar or double-click on its title bar. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== How can I order sheets on the spreadsheet? ===

This can be done by dragging the sheet name on the bottom top bar and drop it to the right place. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Can I control where a cell will be placed on the spreadsheet window? ===

By default, an unoccupied cell on the active sheet will be chosen to display the result. However, you can specify exactly in the pipeline where a spreadsheet cell will be placed by using CellLocation and SheetReference. CellLocation specifies the location (row and column) of a cell when connecting to a spreadsheet cell (VTKCell, ImageViewerCell, ...). Similarly, a SheetReference module (when connecting to a CellLocation) will specify which sheet the cell will be put on given its name, minimum row size and minimum column size. There is an example of this in examples/vtk.xml (select the version below Double Renderer). [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

=== How do I output results to the spreadsheet? ===

By inspecting the VisTrails Spreadsheet package (in the list of packages, to the left of the pipeline builder), you
can see there are built-in cells for different kinds of data, e.g., RichTextCell to display HTML and plain text. op
You (the user) can also define new cell types to display application-specific data. For example, we have developed
VtkCell, MplFigureCell, and OpenGLCell. It is possible to display pretty much anything on the Spreadsheet! [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

Examples of writing cell modules can be found in:
RichTextCell: packages/spreadsheet/widgets/richtext/richtext.py
VTK: packages/vtk/vtkcell.py

Here is the summary of some requirements on a cell widget:

(1) It must be a Qt widget. It should inherit from spreadsheet_cell.QCellWidget in the spreadsheet package. Although any Qt Widget would work, certain features such as animation will not be available (without rewriting it).

(2) It must re-implement the updateContents() function to take a set of inputs (usually coming from input ports of a wrapper Module) and display on the cells. VisTrails uses this function to update/reuse cells on the spreadsheet when new data comes in.

(3) It needs a wrapper VisTrails Module (inherited from basic_widgets.SpreadsheetCell of the spreadsheet package). Inside the compute() method of this module, it may call self.display(CellWidgetType, (inputs)) to trigger the display event on the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#advanced-cell-options Advanced Cell Options]

=== How do I control the default number of cells in the spreadsheet? ===

You can configure the rowCount and colCount using the preferences dialog. Just go to the Module Packages tab, select spreadsheet in the "Enabled packages" and press the Configure button. Then a list of all the configuration options for the spreadsheet will show up. [http://www.vistrails.org/usersguide/spreadsheet.html#custom-layout-options Custom Layout Options]

=== Is it possible to launch a web browser from the vistrails spreadsheet? We would like to output several urls from a parameter sweep and then have the option to click on each one to view the resulting page. I can view the page within the spreadsheet, but it is really too crowded.===

Currently, there isn't a widget that provides exactly this functionality, but I can think of a few solutions that may work for you:

(1) You can use parameter exploration to generate multiple sheets so you might have an exploration that opens each page in a new sheet. Use the third column/dimension in the exploration interface to have a parameter span sheets.

(2) The spreadsheet is extensible so you can write a custom spreadsheet cell widget that has a button or label with the desired link (a QLabel with openExternalLinks set to True, for example).

(3) You can tweak the existing RichTextCell be adding the line "self.browser.setOpenExternalLinks(True)" at line 63 of the source file "vistrails/packages/spreadsheet/widgets/richtext/richtext.py". Then, if your workflow creates a file with html markup text like "<a href="http://www.vistrails.org">VisTrails</a>" connected to a RichTextCell, clicking on the rendered link in the cell will open it in a web browser. You need to add the aforementioned line to the source to let Qt know that you want the link opened externally; by default, it will just issue an event that isn't processed. [http://www.vistrails.org/usersguide/spreadsheet.html#launching-a-web-browser Launching a Web Browser]

== Integrating your software into VisTrails ==

===How can I integrate my own program into VisTrails?===

The easiest way is to create a package. Writing a package is often very simple, here are instructions on how to do it: [[UsersGuideVisTrailsPackages]]

You can also dynamically generate modules. For an example see:

http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#Packages_that_generate_modules_dynamically
[http://www.vistrails.org/usersguide/packages.html#generating-modules-dynamically Generating Modules Dynamically]

In particular, see the new_module call which uses python's type() function to generate new classes dynamically.

===How do I add a port that is not visible on the module (when it appears on the design canvas)?===

This can be accomplished via the "optional" argument. This is the fourth argument of add_input_port (add_output_port) or can be specified as a kwarg. In your example, this would look like:

reg.add_input_port(MyModule, "MyPort", (core.modules.basic_modules.String, 'MyPort Name'), True)

or with kwargs

reg.add_input_port(MyModule, "MyPort", (core.modules.basic_modules.String, 'MyPort Name'), optional=True)

or

_input_ports = [('MyPort', '(core.modules.basic_modules.String)', {"optional": True})]

=== How do modules deal with multiple inputs in a same port? ===

(And should that even be allowed?)

For compatibility reasons, we do need to allow multiple connections to an input port. However, most package developers should never have to use this, and so we do our best to hide it. the default behavior for getting inputs from a port, then, is to always return a single input.

If on your module you need multiple inputs connected to a single port, use the 'forceGetInputListFromPort' method. It will return a list of all the data items coming through the port. The spreadsheet package uses this feature, so look there for usage examples (vistrails/packages/spreadsheet/basic_widgets.py)
[http://www.vistrails.org/usersguide/packages.html#configuring-ports Configuring Ports]

=== Are there mechanisms for attaching widgets to different modules/parameters? ===

Right now, we have a mechanism for putting a specific widget for an input port. For example, if a port is SetColor(red, green, blue), we can put a color wheel widget there. Or we can also replace the SetFileName port with a File Widget. However, this is not per parameter (only per port). We are currently working on this problem.

=== Can I organize my package so it appears hierarchical in the module palette? ===

Yes. Use the namespace keyword argument when adding the module to the registry. For example,

registry.add_module(MyModule, namespace='MyNamespace')
[http://www.vistrails.org/usersguide/packages.html#configuring-modules Configuring Modules - Hierarchy and Visibility]

=== Can I nest namespaces? ===

Yes. Use the '|' character to separate different the hierarchy. For example,

registry.add_module(MyModule, namespace='ParentNamespace|ChildNamespace')

[http://www.vistrails.org/usersguide/packages.html#configuring-modules Configuring Modules - Hierarchy and Visibility]

=== Are there shortcuts for registry initialization? ===

Yes. If you define _modules as a list of classes in the __init__.py file of your package, VisTrails will attempt to load all classes specified as modules. You can provide add_module options as keyword arguments by specifying a tuple (class, kwargs) in the list. For example:

_modules = [MyModule1, (MyModule2, {'namespace': 'MyNamespace'})]

In addition, you need to identify the ports of your modules as a field in your class by defining _input_ports and _output_ports lists. Here, the items in each list must be tuples of the form (portName, portSignature, optional=False, sort_key=-1). For example:

class MyModule(Module):
def compute(self):
pass

_input_ports = [('firstInput', String), ('secondInput', Integer, True)]
_output_ports = [('firstOutput', String), ('secondOutput', String)]

[http://www.vistrails.org/usersguide/packages.html#customizing-modules-and-ports Customizing Modules and Ports]

=== Can I define ports to be of types that I do not import into my package? ===

Yes. You can pass an identifier string as the portSignature instead. The port_signature string is defined by:

<module_string> := <package_identifier>:[<namespace>|]<module_name>,
<port_signature> := (<module_string>*)

For example,

registry.add_input_port(MyModule, 'myInputPort', '(edu.utah.sci.vistrails.basic:String)')

or

_input_ports = [('myInputPort', '(edu.utah.sci.vistrails.basic:String)')]

[http://www.vistrails.org/usersguide/packages.html#configuring-ports Configuring Ports - Port Types]

=== What do I need to change in my package to make it reloadable (new in v1.4.2)? ===
See [http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#How_to_make_you_package_reloadable UsersGuideVisTrailsPackages] for an explanation.
[http://www.vistrails.org/usersguide/packages.html#creating-reloadable-packages Creating Reloadable Packages]

=== Can I add default values or labels for parameters? ===

Yes. Versions 1.4 and greater support these features. See [[UsersGuideVisTrailsPackages#Adding_default_values_and.2For_labels_for_parameters | UsersGuideVisTrailsPacakges]] for more details.
[http://www.vistrails.org/usersguide/packages.html#configuring-ports Configuring Ports - Default Values and Labels]

=== I want to write a module to load HDF data whose output (e.g., data, string) varies according to the input I give it. Is is possible to do this in VisTrails, and if yes, how can I do that? Ideally, I would like to avoid having to change the connection of my output every time I change the input. ===

There are a few ways to tackle this - each has it's own benefits and pitfalls. Firstly, module connections do respect class hierarchies as we're familiar with in object oriented languages. For instance, A module can output a Constant of which String, Float, Integer, etc are specifications. In this way, you can have a subclass of something like HDFData be passed out of the module and the connections will be established regardless of the sub-type. This is a bit dangerous though. Modules downstream of such a class may not really know how to operate on certain types derived from the super-class. Extreme care must be taken both when creating the modules as well as connecting them to prevent things like this from happening.

A second method that I employ in several different packages is the idea of a container class. For instance, the NumSciPy package uses a relatively generic container "Numpy Array" to encapsulate the data. Of course, these encapsulating objects can store dictionaries that other modules can easily access and understand how to operate on. Although this method is slightly more work, the benefits of a stricter typing of ports is beneficial - particularly upon interfacing with other packages that may depend on strongly typed constants (for example).
[http://www.vistrails.org/usersguide/packages.html#configuring-ports Varying Output According to the Input]

=== I need to determine, at run-time, whether or not a "child" module is attached to the output port of a "parent" module. (I do not specifically need to know which child; just if there is one). ===

The outputPorts dictionary of the base Module stores this information. Thus, you should be able to check

("myPortName" in self.outputPorts)

on the parent module to check if there are any downstream connections from the port "myPortName". This might be used, for example, to only set results for output ports that will be used. ***Note***, however, that the caching algorithm assumes that all outputs are set so adding a new connection to a previously unconnected output port will not work as desired if that module is cached. For this reason, I would currently recommend making such a module not cacheable. Another possibility is overriding the update() method to check the outputPorts and set the upToDate flag if they are not equal. In a single, limited test, this seemed to work, but be warned that it is not fully tested. Here is an example:

class TestModule(Module):
_output_ports = [('a1', '(edu.utah.sci.vistrails.basic:String)'),
('a2', '(edu.utah.sci.vistrails.basic:String)')]
def __init__(self):
Module.__init__(self)
self._cached_output_ports = set()

def update(self):
if len(set(self.outputPorts) - self._cached_output_ports) > 0:
self.upToDate = False
Module.update(self)

def compute(self):
if "a1" in self.outputPorts:
self.setResult("a1", "test")
if "a2" in self.outputPorts:
self.setResult("a2", "test2")
self._cached_output_ports = set(self.outputPorts)

[http://www.vistrails.org/usersguide/packages.html#configuring-ports Configuring Ports - Determining Whether or Not a Module is Attached to an Output Port]

=== How can I make a module not display in the modules list? ===

You should set the abstract parameter to True when adding the module to the registry. Using the original syntax, this looks like:

def initialize():
reg = core.modules.module_registry.get_module_registry()
reg.add_module(InvisibleModule, abstract=True)
# ...

With the _modules dictionary shortcut (for more details, see [http://www.vistrails.org/index.php/FAQ#Are_there_shortucts_for_registry_initialization.3F the FAQ section on this]), you include it in a kwargs dict as part of a module tuple:

_modules = [AnotherModule, (InvisibleModule, {'abstract': True})]

There is also a 'hide_descriptor' parameter that prevents the module from appearing in the module palette without declaring it to be abstract.

The technical difference between the two is that 'abstract' will not add the item to the module palette while 'hide_descriptor' does add the item but immediately hides it. If the module should never be instantiated in a workflow, declare it abstract. If you don't want users to be able to add the module to a pipeline, but you have code that may add it programmatically, declare it with hide_descriptor=True.

[http://www.vistrails.org/usersguide/packages.html#configuring-modules Configuring Modules - Hierarchy and Visibility]

== The Console ==

=== Where should I go to find out what I can call from the console and how to import it? ===

We have tried to make some methods more accessible in the console via an api. You can import the api via <code>import api</code> in the console and see the available methods with <code>dir(api)</code>. To open a vistrail:

import api
api.open_vistrail_from_file('/Applications/VisTrails/examples/terminator.vt')

To execute a version of a workflow, you currently have to go through the controller:

api.select_version('Histogram')
api.get_current_controller().execute_current_workflow()

Currently, only a subset of VisTrails functionality is directly available from the api. However, since VisTrails is written in python, you can dig down starting with the VistrailsApplication or controller object to expose most of our internal methods. If you have suggestions for calls to be added to the api, please let us know.

One other feature that we're working on, but is still in progress is the ability to construct workflows via the console. For example:

vtk = load_package('edu.utah.sci.vistrails.vtk')
vtk.vtkDataSetReader() # adds a vtkDataSetReader module to the pipeline
# click on the new module
a = selected_modules()[0] # get the one currently selected module
a.SetFile('/vistrails/examples/data/head120.vtk') # sets the SetFile parmaeter for the data set reader
b = vtk.vtkContourFilter() # adds a vtkContourFilter module to the pipeline and saves to var b
b.SetInputConnection0(a.GetOutputPort0()) # connects a's GetOutputPort0 port to b's SetInputConnection0
[http://www.vistrails.org/usersguide/batch.html#finding-methods-via-the-command-line Finding Methods Via the Command Line]

== Persistence Package ==

=== How do I use the output of one workflow as the input for another using the persistence package? ===

You need to configure the persistence modules using the module's configuration dialog. After adding a <code>PersistentOutputFile</code> to the workflow, click on the triangle in the upper-right corner of the <code>PersistentOutputFile</code>, and select "Edit Configuration" from the menu that appears. In this dialog, select "Create New Reference" and give the reference a name (and any space-delimited tags). Upon running that workflow, the data will be written to the persistent store. In the second workflow where you wish to use that file, add a <code>PersistentInputFile</code> and go to its configuration dialog in the same manner as with the output file. In that dialog, select "Use Existing Reference" and select the data that you just added in the first workflow in the list of files below. Now, when you run that workflow, it will grab the data from the persistent store.

Here is an example: [[Media:offscreen_persistent.vt | offscreen_persistent.vt]]. Run the "persistent offscreen" workflow first, and then run the "display persistent output" to use the output of the first workflow as the input for the second.

== VTK ==

=== Given a VTK visualization, how can I generate a webpage from it? ===

Check out the html pipeline in offscreen.xml.

=== I'm trying to use VTK, but there doesn't seem to be any output. What is wrong? ===

To use VTK on VisTrails, you need a slightly different way of connecting the renderer modules. Instead of using the standard RenderWindow/RenderWindowInteractor infrastructure, you simply connect the renderer to a VTKCell. The examples directory in the distribution has several VTK examples that illustrate.

=== I am trying to add a module to the workflow via Python, but how can I access vtk modules? ===

Here's an example:

import api

vtvtk = 'edu.utah.sci.vistrails.vtk'

module = api.add_module(0, 0, vtvtk, 'vtkContourFilter', '')

The third argument in add_module is the package identifier. You can find this in the "Module Packages" panel of the Preferences; just click on the package you're interested in and it will appear in the information on the right.

== matplotlib ==

=== I'm experiencing a problem with Latex labels and the matplotlib that comes with VisTrails 1.5. The script below entered to the interpreter that comes with VT is sufficient to reproduce it. ===

import matplotlib.pyplot as plt
plt.plot([1,2,3],[1,2,3])
plt.xlabel("$foo$")

Remove your ~/.matplotlib folder and re-start VisTrails

== VisTrails Development ==

=== I would like to build VisTrails from source. Are there instructions on how to do this? ===

Yes! Take a look at http://www.vistrails.org/index.php/Mac_Intel_Instructions
[http://www.vistrails.org/usersguide/getting_started.html#installing-vistrails-from-source Installing VisTrails from source]

== Accessing Provenance Information ==

=== How do I access the information in the execution log? ===

The code responsible for storing execution information is located in the "core/log" directories, and the code that generates much of that information is in "core/interpreter/cached.py". Modules can add execution-specific annotations to provenance via annotate() calls during execution, but much of the data (like timing and errors) is captured by the LogController and CachedInterpreter (the execution engine) objects. To analyze the log from a vistrail (.vt) file, you might have something like the following:

<code>
import core.log.log
import db.services.io

def run(fname):
# open the .vt bundle specified by the filename "fname"
bundle = db.services.io.open_vistrail_bundle_from_zip_xml(fname)[0]
# get the log filename
log_fname = bundle.vistrail.db_log_filename
if log_fname is not None:
# open the log
log = db.services.io.open_log_from_xml(log_fname, True)
# convert the log from a db object
core.log.log.Log.convert(log)
for workflow_exec in log.workflow_execs:
print 'workflow version:', workflow_exec.parent_version
print 'time started:', workflow_exec.ts_start
print 'time ended:', workflow_exec.ts_end
print 'modules executed:', [i.module_id
for i in workflow_exec.item_execs]
if __name__ == '__main__':
run("some_vistrail.vt")
</code>

You should be able to see what information is available by looking at the "core/log" classes. [http://www.vistrails.org/usersguide/log.html Accessing the Execution Log]

FAQ

2011-05-21T16:40:46Z

Ctuttle: /* Integrating your software into VisTrails */

Also check our [[known_issues|Known Issues]] page for troubleshooting.

----
== Running workflows ==

=== How can I run a workflow using the command line? ===

('''Updated for version 1.2''') Call vistrails using the following options:

<code>python vistrails.py -b path_to_vistrails_file:pipeline </code>

where pipeline can be a version '''tag name''' or version '''id'''

'''NOTE:''' If you downloaded the MacOS X bundle, you can run vistrails from the command line via the following commands in the Terminal. Change the current directory to wherever VisTrails was installed (often <code>/Applications</code>), and then type:

<code>Vistrails.app/Contents/MacOS/vistrails [<cmd_line_options>]</code> [http://www.vistrails.org/usersguide/batch.html#running-a-specific-workflow-in-batch-mode Running a Specific Workflow in Batch Mode]

=== Using the command line, we'd like to execute a workflow multiple times, with slightly different parameters, and create a series of output files. Is this possible?===

('''Updated for version 1.2''') We can change parameters that have an alias through the command line.

For example, offscreen pipeline in offscreen.vt always creates the file called image.png. If you want generate it with a different filename:

<code>python vistrails.py -b ../examples/offscreen.vt:offscreen -a"filename=other.png"</code>

<code>filename</code> in the example above is the alias name assigned to the parameter in the <code>value</code> method inside the String module. When running a pipeline from the command line, VisTrails will try to start the spreadsheet automatically if the pipeline requires it. For example, this other execution will also start the spreadsheet (attention to how $ characters are escaped when running on bash):

<code>python vistrails.py -b ../examples/head.vt:aliases -a"isovalue=30\$&\$diffuse_color=0.8,0.4,0.2"</code>

You can also execute more than one pipeline on the command line:

<code>python vistrails.py -b ../examples/head.vt:aliases ../examples/spx.vt:spx \
-a"isovalue=30"</code>

Use the -a parameter only once regardless the number of pipelines. [http://www.vistrails.org/usersguide/batch.html#running-a-workflow-with-specific-parameters Running a Workflow with Specific Parameters]

=== I can load a vistrail, and the version tree shows up fine. However, no pipelines appear when I click on a version. What gives? ===

The most likely reason is that the vistrail uses a package that is not registered with VisTrails. You need to identify the needed package and add it to your <code>.vistrails/startup.py</code>. A single line like the following should be enough:

addPackage('enter_package_name_here')

Some packages might need more information. For example:

addPackage('afront', executable_path='/path/to/afront')

Refer to the package documentation for details. The one inconvenient step is that currently there's no automated way to describe what is the missing package. We're working on this feature for future releases.

=== I have a workflow that reads a file and then does some processing. The first time it runs, it executes correctly. But in subsequent, nothing happens. ===

VisTrails caches by default, so after a workflow is executed, if none of its parameters change, it won't be executed again.

If a workflow reads a file using the basic module File, VisTrails does check whether the file was modified since the last run. It does so by keeping a signature that is based on the modification time of the file. And if the file was modified, the File module and all downstream modules (the ones which depend on File) will be executed.

''Note: If you would like your input and output data to be versioned, you can use the Persistence package.''

If you do not want VisTrails to cache executions, you can turn off caching: go to Menu Edit -> Preferences and in the General Configuration tab, change Cache execution results to Never. [http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

=== Can VisTrails execute workflows in parallel? ===

The VisTrails server can only execute pipelines in parallel if there's more than one instance of VisTrails running. The command

<code> self.rpcserver = ThreadedXMLRPCServer((self.temp_xml_rpc_options.server, self.temp_xml_rpc_options.port))</code>

starts a multithreaded version of the XML-RPC server, so it will create a thread for each request received by the server. The problem is that Qt/PyQT doesn't allow these multiple threads create GUI objects, only in the main thread. To overcome this limitation, the multithreaded version can instantiate other single threaded versions of VisTrails and put them in a queue, so workflow executions and other GUI-related requests, such as generating workflow graphs and history trees can be forwarded to this queue, and each instance takes turns in answering the request. If the results are in the cache, the multithreaded version answers the requests directly.

Note that this infrastructure works on Linux only. To make this work on Windows, you have to create a script similar to start_vistrails_xvfb.sh (located in the scripts folder) where you can send the number of other instances via command-line options to VisTrails. The command line options are:

python vistrails_server.py -T <ADDRESS> -R <PORT> -O<NUMBER_OF_OTHER_VISTRAILS_INSTANCES> [-M]&

If you want the main vistrails instance to be multithreaded, use the -M at the end.

After creating this script, update function start_other_instances in vistrails/gui/application_server.py lines 1007-1023 and set the script variable to point to your script. You may also have to change the arguments sent to your script (line 1016: for example, you don't need to set a virtual display). You will need to change the path to the stop_vistrails_server.py script (on line 1026) according to your installation path. [http://www.vistrails.org/usersguide/batch.html#executing-workflows-in-parallel Executing Workflows in Parallel]

=== When a workflow is executed, what do the colors mean? ===

- lilac: module was notexecuted

- yellow: module is currently being executed

- green: module was successfully executed

- orange: module was cached

- red: the execution of the module failed

[http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

== Building workflows ==

=== Is there a way to give each widget a "display name" in addition to the module name at the center of the widget? ===

Yes, a "display name" can be assigned to a module by selecting the triangle in its top right corner to open a popup menu and selecting the Set Module Label... menu item. You will then be prompted to enter the "display name". [http://www.vistrails.org/usersguide/creating.html#changing-module-labels Changing Module Labels]

=== Is there a way to re-center the picture-in-picture (PiP) view? ===

Yes. If you click on the PIP window to bring it to focus, you can press Ctrl-R (or Command-R on Mac) to re-center the PiP window. [http://www.vistrails.org/usersguide/getting_started.html#vistrails-interaction Vistrails Interaction]

=== How do I search for a literal "?" (question mark) in the search box in the Property panel? ===

Since we allow regular expressions in our search box, question marks are treated as meta-characters. Thus, searching for "?" returns everything and "abc?" will return everything containing "abc". You need to use "\?" instead to search for "?". So the search for "??" would be "\?\?". [http://www.vistrails.org/usersguide/querying.html#textual-queries Textual Queries]

== Using VisTrails as a server ==

=== What is the VisTrails server-mode? ===

Using the VisTrails server mode, it is possible to ''execute workflows and control VisTrails through another application''. For example, the CrowdLabs Web portal (http://www.crowdlabs.org) accesses a VisTrails sever to execute workflows, retrieve and display vistrail trees and workflows. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

=== How do I execute workflows and control VisTrails through another application? ===

The way you access the server is by doing XML-RPC calls. In the current VisTrails release, we include a set of PHP scripts that can talk to a VisTrails server instance. They are in "extensions/http" folder. The files are reasonably well documented. Also, it should be not difficult to create python scripts to access the server (just use xmlrpclib module).

Note that the VisTrails server requires the provenance and workflows to be in a database. More detailed instructions on how to setup the server and the database
are available here:

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_server_setup.html

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_database_setup.html

If what you want is just to execute a series of workflows in batch mode, a simpler solution would be to ''use the VisTrails client in batch mode''. Chapter 12 of the user's guide contains detailed information and examples on that. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

== Control Flow ==

=== Note: using map ===

When using 'map', the module (or subworkflow) used as function port in the map module MUST be a function, i.e., it can only define 1 output port. [http://www.vistrails.org/usersguide/controlflow.html#the-map-operator The Map Operator]

== Spreadsheet ==

Where ''pipeline'' is a version number or a tag.

=== How can I save an image from the spreadsheet? ===

While having the focus on a spreadsheet cell and select the camera on the toolbar to take a snapshot. The system will prompt you for the location and file name where it should be saved. The other icons can be used for saving multiple images that can be used for generating an animation on demand. A whole sheet can also be saved by selecting Export (either from the menu or from the toolbar). [http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet-image Saving a Spreadsheet Image]

===Is it possible to save the complete state of the spreadsheet?===
[http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet Saving a Spreadsheet]

=== Can I view multiple sheets at the same time? ===

Yes. Each sheet on the spreadsheet can be displayed as a dock widget separated from the main spreadsheet window by dragging its tab name out of the tab bar at the bottom of the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Then, how can I put back a separated sheet? ===

A sheet can be docked back to the main window by dragging it back to the tab bar or double-click on its title bar. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== How can I order sheets on the spreadsheet? ===

This can be done by dragging the sheet name on the bottom top bar and drop it to the right place. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Can I control where a cell will be placed on the spreadsheet window? ===

By default, an unoccupied cell on the active sheet will be chosen to display the result. However, you can specify exactly in the pipeline where a spreadsheet cell will be placed by using CellLocation and SheetReference. CellLocation specifies the location (row and column) of a cell when connecting to a spreadsheet cell (VTKCell, ImageViewerCell, ...). Similarly, a SheetReference module (when connecting to a CellLocation) will specify which sheet the cell will be put on given its name, minimum row size and minimum column size. There is an example of this in examples/vtk.xml (select the version below Double Renderer). [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

=== How do I output results to the spreadsheet? ===

By inspecting the VisTrails Spreadsheet package (in the list of packages, to the left of the pipeline builder), you
can see there are built-in cells for different kinds of data, e.g., RichTextCell to display HTML and plain text. op
You (the user) can also define new cell types to display application-specific data. For example, we have developed
VtkCell, MplFigureCell, and OpenGLCell. It is possible to display pretty much anything on the Spreadsheet! [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

Examples of writing cell modules can be found in:
RichTextCell: packages/spreadsheet/widgets/richtext/richtext.py
VTK: packages/vtk/vtkcell.py

Here is the summary of some requirements on a cell widget:

(1) It must be a Qt widget. It should inherit from spreadsheet_cell.QCellWidget in the spreadsheet package. Although any Qt Widget would work, certain features such as animation will not be available (without rewriting it).

(2) It must re-implement the updateContents() function to take a set of inputs (usually coming from input ports of a wrapper Module) and display on the cells. VisTrails uses this function to update/reuse cells on the spreadsheet when new data comes in.

(3) It needs a wrapper VisTrails Module (inherited from basic_widgets.SpreadsheetCell of the spreadsheet package). Inside the compute() method of this module, it may call self.display(CellWidgetType, (inputs)) to trigger the display event on the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#advanced-cell-options Advanced Cell Options]

=== How do I control the default number of cells in the spreadsheet? ===

You can configure the rowCount and colCount using the preferences dialog. Just go to the Module Packages tab, select spreadsheet in the "Enabled packages" and press the Configure button. Then a list of all the configuration options for the spreadsheet will show up. [http://www.vistrails.org/usersguide/spreadsheet.html#custom-layout-options Custom Layout Options]

=== Is it possible to launch a web browser from the vistrails spreadsheet? We would like to output several urls from a parameter sweep and then have the option to click on each one to view the resulting page. I can view the page within the spreadsheet, but it is really too crowded.===

Currently, there isn't a widget that provides exactly this functionality, but I can think of a few solutions that may work for you:

(1) You can use parameter exploration to generate multiple sheets so you might have an exploration that opens each page in a new sheet. Use the third column/dimension in the exploration interface to have a parameter span sheets.

(2) The spreadsheet is extensible so you can write a custom spreadsheet cell widget that has a button or label with the desired link (a QLabel with openExternalLinks set to True, for example).

(3) You can tweak the existing RichTextCell be adding the line "self.browser.setOpenExternalLinks(True)" at line 63 of the source file "vistrails/packages/spreadsheet/widgets/richtext/richtext.py". Then, if your workflow creates a file with html markup text like "<a href="http://www.vistrails.org">VisTrails</a>" connected to a RichTextCell, clicking on the rendered link in the cell will open it in a web browser. You need to add the aforementioned line to the source to let Qt know that you want the link opened externally; by default, it will just issue an event that isn't processed. [http://www.vistrails.org/usersguide/spreadsheet.html#launching-a-web-browser Launching a Web Browser]

== Integrating your software into VisTrails ==

===How can I integrate my own program into VisTrails?===

The easiest way is to create a package. Writing a package is often very simple, here are instructions on how to do it: [[UsersGuideVisTrailsPackages]]

You can also dynamically generate modules. For an example see:

http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#Packages_that_generate_modules_dynamically
[http://www.vistrails.org/usersguide/packages.html#generating-modules-dynamically Generating Modules Dynamically]

In particular, see the new_module call which uses python's type() function to generate new classes dynamically.

===How do I add a port that is not visible on the module (when it appears on the design canvas)?===

This can be accomplished via the "optional" argument. This is the fourth argument of add_input_port (add_output_port) or can be specified as a kwarg. In your example, this would look like:

reg.add_input_port(MyModule, "MyPort", (core.modules.basic_modules.String, 'MyPort Name'), True)

or with kwargs

reg.add_input_port(MyModule, "MyPort", (core.modules.basic_modules.String, 'MyPort Name'), optional=True)

or

_input_ports = [('MyPort', '(core.modules.basic_modules.String)', {"optional": True})]

=== How do modules deal with multiple inputs in a same port? ===

(And should that even be allowed?)

For compatibility reasons, we do need to allow multiple connections to an input port. However, most package developers should never have to use this, and so we do our best to hide it. the default behavior for getting inputs from a port, then, is to always return a single input.

If on your module you need multiple inputs connected to a single port, use the 'forceGetInputListFromPort' method. It will return a list of all the data items coming through the port. The spreadsheet package uses this feature, so look there for usage examples (vistrails/packages/spreadsheet/basic_widgets.py)
[http://www.vistrails.org/usersguide/packages.html#configuring-ports Configuring Ports]

=== Are there mechanisms for attaching widgets to different modules/parameters? ===

Right now, we have a mechanism for putting a specific widget for an input port. For example, if a port is SetColor(red, green, blue), we can put a color wheel widget there. Or we can also replace the SetFileName port with a File Widget. However, this is not per parameter (only per port). We are currently working on this problem.

=== Can I organize my package so it appears hierarchical in the module palette? ===

Yes. Use the namespace keyword argument when adding the module to the registry. For example,

registry.add_module(MyModule, namespace='MyNamespace')
[http://www.vistrails.org/usersguide/packages.html#configuring-modules Configuring Modules - Hierarchy and Visibility]

=== Can I nest namespaces? ===

Yes. Use the '|' character to separate different the hierarchy. For example,

registry.add_module(MyModule, namespace='ParentNamespace|ChildNamespace')

[http://www.vistrails.org/usersguide/packages.html#configuring-modules Configuring Modules - Hierarchy and Visibility]

=== Are there shortcuts for registry initialization? ===

Yes. If you define _modules as a list of classes in the __init__.py file of your package, VisTrails will attempt to load all classes specified as modules. You can provide add_module options as keyword arguments by specifying a tuple (class, kwargs) in the list. For example:

_modules = [MyModule1, (MyModule2, {'namespace': 'MyNamespace'})]

In addition, you need to identify the ports of your modules as a field in your class by defining _input_ports and _output_ports lists. Here, the items in each list must be tuples of the form (portName, portSignature, optional=False, sort_key=-1). For example:

class MyModule(Module):
def compute(self):
pass

_input_ports = [('firstInput', String), ('secondInput', Integer, True)]
_output_ports = [('firstOutput', String), ('secondOutput', String)]

[http://www.vistrails.org/usersguide/packages.html#customizing-modules-and-ports Customizing Modules and Ports]

=== Can I define ports to be of types that I do not import into my package? ===

Yes. You can pass an identifier string as the portSignature instead. The port_signature string is defined by:

<module_string> := <package_identifier>:[<namespace>|]<module_name>,
<port_signature> := (<module_string>*)

For example,

registry.add_input_port(MyModule, 'myInputPort', '(edu.utah.sci.vistrails.basic:String)')

or

_input_ports = [('myInputPort', '(edu.utah.sci.vistrails.basic:String)')]

[http://www.vistrails.org/usersguide/packages.html#configuring-ports Configuring Ports - Port Types]

=== What do I need to change in my package to make it reloadable (new in v1.4.2)? ===
See [http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#How_to_make_you_package_reloadable UsersGuideVisTrailsPackages] for an explanation.
[http://www.vistrails.org/usersguide/packages.html#creating-reloadable-packages Creating Reloadable Packages]

=== Can I add default values or labels for parameters? ===

Yes. Versions 1.4 and greater support these features. See [[UsersGuideVisTrailsPackages#Adding_default_values_and.2For_labels_for_parameters | UsersGuideVisTrailsPacakges]] for more details.
[http://www.vistrails.org/usersguide/packages.html#configuring-ports Configuring Ports - Default Values and Labels]

=== I want to write a module to load HDF data whose output (e.g., data, string) varies according to the input I give it. Is is possible to do this in VisTrails, and if yes, how can I do that? Ideally, I would like to avoid having to change the connection of my output every time I change the input. ===

There are a few ways to tackle this - each has it's own benefits and pitfalls. Firstly, module connections do respect class hierarchies as we're familiar with in object oriented languages. For instance, A module can output a Constant of which String, Float, Integer, etc are specifications. In this way, you can have a subclass of something like HDFData be passed out of the module and the connections will be established regardless of the sub-type. This is a bit dangerous though. Modules downstream of such a class may not really know how to operate on certain types derived from the super-class. Extreme care must be taken both when creating the modules as well as connecting them to prevent things like this from happening.

A second method that I employ in several different packages is the idea of a container class. For instance, the NumSciPy package uses a relatively generic container "Numpy Array" to encapsulate the data. Of course, these encapsulating objects can store dictionaries that other modules can easily access and understand how to operate on. Although this method is slightly more work, the benefits of a stricter typing of ports is beneficial - particularly upon interfacing with other packages that may depend on strongly typed constants (for example).
[http://www.vistrails.org/usersguide/packages.html#configuring-ports Varying Output According to the Input]

=== I need to determine, at run-time, whether or not a "child" module is attached to the output port of a "parent" module. (I do not specifically need to know which child; just if there is one). ===

The outputPorts dictionary of the base Module stores this information. Thus, you should be able to check

("myPortName" in self.outputPorts)

on the parent module to check if there are any downstream connections from the port "myPortName". This might be used, for example, to only set results for output ports that will be used. ***Note***, however, that the caching algorithm assumes that all outputs are set so adding a new connection to a previously unconnected output port will not work as desired if that module is cached. For this reason, I would currently recommend making such a module not cacheable. Another possibility is overriding the update() method to check the outputPorts and set the upToDate flag if they are not equal. In a single, limited test, this seemed to work, but be warned that it is not fully tested. Here is an example:

class TestModule(Module):
_output_ports = [('a1', '(edu.utah.sci.vistrails.basic:String)'),
('a2', '(edu.utah.sci.vistrails.basic:String)')]
def __init__(self):
Module.__init__(self)
self._cached_output_ports = set()

def update(self):
if len(set(self.outputPorts) - self._cached_output_ports) > 0:
self.upToDate = False
Module.update(self)

def compute(self):
if "a1" in self.outputPorts:
self.setResult("a1", "test")
if "a2" in self.outputPorts:
self.setResult("a2", "test2")
self._cached_output_ports = set(self.outputPorts)

[http://www.vistrails.org/usersguide/packages.html#configuring-ports Configuring Ports - Determining Whether or Not a Module is Attached to an Output Port]

=== How can I make a module not display in the modules list? ===

You should set the abstract parameter to True when adding the module to the registry. Using the original syntax, this looks like:

def initialize():
reg = core.modules.module_registry.get_module_registry()
reg.add_module(InvisibleModule, abstract=True)
# ...

With the _modules dictionary shortcut (for more details, see [http://www.vistrails.org/index.php/FAQ#Are_there_shortucts_for_registry_initialization.3F the FAQ section on this]), you include it in a kwargs dict as part of a module tuple:

_modules = [AnotherModule, (InvisibleModule, {'abstract': True})]

There is also a 'hide_descriptor' parameter that prevents the module from appearing in the module palette without declaring it to be abstract.

The technical difference between the two is that 'abstract' will not add the item to the module palette while 'hide_descriptor' does add the item but immediately hides it. If the module should never be instantiated in a workflow, declare it abstract. If you don't want users to be able to add the module to a pipeline, but you have code that may add it programmatically, declare it with hide_descriptor=True.

[http://www.vistrails.org/usersguide/packages.html#configuring-modules Configuring Modules - Hierarchy and Visibility]

== The Console ==

=== Where should I go to find out what I can call from the console and how to import it? ===

We have tried to make some methods more accessible in the console via an api. You can import the api via <code>import api</code> in the console and see the available methods with <code>dir(api)</code>. To open a vistrail:

import api
api.open_vistrail_from_file('/Applications/VisTrails/examples/terminator.vt')

To execute a version of a workflow, you currently have to go through the controller:

api.select_version('Histogram')
api.get_current_controller().execute_current_workflow()

Currently, only a subset of VisTrails functionality is directly available from the api. However, since VisTrails is written in python, you can dig down starting with the VistrailsApplication or controller object to expose most of our internal methods. If you have suggestions for calls to be added to the api, please let us know.

One other feature that we're working on, but is still in progress is the ability to construct workflows via the console. For example:

vtk = load_package('edu.utah.sci.vistrails.vtk')
vtk.vtkDataSetReader() # adds a vtkDataSetReader module to the pipeline
# click on the new module
a = selected_modules()[0] # get the one currently selected module
a.SetFile('/vistrails/examples/data/head120.vtk') # sets the SetFile parmaeter for the data set reader
b = vtk.vtkContourFilter() # adds a vtkContourFilter module to the pipeline and saves to var b
b.SetInputConnection0(a.GetOutputPort0()) # connects a's GetOutputPort0 port to b's SetInputConnection0
[http://www.vistrails.org/usersguide/batch.html#finding-methods-via-the-command-line Finding Methods Via the Command Line]

== Persistence Package ==

=== How do I use the output of one workflow as the input for another using the persistence package? ===

You need to configure the persistence modules using the module's configuration dialog. After adding a <code>PersistentOutputFile</code> to the workflow, click on the triangle in the upper-right corner of the <code>PersistentOutputFile</code>, and select "Edit Configuration" from the menu that appears. In this dialog, select "Create New Reference" and give the reference a name (and any space-delimited tags). Upon running that workflow, the data will be written to the persistent store. In the second workflow where you wish to use that file, add a <code>PersistentInputFile</code> and go to its configuration dialog in the same manner as with the output file. In that dialog, select "Use Existing Reference" and select the data that you just added in the first workflow in the list of files below. Now, when you run that workflow, it will grab the data from the persistent store.

Here is an example: [[Media:offscreen_persistent.vt | offscreen_persistent.vt]]. Run the "persistent offscreen" workflow first, and then run the "display persistent output" to use the output of the first workflow as the input for the second.

== VTK ==

=== Given a VTK visualization, how can I generate a webpage from it? ===

Check out the html pipeline in offscreen.xml.

=== I'm trying to use VTK, but there doesn't seem to be any output. What is wrong? ===

To use VTK on VisTrails, you need a slightly different way of connecting the renderer modules. Instead of using the standard RenderWindow/RenderWindowInteractor infrastructure, you simply connect the renderer to a VTKCell. The examples directory in the distribution has several VTK examples that illustrate.

=== I am trying to add a module to the workflow via Python, but how can I access vtk modules? ===

Here's an example:

import api

vtvtk = 'edu.utah.sci.vistrails.vtk'

module = api.add_module(0, 0, vtvtk, 'vtkContourFilter', '')

The third argument in add_module is the package identifier. You can find this in the "Module Packages" panel of the Preferences; just click on the package you're interested in and it will appear in the information on the right.

== matplotlib ==

=== I'm experiencing a problem with Latex labels and the matplotlib that comes with VisTrails 1.5. The script below entered to the interpreter that comes with VT is sufficient to reproduce it. ===

import matplotlib.pyplot as plt
plt.plot([1,2,3],[1,2,3])
plt.xlabel("$foo$")

Remove your ~/.matplotlib folder and re-start VisTrails

== VisTrails Development ==

=== I would like to build VisTrails from source. Are there instructions on how to do this? ===

Yes! Take a look at http://www.vistrails.org/index.php/Mac_Intel_Instructions
[http://www.vistrails.org/usersguide/getting_started.html#installing-vistrails-from-source Installing VisTrails from source]

== Accessing Provenance Information ==

=== How do I access the information in the execution log? ===

The code responsible for storing execution information is located in the "core/log" directories, and the code that generates much of that information is in "core/interpreter/cached.py". Modules can add execution-specific annotations to provenance via annotate() calls during execution, but much of the data (like timing and errors) is captured by the LogController and CachedInterpreter (the execution engine) objects. To analyze the log from a vistrail (.vt) file, you might have something like the following:

<code>
import core.log.log
import db.services.io

def run(fname):
# open the .vt bundle specified by the filename "fname"
bundle = db.services.io.open_vistrail_bundle_from_zip_xml(fname)[0]
# get the log filename
log_fname = bundle.vistrail.db_log_filename
if log_fname is not None:
# open the log
log = db.services.io.open_log_from_xml(log_fname, True)
# convert the log from a db object
core.log.log.Log.convert(log)
for workflow_exec in log.workflow_execs:
print 'workflow version:', workflow_exec.parent_version
print 'time started:', workflow_exec.ts_start
print 'time ended:', workflow_exec.ts_end
print 'modules executed:', [i.module_id
for i in workflow_exec.item_execs]
if __name__ == '__main__':
run("some_vistrail.vt")
</code>

You should be able to see what information is available by looking at the "core/log" classes. [http://www.vistrails.org/usersguide/log.html Accessing the Execution Log]

Undocumented Features

2011-05-16T15:07:45Z

Ctuttle:

Features that should be added to the VisTrails Users' Guide

* Many items in the [http://www.vistrails.org/index.php/FAQ FAQ] .
* New syntax for specifying modules in packages (<code>_input_ports</code>, <code>_output_ports</code>, <code>_modules</code>).
* Shortcuts for specifying ports (default to current package, allow 'edu.utah.sci.vistrails.basic' to shorten to 'basic').
* Manipulate workflows via the shell (<code>load_package</code> and then add modules, connections, and parameters)
* Better docs on groups/subworkflows and some integration with control flow
* Publication tools (embed in LaTeX, PowerPoint)
* crowdLabs publishing
* Merge documentation (Tommy's code to merge vistrails).
* Persistence package documentation and general tips on data management with VisTrails
* Vistrail/workflow browser documentation

* ...

Undocumented Features

2011-05-11T17:02:41Z

Ctuttle:

Features that should be added to the VisTrails Users' Guide

* Many items in the [http://www.vistrails.org/index.php/FAQ FAQ] .
* New syntax for specifying modules in packages (<code>_input_ports</code>, <code>_output_ports</code>, <code>_modules</code>).
* Shortcuts for specifying ports (default to current package, allow 'edu.utah.sci.vistrails.basic' to shorten to 'basic').
* Manipulate workflows via the shell (<code>load_package</code> and then add modules, connections, and parameters)
* Better docs on groups/subworkflows and some integration with control flow
* Publication tools (embed in LaTeX, PowerPoint)
* crowdLabs publishing
* Merge documentation (Tommy's code to merge vistrails)
* Persistence package documentation and general tips on data management with VisTrails
* Vistrail/workflow browser documentation

* ...

FAQ

2011-04-06T17:48:41Z

Ctuttle: /* Integrating your software into VisTrails */

Also check our [[known_issues|Known Issues]] page for troubleshooting.

----
== Running workflows ==

=== How can I run a workflow using the command line? ===

('''Updated for version 1.2''') Call vistrails using the following options:

<code>python vistrails.py -b path_to_vistrails_file:pipeline </code>

where pipeline can be a version '''tag name''' or version '''id'''

'''NOTE:''' If you downloaded the MacOS X bundle, you can run vistrails from the command line via the following commands in the Terminal. Change the current directory to wherever VisTrails was installed (often <code>/Applications</code>), and then type:

<code>Vistrails.app/Contents/MacOS/vistrails [<cmd_line_options>]</code> [http://www.vistrails.org/usersguide/batch.html#running-a-specific-workflow-in-batch-mode Running a Specific Workflow in Batch Mode]

=== Using the command line, we'd like to execute a workflow multiple times, with slightly different parameters, and create a series of output files. Is this possible?===

('''Updated for version 1.2''') We can change parameters that have an alias through the command line.

For example, offscreen pipeline in offscreen.vt always creates the file called image.png. If you want generate it with a different filename:

<code>python vistrails.py -b ../examples/offscreen.vt:offscreen -a"filename=other.png"</code>

<code>filename</code> in the example above is the alias name assigned to the parameter in the <code>value</code> method inside the String module. When running a pipeline from the command line, VisTrails will try to start the spreadsheet automatically if the pipeline requires it. For example, this other execution will also start the spreadsheet (attention to how $ characters are escaped when running on bash):

<code>python vistrails.py -b ../examples/head.vt:aliases -a"isovalue=30\$&\$diffuse_color=0.8,0.4,0.2"</code>

You can also execute more than one pipeline on the command line:

<code>python vistrails.py -b ../examples/head.vt:aliases ../examples/spx.vt:spx \
-a"isovalue=30"</code>

Use the -a parameter only once regardless the number of pipelines. [http://www.vistrails.org/usersguide/batch.html#running-a-workflow-with-specific-parameters Running a Workflow with Specific Parameters]

=== I can load a vistrail, and the version tree shows up fine. However, no pipelines appear when I click on a version. What gives? ===

The most likely reason is that the vistrail uses a package that is not registered with VisTrails. You need to identify the needed package and add it to your <code>.vistrails/startup.py</code>. A single line like the following should be enough:

addPackage('enter_package_name_here')

Some packages might need more information. For example:

addPackage('afront', executable_path='/path/to/afront')

Refer to the package documentation for details. The one inconvenient step is that currently there's no automated way to describe what is the missing package. We're working on this feature for future releases.

=== I have a workflow that reads a file and then does some processing. The first time it runs, it executes correctly. But in subsequent, nothing happens. ===

VisTrails caches by default, so after a workflow is executed, if none of its parameters change, it won't be executed again.

If a workflow reads a file using the basic module File, VisTrails does check whether the file was modified since the last run. It does so by keeping a signature that is based on the modification time of the file. And if the file was modified, the File module and all downstream modules (the ones which depend on File) will be executed.

''Note: If you would like your input and output data to be versioned, you can use the Persistence package.''

If you do not want VisTrails to cache executions, you can turn off caching: go to Menu Edit -> Preferences and in the General Configuration tab, change Cache execution results to Never. [http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

=== Can VisTrails execute workflows in parallel? ===

The VisTrails server can only execute pipelines in parallel if there's more than one instance of VisTrails running. The command

<code> self.rpcserver = ThreadedXMLRPCServer((self.temp_xml_rpc_options.server, self.temp_xml_rpc_options.port))</code>

starts a multithreaded version of the XML-RPC server, so it will create a thread for each request received by the server. The problem is that Qt/PyQT doesn't allow these multiple threads create GUI objects, only in the main thread. To overcome this limitation, the multithreaded version can instantiate other single threaded versions of VisTrails and put them in a queue, so workflow executions and other GUI-related requests, such as generating workflow graphs and history trees can be forwarded to this queue, and each instance takes turns in answering the request. If the results are in the cache, the multithreaded version answers the requests directly.

Note that this infrastructure works on Linux only. To make this work on Windows, you have to create a script similar to start_vistrails_xvfb.sh (located in the scripts folder) where you can send the number of other instances via command-line options to VisTrails. The command line options are:

python vistrails_server.py -T <ADDRESS> -R <PORT> -O<NUMBER_OF_OTHER_VISTRAILS_INSTANCES> [-M]&

If you want the main vistrails instance to be multithreaded, use the -M at the end.

After creating this script, update function start_other_instances in vistrails/gui/application_server.py lines 1007-1023 and set the script variable to point to your script. You may also have to change the arguments sent to your script (line 1016: for example, you don't need to set a virtual display). You will need to change the path to the stop_vistrails_server.py script (on line 1026) according to your installation path. [http://www.vistrails.org/usersguide/batch.html#executing-workflows-in-parallel Executing Workflows in Parallel]

=== When a workflow is executed, what do the colors mean? ===

- lilac: module was notexecuted

- yellow: module is currently being executed

- green: module was successfully executed

- orange: module was cached

- red: the execution of the module failed

[http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

== Building workflows ==

=== Is there a way to give each widget a "display name" in addition to the module name at the center of the widget? ===

Yes, a "display name" can be assigned to a module by selecting the triangle in its top right corner to open a popup menu and selecting the Set Module Label... menu item. You will then be prompted to enter the "display name". [http://www.vistrails.org/usersguide/creating.html#changing-module-labels Changing Module Labels]

=== Is there a way to re-center the picture-in-picture (PiP) view? ===

Yes. If you click on the PIP window to bring it to focus, you can press Ctrl-R (or Command-R on Mac) to re-center the PiP window. [http://www.vistrails.org/usersguide/getting_started.html#vistrails-interaction Vistrails Interaction]

=== How do I search for a literal "?" (question mark) in the search box in the Property panel? ===

Since we allow regular expressions in our search box, question marks are treated as meta-characters. Thus, searching for "?" returns everything and "abc?" will return everything containing "abc". You need to use "\?" instead to search for "?". So the search for "??" would be "\?\?". [http://www.vistrails.org/usersguide/querying.html#textual-queries Textual Queries]

== Using VisTrails as a server ==

=== What is the VisTrails server-mode? ===

Using the VisTrails server mode, it is possible to ''execute workflows and control VisTrails through another application''. For example, the CrowdLabs Web portal (http://www.crowdlabs.org) accesses a VisTrails sever to execute workflows, retrieve and display vistrail trees and workflows. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

=== How do I execute workflows and control VisTrails through another application? ===

The way you access the server is by doing XML-RPC calls. In the current VisTrails release, we include a set of PHP scripts that can talk to a VisTrails server instance. They are in "extensions/http" folder. The files are reasonably well documented. Also, it should be not difficult to create python scripts to access the server (just use xmlrpclib module).

Note that the VisTrails server requires the provenance and workflows to be in a database. More detailed instructions on how to setup the server and the database
are available here:

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_server_setup.html

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_database_setup.html

If what you want is just to execute a series of workflows in batch mode, a simpler solution would be to ''use the VisTrails client in batch mode''. Chapter 12 of the user's guide contains detailed information and examples on that. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

== Control Flow ==

=== Note: using map ===

When using 'map', the module (or subworkflow) used as function port in the map module MUST be a function, i.e., it can only define 1 output port. [http://www.vistrails.org/usersguide/controlflow.html#the-map-operator The Map Operator]

== Spreadsheet ==

Where ''pipeline'' is a version number or a tag.

=== How can I save an image from the spreadsheet? ===

While having the focus on a spreadsheet cell and select the camera on the toolbar to take a snapshot. The system will prompt you for the location and file name where it should be saved. The other icons can be used for saving multiple images that can be used for generating an animation on demand. A whole sheet can also be saved by selecting Export (either from the menu or from the toolbar). [http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet-image Saving a Spreadsheet Image]

===Is it possible to save the complete state of the spreadsheet?===
[http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet Saving a Spreadsheet]

=== Can I view multiple sheets at the same time? ===

Yes. Each sheet on the spreadsheet can be displayed as a dock widget separated from the main spreadsheet window by dragging its tab name out of the tab bar at the bottom of the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Then, how can I put back a separated sheet? ===

A sheet can be docked back to the main window by dragging it back to the tab bar or double-click on its title bar. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== How can I order sheets on the spreadsheet? ===

This can be done by dragging the sheet name on the bottom top bar and drop it to the right place. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Can I control where a cell will be placed on the spreadsheet window? ===

By default, an unoccupied cell on the active sheet will be chosen to display the result. However, you can specify exactly in the pipeline where a spreadsheet cell will be placed by using CellLocation and SheetReference. CellLocation specifies the location (row and column) of a cell when connecting to a spreadsheet cell (VTKCell, ImageViewerCell, ...). Similarly, a SheetReference module (when connecting to a CellLocation) will specify which sheet the cell will be put on given its name, minimum row size and minimum column size. There is an example of this in examples/vtk.xml (select the version below Double Renderer). [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

=== How do I output results to the spreadsheet? ===

By inspecting the VisTrails Spreadsheet package (in the list of packages, to the left of the pipeline builder), you
can see there are built-in cells for different kinds of data, e.g., RichTextCell to display HTML and plain text. op
You (the user) can also define new cell types to display application-specific data. For example, we have developed
VtkCell, MplFigureCell, and OpenGLCell. It is possible to display pretty much anything on the Spreadsheet! [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

Examples of writing cell modules can be found in:
RichTextCell: packages/spreadsheet/widgets/richtext/richtext.py
VTK: packages/vtk/vtkcell.py

Here is the summary of some requirements on a cell widget:

(1) It must be a Qt widget. It should inherit from spreadsheet_cell.QCellWidget in the spreadsheet package. Although any Qt Widget would work, certain features such as animation will not be available (without rewriting it).

(2) It must re-implement the updateContents() function to take a set of inputs (usually coming from input ports of a wrapper Module) and display on the cells. VisTrails uses this function to update/reuse cells on the spreadsheet when new data comes in.

(3) It needs a wrapper VisTrails Module (inherited from basic_widgets.SpreadsheetCell of the spreadsheet package). Inside the compute() method of this module, it may call self.display(CellWidgetType, (inputs)) to trigger the display event on the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#advanced-cell-options Advanced Cell Options]

=== How do I control the default number of cells in the spreadsheet? ===

You can configure the rowCount and colCount using the preferences dialog. Just go to the Module Packages tab, select spreadsheet in the "Enabled packages" and press the Configure button. Then a list of all the configuration options for the spreadsheet will show up. [http://www.vistrails.org/usersguide/spreadsheet.html#custom-layout-options Custom Layout Options]

=== Is it possible to launch a web browser from the vistrails spreadsheet? We would like to output several urls from a parameter sweep and then have the option to click on each one to view the resulting page. I can view the page within the spreadsheet, but it is really too crowded.===

Currently, there isn't a widget that provides exactly this functionality, but I can think of a few solutions that may work for you:

(1) You can use parameter exploration to generate multiple sheets so you might have an exploration that opens each page in a new sheet. Use the third column/dimension in the exploration interface to have a parameter span sheets.

(2) The spreadsheet is extensible so you can write a custom spreadsheet cell widget that has a button or label with the desired link (a QLabel with openExternalLinks set to True, for example).

(3) You can tweak the existing RichTextCell be adding the line "self.browser.setOpenExternalLinks(True)" at line 63 of the source file "vistrails/packages/spreadsheet/widgets/richtext/richtext.py". Then, if your workflow creates a file with html markup text like "<a href="http://www.vistrails.org">VisTrails</a>" connected to a RichTextCell, clicking on the rendered link in the cell will open it in a web browser. You need to add the aforementioned line to the source to let Qt know that you want the link opened externally; by default, it will just issue an event that isn't processed. [http://www.vistrails.org/usersguide/spreadsheet.html#launching-a-web-browser Launching a Web Browser]

== Integrating your software into VisTrails ==

===How can I integrate my own program into VisTrails?===

The easiest way is to create a package. Writing a package is often very simple, here are instructions on how to do it: [[UsersGuideVisTrailsPackages]]

You can also dynamically generate modules. For an example see:

http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#Packages_that_generate_modules_dynamically
[http://www.vistrails.org/usersguide/packages.html#generating-modules-dynamically Generating Modules Dynamically]

In particular, see the new_module call which uses python's type() function to generate new classes dynamically.

=== How do modules deal with multiple inputs in a same port? ===

(And should that even be allowed?)

For compatibility reasons, we do need to allow multiple connections to an input port. However, most package developers should never have to use this, and so we do our best to hide it. the default behavior for getting inputs from a port, then, is to always return a single input.

If on your module you need multiple inputs connected to a single port, use the 'forceGetInputListFromPort' method. It will return a list of all the data items coming through the port. The spreadsheet package uses this feature, so look there for usage examples (vistrails/packages/spreadsheet/basic_widgets.py)
[http://www.vistrails.org/usersguide/packages.html#configuring-ports Configuring Ports]

=== Are there mechanisms for attaching widgets to different modules/parameters? ===

Right now, we have a mechanism for putting a specific widget for an input port. For example, if a port is SetColor(red, green, blue), we can put a color wheel widget there. Or we can also replace the SetFileName port with a File Widget. However, this is not per parameter (only per port). We are currently working on this problem.

=== Can I organize my package so it appears hierarchical in the module palette? ===

Yes. Use the namespace keyword argument when adding the module to the registry. For example,

registry.add_module(MyModule, namespace='MyNamespace')
[http://www.vistrails.org/usersguide/packages.html#configuring-modules Configuring Modules - Hierarchy and Visibility]

=== Can I nest namespaces? ===

Yes. Use the '|' character to separate different the hierarchy. For example,

registry.add_module(MyModule, namespace='ParentNamespace|ChildNamespace')

[http://www.vistrails.org/usersguide/packages.html#configuring-modules Configuring Modules - Hierarchy and Visibility]

=== Are there shortcuts for registry initialization? ===

Yes. If you define _modules as a list of classes in the __init__.py file of your package, VisTrails will attempt to load all classes specified as modules. You can provide add_module options as keyword arguments by specifying a tuple (class, kwargs) in the list. For example:

_modules = [MyModule1, (MyModule2, {'namespace': 'MyNamespace'})]

In addition, you need to identify the ports of your modules as a field in your class by defining _input_ports and _output_ports lists. Here, the items in each list must be tuples of the form (portName, portSignature, optional=False, sort_key=-1). For example:

class MyModule(Module):
def compute(self):
pass

_input_ports = [('firstInput', String), ('secondInput', Integer, True)]
_output_ports = [('firstOutput', String), ('secondOutput', String)]

[http://www.vistrails.org/usersguide/packages.html#customizing-modules-and-ports Customizing Modules and Ports]

=== Can I define ports to be of types that I do not import into my package? ===

Yes. You can pass an identifier string as the portSignature instead. The port_signature string is defined by:

<module_string> := <package_identifier>:[<namespace>|]<module_name>,
<port_signature> := (<module_string>*)

For example,

registry.add_input_port(MyModule, 'myInputPort', '(edu.utah.sci.vistrails.basic:String)')

or

_input_ports = [('myInputPort', '(edu.utah.sci.vistrails.basic:String)')]

[http://www.vistrails.org/usersguide/packages.html#configuring-ports Configuring Ports - Port Types]

=== What do I need to change in my package to make it reloadable (new in v1.4.2)? ===
See [http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#How_to_make_you_package_reloadable UsersGuideVisTrailsPackages] for an explanation.
[http://www.vistrails.org/usersguide/packages.html#creating-reloadable-packages Creating Reloadable Packages]

=== Can I add default values or labels for parameters? ===

Yes. Versions 1.4 and greater support these features. See [[UsersGuideVisTrailsPackages#Adding_default_values_and.2For_labels_for_parameters | UsersGuideVisTrailsPacakges]] for more details.
[http://www.vistrails.org/usersguide/packages.html#configuring-ports Configuring Ports - Default Values and Labels]

=== I want to write a module to load HDF data whose output (e.g., data, string) varies according to the input I give it. Is is possible to do this in VisTrails, and if yes, how can I do that? Ideally, I would like to avoid having to change the connection of my output every time I change the input. ===

There are a few ways to tackle this - each has it's own benefits and pitfalls. Firstly, module connections do respect class hierarchies as we're familiar with in object oriented languages. For instance, A module can output a Constant of which String, Float, Integer, etc are specifications. In this way, you can have a subclass of something like HDFData be passed out of the module and the connections will be established regardless of the sub-type. This is a bit dangerous though. Modules downstream of such a class may not really know how to operate on certain types derived from the super-class. Extreme care must be taken both when creating the modules as well as connecting them to prevent things like this from happening.

A second method that I employ in several different packages is the idea of a container class. For instance, the NumSciPy package uses a relatively generic container "Numpy Array" to encapsulate the data. Of course, these encapsulating objects can store dictionaries that other modules can easily access and understand how to operate on. Although this method is slightly more work, the benefits of a stricter typing of ports is beneficial - particularly upon interfacing with other packages that may depend on strongly typed constants (for example).
[http://www.vistrails.org/usersguide/packages.html#configuring-ports Varying Output According to the Input]

=== I need to determine, at run-time, whether or not a "child" module is attached to the output port of a "parent" module. (I do not specifically need to know which child; just if there is one). ===

The outputPorts dictionary of the base Module stores this information. Thus, you should be able to check

("myPortName" in self.outputPorts)

on the parent module to check if there are any downstream connections from the port "myPortName". This might be used, for example, to only set results for output ports that will be used. ***Note***, however, that the caching algorithm assumes that all outputs are set so adding a new connection to a previously unconnected output port will not work as desired if that module is cached. For this reason, I would currently recommend making such a module not cacheable. Another possibility is overriding the update() method to check the outputPorts and set the upToDate flag if they are not equal. In a single, limited test, this seemed to work, but be warned that it is not fully tested. Here is an example:

class TestModule(Module):
_output_ports = [('a1', '(edu.utah.sci.vistrails.basic:String)'),
('a2', '(edu.utah.sci.vistrails.basic:String)')]
def __init__(self):
Module.__init__(self)
self._cached_output_ports = set()

def update(self):
if len(set(self.outputPorts) - self._cached_output_ports) > 0:
self.upToDate = False
Module.update(self)

def compute(self):
if "a1" in self.outputPorts:
self.setResult("a1", "test")
if "a2" in self.outputPorts:
self.setResult("a2", "test2")
self._cached_output_ports = set(self.outputPorts)

[http://www.vistrails.org/usersguide/packages.html#configuring-ports Configuring Ports - Determining Whether or Not a Module is Attached to an Output Port]

=== How can I make a module not display in the modules list? ===

You should set the abstract parameter to True when adding the module to the registry. Using the original syntax, this looks like:

def initialize():
reg = core.modules.module_registry.get_module_registry()
reg.add_module(InvisibleModule, abstract=True)
# ...

With the _modules dictionary shortcut (for more details, see [http://www.vistrails.org/index.php/FAQ#Are_there_shortucts_for_registry_initialization.3F the FAQ section on this]), you include it in a kwargs dict as part of a module tuple:

_modules = [AnotherModule, (InvisibleModule, {'abstract': True})]

There is also a 'hide_descriptor' parameter that prevents the module from appearing in the module palette without declaring it to be abstract.

The technical difference between the two is that 'abstract' will not add the item to the module palette while 'hide_descriptor' does add the item but immediately hides it. If the module should never be instantiated in a workflow, declare it abstract. If you don't want users to be able to add the module to a pipeline, but you have code that may add it programmatically, declare it with hide_descriptor=True.

[http://www.vistrails.org/usersguide/packages.html#configuring-modules Configuring Modules - Hierarchy and Visibility]

== The Console ==

=== Where should I go to find out what I can call from the console and how to import it? ===

We have tried to make some methods more accessible in the console via an api. You can import the api via <code>import api</code> in the console and see the available methods with <code>dir(api)</code>. To open a vistrail:

import api
api.open_vistrail_from_file('/Applications/VisTrails/examples/terminator.vt')

To execute a version of a workflow, you currently have to go through the controller:

api.select_version('Histogram')
api.get_current_controller().execute_current_workflow()

Currently, only a subset of VisTrails functionality is directly available from the api. However, since VisTrails is written in python, you can dig down starting with the VistrailsApplication or controller object to expose most of our internal methods. If you have suggestions for calls to be added to the api, please let us know.

One other feature that we're working on, but is still in progress is the ability to construct workflows via the console. For example:

vtk = load_package('edu.utah.sci.vistrails.vtk')
vtk.vtkDataSetReader() # adds a vtkDataSetReader module to the pipeline
# click on the new module
a = selected_modules()[0] # get the one currently selected module
a.SetFile('/vistrails/examples/data/head120.vtk') # sets the SetFile parmaeter for the data set reader
b = vtk.vtkContourFilter() # adds a vtkContourFilter module to the pipeline and saves to var b
b.SetInputConnection0(a.GetOutputPort0()) # connects a's GetOutputPort0 port to b's SetInputConnection0
[http://www.vistrails.org/usersguide/batch.html#finding-methods-via-the-command-line Finding Methods Via the Command Line]

== Persistence Package ==

=== How do I use the output of one workflow as the input for another using the persistence package? ===

You need to configure the persistence modules using the module's configuration dialog. After adding a <code>PersistentOutputFile</code> to the workflow, click on the triangle in the upper-right corner of the <code>PersistentOutputFile</code>, and select "Edit Configuration" from the menu that appears. In this dialog, select "Create New Reference" and give the reference a name (and any space-delimited tags). Upon running that workflow, the data will be written to the persistent store. In the second workflow where you wish to use that file, add a <code>PersistentInputFile</code> and go to its configuration dialog in the same manner as with the output file. In that dialog, select "Use Existing Reference" and select the data that you just added in the first workflow in the list of files below. Now, when you run that workflow, it will grab the data from the persistent store.

Here is an example: [[Media:offscreen_persistent.vt | offscreen_persistent.vt]]. Run the "persistent offscreen" workflow first, and then run the "display persistent output" to use the output of the first workflow as the input for the second.

== VTK ==

=== Given a VTK visualization, how can I generate a webpage from it? ===

Check out the html pipeline in offscreen.xml.

=== I'm trying to use VTK, but there doesn't seem to be any output. What is wrong? ===

To use VTK on VisTrails, you need a slightly different way of connecting the renderer modules. Instead of using the standard RenderWindow/RenderWindowInteractor infrastructure, you simply connect the renderer to a VTKCell. The examples directory in the distribution has several VTK examples that illustrate.

=== I am trying to add a module to the workflow via Python, but how can I access vtk modules? ===

Here's an example:

import api

vtvtk = 'edu.utah.sci.vistrails.vtk'

module = api.add_module(0, 0, vtvtk, 'vtkContourFilter', '')

The third argument in add_module is the package identifier. You can find this in the "Module Packages" panel of the Preferences; just click on the package you're interested in and it will appear in the information on the right.

== matplotlib ==

=== I'm experiencing a problem with Latex labels and the matplotlib that comes with VisTrails 1.5. The script below entered to the interpreter that comes with VT is sufficient to reproduce it. ===

import matplotlib.pyplot as plt
plt.plot([1,2,3],[1,2,3])
plt.xlabel("$foo$")

Remove your ~/.matplotlib folder and re-start VisTrails

== VisTrails Development ==

=== I would like to build VisTrails from source. Are there instructions on how to do this? ===

Yes! Take a look at http://www.vistrails.org/index.php/Mac_Intel_Instructions
[http://www.vistrails.org/usersguide/getting_started.html#installing-vistrails-from-source Installing VisTrails from source]

== Accessing Provenance Information ==

=== How do I access the information in the execution log? ===

The code responsible for storing execution information is located in the "core/log" directories, and the code that generates much of that information is in "core/interpreter/cached.py". Modules can add execution-specific annotations to provenance via annotate() calls during execution, but much of the data (like timing and errors) is captured by the LogController and CachedInterpreter (the execution engine) objects. To analyze the log from a vistrail (.vt) file, you might have something like the following:

<code>
import core.log.log
import db.services.io

def run(fname):
# open the .vt bundle specified by the filename "fname"
bundle = db.services.io.open_vistrail_bundle_from_zip_xml(fname)[0]
# get the log filename
log_fname = bundle.vistrail.db_log_filename
if log_fname is not None:
# open the log
log = db.services.io.open_log_from_xml(log_fname, True)
# convert the log from a db object
core.log.log.Log.convert(log)
for workflow_exec in log.workflow_execs:
print 'workflow version:', workflow_exec.parent_version
print 'time started:', workflow_exec.ts_start
print 'time ended:', workflow_exec.ts_end
print 'modules executed:', [i.module_id
for i in workflow_exec.item_execs]
if __name__ == '__main__':
run("some_vistrail.vt")
</code>

You should be able to see what information is available by looking at the "core/log" classes. [http://www.vistrails.org/usersguide/log.html Accessing the Execution Log]

Documentation Log

2011-04-06T05:26:16Z

Ctuttle: /* Ideas */

Link to undocumented features:
http://www.vistrails.org/index.php/Undocumented_Features

===Ideas===
*Separate the developer's guide from the user's guide.
*Create a tutorial for getting started quickly.

===Questions===
Is registry.addInputPort(Divide, "arg1", (basic.Float, 'dividend')) equivalent to:
_input_ports = [('arg1', '(edu.utah.sci.vistrails.basic:Float)', {"labels": str(["dividend"])})]?

FAQ

2011-04-01T00:33:24Z

Ctuttle: /* Where should I go to find out what I can call from the console and how to import it? */

Also check our [[known_issues|Known Issues]] page for troubleshooting.

----
== Running workflows ==

=== How can I run a workflow using the command line? ===

('''Updated for version 1.2''') Call vistrails using the following options:

<code>python vistrails.py -b path_to_vistrails_file:pipeline </code>

where pipeline can be a version '''tag name''' or version '''id'''

'''NOTE:''' If you downloaded the MacOS X bundle, you can run vistrails from the command line via the following commands in the Terminal. Change the current directory to wherever VisTrails was installed (often <code>/Applications</code>), and then type:

<code>Vistrails.app/Contents/MacOS/vistrails [<cmd_line_options>]</code> [http://www.vistrails.org/usersguide/batch.html#running-a-specific-workflow-in-batch-mode Running a Specific Workflow in Batch Mode]

=== Using the command line, we'd like to execute a workflow multiple times, with slightly different parameters, and create a series of output files. Is this possible?===

('''Updated for version 1.2''') We can change parameters that have an alias through the command line.

For example, offscreen pipeline in offscreen.vt always creates the file called image.png. If you want generate it with a different filename:

<code>python vistrails.py -b ../examples/offscreen.vt:offscreen -a"filename=other.png"</code>

<code>filename</code> in the example above is the alias name assigned to the parameter in the <code>value</code> method inside the String module. When running a pipeline from the command line, VisTrails will try to start the spreadsheet automatically if the pipeline requires it. For example, this other execution will also start the spreadsheet (attention to how $ characters are escaped when running on bash):

<code>python vistrails.py -b ../examples/head.vt:aliases -a"isovalue=30\$&\$diffuse_color=0.8,0.4,0.2"</code>

You can also execute more than one pipeline on the command line:

<code>python vistrails.py -b ../examples/head.vt:aliases ../examples/spx.vt:spx \
-a"isovalue=30"</code>

Use the -a parameter only once regardless the number of pipelines. [http://www.vistrails.org/usersguide/batch.html#running-a-workflow-with-specific-parameters Running a Workflow with Specific Parameters]

=== I can load a vistrail, and the version tree shows up fine. However, no pipelines appear when I click on a version. What gives? ===

The most likely reason is that the vistrail uses a package that is not registered with VisTrails. You need to identify the needed package and add it to your <code>.vistrails/startup.py</code>. A single line like the following should be enough:

addPackage('enter_package_name_here')

Some packages might need more information. For example:

addPackage('afront', executable_path='/path/to/afront')

Refer to the package documentation for details. The one inconvenient step is that currently there's no automated way to describe what is the missing package. We're working on this feature for future releases.

=== I have a workflow that reads a file and then does some processing. The first time it runs, it executes correctly. But in subsequent, nothing happens. ===

VisTrails caches by default, so after a workflow is executed, if none of its parameters change, it won't be executed again.

If a workflow reads a file using the basic module File, VisTrails does check whether the file was modified since the last run. It does so by keeping a signature that is based on the modification time of the file. And if the file was modified, the File module and all downstream modules (the ones which depend on File) will be executed.

''Note: If you would like your input and output data to be versioned, you can use the Persistence package.''

If you do not want VisTrails to cache executions, you can turn off caching: go to Menu Edit -> Preferences and in the General Configuration tab, change Cache execution results to Never. [http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

=== Can VisTrails execute workflows in parallel? ===

The VisTrails server can only execute pipelines in parallel if there's more than one instance of VisTrails running. The command

<code> self.rpcserver = ThreadedXMLRPCServer((self.temp_xml_rpc_options.server, self.temp_xml_rpc_options.port))</code>

starts a multithreaded version of the XML-RPC server, so it will create a thread for each request received by the server. The problem is that Qt/PyQT doesn't allow these multiple threads create GUI objects, only in the main thread. To overcome this limitation, the multithreaded version can instantiate other single threaded versions of VisTrails and put them in a queue, so workflow executions and other GUI-related requests, such as generating workflow graphs and history trees can be forwarded to this queue, and each instance takes turns in answering the request. If the results are in the cache, the multithreaded version answers the requests directly.

Note that this infrastructure works on Linux only. To make this work on Windows, you have to create a script similar to start_vistrails_xvfb.sh (located in the scripts folder) where you can send the number of other instances via command-line options to VisTrails. The command line options are:

python vistrails_server.py -T <ADDRESS> -R <PORT> -O<NUMBER_OF_OTHER_VISTRAILS_INSTANCES> [-M]&

If you want the main vistrails instance to be multithreaded, use the -M at the end.

After creating this script, update function start_other_instances in vistrails/gui/application_server.py lines 1007-1023 and set the script variable to point to your script. You may also have to change the arguments sent to your script (line 1016: for example, you don't need to set a virtual display). You will need to change the path to the stop_vistrails_server.py script (on line 1026) according to your installation path. [http://www.vistrails.org/usersguide/batch.html#executing-workflows-in-parallel Executing Workflows in Parallel]

=== When a workflow is executed, what do the colors mean? ===

- lilac: module was notexecuted

- yellow: module is currently being executed

- green: module was successfully executed

- orange: module was cached

- red: the execution of the module failed

[http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

== Building workflows ==

=== Is there a way to give each widget a "display name" in addition to the module name at the center of the widget? ===

Yes, a "display name" can be assigned to a module by selecting the triangle in its top right corner to open a popup menu and selecting the Set Module Label... menu item. You will then be prompted to enter the "display name". [http://www.vistrails.org/usersguide/creating.html#changing-module-labels Changing Module Labels]

=== Is there a way to re-center the picture-in-picture (PiP) view? ===

Yes. If you click on the PIP window to bring it to focus, you can press Ctrl-R (or Command-R on Mac) to re-center the PiP window. [http://www.vistrails.org/usersguide/getting_started.html#vistrails-interaction Vistrails Interaction]

=== How do I search for a literal "?" (question mark) in the search box in the Property panel? ===

Since we allow regular expressions in our search box, question marks are treated as meta-characters. Thus, searching for "?" returns everything and "abc?" will return everything containing "abc". You need to use "\?" instead to search for "?". So the search for "??" would be "\?\?". [http://www.vistrails.org/usersguide/querying.html#textual-queries Textual Queries]

== Using VisTrails as a server ==

=== What is the VisTrails server-mode? ===

Using the VisTrails server mode, it is possible to ''execute workflows and control VisTrails through another application''. For example, the CrowdLabs Web portal (http://www.crowdlabs.org) accesses a VisTrails sever to execute workflows, retrieve and display vistrail trees and workflows. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

=== How do I execute workflows and control VisTrails through another application? ===

The way you access the server is by doing XML-RPC calls. In the current VisTrails release, we include a set of PHP scripts that can talk to a VisTrails server instance. They are in "extensions/http" folder. The files are reasonably well documented. Also, it should be not difficult to create python scripts to access the server (just use xmlrpclib module).

Note that the VisTrails server requires the provenance and workflows to be in a database. More detailed instructions on how to setup the server and the database
are available here:

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_server_setup.html

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_database_setup.html

If what you want is just to execute a series of workflows in batch mode, a simpler solution would be to ''use the VisTrails client in batch mode''. Chapter 12 of the user's guide contains detailed information and examples on that. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

== Control Flow ==

=== Note: using map ===

When using 'map', the module (or subworkflow) used as function port in the map module MUST be a function, i.e., it can only define 1 output port. [http://www.vistrails.org/usersguide/controlflow.html#the-map-operator The Map Operator]

== Spreadsheet ==

Where ''pipeline'' is a version number or a tag.

=== How can I save an image from the spreadsheet? ===

While having the focus on a spreadsheet cell and select the camera on the toolbar to take a snapshot. The system will prompt you for the location and file name where it should be saved. The other icons can be used for saving multiple images that can be used for generating an animation on demand. A whole sheet can also be saved by selecting Export (either from the menu or from the toolbar). [http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet-image Saving a Spreadsheet Image]

===Is it possible to save the complete state of the spreadsheet?===
[http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet Saving a Spreadsheet]

=== Can I view multiple sheets at the same time? ===

Yes. Each sheet on the spreadsheet can be displayed as a dock widget separated from the main spreadsheet window by dragging its tab name out of the tab bar at the bottom of the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Then, how can I put back a separated sheet? ===

A sheet can be docked back to the main window by dragging it back to the tab bar or double-click on its title bar. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== How can I order sheets on the spreadsheet? ===

This can be done by dragging the sheet name on the bottom top bar and drop it to the right place. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Can I control where a cell will be placed on the spreadsheet window? ===

By default, an unoccupied cell on the active sheet will be chosen to display the result. However, you can specify exactly in the pipeline where a spreadsheet cell will be placed by using CellLocation and SheetReference. CellLocation specifies the location (row and column) of a cell when connecting to a spreadsheet cell (VTKCell, ImageViewerCell, ...). Similarly, a SheetReference module (when connecting to a CellLocation) will specify which sheet the cell will be put on given its name, minimum row size and minimum column size. There is an example of this in examples/vtk.xml (select the version below Double Renderer). [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

=== How do I output results to the spreadsheet? ===

By inspecting the VisTrails Spreadsheet package (in the list of packages, to the left of the pipeline builder), you
can see there are built-in cells for different kinds of data, e.g., RichTextCell to display HTML and plain text. op
You (the user) can also define new cell types to display application-specific data. For example, we have developed
VtkCell, MplFigureCell, and OpenGLCell. It is possible to display pretty much anything on the Spreadsheet! [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

Examples of writing cell modules can be found in:
RichTextCell: packages/spreadsheet/widgets/richtext/richtext.py
VTK: packages/vtk/vtkcell.py

Here is the summary of some requirements on a cell widget:

(1) It must be a Qt widget. It should inherit from spreadsheet_cell.QCellWidget in the spreadsheet package. Although any Qt Widget would work, certain features such as animation will not be available (without rewriting it).

(2) It must re-implement the updateContents() function to take a set of inputs (usually coming from input ports of a wrapper Module) and display on the cells. VisTrails uses this function to update/reuse cells on the spreadsheet when new data comes in.

(3) It needs a wrapper VisTrails Module (inherited from basic_widgets.SpreadsheetCell of the spreadsheet package). Inside the compute() method of this module, it may call self.display(CellWidgetType, (inputs)) to trigger the display event on the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#advanced-cell-options Advanced Cell Options]

=== How do I control the default number of cells in the spreadsheet? ===

You can configure the rowCount and colCount using the preferences dialog. Just go to the Module Packages tab, select spreadsheet in the "Enabled packages" and press the Configure button. Then a list of all the configuration options for the spreadsheet will show up. [http://www.vistrails.org/usersguide/spreadsheet.html#custom-layout-options Custom Layout Options]

=== Is it possible to launch a web browser from the vistrails spreadsheet? We would like to output several urls from a parameter sweep and then have the option to click on each one to view the resulting page. I can view the page within the spreadsheet, but it is really too crowded.===

Currently, there isn't a widget that provides exactly this functionality, but I can think of a few solutions that may work for you:

(1) You can use parameter exploration to generate multiple sheets so you might have an exploration that opens each page in a new sheet. Use the third column/dimension in the exploration interface to have a parameter span sheets.

(2) The spreadsheet is extensible so you can write a custom spreadsheet cell widget that has a button or label with the desired link (a QLabel with openExternalLinks set to True, for example).

(3) You can tweak the existing RichTextCell be adding the line "self.browser.setOpenExternalLinks(True)" at line 63 of the source file "vistrails/packages/spreadsheet/widgets/richtext/richtext.py". Then, if your workflow creates a file with html markup text like "<a href="http://www.vistrails.org">VisTrails</a>" connected to a RichTextCell, clicking on the rendered link in the cell will open it in a web browser. You need to add the aforementioned line to the source to let Qt know that you want the link opened externally; by default, it will just issue an event that isn't processed. [http://www.vistrails.org/usersguide/spreadsheet.html#launching-a-web-browser Launching a Web Browser]

== Integrating your software into VisTrails ==

===How can I integrate my own program into VisTrails?===

The easiest way is to create a package. Writing a package is often very simple, here are instructions on how to do it: [[UsersGuideVisTrailsPackages]]

You can also dynamically generate modules. For an example see:

http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#Packages_that_generate_modules_dynamically

In particular, see the new_module call which uses python's type() function to generate new classes dynamically.

=== How do modules deal with multiple inputs in a same port? ===

(And should that even be allowed?)

For compatibility reasons, we do need to allow multiple connections to an input port. However, most package developers should never have to use this, and so we do our best to hide it. the default behavior for getting inputs from a port, then, is to always return a single input.

If on your module you need multiple inputs connected to a single port, use the 'forceGetInputListFromPort' method. It will return a list of all the data items coming through the port. The spreadsheet package uses this feature, so look there for usage examples (vistrails/packages/spreadsheet/basic_widgets.py)
[http://www.vistrails.org/usersguide/packages.html#configuring-ports Configuring Ports]

=== Are there mechanisms for attaching widgets to different modules/parameters? ===

Right now, we have a mechanism for putting a specific widget for an input port. For example, if a port is SetColor(red, green, blue), we can put a color wheel widget there. Or we can also replace the SetFileName port with a File Widget. However, this is not per parameter (only per port). We are currently working on this problem.

=== Can I organize my package so it appears hierarchical in the module palette? ===

Yes. Use the namespace keyword argument when adding the module to the registry. For example,

registry.add_module(MyModule, namespace='MyNamespace')
[http://www.vistrails.org/usersguide/packages.html#module-hierarchy-and-visibility Module Hierarchy and Visibility]

=== Can I nest namespaces? ===

Yes. Use the '|' character to separate different the hierarchy. For example,

registry.add_module(MyModule, namespace='ParentNamespace|ChildNamespace')

[http://www.vistrails.org/usersguide/packages.html#module-hierarchy-and-visibility Module Hierarchy and Visibility]

=== Are there shortcuts for registry initialization? ===

Yes. If you define _modules as a list of classes in the __init__.py file of your package, VisTrails will attempt to load all classes specified as modules. You can provide add_module options as keyword arguments by specifying a tuple (class, kwargs) in the list. For example:

_modules = [MyModule1, (MyModule2, {'namespace': 'MyNamespace'})]

In addition, you need to identify the ports of your modules as a field in your class by defining _input_ports and _output_ports lists. Here, the items in each list must be tuples of the form (portName, portSignature, optional=False, sort_key=-1). For example:

class MyModule(Module):
def compute(self):
pass

_input_ports = [('firstInput', String), ('secondInput', Integer, True)]
_output_ports = [('firstOutput', String), ('secondOutput', String)]

[http://www.vistrails.org/usersguide/packages.html#options-for-configuring-modules-and-ports Options for Configuring Modules and Ports]

=== Can I define ports to be of types that I do not import into my package? ===

Yes. You can pass an identifier string as the portSignature instead. The port_signature string is defined by:

<module_string> := <package_identifier>:[<namespace>|]<module_name>,
<port_signature> := (<module_string>*)

For example,

registry.add_input_port(MyModule, 'myInputPort', '(edu.utah.sci.vistrails.basic:String)')

or

_input_ports = [('myInputPort', '(edu.utah.sci.vistrails.basic:String)')]

[http://www.vistrails.org/usersguide/packages.html#configuring-ports Configuring Ports]

=== What do I need to change in my package to make it reloadable (new in v1.4.2)? ===
See [http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#How_to_make_you_package_reloadable UsersGuideVisTrailsPackages] for an explanation.

=== Can I add default values or labels for parameters? ===

Yes. Versions 1.4 and greater support these features. See [[UsersGuideVisTrailsPackages#Adding_default_values_and.2For_labels_for_parameters | UsersGuideVisTrailsPacakges]] for more details.
[http://www.vistrails.org/usersguide/packages.html#configuring-ports Configuring Ports]

=== I want to write a module to load HDF data whose output (e.g., data, string) varies according to the input I give it. Is is possible to do this in VisTrails, and if yes, how can I do that? Ideally, I would like to avoid having to change the connection of my output every time I change the input. ===

There are a few ways to tackle this - each has it's own benefits and pitfalls. Firstly, module connections do respect class hierarchies as we're familiar with in object oriented languages. For instance, A module can output a Constant of which String, Float, Integer, etc are specifications. In this way, you can have a subclass of something like HDFData be passed out of the module and the connections will be established regardless of the sub-type. This is a bit dangerous though. Modules downstream of such a class may not really know how to operate on certain types derived from the super-class. Extreme care must be taken both when creating the modules as well as connecting them to prevent things like this from happening.

A second method that I employ in several different packages is the idea of a container class. For instance, the NumSciPy package uses a relatively generic container "Numpy Array" to encapsulate the data. Of course, these encapsulating objects can store dictionaries that other modules can easily access and understand how to operate on. Although this method is slightly more work, the benefits of a stricter typing of ports is beneficial - particularly upon interfacing with other packages that may depend on strongly typed constants (for example).
[http://www.vistrails.org/usersguide/packages.html#configuring-ports Configuring Ports]

=== I need to determine, at run-time, whether or not a "child" module is attached to the output port of a "parent" module. (I do not specifically need to know which child; just if there is one). ===

The outputPorts dictionary of the base Module stores this information. Thus, you should be able to check

("myPortName" in self.outputPorts)

on the parent module to check if there are any downstream connections from the port "myPortName". This might be used, for example, to only set results for output ports that will be used. ***Note***, however, that the caching algorithm assumes that all outputs are set so adding a new connection to a previously unconnected output port will not work as desired if that module is cached. For this reason, I would currently recommend making such a module not cacheable. Another possibility is overriding the update() method to check the outputPorts and set the upToDate flag if they are not equal. In a single, limited test, this seemed to work, but be warned that it is not fully tested. Here is an example:

class TestModule(Module):
_output_ports = [('a1', '(edu.utah.sci.vistrails.basic:String)'),
('a2', '(edu.utah.sci.vistrails.basic:String)')]
def __init__(self):
Module.__init__(self)
self._cached_output_ports = set()

def update(self):
if len(set(self.outputPorts) - self._cached_output_ports) > 0:
self.upToDate = False
Module.update(self)

def compute(self):
if "a1" in self.outputPorts:
self.setResult("a1", "test")
if "a2" in self.outputPorts:
self.setResult("a2", "test2")
self._cached_output_ports = set(self.outputPorts)

[http://www.vistrails.org/usersguide/packages.html#configuring-ports Configuring Ports]

=== How can I make a module not display in the modules list? ===

You should set the abstract parameter to True when adding the module to the registry. Using the original syntax, this looks like:

def initialize():
reg = core.modules.module_registry.get_module_registry()
reg.add_module(InvisibleModule, abstract=True)
# ...

With the _modules dictionary shortcut (for more details, see [http://www.vistrails.org/index.php/FAQ#Are_there_shortucts_for_registry_initialization.3F the FAQ section on this]), you include it in a kwargs dict as part of a module tuple:

_modules = [AnotherModule, (InvisibleModule, {'abstract': True})]

There is also a 'hide_descriptor' parameter that prevents the module from appearing in the module palette without declaring it to be abstract.

The technical difference between the two is that 'abstract' will not add the item to the module palette while 'hide_descriptor' does add the item but immediately hides it. If the module should never be instantiated in a workflow, declare it abstract. If you don't want users to be able to add the module to a pipeline, but you have code that may add it programmatically, declare it with hide_descriptor=True.

[http://www.vistrails.org/usersguide/packages.html#module-hierarchy-and-visibility Module Hierarchy and Visibility]

== The Console ==

=== Where should I go to find out what I can call from the console and how to import it? ===

We have tried to make some methods more accessible in the console via an api. You can import the api via <code>import api</code> in the console and see the available methods with <code>dir(api)</code>. To open a vistrail:

import api
api.open_vistrail_from_file('/Applications/VisTrails/examples/terminator.vt')

To execute a version of a workflow, you currently have to go through the controller:

api.select_version('Histogram')
api.get_current_controller().execute_current_workflow()

Currently, only a subset of VisTrails functionality is directly available from the api. However, since VisTrails is written in python, you can dig down starting with the VistrailsApplication or controller object to expose most of our internal methods. If you have suggestions for calls to be added to the api, please let us know.

One other feature that we're working on, but is still in progress is the ability to construct workflows via the console. For example:

vtk = load_package('edu.utah.sci.vistrails.vtk')
vtk.vtkDataSetReader() # adds a vtkDataSetReader module to the pipeline
# click on the new module
a = selected_modules()[0] # get the one currently selected module
a.SetFile('/vistrails/examples/data/head120.vtk') # sets the SetFile parmaeter for the data set reader
b = vtk.vtkContourFilter() # adds a vtkContourFilter module to the pipeline and saves to var b
b.SetInputConnection0(a.GetOutputPort0()) # connects a's GetOutputPort0 port to b's SetInputConnection0
[http://www.vistrails.org/usersguide/batch.html#finding-methods-via-the-command-line Finding Methods Via the Command Line]

== Persistence Package ==

=== How do I use the output of one workflow as the input for another using the persistence package? ===

You need to configure the persistence modules using the module's configuration dialog. After adding a <code>PersistentOutputFile</code> to the workflow, click on the triangle in the upper-right corner of the <code>PersistentOutputFile</code>, and select "Edit Configuration" from the menu that appears. In this dialog, select "Create New Reference" and give the reference a name (and any space-delimited tags). Upon running that workflow, the data will be written to the persistent store. In the second workflow where you wish to use that file, add a <code>PersistentInputFile</code> and go to its configuration dialog in the same manner as with the output file. In that dialog, select "Use Existing Reference" and select the data that you just added in the first workflow in the list of files below. Now, when you run that workflow, it will grab the data from the persistent store.

Here is an example: [[Media:offscreen_persistent.vt | offscreen_persistent.vt]]. Run the "persistent offscreen" workflow first, and then run the "display persistent output" to use the output of the first workflow as the input for the second.

== VTK ==

=== Given a VTK visualization, how can I generate a webpage from it? ===

Check out the html pipeline in offscreen.xml.

=== I'm trying to use VTK, but there doesn't seem to be any output. What is wrong? ===

To use VTK on VisTrails, you need a slightly different way of connecting the renderer modules. Instead of using the standard RenderWindow/RenderWindowInteractor infrastructure, you simply connect the renderer to a VTKCell. The examples directory in the distribution has several VTK examples that illustrate.

=== I am trying to add a module to the workflow via Python, but how can I access vtk modules? ===

Here's an example:

import api

vtvtk = 'edu.utah.sci.vistrails.vtk'

module = api.add_module(0, 0, vtvtk, 'vtkContourFilter', '')

The third argument in add_module is the package identifier. You can find this in the "Module Packages" panel of the Preferences; just click on the package you're interested in and it will appear in the information on the right.

== matplotlib ==

=== I'm experiencing a problem with Latex labels and the matplotlib that comes with VisTrails 1.5. The script below entered to the interpreter that comes with VT is sufficient to reproduce it. ===

import matplotlib.pyplot as plt
plt.plot([1,2,3],[1,2,3])
plt.xlabel("$foo$")

Remove your ~/.matplotlib folder and re-start VisTrails

== VisTrails Development ==

=== I would like to build VisTrails from source. Are there instructions on how to do this? ===

Yes! Take a look at http://www.vistrails.org/index.php/Mac_Intel_Instructions
[http://www.vistrails.org/usersguide/getting_started.html#installing-vistrails-from-source Installing VisTrails from source]

== Accessing Provenance Information ==

=== How do I access the information in the execution log? ===

The code responsible for storing execution information is located in the "core/log" directories, and the code that generates much of that information is in "core/interpreter/cached.py". Modules can add execution-specific annotations to provenance via annotate() calls during execution, but much of the data (like timing and errors) is captured by the LogController and CachedInterpreter (the execution engine) objects. To analyze the log from a vistrail (.vt) file, you might have something like the following:

<code>
import core.log.log
import db.services.io

def run(fname):
# open the .vt bundle specified by the filename "fname"
bundle = db.services.io.open_vistrail_bundle_from_zip_xml(fname)[0]
# get the log filename
log_fname = bundle.vistrail.db_log_filename
if log_fname is not None:
# open the log
log = db.services.io.open_log_from_xml(log_fname, True)
# convert the log from a db object
core.log.log.Log.convert(log)
for workflow_exec in log.workflow_execs:
print 'workflow version:', workflow_exec.parent_version
print 'time started:', workflow_exec.ts_start
print 'time ended:', workflow_exec.ts_end
print 'modules executed:', [i.module_id
for i in workflow_exec.item_execs]
if __name__ == '__main__':
run("some_vistrail.vt")
</code>

You should be able to see what information is available by looking at the "core/log" classes. [http://www.vistrails.org/usersguide/log.html Accessing the Execution Log]

FAQ

2011-04-01T00:22:52Z

Ctuttle: /* I need to determine, at run-time, whether or not a "child" module is attached to the output port of a "parent" module. (I do not specifically need to know which child; just if there is one). */

Also check our [[known_issues|Known Issues]] page for troubleshooting.

----
== Running workflows ==

=== How can I run a workflow using the command line? ===

('''Updated for version 1.2''') Call vistrails using the following options:

<code>python vistrails.py -b path_to_vistrails_file:pipeline </code>

where pipeline can be a version '''tag name''' or version '''id'''

'''NOTE:''' If you downloaded the MacOS X bundle, you can run vistrails from the command line via the following commands in the Terminal. Change the current directory to wherever VisTrails was installed (often <code>/Applications</code>), and then type:

<code>Vistrails.app/Contents/MacOS/vistrails [<cmd_line_options>]</code> [http://www.vistrails.org/usersguide/batch.html#running-a-specific-workflow-in-batch-mode Running a Specific Workflow in Batch Mode]

=== Using the command line, we'd like to execute a workflow multiple times, with slightly different parameters, and create a series of output files. Is this possible?===

('''Updated for version 1.2''') We can change parameters that have an alias through the command line.

For example, offscreen pipeline in offscreen.vt always creates the file called image.png. If you want generate it with a different filename:

<code>python vistrails.py -b ../examples/offscreen.vt:offscreen -a"filename=other.png"</code>

<code>filename</code> in the example above is the alias name assigned to the parameter in the <code>value</code> method inside the String module. When running a pipeline from the command line, VisTrails will try to start the spreadsheet automatically if the pipeline requires it. For example, this other execution will also start the spreadsheet (attention to how $ characters are escaped when running on bash):

<code>python vistrails.py -b ../examples/head.vt:aliases -a"isovalue=30\$&\$diffuse_color=0.8,0.4,0.2"</code>

You can also execute more than one pipeline on the command line:

<code>python vistrails.py -b ../examples/head.vt:aliases ../examples/spx.vt:spx \
-a"isovalue=30"</code>

Use the -a parameter only once regardless the number of pipelines. [http://www.vistrails.org/usersguide/batch.html#running-a-workflow-with-specific-parameters Running a Workflow with Specific Parameters]

=== I can load a vistrail, and the version tree shows up fine. However, no pipelines appear when I click on a version. What gives? ===

The most likely reason is that the vistrail uses a package that is not registered with VisTrails. You need to identify the needed package and add it to your <code>.vistrails/startup.py</code>. A single line like the following should be enough:

addPackage('enter_package_name_here')

Some packages might need more information. For example:

addPackage('afront', executable_path='/path/to/afront')

Refer to the package documentation for details. The one inconvenient step is that currently there's no automated way to describe what is the missing package. We're working on this feature for future releases.

=== I have a workflow that reads a file and then does some processing. The first time it runs, it executes correctly. But in subsequent, nothing happens. ===

VisTrails caches by default, so after a workflow is executed, if none of its parameters change, it won't be executed again.

If a workflow reads a file using the basic module File, VisTrails does check whether the file was modified since the last run. It does so by keeping a signature that is based on the modification time of the file. And if the file was modified, the File module and all downstream modules (the ones which depend on File) will be executed.

''Note: If you would like your input and output data to be versioned, you can use the Persistence package.''

If you do not want VisTrails to cache executions, you can turn off caching: go to Menu Edit -> Preferences and in the General Configuration tab, change Cache execution results to Never. [http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

=== Can VisTrails execute workflows in parallel? ===

The VisTrails server can only execute pipelines in parallel if there's more than one instance of VisTrails running. The command

<code> self.rpcserver = ThreadedXMLRPCServer((self.temp_xml_rpc_options.server, self.temp_xml_rpc_options.port))</code>

starts a multithreaded version of the XML-RPC server, so it will create a thread for each request received by the server. The problem is that Qt/PyQT doesn't allow these multiple threads create GUI objects, only in the main thread. To overcome this limitation, the multithreaded version can instantiate other single threaded versions of VisTrails and put them in a queue, so workflow executions and other GUI-related requests, such as generating workflow graphs and history trees can be forwarded to this queue, and each instance takes turns in answering the request. If the results are in the cache, the multithreaded version answers the requests directly.

Note that this infrastructure works on Linux only. To make this work on Windows, you have to create a script similar to start_vistrails_xvfb.sh (located in the scripts folder) where you can send the number of other instances via command-line options to VisTrails. The command line options are:

python vistrails_server.py -T <ADDRESS> -R <PORT> -O<NUMBER_OF_OTHER_VISTRAILS_INSTANCES> [-M]&

If you want the main vistrails instance to be multithreaded, use the -M at the end.

After creating this script, update function start_other_instances in vistrails/gui/application_server.py lines 1007-1023 and set the script variable to point to your script. You may also have to change the arguments sent to your script (line 1016: for example, you don't need to set a virtual display). You will need to change the path to the stop_vistrails_server.py script (on line 1026) according to your installation path. [http://www.vistrails.org/usersguide/batch.html#executing-workflows-in-parallel Executing Workflows in Parallel]

=== When a workflow is executed, what do the colors mean? ===

- lilac: module was notexecuted

- yellow: module is currently being executed

- green: module was successfully executed

- orange: module was cached

- red: the execution of the module failed

[http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

== Building workflows ==

=== Is there a way to give each widget a "display name" in addition to the module name at the center of the widget? ===

Yes, a "display name" can be assigned to a module by selecting the triangle in its top right corner to open a popup menu and selecting the Set Module Label... menu item. You will then be prompted to enter the "display name". [http://www.vistrails.org/usersguide/creating.html#changing-module-labels Changing Module Labels]

=== Is there a way to re-center the picture-in-picture (PiP) view? ===

Yes. If you click on the PIP window to bring it to focus, you can press Ctrl-R (or Command-R on Mac) to re-center the PiP window. [http://www.vistrails.org/usersguide/getting_started.html#vistrails-interaction Vistrails Interaction]

=== How do I search for a literal "?" (question mark) in the search box in the Property panel? ===

Since we allow regular expressions in our search box, question marks are treated as meta-characters. Thus, searching for "?" returns everything and "abc?" will return everything containing "abc". You need to use "\?" instead to search for "?". So the search for "??" would be "\?\?". [http://www.vistrails.org/usersguide/querying.html#textual-queries Textual Queries]

== Using VisTrails as a server ==

=== What is the VisTrails server-mode? ===

Using the VisTrails server mode, it is possible to ''execute workflows and control VisTrails through another application''. For example, the CrowdLabs Web portal (http://www.crowdlabs.org) accesses a VisTrails sever to execute workflows, retrieve and display vistrail trees and workflows. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

=== How do I execute workflows and control VisTrails through another application? ===

The way you access the server is by doing XML-RPC calls. In the current VisTrails release, we include a set of PHP scripts that can talk to a VisTrails server instance. They are in "extensions/http" folder. The files are reasonably well documented. Also, it should be not difficult to create python scripts to access the server (just use xmlrpclib module).

Note that the VisTrails server requires the provenance and workflows to be in a database. More detailed instructions on how to setup the server and the database
are available here:

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_server_setup.html

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_database_setup.html

If what you want is just to execute a series of workflows in batch mode, a simpler solution would be to ''use the VisTrails client in batch mode''. Chapter 12 of the user's guide contains detailed information and examples on that. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

== Control Flow ==

=== Note: using map ===

When using 'map', the module (or subworkflow) used as function port in the map module MUST be a function, i.e., it can only define 1 output port. [http://www.vistrails.org/usersguide/controlflow.html#the-map-operator The Map Operator]

== Spreadsheet ==

Where ''pipeline'' is a version number or a tag.

=== How can I save an image from the spreadsheet? ===

While having the focus on a spreadsheet cell and select the camera on the toolbar to take a snapshot. The system will prompt you for the location and file name where it should be saved. The other icons can be used for saving multiple images that can be used for generating an animation on demand. A whole sheet can also be saved by selecting Export (either from the menu or from the toolbar). [http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet-image Saving a Spreadsheet Image]

===Is it possible to save the complete state of the spreadsheet?===
[http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet Saving a Spreadsheet]

=== Can I view multiple sheets at the same time? ===

Yes. Each sheet on the spreadsheet can be displayed as a dock widget separated from the main spreadsheet window by dragging its tab name out of the tab bar at the bottom of the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Then, how can I put back a separated sheet? ===

A sheet can be docked back to the main window by dragging it back to the tab bar or double-click on its title bar. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== How can I order sheets on the spreadsheet? ===

This can be done by dragging the sheet name on the bottom top bar and drop it to the right place. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Can I control where a cell will be placed on the spreadsheet window? ===

By default, an unoccupied cell on the active sheet will be chosen to display the result. However, you can specify exactly in the pipeline where a spreadsheet cell will be placed by using CellLocation and SheetReference. CellLocation specifies the location (row and column) of a cell when connecting to a spreadsheet cell (VTKCell, ImageViewerCell, ...). Similarly, a SheetReference module (when connecting to a CellLocation) will specify which sheet the cell will be put on given its name, minimum row size and minimum column size. There is an example of this in examples/vtk.xml (select the version below Double Renderer). [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

=== How do I output results to the spreadsheet? ===

By inspecting the VisTrails Spreadsheet package (in the list of packages, to the left of the pipeline builder), you
can see there are built-in cells for different kinds of data, e.g., RichTextCell to display HTML and plain text. op
You (the user) can also define new cell types to display application-specific data. For example, we have developed
VtkCell, MplFigureCell, and OpenGLCell. It is possible to display pretty much anything on the Spreadsheet! [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

Examples of writing cell modules can be found in:
RichTextCell: packages/spreadsheet/widgets/richtext/richtext.py
VTK: packages/vtk/vtkcell.py

Here is the summary of some requirements on a cell widget:

(1) It must be a Qt widget. It should inherit from spreadsheet_cell.QCellWidget in the spreadsheet package. Although any Qt Widget would work, certain features such as animation will not be available (without rewriting it).

(2) It must re-implement the updateContents() function to take a set of inputs (usually coming from input ports of a wrapper Module) and display on the cells. VisTrails uses this function to update/reuse cells on the spreadsheet when new data comes in.

(3) It needs a wrapper VisTrails Module (inherited from basic_widgets.SpreadsheetCell of the spreadsheet package). Inside the compute() method of this module, it may call self.display(CellWidgetType, (inputs)) to trigger the display event on the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#advanced-cell-options Advanced Cell Options]

=== How do I control the default number of cells in the spreadsheet? ===

You can configure the rowCount and colCount using the preferences dialog. Just go to the Module Packages tab, select spreadsheet in the "Enabled packages" and press the Configure button. Then a list of all the configuration options for the spreadsheet will show up. [http://www.vistrails.org/usersguide/spreadsheet.html#custom-layout-options Custom Layout Options]

=== Is it possible to launch a web browser from the vistrails spreadsheet? We would like to output several urls from a parameter sweep and then have the option to click on each one to view the resulting page. I can view the page within the spreadsheet, but it is really too crowded.===

Currently, there isn't a widget that provides exactly this functionality, but I can think of a few solutions that may work for you:

(1) You can use parameter exploration to generate multiple sheets so you might have an exploration that opens each page in a new sheet. Use the third column/dimension in the exploration interface to have a parameter span sheets.

(2) The spreadsheet is extensible so you can write a custom spreadsheet cell widget that has a button or label with the desired link (a QLabel with openExternalLinks set to True, for example).

(3) You can tweak the existing RichTextCell be adding the line "self.browser.setOpenExternalLinks(True)" at line 63 of the source file "vistrails/packages/spreadsheet/widgets/richtext/richtext.py". Then, if your workflow creates a file with html markup text like "<a href="http://www.vistrails.org">VisTrails</a>" connected to a RichTextCell, clicking on the rendered link in the cell will open it in a web browser. You need to add the aforementioned line to the source to let Qt know that you want the link opened externally; by default, it will just issue an event that isn't processed. [http://www.vistrails.org/usersguide/spreadsheet.html#launching-a-web-browser Launching a Web Browser]

== Integrating your software into VisTrails ==

===How can I integrate my own program into VisTrails?===

The easiest way is to create a package. Writing a package is often very simple, here are instructions on how to do it: [[UsersGuideVisTrailsPackages]]

You can also dynamically generate modules. For an example see:

http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#Packages_that_generate_modules_dynamically

In particular, see the new_module call which uses python's type() function to generate new classes dynamically.

=== How do modules deal with multiple inputs in a same port? ===

(And should that even be allowed?)

For compatibility reasons, we do need to allow multiple connections to an input port. However, most package developers should never have to use this, and so we do our best to hide it. the default behavior for getting inputs from a port, then, is to always return a single input.

If on your module you need multiple inputs connected to a single port, use the 'forceGetInputListFromPort' method. It will return a list of all the data items coming through the port. The spreadsheet package uses this feature, so look there for usage examples (vistrails/packages/spreadsheet/basic_widgets.py)
[http://www.vistrails.org/usersguide/packages.html#configuring-ports Configuring Ports]

=== Are there mechanisms for attaching widgets to different modules/parameters? ===

Right now, we have a mechanism for putting a specific widget for an input port. For example, if a port is SetColor(red, green, blue), we can put a color wheel widget there. Or we can also replace the SetFileName port with a File Widget. However, this is not per parameter (only per port). We are currently working on this problem.

=== Can I organize my package so it appears hierarchical in the module palette? ===

Yes. Use the namespace keyword argument when adding the module to the registry. For example,

registry.add_module(MyModule, namespace='MyNamespace')
[http://www.vistrails.org/usersguide/packages.html#module-hierarchy-and-visibility Module Hierarchy and Visibility]

=== Can I nest namespaces? ===

Yes. Use the '|' character to separate different the hierarchy. For example,

registry.add_module(MyModule, namespace='ParentNamespace|ChildNamespace')

[http://www.vistrails.org/usersguide/packages.html#module-hierarchy-and-visibility Module Hierarchy and Visibility]

=== Are there shortcuts for registry initialization? ===

Yes. If you define _modules as a list of classes in the __init__.py file of your package, VisTrails will attempt to load all classes specified as modules. You can provide add_module options as keyword arguments by specifying a tuple (class, kwargs) in the list. For example:

_modules = [MyModule1, (MyModule2, {'namespace': 'MyNamespace'})]

In addition, you need to identify the ports of your modules as a field in your class by defining _input_ports and _output_ports lists. Here, the items in each list must be tuples of the form (portName, portSignature, optional=False, sort_key=-1). For example:

class MyModule(Module):
def compute(self):
pass

_input_ports = [('firstInput', String), ('secondInput', Integer, True)]
_output_ports = [('firstOutput', String), ('secondOutput', String)]

[http://www.vistrails.org/usersguide/packages.html#options-for-configuring-modules-and-ports Options for Configuring Modules and Ports]

=== Can I define ports to be of types that I do not import into my package? ===

Yes. You can pass an identifier string as the portSignature instead. The port_signature string is defined by:

<module_string> := <package_identifier>:[<namespace>|]<module_name>,
<port_signature> := (<module_string>*)

For example,

registry.add_input_port(MyModule, 'myInputPort', '(edu.utah.sci.vistrails.basic:String)')

or

_input_ports = [('myInputPort', '(edu.utah.sci.vistrails.basic:String)')]

[http://www.vistrails.org/usersguide/packages.html#configuring-ports Configuring Ports]

=== What do I need to change in my package to make it reloadable (new in v1.4.2)? ===
See [http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#How_to_make_you_package_reloadable UsersGuideVisTrailsPackages] for an explanation.

=== Can I add default values or labels for parameters? ===

Yes. Versions 1.4 and greater support these features. See [[UsersGuideVisTrailsPackages#Adding_default_values_and.2For_labels_for_parameters | UsersGuideVisTrailsPacakges]] for more details.
[http://www.vistrails.org/usersguide/packages.html#configuring-ports Configuring Ports]

=== I want to write a module to load HDF data whose output (e.g., data, string) varies according to the input I give it. Is is possible to do this in VisTrails, and if yes, how can I do that? Ideally, I would like to avoid having to change the connection of my output every time I change the input. ===

There are a few ways to tackle this - each has it's own benefits and pitfalls. Firstly, module connections do respect class hierarchies as we're familiar with in object oriented languages. For instance, A module can output a Constant of which String, Float, Integer, etc are specifications. In this way, you can have a subclass of something like HDFData be passed out of the module and the connections will be established regardless of the sub-type. This is a bit dangerous though. Modules downstream of such a class may not really know how to operate on certain types derived from the super-class. Extreme care must be taken both when creating the modules as well as connecting them to prevent things like this from happening.

A second method that I employ in several different packages is the idea of a container class. For instance, the NumSciPy package uses a relatively generic container "Numpy Array" to encapsulate the data. Of course, these encapsulating objects can store dictionaries that other modules can easily access and understand how to operate on. Although this method is slightly more work, the benefits of a stricter typing of ports is beneficial - particularly upon interfacing with other packages that may depend on strongly typed constants (for example).
[http://www.vistrails.org/usersguide/packages.html#configuring-ports Configuring Ports]

=== I need to determine, at run-time, whether or not a "child" module is attached to the output port of a "parent" module. (I do not specifically need to know which child; just if there is one). ===

The outputPorts dictionary of the base Module stores this information. Thus, you should be able to check

("myPortName" in self.outputPorts)

on the parent module to check if there are any downstream connections from the port "myPortName". This might be used, for example, to only set results for output ports that will be used. ***Note***, however, that the caching algorithm assumes that all outputs are set so adding a new connection to a previously unconnected output port will not work as desired if that module is cached. For this reason, I would currently recommend making such a module not cacheable. Another possibility is overriding the update() method to check the outputPorts and set the upToDate flag if they are not equal. In a single, limited test, this seemed to work, but be warned that it is not fully tested. Here is an example:

class TestModule(Module):
_output_ports = [('a1', '(edu.utah.sci.vistrails.basic:String)'),
('a2', '(edu.utah.sci.vistrails.basic:String)')]
def __init__(self):
Module.__init__(self)
self._cached_output_ports = set()

def update(self):
if len(set(self.outputPorts) - self._cached_output_ports) > 0:
self.upToDate = False
Module.update(self)

def compute(self):
if "a1" in self.outputPorts:
self.setResult("a1", "test")
if "a2" in self.outputPorts:
self.setResult("a2", "test2")
self._cached_output_ports = set(self.outputPorts)

[http://www.vistrails.org/usersguide/packages.html#configuring-ports Configuring Ports]

=== How can I make a module not display in the modules list? ===

You should set the abstract parameter to True when adding the module to the registry. Using the original syntax, this looks like:

def initialize():
reg = core.modules.module_registry.get_module_registry()
reg.add_module(InvisibleModule, abstract=True)
# ...

With the _modules dictionary shortcut (for more details, see [http://www.vistrails.org/index.php/FAQ#Are_there_shortucts_for_registry_initialization.3F the FAQ section on this]), you include it in a kwargs dict as part of a module tuple:

_modules = [AnotherModule, (InvisibleModule, {'abstract': True})]

There is also a 'hide_descriptor' parameter that prevents the module from appearing in the module palette without declaring it to be abstract.

The technical difference between the two is that 'abstract' will not add the item to the module palette while 'hide_descriptor' does add the item but immediately hides it. If the module should never be instantiated in a workflow, declare it abstract. If you don't want users to be able to add the module to a pipeline, but you have code that may add it programmatically, declare it with hide_descriptor=True.

[http://www.vistrails.org/usersguide/packages.html#module-hierarchy-and-visibility Module Hierarchy and Visibility]

== The Console ==

=== Where should I go to find out what I can call from the console and how to import it? ===

We have tried to make some methods more accessible in the console via an api. You can import the api via <code>import api</code> in the console and see the available methods with <code>dir(api)</code>. To open a vistrail:

import api
api.open_vistrail_from_file('/Applications/VisTrails/examples/terminator.vt')

To execute a version of a workflow, you currently have to go through the controller:

api.select_version('Histogram')
api.get_current_controller().execute_current_workflow()

Currently, only a subset of VisTrails functionality is directly available from the api. However, since VisTrails is written in python, you can dig down starting with the VistrailsApplication or controller object to expose most of our internal methods. If you have suggestions for calls to be added to the api, please let us know.

One other feature that we're working on, but is still in progress is the ability to construct workflows via the console. For example:

vtk = load_package('edu.utah.sci.vistrails.vtk')
vtk.vtkDataSetReader() # adds a vtkDataSetReader module to the pipeline
# click on the new module
a = selected_modules()[0] # get the one currently selected module
a.SetFile('/vistrails/examples/data/head120.vtk') # sets the SetFile parmaeter for the data set reader
b = vtk.vtkContourFilter() # adds a vtkContourFilter module to the pipeline and saves to var b
b.SetInputConnection0(a.GetOutputPort0()) # connects a's GetOutputPort0 port to b's SetInputConnection0

== Persistence Package ==

=== How do I use the output of one workflow as the input for another using the persistence package? ===

You need to configure the persistence modules using the module's configuration dialog. After adding a <code>PersistentOutputFile</code> to the workflow, click on the triangle in the upper-right corner of the <code>PersistentOutputFile</code>, and select "Edit Configuration" from the menu that appears. In this dialog, select "Create New Reference" and give the reference a name (and any space-delimited tags). Upon running that workflow, the data will be written to the persistent store. In the second workflow where you wish to use that file, add a <code>PersistentInputFile</code> and go to its configuration dialog in the same manner as with the output file. In that dialog, select "Use Existing Reference" and select the data that you just added in the first workflow in the list of files below. Now, when you run that workflow, it will grab the data from the persistent store.

Here is an example: [[Media:offscreen_persistent.vt | offscreen_persistent.vt]]. Run the "persistent offscreen" workflow first, and then run the "display persistent output" to use the output of the first workflow as the input for the second.

== VTK ==

=== Given a VTK visualization, how can I generate a webpage from it? ===

Check out the html pipeline in offscreen.xml.

=== I'm trying to use VTK, but there doesn't seem to be any output. What is wrong? ===

To use VTK on VisTrails, you need a slightly different way of connecting the renderer modules. Instead of using the standard RenderWindow/RenderWindowInteractor infrastructure, you simply connect the renderer to a VTKCell. The examples directory in the distribution has several VTK examples that illustrate.

=== I am trying to add a module to the workflow via Python, but how can I access vtk modules? ===

Here's an example:

import api

vtvtk = 'edu.utah.sci.vistrails.vtk'

module = api.add_module(0, 0, vtvtk, 'vtkContourFilter', '')

The third argument in add_module is the package identifier. You can find this in the "Module Packages" panel of the Preferences; just click on the package you're interested in and it will appear in the information on the right.

== matplotlib ==

=== I'm experiencing a problem with Latex labels and the matplotlib that comes with VisTrails 1.5. The script below entered to the interpreter that comes with VT is sufficient to reproduce it. ===

import matplotlib.pyplot as plt
plt.plot([1,2,3],[1,2,3])
plt.xlabel("$foo$")

Remove your ~/.matplotlib folder and re-start VisTrails

== VisTrails Development ==

=== I would like to build VisTrails from source. Are there instructions on how to do this? ===

Yes! Take a look at http://www.vistrails.org/index.php/Mac_Intel_Instructions
[http://www.vistrails.org/usersguide/getting_started.html#installing-vistrails-from-source Installing VisTrails from source]

== Accessing Provenance Information ==

=== How do I access the information in the execution log? ===

The code responsible for storing execution information is located in the "core/log" directories, and the code that generates much of that information is in "core/interpreter/cached.py". Modules can add execution-specific annotations to provenance via annotate() calls during execution, but much of the data (like timing and errors) is captured by the LogController and CachedInterpreter (the execution engine) objects. To analyze the log from a vistrail (.vt) file, you might have something like the following:

<code>
import core.log.log
import db.services.io

def run(fname):
# open the .vt bundle specified by the filename "fname"
bundle = db.services.io.open_vistrail_bundle_from_zip_xml(fname)[0]
# get the log filename
log_fname = bundle.vistrail.db_log_filename
if log_fname is not None:
# open the log
log = db.services.io.open_log_from_xml(log_fname, True)
# convert the log from a db object
core.log.log.Log.convert(log)
for workflow_exec in log.workflow_execs:
print 'workflow version:', workflow_exec.parent_version
print 'time started:', workflow_exec.ts_start
print 'time ended:', workflow_exec.ts_end
print 'modules executed:', [i.module_id
for i in workflow_exec.item_execs]
if __name__ == '__main__':
run("some_vistrail.vt")
</code>

You should be able to see what information is available by looking at the "core/log" classes. [http://www.vistrails.org/usersguide/log.html Accessing the Execution Log]

FAQ

2011-04-01T00:22:23Z

Also check our [[known_issues|Known Issues]] page for troubleshooting.

----
== Running workflows ==

=== How can I run a workflow using the command line? ===

('''Updated for version 1.2''') Call vistrails using the following options:

<code>python vistrails.py -b path_to_vistrails_file:pipeline </code>

where pipeline can be a version '''tag name''' or version '''id'''

'''NOTE:''' If you downloaded the MacOS X bundle, you can run vistrails from the command line via the following commands in the Terminal. Change the current directory to wherever VisTrails was installed (often <code>/Applications</code>), and then type:

<code>Vistrails.app/Contents/MacOS/vistrails [<cmd_line_options>]</code> [http://www.vistrails.org/usersguide/batch.html#running-a-specific-workflow-in-batch-mode Running a Specific Workflow in Batch Mode]

=== Using the command line, we'd like to execute a workflow multiple times, with slightly different parameters, and create a series of output files. Is this possible?===

('''Updated for version 1.2''') We can change parameters that have an alias through the command line.

For example, offscreen pipeline in offscreen.vt always creates the file called image.png. If you want generate it with a different filename:

<code>python vistrails.py -b ../examples/offscreen.vt:offscreen -a"filename=other.png"</code>

<code>filename</code> in the example above is the alias name assigned to the parameter in the <code>value</code> method inside the String module. When running a pipeline from the command line, VisTrails will try to start the spreadsheet automatically if the pipeline requires it. For example, this other execution will also start the spreadsheet (attention to how $ characters are escaped when running on bash):

<code>python vistrails.py -b ../examples/head.vt:aliases -a"isovalue=30\$&\$diffuse_color=0.8,0.4,0.2"</code>

You can also execute more than one pipeline on the command line:

<code>python vistrails.py -b ../examples/head.vt:aliases ../examples/spx.vt:spx \
-a"isovalue=30"</code>

Use the -a parameter only once regardless the number of pipelines. [http://www.vistrails.org/usersguide/batch.html#running-a-workflow-with-specific-parameters Running a Workflow with Specific Parameters]

=== I can load a vistrail, and the version tree shows up fine. However, no pipelines appear when I click on a version. What gives? ===

The most likely reason is that the vistrail uses a package that is not registered with VisTrails. You need to identify the needed package and add it to your <code>.vistrails/startup.py</code>. A single line like the following should be enough:

addPackage('enter_package_name_here')

Some packages might need more information. For example:

addPackage('afront', executable_path='/path/to/afront')

Refer to the package documentation for details. The one inconvenient step is that currently there's no automated way to describe what is the missing package. We're working on this feature for future releases.

=== I have a workflow that reads a file and then does some processing. The first time it runs, it executes correctly. But in subsequent, nothing happens. ===

VisTrails caches by default, so after a workflow is executed, if none of its parameters change, it won't be executed again.

If a workflow reads a file using the basic module File, VisTrails does check whether the file was modified since the last run. It does so by keeping a signature that is based on the modification time of the file. And if the file was modified, the File module and all downstream modules (the ones which depend on File) will be executed.

''Note: If you would like your input and output data to be versioned, you can use the Persistence package.''

If you do not want VisTrails to cache executions, you can turn off caching: go to Menu Edit -> Preferences and in the General Configuration tab, change Cache execution results to Never. [http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

=== Can VisTrails execute workflows in parallel? ===

The VisTrails server can only execute pipelines in parallel if there's more than one instance of VisTrails running. The command

<code> self.rpcserver = ThreadedXMLRPCServer((self.temp_xml_rpc_options.server, self.temp_xml_rpc_options.port))</code>

starts a multithreaded version of the XML-RPC server, so it will create a thread for each request received by the server. The problem is that Qt/PyQT doesn't allow these multiple threads create GUI objects, only in the main thread. To overcome this limitation, the multithreaded version can instantiate other single threaded versions of VisTrails and put them in a queue, so workflow executions and other GUI-related requests, such as generating workflow graphs and history trees can be forwarded to this queue, and each instance takes turns in answering the request. If the results are in the cache, the multithreaded version answers the requests directly.

Note that this infrastructure works on Linux only. To make this work on Windows, you have to create a script similar to start_vistrails_xvfb.sh (located in the scripts folder) where you can send the number of other instances via command-line options to VisTrails. The command line options are:

python vistrails_server.py -T <ADDRESS> -R <PORT> -O<NUMBER_OF_OTHER_VISTRAILS_INSTANCES> [-M]&

If you want the main vistrails instance to be multithreaded, use the -M at the end.

After creating this script, update function start_other_instances in vistrails/gui/application_server.py lines 1007-1023 and set the script variable to point to your script. You may also have to change the arguments sent to your script (line 1016: for example, you don't need to set a virtual display). You will need to change the path to the stop_vistrails_server.py script (on line 1026) according to your installation path. [http://www.vistrails.org/usersguide/batch.html#executing-workflows-in-parallel Executing Workflows in Parallel]

=== When a workflow is executed, what do the colors mean? ===

- lilac: module was notexecuted

- yellow: module is currently being executed

- green: module was successfully executed

- orange: module was cached

- red: the execution of the module failed

[http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

== Building workflows ==

=== Is there a way to give each widget a "display name" in addition to the module name at the center of the widget? ===

Yes, a "display name" can be assigned to a module by selecting the triangle in its top right corner to open a popup menu and selecting the Set Module Label... menu item. You will then be prompted to enter the "display name". [http://www.vistrails.org/usersguide/creating.html#changing-module-labels Changing Module Labels]

=== Is there a way to re-center the picture-in-picture (PiP) view? ===

Yes. If you click on the PIP window to bring it to focus, you can press Ctrl-R (or Command-R on Mac) to re-center the PiP window. [http://www.vistrails.org/usersguide/getting_started.html#vistrails-interaction Vistrails Interaction]

=== How do I search for a literal "?" (question mark) in the search box in the Property panel? ===

Since we allow regular expressions in our search box, question marks are treated as meta-characters. Thus, searching for "?" returns everything and "abc?" will return everything containing "abc". You need to use "\?" instead to search for "?". So the search for "??" would be "\?\?". [http://www.vistrails.org/usersguide/querying.html#textual-queries Textual Queries]

== Using VisTrails as a server ==

=== What is the VisTrails server-mode? ===

Using the VisTrails server mode, it is possible to ''execute workflows and control VisTrails through another application''. For example, the CrowdLabs Web portal (http://www.crowdlabs.org) accesses a VisTrails sever to execute workflows, retrieve and display vistrail trees and workflows. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

=== How do I execute workflows and control VisTrails through another application? ===

The way you access the server is by doing XML-RPC calls. In the current VisTrails release, we include a set of PHP scripts that can talk to a VisTrails server instance. They are in "extensions/http" folder. The files are reasonably well documented. Also, it should be not difficult to create python scripts to access the server (just use xmlrpclib module).

Note that the VisTrails server requires the provenance and workflows to be in a database. More detailed instructions on how to setup the server and the database
are available here:

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_server_setup.html

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_database_setup.html

If what you want is just to execute a series of workflows in batch mode, a simpler solution would be to ''use the VisTrails client in batch mode''. Chapter 12 of the user's guide contains detailed information and examples on that. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

== Control Flow ==

=== Note: using map ===

When using 'map', the module (or subworkflow) used as function port in the map module MUST be a function, i.e., it can only define 1 output port. [http://www.vistrails.org/usersguide/controlflow.html#the-map-operator The Map Operator]

== Spreadsheet ==

Where ''pipeline'' is a version number or a tag.

=== How can I save an image from the spreadsheet? ===

While having the focus on a spreadsheet cell and select the camera on the toolbar to take a snapshot. The system will prompt you for the location and file name where it should be saved. The other icons can be used for saving multiple images that can be used for generating an animation on demand. A whole sheet can also be saved by selecting Export (either from the menu or from the toolbar). [http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet-image Saving a Spreadsheet Image]

===Is it possible to save the complete state of the spreadsheet?===
[http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet Saving a Spreadsheet]

=== Can I view multiple sheets at the same time? ===

Yes. Each sheet on the spreadsheet can be displayed as a dock widget separated from the main spreadsheet window by dragging its tab name out of the tab bar at the bottom of the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Then, how can I put back a separated sheet? ===

A sheet can be docked back to the main window by dragging it back to the tab bar or double-click on its title bar. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== How can I order sheets on the spreadsheet? ===

This can be done by dragging the sheet name on the bottom top bar and drop it to the right place. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Can I control where a cell will be placed on the spreadsheet window? ===

By default, an unoccupied cell on the active sheet will be chosen to display the result. However, you can specify exactly in the pipeline where a spreadsheet cell will be placed by using CellLocation and SheetReference. CellLocation specifies the location (row and column) of a cell when connecting to a spreadsheet cell (VTKCell, ImageViewerCell, ...). Similarly, a SheetReference module (when connecting to a CellLocation) will specify which sheet the cell will be put on given its name, minimum row size and minimum column size. There is an example of this in examples/vtk.xml (select the version below Double Renderer). [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

=== How do I output results to the spreadsheet? ===

By inspecting the VisTrails Spreadsheet package (in the list of packages, to the left of the pipeline builder), you
can see there are built-in cells for different kinds of data, e.g., RichTextCell to display HTML and plain text. op
You (the user) can also define new cell types to display application-specific data. For example, we have developed
VtkCell, MplFigureCell, and OpenGLCell. It is possible to display pretty much anything on the Spreadsheet! [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

Examples of writing cell modules can be found in:
RichTextCell: packages/spreadsheet/widgets/richtext/richtext.py
VTK: packages/vtk/vtkcell.py

Here is the summary of some requirements on a cell widget:

(1) It must be a Qt widget. It should inherit from spreadsheet_cell.QCellWidget in the spreadsheet package. Although any Qt Widget would work, certain features such as animation will not be available (without rewriting it).

(2) It must re-implement the updateContents() function to take a set of inputs (usually coming from input ports of a wrapper Module) and display on the cells. VisTrails uses this function to update/reuse cells on the spreadsheet when new data comes in.

(3) It needs a wrapper VisTrails Module (inherited from basic_widgets.SpreadsheetCell of the spreadsheet package). Inside the compute() method of this module, it may call self.display(CellWidgetType, (inputs)) to trigger the display event on the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#advanced-cell-options Advanced Cell Options]

=== How do I control the default number of cells in the spreadsheet? ===

You can configure the rowCount and colCount using the preferences dialog. Just go to the Module Packages tab, select spreadsheet in the "Enabled packages" and press the Configure button. Then a list of all the configuration options for the spreadsheet will show up. [http://www.vistrails.org/usersguide/spreadsheet.html#custom-layout-options Custom Layout Options]

=== Is it possible to launch a web browser from the vistrails spreadsheet? We would like to output several urls from a parameter sweep and then have the option to click on each one to view the resulting page. I can view the page within the spreadsheet, but it is really too crowded.===

Currently, there isn't a widget that provides exactly this functionality, but I can think of a few solutions that may work for you:

(1) You can use parameter exploration to generate multiple sheets so you might have an exploration that opens each page in a new sheet. Use the third column/dimension in the exploration interface to have a parameter span sheets.

(2) The spreadsheet is extensible so you can write a custom spreadsheet cell widget that has a button or label with the desired link (a QLabel with openExternalLinks set to True, for example).

(3) You can tweak the existing RichTextCell be adding the line "self.browser.setOpenExternalLinks(True)" at line 63 of the source file "vistrails/packages/spreadsheet/widgets/richtext/richtext.py". Then, if your workflow creates a file with html markup text like "<a href="http://www.vistrails.org">VisTrails</a>" connected to a RichTextCell, clicking on the rendered link in the cell will open it in a web browser. You need to add the aforementioned line to the source to let Qt know that you want the link opened externally; by default, it will just issue an event that isn't processed. [http://www.vistrails.org/usersguide/spreadsheet.html#launching-a-web-browser Launching a Web Browser]

== Integrating your software into VisTrails ==

===How can I integrate my own program into VisTrails?===

The easiest way is to create a package. Writing a package is often very simple, here are instructions on how to do it: [[UsersGuideVisTrailsPackages]]

You can also dynamically generate modules. For an example see:

http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#Packages_that_generate_modules_dynamically

In particular, see the new_module call which uses python's type() function to generate new classes dynamically.

=== How do modules deal with multiple inputs in a same port? ===

(And should that even be allowed?)

For compatibility reasons, we do need to allow multiple connections to an input port. However, most package developers should never have to use this, and so we do our best to hide it. the default behavior for getting inputs from a port, then, is to always return a single input.

If on your module you need multiple inputs connected to a single port, use the 'forceGetInputListFromPort' method. It will return a list of all the data items coming through the port. The spreadsheet package uses this feature, so look there for usage examples (vistrails/packages/spreadsheet/basic_widgets.py)
[http://www.vistrails.org/usersguide/packages.html#configuring-ports Configuring Ports]

=== Are there mechanisms for attaching widgets to different modules/parameters? ===

Right now, we have a mechanism for putting a specific widget for an input port. For example, if a port is SetColor(red, green, blue), we can put a color wheel widget there. Or we can also replace the SetFileName port with a File Widget. However, this is not per parameter (only per port). We are currently working on this problem.

=== Can I organize my package so it appears hierarchical in the module palette? ===

Yes. Use the namespace keyword argument when adding the module to the registry. For example,

registry.add_module(MyModule, namespace='MyNamespace')
[http://www.vistrails.org/usersguide/packages.html#module-hierarchy-and-visibility Module Hierarchy and Visibility]

=== Can I nest namespaces? ===

Yes. Use the '|' character to separate different the hierarchy. For example,

registry.add_module(MyModule, namespace='ParentNamespace|ChildNamespace')

[http://www.vistrails.org/usersguide/packages.html#module-hierarchy-and-visibility Module Hierarchy and Visibility]

=== Are there shortcuts for registry initialization? ===

Yes. If you define _modules as a list of classes in the __init__.py file of your package, VisTrails will attempt to load all classes specified as modules. You can provide add_module options as keyword arguments by specifying a tuple (class, kwargs) in the list. For example:

_modules = [MyModule1, (MyModule2, {'namespace': 'MyNamespace'})]

In addition, you need to identify the ports of your modules as a field in your class by defining _input_ports and _output_ports lists. Here, the items in each list must be tuples of the form (portName, portSignature, optional=False, sort_key=-1). For example:

class MyModule(Module):
def compute(self):
pass

_input_ports = [('firstInput', String), ('secondInput', Integer, True)]
_output_ports = [('firstOutput', String), ('secondOutput', String)]

[http://www.vistrails.org/usersguide/packages.html#options-for-configuring-modules-and-ports Options for Configuring Modules and Ports]

=== Can I define ports to be of types that I do not import into my package? ===

Yes. You can pass an identifier string as the portSignature instead. The port_signature string is defined by:

<module_string> := <package_identifier>:[<namespace>|]<module_name>,
<port_signature> := (<module_string>*)

For example,

registry.add_input_port(MyModule, 'myInputPort', '(edu.utah.sci.vistrails.basic:String)')

or

_input_ports = [('myInputPort', '(edu.utah.sci.vistrails.basic:String)')]

[http://www.vistrails.org/usersguide/packages.html#configuring-ports Configuring Ports]

=== What do I need to change in my package to make it reloadable (new in v1.4.2)? ===
See [http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#How_to_make_you_package_reloadable UsersGuideVisTrailsPackages] for an explanation.

=== Can I add default values or labels for parameters? ===

Yes. Versions 1.4 and greater support these features. See [[UsersGuideVisTrailsPackages#Adding_default_values_and.2For_labels_for_parameters | UsersGuideVisTrailsPacakges]] for more details.
[http://www.vistrails.org/usersguide/packages.html#configuring-ports Configuring Ports]

=== I want to write a module to load HDF data whose output (e.g., data, string) varies according to the input I give it. Is is possible to do this in VisTrails, and if yes, how can I do that? Ideally, I would like to avoid having to change the connection of my output every time I change the input. ===

There are a few ways to tackle this - each has it's own benefits and pitfalls. Firstly, module connections do respect class hierarchies as we're familiar with in object oriented languages. For instance, A module can output a Constant of which String, Float, Integer, etc are specifications. In this way, you can have a subclass of something like HDFData be passed out of the module and the connections will be established regardless of the sub-type. This is a bit dangerous though. Modules downstream of such a class may not really know how to operate on certain types derived from the super-class. Extreme care must be taken both when creating the modules as well as connecting them to prevent things like this from happening.

A second method that I employ in several different packages is the idea of a container class. For instance, the NumSciPy package uses a relatively generic container "Numpy Array" to encapsulate the data. Of course, these encapsulating objects can store dictionaries that other modules can easily access and understand how to operate on. Although this method is slightly more work, the benefits of a stricter typing of ports is beneficial - particularly upon interfacing with other packages that may depend on strongly typed constants (for example).
[http://www.vistrails.org/usersguide/packages.html#configuring-ports Configuring Ports]

=== I need to determine, at run-time, whether or not a "child" module is attached to the output port of a "parent" module. (I do not specifically need to know which child; just if there is one). ===

The outputPorts dictionary of the base Module stores this information. Thus, you should be able to check

("myPortName" in self.outputPorts)

on the parent module to check if there are any downstream connections from the port "myPortName". This might be used, for example, to only set results for output ports that will be used. ***Note***, however, that the caching algorithm assumes that all outputs are set so adding a new connection to a previously unconnected output port will not work as desired if that module is cached. For this reason, I would currently recommend making such a module not cacheable. Another possibility is overriding the update() method to check the outputPorts and set the upToDate flag if they are not equal. In a single, limited test, this seemed to work, but be warned that it is not fully tested. Here is an example:

class TestModule(Module):
_output_ports = [('a1', '(edu.utah.sci.vistrails.basic:String)'),
('a2', '(edu.utah.sci.vistrails.basic:String)')]
def __init__(self):
Module.__init__(self)
self._cached_output_ports = set()

def update(self):
if len(set(self.outputPorts) - self._cached_output_ports) > 0:
self.upToDate = False
Module.update(self)
[http://www.vistrails.org/usersguide/packages.html#configuring-ports Configuring Ports]

def compute(self):
if "a1" in self.outputPorts:
self.setResult("a1", "test")
if "a2" in self.outputPorts:
self.setResult("a2", "test2")
self._cached_output_ports = set(self.outputPorts)

=== How can I make a module not display in the modules list? ===

You should set the abstract parameter to True when adding the module to the registry. Using the original syntax, this looks like:

def initialize():
reg = core.modules.module_registry.get_module_registry()
reg.add_module(InvisibleModule, abstract=True)
# ...

With the _modules dictionary shortcut (for more details, see [http://www.vistrails.org/index.php/FAQ#Are_there_shortucts_for_registry_initialization.3F the FAQ section on this]), you include it in a kwargs dict as part of a module tuple:

_modules = [AnotherModule, (InvisibleModule, {'abstract': True})]

There is also a 'hide_descriptor' parameter that prevents the module from appearing in the module palette without declaring it to be abstract.

The technical difference between the two is that 'abstract' will not add the item to the module palette while 'hide_descriptor' does add the item but immediately hides it. If the module should never be instantiated in a workflow, declare it abstract. If you don't want users to be able to add the module to a pipeline, but you have code that may add it programmatically, declare it with hide_descriptor=True.

[http://www.vistrails.org/usersguide/packages.html#module-hierarchy-and-visibility Module Hierarchy and Visibility]

== The Console ==

=== Where should I go to find out what I can call from the console and how to import it? ===

We have tried to make some methods more accessible in the console via an api. You can import the api via <code>import api</code> in the console and see the available methods with <code>dir(api)</code>. To open a vistrail:

import api
api.open_vistrail_from_file('/Applications/VisTrails/examples/terminator.vt')

To execute a version of a workflow, you currently have to go through the controller:

api.select_version('Histogram')
api.get_current_controller().execute_current_workflow()

Currently, only a subset of VisTrails functionality is directly available from the api. However, since VisTrails is written in python, you can dig down starting with the VistrailsApplication or controller object to expose most of our internal methods. If you have suggestions for calls to be added to the api, please let us know.

One other feature that we're working on, but is still in progress is the ability to construct workflows via the console. For example:

vtk = load_package('edu.utah.sci.vistrails.vtk')
vtk.vtkDataSetReader() # adds a vtkDataSetReader module to the pipeline
# click on the new module
a = selected_modules()[0] # get the one currently selected module
a.SetFile('/vistrails/examples/data/head120.vtk') # sets the SetFile parmaeter for the data set reader
b = vtk.vtkContourFilter() # adds a vtkContourFilter module to the pipeline and saves to var b
b.SetInputConnection0(a.GetOutputPort0()) # connects a's GetOutputPort0 port to b's SetInputConnection0

== Persistence Package ==

=== How do I use the output of one workflow as the input for another using the persistence package? ===

You need to configure the persistence modules using the module's configuration dialog. After adding a <code>PersistentOutputFile</code> to the workflow, click on the triangle in the upper-right corner of the <code>PersistentOutputFile</code>, and select "Edit Configuration" from the menu that appears. In this dialog, select "Create New Reference" and give the reference a name (and any space-delimited tags). Upon running that workflow, the data will be written to the persistent store. In the second workflow where you wish to use that file, add a <code>PersistentInputFile</code> and go to its configuration dialog in the same manner as with the output file. In that dialog, select "Use Existing Reference" and select the data that you just added in the first workflow in the list of files below. Now, when you run that workflow, it will grab the data from the persistent store.

Here is an example: [[Media:offscreen_persistent.vt | offscreen_persistent.vt]]. Run the "persistent offscreen" workflow first, and then run the "display persistent output" to use the output of the first workflow as the input for the second.

== VTK ==

=== Given a VTK visualization, how can I generate a webpage from it? ===

Check out the html pipeline in offscreen.xml.

=== I'm trying to use VTK, but there doesn't seem to be any output. What is wrong? ===

To use VTK on VisTrails, you need a slightly different way of connecting the renderer modules. Instead of using the standard RenderWindow/RenderWindowInteractor infrastructure, you simply connect the renderer to a VTKCell. The examples directory in the distribution has several VTK examples that illustrate.

=== I am trying to add a module to the workflow via Python, but how can I access vtk modules? ===

Here's an example:

import api

vtvtk = 'edu.utah.sci.vistrails.vtk'

module = api.add_module(0, 0, vtvtk, 'vtkContourFilter', '')

The third argument in add_module is the package identifier. You can find this in the "Module Packages" panel of the Preferences; just click on the package you're interested in and it will appear in the information on the right.

== matplotlib ==

=== I'm experiencing a problem with Latex labels and the matplotlib that comes with VisTrails 1.5. The script below entered to the interpreter that comes with VT is sufficient to reproduce it. ===

import matplotlib.pyplot as plt
plt.plot([1,2,3],[1,2,3])
plt.xlabel("$foo$")

Remove your ~/.matplotlib folder and re-start VisTrails

== VisTrails Development ==

=== I would like to build VisTrails from source. Are there instructions on how to do this? ===

Yes! Take a look at http://www.vistrails.org/index.php/Mac_Intel_Instructions
[http://www.vistrails.org/usersguide/getting_started.html#installing-vistrails-from-source Installing VisTrails from source]

== Accessing Provenance Information ==

=== How do I access the information in the execution log? ===

The code responsible for storing execution information is located in the "core/log" directories, and the code that generates much of that information is in "core/interpreter/cached.py". Modules can add execution-specific annotations to provenance via annotate() calls during execution, but much of the data (like timing and errors) is captured by the LogController and CachedInterpreter (the execution engine) objects. To analyze the log from a vistrail (.vt) file, you might have something like the following:

<code>
import core.log.log
import db.services.io

def run(fname):
# open the .vt bundle specified by the filename "fname"
bundle = db.services.io.open_vistrail_bundle_from_zip_xml(fname)[0]
# get the log filename
log_fname = bundle.vistrail.db_log_filename
if log_fname is not None:
# open the log
log = db.services.io.open_log_from_xml(log_fname, True)
# convert the log from a db object
core.log.log.Log.convert(log)
for workflow_exec in log.workflow_execs:
print 'workflow version:', workflow_exec.parent_version
print 'time started:', workflow_exec.ts_start
print 'time ended:', workflow_exec.ts_end
print 'modules executed:', [i.module_id
for i in workflow_exec.item_execs]
if __name__ == '__main__':
run("some_vistrail.vt")
</code>

You should be able to see what information is available by looking at the "core/log" classes. [http://www.vistrails.org/usersguide/log.html Accessing the Execution Log]

FAQ

2011-04-01T00:17:22Z

Ctuttle: /* I want to write a module to load HDF data whose output (e.g., data, string) varies according to the input I give it. Is is possible to do this in VisTrails, and if yes, how can I do that? Ideally, I would like to avoid having to change the connec

Also check our [[known_issues|Known Issues]] page for troubleshooting.

----
== Running workflows ==

=== How can I run a workflow using the command line? ===

('''Updated for version 1.2''') Call vistrails using the following options:

<code>python vistrails.py -b path_to_vistrails_file:pipeline </code>

where pipeline can be a version '''tag name''' or version '''id'''

'''NOTE:''' If you downloaded the MacOS X bundle, you can run vistrails from the command line via the following commands in the Terminal. Change the current directory to wherever VisTrails was installed (often <code>/Applications</code>), and then type:

<code>Vistrails.app/Contents/MacOS/vistrails [<cmd_line_options>]</code> [http://www.vistrails.org/usersguide/batch.html#running-a-specific-workflow-in-batch-mode Running a Specific Workflow in Batch Mode]

=== Using the command line, we'd like to execute a workflow multiple times, with slightly different parameters, and create a series of output files. Is this possible?===

('''Updated for version 1.2''') We can change parameters that have an alias through the command line.

For example, offscreen pipeline in offscreen.vt always creates the file called image.png. If you want generate it with a different filename:

<code>python vistrails.py -b ../examples/offscreen.vt:offscreen -a"filename=other.png"</code>

<code>filename</code> in the example above is the alias name assigned to the parameter in the <code>value</code> method inside the String module. When running a pipeline from the command line, VisTrails will try to start the spreadsheet automatically if the pipeline requires it. For example, this other execution will also start the spreadsheet (attention to how $ characters are escaped when running on bash):

<code>python vistrails.py -b ../examples/head.vt:aliases -a"isovalue=30\$&\$diffuse_color=0.8,0.4,0.2"</code>

You can also execute more than one pipeline on the command line:

<code>python vistrails.py -b ../examples/head.vt:aliases ../examples/spx.vt:spx \
-a"isovalue=30"</code>

Use the -a parameter only once regardless the number of pipelines. [http://www.vistrails.org/usersguide/batch.html#running-a-workflow-with-specific-parameters Running a Workflow with Specific Parameters]

=== I can load a vistrail, and the version tree shows up fine. However, no pipelines appear when I click on a version. What gives? ===

The most likely reason is that the vistrail uses a package that is not registered with VisTrails. You need to identify the needed package and add it to your <code>.vistrails/startup.py</code>. A single line like the following should be enough:

addPackage('enter_package_name_here')

Some packages might need more information. For example:

addPackage('afront', executable_path='/path/to/afront')

Refer to the package documentation for details. The one inconvenient step is that currently there's no automated way to describe what is the missing package. We're working on this feature for future releases.

=== I have a workflow that reads a file and then does some processing. The first time it runs, it executes correctly. But in subsequent, nothing happens. ===

VisTrails caches by default, so after a workflow is executed, if none of its parameters change, it won't be executed again.

If a workflow reads a file using the basic module File, VisTrails does check whether the file was modified since the last run. It does so by keeping a signature that is based on the modification time of the file. And if the file was modified, the File module and all downstream modules (the ones which depend on File) will be executed.

''Note: If you would like your input and output data to be versioned, you can use the Persistence package.''

If you do not want VisTrails to cache executions, you can turn off caching: go to Menu Edit -> Preferences and in the General Configuration tab, change Cache execution results to Never. [http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

=== Can VisTrails execute workflows in parallel? ===

The VisTrails server can only execute pipelines in parallel if there's more than one instance of VisTrails running. The command

<code> self.rpcserver = ThreadedXMLRPCServer((self.temp_xml_rpc_options.server, self.temp_xml_rpc_options.port))</code>

starts a multithreaded version of the XML-RPC server, so it will create a thread for each request received by the server. The problem is that Qt/PyQT doesn't allow these multiple threads create GUI objects, only in the main thread. To overcome this limitation, the multithreaded version can instantiate other single threaded versions of VisTrails and put them in a queue, so workflow executions and other GUI-related requests, such as generating workflow graphs and history trees can be forwarded to this queue, and each instance takes turns in answering the request. If the results are in the cache, the multithreaded version answers the requests directly.

Note that this infrastructure works on Linux only. To make this work on Windows, you have to create a script similar to start_vistrails_xvfb.sh (located in the scripts folder) where you can send the number of other instances via command-line options to VisTrails. The command line options are:

python vistrails_server.py -T <ADDRESS> -R <PORT> -O<NUMBER_OF_OTHER_VISTRAILS_INSTANCES> [-M]&

If you want the main vistrails instance to be multithreaded, use the -M at the end.

After creating this script, update function start_other_instances in vistrails/gui/application_server.py lines 1007-1023 and set the script variable to point to your script. You may also have to change the arguments sent to your script (line 1016: for example, you don't need to set a virtual display). You will need to change the path to the stop_vistrails_server.py script (on line 1026) according to your installation path. [http://www.vistrails.org/usersguide/batch.html#executing-workflows-in-parallel Executing Workflows in Parallel]

=== When a workflow is executed, what do the colors mean? ===

- lilac: module was notexecuted

- yellow: module is currently being executed

- green: module was successfully executed

- orange: module was cached

- red: the execution of the module failed

[http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

== Building workflows ==

=== Is there a way to give each widget a "display name" in addition to the module name at the center of the widget? ===

Yes, a "display name" can be assigned to a module by selecting the triangle in its top right corner to open a popup menu and selecting the Set Module Label... menu item. You will then be prompted to enter the "display name". [http://www.vistrails.org/usersguide/creating.html#changing-module-labels Changing Module Labels]

=== Is there a way to re-center the picture-in-picture (PiP) view? ===

Yes. If you click on the PIP window to bring it to focus, you can press Ctrl-R (or Command-R on Mac) to re-center the PiP window. [http://www.vistrails.org/usersguide/getting_started.html#vistrails-interaction Vistrails Interaction]

=== How do I search for a literal "?" (question mark) in the search box in the Property panel? ===

Since we allow regular expressions in our search box, question marks are treated as meta-characters. Thus, searching for "?" returns everything and "abc?" will return everything containing "abc". You need to use "\?" instead to search for "?". So the search for "??" would be "\?\?". [http://www.vistrails.org/usersguide/querying.html#textual-queries Textual Queries]

== Using VisTrails as a server ==

=== What is the VisTrails server-mode? ===

Using the VisTrails server mode, it is possible to ''execute workflows and control VisTrails through another application''. For example, the CrowdLabs Web portal (http://www.crowdlabs.org) accesses a VisTrails sever to execute workflows, retrieve and display vistrail trees and workflows. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

=== How do I execute workflows and control VisTrails through another application? ===

The way you access the server is by doing XML-RPC calls. In the current VisTrails release, we include a set of PHP scripts that can talk to a VisTrails server instance. They are in "extensions/http" folder. The files are reasonably well documented. Also, it should be not difficult to create python scripts to access the server (just use xmlrpclib module).

Note that the VisTrails server requires the provenance and workflows to be in a database. More detailed instructions on how to setup the server and the database
are available here:

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_server_setup.html

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_database_setup.html

If what you want is just to execute a series of workflows in batch mode, a simpler solution would be to ''use the VisTrails client in batch mode''. Chapter 12 of the user's guide contains detailed information and examples on that. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

== Control Flow ==

=== Note: using map ===

When using 'map', the module (or subworkflow) used as function port in the map module MUST be a function, i.e., it can only define 1 output port. [http://www.vistrails.org/usersguide/controlflow.html#the-map-operator The Map Operator]

== Spreadsheet ==

Where ''pipeline'' is a version number or a tag.

=== How can I save an image from the spreadsheet? ===

While having the focus on a spreadsheet cell and select the camera on the toolbar to take a snapshot. The system will prompt you for the location and file name where it should be saved. The other icons can be used for saving multiple images that can be used for generating an animation on demand. A whole sheet can also be saved by selecting Export (either from the menu or from the toolbar). [http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet-image Saving a Spreadsheet Image]

===Is it possible to save the complete state of the spreadsheet?===
[http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet Saving a Spreadsheet]

=== Can I view multiple sheets at the same time? ===

Yes. Each sheet on the spreadsheet can be displayed as a dock widget separated from the main spreadsheet window by dragging its tab name out of the tab bar at the bottom of the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Then, how can I put back a separated sheet? ===

A sheet can be docked back to the main window by dragging it back to the tab bar or double-click on its title bar. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== How can I order sheets on the spreadsheet? ===

This can be done by dragging the sheet name on the bottom top bar and drop it to the right place. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Can I control where a cell will be placed on the spreadsheet window? ===

By default, an unoccupied cell on the active sheet will be chosen to display the result. However, you can specify exactly in the pipeline where a spreadsheet cell will be placed by using CellLocation and SheetReference. CellLocation specifies the location (row and column) of a cell when connecting to a spreadsheet cell (VTKCell, ImageViewerCell, ...). Similarly, a SheetReference module (when connecting to a CellLocation) will specify which sheet the cell will be put on given its name, minimum row size and minimum column size. There is an example of this in examples/vtk.xml (select the version below Double Renderer). [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

=== How do I output results to the spreadsheet? ===

By inspecting the VisTrails Spreadsheet package (in the list of packages, to the left of the pipeline builder), you
can see there are built-in cells for different kinds of data, e.g., RichTextCell to display HTML and plain text. op
You (the user) can also define new cell types to display application-specific data. For example, we have developed
VtkCell, MplFigureCell, and OpenGLCell. It is possible to display pretty much anything on the Spreadsheet! [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

Examples of writing cell modules can be found in:
RichTextCell: packages/spreadsheet/widgets/richtext/richtext.py
VTK: packages/vtk/vtkcell.py

Here is the summary of some requirements on a cell widget:

(1) It must be a Qt widget. It should inherit from spreadsheet_cell.QCellWidget in the spreadsheet package. Although any Qt Widget would work, certain features such as animation will not be available (without rewriting it).

(2) It must re-implement the updateContents() function to take a set of inputs (usually coming from input ports of a wrapper Module) and display on the cells. VisTrails uses this function to update/reuse cells on the spreadsheet when new data comes in.

(3) It needs a wrapper VisTrails Module (inherited from basic_widgets.SpreadsheetCell of the spreadsheet package). Inside the compute() method of this module, it may call self.display(CellWidgetType, (inputs)) to trigger the display event on the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#advanced-cell-options Advanced Cell Options]

=== How do I control the default number of cells in the spreadsheet? ===

You can configure the rowCount and colCount using the preferences dialog. Just go to the Module Packages tab, select spreadsheet in the "Enabled packages" and press the Configure button. Then a list of all the configuration options for the spreadsheet will show up. [http://www.vistrails.org/usersguide/spreadsheet.html#custom-layout-options Custom Layout Options]

=== Is it possible to launch a web browser from the vistrails spreadsheet? We would like to output several urls from a parameter sweep and then have the option to click on each one to view the resulting page. I can view the page within the spreadsheet, but it is really too crowded.===

Currently, there isn't a widget that provides exactly this functionality, but I can think of a few solutions that may work for you:

(1) You can use parameter exploration to generate multiple sheets so you might have an exploration that opens each page in a new sheet. Use the third column/dimension in the exploration interface to have a parameter span sheets.

(2) The spreadsheet is extensible so you can write a custom spreadsheet cell widget that has a button or label with the desired link (a QLabel with openExternalLinks set to True, for example).

(3) You can tweak the existing RichTextCell be adding the line "self.browser.setOpenExternalLinks(True)" at line 63 of the source file "vistrails/packages/spreadsheet/widgets/richtext/richtext.py". Then, if your workflow creates a file with html markup text like "<a href="http://www.vistrails.org">VisTrails</a>" connected to a RichTextCell, clicking on the rendered link in the cell will open it in a web browser. You need to add the aforementioned line to the source to let Qt know that you want the link opened externally; by default, it will just issue an event that isn't processed. [http://www.vistrails.org/usersguide/spreadsheet.html#launching-a-web-browser Launching a Web Browser]

== Integrating your software into VisTrails ==

===How can I integrate my own program into VisTrails?===

The easiest way is to create a package. Writing a package is often very simple, here are instructions on how to do it: [[UsersGuideVisTrailsPackages]]

You can also dynamically generate modules. For an example see:

http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#Packages_that_generate_modules_dynamically

In particular, see the new_module call which uses python's type() function to generate new classes dynamically.

=== How do modules deal with multiple inputs in a same port? ===

(And should that even be allowed?)

For compatibility reasons, we do need to allow multiple connections to an input port. However, most package developers should never have to use this, and so we do our best to hide it. the default behavior for getting inputs from a port, then, is to always return a single input.

If on your module you need multiple inputs connected to a single port, use the 'forceGetInputListFromPort' method. It will return a list of all the data items coming through the port. The spreadsheet package uses this feature, so look there for usage examples (vistrails/packages/spreadsheet/basic_widgets.py)
[http://www.vistrails.org/usersguide/packages.html#configuring-ports Configuring Ports]

=== Are there mechanisms for attaching widgets to different modules/parameters? ===

Right now, we have a mechanism for putting a specific widget for an input port. For example, if a port is SetColor(red, green, blue), we can put a color wheel widget there. Or we can also replace the SetFileName port with a File Widget. However, this is not per parameter (only per port). We are currently working on this problem.

=== Can I organize my package so it appears hierarchical in the module palette? ===

Yes. Use the namespace keyword argument when adding the module to the registry. For example,

registry.add_module(MyModule, namespace='MyNamespace')
[http://www.vistrails.org/usersguide/packages.html#module-hierarchy-and-visibility Module Hierarchy and Visibility]

=== Can I nest namespaces? ===

Yes. Use the '|' character to separate different the hierarchy. For example,

registry.add_module(MyModule, namespace='ParentNamespace|ChildNamespace')

[http://www.vistrails.org/usersguide/packages.html#module-hierarchy-and-visibility Module Hierarchy and Visibility]

=== Are there shortcuts for registry initialization? ===

Yes. If you define _modules as a list of classes in the __init__.py file of your package, VisTrails will attempt to load all classes specified as modules. You can provide add_module options as keyword arguments by specifying a tuple (class, kwargs) in the list. For example:

_modules = [MyModule1, (MyModule2, {'namespace': 'MyNamespace'})]

In addition, you need to identify the ports of your modules as a field in your class by defining _input_ports and _output_ports lists. Here, the items in each list must be tuples of the form (portName, portSignature, optional=False, sort_key=-1). For example:

class MyModule(Module):
def compute(self):
pass

_input_ports = [('firstInput', String), ('secondInput', Integer, True)]
_output_ports = [('firstOutput', String), ('secondOutput', String)]

[http://www.vistrails.org/usersguide/packages.html#options-for-configuring-modules-and-ports Options for Configuring Modules and Ports]

=== Can I define ports to be of types that I do not import into my package? ===

Yes. You can pass an identifier string as the portSignature instead. The port_signature string is defined by:

<module_string> := <package_identifier>:[<namespace>|]<module_name>,
<port_signature> := (<module_string>*)

For example,

registry.add_input_port(MyModule, 'myInputPort', '(edu.utah.sci.vistrails.basic:String)')

or

_input_ports = [('myInputPort', '(edu.utah.sci.vistrails.basic:String)')]

[http://www.vistrails.org/usersguide/packages.html#configuring-ports Configuring Ports]

=== What do I need to change in my package to make it reloadable (new in v1.4.2)? ===
See [http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#How_to_make_you_package_reloadable UsersGuideVisTrailsPackages] for an explanation.

=== Can I add default values or labels for parameters? ===

Yes. Versions 1.4 and greater support these features. See [[UsersGuideVisTrailsPackages#Adding_default_values_and.2For_labels_for_parameters | UsersGuideVisTrailsPacakges]] for more details.
[http://www.vistrails.org/usersguide/packages.html#configuring-ports Configuring Ports]

=== I want to write a module to load HDF data whose output (e.g., data, string) varies according to the input I give it. Is is possible to do this in VisTrails, and if yes, how can I do that? Ideally, I would like to avoid having to change the connection of my output every time I change the input. ===

There are a few ways to tackle this - each has it's own benefits and pitfalls. Firstly, module connections do respect class hierarchies as we're familiar with in object oriented languages. For instance, A module can output a Constant of which String, Float, Integer, etc are specifications. In this way, you can have a subclass of something like HDFData be passed out of the module and the connections will be established regardless of the sub-type. This is a bit dangerous though. Modules downstream of such a class may not really know how to operate on certain types derived from the super-class. Extreme care must be taken both when creating the modules as well as connecting them to prevent things like this from happening.

A second method that I employ in several different packages is the idea of a container class. For instance, the NumSciPy package uses a relatively generic container "Numpy Array" to encapsulate the data. Of course, these encapsulating objects can store dictionaries that other modules can easily access and understand how to operate on. Although this method is slightly more work, the benefits of a stricter typing of ports is beneficial - particularly upon interfacing with other packages that may depend on strongly typed constants (for example).
[http://www.vistrails.org/usersguide/packages.html#configuring-ports Configuring Ports]

=== I need to determine, at run-time, whether or not a "child" module is attached to the output port of a "parent" module. (I do not specifically need to know which child; just if there is one). ===

The outputPorts dictionary of the base Module stores this information. Thus, you should be able to check

("myPortName" in self.outputPorts)

on the parent module to check if there are any downstream connections from the port "myPortName". This might be used, for example, to only set results for output ports that will be used. ***Note***, however, that the caching algorithm assumes that all outputs are set so adding a new connection to a previously unconnected output port will not work as desired if that module is cached. For this reason, I would currently recommend making such a module not cacheable. Another possibility is overriding the update() method to check the outputPorts and set the upToDate flag if they are not equal. In a single, limited test, this seemed to work, but be warned that it is not fully tested. Here is an example:

class TestModule(Module):
_output_ports = [('a1', '(edu.utah.sci.vistrails.basic:String)'),
('a2', '(edu.utah.sci.vistrails.basic:String)')]
def __init__(self):
Module.__init__(self)
self._cached_output_ports = set()

def update(self):
if len(set(self.outputPorts) - self._cached_output_ports) > 0:
self.upToDate = False
Module.update(self)

def compute(self):
if "a1" in self.outputPorts:
self.setResult("a1", "test")
if "a2" in self.outputPorts:
self.setResult("a2", "test2")
self._cached_output_ports = set(self.outputPorts)

=== How can I make a module not display in the modules list? ===

You should set the abstract parameter to True when adding the module to the registry. Using the original syntax, this looks like:

def initialize():
reg = core.modules.module_registry.get_module_registry()
reg.add_module(InvisibleModule, abstract=True)
# ...

With the _modules dictionary shortcut (for more details, see [http://www.vistrails.org/index.php/FAQ#Are_there_shortucts_for_registry_initialization.3F the FAQ section on this]), you include it in a kwargs dict as part of a module tuple:

_modules = [AnotherModule, (InvisibleModule, {'abstract': True})]

There is also a 'hide_descriptor' parameter that prevents the module from appearing in the module palette without declaring it to be abstract.

The technical difference between the two is that 'abstract' will not add the item to the module palette while 'hide_descriptor' does add the item but immediately hides it. If the module should never be instantiated in a workflow, declare it abstract. If you don't want users to be able to add the module to a pipeline, but you have code that may add it programmatically, declare it with hide_descriptor=True.

[http://www.vistrails.org/usersguide/packages.html#module-hierarchy-and-visibility Module Hierarchy and Visibility]

== The Console ==

=== Where should I go to find out what I can call from the console and how to import it? ===

We have tried to make some methods more accessible in the console via an api. You can import the api via <code>import api</code> in the console and see the available methods with <code>dir(api)</code>. To open a vistrail:

import api
api.open_vistrail_from_file('/Applications/VisTrails/examples/terminator.vt')

To execute a version of a workflow, you currently have to go through the controller:

api.select_version('Histogram')
api.get_current_controller().execute_current_workflow()

Currently, only a subset of VisTrails functionality is directly available from the api. However, since VisTrails is written in python, you can dig down starting with the VistrailsApplication or controller object to expose most of our internal methods. If you have suggestions for calls to be added to the api, please let us know.

One other feature that we're working on, but is still in progress is the ability to construct workflows via the console. For example:

vtk = load_package('edu.utah.sci.vistrails.vtk')
vtk.vtkDataSetReader() # adds a vtkDataSetReader module to the pipeline
# click on the new module
a = selected_modules()[0] # get the one currently selected module
a.SetFile('/vistrails/examples/data/head120.vtk') # sets the SetFile parmaeter for the data set reader
b = vtk.vtkContourFilter() # adds a vtkContourFilter module to the pipeline and saves to var b
b.SetInputConnection0(a.GetOutputPort0()) # connects a's GetOutputPort0 port to b's SetInputConnection0

== Persistence Package ==

=== How do I use the output of one workflow as the input for another using the persistence package? ===

You need to configure the persistence modules using the module's configuration dialog. After adding a <code>PersistentOutputFile</code> to the workflow, click on the triangle in the upper-right corner of the <code>PersistentOutputFile</code>, and select "Edit Configuration" from the menu that appears. In this dialog, select "Create New Reference" and give the reference a name (and any space-delimited tags). Upon running that workflow, the data will be written to the persistent store. In the second workflow where you wish to use that file, add a <code>PersistentInputFile</code> and go to its configuration dialog in the same manner as with the output file. In that dialog, select "Use Existing Reference" and select the data that you just added in the first workflow in the list of files below. Now, when you run that workflow, it will grab the data from the persistent store.

Here is an example: [[Media:offscreen_persistent.vt | offscreen_persistent.vt]]. Run the "persistent offscreen" workflow first, and then run the "display persistent output" to use the output of the first workflow as the input for the second.

== VTK ==

=== Given a VTK visualization, how can I generate a webpage from it? ===

Check out the html pipeline in offscreen.xml.

=== I'm trying to use VTK, but there doesn't seem to be any output. What is wrong? ===

To use VTK on VisTrails, you need a slightly different way of connecting the renderer modules. Instead of using the standard RenderWindow/RenderWindowInteractor infrastructure, you simply connect the renderer to a VTKCell. The examples directory in the distribution has several VTK examples that illustrate.

=== I am trying to add a module to the workflow via Python, but how can I access vtk modules? ===

Here's an example:

import api

vtvtk = 'edu.utah.sci.vistrails.vtk'

module = api.add_module(0, 0, vtvtk, 'vtkContourFilter', '')

The third argument in add_module is the package identifier. You can find this in the "Module Packages" panel of the Preferences; just click on the package you're interested in and it will appear in the information on the right.

== matplotlib ==

=== I'm experiencing a problem with Latex labels and the matplotlib that comes with VisTrails 1.5. The script below entered to the interpreter that comes with VT is sufficient to reproduce it. ===

import matplotlib.pyplot as plt
plt.plot([1,2,3],[1,2,3])
plt.xlabel("$foo$")

Remove your ~/.matplotlib folder and re-start VisTrails

== VisTrails Development ==

=== I would like to build VisTrails from source. Are there instructions on how to do this? ===

Yes! Take a look at http://www.vistrails.org/index.php/Mac_Intel_Instructions
[http://www.vistrails.org/usersguide/getting_started.html#installing-vistrails-from-source Installing VisTrails from source]

== Accessing Provenance Information ==

=== How do I access the information in the execution log? ===

The code responsible for storing execution information is located in the "core/log" directories, and the code that generates much of that information is in "core/interpreter/cached.py". Modules can add execution-specific annotations to provenance via annotate() calls during execution, but much of the data (like timing and errors) is captured by the LogController and CachedInterpreter (the execution engine) objects. To analyze the log from a vistrail (.vt) file, you might have something like the following:

<code>
import core.log.log
import db.services.io

def run(fname):
# open the .vt bundle specified by the filename "fname"
bundle = db.services.io.open_vistrail_bundle_from_zip_xml(fname)[0]
# get the log filename
log_fname = bundle.vistrail.db_log_filename
if log_fname is not None:
# open the log
log = db.services.io.open_log_from_xml(log_fname, True)
# convert the log from a db object
core.log.log.Log.convert(log)
for workflow_exec in log.workflow_execs:
print 'workflow version:', workflow_exec.parent_version
print 'time started:', workflow_exec.ts_start
print 'time ended:', workflow_exec.ts_end
print 'modules executed:', [i.module_id
for i in workflow_exec.item_execs]
if __name__ == '__main__':
run("some_vistrail.vt")
</code>

You should be able to see what information is available by looking at the "core/log" classes. [http://www.vistrails.org/usersguide/log.html Accessing the Execution Log]

FAQ

2011-04-01T00:05:05Z

Ctuttle: /* Can I define ports to be of types that I do not import into my package? */

Also check our [[known_issues|Known Issues]] page for troubleshooting.

----
== Running workflows ==

=== How can I run a workflow using the command line? ===

('''Updated for version 1.2''') Call vistrails using the following options:

<code>python vistrails.py -b path_to_vistrails_file:pipeline </code>

where pipeline can be a version '''tag name''' or version '''id'''

'''NOTE:''' If you downloaded the MacOS X bundle, you can run vistrails from the command line via the following commands in the Terminal. Change the current directory to wherever VisTrails was installed (often <code>/Applications</code>), and then type:

<code>Vistrails.app/Contents/MacOS/vistrails [<cmd_line_options>]</code> [http://www.vistrails.org/usersguide/batch.html#running-a-specific-workflow-in-batch-mode Running a Specific Workflow in Batch Mode]

=== Using the command line, we'd like to execute a workflow multiple times, with slightly different parameters, and create a series of output files. Is this possible?===

('''Updated for version 1.2''') We can change parameters that have an alias through the command line.

For example, offscreen pipeline in offscreen.vt always creates the file called image.png. If you want generate it with a different filename:

<code>python vistrails.py -b ../examples/offscreen.vt:offscreen -a"filename=other.png"</code>

<code>filename</code> in the example above is the alias name assigned to the parameter in the <code>value</code> method inside the String module. When running a pipeline from the command line, VisTrails will try to start the spreadsheet automatically if the pipeline requires it. For example, this other execution will also start the spreadsheet (attention to how $ characters are escaped when running on bash):

<code>python vistrails.py -b ../examples/head.vt:aliases -a"isovalue=30\$&\$diffuse_color=0.8,0.4,0.2"</code>

You can also execute more than one pipeline on the command line:

<code>python vistrails.py -b ../examples/head.vt:aliases ../examples/spx.vt:spx \
-a"isovalue=30"</code>

Use the -a parameter only once regardless the number of pipelines. [http://www.vistrails.org/usersguide/batch.html#running-a-workflow-with-specific-parameters Running a Workflow with Specific Parameters]

=== I can load a vistrail, and the version tree shows up fine. However, no pipelines appear when I click on a version. What gives? ===

The most likely reason is that the vistrail uses a package that is not registered with VisTrails. You need to identify the needed package and add it to your <code>.vistrails/startup.py</code>. A single line like the following should be enough:

addPackage('enter_package_name_here')

Some packages might need more information. For example:

addPackage('afront', executable_path='/path/to/afront')

Refer to the package documentation for details. The one inconvenient step is that currently there's no automated way to describe what is the missing package. We're working on this feature for future releases.

=== I have a workflow that reads a file and then does some processing. The first time it runs, it executes correctly. But in subsequent, nothing happens. ===

VisTrails caches by default, so after a workflow is executed, if none of its parameters change, it won't be executed again.

If a workflow reads a file using the basic module File, VisTrails does check whether the file was modified since the last run. It does so by keeping a signature that is based on the modification time of the file. And if the file was modified, the File module and all downstream modules (the ones which depend on File) will be executed.

''Note: If you would like your input and output data to be versioned, you can use the Persistence package.''

If you do not want VisTrails to cache executions, you can turn off caching: go to Menu Edit -> Preferences and in the General Configuration tab, change Cache execution results to Never. [http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

=== Can VisTrails execute workflows in parallel? ===

The VisTrails server can only execute pipelines in parallel if there's more than one instance of VisTrails running. The command

<code> self.rpcserver = ThreadedXMLRPCServer((self.temp_xml_rpc_options.server, self.temp_xml_rpc_options.port))</code>

starts a multithreaded version of the XML-RPC server, so it will create a thread for each request received by the server. The problem is that Qt/PyQT doesn't allow these multiple threads create GUI objects, only in the main thread. To overcome this limitation, the multithreaded version can instantiate other single threaded versions of VisTrails and put them in a queue, so workflow executions and other GUI-related requests, such as generating workflow graphs and history trees can be forwarded to this queue, and each instance takes turns in answering the request. If the results are in the cache, the multithreaded version answers the requests directly.

Note that this infrastructure works on Linux only. To make this work on Windows, you have to create a script similar to start_vistrails_xvfb.sh (located in the scripts folder) where you can send the number of other instances via command-line options to VisTrails. The command line options are:

python vistrails_server.py -T <ADDRESS> -R <PORT> -O<NUMBER_OF_OTHER_VISTRAILS_INSTANCES> [-M]&

If you want the main vistrails instance to be multithreaded, use the -M at the end.

After creating this script, update function start_other_instances in vistrails/gui/application_server.py lines 1007-1023 and set the script variable to point to your script. You may also have to change the arguments sent to your script (line 1016: for example, you don't need to set a virtual display). You will need to change the path to the stop_vistrails_server.py script (on line 1026) according to your installation path. [http://www.vistrails.org/usersguide/batch.html#executing-workflows-in-parallel Executing Workflows in Parallel]

=== When a workflow is executed, what do the colors mean? ===

- lilac: module was notexecuted

- yellow: module is currently being executed

- green: module was successfully executed

- orange: module was cached

- red: the execution of the module failed

[http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

== Building workflows ==

=== Is there a way to give each widget a "display name" in addition to the module name at the center of the widget? ===

Yes, a "display name" can be assigned to a module by selecting the triangle in its top right corner to open a popup menu and selecting the Set Module Label... menu item. You will then be prompted to enter the "display name". [http://www.vistrails.org/usersguide/creating.html#changing-module-labels Changing Module Labels]

=== Is there a way to re-center the picture-in-picture (PiP) view? ===

Yes. If you click on the PIP window to bring it to focus, you can press Ctrl-R (or Command-R on Mac) to re-center the PiP window. [http://www.vistrails.org/usersguide/getting_started.html#vistrails-interaction Vistrails Interaction]

=== How do I search for a literal "?" (question mark) in the search box in the Property panel? ===

Since we allow regular expressions in our search box, question marks are treated as meta-characters. Thus, searching for "?" returns everything and "abc?" will return everything containing "abc". You need to use "\?" instead to search for "?". So the search for "??" would be "\?\?". [http://www.vistrails.org/usersguide/querying.html#textual-queries Textual Queries]

== Using VisTrails as a server ==

=== What is the VisTrails server-mode? ===

Using the VisTrails server mode, it is possible to ''execute workflows and control VisTrails through another application''. For example, the CrowdLabs Web portal (http://www.crowdlabs.org) accesses a VisTrails sever to execute workflows, retrieve and display vistrail trees and workflows. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

=== How do I execute workflows and control VisTrails through another application? ===

The way you access the server is by doing XML-RPC calls. In the current VisTrails release, we include a set of PHP scripts that can talk to a VisTrails server instance. They are in "extensions/http" folder. The files are reasonably well documented. Also, it should be not difficult to create python scripts to access the server (just use xmlrpclib module).

Note that the VisTrails server requires the provenance and workflows to be in a database. More detailed instructions on how to setup the server and the database
are available here:

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_server_setup.html

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_database_setup.html

If what you want is just to execute a series of workflows in batch mode, a simpler solution would be to ''use the VisTrails client in batch mode''. Chapter 12 of the user's guide contains detailed information and examples on that. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

== Control Flow ==

=== Note: using map ===

When using 'map', the module (or subworkflow) used as function port in the map module MUST be a function, i.e., it can only define 1 output port. [http://www.vistrails.org/usersguide/controlflow.html#the-map-operator The Map Operator]

== Spreadsheet ==

Where ''pipeline'' is a version number or a tag.

=== How can I save an image from the spreadsheet? ===

While having the focus on a spreadsheet cell and select the camera on the toolbar to take a snapshot. The system will prompt you for the location and file name where it should be saved. The other icons can be used for saving multiple images that can be used for generating an animation on demand. A whole sheet can also be saved by selecting Export (either from the menu or from the toolbar). [http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet-image Saving a Spreadsheet Image]

===Is it possible to save the complete state of the spreadsheet?===
[http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet Saving a Spreadsheet]

=== Can I view multiple sheets at the same time? ===

Yes. Each sheet on the spreadsheet can be displayed as a dock widget separated from the main spreadsheet window by dragging its tab name out of the tab bar at the bottom of the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Then, how can I put back a separated sheet? ===

A sheet can be docked back to the main window by dragging it back to the tab bar or double-click on its title bar. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== How can I order sheets on the spreadsheet? ===

This can be done by dragging the sheet name on the bottom top bar and drop it to the right place. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Can I control where a cell will be placed on the spreadsheet window? ===

By default, an unoccupied cell on the active sheet will be chosen to display the result. However, you can specify exactly in the pipeline where a spreadsheet cell will be placed by using CellLocation and SheetReference. CellLocation specifies the location (row and column) of a cell when connecting to a spreadsheet cell (VTKCell, ImageViewerCell, ...). Similarly, a SheetReference module (when connecting to a CellLocation) will specify which sheet the cell will be put on given its name, minimum row size and minimum column size. There is an example of this in examples/vtk.xml (select the version below Double Renderer). [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

=== How do I output results to the spreadsheet? ===

By inspecting the VisTrails Spreadsheet package (in the list of packages, to the left of the pipeline builder), you
can see there are built-in cells for different kinds of data, e.g., RichTextCell to display HTML and plain text. op
You (the user) can also define new cell types to display application-specific data. For example, we have developed
VtkCell, MplFigureCell, and OpenGLCell. It is possible to display pretty much anything on the Spreadsheet! [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

Examples of writing cell modules can be found in:
RichTextCell: packages/spreadsheet/widgets/richtext/richtext.py
VTK: packages/vtk/vtkcell.py

Here is the summary of some requirements on a cell widget:

(1) It must be a Qt widget. It should inherit from spreadsheet_cell.QCellWidget in the spreadsheet package. Although any Qt Widget would work, certain features such as animation will not be available (without rewriting it).

(2) It must re-implement the updateContents() function to take a set of inputs (usually coming from input ports of a wrapper Module) and display on the cells. VisTrails uses this function to update/reuse cells on the spreadsheet when new data comes in.

(3) It needs a wrapper VisTrails Module (inherited from basic_widgets.SpreadsheetCell of the spreadsheet package). Inside the compute() method of this module, it may call self.display(CellWidgetType, (inputs)) to trigger the display event on the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#advanced-cell-options Advanced Cell Options]

=== How do I control the default number of cells in the spreadsheet? ===

You can configure the rowCount and colCount using the preferences dialog. Just go to the Module Packages tab, select spreadsheet in the "Enabled packages" and press the Configure button. Then a list of all the configuration options for the spreadsheet will show up. [http://www.vistrails.org/usersguide/spreadsheet.html#custom-layout-options Custom Layout Options]

=== Is it possible to launch a web browser from the vistrails spreadsheet? We would like to output several urls from a parameter sweep and then have the option to click on each one to view the resulting page. I can view the page within the spreadsheet, but it is really too crowded.===

Currently, there isn't a widget that provides exactly this functionality, but I can think of a few solutions that may work for you:

(1) You can use parameter exploration to generate multiple sheets so you might have an exploration that opens each page in a new sheet. Use the third column/dimension in the exploration interface to have a parameter span sheets.

(2) The spreadsheet is extensible so you can write a custom spreadsheet cell widget that has a button or label with the desired link (a QLabel with openExternalLinks set to True, for example).

(3) You can tweak the existing RichTextCell be adding the line "self.browser.setOpenExternalLinks(True)" at line 63 of the source file "vistrails/packages/spreadsheet/widgets/richtext/richtext.py". Then, if your workflow creates a file with html markup text like "<a href="http://www.vistrails.org">VisTrails</a>" connected to a RichTextCell, clicking on the rendered link in the cell will open it in a web browser. You need to add the aforementioned line to the source to let Qt know that you want the link opened externally; by default, it will just issue an event that isn't processed. [http://www.vistrails.org/usersguide/spreadsheet.html#launching-a-web-browser Launching a Web Browser]

== Integrating your software into VisTrails ==

===How can I integrate my own program into VisTrails?===

The easiest way is to create a package. Writing a package is often very simple, here are instructions on how to do it: [[UsersGuideVisTrailsPackages]]

You can also dynamically generate modules. For an example see:

http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#Packages_that_generate_modules_dynamically

In particular, see the new_module call which uses python's type() function to generate new classes dynamically.

=== How do modules deal with multiple inputs in a same port? ===

(And should that even be allowed?)

For compatibility reasons, we do need to allow multiple connections to an input port. However, most package developers should never have to use this, and so we do our best to hide it. the default behavior for getting inputs from a port, then, is to always return a single input.

If on your module you need multiple inputs connected to a single port, use the 'forceGetInputListFromPort' method. It will return a list of all the data items coming through the port. The spreadsheet package uses this feature, so look there for usage examples (vistrails/packages/spreadsheet/basic_widgets.py)
[http://www.vistrails.org/usersguide/packages.html#configuring-ports Configuring Ports]

=== Are there mechanisms for attaching widgets to different modules/parameters? ===

Right now, we have a mechanism for putting a specific widget for an input port. For example, if a port is SetColor(red, green, blue), we can put a color wheel widget there. Or we can also replace the SetFileName port with a File Widget. However, this is not per parameter (only per port). We are currently working on this problem.

=== Can I organize my package so it appears hierarchical in the module palette? ===

Yes. Use the namespace keyword argument when adding the module to the registry. For example,

registry.add_module(MyModule, namespace='MyNamespace')
[http://www.vistrails.org/usersguide/packages.html#module-hierarchy-and-visibility Module Hierarchy and Visibility]

=== Can I nest namespaces? ===

Yes. Use the '|' character to separate different the hierarchy. For example,

registry.add_module(MyModule, namespace='ParentNamespace|ChildNamespace')

[http://www.vistrails.org/usersguide/packages.html#module-hierarchy-and-visibility Module Hierarchy and Visibility]

=== Are there shortcuts for registry initialization? ===

Yes. If you define _modules as a list of classes in the __init__.py file of your package, VisTrails will attempt to load all classes specified as modules. You can provide add_module options as keyword arguments by specifying a tuple (class, kwargs) in the list. For example:

_modules = [MyModule1, (MyModule2, {'namespace': 'MyNamespace'})]

In addition, you need to identify the ports of your modules as a field in your class by defining _input_ports and _output_ports lists. Here, the items in each list must be tuples of the form (portName, portSignature, optional=False, sort_key=-1). For example:

class MyModule(Module):
def compute(self):
pass

_input_ports = [('firstInput', String), ('secondInput', Integer, True)]
_output_ports = [('firstOutput', String), ('secondOutput', String)]

[http://www.vistrails.org/usersguide/packages.html#options-for-configuring-modules-and-ports Options for Configuring Modules and Ports]

=== Can I define ports to be of types that I do not import into my package? ===

Yes. You can pass an identifier string as the portSignature instead. The port_signature string is defined by:

<module_string> := <package_identifier>:[<namespace>|]<module_name>,
<port_signature> := (<module_string>*)

For example,

registry.add_input_port(MyModule, 'myInputPort', '(edu.utah.sci.vistrails.basic:String)')

or

_input_ports = [('myInputPort', '(edu.utah.sci.vistrails.basic:String)')]

[http://www.vistrails.org/usersguide/packages.html#configuring-ports Configuring Ports]

=== What do I need to change in my package to make it reloadable (new in v1.4.2)? ===
See [http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#How_to_make_you_package_reloadable UsersGuideVisTrailsPackages] for an explanation.

=== Can I add default values or labels for parameters? ===

Yes. Versions 1.4 and greater support these features. See [[UsersGuideVisTrailsPackages#Adding_default_values_and.2For_labels_for_parameters | UsersGuideVisTrailsPacakges]] for more details.
[http://www.vistrails.org/usersguide/packages.html#configuring-ports Configuring Ports]

=== I want to write a module to load HDF data whose output (e.g., data, string) varies according to the input I give it. Is is possible to do this in VisTrails, and if yes, how can I do that? Ideally, I would like to avoid having to change the connection of my output every time I change the input. ===

There are a few ways to tackle this - each has it's own benefits and pitfalls. Firstly, module connections do respect class hierarchies as we're familiar with in object oriented languages. For instance, A module can output a Constant of which String, Float, Integer, etc are specifications. In this way, you can have a subclass of something like HDFData be passed out of the module and the connections will be established regardless of the sub-type. This is a bit dangerous though. Modules downstream of such a class may not really know how to operate on certain types derived from the super-class. Extreme care must be taken both when creating the modules as well as connecting them to prevent things like this from happening.

A second method that I employ in several different packages is the idea of a container class. For instance, the NumSciPy package uses a relatively generic container "Numpy Array" to encapsulate the data. Of course, these encapsulating objects can store dictionaries that other modules can easily access and understand how to operate on. Although this method is slightly more work, the benefits of a stricter typing of ports is beneficial - particularly upon interfacing with other packages that may depend on strongly typed constants (for example).

=== I need to determine, at run-time, whether or not a "child" module is attached to the output port of a "parent" module. (I do not specifically need to know which child; just if there is one). ===

The outputPorts dictionary of the base Module stores this information. Thus, you should be able to check

("myPortName" in self.outputPorts)

on the parent module to check if there are any downstream connections from the port "myPortName". This might be used, for example, to only set results for output ports that will be used. ***Note***, however, that the caching algorithm assumes that all outputs are set so adding a new connection to a previously unconnected output port will not work as desired if that module is cached. For this reason, I would currently recommend making such a module not cacheable. Another possibility is overriding the update() method to check the outputPorts and set the upToDate flag if they are not equal. In a single, limited test, this seemed to work, but be warned that it is not fully tested. Here is an example:

class TestModule(Module):
_output_ports = [('a1', '(edu.utah.sci.vistrails.basic:String)'),
('a2', '(edu.utah.sci.vistrails.basic:String)')]
def __init__(self):
Module.__init__(self)
self._cached_output_ports = set()

def update(self):
if len(set(self.outputPorts) - self._cached_output_ports) > 0:
self.upToDate = False
Module.update(self)

def compute(self):
if "a1" in self.outputPorts:
self.setResult("a1", "test")
if "a2" in self.outputPorts:
self.setResult("a2", "test2")
self._cached_output_ports = set(self.outputPorts)

=== How can I make a module not display in the modules list? ===

You should set the abstract parameter to True when adding the module to the registry. Using the original syntax, this looks like:

def initialize():
reg = core.modules.module_registry.get_module_registry()
reg.add_module(InvisibleModule, abstract=True)
# ...

With the _modules dictionary shortcut (for more details, see [http://www.vistrails.org/index.php/FAQ#Are_there_shortucts_for_registry_initialization.3F the FAQ section on this]), you include it in a kwargs dict as part of a module tuple:

_modules = [AnotherModule, (InvisibleModule, {'abstract': True})]

There is also a 'hide_descriptor' parameter that prevents the module from appearing in the module palette without declaring it to be abstract.

The technical difference between the two is that 'abstract' will not add the item to the module palette while 'hide_descriptor' does add the item but immediately hides it. If the module should never be instantiated in a workflow, declare it abstract. If you don't want users to be able to add the module to a pipeline, but you have code that may add it programmatically, declare it with hide_descriptor=True.

[http://www.vistrails.org/usersguide/packages.html#module-hierarchy-and-visibility Module Hierarchy and Visibility]

== The Console ==

=== Where should I go to find out what I can call from the console and how to import it? ===

We have tried to make some methods more accessible in the console via an api. You can import the api via <code>import api</code> in the console and see the available methods with <code>dir(api)</code>. To open a vistrail:

import api
api.open_vistrail_from_file('/Applications/VisTrails/examples/terminator.vt')

To execute a version of a workflow, you currently have to go through the controller:

api.select_version('Histogram')
api.get_current_controller().execute_current_workflow()

Currently, only a subset of VisTrails functionality is directly available from the api. However, since VisTrails is written in python, you can dig down starting with the VistrailsApplication or controller object to expose most of our internal methods. If you have suggestions for calls to be added to the api, please let us know.

One other feature that we're working on, but is still in progress is the ability to construct workflows via the console. For example:

vtk = load_package('edu.utah.sci.vistrails.vtk')
vtk.vtkDataSetReader() # adds a vtkDataSetReader module to the pipeline
# click on the new module
a = selected_modules()[0] # get the one currently selected module
a.SetFile('/vistrails/examples/data/head120.vtk') # sets the SetFile parmaeter for the data set reader
b = vtk.vtkContourFilter() # adds a vtkContourFilter module to the pipeline and saves to var b
b.SetInputConnection0(a.GetOutputPort0()) # connects a's GetOutputPort0 port to b's SetInputConnection0

== Persistence Package ==

=== How do I use the output of one workflow as the input for another using the persistence package? ===

You need to configure the persistence modules using the module's configuration dialog. After adding a <code>PersistentOutputFile</code> to the workflow, click on the triangle in the upper-right corner of the <code>PersistentOutputFile</code>, and select "Edit Configuration" from the menu that appears. In this dialog, select "Create New Reference" and give the reference a name (and any space-delimited tags). Upon running that workflow, the data will be written to the persistent store. In the second workflow where you wish to use that file, add a <code>PersistentInputFile</code> and go to its configuration dialog in the same manner as with the output file. In that dialog, select "Use Existing Reference" and select the data that you just added in the first workflow in the list of files below. Now, when you run that workflow, it will grab the data from the persistent store.

Here is an example: [[Media:offscreen_persistent.vt | offscreen_persistent.vt]]. Run the "persistent offscreen" workflow first, and then run the "display persistent output" to use the output of the first workflow as the input for the second.

== VTK ==

=== Given a VTK visualization, how can I generate a webpage from it? ===

Check out the html pipeline in offscreen.xml.

=== I'm trying to use VTK, but there doesn't seem to be any output. What is wrong? ===

To use VTK on VisTrails, you need a slightly different way of connecting the renderer modules. Instead of using the standard RenderWindow/RenderWindowInteractor infrastructure, you simply connect the renderer to a VTKCell. The examples directory in the distribution has several VTK examples that illustrate.

=== I am trying to add a module to the workflow via Python, but how can I access vtk modules? ===

Here's an example:

import api

vtvtk = 'edu.utah.sci.vistrails.vtk'

module = api.add_module(0, 0, vtvtk, 'vtkContourFilter', '')

The third argument in add_module is the package identifier. You can find this in the "Module Packages" panel of the Preferences; just click on the package you're interested in and it will appear in the information on the right.

== matplotlib ==

=== I'm experiencing a problem with Latex labels and the matplotlib that comes with VisTrails 1.5. The script below entered to the interpreter that comes with VT is sufficient to reproduce it. ===

import matplotlib.pyplot as plt
plt.plot([1,2,3],[1,2,3])
plt.xlabel("$foo$")

Remove your ~/.matplotlib folder and re-start VisTrails

== VisTrails Development ==

=== I would like to build VisTrails from source. Are there instructions on how to do this? ===

Yes! Take a look at http://www.vistrails.org/index.php/Mac_Intel_Instructions
[http://www.vistrails.org/usersguide/getting_started.html#installing-vistrails-from-source Installing VisTrails from source]

== Accessing Provenance Information ==

=== How do I access the information in the execution log? ===

The code responsible for storing execution information is located in the "core/log" directories, and the code that generates much of that information is in "core/interpreter/cached.py". Modules can add execution-specific annotations to provenance via annotate() calls during execution, but much of the data (like timing and errors) is captured by the LogController and CachedInterpreter (the execution engine) objects. To analyze the log from a vistrail (.vt) file, you might have something like the following:

<code>
import core.log.log
import db.services.io

def run(fname):
# open the .vt bundle specified by the filename "fname"
bundle = db.services.io.open_vistrail_bundle_from_zip_xml(fname)[0]
# get the log filename
log_fname = bundle.vistrail.db_log_filename
if log_fname is not None:
# open the log
log = db.services.io.open_log_from_xml(log_fname, True)
# convert the log from a db object
core.log.log.Log.convert(log)
for workflow_exec in log.workflow_execs:
print 'workflow version:', workflow_exec.parent_version
print 'time started:', workflow_exec.ts_start
print 'time ended:', workflow_exec.ts_end
print 'modules executed:', [i.module_id
for i in workflow_exec.item_execs]
if __name__ == '__main__':
run("some_vistrail.vt")
</code>

You should be able to see what information is available by looking at the "core/log" classes. [http://www.vistrails.org/usersguide/log.html Accessing the Execution Log]

FAQ

2011-03-31T23:54:44Z

Ctuttle: /* How do modules deal with multiple inputs in a same port? */

Also check our [[known_issues|Known Issues]] page for troubleshooting.

----
== Running workflows ==

=== How can I run a workflow using the command line? ===

('''Updated for version 1.2''') Call vistrails using the following options:

<code>python vistrails.py -b path_to_vistrails_file:pipeline </code>

where pipeline can be a version '''tag name''' or version '''id'''

'''NOTE:''' If you downloaded the MacOS X bundle, you can run vistrails from the command line via the following commands in the Terminal. Change the current directory to wherever VisTrails was installed (often <code>/Applications</code>), and then type:

<code>Vistrails.app/Contents/MacOS/vistrails [<cmd_line_options>]</code> [http://www.vistrails.org/usersguide/batch.html#running-a-specific-workflow-in-batch-mode Running a Specific Workflow in Batch Mode]

=== Using the command line, we'd like to execute a workflow multiple times, with slightly different parameters, and create a series of output files. Is this possible?===

('''Updated for version 1.2''') We can change parameters that have an alias through the command line.

For example, offscreen pipeline in offscreen.vt always creates the file called image.png. If you want generate it with a different filename:

<code>python vistrails.py -b ../examples/offscreen.vt:offscreen -a"filename=other.png"</code>

<code>filename</code> in the example above is the alias name assigned to the parameter in the <code>value</code> method inside the String module. When running a pipeline from the command line, VisTrails will try to start the spreadsheet automatically if the pipeline requires it. For example, this other execution will also start the spreadsheet (attention to how $ characters are escaped when running on bash):

<code>python vistrails.py -b ../examples/head.vt:aliases -a"isovalue=30\$&\$diffuse_color=0.8,0.4,0.2"</code>

You can also execute more than one pipeline on the command line:

<code>python vistrails.py -b ../examples/head.vt:aliases ../examples/spx.vt:spx \
-a"isovalue=30"</code>

Use the -a parameter only once regardless the number of pipelines. [http://www.vistrails.org/usersguide/batch.html#running-a-workflow-with-specific-parameters Running a Workflow with Specific Parameters]

=== I can load a vistrail, and the version tree shows up fine. However, no pipelines appear when I click on a version. What gives? ===

The most likely reason is that the vistrail uses a package that is not registered with VisTrails. You need to identify the needed package and add it to your <code>.vistrails/startup.py</code>. A single line like the following should be enough:

addPackage('enter_package_name_here')

Some packages might need more information. For example:

addPackage('afront', executable_path='/path/to/afront')

Refer to the package documentation for details. The one inconvenient step is that currently there's no automated way to describe what is the missing package. We're working on this feature for future releases.

=== I have a workflow that reads a file and then does some processing. The first time it runs, it executes correctly. But in subsequent, nothing happens. ===

VisTrails caches by default, so after a workflow is executed, if none of its parameters change, it won't be executed again.

If a workflow reads a file using the basic module File, VisTrails does check whether the file was modified since the last run. It does so by keeping a signature that is based on the modification time of the file. And if the file was modified, the File module and all downstream modules (the ones which depend on File) will be executed.

''Note: If you would like your input and output data to be versioned, you can use the Persistence package.''

If you do not want VisTrails to cache executions, you can turn off caching: go to Menu Edit -> Preferences and in the General Configuration tab, change Cache execution results to Never. [http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

=== Can VisTrails execute workflows in parallel? ===

The VisTrails server can only execute pipelines in parallel if there's more than one instance of VisTrails running. The command

<code> self.rpcserver = ThreadedXMLRPCServer((self.temp_xml_rpc_options.server, self.temp_xml_rpc_options.port))</code>

starts a multithreaded version of the XML-RPC server, so it will create a thread for each request received by the server. The problem is that Qt/PyQT doesn't allow these multiple threads create GUI objects, only in the main thread. To overcome this limitation, the multithreaded version can instantiate other single threaded versions of VisTrails and put them in a queue, so workflow executions and other GUI-related requests, such as generating workflow graphs and history trees can be forwarded to this queue, and each instance takes turns in answering the request. If the results are in the cache, the multithreaded version answers the requests directly.

Note that this infrastructure works on Linux only. To make this work on Windows, you have to create a script similar to start_vistrails_xvfb.sh (located in the scripts folder) where you can send the number of other instances via command-line options to VisTrails. The command line options are:

python vistrails_server.py -T <ADDRESS> -R <PORT> -O<NUMBER_OF_OTHER_VISTRAILS_INSTANCES> [-M]&

If you want the main vistrails instance to be multithreaded, use the -M at the end.

After creating this script, update function start_other_instances in vistrails/gui/application_server.py lines 1007-1023 and set the script variable to point to your script. You may also have to change the arguments sent to your script (line 1016: for example, you don't need to set a virtual display). You will need to change the path to the stop_vistrails_server.py script (on line 1026) according to your installation path. [http://www.vistrails.org/usersguide/batch.html#executing-workflows-in-parallel Executing Workflows in Parallel]

=== When a workflow is executed, what do the colors mean? ===

- lilac: module was notexecuted

- yellow: module is currently being executed

- green: module was successfully executed

- orange: module was cached

- red: the execution of the module failed

[http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

== Building workflows ==

=== Is there a way to give each widget a "display name" in addition to the module name at the center of the widget? ===

Yes, a "display name" can be assigned to a module by selecting the triangle in its top right corner to open a popup menu and selecting the Set Module Label... menu item. You will then be prompted to enter the "display name". [http://www.vistrails.org/usersguide/creating.html#changing-module-labels Changing Module Labels]

=== Is there a way to re-center the picture-in-picture (PiP) view? ===

Yes. If you click on the PIP window to bring it to focus, you can press Ctrl-R (or Command-R on Mac) to re-center the PiP window. [http://www.vistrails.org/usersguide/getting_started.html#vistrails-interaction Vistrails Interaction]

=== How do I search for a literal "?" (question mark) in the search box in the Property panel? ===

Since we allow regular expressions in our search box, question marks are treated as meta-characters. Thus, searching for "?" returns everything and "abc?" will return everything containing "abc". You need to use "\?" instead to search for "?". So the search for "??" would be "\?\?". [http://www.vistrails.org/usersguide/querying.html#textual-queries Textual Queries]

== Using VisTrails as a server ==

=== What is the VisTrails server-mode? ===

Using the VisTrails server mode, it is possible to ''execute workflows and control VisTrails through another application''. For example, the CrowdLabs Web portal (http://www.crowdlabs.org) accesses a VisTrails sever to execute workflows, retrieve and display vistrail trees and workflows. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

=== How do I execute workflows and control VisTrails through another application? ===

The way you access the server is by doing XML-RPC calls. In the current VisTrails release, we include a set of PHP scripts that can talk to a VisTrails server instance. They are in "extensions/http" folder. The files are reasonably well documented. Also, it should be not difficult to create python scripts to access the server (just use xmlrpclib module).

Note that the VisTrails server requires the provenance and workflows to be in a database. More detailed instructions on how to setup the server and the database
are available here:

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_server_setup.html

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_database_setup.html

If what you want is just to execute a series of workflows in batch mode, a simpler solution would be to ''use the VisTrails client in batch mode''. Chapter 12 of the user's guide contains detailed information and examples on that. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

== Control Flow ==

=== Note: using map ===

When using 'map', the module (or subworkflow) used as function port in the map module MUST be a function, i.e., it can only define 1 output port. [http://www.vistrails.org/usersguide/controlflow.html#the-map-operator The Map Operator]

== Spreadsheet ==

Where ''pipeline'' is a version number or a tag.

=== How can I save an image from the spreadsheet? ===

While having the focus on a spreadsheet cell and select the camera on the toolbar to take a snapshot. The system will prompt you for the location and file name where it should be saved. The other icons can be used for saving multiple images that can be used for generating an animation on demand. A whole sheet can also be saved by selecting Export (either from the menu or from the toolbar). [http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet-image Saving a Spreadsheet Image]

===Is it possible to save the complete state of the spreadsheet?===
[http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet Saving a Spreadsheet]

=== Can I view multiple sheets at the same time? ===

Yes. Each sheet on the spreadsheet can be displayed as a dock widget separated from the main spreadsheet window by dragging its tab name out of the tab bar at the bottom of the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Then, how can I put back a separated sheet? ===

A sheet can be docked back to the main window by dragging it back to the tab bar or double-click on its title bar. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== How can I order sheets on the spreadsheet? ===

This can be done by dragging the sheet name on the bottom top bar and drop it to the right place. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Can I control where a cell will be placed on the spreadsheet window? ===

By default, an unoccupied cell on the active sheet will be chosen to display the result. However, you can specify exactly in the pipeline where a spreadsheet cell will be placed by using CellLocation and SheetReference. CellLocation specifies the location (row and column) of a cell when connecting to a spreadsheet cell (VTKCell, ImageViewerCell, ...). Similarly, a SheetReference module (when connecting to a CellLocation) will specify which sheet the cell will be put on given its name, minimum row size and minimum column size. There is an example of this in examples/vtk.xml (select the version below Double Renderer). [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

=== How do I output results to the spreadsheet? ===

By inspecting the VisTrails Spreadsheet package (in the list of packages, to the left of the pipeline builder), you
can see there are built-in cells for different kinds of data, e.g., RichTextCell to display HTML and plain text. op
You (the user) can also define new cell types to display application-specific data. For example, we have developed
VtkCell, MplFigureCell, and OpenGLCell. It is possible to display pretty much anything on the Spreadsheet! [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

Examples of writing cell modules can be found in:
RichTextCell: packages/spreadsheet/widgets/richtext/richtext.py
VTK: packages/vtk/vtkcell.py

Here is the summary of some requirements on a cell widget:

(1) It must be a Qt widget. It should inherit from spreadsheet_cell.QCellWidget in the spreadsheet package. Although any Qt Widget would work, certain features such as animation will not be available (without rewriting it).

(2) It must re-implement the updateContents() function to take a set of inputs (usually coming from input ports of a wrapper Module) and display on the cells. VisTrails uses this function to update/reuse cells on the spreadsheet when new data comes in.

(3) It needs a wrapper VisTrails Module (inherited from basic_widgets.SpreadsheetCell of the spreadsheet package). Inside the compute() method of this module, it may call self.display(CellWidgetType, (inputs)) to trigger the display event on the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#advanced-cell-options Advanced Cell Options]

=== How do I control the default number of cells in the spreadsheet? ===

You can configure the rowCount and colCount using the preferences dialog. Just go to the Module Packages tab, select spreadsheet in the "Enabled packages" and press the Configure button. Then a list of all the configuration options for the spreadsheet will show up. [http://www.vistrails.org/usersguide/spreadsheet.html#custom-layout-options Custom Layout Options]

=== Is it possible to launch a web browser from the vistrails spreadsheet? We would like to output several urls from a parameter sweep and then have the option to click on each one to view the resulting page. I can view the page within the spreadsheet, but it is really too crowded.===

Currently, there isn't a widget that provides exactly this functionality, but I can think of a few solutions that may work for you:

(1) You can use parameter exploration to generate multiple sheets so you might have an exploration that opens each page in a new sheet. Use the third column/dimension in the exploration interface to have a parameter span sheets.

(2) The spreadsheet is extensible so you can write a custom spreadsheet cell widget that has a button or label with the desired link (a QLabel with openExternalLinks set to True, for example).

(3) You can tweak the existing RichTextCell be adding the line "self.browser.setOpenExternalLinks(True)" at line 63 of the source file "vistrails/packages/spreadsheet/widgets/richtext/richtext.py". Then, if your workflow creates a file with html markup text like "<a href="http://www.vistrails.org">VisTrails</a>" connected to a RichTextCell, clicking on the rendered link in the cell will open it in a web browser. You need to add the aforementioned line to the source to let Qt know that you want the link opened externally; by default, it will just issue an event that isn't processed. [http://www.vistrails.org/usersguide/spreadsheet.html#launching-a-web-browser Launching a Web Browser]

== Integrating your software into VisTrails ==

===How can I integrate my own program into VisTrails?===

The easiest way is to create a package. Writing a package is often very simple, here are instructions on how to do it: [[UsersGuideVisTrailsPackages]]

You can also dynamically generate modules. For an example see:

http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#Packages_that_generate_modules_dynamically

In particular, see the new_module call which uses python's type() function to generate new classes dynamically.

=== How do modules deal with multiple inputs in a same port? ===

(And should that even be allowed?)

For compatibility reasons, we do need to allow multiple connections to an input port. However, most package developers should never have to use this, and so we do our best to hide it. the default behavior for getting inputs from a port, then, is to always return a single input.

If on your module you need multiple inputs connected to a single port, use the 'forceGetInputListFromPort' method. It will return a list of all the data items coming through the port. The spreadsheet package uses this feature, so look there for usage examples (vistrails/packages/spreadsheet/basic_widgets.py)
[http://www.vistrails.org/usersguide/packages.html#configuring-ports Configuring Ports]

=== Are there mechanisms for attaching widgets to different modules/parameters? ===

Right now, we have a mechanism for putting a specific widget for an input port. For example, if a port is SetColor(red, green, blue), we can put a color wheel widget there. Or we can also replace the SetFileName port with a File Widget. However, this is not per parameter (only per port). We are currently working on this problem.

=== Can I organize my package so it appears hierarchical in the module palette? ===

Yes. Use the namespace keyword argument when adding the module to the registry. For example,

registry.add_module(MyModule, namespace='MyNamespace')
[http://www.vistrails.org/usersguide/packages.html#module-hierarchy-and-visibility Module Hierarchy and Visibility]

=== Can I nest namespaces? ===

Yes. Use the '|' character to separate different the hierarchy. For example,

registry.add_module(MyModule, namespace='ParentNamespace|ChildNamespace')

[http://www.vistrails.org/usersguide/packages.html#module-hierarchy-and-visibility Module Hierarchy and Visibility]

=== Are there shortcuts for registry initialization? ===

Yes. If you define _modules as a list of classes in the __init__.py file of your package, VisTrails will attempt to load all classes specified as modules. You can provide add_module options as keyword arguments by specifying a tuple (class, kwargs) in the list. For example:

_modules = [MyModule1, (MyModule2, {'namespace': 'MyNamespace'})]

In addition, you need to identify the ports of your modules as a field in your class by defining _input_ports and _output_ports lists. Here, the items in each list must be tuples of the form (portName, portSignature, optional=False, sort_key=-1). For example:

class MyModule(Module):
def compute(self):
pass

_input_ports = [('firstInput', String), ('secondInput', Integer, True)]
_output_ports = [('firstOutput', String), ('secondOutput', String)]

[http://www.vistrails.org/usersguide/packages.html#options-for-configuring-modules-and-ports Options for Configuring Modules and Ports]

=== Can I define ports to be of types that I do not import into my package? ===

Yes. You can pass an identifier string as the portSignature instead. The port_signature string is defined by:

<module_string> := <package_identifier>:[<namespace>|]<module_name>,
<port_signature> := (<module_string>*)

For example,

registry.add_input_port(MyModule, 'myInputPort', '(edu.utah.sci.vistrails.basic:String)')

or

_input_ports = [('myInputPort', '(edu.utah.sci.vistrails.basic:String)')]

=== What do I need to change in my package to make it reloadable (new in v1.4.2)? ===
See [http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#How_to_make_you_package_reloadable UsersGuideVisTrailsPackages] for an explanation.

=== Can I add default values or labels for parameters? ===

Yes. Versions 1.4 and greater support these features. See [[UsersGuideVisTrailsPackages#Adding_default_values_and.2For_labels_for_parameters | UsersGuideVisTrailsPacakges]] for more details.
[http://www.vistrails.org/usersguide/packages.html#configuring-ports Configuring Ports]

=== I want to write a module to load HDF data whose output (e.g., data, string) varies according to the input I give it. Is is possible to do this in VisTrails, and if yes, how can I do that? Ideally, I would like to avoid having to change the connection of my output every time I change the input. ===

There are a few ways to tackle this - each has it's own benefits and pitfalls. Firstly, module connections do respect class hierarchies as we're familiar with in object oriented languages. For instance, A module can output a Constant of which String, Float, Integer, etc are specifications. In this way, you can have a subclass of something like HDFData be passed out of the module and the connections will be established regardless of the sub-type. This is a bit dangerous though. Modules downstream of such a class may not really know how to operate on certain types derived from the super-class. Extreme care must be taken both when creating the modules as well as connecting them to prevent things like this from happening.

A second method that I employ in several different packages is the idea of a container class. For instance, the NumSciPy package uses a relatively generic container "Numpy Array" to encapsulate the data. Of course, these encapsulating objects can store dictionaries that other modules can easily access and understand how to operate on. Although this method is slightly more work, the benefits of a stricter typing of ports is beneficial - particularly upon interfacing with other packages that may depend on strongly typed constants (for example).

=== I need to determine, at run-time, whether or not a "child" module is attached to the output port of a "parent" module. (I do not specifically need to know which child; just if there is one). ===

The outputPorts dictionary of the base Module stores this information. Thus, you should be able to check

("myPortName" in self.outputPorts)

on the parent module to check if there are any downstream connections from the port "myPortName". This might be used, for example, to only set results for output ports that will be used. ***Note***, however, that the caching algorithm assumes that all outputs are set so adding a new connection to a previously unconnected output port will not work as desired if that module is cached. For this reason, I would currently recommend making such a module not cacheable. Another possibility is overriding the update() method to check the outputPorts and set the upToDate flag if they are not equal. In a single, limited test, this seemed to work, but be warned that it is not fully tested. Here is an example:

class TestModule(Module):
_output_ports = [('a1', '(edu.utah.sci.vistrails.basic:String)'),
('a2', '(edu.utah.sci.vistrails.basic:String)')]
def __init__(self):
Module.__init__(self)
self._cached_output_ports = set()

def update(self):
if len(set(self.outputPorts) - self._cached_output_ports) > 0:
self.upToDate = False
Module.update(self)

def compute(self):
if "a1" in self.outputPorts:
self.setResult("a1", "test")
if "a2" in self.outputPorts:
self.setResult("a2", "test2")
self._cached_output_ports = set(self.outputPorts)

=== How can I make a module not display in the modules list? ===

You should set the abstract parameter to True when adding the module to the registry. Using the original syntax, this looks like:

def initialize():
reg = core.modules.module_registry.get_module_registry()
reg.add_module(InvisibleModule, abstract=True)
# ...

With the _modules dictionary shortcut (for more details, see [http://www.vistrails.org/index.php/FAQ#Are_there_shortucts_for_registry_initialization.3F the FAQ section on this]), you include it in a kwargs dict as part of a module tuple:

_modules = [AnotherModule, (InvisibleModule, {'abstract': True})]

There is also a 'hide_descriptor' parameter that prevents the module from appearing in the module palette without declaring it to be abstract.

The technical difference between the two is that 'abstract' will not add the item to the module palette while 'hide_descriptor' does add the item but immediately hides it. If the module should never be instantiated in a workflow, declare it abstract. If you don't want users to be able to add the module to a pipeline, but you have code that may add it programmatically, declare it with hide_descriptor=True.

[http://www.vistrails.org/usersguide/packages.html#module-hierarchy-and-visibility Module Hierarchy and Visibility]

== The Console ==

=== Where should I go to find out what I can call from the console and how to import it? ===

We have tried to make some methods more accessible in the console via an api. You can import the api via <code>import api</code> in the console and see the available methods with <code>dir(api)</code>. To open a vistrail:

import api
api.open_vistrail_from_file('/Applications/VisTrails/examples/terminator.vt')

To execute a version of a workflow, you currently have to go through the controller:

api.select_version('Histogram')
api.get_current_controller().execute_current_workflow()

Currently, only a subset of VisTrails functionality is directly available from the api. However, since VisTrails is written in python, you can dig down starting with the VistrailsApplication or controller object to expose most of our internal methods. If you have suggestions for calls to be added to the api, please let us know.

One other feature that we're working on, but is still in progress is the ability to construct workflows via the console. For example:

vtk = load_package('edu.utah.sci.vistrails.vtk')
vtk.vtkDataSetReader() # adds a vtkDataSetReader module to the pipeline
# click on the new module
a = selected_modules()[0] # get the one currently selected module
a.SetFile('/vistrails/examples/data/head120.vtk') # sets the SetFile parmaeter for the data set reader
b = vtk.vtkContourFilter() # adds a vtkContourFilter module to the pipeline and saves to var b
b.SetInputConnection0(a.GetOutputPort0()) # connects a's GetOutputPort0 port to b's SetInputConnection0

== Persistence Package ==

=== How do I use the output of one workflow as the input for another using the persistence package? ===

You need to configure the persistence modules using the module's configuration dialog. After adding a <code>PersistentOutputFile</code> to the workflow, click on the triangle in the upper-right corner of the <code>PersistentOutputFile</code>, and select "Edit Configuration" from the menu that appears. In this dialog, select "Create New Reference" and give the reference a name (and any space-delimited tags). Upon running that workflow, the data will be written to the persistent store. In the second workflow where you wish to use that file, add a <code>PersistentInputFile</code> and go to its configuration dialog in the same manner as with the output file. In that dialog, select "Use Existing Reference" and select the data that you just added in the first workflow in the list of files below. Now, when you run that workflow, it will grab the data from the persistent store.

Here is an example: [[Media:offscreen_persistent.vt | offscreen_persistent.vt]]. Run the "persistent offscreen" workflow first, and then run the "display persistent output" to use the output of the first workflow as the input for the second.

== VTK ==

=== Given a VTK visualization, how can I generate a webpage from it? ===

Check out the html pipeline in offscreen.xml.

=== I'm trying to use VTK, but there doesn't seem to be any output. What is wrong? ===

To use VTK on VisTrails, you need a slightly different way of connecting the renderer modules. Instead of using the standard RenderWindow/RenderWindowInteractor infrastructure, you simply connect the renderer to a VTKCell. The examples directory in the distribution has several VTK examples that illustrate.

=== I am trying to add a module to the workflow via Python, but how can I access vtk modules? ===

Here's an example:

import api

vtvtk = 'edu.utah.sci.vistrails.vtk'

module = api.add_module(0, 0, vtvtk, 'vtkContourFilter', '')

The third argument in add_module is the package identifier. You can find this in the "Module Packages" panel of the Preferences; just click on the package you're interested in and it will appear in the information on the right.

== matplotlib ==

=== I'm experiencing a problem with Latex labels and the matplotlib that comes with VisTrails 1.5. The script below entered to the interpreter that comes with VT is sufficient to reproduce it. ===

import matplotlib.pyplot as plt
plt.plot([1,2,3],[1,2,3])
plt.xlabel("$foo$")

Remove your ~/.matplotlib folder and re-start VisTrails

== VisTrails Development ==

=== I would like to build VisTrails from source. Are there instructions on how to do this? ===

Yes! Take a look at http://www.vistrails.org/index.php/Mac_Intel_Instructions
[http://www.vistrails.org/usersguide/getting_started.html#installing-vistrails-from-source Installing VisTrails from source]

== Accessing Provenance Information ==

=== How do I access the information in the execution log? ===

The code responsible for storing execution information is located in the "core/log" directories, and the code that generates much of that information is in "core/interpreter/cached.py". Modules can add execution-specific annotations to provenance via annotate() calls during execution, but much of the data (like timing and errors) is captured by the LogController and CachedInterpreter (the execution engine) objects. To analyze the log from a vistrail (.vt) file, you might have something like the following:

<code>
import core.log.log
import db.services.io

def run(fname):
# open the .vt bundle specified by the filename "fname"
bundle = db.services.io.open_vistrail_bundle_from_zip_xml(fname)[0]
# get the log filename
log_fname = bundle.vistrail.db_log_filename
if log_fname is not None:
# open the log
log = db.services.io.open_log_from_xml(log_fname, True)
# convert the log from a db object
core.log.log.Log.convert(log)
for workflow_exec in log.workflow_execs:
print 'workflow version:', workflow_exec.parent_version
print 'time started:', workflow_exec.ts_start
print 'time ended:', workflow_exec.ts_end
print 'modules executed:', [i.module_id
for i in workflow_exec.item_execs]
if __name__ == '__main__':
run("some_vistrail.vt")
</code>

You should be able to see what information is available by looking at the "core/log" classes. [http://www.vistrails.org/usersguide/log.html Accessing the Execution Log]

FAQ

2011-03-31T23:50:07Z

Ctuttle: /* Can I add default values or labels for parameters? */

Also check our [[known_issues|Known Issues]] page for troubleshooting.

----
== Running workflows ==

=== How can I run a workflow using the command line? ===

('''Updated for version 1.2''') Call vistrails using the following options:

<code>python vistrails.py -b path_to_vistrails_file:pipeline </code>

where pipeline can be a version '''tag name''' or version '''id'''

'''NOTE:''' If you downloaded the MacOS X bundle, you can run vistrails from the command line via the following commands in the Terminal. Change the current directory to wherever VisTrails was installed (often <code>/Applications</code>), and then type:

<code>Vistrails.app/Contents/MacOS/vistrails [<cmd_line_options>]</code> [http://www.vistrails.org/usersguide/batch.html#running-a-specific-workflow-in-batch-mode Running a Specific Workflow in Batch Mode]

=== Using the command line, we'd like to execute a workflow multiple times, with slightly different parameters, and create a series of output files. Is this possible?===

('''Updated for version 1.2''') We can change parameters that have an alias through the command line.

For example, offscreen pipeline in offscreen.vt always creates the file called image.png. If you want generate it with a different filename:

<code>python vistrails.py -b ../examples/offscreen.vt:offscreen -a"filename=other.png"</code>

<code>filename</code> in the example above is the alias name assigned to the parameter in the <code>value</code> method inside the String module. When running a pipeline from the command line, VisTrails will try to start the spreadsheet automatically if the pipeline requires it. For example, this other execution will also start the spreadsheet (attention to how $ characters are escaped when running on bash):

<code>python vistrails.py -b ../examples/head.vt:aliases -a"isovalue=30\$&\$diffuse_color=0.8,0.4,0.2"</code>

You can also execute more than one pipeline on the command line:

<code>python vistrails.py -b ../examples/head.vt:aliases ../examples/spx.vt:spx \
-a"isovalue=30"</code>

Use the -a parameter only once regardless the number of pipelines. [http://www.vistrails.org/usersguide/batch.html#running-a-workflow-with-specific-parameters Running a Workflow with Specific Parameters]

=== I can load a vistrail, and the version tree shows up fine. However, no pipelines appear when I click on a version. What gives? ===

The most likely reason is that the vistrail uses a package that is not registered with VisTrails. You need to identify the needed package and add it to your <code>.vistrails/startup.py</code>. A single line like the following should be enough:

addPackage('enter_package_name_here')

Some packages might need more information. For example:

addPackage('afront', executable_path='/path/to/afront')

Refer to the package documentation for details. The one inconvenient step is that currently there's no automated way to describe what is the missing package. We're working on this feature for future releases.

=== I have a workflow that reads a file and then does some processing. The first time it runs, it executes correctly. But in subsequent, nothing happens. ===

VisTrails caches by default, so after a workflow is executed, if none of its parameters change, it won't be executed again.

If a workflow reads a file using the basic module File, VisTrails does check whether the file was modified since the last run. It does so by keeping a signature that is based on the modification time of the file. And if the file was modified, the File module and all downstream modules (the ones which depend on File) will be executed.

''Note: If you would like your input and output data to be versioned, you can use the Persistence package.''

If you do not want VisTrails to cache executions, you can turn off caching: go to Menu Edit -> Preferences and in the General Configuration tab, change Cache execution results to Never. [http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

=== Can VisTrails execute workflows in parallel? ===

The VisTrails server can only execute pipelines in parallel if there's more than one instance of VisTrails running. The command

<code> self.rpcserver = ThreadedXMLRPCServer((self.temp_xml_rpc_options.server, self.temp_xml_rpc_options.port))</code>

starts a multithreaded version of the XML-RPC server, so it will create a thread for each request received by the server. The problem is that Qt/PyQT doesn't allow these multiple threads create GUI objects, only in the main thread. To overcome this limitation, the multithreaded version can instantiate other single threaded versions of VisTrails and put them in a queue, so workflow executions and other GUI-related requests, such as generating workflow graphs and history trees can be forwarded to this queue, and each instance takes turns in answering the request. If the results are in the cache, the multithreaded version answers the requests directly.

Note that this infrastructure works on Linux only. To make this work on Windows, you have to create a script similar to start_vistrails_xvfb.sh (located in the scripts folder) where you can send the number of other instances via command-line options to VisTrails. The command line options are:

python vistrails_server.py -T <ADDRESS> -R <PORT> -O<NUMBER_OF_OTHER_VISTRAILS_INSTANCES> [-M]&

If you want the main vistrails instance to be multithreaded, use the -M at the end.

After creating this script, update function start_other_instances in vistrails/gui/application_server.py lines 1007-1023 and set the script variable to point to your script. You may also have to change the arguments sent to your script (line 1016: for example, you don't need to set a virtual display). You will need to change the path to the stop_vistrails_server.py script (on line 1026) according to your installation path. [http://www.vistrails.org/usersguide/batch.html#executing-workflows-in-parallel Executing Workflows in Parallel]

=== When a workflow is executed, what do the colors mean? ===

- lilac: module was notexecuted

- yellow: module is currently being executed

- green: module was successfully executed

- orange: module was cached

- red: the execution of the module failed

[http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

== Building workflows ==

=== Is there a way to give each widget a "display name" in addition to the module name at the center of the widget? ===

Yes, a "display name" can be assigned to a module by selecting the triangle in its top right corner to open a popup menu and selecting the Set Module Label... menu item. You will then be prompted to enter the "display name". [http://www.vistrails.org/usersguide/creating.html#changing-module-labels Changing Module Labels]

=== Is there a way to re-center the picture-in-picture (PiP) view? ===

Yes. If you click on the PIP window to bring it to focus, you can press Ctrl-R (or Command-R on Mac) to re-center the PiP window. [http://www.vistrails.org/usersguide/getting_started.html#vistrails-interaction Vistrails Interaction]

=== How do I search for a literal "?" (question mark) in the search box in the Property panel? ===

Since we allow regular expressions in our search box, question marks are treated as meta-characters. Thus, searching for "?" returns everything and "abc?" will return everything containing "abc". You need to use "\?" instead to search for "?". So the search for "??" would be "\?\?". [http://www.vistrails.org/usersguide/querying.html#textual-queries Textual Queries]

== Using VisTrails as a server ==

=== What is the VisTrails server-mode? ===

Using the VisTrails server mode, it is possible to ''execute workflows and control VisTrails through another application''. For example, the CrowdLabs Web portal (http://www.crowdlabs.org) accesses a VisTrails sever to execute workflows, retrieve and display vistrail trees and workflows. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

=== How do I execute workflows and control VisTrails through another application? ===

The way you access the server is by doing XML-RPC calls. In the current VisTrails release, we include a set of PHP scripts that can talk to a VisTrails server instance. They are in "extensions/http" folder. The files are reasonably well documented. Also, it should be not difficult to create python scripts to access the server (just use xmlrpclib module).

Note that the VisTrails server requires the provenance and workflows to be in a database. More detailed instructions on how to setup the server and the database
are available here:

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_server_setup.html

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_database_setup.html

If what you want is just to execute a series of workflows in batch mode, a simpler solution would be to ''use the VisTrails client in batch mode''. Chapter 12 of the user's guide contains detailed information and examples on that. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

== Control Flow ==

=== Note: using map ===

When using 'map', the module (or subworkflow) used as function port in the map module MUST be a function, i.e., it can only define 1 output port. [http://www.vistrails.org/usersguide/controlflow.html#the-map-operator The Map Operator]

== Spreadsheet ==

Where ''pipeline'' is a version number or a tag.

=== How can I save an image from the spreadsheet? ===

While having the focus on a spreadsheet cell and select the camera on the toolbar to take a snapshot. The system will prompt you for the location and file name where it should be saved. The other icons can be used for saving multiple images that can be used for generating an animation on demand. A whole sheet can also be saved by selecting Export (either from the menu or from the toolbar). [http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet-image Saving a Spreadsheet Image]

===Is it possible to save the complete state of the spreadsheet?===
[http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet Saving a Spreadsheet]

=== Can I view multiple sheets at the same time? ===

Yes. Each sheet on the spreadsheet can be displayed as a dock widget separated from the main spreadsheet window by dragging its tab name out of the tab bar at the bottom of the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Then, how can I put back a separated sheet? ===

A sheet can be docked back to the main window by dragging it back to the tab bar or double-click on its title bar. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== How can I order sheets on the spreadsheet? ===

This can be done by dragging the sheet name on the bottom top bar and drop it to the right place. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Can I control where a cell will be placed on the spreadsheet window? ===

By default, an unoccupied cell on the active sheet will be chosen to display the result. However, you can specify exactly in the pipeline where a spreadsheet cell will be placed by using CellLocation and SheetReference. CellLocation specifies the location (row and column) of a cell when connecting to a spreadsheet cell (VTKCell, ImageViewerCell, ...). Similarly, a SheetReference module (when connecting to a CellLocation) will specify which sheet the cell will be put on given its name, minimum row size and minimum column size. There is an example of this in examples/vtk.xml (select the version below Double Renderer). [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

=== How do I output results to the spreadsheet? ===

By inspecting the VisTrails Spreadsheet package (in the list of packages, to the left of the pipeline builder), you
can see there are built-in cells for different kinds of data, e.g., RichTextCell to display HTML and plain text. op
You (the user) can also define new cell types to display application-specific data. For example, we have developed
VtkCell, MplFigureCell, and OpenGLCell. It is possible to display pretty much anything on the Spreadsheet! [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

Examples of writing cell modules can be found in:
RichTextCell: packages/spreadsheet/widgets/richtext/richtext.py
VTK: packages/vtk/vtkcell.py

Here is the summary of some requirements on a cell widget:

(1) It must be a Qt widget. It should inherit from spreadsheet_cell.QCellWidget in the spreadsheet package. Although any Qt Widget would work, certain features such as animation will not be available (without rewriting it).

(2) It must re-implement the updateContents() function to take a set of inputs (usually coming from input ports of a wrapper Module) and display on the cells. VisTrails uses this function to update/reuse cells on the spreadsheet when new data comes in.

(3) It needs a wrapper VisTrails Module (inherited from basic_widgets.SpreadsheetCell of the spreadsheet package). Inside the compute() method of this module, it may call self.display(CellWidgetType, (inputs)) to trigger the display event on the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#advanced-cell-options Advanced Cell Options]

=== How do I control the default number of cells in the spreadsheet? ===

You can configure the rowCount and colCount using the preferences dialog. Just go to the Module Packages tab, select spreadsheet in the "Enabled packages" and press the Configure button. Then a list of all the configuration options for the spreadsheet will show up. [http://www.vistrails.org/usersguide/spreadsheet.html#custom-layout-options Custom Layout Options]

=== Is it possible to launch a web browser from the vistrails spreadsheet? We would like to output several urls from a parameter sweep and then have the option to click on each one to view the resulting page. I can view the page within the spreadsheet, but it is really too crowded.===

Currently, there isn't a widget that provides exactly this functionality, but I can think of a few solutions that may work for you:

(1) You can use parameter exploration to generate multiple sheets so you might have an exploration that opens each page in a new sheet. Use the third column/dimension in the exploration interface to have a parameter span sheets.

(2) The spreadsheet is extensible so you can write a custom spreadsheet cell widget that has a button or label with the desired link (a QLabel with openExternalLinks set to True, for example).

(3) You can tweak the existing RichTextCell be adding the line "self.browser.setOpenExternalLinks(True)" at line 63 of the source file "vistrails/packages/spreadsheet/widgets/richtext/richtext.py". Then, if your workflow creates a file with html markup text like "<a href="http://www.vistrails.org">VisTrails</a>" connected to a RichTextCell, clicking on the rendered link in the cell will open it in a web browser. You need to add the aforementioned line to the source to let Qt know that you want the link opened externally; by default, it will just issue an event that isn't processed. [http://www.vistrails.org/usersguide/spreadsheet.html#launching-a-web-browser Launching a Web Browser]

== Integrating your software into VisTrails ==

===How can I integrate my own program into VisTrails?===

The easiest way is to create a package. Writing a package is often very simple, here are instructions on how to do it: [[UsersGuideVisTrailsPackages]]

You can also dynamically generate modules. For an example see:

http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#Packages_that_generate_modules_dynamically

In particular, see the new_module call which uses python's type() function to generate new classes dynamically.

=== How do modules deal with multiple inputs in a same port? ===

(And should that even be allowed?)

For compatibility reasons, we do need to allow multiple connections to an input port. However, most package developers should never have to use this, and so we do our best to hide it. the default behavior for getting inputs from a port, then, is to always return a single input.

If on your module you need multiple inputs connected to a single port, use the 'forceGetInputListFromPort' method. It will return a list of all the data items coming through the port. The spreadsheet package uses this feature, so look there for usage examples (vistrails/packages/spreadsheet/basic_widgets.py)

=== Are there mechanisms for attaching widgets to different modules/parameters? ===

Right now, we have a mechanism for putting a specific widget for an input port. For example, if a port is SetColor(red, green, blue), we can put a color wheel widget there. Or we can also replace the SetFileName port with a File Widget. However, this is not per parameter (only per port). We are currently working on this problem.

=== Can I organize my package so it appears hierarchical in the module palette? ===

Yes. Use the namespace keyword argument when adding the module to the registry. For example,

registry.add_module(MyModule, namespace='MyNamespace')
[http://www.vistrails.org/usersguide/packages.html#module-hierarchy-and-visibility Module Hierarchy and Visibility]

=== Can I nest namespaces? ===

Yes. Use the '|' character to separate different the hierarchy. For example,

registry.add_module(MyModule, namespace='ParentNamespace|ChildNamespace')

[http://www.vistrails.org/usersguide/packages.html#module-hierarchy-and-visibility Module Hierarchy and Visibility]

=== Are there shortcuts for registry initialization? ===

Yes. If you define _modules as a list of classes in the __init__.py file of your package, VisTrails will attempt to load all classes specified as modules. You can provide add_module options as keyword arguments by specifying a tuple (class, kwargs) in the list. For example:

_modules = [MyModule1, (MyModule2, {'namespace': 'MyNamespace'})]

In addition, you need to identify the ports of your modules as a field in your class by defining _input_ports and _output_ports lists. Here, the items in each list must be tuples of the form (portName, portSignature, optional=False, sort_key=-1). For example:

class MyModule(Module):
def compute(self):
pass

_input_ports = [('firstInput', String), ('secondInput', Integer, True)]
_output_ports = [('firstOutput', String), ('secondOutput', String)]

[http://www.vistrails.org/usersguide/packages.html#options-for-configuring-modules-and-ports Options for Configuring Modules and Ports]

=== Can I define ports to be of types that I do not import into my package? ===

Yes. You can pass an identifier string as the portSignature instead. The port_signature string is defined by:

<module_string> := <package_identifier>:[<namespace>|]<module_name>,
<port_signature> := (<module_string>*)

For example,

registry.add_input_port(MyModule, 'myInputPort', '(edu.utah.sci.vistrails.basic:String)')

or

_input_ports = [('myInputPort', '(edu.utah.sci.vistrails.basic:String)')]

=== What do I need to change in my package to make it reloadable (new in v1.4.2)? ===
See [http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#How_to_make_you_package_reloadable UsersGuideVisTrailsPackages] for an explanation.

=== Can I add default values or labels for parameters? ===

Yes. Versions 1.4 and greater support these features. See [[UsersGuideVisTrailsPackages#Adding_default_values_and.2For_labels_for_parameters | UsersGuideVisTrailsPacakges]] for more details.
[http://www.vistrails.org/usersguide/packages.html#configuring-ports Configuring Ports]

=== I want to write a module to load HDF data whose output (e.g., data, string) varies according to the input I give it. Is is possible to do this in VisTrails, and if yes, how can I do that? Ideally, I would like to avoid having to change the connection of my output every time I change the input. ===

There are a few ways to tackle this - each has it's own benefits and pitfalls. Firstly, module connections do respect class hierarchies as we're familiar with in object oriented languages. For instance, A module can output a Constant of which String, Float, Integer, etc are specifications. In this way, you can have a subclass of something like HDFData be passed out of the module and the connections will be established regardless of the sub-type. This is a bit dangerous though. Modules downstream of such a class may not really know how to operate on certain types derived from the super-class. Extreme care must be taken both when creating the modules as well as connecting them to prevent things like this from happening.

A second method that I employ in several different packages is the idea of a container class. For instance, the NumSciPy package uses a relatively generic container "Numpy Array" to encapsulate the data. Of course, these encapsulating objects can store dictionaries that other modules can easily access and understand how to operate on. Although this method is slightly more work, the benefits of a stricter typing of ports is beneficial - particularly upon interfacing with other packages that may depend on strongly typed constants (for example).

=== I need to determine, at run-time, whether or not a "child" module is attached to the output port of a "parent" module. (I do not specifically need to know which child; just if there is one). ===

The outputPorts dictionary of the base Module stores this information. Thus, you should be able to check

("myPortName" in self.outputPorts)

on the parent module to check if there are any downstream connections from the port "myPortName". This might be used, for example, to only set results for output ports that will be used. ***Note***, however, that the caching algorithm assumes that all outputs are set so adding a new connection to a previously unconnected output port will not work as desired if that module is cached. For this reason, I would currently recommend making such a module not cacheable. Another possibility is overriding the update() method to check the outputPorts and set the upToDate flag if they are not equal. In a single, limited test, this seemed to work, but be warned that it is not fully tested. Here is an example:

class TestModule(Module):
_output_ports = [('a1', '(edu.utah.sci.vistrails.basic:String)'),
('a2', '(edu.utah.sci.vistrails.basic:String)')]
def __init__(self):
Module.__init__(self)
self._cached_output_ports = set()

def update(self):
if len(set(self.outputPorts) - self._cached_output_ports) > 0:
self.upToDate = False
Module.update(self)

def compute(self):
if "a1" in self.outputPorts:
self.setResult("a1", "test")
if "a2" in self.outputPorts:
self.setResult("a2", "test2")
self._cached_output_ports = set(self.outputPorts)

=== How can I make a module not display in the modules list? ===

You should set the abstract parameter to True when adding the module to the registry. Using the original syntax, this looks like:

def initialize():
reg = core.modules.module_registry.get_module_registry()
reg.add_module(InvisibleModule, abstract=True)
# ...

With the _modules dictionary shortcut (for more details, see [http://www.vistrails.org/index.php/FAQ#Are_there_shortucts_for_registry_initialization.3F the FAQ section on this]), you include it in a kwargs dict as part of a module tuple:

_modules = [AnotherModule, (InvisibleModule, {'abstract': True})]

There is also a 'hide_descriptor' parameter that prevents the module from appearing in the module palette without declaring it to be abstract.

The technical difference between the two is that 'abstract' will not add the item to the module palette while 'hide_descriptor' does add the item but immediately hides it. If the module should never be instantiated in a workflow, declare it abstract. If you don't want users to be able to add the module to a pipeline, but you have code that may add it programmatically, declare it with hide_descriptor=True.

[http://www.vistrails.org/usersguide/packages.html#module-hierarchy-and-visibility Module Hierarchy and Visibility]

== The Console ==

=== Where should I go to find out what I can call from the console and how to import it? ===

We have tried to make some methods more accessible in the console via an api. You can import the api via <code>import api</code> in the console and see the available methods with <code>dir(api)</code>. To open a vistrail:

import api
api.open_vistrail_from_file('/Applications/VisTrails/examples/terminator.vt')

To execute a version of a workflow, you currently have to go through the controller:

api.select_version('Histogram')
api.get_current_controller().execute_current_workflow()

Currently, only a subset of VisTrails functionality is directly available from the api. However, since VisTrails is written in python, you can dig down starting with the VistrailsApplication or controller object to expose most of our internal methods. If you have suggestions for calls to be added to the api, please let us know.

One other feature that we're working on, but is still in progress is the ability to construct workflows via the console. For example:

vtk = load_package('edu.utah.sci.vistrails.vtk')
vtk.vtkDataSetReader() # adds a vtkDataSetReader module to the pipeline
# click on the new module
a = selected_modules()[0] # get the one currently selected module
a.SetFile('/vistrails/examples/data/head120.vtk') # sets the SetFile parmaeter for the data set reader
b = vtk.vtkContourFilter() # adds a vtkContourFilter module to the pipeline and saves to var b
b.SetInputConnection0(a.GetOutputPort0()) # connects a's GetOutputPort0 port to b's SetInputConnection0

== Persistence Package ==

=== How do I use the output of one workflow as the input for another using the persistence package? ===

You need to configure the persistence modules using the module's configuration dialog. After adding a <code>PersistentOutputFile</code> to the workflow, click on the triangle in the upper-right corner of the <code>PersistentOutputFile</code>, and select "Edit Configuration" from the menu that appears. In this dialog, select "Create New Reference" and give the reference a name (and any space-delimited tags). Upon running that workflow, the data will be written to the persistent store. In the second workflow where you wish to use that file, add a <code>PersistentInputFile</code> and go to its configuration dialog in the same manner as with the output file. In that dialog, select "Use Existing Reference" and select the data that you just added in the first workflow in the list of files below. Now, when you run that workflow, it will grab the data from the persistent store.

Here is an example: [[Media:offscreen_persistent.vt | offscreen_persistent.vt]]. Run the "persistent offscreen" workflow first, and then run the "display persistent output" to use the output of the first workflow as the input for the second.

== VTK ==

=== Given a VTK visualization, how can I generate a webpage from it? ===

Check out the html pipeline in offscreen.xml.

=== I'm trying to use VTK, but there doesn't seem to be any output. What is wrong? ===

To use VTK on VisTrails, you need a slightly different way of connecting the renderer modules. Instead of using the standard RenderWindow/RenderWindowInteractor infrastructure, you simply connect the renderer to a VTKCell. The examples directory in the distribution has several VTK examples that illustrate.

=== I am trying to add a module to the workflow via Python, but how can I access vtk modules? ===

Here's an example:

import api

vtvtk = 'edu.utah.sci.vistrails.vtk'

module = api.add_module(0, 0, vtvtk, 'vtkContourFilter', '')

The third argument in add_module is the package identifier. You can find this in the "Module Packages" panel of the Preferences; just click on the package you're interested in and it will appear in the information on the right.

== matplotlib ==

=== I'm experiencing a problem with Latex labels and the matplotlib that comes with VisTrails 1.5. The script below entered to the interpreter that comes with VT is sufficient to reproduce it. ===

import matplotlib.pyplot as plt
plt.plot([1,2,3],[1,2,3])
plt.xlabel("$foo$")

Remove your ~/.matplotlib folder and re-start VisTrails

== VisTrails Development ==

=== I would like to build VisTrails from source. Are there instructions on how to do this? ===

Yes! Take a look at http://www.vistrails.org/index.php/Mac_Intel_Instructions
[http://www.vistrails.org/usersguide/getting_started.html#installing-vistrails-from-source Installing VisTrails from source]

== Accessing Provenance Information ==

=== How do I access the information in the execution log? ===

The code responsible for storing execution information is located in the "core/log" directories, and the code that generates much of that information is in "core/interpreter/cached.py". Modules can add execution-specific annotations to provenance via annotate() calls during execution, but much of the data (like timing and errors) is captured by the LogController and CachedInterpreter (the execution engine) objects. To analyze the log from a vistrail (.vt) file, you might have something like the following:

<code>
import core.log.log
import db.services.io

def run(fname):
# open the .vt bundle specified by the filename "fname"
bundle = db.services.io.open_vistrail_bundle_from_zip_xml(fname)[0]
# get the log filename
log_fname = bundle.vistrail.db_log_filename
if log_fname is not None:
# open the log
log = db.services.io.open_log_from_xml(log_fname, True)
# convert the log from a db object
core.log.log.Log.convert(log)
for workflow_exec in log.workflow_execs:
print 'workflow version:', workflow_exec.parent_version
print 'time started:', workflow_exec.ts_start
print 'time ended:', workflow_exec.ts_end
print 'modules executed:', [i.module_id
for i in workflow_exec.item_execs]
if __name__ == '__main__':
run("some_vistrail.vt")
</code>

You should be able to see what information is available by looking at the "core/log" classes. [http://www.vistrails.org/usersguide/log.html Accessing the Execution Log]

FAQ

2011-03-31T23:40:07Z

Ctuttle: /* How can I make a module not display in the modules list? */

Also check our [[known_issues|Known Issues]] page for troubleshooting.

----
== Running workflows ==

=== How can I run a workflow using the command line? ===

('''Updated for version 1.2''') Call vistrails using the following options:

<code>python vistrails.py -b path_to_vistrails_file:pipeline </code>

where pipeline can be a version '''tag name''' or version '''id'''

'''NOTE:''' If you downloaded the MacOS X bundle, you can run vistrails from the command line via the following commands in the Terminal. Change the current directory to wherever VisTrails was installed (often <code>/Applications</code>), and then type:

<code>Vistrails.app/Contents/MacOS/vistrails [<cmd_line_options>]</code> [http://www.vistrails.org/usersguide/batch.html#running-a-specific-workflow-in-batch-mode Running a Specific Workflow in Batch Mode]

=== Using the command line, we'd like to execute a workflow multiple times, with slightly different parameters, and create a series of output files. Is this possible?===

('''Updated for version 1.2''') We can change parameters that have an alias through the command line.

For example, offscreen pipeline in offscreen.vt always creates the file called image.png. If you want generate it with a different filename:

<code>python vistrails.py -b ../examples/offscreen.vt:offscreen -a"filename=other.png"</code>

<code>filename</code> in the example above is the alias name assigned to the parameter in the <code>value</code> method inside the String module. When running a pipeline from the command line, VisTrails will try to start the spreadsheet automatically if the pipeline requires it. For example, this other execution will also start the spreadsheet (attention to how $ characters are escaped when running on bash):

<code>python vistrails.py -b ../examples/head.vt:aliases -a"isovalue=30\$&\$diffuse_color=0.8,0.4,0.2"</code>

You can also execute more than one pipeline on the command line:

<code>python vistrails.py -b ../examples/head.vt:aliases ../examples/spx.vt:spx \
-a"isovalue=30"</code>

Use the -a parameter only once regardless the number of pipelines. [http://www.vistrails.org/usersguide/batch.html#running-a-workflow-with-specific-parameters Running a Workflow with Specific Parameters]

=== I can load a vistrail, and the version tree shows up fine. However, no pipelines appear when I click on a version. What gives? ===

The most likely reason is that the vistrail uses a package that is not registered with VisTrails. You need to identify the needed package and add it to your <code>.vistrails/startup.py</code>. A single line like the following should be enough:

addPackage('enter_package_name_here')

Some packages might need more information. For example:

addPackage('afront', executable_path='/path/to/afront')

Refer to the package documentation for details. The one inconvenient step is that currently there's no automated way to describe what is the missing package. We're working on this feature for future releases.

=== I have a workflow that reads a file and then does some processing. The first time it runs, it executes correctly. But in subsequent, nothing happens. ===

VisTrails caches by default, so after a workflow is executed, if none of its parameters change, it won't be executed again.

If a workflow reads a file using the basic module File, VisTrails does check whether the file was modified since the last run. It does so by keeping a signature that is based on the modification time of the file. And if the file was modified, the File module and all downstream modules (the ones which depend on File) will be executed.

''Note: If you would like your input and output data to be versioned, you can use the Persistence package.''

If you do not want VisTrails to cache executions, you can turn off caching: go to Menu Edit -> Preferences and in the General Configuration tab, change Cache execution results to Never. [http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

=== Can VisTrails execute workflows in parallel? ===

The VisTrails server can only execute pipelines in parallel if there's more than one instance of VisTrails running. The command

<code> self.rpcserver = ThreadedXMLRPCServer((self.temp_xml_rpc_options.server, self.temp_xml_rpc_options.port))</code>

starts a multithreaded version of the XML-RPC server, so it will create a thread for each request received by the server. The problem is that Qt/PyQT doesn't allow these multiple threads create GUI objects, only in the main thread. To overcome this limitation, the multithreaded version can instantiate other single threaded versions of VisTrails and put them in a queue, so workflow executions and other GUI-related requests, such as generating workflow graphs and history trees can be forwarded to this queue, and each instance takes turns in answering the request. If the results are in the cache, the multithreaded version answers the requests directly.

Note that this infrastructure works on Linux only. To make this work on Windows, you have to create a script similar to start_vistrails_xvfb.sh (located in the scripts folder) where you can send the number of other instances via command-line options to VisTrails. The command line options are:

python vistrails_server.py -T <ADDRESS> -R <PORT> -O<NUMBER_OF_OTHER_VISTRAILS_INSTANCES> [-M]&

If you want the main vistrails instance to be multithreaded, use the -M at the end.

After creating this script, update function start_other_instances in vistrails/gui/application_server.py lines 1007-1023 and set the script variable to point to your script. You may also have to change the arguments sent to your script (line 1016: for example, you don't need to set a virtual display). You will need to change the path to the stop_vistrails_server.py script (on line 1026) according to your installation path. [http://www.vistrails.org/usersguide/batch.html#executing-workflows-in-parallel Executing Workflows in Parallel]

=== When a workflow is executed, what do the colors mean? ===

- lilac: module was notexecuted

- yellow: module is currently being executed

- green: module was successfully executed

- orange: module was cached

- red: the execution of the module failed

[http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

== Building workflows ==

=== Is there a way to give each widget a "display name" in addition to the module name at the center of the widget? ===

Yes, a "display name" can be assigned to a module by selecting the triangle in its top right corner to open a popup menu and selecting the Set Module Label... menu item. You will then be prompted to enter the "display name". [http://www.vistrails.org/usersguide/creating.html#changing-module-labels Changing Module Labels]

=== Is there a way to re-center the picture-in-picture (PiP) view? ===

Yes. If you click on the PIP window to bring it to focus, you can press Ctrl-R (or Command-R on Mac) to re-center the PiP window. [http://www.vistrails.org/usersguide/getting_started.html#vistrails-interaction Vistrails Interaction]

=== How do I search for a literal "?" (question mark) in the search box in the Property panel? ===

Since we allow regular expressions in our search box, question marks are treated as meta-characters. Thus, searching for "?" returns everything and "abc?" will return everything containing "abc". You need to use "\?" instead to search for "?". So the search for "??" would be "\?\?". [http://www.vistrails.org/usersguide/querying.html#textual-queries Textual Queries]

== Using VisTrails as a server ==

=== What is the VisTrails server-mode? ===

Using the VisTrails server mode, it is possible to ''execute workflows and control VisTrails through another application''. For example, the CrowdLabs Web portal (http://www.crowdlabs.org) accesses a VisTrails sever to execute workflows, retrieve and display vistrail trees and workflows. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

=== How do I execute workflows and control VisTrails through another application? ===

The way you access the server is by doing XML-RPC calls. In the current VisTrails release, we include a set of PHP scripts that can talk to a VisTrails server instance. They are in "extensions/http" folder. The files are reasonably well documented. Also, it should be not difficult to create python scripts to access the server (just use xmlrpclib module).

Note that the VisTrails server requires the provenance and workflows to be in a database. More detailed instructions on how to setup the server and the database
are available here:

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_server_setup.html

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_database_setup.html

If what you want is just to execute a series of workflows in batch mode, a simpler solution would be to ''use the VisTrails client in batch mode''. Chapter 12 of the user's guide contains detailed information and examples on that. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

== Control Flow ==

=== Note: using map ===

When using 'map', the module (or subworkflow) used as function port in the map module MUST be a function, i.e., it can only define 1 output port. [http://www.vistrails.org/usersguide/controlflow.html#the-map-operator The Map Operator]

== Spreadsheet ==

Where ''pipeline'' is a version number or a tag.

=== How can I save an image from the spreadsheet? ===

While having the focus on a spreadsheet cell and select the camera on the toolbar to take a snapshot. The system will prompt you for the location and file name where it should be saved. The other icons can be used for saving multiple images that can be used for generating an animation on demand. A whole sheet can also be saved by selecting Export (either from the menu or from the toolbar). [http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet-image Saving a Spreadsheet Image]

===Is it possible to save the complete state of the spreadsheet?===
[http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet Saving a Spreadsheet]

=== Can I view multiple sheets at the same time? ===

Yes. Each sheet on the spreadsheet can be displayed as a dock widget separated from the main spreadsheet window by dragging its tab name out of the tab bar at the bottom of the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Then, how can I put back a separated sheet? ===

A sheet can be docked back to the main window by dragging it back to the tab bar or double-click on its title bar. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== How can I order sheets on the spreadsheet? ===

This can be done by dragging the sheet name on the bottom top bar and drop it to the right place. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Can I control where a cell will be placed on the spreadsheet window? ===

By default, an unoccupied cell on the active sheet will be chosen to display the result. However, you can specify exactly in the pipeline where a spreadsheet cell will be placed by using CellLocation and SheetReference. CellLocation specifies the location (row and column) of a cell when connecting to a spreadsheet cell (VTKCell, ImageViewerCell, ...). Similarly, a SheetReference module (when connecting to a CellLocation) will specify which sheet the cell will be put on given its name, minimum row size and minimum column size. There is an example of this in examples/vtk.xml (select the version below Double Renderer). [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

=== How do I output results to the spreadsheet? ===

By inspecting the VisTrails Spreadsheet package (in the list of packages, to the left of the pipeline builder), you
can see there are built-in cells for different kinds of data, e.g., RichTextCell to display HTML and plain text. op
You (the user) can also define new cell types to display application-specific data. For example, we have developed
VtkCell, MplFigureCell, and OpenGLCell. It is possible to display pretty much anything on the Spreadsheet! [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

Examples of writing cell modules can be found in:
RichTextCell: packages/spreadsheet/widgets/richtext/richtext.py
VTK: packages/vtk/vtkcell.py

Here is the summary of some requirements on a cell widget:

(1) It must be a Qt widget. It should inherit from spreadsheet_cell.QCellWidget in the spreadsheet package. Although any Qt Widget would work, certain features such as animation will not be available (without rewriting it).

(2) It must re-implement the updateContents() function to take a set of inputs (usually coming from input ports of a wrapper Module) and display on the cells. VisTrails uses this function to update/reuse cells on the spreadsheet when new data comes in.

(3) It needs a wrapper VisTrails Module (inherited from basic_widgets.SpreadsheetCell of the spreadsheet package). Inside the compute() method of this module, it may call self.display(CellWidgetType, (inputs)) to trigger the display event on the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#advanced-cell-options Advanced Cell Options]

=== How do I control the default number of cells in the spreadsheet? ===

You can configure the rowCount and colCount using the preferences dialog. Just go to the Module Packages tab, select spreadsheet in the "Enabled packages" and press the Configure button. Then a list of all the configuration options for the spreadsheet will show up. [http://www.vistrails.org/usersguide/spreadsheet.html#custom-layout-options Custom Layout Options]

=== Is it possible to launch a web browser from the vistrails spreadsheet? We would like to output several urls from a parameter sweep and then have the option to click on each one to view the resulting page. I can view the page within the spreadsheet, but it is really too crowded.===

Currently, there isn't a widget that provides exactly this functionality, but I can think of a few solutions that may work for you:

(1) You can use parameter exploration to generate multiple sheets so you might have an exploration that opens each page in a new sheet. Use the third column/dimension in the exploration interface to have a parameter span sheets.

(2) The spreadsheet is extensible so you can write a custom spreadsheet cell widget that has a button or label with the desired link (a QLabel with openExternalLinks set to True, for example).

(3) You can tweak the existing RichTextCell be adding the line "self.browser.setOpenExternalLinks(True)" at line 63 of the source file "vistrails/packages/spreadsheet/widgets/richtext/richtext.py". Then, if your workflow creates a file with html markup text like "<a href="http://www.vistrails.org">VisTrails</a>" connected to a RichTextCell, clicking on the rendered link in the cell will open it in a web browser. You need to add the aforementioned line to the source to let Qt know that you want the link opened externally; by default, it will just issue an event that isn't processed. [http://www.vistrails.org/usersguide/spreadsheet.html#launching-a-web-browser Launching a Web Browser]

== Integrating your software into VisTrails ==

===How can I integrate my own program into VisTrails?===

The easiest way is to create a package. Writing a package is often very simple, here are instructions on how to do it: [[UsersGuideVisTrailsPackages]]

You can also dynamically generate modules. For an example see:

http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#Packages_that_generate_modules_dynamically

In particular, see the new_module call which uses python's type() function to generate new classes dynamically.

=== How do modules deal with multiple inputs in a same port? ===

(And should that even be allowed?)

For compatibility reasons, we do need to allow multiple connections to an input port. However, most package developers should never have to use this, and so we do our best to hide it. the default behavior for getting inputs from a port, then, is to always return a single input.

If on your module you need multiple inputs connected to a single port, use the 'forceGetInputListFromPort' method. It will return a list of all the data items coming through the port. The spreadsheet package uses this feature, so look there for usage examples (vistrails/packages/spreadsheet/basic_widgets.py)

=== Are there mechanisms for attaching widgets to different modules/parameters? ===

Right now, we have a mechanism for putting a specific widget for an input port. For example, if a port is SetColor(red, green, blue), we can put a color wheel widget there. Or we can also replace the SetFileName port with a File Widget. However, this is not per parameter (only per port). We are currently working on this problem.

=== Can I organize my package so it appears hierarchical in the module palette? ===

Yes. Use the namespace keyword argument when adding the module to the registry. For example,

registry.add_module(MyModule, namespace='MyNamespace')
[http://www.vistrails.org/usersguide/packages.html#module-hierarchy-and-visibility Module Hierarchy and Visibility]

=== Can I nest namespaces? ===

Yes. Use the '|' character to separate different the hierarchy. For example,

registry.add_module(MyModule, namespace='ParentNamespace|ChildNamespace')

[http://www.vistrails.org/usersguide/packages.html#module-hierarchy-and-visibility Module Hierarchy and Visibility]

=== Are there shortcuts for registry initialization? ===

Yes. If you define _modules as a list of classes in the __init__.py file of your package, VisTrails will attempt to load all classes specified as modules. You can provide add_module options as keyword arguments by specifying a tuple (class, kwargs) in the list. For example:

_modules = [MyModule1, (MyModule2, {'namespace': 'MyNamespace'})]

In addition, you need to identify the ports of your modules as a field in your class by defining _input_ports and _output_ports lists. Here, the items in each list must be tuples of the form (portName, portSignature, optional=False, sort_key=-1). For example:

class MyModule(Module):
def compute(self):
pass

_input_ports = [('firstInput', String), ('secondInput', Integer, True)]
_output_ports = [('firstOutput', String), ('secondOutput', String)]

[http://www.vistrails.org/usersguide/packages.html#options-for-configuring-modules-and-ports Options for Configuring Modules and Ports]

=== Can I define ports to be of types that I do not import into my package? ===

Yes. You can pass an identifier string as the portSignature instead. The port_signature string is defined by:

<module_string> := <package_identifier>:[<namespace>|]<module_name>,
<port_signature> := (<module_string>*)

For example,

registry.add_input_port(MyModule, 'myInputPort', '(edu.utah.sci.vistrails.basic:String)')

or

_input_ports = [('myInputPort', '(edu.utah.sci.vistrails.basic:String)')]

=== What do I need to change in my package to make it reloadable (new in v1.4.2)? ===
See [http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#How_to_make_you_package_reloadable UsersGuideVisTrailsPackages] for an explanation.

=== Can I add default values or labels for parameters? ===

Yes. Versions 1.4 and greater support these features. See [[UsersGuideVisTrailsPackages#Adding_default_values_and.2For_labels_for_parameters | UsersGuideVisTrailsPacakges]] for more details.

=== I want to write a module to load HDF data whose output (e.g., data, string) varies according to the input I give it. Is is possible to do this in VisTrails, and if yes, how can I do that? Ideally, I would like to avoid having to change the connection of my output every time I change the input. ===

There are a few ways to tackle this - each has it's own benefits and pitfalls. Firstly, module connections do respect class hierarchies as we're familiar with in object oriented languages. For instance, A module can output a Constant of which String, Float, Integer, etc are specifications. In this way, you can have a subclass of something like HDFData be passed out of the module and the connections will be established regardless of the sub-type. This is a bit dangerous though. Modules downstream of such a class may not really know how to operate on certain types derived from the super-class. Extreme care must be taken both when creating the modules as well as connecting them to prevent things like this from happening.

A second method that I employ in several different packages is the idea of a container class. For instance, the NumSciPy package uses a relatively generic container "Numpy Array" to encapsulate the data. Of course, these encapsulating objects can store dictionaries that other modules can easily access and understand how to operate on. Although this method is slightly more work, the benefits of a stricter typing of ports is beneficial - particularly upon interfacing with other packages that may depend on strongly typed constants (for example).

=== I need to determine, at run-time, whether or not a "child" module is attached to the output port of a "parent" module. (I do not specifically need to know which child; just if there is one). ===

The outputPorts dictionary of the base Module stores this information. Thus, you should be able to check

("myPortName" in self.outputPorts)

on the parent module to check if there are any downstream connections from the port "myPortName". This might be used, for example, to only set results for output ports that will be used. ***Note***, however, that the caching algorithm assumes that all outputs are set so adding a new connection to a previously unconnected output port will not work as desired if that module is cached. For this reason, I would currently recommend making such a module not cacheable. Another possibility is overriding the update() method to check the outputPorts and set the upToDate flag if they are not equal. In a single, limited test, this seemed to work, but be warned that it is not fully tested. Here is an example:

class TestModule(Module):
_output_ports = [('a1', '(edu.utah.sci.vistrails.basic:String)'),
('a2', '(edu.utah.sci.vistrails.basic:String)')]
def __init__(self):
Module.__init__(self)
self._cached_output_ports = set()

def update(self):
if len(set(self.outputPorts) - self._cached_output_ports) > 0:
self.upToDate = False
Module.update(self)

def compute(self):
if "a1" in self.outputPorts:
self.setResult("a1", "test")
if "a2" in self.outputPorts:
self.setResult("a2", "test2")
self._cached_output_ports = set(self.outputPorts)

=== How can I make a module not display in the modules list? ===

You should set the abstract parameter to True when adding the module to the registry. Using the original syntax, this looks like:

def initialize():
reg = core.modules.module_registry.get_module_registry()
reg.add_module(InvisibleModule, abstract=True)
# ...

With the _modules dictionary shortcut (for more details, see [http://www.vistrails.org/index.php/FAQ#Are_there_shortucts_for_registry_initialization.3F the FAQ section on this]), you include it in a kwargs dict as part of a module tuple:

_modules = [AnotherModule, (InvisibleModule, {'abstract': True})]

There is also a 'hide_descriptor' parameter that prevents the module from appearing in the module palette without declaring it to be abstract.

The technical difference between the two is that 'abstract' will not add the item to the module palette while 'hide_descriptor' does add the item but immediately hides it. If the module should never be instantiated in a workflow, declare it abstract. If you don't want users to be able to add the module to a pipeline, but you have code that may add it programmatically, declare it with hide_descriptor=True.

[http://www.vistrails.org/usersguide/packages.html#module-hierarchy-and-visibility Module Hierarchy and Visibility]

== The Console ==

=== Where should I go to find out what I can call from the console and how to import it? ===

We have tried to make some methods more accessible in the console via an api. You can import the api via <code>import api</code> in the console and see the available methods with <code>dir(api)</code>. To open a vistrail:

import api
api.open_vistrail_from_file('/Applications/VisTrails/examples/terminator.vt')

To execute a version of a workflow, you currently have to go through the controller:

api.select_version('Histogram')
api.get_current_controller().execute_current_workflow()

Currently, only a subset of VisTrails functionality is directly available from the api. However, since VisTrails is written in python, you can dig down starting with the VistrailsApplication or controller object to expose most of our internal methods. If you have suggestions for calls to be added to the api, please let us know.

One other feature that we're working on, but is still in progress is the ability to construct workflows via the console. For example:

vtk = load_package('edu.utah.sci.vistrails.vtk')
vtk.vtkDataSetReader() # adds a vtkDataSetReader module to the pipeline
# click on the new module
a = selected_modules()[0] # get the one currently selected module
a.SetFile('/vistrails/examples/data/head120.vtk') # sets the SetFile parmaeter for the data set reader
b = vtk.vtkContourFilter() # adds a vtkContourFilter module to the pipeline and saves to var b
b.SetInputConnection0(a.GetOutputPort0()) # connects a's GetOutputPort0 port to b's SetInputConnection0

== Persistence Package ==

=== How do I use the output of one workflow as the input for another using the persistence package? ===

You need to configure the persistence modules using the module's configuration dialog. After adding a <code>PersistentOutputFile</code> to the workflow, click on the triangle in the upper-right corner of the <code>PersistentOutputFile</code>, and select "Edit Configuration" from the menu that appears. In this dialog, select "Create New Reference" and give the reference a name (and any space-delimited tags). Upon running that workflow, the data will be written to the persistent store. In the second workflow where you wish to use that file, add a <code>PersistentInputFile</code> and go to its configuration dialog in the same manner as with the output file. In that dialog, select "Use Existing Reference" and select the data that you just added in the first workflow in the list of files below. Now, when you run that workflow, it will grab the data from the persistent store.

Here is an example: [[Media:offscreen_persistent.vt | offscreen_persistent.vt]]. Run the "persistent offscreen" workflow first, and then run the "display persistent output" to use the output of the first workflow as the input for the second.

== VTK ==

=== Given a VTK visualization, how can I generate a webpage from it? ===

Check out the html pipeline in offscreen.xml.

=== I'm trying to use VTK, but there doesn't seem to be any output. What is wrong? ===

To use VTK on VisTrails, you need a slightly different way of connecting the renderer modules. Instead of using the standard RenderWindow/RenderWindowInteractor infrastructure, you simply connect the renderer to a VTKCell. The examples directory in the distribution has several VTK examples that illustrate.

=== I am trying to add a module to the workflow via Python, but how can I access vtk modules? ===

Here's an example:

import api

vtvtk = 'edu.utah.sci.vistrails.vtk'

module = api.add_module(0, 0, vtvtk, 'vtkContourFilter', '')

The third argument in add_module is the package identifier. You can find this in the "Module Packages" panel of the Preferences; just click on the package you're interested in and it will appear in the information on the right.

== matplotlib ==

=== I'm experiencing a problem with Latex labels and the matplotlib that comes with VisTrails 1.5. The script below entered to the interpreter that comes with VT is sufficient to reproduce it. ===

import matplotlib.pyplot as plt
plt.plot([1,2,3],[1,2,3])
plt.xlabel("$foo$")

Remove your ~/.matplotlib folder and re-start VisTrails

== VisTrails Development ==

=== I would like to build VisTrails from source. Are there instructions on how to do this? ===

Yes! Take a look at http://www.vistrails.org/index.php/Mac_Intel_Instructions
[http://www.vistrails.org/usersguide/getting_started.html#installing-vistrails-from-source Installing VisTrails from source]

== Accessing Provenance Information ==

=== How do I access the information in the execution log? ===

The code responsible for storing execution information is located in the "core/log" directories, and the code that generates much of that information is in "core/interpreter/cached.py". Modules can add execution-specific annotations to provenance via annotate() calls during execution, but much of the data (like timing and errors) is captured by the LogController and CachedInterpreter (the execution engine) objects. To analyze the log from a vistrail (.vt) file, you might have something like the following:

<code>
import core.log.log
import db.services.io

def run(fname):
# open the .vt bundle specified by the filename "fname"
bundle = db.services.io.open_vistrail_bundle_from_zip_xml(fname)[0]
# get the log filename
log_fname = bundle.vistrail.db_log_filename
if log_fname is not None:
# open the log
log = db.services.io.open_log_from_xml(log_fname, True)
# convert the log from a db object
core.log.log.Log.convert(log)
for workflow_exec in log.workflow_execs:
print 'workflow version:', workflow_exec.parent_version
print 'time started:', workflow_exec.ts_start
print 'time ended:', workflow_exec.ts_end
print 'modules executed:', [i.module_id
for i in workflow_exec.item_execs]
if __name__ == '__main__':
run("some_vistrail.vt")
</code>

You should be able to see what information is available by looking at the "core/log" classes. [http://www.vistrails.org/usersguide/log.html Accessing the Execution Log]

FAQ

2011-03-31T23:38:48Z

Ctuttle: /* Are there shortcuts for registry initialization? */

Also check our [[known_issues|Known Issues]] page for troubleshooting.

----
== Running workflows ==

=== How can I run a workflow using the command line? ===

('''Updated for version 1.2''') Call vistrails using the following options:

<code>python vistrails.py -b path_to_vistrails_file:pipeline </code>

where pipeline can be a version '''tag name''' or version '''id'''

'''NOTE:''' If you downloaded the MacOS X bundle, you can run vistrails from the command line via the following commands in the Terminal. Change the current directory to wherever VisTrails was installed (often <code>/Applications</code>), and then type:

<code>Vistrails.app/Contents/MacOS/vistrails [<cmd_line_options>]</code> [http://www.vistrails.org/usersguide/batch.html#running-a-specific-workflow-in-batch-mode Running a Specific Workflow in Batch Mode]

=== Using the command line, we'd like to execute a workflow multiple times, with slightly different parameters, and create a series of output files. Is this possible?===

('''Updated for version 1.2''') We can change parameters that have an alias through the command line.

For example, offscreen pipeline in offscreen.vt always creates the file called image.png. If you want generate it with a different filename:

<code>python vistrails.py -b ../examples/offscreen.vt:offscreen -a"filename=other.png"</code>

<code>filename</code> in the example above is the alias name assigned to the parameter in the <code>value</code> method inside the String module. When running a pipeline from the command line, VisTrails will try to start the spreadsheet automatically if the pipeline requires it. For example, this other execution will also start the spreadsheet (attention to how $ characters are escaped when running on bash):

<code>python vistrails.py -b ../examples/head.vt:aliases -a"isovalue=30\$&\$diffuse_color=0.8,0.4,0.2"</code>

You can also execute more than one pipeline on the command line:

<code>python vistrails.py -b ../examples/head.vt:aliases ../examples/spx.vt:spx \
-a"isovalue=30"</code>

Use the -a parameter only once regardless the number of pipelines. [http://www.vistrails.org/usersguide/batch.html#running-a-workflow-with-specific-parameters Running a Workflow with Specific Parameters]

=== I can load a vistrail, and the version tree shows up fine. However, no pipelines appear when I click on a version. What gives? ===

The most likely reason is that the vistrail uses a package that is not registered with VisTrails. You need to identify the needed package and add it to your <code>.vistrails/startup.py</code>. A single line like the following should be enough:

addPackage('enter_package_name_here')

Some packages might need more information. For example:

addPackage('afront', executable_path='/path/to/afront')

Refer to the package documentation for details. The one inconvenient step is that currently there's no automated way to describe what is the missing package. We're working on this feature for future releases.

=== I have a workflow that reads a file and then does some processing. The first time it runs, it executes correctly. But in subsequent, nothing happens. ===

VisTrails caches by default, so after a workflow is executed, if none of its parameters change, it won't be executed again.

If a workflow reads a file using the basic module File, VisTrails does check whether the file was modified since the last run. It does so by keeping a signature that is based on the modification time of the file. And if the file was modified, the File module and all downstream modules (the ones which depend on File) will be executed.

''Note: If you would like your input and output data to be versioned, you can use the Persistence package.''

If you do not want VisTrails to cache executions, you can turn off caching: go to Menu Edit -> Preferences and in the General Configuration tab, change Cache execution results to Never. [http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

=== Can VisTrails execute workflows in parallel? ===

The VisTrails server can only execute pipelines in parallel if there's more than one instance of VisTrails running. The command

<code> self.rpcserver = ThreadedXMLRPCServer((self.temp_xml_rpc_options.server, self.temp_xml_rpc_options.port))</code>

starts a multithreaded version of the XML-RPC server, so it will create a thread for each request received by the server. The problem is that Qt/PyQT doesn't allow these multiple threads create GUI objects, only in the main thread. To overcome this limitation, the multithreaded version can instantiate other single threaded versions of VisTrails and put them in a queue, so workflow executions and other GUI-related requests, such as generating workflow graphs and history trees can be forwarded to this queue, and each instance takes turns in answering the request. If the results are in the cache, the multithreaded version answers the requests directly.

Note that this infrastructure works on Linux only. To make this work on Windows, you have to create a script similar to start_vistrails_xvfb.sh (located in the scripts folder) where you can send the number of other instances via command-line options to VisTrails. The command line options are:

python vistrails_server.py -T <ADDRESS> -R <PORT> -O<NUMBER_OF_OTHER_VISTRAILS_INSTANCES> [-M]&

If you want the main vistrails instance to be multithreaded, use the -M at the end.

After creating this script, update function start_other_instances in vistrails/gui/application_server.py lines 1007-1023 and set the script variable to point to your script. You may also have to change the arguments sent to your script (line 1016: for example, you don't need to set a virtual display). You will need to change the path to the stop_vistrails_server.py script (on line 1026) according to your installation path. [http://www.vistrails.org/usersguide/batch.html#executing-workflows-in-parallel Executing Workflows in Parallel]

=== When a workflow is executed, what do the colors mean? ===

- lilac: module was notexecuted

- yellow: module is currently being executed

- green: module was successfully executed

- orange: module was cached

- red: the execution of the module failed

[http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

== Building workflows ==

=== Is there a way to give each widget a "display name" in addition to the module name at the center of the widget? ===

Yes, a "display name" can be assigned to a module by selecting the triangle in its top right corner to open a popup menu and selecting the Set Module Label... menu item. You will then be prompted to enter the "display name". [http://www.vistrails.org/usersguide/creating.html#changing-module-labels Changing Module Labels]

=== Is there a way to re-center the picture-in-picture (PiP) view? ===

Yes. If you click on the PIP window to bring it to focus, you can press Ctrl-R (or Command-R on Mac) to re-center the PiP window. [http://www.vistrails.org/usersguide/getting_started.html#vistrails-interaction Vistrails Interaction]

=== How do I search for a literal "?" (question mark) in the search box in the Property panel? ===

Since we allow regular expressions in our search box, question marks are treated as meta-characters. Thus, searching for "?" returns everything and "abc?" will return everything containing "abc". You need to use "\?" instead to search for "?". So the search for "??" would be "\?\?". [http://www.vistrails.org/usersguide/querying.html#textual-queries Textual Queries]

== Using VisTrails as a server ==

=== What is the VisTrails server-mode? ===

Using the VisTrails server mode, it is possible to ''execute workflows and control VisTrails through another application''. For example, the CrowdLabs Web portal (http://www.crowdlabs.org) accesses a VisTrails sever to execute workflows, retrieve and display vistrail trees and workflows. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

=== How do I execute workflows and control VisTrails through another application? ===

The way you access the server is by doing XML-RPC calls. In the current VisTrails release, we include a set of PHP scripts that can talk to a VisTrails server instance. They are in "extensions/http" folder. The files are reasonably well documented. Also, it should be not difficult to create python scripts to access the server (just use xmlrpclib module).

Note that the VisTrails server requires the provenance and workflows to be in a database. More detailed instructions on how to setup the server and the database
are available here:

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_server_setup.html

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_database_setup.html

If what you want is just to execute a series of workflows in batch mode, a simpler solution would be to ''use the VisTrails client in batch mode''. Chapter 12 of the user's guide contains detailed information and examples on that. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

== Control Flow ==

=== Note: using map ===

When using 'map', the module (or subworkflow) used as function port in the map module MUST be a function, i.e., it can only define 1 output port. [http://www.vistrails.org/usersguide/controlflow.html#the-map-operator The Map Operator]

== Spreadsheet ==

Where ''pipeline'' is a version number or a tag.

=== How can I save an image from the spreadsheet? ===

While having the focus on a spreadsheet cell and select the camera on the toolbar to take a snapshot. The system will prompt you for the location and file name where it should be saved. The other icons can be used for saving multiple images that can be used for generating an animation on demand. A whole sheet can also be saved by selecting Export (either from the menu or from the toolbar). [http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet-image Saving a Spreadsheet Image]

===Is it possible to save the complete state of the spreadsheet?===
[http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet Saving a Spreadsheet]

=== Can I view multiple sheets at the same time? ===

Yes. Each sheet on the spreadsheet can be displayed as a dock widget separated from the main spreadsheet window by dragging its tab name out of the tab bar at the bottom of the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Then, how can I put back a separated sheet? ===

A sheet can be docked back to the main window by dragging it back to the tab bar or double-click on its title bar. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== How can I order sheets on the spreadsheet? ===

This can be done by dragging the sheet name on the bottom top bar and drop it to the right place. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Can I control where a cell will be placed on the spreadsheet window? ===

By default, an unoccupied cell on the active sheet will be chosen to display the result. However, you can specify exactly in the pipeline where a spreadsheet cell will be placed by using CellLocation and SheetReference. CellLocation specifies the location (row and column) of a cell when connecting to a spreadsheet cell (VTKCell, ImageViewerCell, ...). Similarly, a SheetReference module (when connecting to a CellLocation) will specify which sheet the cell will be put on given its name, minimum row size and minimum column size. There is an example of this in examples/vtk.xml (select the version below Double Renderer). [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

=== How do I output results to the spreadsheet? ===

By inspecting the VisTrails Spreadsheet package (in the list of packages, to the left of the pipeline builder), you
can see there are built-in cells for different kinds of data, e.g., RichTextCell to display HTML and plain text. op
You (the user) can also define new cell types to display application-specific data. For example, we have developed
VtkCell, MplFigureCell, and OpenGLCell. It is possible to display pretty much anything on the Spreadsheet! [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

Examples of writing cell modules can be found in:
RichTextCell: packages/spreadsheet/widgets/richtext/richtext.py
VTK: packages/vtk/vtkcell.py

Here is the summary of some requirements on a cell widget:

(1) It must be a Qt widget. It should inherit from spreadsheet_cell.QCellWidget in the spreadsheet package. Although any Qt Widget would work, certain features such as animation will not be available (without rewriting it).

(2) It must re-implement the updateContents() function to take a set of inputs (usually coming from input ports of a wrapper Module) and display on the cells. VisTrails uses this function to update/reuse cells on the spreadsheet when new data comes in.

(3) It needs a wrapper VisTrails Module (inherited from basic_widgets.SpreadsheetCell of the spreadsheet package). Inside the compute() method of this module, it may call self.display(CellWidgetType, (inputs)) to trigger the display event on the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#advanced-cell-options Advanced Cell Options]

=== How do I control the default number of cells in the spreadsheet? ===

You can configure the rowCount and colCount using the preferences dialog. Just go to the Module Packages tab, select spreadsheet in the "Enabled packages" and press the Configure button. Then a list of all the configuration options for the spreadsheet will show up. [http://www.vistrails.org/usersguide/spreadsheet.html#custom-layout-options Custom Layout Options]

=== Is it possible to launch a web browser from the vistrails spreadsheet? We would like to output several urls from a parameter sweep and then have the option to click on each one to view the resulting page. I can view the page within the spreadsheet, but it is really too crowded.===

Currently, there isn't a widget that provides exactly this functionality, but I can think of a few solutions that may work for you:

(1) You can use parameter exploration to generate multiple sheets so you might have an exploration that opens each page in a new sheet. Use the third column/dimension in the exploration interface to have a parameter span sheets.

(2) The spreadsheet is extensible so you can write a custom spreadsheet cell widget that has a button or label with the desired link (a QLabel with openExternalLinks set to True, for example).

(3) You can tweak the existing RichTextCell be adding the line "self.browser.setOpenExternalLinks(True)" at line 63 of the source file "vistrails/packages/spreadsheet/widgets/richtext/richtext.py". Then, if your workflow creates a file with html markup text like "<a href="http://www.vistrails.org">VisTrails</a>" connected to a RichTextCell, clicking on the rendered link in the cell will open it in a web browser. You need to add the aforementioned line to the source to let Qt know that you want the link opened externally; by default, it will just issue an event that isn't processed. [http://www.vistrails.org/usersguide/spreadsheet.html#launching-a-web-browser Launching a Web Browser]

== Integrating your software into VisTrails ==

===How can I integrate my own program into VisTrails?===

The easiest way is to create a package. Writing a package is often very simple, here are instructions on how to do it: [[UsersGuideVisTrailsPackages]]

You can also dynamically generate modules. For an example see:

http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#Packages_that_generate_modules_dynamically

In particular, see the new_module call which uses python's type() function to generate new classes dynamically.

=== How do modules deal with multiple inputs in a same port? ===

(And should that even be allowed?)

For compatibility reasons, we do need to allow multiple connections to an input port. However, most package developers should never have to use this, and so we do our best to hide it. the default behavior for getting inputs from a port, then, is to always return a single input.

If on your module you need multiple inputs connected to a single port, use the 'forceGetInputListFromPort' method. It will return a list of all the data items coming through the port. The spreadsheet package uses this feature, so look there for usage examples (vistrails/packages/spreadsheet/basic_widgets.py)

=== Are there mechanisms for attaching widgets to different modules/parameters? ===

Right now, we have a mechanism for putting a specific widget for an input port. For example, if a port is SetColor(red, green, blue), we can put a color wheel widget there. Or we can also replace the SetFileName port with a File Widget. However, this is not per parameter (only per port). We are currently working on this problem.

=== Can I organize my package so it appears hierarchical in the module palette? ===

Yes. Use the namespace keyword argument when adding the module to the registry. For example,

registry.add_module(MyModule, namespace='MyNamespace')
[http://www.vistrails.org/usersguide/packages.html#module-hierarchy-and-visibility Module Hierarchy and Visibility]

=== Can I nest namespaces? ===

Yes. Use the '|' character to separate different the hierarchy. For example,

registry.add_module(MyModule, namespace='ParentNamespace|ChildNamespace')

[http://www.vistrails.org/usersguide/packages.html#module-hierarchy-and-visibility Module Hierarchy and Visibility]

=== Are there shortcuts for registry initialization? ===

Yes. If you define _modules as a list of classes in the __init__.py file of your package, VisTrails will attempt to load all classes specified as modules. You can provide add_module options as keyword arguments by specifying a tuple (class, kwargs) in the list. For example:

_modules = [MyModule1, (MyModule2, {'namespace': 'MyNamespace'})]

In addition, you need to identify the ports of your modules as a field in your class by defining _input_ports and _output_ports lists. Here, the items in each list must be tuples of the form (portName, portSignature, optional=False, sort_key=-1). For example:

class MyModule(Module):
def compute(self):
pass

_input_ports = [('firstInput', String), ('secondInput', Integer, True)]
_output_ports = [('firstOutput', String), ('secondOutput', String)]

[http://www.vistrails.org/usersguide/packages.html#options-for-configuring-modules-and-ports Options for Configuring Modules and Ports]

=== Can I define ports to be of types that I do not import into my package? ===

Yes. You can pass an identifier string as the portSignature instead. The port_signature string is defined by:

<module_string> := <package_identifier>:[<namespace>|]<module_name>,
<port_signature> := (<module_string>*)

For example,

registry.add_input_port(MyModule, 'myInputPort', '(edu.utah.sci.vistrails.basic:String)')

or

_input_ports = [('myInputPort', '(edu.utah.sci.vistrails.basic:String)')]

=== What do I need to change in my package to make it reloadable (new in v1.4.2)? ===
See [http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#How_to_make_you_package_reloadable UsersGuideVisTrailsPackages] for an explanation.

=== Can I add default values or labels for parameters? ===

Yes. Versions 1.4 and greater support these features. See [[UsersGuideVisTrailsPackages#Adding_default_values_and.2For_labels_for_parameters | UsersGuideVisTrailsPacakges]] for more details.

=== I want to write a module to load HDF data whose output (e.g., data, string) varies according to the input I give it. Is is possible to do this in VisTrails, and if yes, how can I do that? Ideally, I would like to avoid having to change the connection of my output every time I change the input. ===

There are a few ways to tackle this - each has it's own benefits and pitfalls. Firstly, module connections do respect class hierarchies as we're familiar with in object oriented languages. For instance, A module can output a Constant of which String, Float, Integer, etc are specifications. In this way, you can have a subclass of something like HDFData be passed out of the module and the connections will be established regardless of the sub-type. This is a bit dangerous though. Modules downstream of such a class may not really know how to operate on certain types derived from the super-class. Extreme care must be taken both when creating the modules as well as connecting them to prevent things like this from happening.

A second method that I employ in several different packages is the idea of a container class. For instance, the NumSciPy package uses a relatively generic container "Numpy Array" to encapsulate the data. Of course, these encapsulating objects can store dictionaries that other modules can easily access and understand how to operate on. Although this method is slightly more work, the benefits of a stricter typing of ports is beneficial - particularly upon interfacing with other packages that may depend on strongly typed constants (for example).

=== I need to determine, at run-time, whether or not a "child" module is attached to the output port of a "parent" module. (I do not specifically need to know which child; just if there is one). ===

The outputPorts dictionary of the base Module stores this information. Thus, you should be able to check

("myPortName" in self.outputPorts)

on the parent module to check if there are any downstream connections from the port "myPortName". This might be used, for example, to only set results for output ports that will be used. ***Note***, however, that the caching algorithm assumes that all outputs are set so adding a new connection to a previously unconnected output port will not work as desired if that module is cached. For this reason, I would currently recommend making such a module not cacheable. Another possibility is overriding the update() method to check the outputPorts and set the upToDate flag if they are not equal. In a single, limited test, this seemed to work, but be warned that it is not fully tested. Here is an example:

class TestModule(Module):
_output_ports = [('a1', '(edu.utah.sci.vistrails.basic:String)'),
('a2', '(edu.utah.sci.vistrails.basic:String)')]
def __init__(self):
Module.__init__(self)
self._cached_output_ports = set()

def update(self):
if len(set(self.outputPorts) - self._cached_output_ports) > 0:
self.upToDate = False
Module.update(self)

def compute(self):
if "a1" in self.outputPorts:
self.setResult("a1", "test")
if "a2" in self.outputPorts:
self.setResult("a2", "test2")
self._cached_output_ports = set(self.outputPorts)

=== How can I make a module not display in the modules list? ===

You should set the abstract parameter to True when adding the module to the registry. Using the original syntax, this looks like:

def initialize():
reg = core.modules.module_registry.get_module_registry()
reg.add_module(InvisibleModule, abstract=True)
# ...

With the _modules dictionary shortcut (for more details, see [http://www.vistrails.org/index.php/FAQ#Are_there_shortucts_for_registry_initialization.3F the FAQ section on this]), you include it in a kwargs dict as part of a module tuple:

_modules = [AnotherModule, (InvisibleModule, {'abstract': True})]

There is also a 'hide_descriptor' parameter that prevents the module from appearing in the module palette without declaring it to be abstract.

The technical difference between the two is that 'abstract' will not add the item to the module palette while 'hide_descriptor' does add the item but immediately hides it. If the module should never be instantiated in a workflow, declare it abstract. If you don't want users to be able to add the module to a pipeline, but you have code that may add it programmatically, declare it with hide_descriptor=True.

== The Console ==

=== Where should I go to find out what I can call from the console and how to import it? ===

We have tried to make some methods more accessible in the console via an api. You can import the api via <code>import api</code> in the console and see the available methods with <code>dir(api)</code>. To open a vistrail:

import api
api.open_vistrail_from_file('/Applications/VisTrails/examples/terminator.vt')

To execute a version of a workflow, you currently have to go through the controller:

api.select_version('Histogram')
api.get_current_controller().execute_current_workflow()

Currently, only a subset of VisTrails functionality is directly available from the api. However, since VisTrails is written in python, you can dig down starting with the VistrailsApplication or controller object to expose most of our internal methods. If you have suggestions for calls to be added to the api, please let us know.

One other feature that we're working on, but is still in progress is the ability to construct workflows via the console. For example:

vtk = load_package('edu.utah.sci.vistrails.vtk')
vtk.vtkDataSetReader() # adds a vtkDataSetReader module to the pipeline
# click on the new module
a = selected_modules()[0] # get the one currently selected module
a.SetFile('/vistrails/examples/data/head120.vtk') # sets the SetFile parmaeter for the data set reader
b = vtk.vtkContourFilter() # adds a vtkContourFilter module to the pipeline and saves to var b
b.SetInputConnection0(a.GetOutputPort0()) # connects a's GetOutputPort0 port to b's SetInputConnection0

== Persistence Package ==

=== How do I use the output of one workflow as the input for another using the persistence package? ===

You need to configure the persistence modules using the module's configuration dialog. After adding a <code>PersistentOutputFile</code> to the workflow, click on the triangle in the upper-right corner of the <code>PersistentOutputFile</code>, and select "Edit Configuration" from the menu that appears. In this dialog, select "Create New Reference" and give the reference a name (and any space-delimited tags). Upon running that workflow, the data will be written to the persistent store. In the second workflow where you wish to use that file, add a <code>PersistentInputFile</code> and go to its configuration dialog in the same manner as with the output file. In that dialog, select "Use Existing Reference" and select the data that you just added in the first workflow in the list of files below. Now, when you run that workflow, it will grab the data from the persistent store.

Here is an example: [[Media:offscreen_persistent.vt | offscreen_persistent.vt]]. Run the "persistent offscreen" workflow first, and then run the "display persistent output" to use the output of the first workflow as the input for the second.

== VTK ==

=== Given a VTK visualization, how can I generate a webpage from it? ===

Check out the html pipeline in offscreen.xml.

=== I'm trying to use VTK, but there doesn't seem to be any output. What is wrong? ===

To use VTK on VisTrails, you need a slightly different way of connecting the renderer modules. Instead of using the standard RenderWindow/RenderWindowInteractor infrastructure, you simply connect the renderer to a VTKCell. The examples directory in the distribution has several VTK examples that illustrate.

=== I am trying to add a module to the workflow via Python, but how can I access vtk modules? ===

Here's an example:

import api

vtvtk = 'edu.utah.sci.vistrails.vtk'

module = api.add_module(0, 0, vtvtk, 'vtkContourFilter', '')

The third argument in add_module is the package identifier. You can find this in the "Module Packages" panel of the Preferences; just click on the package you're interested in and it will appear in the information on the right.

== matplotlib ==

=== I'm experiencing a problem with Latex labels and the matplotlib that comes with VisTrails 1.5. The script below entered to the interpreter that comes with VT is sufficient to reproduce it. ===

import matplotlib.pyplot as plt
plt.plot([1,2,3],[1,2,3])
plt.xlabel("$foo$")

Remove your ~/.matplotlib folder and re-start VisTrails

== VisTrails Development ==

=== I would like to build VisTrails from source. Are there instructions on how to do this? ===

Yes! Take a look at http://www.vistrails.org/index.php/Mac_Intel_Instructions
[http://www.vistrails.org/usersguide/getting_started.html#installing-vistrails-from-source Installing VisTrails from source]

== Accessing Provenance Information ==

=== How do I access the information in the execution log? ===

The code responsible for storing execution information is located in the "core/log" directories, and the code that generates much of that information is in "core/interpreter/cached.py". Modules can add execution-specific annotations to provenance via annotate() calls during execution, but much of the data (like timing and errors) is captured by the LogController and CachedInterpreter (the execution engine) objects. To analyze the log from a vistrail (.vt) file, you might have something like the following:

<code>
import core.log.log
import db.services.io

def run(fname):
# open the .vt bundle specified by the filename "fname"
bundle = db.services.io.open_vistrail_bundle_from_zip_xml(fname)[0]
# get the log filename
log_fname = bundle.vistrail.db_log_filename
if log_fname is not None:
# open the log
log = db.services.io.open_log_from_xml(log_fname, True)
# convert the log from a db object
core.log.log.Log.convert(log)
for workflow_exec in log.workflow_execs:
print 'workflow version:', workflow_exec.parent_version
print 'time started:', workflow_exec.ts_start
print 'time ended:', workflow_exec.ts_end
print 'modules executed:', [i.module_id
for i in workflow_exec.item_execs]
if __name__ == '__main__':
run("some_vistrail.vt")
</code>

You should be able to see what information is available by looking at the "core/log" classes. [http://www.vistrails.org/usersguide/log.html Accessing the Execution Log]

FAQ

2011-03-31T23:37:17Z

Ctuttle: /* Can I nest namespaces? */

Also check our [[known_issues|Known Issues]] page for troubleshooting.

----
== Running workflows ==

=== How can I run a workflow using the command line? ===

('''Updated for version 1.2''') Call vistrails using the following options:

<code>python vistrails.py -b path_to_vistrails_file:pipeline </code>

where pipeline can be a version '''tag name''' or version '''id'''

'''NOTE:''' If you downloaded the MacOS X bundle, you can run vistrails from the command line via the following commands in the Terminal. Change the current directory to wherever VisTrails was installed (often <code>/Applications</code>), and then type:

<code>Vistrails.app/Contents/MacOS/vistrails [<cmd_line_options>]</code> [http://www.vistrails.org/usersguide/batch.html#running-a-specific-workflow-in-batch-mode Running a Specific Workflow in Batch Mode]

=== Using the command line, we'd like to execute a workflow multiple times, with slightly different parameters, and create a series of output files. Is this possible?===

('''Updated for version 1.2''') We can change parameters that have an alias through the command line.

For example, offscreen pipeline in offscreen.vt always creates the file called image.png. If you want generate it with a different filename:

<code>python vistrails.py -b ../examples/offscreen.vt:offscreen -a"filename=other.png"</code>

<code>filename</code> in the example above is the alias name assigned to the parameter in the <code>value</code> method inside the String module. When running a pipeline from the command line, VisTrails will try to start the spreadsheet automatically if the pipeline requires it. For example, this other execution will also start the spreadsheet (attention to how $ characters are escaped when running on bash):

<code>python vistrails.py -b ../examples/head.vt:aliases -a"isovalue=30\$&\$diffuse_color=0.8,0.4,0.2"</code>

You can also execute more than one pipeline on the command line:

<code>python vistrails.py -b ../examples/head.vt:aliases ../examples/spx.vt:spx \
-a"isovalue=30"</code>

Use the -a parameter only once regardless the number of pipelines. [http://www.vistrails.org/usersguide/batch.html#running-a-workflow-with-specific-parameters Running a Workflow with Specific Parameters]

=== I can load a vistrail, and the version tree shows up fine. However, no pipelines appear when I click on a version. What gives? ===

The most likely reason is that the vistrail uses a package that is not registered with VisTrails. You need to identify the needed package and add it to your <code>.vistrails/startup.py</code>. A single line like the following should be enough:

addPackage('enter_package_name_here')

Some packages might need more information. For example:

addPackage('afront', executable_path='/path/to/afront')

Refer to the package documentation for details. The one inconvenient step is that currently there's no automated way to describe what is the missing package. We're working on this feature for future releases.

=== I have a workflow that reads a file and then does some processing. The first time it runs, it executes correctly. But in subsequent, nothing happens. ===

VisTrails caches by default, so after a workflow is executed, if none of its parameters change, it won't be executed again.

If a workflow reads a file using the basic module File, VisTrails does check whether the file was modified since the last run. It does so by keeping a signature that is based on the modification time of the file. And if the file was modified, the File module and all downstream modules (the ones which depend on File) will be executed.

''Note: If you would like your input and output data to be versioned, you can use the Persistence package.''

If you do not want VisTrails to cache executions, you can turn off caching: go to Menu Edit -> Preferences and in the General Configuration tab, change Cache execution results to Never. [http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

=== Can VisTrails execute workflows in parallel? ===

The VisTrails server can only execute pipelines in parallel if there's more than one instance of VisTrails running. The command

<code> self.rpcserver = ThreadedXMLRPCServer((self.temp_xml_rpc_options.server, self.temp_xml_rpc_options.port))</code>

starts a multithreaded version of the XML-RPC server, so it will create a thread for each request received by the server. The problem is that Qt/PyQT doesn't allow these multiple threads create GUI objects, only in the main thread. To overcome this limitation, the multithreaded version can instantiate other single threaded versions of VisTrails and put them in a queue, so workflow executions and other GUI-related requests, such as generating workflow graphs and history trees can be forwarded to this queue, and each instance takes turns in answering the request. If the results are in the cache, the multithreaded version answers the requests directly.

Note that this infrastructure works on Linux only. To make this work on Windows, you have to create a script similar to start_vistrails_xvfb.sh (located in the scripts folder) where you can send the number of other instances via command-line options to VisTrails. The command line options are:

python vistrails_server.py -T <ADDRESS> -R <PORT> -O<NUMBER_OF_OTHER_VISTRAILS_INSTANCES> [-M]&

If you want the main vistrails instance to be multithreaded, use the -M at the end.

After creating this script, update function start_other_instances in vistrails/gui/application_server.py lines 1007-1023 and set the script variable to point to your script. You may also have to change the arguments sent to your script (line 1016: for example, you don't need to set a virtual display). You will need to change the path to the stop_vistrails_server.py script (on line 1026) according to your installation path. [http://www.vistrails.org/usersguide/batch.html#executing-workflows-in-parallel Executing Workflows in Parallel]

=== When a workflow is executed, what do the colors mean? ===

- lilac: module was notexecuted

- yellow: module is currently being executed

- green: module was successfully executed

- orange: module was cached

- red: the execution of the module failed

[http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

== Building workflows ==

=== Is there a way to give each widget a "display name" in addition to the module name at the center of the widget? ===

Yes, a "display name" can be assigned to a module by selecting the triangle in its top right corner to open a popup menu and selecting the Set Module Label... menu item. You will then be prompted to enter the "display name". [http://www.vistrails.org/usersguide/creating.html#changing-module-labels Changing Module Labels]

=== Is there a way to re-center the picture-in-picture (PiP) view? ===

Yes. If you click on the PIP window to bring it to focus, you can press Ctrl-R (or Command-R on Mac) to re-center the PiP window. [http://www.vistrails.org/usersguide/getting_started.html#vistrails-interaction Vistrails Interaction]

=== How do I search for a literal "?" (question mark) in the search box in the Property panel? ===

Since we allow regular expressions in our search box, question marks are treated as meta-characters. Thus, searching for "?" returns everything and "abc?" will return everything containing "abc". You need to use "\?" instead to search for "?". So the search for "??" would be "\?\?". [http://www.vistrails.org/usersguide/querying.html#textual-queries Textual Queries]

== Using VisTrails as a server ==

=== What is the VisTrails server-mode? ===

Using the VisTrails server mode, it is possible to ''execute workflows and control VisTrails through another application''. For example, the CrowdLabs Web portal (http://www.crowdlabs.org) accesses a VisTrails sever to execute workflows, retrieve and display vistrail trees and workflows. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

=== How do I execute workflows and control VisTrails through another application? ===

The way you access the server is by doing XML-RPC calls. In the current VisTrails release, we include a set of PHP scripts that can talk to a VisTrails server instance. They are in "extensions/http" folder. The files are reasonably well documented. Also, it should be not difficult to create python scripts to access the server (just use xmlrpclib module).

Note that the VisTrails server requires the provenance and workflows to be in a database. More detailed instructions on how to setup the server and the database
are available here:

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_server_setup.html

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_database_setup.html

If what you want is just to execute a series of workflows in batch mode, a simpler solution would be to ''use the VisTrails client in batch mode''. Chapter 12 of the user's guide contains detailed information and examples on that. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

== Control Flow ==

=== Note: using map ===

When using 'map', the module (or subworkflow) used as function port in the map module MUST be a function, i.e., it can only define 1 output port. [http://www.vistrails.org/usersguide/controlflow.html#the-map-operator The Map Operator]

== Spreadsheet ==

Where ''pipeline'' is a version number or a tag.

=== How can I save an image from the spreadsheet? ===

While having the focus on a spreadsheet cell and select the camera on the toolbar to take a snapshot. The system will prompt you for the location and file name where it should be saved. The other icons can be used for saving multiple images that can be used for generating an animation on demand. A whole sheet can also be saved by selecting Export (either from the menu or from the toolbar). [http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet-image Saving a Spreadsheet Image]

===Is it possible to save the complete state of the spreadsheet?===
[http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet Saving a Spreadsheet]

=== Can I view multiple sheets at the same time? ===

Yes. Each sheet on the spreadsheet can be displayed as a dock widget separated from the main spreadsheet window by dragging its tab name out of the tab bar at the bottom of the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Then, how can I put back a separated sheet? ===

A sheet can be docked back to the main window by dragging it back to the tab bar or double-click on its title bar. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== How can I order sheets on the spreadsheet? ===

This can be done by dragging the sheet name on the bottom top bar and drop it to the right place. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Can I control where a cell will be placed on the spreadsheet window? ===

By default, an unoccupied cell on the active sheet will be chosen to display the result. However, you can specify exactly in the pipeline where a spreadsheet cell will be placed by using CellLocation and SheetReference. CellLocation specifies the location (row and column) of a cell when connecting to a spreadsheet cell (VTKCell, ImageViewerCell, ...). Similarly, a SheetReference module (when connecting to a CellLocation) will specify which sheet the cell will be put on given its name, minimum row size and minimum column size. There is an example of this in examples/vtk.xml (select the version below Double Renderer). [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

=== How do I output results to the spreadsheet? ===

By inspecting the VisTrails Spreadsheet package (in the list of packages, to the left of the pipeline builder), you
can see there are built-in cells for different kinds of data, e.g., RichTextCell to display HTML and plain text. op
You (the user) can also define new cell types to display application-specific data. For example, we have developed
VtkCell, MplFigureCell, and OpenGLCell. It is possible to display pretty much anything on the Spreadsheet! [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

Examples of writing cell modules can be found in:
RichTextCell: packages/spreadsheet/widgets/richtext/richtext.py
VTK: packages/vtk/vtkcell.py

Here is the summary of some requirements on a cell widget:

(1) It must be a Qt widget. It should inherit from spreadsheet_cell.QCellWidget in the spreadsheet package. Although any Qt Widget would work, certain features such as animation will not be available (without rewriting it).

(2) It must re-implement the updateContents() function to take a set of inputs (usually coming from input ports of a wrapper Module) and display on the cells. VisTrails uses this function to update/reuse cells on the spreadsheet when new data comes in.

(3) It needs a wrapper VisTrails Module (inherited from basic_widgets.SpreadsheetCell of the spreadsheet package). Inside the compute() method of this module, it may call self.display(CellWidgetType, (inputs)) to trigger the display event on the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#advanced-cell-options Advanced Cell Options]

=== How do I control the default number of cells in the spreadsheet? ===

You can configure the rowCount and colCount using the preferences dialog. Just go to the Module Packages tab, select spreadsheet in the "Enabled packages" and press the Configure button. Then a list of all the configuration options for the spreadsheet will show up. [http://www.vistrails.org/usersguide/spreadsheet.html#custom-layout-options Custom Layout Options]

=== Is it possible to launch a web browser from the vistrails spreadsheet? We would like to output several urls from a parameter sweep and then have the option to click on each one to view the resulting page. I can view the page within the spreadsheet, but it is really too crowded.===

Currently, there isn't a widget that provides exactly this functionality, but I can think of a few solutions that may work for you:

(1) You can use parameter exploration to generate multiple sheets so you might have an exploration that opens each page in a new sheet. Use the third column/dimension in the exploration interface to have a parameter span sheets.

(2) The spreadsheet is extensible so you can write a custom spreadsheet cell widget that has a button or label with the desired link (a QLabel with openExternalLinks set to True, for example).

(3) You can tweak the existing RichTextCell be adding the line "self.browser.setOpenExternalLinks(True)" at line 63 of the source file "vistrails/packages/spreadsheet/widgets/richtext/richtext.py". Then, if your workflow creates a file with html markup text like "<a href="http://www.vistrails.org">VisTrails</a>" connected to a RichTextCell, clicking on the rendered link in the cell will open it in a web browser. You need to add the aforementioned line to the source to let Qt know that you want the link opened externally; by default, it will just issue an event that isn't processed. [http://www.vistrails.org/usersguide/spreadsheet.html#launching-a-web-browser Launching a Web Browser]

== Integrating your software into VisTrails ==

===How can I integrate my own program into VisTrails?===

The easiest way is to create a package. Writing a package is often very simple, here are instructions on how to do it: [[UsersGuideVisTrailsPackages]]

You can also dynamically generate modules. For an example see:

http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#Packages_that_generate_modules_dynamically

In particular, see the new_module call which uses python's type() function to generate new classes dynamically.

=== How do modules deal with multiple inputs in a same port? ===

(And should that even be allowed?)

For compatibility reasons, we do need to allow multiple connections to an input port. However, most package developers should never have to use this, and so we do our best to hide it. the default behavior for getting inputs from a port, then, is to always return a single input.

If on your module you need multiple inputs connected to a single port, use the 'forceGetInputListFromPort' method. It will return a list of all the data items coming through the port. The spreadsheet package uses this feature, so look there for usage examples (vistrails/packages/spreadsheet/basic_widgets.py)

=== Are there mechanisms for attaching widgets to different modules/parameters? ===

Right now, we have a mechanism for putting a specific widget for an input port. For example, if a port is SetColor(red, green, blue), we can put a color wheel widget there. Or we can also replace the SetFileName port with a File Widget. However, this is not per parameter (only per port). We are currently working on this problem.

=== Can I organize my package so it appears hierarchical in the module palette? ===

Yes. Use the namespace keyword argument when adding the module to the registry. For example,

registry.add_module(MyModule, namespace='MyNamespace')
[http://www.vistrails.org/usersguide/packages.html#module-hierarchy-and-visibility Module Hierarchy and Visibility]

=== Can I nest namespaces? ===

Yes. Use the '|' character to separate different the hierarchy. For example,

registry.add_module(MyModule, namespace='ParentNamespace|ChildNamespace')

[http://www.vistrails.org/usersguide/packages.html#module-hierarchy-and-visibility Module Hierarchy and Visibility]

=== Are there shortcuts for registry initialization? ===

Yes. If you define _modules as a list of classes in the __init__.py file of your package, VisTrails will attempt to load all classes specified as modules. You can provide add_module options as keyword arguments by specifying a tuple (class, kwargs) in the list. For example:

_modules = [MyModule1, (MyModule2, {'namespace': 'MyNamespace'})]

In addition, you need to identify the ports of your modules as a field in your class by defining _input_ports and _output_ports lists. Here, the items in each list must be tuples of the form (portName, portSignature, optional=False, sort_key=-1). For example:

class MyModule(Module):
def compute(self):
pass

_input_ports = [('firstInput', String), ('secondInput', Integer, True)]
_output_ports = [('firstOutput', String), ('secondOutput', String)]

=== Can I define ports to be of types that I do not import into my package? ===

Yes. You can pass an identifier string as the portSignature instead. The port_signature string is defined by:

<module_string> := <package_identifier>:[<namespace>|]<module_name>,
<port_signature> := (<module_string>*)

For example,

registry.add_input_port(MyModule, 'myInputPort', '(edu.utah.sci.vistrails.basic:String)')

or

_input_ports = [('myInputPort', '(edu.utah.sci.vistrails.basic:String)')]

=== What do I need to change in my package to make it reloadable (new in v1.4.2)? ===
See [http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#How_to_make_you_package_reloadable UsersGuideVisTrailsPackages] for an explanation.

=== Can I add default values or labels for parameters? ===

Yes. Versions 1.4 and greater support these features. See [[UsersGuideVisTrailsPackages#Adding_default_values_and.2For_labels_for_parameters | UsersGuideVisTrailsPacakges]] for more details.

=== I want to write a module to load HDF data whose output (e.g., data, string) varies according to the input I give it. Is is possible to do this in VisTrails, and if yes, how can I do that? Ideally, I would like to avoid having to change the connection of my output every time I change the input. ===

There are a few ways to tackle this - each has it's own benefits and pitfalls. Firstly, module connections do respect class hierarchies as we're familiar with in object oriented languages. For instance, A module can output a Constant of which String, Float, Integer, etc are specifications. In this way, you can have a subclass of something like HDFData be passed out of the module and the connections will be established regardless of the sub-type. This is a bit dangerous though. Modules downstream of such a class may not really know how to operate on certain types derived from the super-class. Extreme care must be taken both when creating the modules as well as connecting them to prevent things like this from happening.

A second method that I employ in several different packages is the idea of a container class. For instance, the NumSciPy package uses a relatively generic container "Numpy Array" to encapsulate the data. Of course, these encapsulating objects can store dictionaries that other modules can easily access and understand how to operate on. Although this method is slightly more work, the benefits of a stricter typing of ports is beneficial - particularly upon interfacing with other packages that may depend on strongly typed constants (for example).

=== I need to determine, at run-time, whether or not a "child" module is attached to the output port of a "parent" module. (I do not specifically need to know which child; just if there is one). ===

The outputPorts dictionary of the base Module stores this information. Thus, you should be able to check

("myPortName" in self.outputPorts)

on the parent module to check if there are any downstream connections from the port "myPortName". This might be used, for example, to only set results for output ports that will be used. ***Note***, however, that the caching algorithm assumes that all outputs are set so adding a new connection to a previously unconnected output port will not work as desired if that module is cached. For this reason, I would currently recommend making such a module not cacheable. Another possibility is overriding the update() method to check the outputPorts and set the upToDate flag if they are not equal. In a single, limited test, this seemed to work, but be warned that it is not fully tested. Here is an example:

class TestModule(Module):
_output_ports = [('a1', '(edu.utah.sci.vistrails.basic:String)'),
('a2', '(edu.utah.sci.vistrails.basic:String)')]
def __init__(self):
Module.__init__(self)
self._cached_output_ports = set()

def update(self):
if len(set(self.outputPorts) - self._cached_output_ports) > 0:
self.upToDate = False
Module.update(self)

def compute(self):
if "a1" in self.outputPorts:
self.setResult("a1", "test")
if "a2" in self.outputPorts:
self.setResult("a2", "test2")
self._cached_output_ports = set(self.outputPorts)

=== How can I make a module not display in the modules list? ===

You should set the abstract parameter to True when adding the module to the registry. Using the original syntax, this looks like:

def initialize():
reg = core.modules.module_registry.get_module_registry()
reg.add_module(InvisibleModule, abstract=True)
# ...

With the _modules dictionary shortcut (for more details, see [http://www.vistrails.org/index.php/FAQ#Are_there_shortucts_for_registry_initialization.3F the FAQ section on this]), you include it in a kwargs dict as part of a module tuple:

_modules = [AnotherModule, (InvisibleModule, {'abstract': True})]

There is also a 'hide_descriptor' parameter that prevents the module from appearing in the module palette without declaring it to be abstract.

The technical difference between the two is that 'abstract' will not add the item to the module palette while 'hide_descriptor' does add the item but immediately hides it. If the module should never be instantiated in a workflow, declare it abstract. If you don't want users to be able to add the module to a pipeline, but you have code that may add it programmatically, declare it with hide_descriptor=True.

== The Console ==

=== Where should I go to find out what I can call from the console and how to import it? ===

We have tried to make some methods more accessible in the console via an api. You can import the api via <code>import api</code> in the console and see the available methods with <code>dir(api)</code>. To open a vistrail:

import api
api.open_vistrail_from_file('/Applications/VisTrails/examples/terminator.vt')

To execute a version of a workflow, you currently have to go through the controller:

api.select_version('Histogram')
api.get_current_controller().execute_current_workflow()

Currently, only a subset of VisTrails functionality is directly available from the api. However, since VisTrails is written in python, you can dig down starting with the VistrailsApplication or controller object to expose most of our internal methods. If you have suggestions for calls to be added to the api, please let us know.

One other feature that we're working on, but is still in progress is the ability to construct workflows via the console. For example:

vtk = load_package('edu.utah.sci.vistrails.vtk')
vtk.vtkDataSetReader() # adds a vtkDataSetReader module to the pipeline
# click on the new module
a = selected_modules()[0] # get the one currently selected module
a.SetFile('/vistrails/examples/data/head120.vtk') # sets the SetFile parmaeter for the data set reader
b = vtk.vtkContourFilter() # adds a vtkContourFilter module to the pipeline and saves to var b
b.SetInputConnection0(a.GetOutputPort0()) # connects a's GetOutputPort0 port to b's SetInputConnection0

== Persistence Package ==

=== How do I use the output of one workflow as the input for another using the persistence package? ===

You need to configure the persistence modules using the module's configuration dialog. After adding a <code>PersistentOutputFile</code> to the workflow, click on the triangle in the upper-right corner of the <code>PersistentOutputFile</code>, and select "Edit Configuration" from the menu that appears. In this dialog, select "Create New Reference" and give the reference a name (and any space-delimited tags). Upon running that workflow, the data will be written to the persistent store. In the second workflow where you wish to use that file, add a <code>PersistentInputFile</code> and go to its configuration dialog in the same manner as with the output file. In that dialog, select "Use Existing Reference" and select the data that you just added in the first workflow in the list of files below. Now, when you run that workflow, it will grab the data from the persistent store.

Here is an example: [[Media:offscreen_persistent.vt | offscreen_persistent.vt]]. Run the "persistent offscreen" workflow first, and then run the "display persistent output" to use the output of the first workflow as the input for the second.

== VTK ==

=== Given a VTK visualization, how can I generate a webpage from it? ===

Check out the html pipeline in offscreen.xml.

=== I'm trying to use VTK, but there doesn't seem to be any output. What is wrong? ===

To use VTK on VisTrails, you need a slightly different way of connecting the renderer modules. Instead of using the standard RenderWindow/RenderWindowInteractor infrastructure, you simply connect the renderer to a VTKCell. The examples directory in the distribution has several VTK examples that illustrate.

=== I am trying to add a module to the workflow via Python, but how can I access vtk modules? ===

Here's an example:

import api

vtvtk = 'edu.utah.sci.vistrails.vtk'

module = api.add_module(0, 0, vtvtk, 'vtkContourFilter', '')

The third argument in add_module is the package identifier. You can find this in the "Module Packages" panel of the Preferences; just click on the package you're interested in and it will appear in the information on the right.

== matplotlib ==

=== I'm experiencing a problem with Latex labels and the matplotlib that comes with VisTrails 1.5. The script below entered to the interpreter that comes with VT is sufficient to reproduce it. ===

import matplotlib.pyplot as plt
plt.plot([1,2,3],[1,2,3])
plt.xlabel("$foo$")

Remove your ~/.matplotlib folder and re-start VisTrails

== VisTrails Development ==

=== I would like to build VisTrails from source. Are there instructions on how to do this? ===

Yes! Take a look at http://www.vistrails.org/index.php/Mac_Intel_Instructions
[http://www.vistrails.org/usersguide/getting_started.html#installing-vistrails-from-source Installing VisTrails from source]

== Accessing Provenance Information ==

=== How do I access the information in the execution log? ===

The code responsible for storing execution information is located in the "core/log" directories, and the code that generates much of that information is in "core/interpreter/cached.py". Modules can add execution-specific annotations to provenance via annotate() calls during execution, but much of the data (like timing and errors) is captured by the LogController and CachedInterpreter (the execution engine) objects. To analyze the log from a vistrail (.vt) file, you might have something like the following:

<code>
import core.log.log
import db.services.io

def run(fname):
# open the .vt bundle specified by the filename "fname"
bundle = db.services.io.open_vistrail_bundle_from_zip_xml(fname)[0]
# get the log filename
log_fname = bundle.vistrail.db_log_filename
if log_fname is not None:
# open the log
log = db.services.io.open_log_from_xml(log_fname, True)
# convert the log from a db object
core.log.log.Log.convert(log)
for workflow_exec in log.workflow_execs:
print 'workflow version:', workflow_exec.parent_version
print 'time started:', workflow_exec.ts_start
print 'time ended:', workflow_exec.ts_end
print 'modules executed:', [i.module_id
for i in workflow_exec.item_execs]
if __name__ == '__main__':
run("some_vistrail.vt")
</code>

You should be able to see what information is available by looking at the "core/log" classes. [http://www.vistrails.org/usersguide/log.html Accessing the Execution Log]

FAQ

2011-03-31T23:37:00Z

Ctuttle: /* Can I organize my package so it appears hierarchical in the module palette? */

Also check our [[known_issues|Known Issues]] page for troubleshooting.

----
== Running workflows ==

=== How can I run a workflow using the command line? ===

('''Updated for version 1.2''') Call vistrails using the following options:

<code>python vistrails.py -b path_to_vistrails_file:pipeline </code>

where pipeline can be a version '''tag name''' or version '''id'''

'''NOTE:''' If you downloaded the MacOS X bundle, you can run vistrails from the command line via the following commands in the Terminal. Change the current directory to wherever VisTrails was installed (often <code>/Applications</code>), and then type:

<code>Vistrails.app/Contents/MacOS/vistrails [<cmd_line_options>]</code> [http://www.vistrails.org/usersguide/batch.html#running-a-specific-workflow-in-batch-mode Running a Specific Workflow in Batch Mode]

=== Using the command line, we'd like to execute a workflow multiple times, with slightly different parameters, and create a series of output files. Is this possible?===

('''Updated for version 1.2''') We can change parameters that have an alias through the command line.

For example, offscreen pipeline in offscreen.vt always creates the file called image.png. If you want generate it with a different filename:

<code>python vistrails.py -b ../examples/offscreen.vt:offscreen -a"filename=other.png"</code>

<code>filename</code> in the example above is the alias name assigned to the parameter in the <code>value</code> method inside the String module. When running a pipeline from the command line, VisTrails will try to start the spreadsheet automatically if the pipeline requires it. For example, this other execution will also start the spreadsheet (attention to how $ characters are escaped when running on bash):

<code>python vistrails.py -b ../examples/head.vt:aliases -a"isovalue=30\$&\$diffuse_color=0.8,0.4,0.2"</code>

You can also execute more than one pipeline on the command line:

<code>python vistrails.py -b ../examples/head.vt:aliases ../examples/spx.vt:spx \
-a"isovalue=30"</code>

Use the -a parameter only once regardless the number of pipelines. [http://www.vistrails.org/usersguide/batch.html#running-a-workflow-with-specific-parameters Running a Workflow with Specific Parameters]

=== I can load a vistrail, and the version tree shows up fine. However, no pipelines appear when I click on a version. What gives? ===

The most likely reason is that the vistrail uses a package that is not registered with VisTrails. You need to identify the needed package and add it to your <code>.vistrails/startup.py</code>. A single line like the following should be enough:

addPackage('enter_package_name_here')

Some packages might need more information. For example:

addPackage('afront', executable_path='/path/to/afront')

Refer to the package documentation for details. The one inconvenient step is that currently there's no automated way to describe what is the missing package. We're working on this feature for future releases.

=== I have a workflow that reads a file and then does some processing. The first time it runs, it executes correctly. But in subsequent, nothing happens. ===

VisTrails caches by default, so after a workflow is executed, if none of its parameters change, it won't be executed again.

If a workflow reads a file using the basic module File, VisTrails does check whether the file was modified since the last run. It does so by keeping a signature that is based on the modification time of the file. And if the file was modified, the File module and all downstream modules (the ones which depend on File) will be executed.

''Note: If you would like your input and output data to be versioned, you can use the Persistence package.''

If you do not want VisTrails to cache executions, you can turn off caching: go to Menu Edit -> Preferences and in the General Configuration tab, change Cache execution results to Never. [http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

=== Can VisTrails execute workflows in parallel? ===

The VisTrails server can only execute pipelines in parallel if there's more than one instance of VisTrails running. The command

<code> self.rpcserver = ThreadedXMLRPCServer((self.temp_xml_rpc_options.server, self.temp_xml_rpc_options.port))</code>

starts a multithreaded version of the XML-RPC server, so it will create a thread for each request received by the server. The problem is that Qt/PyQT doesn't allow these multiple threads create GUI objects, only in the main thread. To overcome this limitation, the multithreaded version can instantiate other single threaded versions of VisTrails and put them in a queue, so workflow executions and other GUI-related requests, such as generating workflow graphs and history trees can be forwarded to this queue, and each instance takes turns in answering the request. If the results are in the cache, the multithreaded version answers the requests directly.

Note that this infrastructure works on Linux only. To make this work on Windows, you have to create a script similar to start_vistrails_xvfb.sh (located in the scripts folder) where you can send the number of other instances via command-line options to VisTrails. The command line options are:

python vistrails_server.py -T <ADDRESS> -R <PORT> -O<NUMBER_OF_OTHER_VISTRAILS_INSTANCES> [-M]&

If you want the main vistrails instance to be multithreaded, use the -M at the end.

After creating this script, update function start_other_instances in vistrails/gui/application_server.py lines 1007-1023 and set the script variable to point to your script. You may also have to change the arguments sent to your script (line 1016: for example, you don't need to set a virtual display). You will need to change the path to the stop_vistrails_server.py script (on line 1026) according to your installation path. [http://www.vistrails.org/usersguide/batch.html#executing-workflows-in-parallel Executing Workflows in Parallel]

=== When a workflow is executed, what do the colors mean? ===

- lilac: module was notexecuted

- yellow: module is currently being executed

- green: module was successfully executed

- orange: module was cached

- red: the execution of the module failed

[http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

== Building workflows ==

=== Is there a way to give each widget a "display name" in addition to the module name at the center of the widget? ===

Yes, a "display name" can be assigned to a module by selecting the triangle in its top right corner to open a popup menu and selecting the Set Module Label... menu item. You will then be prompted to enter the "display name". [http://www.vistrails.org/usersguide/creating.html#changing-module-labels Changing Module Labels]

=== Is there a way to re-center the picture-in-picture (PiP) view? ===

Yes. If you click on the PIP window to bring it to focus, you can press Ctrl-R (or Command-R on Mac) to re-center the PiP window. [http://www.vistrails.org/usersguide/getting_started.html#vistrails-interaction Vistrails Interaction]

=== How do I search for a literal "?" (question mark) in the search box in the Property panel? ===

Since we allow regular expressions in our search box, question marks are treated as meta-characters. Thus, searching for "?" returns everything and "abc?" will return everything containing "abc". You need to use "\?" instead to search for "?". So the search for "??" would be "\?\?". [http://www.vistrails.org/usersguide/querying.html#textual-queries Textual Queries]

== Using VisTrails as a server ==

=== What is the VisTrails server-mode? ===

Using the VisTrails server mode, it is possible to ''execute workflows and control VisTrails through another application''. For example, the CrowdLabs Web portal (http://www.crowdlabs.org) accesses a VisTrails sever to execute workflows, retrieve and display vistrail trees and workflows. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

=== How do I execute workflows and control VisTrails through another application? ===

The way you access the server is by doing XML-RPC calls. In the current VisTrails release, we include a set of PHP scripts that can talk to a VisTrails server instance. They are in "extensions/http" folder. The files are reasonably well documented. Also, it should be not difficult to create python scripts to access the server (just use xmlrpclib module).

Note that the VisTrails server requires the provenance and workflows to be in a database. More detailed instructions on how to setup the server and the database
are available here:

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_server_setup.html

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_database_setup.html

If what you want is just to execute a series of workflows in batch mode, a simpler solution would be to ''use the VisTrails client in batch mode''. Chapter 12 of the user's guide contains detailed information and examples on that. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

== Control Flow ==

=== Note: using map ===

When using 'map', the module (or subworkflow) used as function port in the map module MUST be a function, i.e., it can only define 1 output port. [http://www.vistrails.org/usersguide/controlflow.html#the-map-operator The Map Operator]

== Spreadsheet ==

Where ''pipeline'' is a version number or a tag.

=== How can I save an image from the spreadsheet? ===

While having the focus on a spreadsheet cell and select the camera on the toolbar to take a snapshot. The system will prompt you for the location and file name where it should be saved. The other icons can be used for saving multiple images that can be used for generating an animation on demand. A whole sheet can also be saved by selecting Export (either from the menu or from the toolbar). [http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet-image Saving a Spreadsheet Image]

===Is it possible to save the complete state of the spreadsheet?===
[http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet Saving a Spreadsheet]

=== Can I view multiple sheets at the same time? ===

Yes. Each sheet on the spreadsheet can be displayed as a dock widget separated from the main spreadsheet window by dragging its tab name out of the tab bar at the bottom of the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Then, how can I put back a separated sheet? ===

A sheet can be docked back to the main window by dragging it back to the tab bar or double-click on its title bar. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== How can I order sheets on the spreadsheet? ===

This can be done by dragging the sheet name on the bottom top bar and drop it to the right place. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Can I control where a cell will be placed on the spreadsheet window? ===

By default, an unoccupied cell on the active sheet will be chosen to display the result. However, you can specify exactly in the pipeline where a spreadsheet cell will be placed by using CellLocation and SheetReference. CellLocation specifies the location (row and column) of a cell when connecting to a spreadsheet cell (VTKCell, ImageViewerCell, ...). Similarly, a SheetReference module (when connecting to a CellLocation) will specify which sheet the cell will be put on given its name, minimum row size and minimum column size. There is an example of this in examples/vtk.xml (select the version below Double Renderer). [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

=== How do I output results to the spreadsheet? ===

By inspecting the VisTrails Spreadsheet package (in the list of packages, to the left of the pipeline builder), you
can see there are built-in cells for different kinds of data, e.g., RichTextCell to display HTML and plain text. op
You (the user) can also define new cell types to display application-specific data. For example, we have developed
VtkCell, MplFigureCell, and OpenGLCell. It is possible to display pretty much anything on the Spreadsheet! [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

Examples of writing cell modules can be found in:
RichTextCell: packages/spreadsheet/widgets/richtext/richtext.py
VTK: packages/vtk/vtkcell.py

Here is the summary of some requirements on a cell widget:

(1) It must be a Qt widget. It should inherit from spreadsheet_cell.QCellWidget in the spreadsheet package. Although any Qt Widget would work, certain features such as animation will not be available (without rewriting it).

(2) It must re-implement the updateContents() function to take a set of inputs (usually coming from input ports of a wrapper Module) and display on the cells. VisTrails uses this function to update/reuse cells on the spreadsheet when new data comes in.

(3) It needs a wrapper VisTrails Module (inherited from basic_widgets.SpreadsheetCell of the spreadsheet package). Inside the compute() method of this module, it may call self.display(CellWidgetType, (inputs)) to trigger the display event on the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#advanced-cell-options Advanced Cell Options]

=== How do I control the default number of cells in the spreadsheet? ===

You can configure the rowCount and colCount using the preferences dialog. Just go to the Module Packages tab, select spreadsheet in the "Enabled packages" and press the Configure button. Then a list of all the configuration options for the spreadsheet will show up. [http://www.vistrails.org/usersguide/spreadsheet.html#custom-layout-options Custom Layout Options]

=== Is it possible to launch a web browser from the vistrails spreadsheet? We would like to output several urls from a parameter sweep and then have the option to click on each one to view the resulting page. I can view the page within the spreadsheet, but it is really too crowded.===

Currently, there isn't a widget that provides exactly this functionality, but I can think of a few solutions that may work for you:

(1) You can use parameter exploration to generate multiple sheets so you might have an exploration that opens each page in a new sheet. Use the third column/dimension in the exploration interface to have a parameter span sheets.

(2) The spreadsheet is extensible so you can write a custom spreadsheet cell widget that has a button or label with the desired link (a QLabel with openExternalLinks set to True, for example).

(3) You can tweak the existing RichTextCell be adding the line "self.browser.setOpenExternalLinks(True)" at line 63 of the source file "vistrails/packages/spreadsheet/widgets/richtext/richtext.py". Then, if your workflow creates a file with html markup text like "<a href="http://www.vistrails.org">VisTrails</a>" connected to a RichTextCell, clicking on the rendered link in the cell will open it in a web browser. You need to add the aforementioned line to the source to let Qt know that you want the link opened externally; by default, it will just issue an event that isn't processed. [http://www.vistrails.org/usersguide/spreadsheet.html#launching-a-web-browser Launching a Web Browser]

== Integrating your software into VisTrails ==

===How can I integrate my own program into VisTrails?===

The easiest way is to create a package. Writing a package is often very simple, here are instructions on how to do it: [[UsersGuideVisTrailsPackages]]

You can also dynamically generate modules. For an example see:

http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#Packages_that_generate_modules_dynamically

In particular, see the new_module call which uses python's type() function to generate new classes dynamically.

=== How do modules deal with multiple inputs in a same port? ===

(And should that even be allowed?)

For compatibility reasons, we do need to allow multiple connections to an input port. However, most package developers should never have to use this, and so we do our best to hide it. the default behavior for getting inputs from a port, then, is to always return a single input.

If on your module you need multiple inputs connected to a single port, use the 'forceGetInputListFromPort' method. It will return a list of all the data items coming through the port. The spreadsheet package uses this feature, so look there for usage examples (vistrails/packages/spreadsheet/basic_widgets.py)

=== Are there mechanisms for attaching widgets to different modules/parameters? ===

Right now, we have a mechanism for putting a specific widget for an input port. For example, if a port is SetColor(red, green, blue), we can put a color wheel widget there. Or we can also replace the SetFileName port with a File Widget. However, this is not per parameter (only per port). We are currently working on this problem.

=== Can I organize my package so it appears hierarchical in the module palette? ===

Yes. Use the namespace keyword argument when adding the module to the registry. For example,

registry.add_module(MyModule, namespace='MyNamespace')
[http://www.vistrails.org/usersguide/packages.html#module-hierarchy-and-visibility Module Hierarchy and Visibility]

=== Can I nest namespaces? ===

Yes. Use the '|' character to separate different the hierarchy. For example,

registry.add_module(MyModule, namespace='ParentNamespace|ChildNamespace')

=== Are there shortcuts for registry initialization? ===

Yes. If you define _modules as a list of classes in the __init__.py file of your package, VisTrails will attempt to load all classes specified as modules. You can provide add_module options as keyword arguments by specifying a tuple (class, kwargs) in the list. For example:

_modules = [MyModule1, (MyModule2, {'namespace': 'MyNamespace'})]

In addition, you need to identify the ports of your modules as a field in your class by defining _input_ports and _output_ports lists. Here, the items in each list must be tuples of the form (portName, portSignature, optional=False, sort_key=-1). For example:

class MyModule(Module):
def compute(self):
pass

_input_ports = [('firstInput', String), ('secondInput', Integer, True)]
_output_ports = [('firstOutput', String), ('secondOutput', String)]

=== Can I define ports to be of types that I do not import into my package? ===

Yes. You can pass an identifier string as the portSignature instead. The port_signature string is defined by:

<module_string> := <package_identifier>:[<namespace>|]<module_name>,
<port_signature> := (<module_string>*)

For example,

registry.add_input_port(MyModule, 'myInputPort', '(edu.utah.sci.vistrails.basic:String)')

or

_input_ports = [('myInputPort', '(edu.utah.sci.vistrails.basic:String)')]

=== What do I need to change in my package to make it reloadable (new in v1.4.2)? ===
See [http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#How_to_make_you_package_reloadable UsersGuideVisTrailsPackages] for an explanation.

=== Can I add default values or labels for parameters? ===

Yes. Versions 1.4 and greater support these features. See [[UsersGuideVisTrailsPackages#Adding_default_values_and.2For_labels_for_parameters | UsersGuideVisTrailsPacakges]] for more details.

=== I want to write a module to load HDF data whose output (e.g., data, string) varies according to the input I give it. Is is possible to do this in VisTrails, and if yes, how can I do that? Ideally, I would like to avoid having to change the connection of my output every time I change the input. ===

There are a few ways to tackle this - each has it's own benefits and pitfalls. Firstly, module connections do respect class hierarchies as we're familiar with in object oriented languages. For instance, A module can output a Constant of which String, Float, Integer, etc are specifications. In this way, you can have a subclass of something like HDFData be passed out of the module and the connections will be established regardless of the sub-type. This is a bit dangerous though. Modules downstream of such a class may not really know how to operate on certain types derived from the super-class. Extreme care must be taken both when creating the modules as well as connecting them to prevent things like this from happening.

A second method that I employ in several different packages is the idea of a container class. For instance, the NumSciPy package uses a relatively generic container "Numpy Array" to encapsulate the data. Of course, these encapsulating objects can store dictionaries that other modules can easily access and understand how to operate on. Although this method is slightly more work, the benefits of a stricter typing of ports is beneficial - particularly upon interfacing with other packages that may depend on strongly typed constants (for example).

=== I need to determine, at run-time, whether or not a "child" module is attached to the output port of a "parent" module. (I do not specifically need to know which child; just if there is one). ===

The outputPorts dictionary of the base Module stores this information. Thus, you should be able to check

("myPortName" in self.outputPorts)

on the parent module to check if there are any downstream connections from the port "myPortName". This might be used, for example, to only set results for output ports that will be used. ***Note***, however, that the caching algorithm assumes that all outputs are set so adding a new connection to a previously unconnected output port will not work as desired if that module is cached. For this reason, I would currently recommend making such a module not cacheable. Another possibility is overriding the update() method to check the outputPorts and set the upToDate flag if they are not equal. In a single, limited test, this seemed to work, but be warned that it is not fully tested. Here is an example:

class TestModule(Module):
_output_ports = [('a1', '(edu.utah.sci.vistrails.basic:String)'),
('a2', '(edu.utah.sci.vistrails.basic:String)')]
def __init__(self):
Module.__init__(self)
self._cached_output_ports = set()

def update(self):
if len(set(self.outputPorts) - self._cached_output_ports) > 0:
self.upToDate = False
Module.update(self)

def compute(self):
if "a1" in self.outputPorts:
self.setResult("a1", "test")
if "a2" in self.outputPorts:
self.setResult("a2", "test2")
self._cached_output_ports = set(self.outputPorts)

=== How can I make a module not display in the modules list? ===

You should set the abstract parameter to True when adding the module to the registry. Using the original syntax, this looks like:

def initialize():
reg = core.modules.module_registry.get_module_registry()
reg.add_module(InvisibleModule, abstract=True)
# ...

With the _modules dictionary shortcut (for more details, see [http://www.vistrails.org/index.php/FAQ#Are_there_shortucts_for_registry_initialization.3F the FAQ section on this]), you include it in a kwargs dict as part of a module tuple:

_modules = [AnotherModule, (InvisibleModule, {'abstract': True})]

There is also a 'hide_descriptor' parameter that prevents the module from appearing in the module palette without declaring it to be abstract.

The technical difference between the two is that 'abstract' will not add the item to the module palette while 'hide_descriptor' does add the item but immediately hides it. If the module should never be instantiated in a workflow, declare it abstract. If you don't want users to be able to add the module to a pipeline, but you have code that may add it programmatically, declare it with hide_descriptor=True.

== The Console ==

=== Where should I go to find out what I can call from the console and how to import it? ===

We have tried to make some methods more accessible in the console via an api. You can import the api via <code>import api</code> in the console and see the available methods with <code>dir(api)</code>. To open a vistrail:

import api
api.open_vistrail_from_file('/Applications/VisTrails/examples/terminator.vt')

To execute a version of a workflow, you currently have to go through the controller:

api.select_version('Histogram')
api.get_current_controller().execute_current_workflow()

Currently, only a subset of VisTrails functionality is directly available from the api. However, since VisTrails is written in python, you can dig down starting with the VistrailsApplication or controller object to expose most of our internal methods. If you have suggestions for calls to be added to the api, please let us know.

One other feature that we're working on, but is still in progress is the ability to construct workflows via the console. For example:

vtk = load_package('edu.utah.sci.vistrails.vtk')
vtk.vtkDataSetReader() # adds a vtkDataSetReader module to the pipeline
# click on the new module
a = selected_modules()[0] # get the one currently selected module
a.SetFile('/vistrails/examples/data/head120.vtk') # sets the SetFile parmaeter for the data set reader
b = vtk.vtkContourFilter() # adds a vtkContourFilter module to the pipeline and saves to var b
b.SetInputConnection0(a.GetOutputPort0()) # connects a's GetOutputPort0 port to b's SetInputConnection0

== Persistence Package ==

=== How do I use the output of one workflow as the input for another using the persistence package? ===

You need to configure the persistence modules using the module's configuration dialog. After adding a <code>PersistentOutputFile</code> to the workflow, click on the triangle in the upper-right corner of the <code>PersistentOutputFile</code>, and select "Edit Configuration" from the menu that appears. In this dialog, select "Create New Reference" and give the reference a name (and any space-delimited tags). Upon running that workflow, the data will be written to the persistent store. In the second workflow where you wish to use that file, add a <code>PersistentInputFile</code> and go to its configuration dialog in the same manner as with the output file. In that dialog, select "Use Existing Reference" and select the data that you just added in the first workflow in the list of files below. Now, when you run that workflow, it will grab the data from the persistent store.

Here is an example: [[Media:offscreen_persistent.vt | offscreen_persistent.vt]]. Run the "persistent offscreen" workflow first, and then run the "display persistent output" to use the output of the first workflow as the input for the second.

== VTK ==

=== Given a VTK visualization, how can I generate a webpage from it? ===

Check out the html pipeline in offscreen.xml.

=== I'm trying to use VTK, but there doesn't seem to be any output. What is wrong? ===

To use VTK on VisTrails, you need a slightly different way of connecting the renderer modules. Instead of using the standard RenderWindow/RenderWindowInteractor infrastructure, you simply connect the renderer to a VTKCell. The examples directory in the distribution has several VTK examples that illustrate.

=== I am trying to add a module to the workflow via Python, but how can I access vtk modules? ===

Here's an example:

import api

vtvtk = 'edu.utah.sci.vistrails.vtk'

module = api.add_module(0, 0, vtvtk, 'vtkContourFilter', '')

The third argument in add_module is the package identifier. You can find this in the "Module Packages" panel of the Preferences; just click on the package you're interested in and it will appear in the information on the right.

== matplotlib ==

=== I'm experiencing a problem with Latex labels and the matplotlib that comes with VisTrails 1.5. The script below entered to the interpreter that comes with VT is sufficient to reproduce it. ===

import matplotlib.pyplot as plt
plt.plot([1,2,3],[1,2,3])
plt.xlabel("$foo$")

Remove your ~/.matplotlib folder and re-start VisTrails

== VisTrails Development ==

=== I would like to build VisTrails from source. Are there instructions on how to do this? ===

Yes! Take a look at http://www.vistrails.org/index.php/Mac_Intel_Instructions
[http://www.vistrails.org/usersguide/getting_started.html#installing-vistrails-from-source Installing VisTrails from source]

== Accessing Provenance Information ==

=== How do I access the information in the execution log? ===

The code responsible for storing execution information is located in the "core/log" directories, and the code that generates much of that information is in "core/interpreter/cached.py". Modules can add execution-specific annotations to provenance via annotate() calls during execution, but much of the data (like timing and errors) is captured by the LogController and CachedInterpreter (the execution engine) objects. To analyze the log from a vistrail (.vt) file, you might have something like the following:

<code>
import core.log.log
import db.services.io

def run(fname):
# open the .vt bundle specified by the filename "fname"
bundle = db.services.io.open_vistrail_bundle_from_zip_xml(fname)[0]
# get the log filename
log_fname = bundle.vistrail.db_log_filename
if log_fname is not None:
# open the log
log = db.services.io.open_log_from_xml(log_fname, True)
# convert the log from a db object
core.log.log.Log.convert(log)
for workflow_exec in log.workflow_execs:
print 'workflow version:', workflow_exec.parent_version
print 'time started:', workflow_exec.ts_start
print 'time ended:', workflow_exec.ts_end
print 'modules executed:', [i.module_id
for i in workflow_exec.item_execs]
if __name__ == '__main__':
run("some_vistrail.vt")
</code>

You should be able to see what information is available by looking at the "core/log" classes. [http://www.vistrails.org/usersguide/log.html Accessing the Execution Log]

FAQ

2011-03-31T16:09:50Z

Also check our [[known_issues|Known Issues]] page for troubleshooting.

----
== Running workflows ==

=== How can I run a workflow using the command line? ===

('''Updated for version 1.2''') Call vistrails using the following options:

<code>python vistrails.py -b path_to_vistrails_file:pipeline </code>

where pipeline can be a version '''tag name''' or version '''id'''

'''NOTE:''' If you downloaded the MacOS X bundle, you can run vistrails from the command line via the following commands in the Terminal. Change the current directory to wherever VisTrails was installed (often <code>/Applications</code>), and then type:

<code>Vistrails.app/Contents/MacOS/vistrails [<cmd_line_options>]</code> [http://www.vistrails.org/usersguide/batch.html#running-a-specific-workflow-in-batch-mode Running a Specific Workflow in Batch Mode]

=== Using the command line, we'd like to execute a workflow multiple times, with slightly different parameters, and create a series of output files. Is this possible?===

('''Updated for version 1.2''') We can change parameters that have an alias through the command line.

For example, offscreen pipeline in offscreen.vt always creates the file called image.png. If you want generate it with a different filename:

<code>python vistrails.py -b ../examples/offscreen.vt:offscreen -a"filename=other.png"</code>

<code>filename</code> in the example above is the alias name assigned to the parameter in the <code>value</code> method inside the String module. When running a pipeline from the command line, VisTrails will try to start the spreadsheet automatically if the pipeline requires it. For example, this other execution will also start the spreadsheet (attention to how $ characters are escaped when running on bash):

<code>python vistrails.py -b ../examples/head.vt:aliases -a"isovalue=30\$&\$diffuse_color=0.8,0.4,0.2"</code>

You can also execute more than one pipeline on the command line:

<code>python vistrails.py -b ../examples/head.vt:aliases ../examples/spx.vt:spx \
-a"isovalue=30"</code>

Use the -a parameter only once regardless the number of pipelines. [http://www.vistrails.org/usersguide/batch.html#running-a-workflow-with-specific-parameters Running a Workflow with Specific Parameters]

=== I can load a vistrail, and the version tree shows up fine. However, no pipelines appear when I click on a version. What gives? ===

The most likely reason is that the vistrail uses a package that is not registered with VisTrails. You need to identify the needed package and add it to your <code>.vistrails/startup.py</code>. A single line like the following should be enough:

addPackage('enter_package_name_here')

Some packages might need more information. For example:

addPackage('afront', executable_path='/path/to/afront')

Refer to the package documentation for details. The one inconvenient step is that currently there's no automated way to describe what is the missing package. We're working on this feature for future releases.

=== I have a workflow that reads a file and then does some processing. The first time it runs, it executes correctly. But in subsequent, nothing happens. ===

VisTrails caches by default, so after a workflow is executed, if none of its parameters change, it won't be executed again.

If a workflow reads a file using the basic module File, VisTrails does check whether the file was modified since the last run. It does so by keeping a signature that is based on the modification time of the file. And if the file was modified, the File module and all downstream modules (the ones which depend on File) will be executed.

''Note: If you would like your input and output data to be versioned, you can use the Persistence package.''

If you do not want VisTrails to cache executions, you can turn off caching: go to Menu Edit -> Preferences and in the General Configuration tab, change Cache execution results to Never. [http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

=== Can VisTrails execute workflows in parallel? ===

The VisTrails server can only execute pipelines in parallel if there's more than one instance of VisTrails running. The command

<code> self.rpcserver = ThreadedXMLRPCServer((self.temp_xml_rpc_options.server, self.temp_xml_rpc_options.port))</code>

starts a multithreaded version of the XML-RPC server, so it will create a thread for each request received by the server. The problem is that Qt/PyQT doesn't allow these multiple threads create GUI objects, only in the main thread. To overcome this limitation, the multithreaded version can instantiate other single threaded versions of VisTrails and put them in a queue, so workflow executions and other GUI-related requests, such as generating workflow graphs and history trees can be forwarded to this queue, and each instance takes turns in answering the request. If the results are in the cache, the multithreaded version answers the requests directly.

Note that this infrastructure works on Linux only. To make this work on Windows, you have to create a script similar to start_vistrails_xvfb.sh (located in the scripts folder) where you can send the number of other instances via command-line options to VisTrails. The command line options are:

python vistrails_server.py -T <ADDRESS> -R <PORT> -O<NUMBER_OF_OTHER_VISTRAILS_INSTANCES> [-M]&

If you want the main vistrails instance to be multithreaded, use the -M at the end.

After creating this script, update function start_other_instances in vistrails/gui/application_server.py lines 1007-1023 and set the script variable to point to your script. You may also have to change the arguments sent to your script (line 1016: for example, you don't need to set a virtual display). You will need to change the path to the stop_vistrails_server.py script (on line 1026) according to your installation path. [http://www.vistrails.org/usersguide/batch.html#executing-workflows-in-parallel Executing Workflows in Parallel]

=== When a workflow is executed, what do the colors mean? ===

- lilac: module was notexecuted

- yellow: module is currently being executed

- green: module was successfully executed

- orange: module was cached

- red: the execution of the module failed

[http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

== Building workflows ==

=== Is there a way to give each widget a "display name" in addition to the module name at the center of the widget? ===

Yes, a "display name" can be assigned to a module by selecting the triangle in its top right corner to open a popup menu and selecting the Set Module Label... menu item. You will then be prompted to enter the "display name". [http://www.vistrails.org/usersguide/creating.html#changing-module-labels Changing Module Labels]

=== Is there a way to re-center the picture-in-picture (PiP) view? ===

Yes. If you click on the PIP window to bring it to focus, you can press Ctrl-R (or Command-R on Mac) to re-center the PiP window. [http://www.vistrails.org/usersguide/getting_started.html#vistrails-interaction Vistrails Interaction]

=== How do I search for a literal "?" (question mark) in the search box in the Property panel? ===

Since we allow regular expressions in our search box, question marks are treated as meta-characters. Thus, searching for "?" returns everything and "abc?" will return everything containing "abc". You need to use "\?" instead to search for "?". So the search for "??" would be "\?\?". [http://www.vistrails.org/usersguide/querying.html#textual-queries Textual Queries]

== Using VisTrails as a server ==

=== What is the VisTrails server-mode? ===

Using the VisTrails server mode, it is possible to ''execute workflows and control VisTrails through another application''. For example, the CrowdLabs Web portal (http://www.crowdlabs.org) accesses a VisTrails sever to execute workflows, retrieve and display vistrail trees and workflows. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

=== How do I execute workflows and control VisTrails through another application? ===

The way you access the server is by doing XML-RPC calls. In the current VisTrails release, we include a set of PHP scripts that can talk to a VisTrails server instance. They are in "extensions/http" folder. The files are reasonably well documented. Also, it should be not difficult to create python scripts to access the server (just use xmlrpclib module).

Note that the VisTrails server requires the provenance and workflows to be in a database. More detailed instructions on how to setup the server and the database
are available here:

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_server_setup.html

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_database_setup.html

If what you want is just to execute a series of workflows in batch mode, a simpler solution would be to ''use the VisTrails client in batch mode''. Chapter 12 of the user's guide contains detailed information and examples on that. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

== Control Flow ==

=== Note: using map ===

When using 'map', the module (or subworkflow) used as function port in the map module MUST be a function, i.e., it can only define 1 output port. [http://www.vistrails.org/usersguide/controlflow.html#the-map-operator The Map Operator]

== Spreadsheet ==

Where ''pipeline'' is a version number or a tag.

=== How can I save an image from the spreadsheet? ===

While having the focus on a spreadsheet cell and select the camera on the toolbar to take a snapshot. The system will prompt you for the location and file name where it should be saved. The other icons can be used for saving multiple images that can be used for generating an animation on demand. A whole sheet can also be saved by selecting Export (either from the menu or from the toolbar). [http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet-image Saving a Spreadsheet Image]

===Is it possible to save the complete state of the spreadsheet?===
[http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet Saving a Spreadsheet]

=== Can I view multiple sheets at the same time? ===

Yes. Each sheet on the spreadsheet can be displayed as a dock widget separated from the main spreadsheet window by dragging its tab name out of the tab bar at the bottom of the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Then, how can I put back a separated sheet? ===

A sheet can be docked back to the main window by dragging it back to the tab bar or double-click on its title bar. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== How can I order sheets on the spreadsheet? ===

This can be done by dragging the sheet name on the bottom top bar and drop it to the right place. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Can I control where a cell will be placed on the spreadsheet window? ===

By default, an unoccupied cell on the active sheet will be chosen to display the result. However, you can specify exactly in the pipeline where a spreadsheet cell will be placed by using CellLocation and SheetReference. CellLocation specifies the location (row and column) of a cell when connecting to a spreadsheet cell (VTKCell, ImageViewerCell, ...). Similarly, a SheetReference module (when connecting to a CellLocation) will specify which sheet the cell will be put on given its name, minimum row size and minimum column size. There is an example of this in examples/vtk.xml (select the version below Double Renderer). [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

=== How do I output results to the spreadsheet? ===

By inspecting the VisTrails Spreadsheet package (in the list of packages, to the left of the pipeline builder), you
can see there are built-in cells for different kinds of data, e.g., RichTextCell to display HTML and plain text. op
You (the user) can also define new cell types to display application-specific data. For example, we have developed
VtkCell, MplFigureCell, and OpenGLCell. It is possible to display pretty much anything on the Spreadsheet! [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

Examples of writing cell modules can be found in:
RichTextCell: packages/spreadsheet/widgets/richtext/richtext.py
VTK: packages/vtk/vtkcell.py

Here is the summary of some requirements on a cell widget:

(1) It must be a Qt widget. It should inherit from spreadsheet_cell.QCellWidget in the spreadsheet package. Although any Qt Widget would work, certain features such as animation will not be available (without rewriting it).

(2) It must re-implement the updateContents() function to take a set of inputs (usually coming from input ports of a wrapper Module) and display on the cells. VisTrails uses this function to update/reuse cells on the spreadsheet when new data comes in.

(3) It needs a wrapper VisTrails Module (inherited from basic_widgets.SpreadsheetCell of the spreadsheet package). Inside the compute() method of this module, it may call self.display(CellWidgetType, (inputs)) to trigger the display event on the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#advanced-cell-options Advanced Cell Options]

=== How do I control the default number of cells in the spreadsheet? ===

You can configure the rowCount and colCount using the preferences dialog. Just go to the Module Packages tab, select spreadsheet in the "Enabled packages" and press the Configure button. Then a list of all the configuration options for the spreadsheet will show up. [http://www.vistrails.org/usersguide/spreadsheet.html#custom-layout-options Custom Layout Options]

=== Is it possible to launch a web browser from the vistrails spreadsheet? We would like to output several urls from a parameter sweep and then have the option to click on each one to view the resulting page. I can view the page within the spreadsheet, but it is really too crowded.===

Currently, there isn't a widget that provides exactly this functionality, but I can think of a few solutions that may work for you:

(1) You can use parameter exploration to generate multiple sheets so you might have an exploration that opens each page in a new sheet. Use the third column/dimension in the exploration interface to have a parameter span sheets.

(2) The spreadsheet is extensible so you can write a custom spreadsheet cell widget that has a button or label with the desired link (a QLabel with openExternalLinks set to True, for example).

(3) You can tweak the existing RichTextCell be adding the line "self.browser.setOpenExternalLinks(True)" at line 63 of the source file "vistrails/packages/spreadsheet/widgets/richtext/richtext.py". Then, if your workflow creates a file with html markup text like "<a href="http://www.vistrails.org">VisTrails</a>" connected to a RichTextCell, clicking on the rendered link in the cell will open it in a web browser. You need to add the aforementioned line to the source to let Qt know that you want the link opened externally; by default, it will just issue an event that isn't processed. [http://www.vistrails.org/usersguide/spreadsheet.html#launching-a-web-browser Launching a Web Browser]

== Integrating your software into VisTrails ==

===How can I integrate my own program into VisTrails?===

The easiest way is to create a package. Writing a package is often very simple, here are instructions on how to do it: [[UsersGuideVisTrailsPackages]]

You can also dynamically generate modules. For an example see:

http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#Packages_that_generate_modules_dynamically

In particular, see the new_module call which uses python's type() function to generate new classes dynamically.

=== How do modules deal with multiple inputs in a same port? ===

(And should that even be allowed?)

For compatibility reasons, we do need to allow multiple connections to an input port. However, most package developers should never have to use this, and so we do our best to hide it. the default behavior for getting inputs from a port, then, is to always return a single input.

If on your module you need multiple inputs connected to a single port, use the 'forceGetInputListFromPort' method. It will return a list of all the data items coming through the port. The spreadsheet package uses this feature, so look there for usage examples (vistrails/packages/spreadsheet/basic_widgets.py)

=== Are there mechanisms for attaching widgets to different modules/parameters? ===

Right now, we have a mechanism for putting a specific widget for an input port. For example, if a port is SetColor(red, green, blue), we can put a color wheel widget there. Or we can also replace the SetFileName port with a File Widget. However, this is not per parameter (only per port). We are currently working on this problem.

=== Can I organize my package so it appears hierarchical in the module palette? ===

Yes. Use the namespace keyword argument when adding the module to the registry. For example,

registry.add_module(MyModule, namespace='MyNamespace')

=== Can I nest namespaces? ===

Yes. Use the '|' character to separate different the hierarchy. For example,

registry.add_module(MyModule, namespace='ParentNamespace|ChildNamespace')

=== Are there shortcuts for registry initialization? ===

Yes. If you define _modules as a list of classes in the __init__.py file of your package, VisTrails will attempt to load all classes specified as modules. You can provide add_module options as keyword arguments by specifying a tuple (class, kwargs) in the list. For example:

_modules = [MyModule1, (MyModule2, {'namespace': 'MyNamespace'})]

In addition, you need to identify the ports of your modules as a field in your class by defining _input_ports and _output_ports lists. Here, the items in each list must be tuples of the form (portName, portSignature, optional=False, sort_key=-1). For example:

class MyModule(Module):
def compute(self):
pass

_input_ports = [('firstInput', String), ('secondInput', Integer, True)]
_output_ports = [('firstOutput', String), ('secondOutput', String)]

=== Can I define ports to be of types that I do not import into my package? ===

Yes. You can pass an identifier string as the portSignature instead. The port_signature string is defined by:

<module_string> := <package_identifier>:[<namespace>|]<module_name>,
<port_signature> := (<module_string>*)

For example,

registry.add_input_port(MyModule, 'myInputPort', '(edu.utah.sci.vistrails.basic:String)')

or

_input_ports = [('myInputPort', '(edu.utah.sci.vistrails.basic:String)')]

=== What do I need to change in my package to make it reloadable (new in v1.4.2)? ===
See [http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#How_to_make_you_package_reloadable UsersGuideVisTrailsPackages] for an explanation.

=== Can I add default values or labels for parameters? ===

Yes. Versions 1.4 and greater support these features. See [[UsersGuideVisTrailsPackages#Adding_default_values_and.2For_labels_for_parameters | UsersGuideVisTrailsPacakges]] for more details.

=== I want to write a module to load HDF data whose output (e.g., data, string) varies according to the input I give it. Is is possible to do this in VisTrails, and if yes, how can I do that? Ideally, I would like to avoid having to change the connection of my output every time I change the input. ===

There are a few ways to tackle this - each has it's own benefits and pitfalls. Firstly, module connections do respect class hierarchies as we're familiar with in object oriented languages. For instance, A module can output a Constant of which String, Float, Integer, etc are specifications. In this way, you can have a subclass of something like HDFData be passed out of the module and the connections will be established regardless of the sub-type. This is a bit dangerous though. Modules downstream of such a class may not really know how to operate on certain types derived from the super-class. Extreme care must be taken both when creating the modules as well as connecting them to prevent things like this from happening.

A second method that I employ in several different packages is the idea of a container class. For instance, the NumSciPy package uses a relatively generic container "Numpy Array" to encapsulate the data. Of course, these encapsulating objects can store dictionaries that other modules can easily access and understand how to operate on. Although this method is slightly more work, the benefits of a stricter typing of ports is beneficial - particularly upon interfacing with other packages that may depend on strongly typed constants (for example).

=== I need to determine, at run-time, whether or not a "child" module is attached to the output port of a "parent" module. (I do not specifically need to know which child; just if there is one). ===

The outputPorts dictionary of the base Module stores this information. Thus, you should be able to check

("myPortName" in self.outputPorts)

on the parent module to check if there are any downstream connections from the port "myPortName". This might be used, for example, to only set results for output ports that will be used. ***Note***, however, that the caching algorithm assumes that all outputs are set so adding a new connection to a previously unconnected output port will not work as desired if that module is cached. For this reason, I would currently recommend making such a module not cacheable. Another possibility is overriding the update() method to check the outputPorts and set the upToDate flag if they are not equal. In a single, limited test, this seemed to work, but be warned that it is not fully tested. Here is an example:

class TestModule(Module):
_output_ports = [('a1', '(edu.utah.sci.vistrails.basic:String)'),
('a2', '(edu.utah.sci.vistrails.basic:String)')]
def __init__(self):
Module.__init__(self)
self._cached_output_ports = set()

def update(self):
if len(set(self.outputPorts) - self._cached_output_ports) > 0:
self.upToDate = False
Module.update(self)

def compute(self):
if "a1" in self.outputPorts:
self.setResult("a1", "test")
if "a2" in self.outputPorts:
self.setResult("a2", "test2")
self._cached_output_ports = set(self.outputPorts)

=== How can I make a module not display in the modules list? ===

You should set the abstract parameter to True when adding the module to the registry. Using the original syntax, this looks like:

def initialize():
reg = core.modules.module_registry.get_module_registry()
reg.add_module(InvisibleModule, abstract=True)
# ...

With the _modules dictionary shortcut (for more details, see [http://www.vistrails.org/index.php/FAQ#Are_there_shortucts_for_registry_initialization.3F the FAQ section on this]), you include it in a kwargs dict as part of a module tuple:

_modules = [AnotherModule, (InvisibleModule, {'abstract': True})]

There is also a 'hide_descriptor' parameter that prevents the module from appearing in the module palette without declaring it to be abstract.

The technical difference between the two is that 'abstract' will not add the item to the module palette while 'hide_descriptor' does add the item but immediately hides it. If the module should never be instantiated in a workflow, declare it abstract. If you don't want users to be able to add the module to a pipeline, but you have code that may add it programmatically, declare it with hide_descriptor=True.

== The Console ==

=== Where should I go to find out what I can call from the console and how to import it? ===

We have tried to make some methods more accessible in the console via an api. You can import the api via <code>import api</code> in the console and see the available methods with <code>dir(api)</code>. To open a vistrail:

import api
api.open_vistrail_from_file('/Applications/VisTrails/examples/terminator.vt')

To execute a version of a workflow, you currently have to go through the controller:

api.select_version('Histogram')
api.get_current_controller().execute_current_workflow()

Currently, only a subset of VisTrails functionality is directly available from the api. However, since VisTrails is written in python, you can dig down starting with the VistrailsApplication or controller object to expose most of our internal methods. If you have suggestions for calls to be added to the api, please let us know.

One other feature that we're working on, but is still in progress is the ability to construct workflows via the console. For example:

vtk = load_package('edu.utah.sci.vistrails.vtk')
vtk.vtkDataSetReader() # adds a vtkDataSetReader module to the pipeline
# click on the new module
a = selected_modules()[0] # get the one currently selected module
a.SetFile('/vistrails/examples/data/head120.vtk') # sets the SetFile parmaeter for the data set reader
b = vtk.vtkContourFilter() # adds a vtkContourFilter module to the pipeline and saves to var b
b.SetInputConnection0(a.GetOutputPort0()) # connects a's GetOutputPort0 port to b's SetInputConnection0

== Persistence Package ==

=== How do I use the output of one workflow as the input for another using the persistence package? ===

You need to configure the persistence modules using the module's configuration dialog. After adding a <code>PersistentOutputFile</code> to the workflow, click on the triangle in the upper-right corner of the <code>PersistentOutputFile</code>, and select "Edit Configuration" from the menu that appears. In this dialog, select "Create New Reference" and give the reference a name (and any space-delimited tags). Upon running that workflow, the data will be written to the persistent store. In the second workflow where you wish to use that file, add a <code>PersistentInputFile</code> and go to its configuration dialog in the same manner as with the output file. In that dialog, select "Use Existing Reference" and select the data that you just added in the first workflow in the list of files below. Now, when you run that workflow, it will grab the data from the persistent store.

Here is an example: [[Media:offscreen_persistent.vt | offscreen_persistent.vt]]. Run the "persistent offscreen" workflow first, and then run the "display persistent output" to use the output of the first workflow as the input for the second.

== VTK ==

=== Given a VTK visualization, how can I generate a webpage from it? ===

Check out the html pipeline in offscreen.xml.

=== I'm trying to use VTK, but there doesn't seem to be any output. What is wrong? ===

To use VTK on VisTrails, you need a slightly different way of connecting the renderer modules. Instead of using the standard RenderWindow/RenderWindowInteractor infrastructure, you simply connect the renderer to a VTKCell. The examples directory in the distribution has several VTK examples that illustrate.

=== I am trying to add a module to the workflow via Python, but how can I access vtk modules? ===

Here's an example:

import api

vtvtk = 'edu.utah.sci.vistrails.vtk'

module = api.add_module(0, 0, vtvtk, 'vtkContourFilter', '')

The third argument in add_module is the package identifier. You can find this in the "Module Packages" panel of the Preferences; just click on the package you're interested in and it will appear in the information on the right.

== matplotlib ==

=== I'm experiencing a problem with Latex labels and the matplotlib that comes with VisTrails 1.5. The script below entered to the interpreter that comes with VT is sufficient to reproduce it. ===

import matplotlib.pyplot as plt
plt.plot([1,2,3],[1,2,3])
plt.xlabel("$foo$")

Remove your ~/.matplotlib folder and re-start VisTrails

== VisTrails Development ==

=== I would like to build VisTrails from source. Are there instructions on how to do this? ===

Yes! Take a look at http://www.vistrails.org/index.php/Mac_Intel_Instructions
[http://www.vistrails.org/usersguide/getting_started.html#installing-vistrails-from-source Installing VisTrails from source]

== Accessing Provenance Information ==

=== How do I access the information in the execution log? ===

The code responsible for storing execution information is located in the "core/log" directories, and the code that generates much of that information is in "core/interpreter/cached.py". Modules can add execution-specific annotations to provenance via annotate() calls during execution, but much of the data (like timing and errors) is captured by the LogController and CachedInterpreter (the execution engine) objects. To analyze the log from a vistrail (.vt) file, you might have something like the following:

<code>
import core.log.log
import db.services.io

def run(fname):
# open the .vt bundle specified by the filename "fname"
bundle = db.services.io.open_vistrail_bundle_from_zip_xml(fname)[0]
# get the log filename
log_fname = bundle.vistrail.db_log_filename
if log_fname is not None:
# open the log
log = db.services.io.open_log_from_xml(log_fname, True)
# convert the log from a db object
core.log.log.Log.convert(log)
for workflow_exec in log.workflow_execs:
print 'workflow version:', workflow_exec.parent_version
print 'time started:', workflow_exec.ts_start
print 'time ended:', workflow_exec.ts_end
print 'modules executed:', [i.module_id
for i in workflow_exec.item_execs]
if __name__ == '__main__':
run("some_vistrail.vt")
</code>

You should be able to see what information is available by looking at the "core/log" classes. [http://www.vistrails.org/usersguide/log.html Accessing the Execution Log]

UsersGuideVisTrailsPackages

2011-03-31T15:24:14Z

Ctuttle: /* Adding default values and/or labels for parameters */

== Introduction ==

VisTrails provides a plugin infrastructure to integrate user-defined
functions and libraries. Specifically, users can incorporate their own visualization and simulation codes into pipelines by defining custom modules (or wrappers). These modules are bundled in what we call '''packages'''. A VisTrails package is simply a collection of Python classes -- each of these classes will represent a new module -- created by the user that respects a certain convention. Here's a simplified example of a very simple user-defined module:

class Divide(Module):
def compute(self):
arg1 = self.getInputFromPort("arg1")
arg2 = self.getInputFromPort("arg2")
if arg2 == 0.0:
raise ModuleError(self, "Division by zero")
self.setResult("result", arg1 / arg2)

registry.addModule(Divide)
registry.addInputPort(Divide, "arg1", (basic.Float, 'dividend'))
registry.addInputPort(Divide, "arg2", (basic.Float, 'divisor'))
registry.addOutputPort(Divide, "result", (basic.Float, 'quotient'))

New VisTrails modules must subclass from Module, the base class that defines basic functionality. The only required override is the <tt>compute()</tt> method, which performs the actual module computation. Input and output is specified through ''ports'', which currently have to be explicitly registered with VisTrails. However, this is straightforward, and done through method calls to the module registry. A complete documented example of a (slightly) more complicated module is available [[VistrailsPackagePythonCalcExample | here]].

== Dealing with command line tools and side effects ==

In an ideal world a module's outputs should be completely determined by its inputs. This is important for provenance purposes - if modules have implicit dependencies, it is not possible to be certain that the same results will be generated when the process is reexecuted.

However, it is clear that certain modules are inherently side-effectful (reading/writing files, network, etc). For the common case of temporary files, VisTrails provides a convenience layer that removes part of the burden of managing temporary files. As an illustrative example, consider one of the packages we make available for image conversion, using the [http://www.imagemagick.org/ ImageMagick] suite:

class Convert(ImageMagick):
"""Convert is the base Module for VisTrails Modules in the ImageMagick
package that deal with operations on images. Convert is a bit of a misnomer since
the 'convert' tool does more than simply file format conversion. Each subclass
has a descriptive name of the operation it implements."""

def create_output_file(self):
"""Creates a File with the output format given by the
outputFormat port."""
if self.hasInputFromPort('outputFormat'):
s = '.' + self.getInputFromPort('outputFormat')
return self.interpreter.filePool.create_file(suffix=s)

def geometry_description(self):
"""returns a string with the description of the geometry as
indicated by the appropriate ports (geometry or width and height)"""
# if complete geometry is available, ignore rest
if self.hasInputFromPort("geometry"):
return self.getInputFromPort("geometry")
elif self.hasInputFromPort("width"):
w = self.getInputFromPort("width")
h = self.getInputFromPort("height")
return "'%sx%s'" % (w, h)
else:
raise ModuleError(self, "Needs geometry or width/height")

def run(self, *args):
"""run(*args), runs ImageMagick's 'convert' on a shell, passing all
arguments to the program."""
cmdline = ("convert" + (" %s" * len(args))) % args
if not self.__quiet:
print cmdline
r = os.system(cmdline)
if r != 0:
raise ModuleError(self, "system call failed: '%s'" % cmdline)

def compute(self):
o = self.create_output_file()
i = self.input_file_description()
self.run(i, o.name)
self.setResult("output", o)

(...)

reg.addModule(Convert)
reg.addInputPort(Convert, "geometry", (basic.String, 'ImageMagick geometry'))
reg.addInputPort(Convert, "width", (basic.String, 'width of the geometry for operation'))
reg.addInputPort(Convert, "height", (basic.String, 'height of the geometry for operation'))
reg.addOutputPort(Convert, "output", (basic.File, 'the output file'))

This example introduces several new VisTrails features. The last line of the snippet registers an output port that provides a file. Immediately, a file output presents several problems when a pipeline is to be shared among users in heterogenous environments. For example, where should a file be written to? For temporary files, VisTrails provides a ''file pool'' class, that manages temporary files and their lifetimes automatically, so that users don't have to worry about deleting them post-execution. To create a temporary file, a user calls, for example

fileObj = self.interpreter.filePool.create(suffix=".png")

<tt>fileObj</tt> will then contain a module that represents a file. The file pool simply creates a temporary file with write permissions, whose local name is available, in this case, as <tt>fileObj.name</tt>. The package developer is then free to use this file for any purpose.

Another feature of this example is the use of command line tools. Notice that Python provides a very convenient way to execute commands through a shell. In this case, we use <tt>os.system</tt> on a command-line that executes the appropriate program.

== Interaction with Caching ==

VisTrails provides a caching mechanism, in which portions of pipelines that are common across different executions are automatically shared. However, some modules are intrinsically side-effectful (writing a report to stdout, or a file to disk, or creating a user interface widget), and should not be shared. Caching control is therefore up to the package developer. By default, '''''caching is enabled'''''. So a developer that doesn't want caching to apply must make small changes to the module. There's a convenient way to disable caching entirely, by using multiple inheritance, and deriving from a ''mixin'' class that's provided by VisTrails. For example, look at the StandardOutput module:

from core.modules.vistrails_module import Module, newModule, \
NotCacheable, ModuleError
(...)
class StandardOutput(NotCacheable, Module):
"""StandardOutput is a VisTrails Module that simply prints the
value connected on its port to standard output. It is intended
mostly as a debugging device."""

def compute(self):
v = self.getInputFromPort("value")
print v

By subclassing from <tt>NotCacheable</tt> as well as from Module (or one of its subclasses), VisTrails automatically will not cache this module, or anything downstream from it.

VisTrails also allows a more sophisticated decision on whether to use caching or not. To do that, a user simply overrides the method <tt>is_cacheable</tt> to return the correct value. This allows context-dependent decisions. For example, in the [http://teem.sourceforge.net teem] package, there's a module that generates a scalar field with random numbers. This is non-deterministic, so shouldn't be cached. However, this module only generates non-deterministic values in special occasions, depending on its input port values. To keep efficiency when caching is possible, while still maintaining correctness, that module implements the following override:

class Unu1op(Unu):
(...)
def is_cacheable(self):
return not self.getInputFromPort('op') in ['rand', 'nrand']
(...)

Notice that the module explicitly uses inputs to decide whether it should be cached. This allows reasonably fine-grained control over the process.

== Interaction with Other Packages ==

When developing more complicated packages, it becomes natural to split code among different VisTrails packages, and have one logically depend on the other. For example, in one package (say, named ' ''package_base'' '), a user might define

class PackageBaseModule(Module):
...

def initialize():
registry.addModule(PackageBaseModule)
...

And then, in another package (say, ' ''package_derived'' '),

class DerivedModule(PackageBaseModule):
...

def initialize():
registry.addModule(DerivedModule)
...

Because of the way packages are loaded, ''package_derived'' cannot be initialized before ''package_base''. VisTrails provides a mechanism for specifying interpackage dependencies. Every VisTrails package can provide a list of necessary installed packages. This is done by providing a callable in the package under the name '''package_dependencies'''. For example, here's how the VTK VisTrails package declares dependencies:

def package_dependencies():
import core.packagemanager
manager = core.packagemanager.get_package_manager()
if manager.has_package('spreadsheet'):
return ['spreadsheet']
else:
return []

The callable must return a list of strings, representing the name of the packages it depends on. We also use this example to introduce the package manager API, that is useful here for inspecting packages present in the system. Notice that the dependencies are not static. '''vtk''' depends on '''spreadsheet''' if and only if '''spreadsheet''' is present in the system. Otherwise, it has no dependencies.

''Note'': Circular dependencies are not allowed. They will be detected by VisTrails and an error will be signalled.

''Note'': Currently, package names are reasonably brittle, in the sense that conflicts in package naming might become an issue. We are in the process of designing an API that will allow more robust naming schemes.

== User-defined module shapes and colors ==

VisTrails allows users to define custom colors and shapes to modules. This must be done at module registration time, by passing special parameters to addModule. For example:

reg.addModule(Afront,
moduleColor=(1.0,0.0,0.0),
moduleFringe=[(0.0, 0.0),
(0.2, 0.0),
(0.2, 0.4),
(0.0, 0.4),
(0.0, 1.0)])

gives this result:

[[Image:PackageCustomColorShape1.png]]

This piece of code

reg = core.modules.module_registry
reg.addModule(Afront,
moduleColor=(0.4,0.6,0.8),
moduleFringe=[(0.0, 0.0),
(0.2, 0.0),
(0.0, 0.2),
(0.2, 0.4),
(0.0, 0.6),
(0.2, 0.8),
(0.0, 1.0)])

gives this result:

[[Image:PackageCustomColorShape2.png]]

The moduleColor parameter must be a tuple of three floats between 0 and 1 that specify RGB colors for the module background, while moduleFringe is a list of pairs of floats that specify points as they go around a side of the module (the same one is used to go from the top-right corner to bottom-right corner, and from the bottom-left corner to the top-left one. If this is not enough, let the developers know!)

Alternatively, you can use different fringes for the left and right borders:

reg.addModule(Afront,
moduleColor=(1.0,0.8,0.6),
moduleLeftFringe=[(0.0, 0.0),
(-0.2, 0.0),
(0.0, 1.0)],
moduleRightFringe=[(0.0, 0.0),
(0.2, 1.0),
(0.0, 1.0)])

which gives this:

[[Image:PackageCustomColorShape3.png]]

== How to make your package reloadable ==

You need to move almost everything in the __init__.py file to a new file "init.py", but keep the identifier, name, version, configuration, and package_dependencies fields/methods in the __init__.py file. Specifically, make sure that imports (excluding things like core.configuration) and the initialize method are in the init.py file. For example, take a look at the __init__.py file of the pylab package included in VisTrails:

identifier = 'edu.utah.sci.vistrails.matplotlib'
name = 'matplotlib'
version = '0.9.0'

def package_dependencies():
import core.packagemanager
manager = core.packagemanager.get_package_manager()
if manager.has_package('edu.utah.sci.vistrails.spreadsheet'):
return ['edu.utah.sci.vistrails.spreadsheet']
else:
return []

def package_requirements():
import core.requirements
if not core.requirements.python_module_exists('matplotlib'):
raise core.requirements.MissingRequirement('matplotlib')
if not core.requirements.python_module_exists('pylab'):
raise core.requirements.MissingRequirement('pylab')

And the init.py contains the other imports, class definitions and the initialize method.

== Adding default values and/or labels for parameters ==

In versions 1.4 and greater, package developers can add labels and default values for parameters. To add this functionality, you need to use the <code>defaults</code> and <code>labels</code> keyword arguments and pass the values as '''strings'''. For example,

class TestDefaults(Module):
_input_ports = [("f1", "(edu.utah.sci.vistrails.basic:Float)",
{"defaults": str([1.23]), "labels": str(["temp"])})]
_modules = [TestDefaults]

or in the older syntax,

def initialize():
reg = core.modules.module_registry.get_module_registry()
reg.add_module(TestDefaults2)
reg.add_input_port(TestDefaults2, "f2", [Float, String],
defaults=str([4.56, "abc"]),
labels=str(["temp", "name"]))

== Packages that generate modules dynamically ==

When wrapping existing libraries or trying to generate modules in a more procedural manner, it is useful to dynamically generate modules. In our work, we have created some shortcuts to make this easier. In addition, the list of modules can also be based based on the package configuration. Here is some example code:

=== __init__.py ===

from core.configuration import ConfigurationObject

identifier = "edu.utah.sci.dakoop.auto_example"
version = "0.0.1"
name = "AutoExample"

configuration = ConfigurationObject(use_b=True)

=== init.py ===

The <code>expand_ports</code> and <code>build_modules</code> methods are functions to help the construction of the modules easier. The key parts are the <code>new_module</code> call and setting the <code>_modules</code> variable.

from core.modules.vistrails_module import new_module, Module

identifier = "edu.utah.sci.dakoop.auto_example"

def expand_ports(port_list):
new_port_list = []
for port in port_list:
port_spec = port[1]
if type(port_spec) == str: # or unicode...
if port_spec.startswith('('):
port_spec = port_spec[1:]
if port_spec.endswith(')'):
port_spec = port_spec[:-1]
new_spec_list = []
for spec in port_spec.split(','):
spec = spec.strip()
parts = spec.split(':', 1)
print 'parts:', parts
namespace = None
if len(parts) > 1:
mod_parts = parts[1].rsplit('|', 1)
if len(mod_parts) > 1:
namespace, module_name = mod_parts
else:
module_name = parts[1]
if len(parts[0].split('.')) == 1:
id_str = 'edu.utah.sci.vistrails.' + parts[0]
else:
id_str = parts[0]
else:
mod_parts = spec.rsplit('|', 1)
if len(mod_parts) > 1:
namespace, module_name = mod_parts
else:
module_name = spec
id_str = identifier
if namespace:
new_spec_list.append(id_str + ':' + module_name + ':' + \
namespace)
else:
new_spec_list.append(id_str + ':' + module_name)
port_spec = '(' + ','.join(new_spec_list) + ')'
new_port_list.append((port[0], port_spec) + port[2:])
print new_port_list
return new_port_list

def build_modules(module_descs):
new_classes = {}
for m_name, m_dict in module_descs:
m_doc = m_dict.get("_doc", None)
m_inputs = m_dict.get("_inputs", [])
m_outputs = m_dict.get("_outputs", [])
if "_inputs" in m_dict:
del m_dict["_inputs"]
if "_outputs" in m_dict:
del m_dict["_outputs"]
if "_doc" in m_dict:
del m_dict["_doc"]
klass_dict = {}
if "_compute" in m_dict:
klass_dict["compute"] = m_dict["_compute"]
del m_dict["_compute"]
m_class = new_module(Module, m_name, klass_dict, m_doc)
m_class._input_ports = expand_ports(m_inputs)
m_class._output_ports = expand_ports(m_outputs)
new_classes[m_name] = (m_class, m_dict)
return new_classes.values()

def initialize():
global _modules
def a_compute(self):
a = self.getInputFromPort("a")
i = 0
if self.hasInputFromPort("i"):
i = self.getInputFromPort("i")
if a == "abc":
i += 100
self.setResult("b", i)

module_descs = [("ModuleA", {"_inputs": [("a", "basic:String")],
"_outputs": [("b", "basic:Integer")],
"_doc": "ModuleA documentation",
"_compute": a_compute,
"namespace": "Test"}),
("ModuleB", {"_inputs": [("a", "Test|ModuleA")],
"_outputs": [("b", "Test|ModuleA")],
"_doc": "ModuleB documentation"})
]

if configuration.use_b:
_modules = build_modules(module_descs)
else:
_modules = build_modules(module_descs[:1])

_modules = []

== Help! This documentation wasn't good enough! ==

Sorry, it's our fault! If you need help, join the vistrails-users list and post your question there.

UsersGuideVisTrailsPackages

2011-03-31T15:23:14Z

Ctuttle: /* How to make you package reloadable */

== Introduction ==

VisTrails provides a plugin infrastructure to integrate user-defined
functions and libraries. Specifically, users can incorporate their own visualization and simulation codes into pipelines by defining custom modules (or wrappers). These modules are bundled in what we call '''packages'''. A VisTrails package is simply a collection of Python classes -- each of these classes will represent a new module -- created by the user that respects a certain convention. Here's a simplified example of a very simple user-defined module:

class Divide(Module):
def compute(self):
arg1 = self.getInputFromPort("arg1")
arg2 = self.getInputFromPort("arg2")
if arg2 == 0.0:
raise ModuleError(self, "Division by zero")
self.setResult("result", arg1 / arg2)

registry.addModule(Divide)
registry.addInputPort(Divide, "arg1", (basic.Float, 'dividend'))
registry.addInputPort(Divide, "arg2", (basic.Float, 'divisor'))
registry.addOutputPort(Divide, "result", (basic.Float, 'quotient'))

New VisTrails modules must subclass from Module, the base class that defines basic functionality. The only required override is the <tt>compute()</tt> method, which performs the actual module computation. Input and output is specified through ''ports'', which currently have to be explicitly registered with VisTrails. However, this is straightforward, and done through method calls to the module registry. A complete documented example of a (slightly) more complicated module is available [[VistrailsPackagePythonCalcExample | here]].

== Dealing with command line tools and side effects ==

In an ideal world a module's outputs should be completely determined by its inputs. This is important for provenance purposes - if modules have implicit dependencies, it is not possible to be certain that the same results will be generated when the process is reexecuted.

However, it is clear that certain modules are inherently side-effectful (reading/writing files, network, etc). For the common case of temporary files, VisTrails provides a convenience layer that removes part of the burden of managing temporary files. As an illustrative example, consider one of the packages we make available for image conversion, using the [http://www.imagemagick.org/ ImageMagick] suite:

class Convert(ImageMagick):
"""Convert is the base Module for VisTrails Modules in the ImageMagick
package that deal with operations on images. Convert is a bit of a misnomer since
the 'convert' tool does more than simply file format conversion. Each subclass
has a descriptive name of the operation it implements."""

def create_output_file(self):
"""Creates a File with the output format given by the
outputFormat port."""
if self.hasInputFromPort('outputFormat'):
s = '.' + self.getInputFromPort('outputFormat')
return self.interpreter.filePool.create_file(suffix=s)

def geometry_description(self):
"""returns a string with the description of the geometry as
indicated by the appropriate ports (geometry or width and height)"""
# if complete geometry is available, ignore rest
if self.hasInputFromPort("geometry"):
return self.getInputFromPort("geometry")
elif self.hasInputFromPort("width"):
w = self.getInputFromPort("width")
h = self.getInputFromPort("height")
return "'%sx%s'" % (w, h)
else:
raise ModuleError(self, "Needs geometry or width/height")

def run(self, *args):
"""run(*args), runs ImageMagick's 'convert' on a shell, passing all
arguments to the program."""
cmdline = ("convert" + (" %s" * len(args))) % args
if not self.__quiet:
print cmdline
r = os.system(cmdline)
if r != 0:
raise ModuleError(self, "system call failed: '%s'" % cmdline)

def compute(self):
o = self.create_output_file()
i = self.input_file_description()
self.run(i, o.name)
self.setResult("output", o)

(...)

reg.addModule(Convert)
reg.addInputPort(Convert, "geometry", (basic.String, 'ImageMagick geometry'))
reg.addInputPort(Convert, "width", (basic.String, 'width of the geometry for operation'))
reg.addInputPort(Convert, "height", (basic.String, 'height of the geometry for operation'))
reg.addOutputPort(Convert, "output", (basic.File, 'the output file'))

This example introduces several new VisTrails features. The last line of the snippet registers an output port that provides a file. Immediately, a file output presents several problems when a pipeline is to be shared among users in heterogenous environments. For example, where should a file be written to? For temporary files, VisTrails provides a ''file pool'' class, that manages temporary files and their lifetimes automatically, so that users don't have to worry about deleting them post-execution. To create a temporary file, a user calls, for example

fileObj = self.interpreter.filePool.create(suffix=".png")

<tt>fileObj</tt> will then contain a module that represents a file. The file pool simply creates a temporary file with write permissions, whose local name is available, in this case, as <tt>fileObj.name</tt>. The package developer is then free to use this file for any purpose.

Another feature of this example is the use of command line tools. Notice that Python provides a very convenient way to execute commands through a shell. In this case, we use <tt>os.system</tt> on a command-line that executes the appropriate program.

== Interaction with Caching ==

VisTrails provides a caching mechanism, in which portions of pipelines that are common across different executions are automatically shared. However, some modules are intrinsically side-effectful (writing a report to stdout, or a file to disk, or creating a user interface widget), and should not be shared. Caching control is therefore up to the package developer. By default, '''''caching is enabled'''''. So a developer that doesn't want caching to apply must make small changes to the module. There's a convenient way to disable caching entirely, by using multiple inheritance, and deriving from a ''mixin'' class that's provided by VisTrails. For example, look at the StandardOutput module:

from core.modules.vistrails_module import Module, newModule, \
NotCacheable, ModuleError
(...)
class StandardOutput(NotCacheable, Module):
"""StandardOutput is a VisTrails Module that simply prints the
value connected on its port to standard output. It is intended
mostly as a debugging device."""

def compute(self):
v = self.getInputFromPort("value")
print v

By subclassing from <tt>NotCacheable</tt> as well as from Module (or one of its subclasses), VisTrails automatically will not cache this module, or anything downstream from it.

VisTrails also allows a more sophisticated decision on whether to use caching or not. To do that, a user simply overrides the method <tt>is_cacheable</tt> to return the correct value. This allows context-dependent decisions. For example, in the [http://teem.sourceforge.net teem] package, there's a module that generates a scalar field with random numbers. This is non-deterministic, so shouldn't be cached. However, this module only generates non-deterministic values in special occasions, depending on its input port values. To keep efficiency when caching is possible, while still maintaining correctness, that module implements the following override:

class Unu1op(Unu):
(...)
def is_cacheable(self):
return not self.getInputFromPort('op') in ['rand', 'nrand']
(...)

Notice that the module explicitly uses inputs to decide whether it should be cached. This allows reasonably fine-grained control over the process.

== Interaction with Other Packages ==

When developing more complicated packages, it becomes natural to split code among different VisTrails packages, and have one logically depend on the other. For example, in one package (say, named ' ''package_base'' '), a user might define

class PackageBaseModule(Module):
...

def initialize():
registry.addModule(PackageBaseModule)
...

And then, in another package (say, ' ''package_derived'' '),

class DerivedModule(PackageBaseModule):
...

def initialize():
registry.addModule(DerivedModule)
...

Because of the way packages are loaded, ''package_derived'' cannot be initialized before ''package_base''. VisTrails provides a mechanism for specifying interpackage dependencies. Every VisTrails package can provide a list of necessary installed packages. This is done by providing a callable in the package under the name '''package_dependencies'''. For example, here's how the VTK VisTrails package declares dependencies:

def package_dependencies():
import core.packagemanager
manager = core.packagemanager.get_package_manager()
if manager.has_package('spreadsheet'):
return ['spreadsheet']
else:
return []

The callable must return a list of strings, representing the name of the packages it depends on. We also use this example to introduce the package manager API, that is useful here for inspecting packages present in the system. Notice that the dependencies are not static. '''vtk''' depends on '''spreadsheet''' if and only if '''spreadsheet''' is present in the system. Otherwise, it has no dependencies.

''Note'': Circular dependencies are not allowed. They will be detected by VisTrails and an error will be signalled.

''Note'': Currently, package names are reasonably brittle, in the sense that conflicts in package naming might become an issue. We are in the process of designing an API that will allow more robust naming schemes.

== User-defined module shapes and colors ==

VisTrails allows users to define custom colors and shapes to modules. This must be done at module registration time, by passing special parameters to addModule. For example:

reg.addModule(Afront,
moduleColor=(1.0,0.0,0.0),
moduleFringe=[(0.0, 0.0),
(0.2, 0.0),
(0.2, 0.4),
(0.0, 0.4),
(0.0, 1.0)])

gives this result:

[[Image:PackageCustomColorShape1.png]]

This piece of code

reg = core.modules.module_registry
reg.addModule(Afront,
moduleColor=(0.4,0.6,0.8),
moduleFringe=[(0.0, 0.0),
(0.2, 0.0),
(0.0, 0.2),
(0.2, 0.4),
(0.0, 0.6),
(0.2, 0.8),
(0.0, 1.0)])

gives this result:

[[Image:PackageCustomColorShape2.png]]

The moduleColor parameter must be a tuple of three floats between 0 and 1 that specify RGB colors for the module background, while moduleFringe is a list of pairs of floats that specify points as they go around a side of the module (the same one is used to go from the top-right corner to bottom-right corner, and from the bottom-left corner to the top-left one. If this is not enough, let the developers know!)

Alternatively, you can use different fringes for the left and right borders:

reg.addModule(Afront,
moduleColor=(1.0,0.8,0.6),
moduleLeftFringe=[(0.0, 0.0),
(-0.2, 0.0),
(0.0, 1.0)],
moduleRightFringe=[(0.0, 0.0),
(0.2, 1.0),
(0.0, 1.0)])

which gives this:

[[Image:PackageCustomColorShape3.png]]

== How to make your package reloadable ==

You need to move almost everything in the __init__.py file to a new file "init.py", but keep the identifier, name, version, configuration, and package_dependencies fields/methods in the __init__.py file. Specifically, make sure that imports (excluding things like core.configuration) and the initialize method are in the init.py file. For example, take a look at the __init__.py file of the pylab package included in VisTrails:

identifier = 'edu.utah.sci.vistrails.matplotlib'
name = 'matplotlib'
version = '0.9.0'

def package_dependencies():
import core.packagemanager
manager = core.packagemanager.get_package_manager()
if manager.has_package('edu.utah.sci.vistrails.spreadsheet'):
return ['edu.utah.sci.vistrails.spreadsheet']
else:
return []

def package_requirements():
import core.requirements
if not core.requirements.python_module_exists('matplotlib'):
raise core.requirements.MissingRequirement('matplotlib')
if not core.requirements.python_module_exists('pylab'):
raise core.requirements.MissingRequirement('pylab')

And the init.py contains the other imports, class definitions and the initialize method.

== Adding default values and/or labels for parameters ==

In versions 1.4 and greater, package developers can add labels and default values for parameters. To add this functionality, you need to use the <code>defaults</code> and <code>labels</code> keyward arguments and pass the values as '''strings'''. For example,

class TestDefaults(Module):
_input_ports = [("f1", "(edu.utah.sci.vistrails.basic:Float)",
{"defaults": str([1.23]), "labels": str(["temp"])})]
_modules = [TestDefaults]

or in the older syntax,

def initialize():
reg = core.modules.module_registry.get_module_registry()
reg.add_module(TestDefaults2)
reg.add_input_port(TestDefaults2, "f2", [Float, String],
defaults=str([4.56, "abc"]),
labels=str(["temp", "name"]))

== Packages that generate modules dynamically ==

When wrapping existing libraries or trying to generate modules in a more procedural manner, it is useful to dynamically generate modules. In our work, we have created some shortcuts to make this easier. In addition, the list of modules can also be based based on the package configuration. Here is some example code:

=== __init__.py ===

from core.configuration import ConfigurationObject

identifier = "edu.utah.sci.dakoop.auto_example"
version = "0.0.1"
name = "AutoExample"

configuration = ConfigurationObject(use_b=True)

=== init.py ===

The <code>expand_ports</code> and <code>build_modules</code> methods are functions to help the construction of the modules easier. The key parts are the <code>new_module</code> call and setting the <code>_modules</code> variable.

from core.modules.vistrails_module import new_module, Module

identifier = "edu.utah.sci.dakoop.auto_example"

def expand_ports(port_list):
new_port_list = []
for port in port_list:
port_spec = port[1]
if type(port_spec) == str: # or unicode...
if port_spec.startswith('('):
port_spec = port_spec[1:]
if port_spec.endswith(')'):
port_spec = port_spec[:-1]
new_spec_list = []
for spec in port_spec.split(','):
spec = spec.strip()
parts = spec.split(':', 1)
print 'parts:', parts
namespace = None
if len(parts) > 1:
mod_parts = parts[1].rsplit('|', 1)
if len(mod_parts) > 1:
namespace, module_name = mod_parts
else:
module_name = parts[1]
if len(parts[0].split('.')) == 1:
id_str = 'edu.utah.sci.vistrails.' + parts[0]
else:
id_str = parts[0]
else:
mod_parts = spec.rsplit('|', 1)
if len(mod_parts) > 1:
namespace, module_name = mod_parts
else:
module_name = spec
id_str = identifier
if namespace:
new_spec_list.append(id_str + ':' + module_name + ':' + \
namespace)
else:
new_spec_list.append(id_str + ':' + module_name)
port_spec = '(' + ','.join(new_spec_list) + ')'
new_port_list.append((port[0], port_spec) + port[2:])
print new_port_list
return new_port_list

def build_modules(module_descs):
new_classes = {}
for m_name, m_dict in module_descs:
m_doc = m_dict.get("_doc", None)
m_inputs = m_dict.get("_inputs", [])
m_outputs = m_dict.get("_outputs", [])
if "_inputs" in m_dict:
del m_dict["_inputs"]
if "_outputs" in m_dict:
del m_dict["_outputs"]
if "_doc" in m_dict:
del m_dict["_doc"]
klass_dict = {}
if "_compute" in m_dict:
klass_dict["compute"] = m_dict["_compute"]
del m_dict["_compute"]
m_class = new_module(Module, m_name, klass_dict, m_doc)
m_class._input_ports = expand_ports(m_inputs)
m_class._output_ports = expand_ports(m_outputs)
new_classes[m_name] = (m_class, m_dict)
return new_classes.values()

def initialize():
global _modules
def a_compute(self):
a = self.getInputFromPort("a")
i = 0
if self.hasInputFromPort("i"):
i = self.getInputFromPort("i")
if a == "abc":
i += 100
self.setResult("b", i)

module_descs = [("ModuleA", {"_inputs": [("a", "basic:String")],
"_outputs": [("b", "basic:Integer")],
"_doc": "ModuleA documentation",
"_compute": a_compute,
"namespace": "Test"}),
("ModuleB", {"_inputs": [("a", "Test|ModuleA")],
"_outputs": [("b", "Test|ModuleA")],
"_doc": "ModuleB documentation"})
]

if configuration.use_b:
_modules = build_modules(module_descs)
else:
_modules = build_modules(module_descs[:1])

_modules = []

== Help! This documentation wasn't good enough! ==

Sorry, it's our fault! If you need help, join the vistrails-users list and post your question there.

UsersGuideVisTrailsPackages

2011-03-31T15:17:41Z

Ctuttle: /* Dealing with command line tools and side effects */

== Introduction ==

VisTrails provides a plugin infrastructure to integrate user-defined
functions and libraries. Specifically, users can incorporate their own visualization and simulation codes into pipelines by defining custom modules (or wrappers). These modules are bundled in what we call '''packages'''. A VisTrails package is simply a collection of Python classes -- each of these classes will represent a new module -- created by the user that respects a certain convention. Here's a simplified example of a very simple user-defined module:

class Divide(Module):
def compute(self):
arg1 = self.getInputFromPort("arg1")
arg2 = self.getInputFromPort("arg2")
if arg2 == 0.0:
raise ModuleError(self, "Division by zero")
self.setResult("result", arg1 / arg2)

registry.addModule(Divide)
registry.addInputPort(Divide, "arg1", (basic.Float, 'dividend'))
registry.addInputPort(Divide, "arg2", (basic.Float, 'divisor'))
registry.addOutputPort(Divide, "result", (basic.Float, 'quotient'))

New VisTrails modules must subclass from Module, the base class that defines basic functionality. The only required override is the <tt>compute()</tt> method, which performs the actual module computation. Input and output is specified through ''ports'', which currently have to be explicitly registered with VisTrails. However, this is straightforward, and done through method calls to the module registry. A complete documented example of a (slightly) more complicated module is available [[VistrailsPackagePythonCalcExample | here]].

== Dealing with command line tools and side effects ==

In an ideal world a module's outputs should be completely determined by its inputs. This is important for provenance purposes - if modules have implicit dependencies, it is not possible to be certain that the same results will be generated when the process is reexecuted.

However, it is clear that certain modules are inherently side-effectful (reading/writing files, network, etc). For the common case of temporary files, VisTrails provides a convenience layer that removes part of the burden of managing temporary files. As an illustrative example, consider one of the packages we make available for image conversion, using the [http://www.imagemagick.org/ ImageMagick] suite:

class Convert(ImageMagick):
"""Convert is the base Module for VisTrails Modules in the ImageMagick
package that deal with operations on images. Convert is a bit of a misnomer since
the 'convert' tool does more than simply file format conversion. Each subclass
has a descriptive name of the operation it implements."""

def create_output_file(self):
"""Creates a File with the output format given by the
outputFormat port."""
if self.hasInputFromPort('outputFormat'):
s = '.' + self.getInputFromPort('outputFormat')
return self.interpreter.filePool.create_file(suffix=s)

def geometry_description(self):
"""returns a string with the description of the geometry as
indicated by the appropriate ports (geometry or width and height)"""
# if complete geometry is available, ignore rest
if self.hasInputFromPort("geometry"):
return self.getInputFromPort("geometry")
elif self.hasInputFromPort("width"):
w = self.getInputFromPort("width")
h = self.getInputFromPort("height")
return "'%sx%s'" % (w, h)
else:
raise ModuleError(self, "Needs geometry or width/height")

def run(self, *args):
"""run(*args), runs ImageMagick's 'convert' on a shell, passing all
arguments to the program."""
cmdline = ("convert" + (" %s" * len(args))) % args
if not self.__quiet:
print cmdline
r = os.system(cmdline)
if r != 0:
raise ModuleError(self, "system call failed: '%s'" % cmdline)

def compute(self):
o = self.create_output_file()
i = self.input_file_description()
self.run(i, o.name)
self.setResult("output", o)

(...)

reg.addModule(Convert)
reg.addInputPort(Convert, "geometry", (basic.String, 'ImageMagick geometry'))
reg.addInputPort(Convert, "width", (basic.String, 'width of the geometry for operation'))
reg.addInputPort(Convert, "height", (basic.String, 'height of the geometry for operation'))
reg.addOutputPort(Convert, "output", (basic.File, 'the output file'))

This example introduces several new VisTrails features. The last line of the snippet registers an output port that provides a file. Immediately, a file output presents several problems when a pipeline is to be shared among users in heterogenous environments. For example, where should a file be written to? For temporary files, VisTrails provides a ''file pool'' class, that manages temporary files and their lifetimes automatically, so that users don't have to worry about deleting them post-execution. To create a temporary file, a user calls, for example

fileObj = self.interpreter.filePool.create(suffix=".png")

<tt>fileObj</tt> will then contain a module that represents a file. The file pool simply creates a temporary file with write permissions, whose local name is available, in this case, as <tt>fileObj.name</tt>. The package developer is then free to use this file for any purpose.

Another feature of this example is the use of command line tools. Notice that Python provides a very convenient way to execute commands through a shell. In this case, we use <tt>os.system</tt> on a command-line that executes the appropriate program.

== Interaction with Caching ==

VisTrails provides a caching mechanism, in which portions of pipelines that are common across different executions are automatically shared. However, some modules are intrinsically side-effectful (writing a report to stdout, or a file to disk, or creating a user interface widget), and should not be shared. Caching control is therefore up to the package developer. By default, '''''caching is enabled'''''. So a developer that doesn't want caching to apply must make small changes to the module. There's a convenient way to disable caching entirely, by using multiple inheritance, and deriving from a ''mixin'' class that's provided by VisTrails. For example, look at the StandardOutput module:

from core.modules.vistrails_module import Module, newModule, \
NotCacheable, ModuleError
(...)
class StandardOutput(NotCacheable, Module):
"""StandardOutput is a VisTrails Module that simply prints the
value connected on its port to standard output. It is intended
mostly as a debugging device."""

def compute(self):
v = self.getInputFromPort("value")
print v

By subclassing from <tt>NotCacheable</tt> as well as from Module (or one of its subclasses), VisTrails automatically will not cache this module, or anything downstream from it.

VisTrails also allows a more sophisticated decision on whether to use caching or not. To do that, a user simply overrides the method <tt>is_cacheable</tt> to return the correct value. This allows context-dependent decisions. For example, in the [http://teem.sourceforge.net teem] package, there's a module that generates a scalar field with random numbers. This is non-deterministic, so shouldn't be cached. However, this module only generates non-deterministic values in special occasions, depending on its input port values. To keep efficiency when caching is possible, while still maintaining correctness, that module implements the following override:

class Unu1op(Unu):
(...)
def is_cacheable(self):
return not self.getInputFromPort('op') in ['rand', 'nrand']
(...)

Notice that the module explicitly uses inputs to decide whether it should be cached. This allows reasonably fine-grained control over the process.

== Interaction with Other Packages ==

When developing more complicated packages, it becomes natural to split code among different VisTrails packages, and have one logically depend on the other. For example, in one package (say, named ' ''package_base'' '), a user might define

class PackageBaseModule(Module):
...

def initialize():
registry.addModule(PackageBaseModule)
...

And then, in another package (say, ' ''package_derived'' '),

class DerivedModule(PackageBaseModule):
...

def initialize():
registry.addModule(DerivedModule)
...

Because of the way packages are loaded, ''package_derived'' cannot be initialized before ''package_base''. VisTrails provides a mechanism for specifying interpackage dependencies. Every VisTrails package can provide a list of necessary installed packages. This is done by providing a callable in the package under the name '''package_dependencies'''. For example, here's how the VTK VisTrails package declares dependencies:

def package_dependencies():
import core.packagemanager
manager = core.packagemanager.get_package_manager()
if manager.has_package('spreadsheet'):
return ['spreadsheet']
else:
return []

The callable must return a list of strings, representing the name of the packages it depends on. We also use this example to introduce the package manager API, that is useful here for inspecting packages present in the system. Notice that the dependencies are not static. '''vtk''' depends on '''spreadsheet''' if and only if '''spreadsheet''' is present in the system. Otherwise, it has no dependencies.

''Note'': Circular dependencies are not allowed. They will be detected by VisTrails and an error will be signalled.

''Note'': Currently, package names are reasonably brittle, in the sense that conflicts in package naming might become an issue. We are in the process of designing an API that will allow more robust naming schemes.

== User-defined module shapes and colors ==

VisTrails allows users to define custom colors and shapes to modules. This must be done at module registration time, by passing special parameters to addModule. For example:

reg.addModule(Afront,
moduleColor=(1.0,0.0,0.0),
moduleFringe=[(0.0, 0.0),
(0.2, 0.0),
(0.2, 0.4),
(0.0, 0.4),
(0.0, 1.0)])

gives this result:

[[Image:PackageCustomColorShape1.png]]

This piece of code

reg = core.modules.module_registry
reg.addModule(Afront,
moduleColor=(0.4,0.6,0.8),
moduleFringe=[(0.0, 0.0),
(0.2, 0.0),
(0.0, 0.2),
(0.2, 0.4),
(0.0, 0.6),
(0.2, 0.8),
(0.0, 1.0)])

gives this result:

[[Image:PackageCustomColorShape2.png]]

The moduleColor parameter must be a tuple of three floats between 0 and 1 that specify RGB colors for the module background, while moduleFringe is a list of pairs of floats that specify points as they go around a side of the module (the same one is used to go from the top-right corner to bottom-right corner, and from the bottom-left corner to the top-left one. If this is not enough, let the developers know!)

Alternatively, you can use different fringes for the left and right borders:

reg.addModule(Afront,
moduleColor=(1.0,0.8,0.6),
moduleLeftFringe=[(0.0, 0.0),
(-0.2, 0.0),
(0.0, 1.0)],
moduleRightFringe=[(0.0, 0.0),
(0.2, 1.0),
(0.0, 1.0)])

which gives this:

[[Image:PackageCustomColorShape3.png]]

== How to make you package reloadable ==

You need to move almost everything in the __init__.py file to a new file "init.py", but keep the identifier, name, version, configuration, and package_dependencies fields/methods in the __init__.py file. Specifically, make sure that imports (excluding things like core.configuration) and the initialize method are in the init.py file. For example, take a look at the __init__.py file of the pylab package included in VisTrails:

identifier = 'edu.utah.sci.vistrails.matplotlib'
name = 'matplotlib'
version = '0.9.0'

def package_dependencies():
import core.packagemanager
manager = core.packagemanager.get_package_manager()
if manager.has_package('edu.utah.sci.vistrails.spreadsheet'):
return ['edu.utah.sci.vistrails.spreadsheet']
else:
return []

def package_requirements():
import core.requirements
if not core.requirements.python_module_exists('matplotlib'):
raise core.requirements.MissingRequirement('matplotlib')
if not core.requirements.python_module_exists('pylab'):
raise core.requirements.MissingRequirement('pylab')

And the init.py contains the other imports, class definitions and the initialize method.

== Adding default values and/or labels for parameters ==

In versions 1.4 and greater, package developers can add labels and default values for parameters. To add this functionality, you need to use the <code>defaults</code> and <code>labels</code> keyward arguments and pass the values as '''strings'''. For example,

class TestDefaults(Module):
_input_ports = [("f1", "(edu.utah.sci.vistrails.basic:Float)",
{"defaults": str([1.23]), "labels": str(["temp"])})]
_modules = [TestDefaults]

or in the older syntax,

def initialize():
reg = core.modules.module_registry.get_module_registry()
reg.add_module(TestDefaults2)
reg.add_input_port(TestDefaults2, "f2", [Float, String],
defaults=str([4.56, "abc"]),
labels=str(["temp", "name"]))

== Packages that generate modules dynamically ==

When wrapping existing libraries or trying to generate modules in a more procedural manner, it is useful to dynamically generate modules. In our work, we have created some shortcuts to make this easier. In addition, the list of modules can also be based based on the package configuration. Here is some example code:

=== __init__.py ===

from core.configuration import ConfigurationObject

identifier = "edu.utah.sci.dakoop.auto_example"
version = "0.0.1"
name = "AutoExample"

configuration = ConfigurationObject(use_b=True)

=== init.py ===

The <code>expand_ports</code> and <code>build_modules</code> methods are functions to help the construction of the modules easier. The key parts are the <code>new_module</code> call and setting the <code>_modules</code> variable.

from core.modules.vistrails_module import new_module, Module

identifier = "edu.utah.sci.dakoop.auto_example"

def expand_ports(port_list):
new_port_list = []
for port in port_list:
port_spec = port[1]
if type(port_spec) == str: # or unicode...
if port_spec.startswith('('):
port_spec = port_spec[1:]
if port_spec.endswith(')'):
port_spec = port_spec[:-1]
new_spec_list = []
for spec in port_spec.split(','):
spec = spec.strip()
parts = spec.split(':', 1)
print 'parts:', parts
namespace = None
if len(parts) > 1:
mod_parts = parts[1].rsplit('|', 1)
if len(mod_parts) > 1:
namespace, module_name = mod_parts
else:
module_name = parts[1]
if len(parts[0].split('.')) == 1:
id_str = 'edu.utah.sci.vistrails.' + parts[0]
else:
id_str = parts[0]
else:
mod_parts = spec.rsplit('|', 1)
if len(mod_parts) > 1:
namespace, module_name = mod_parts
else:
module_name = spec
id_str = identifier
if namespace:
new_spec_list.append(id_str + ':' + module_name + ':' + \
namespace)
else:
new_spec_list.append(id_str + ':' + module_name)
port_spec = '(' + ','.join(new_spec_list) + ')'
new_port_list.append((port[0], port_spec) + port[2:])
print new_port_list
return new_port_list

def build_modules(module_descs):
new_classes = {}
for m_name, m_dict in module_descs:
m_doc = m_dict.get("_doc", None)
m_inputs = m_dict.get("_inputs", [])
m_outputs = m_dict.get("_outputs", [])
if "_inputs" in m_dict:
del m_dict["_inputs"]
if "_outputs" in m_dict:
del m_dict["_outputs"]
if "_doc" in m_dict:
del m_dict["_doc"]
klass_dict = {}
if "_compute" in m_dict:
klass_dict["compute"] = m_dict["_compute"]
del m_dict["_compute"]
m_class = new_module(Module, m_name, klass_dict, m_doc)
m_class._input_ports = expand_ports(m_inputs)
m_class._output_ports = expand_ports(m_outputs)
new_classes[m_name] = (m_class, m_dict)
return new_classes.values()

def initialize():
global _modules
def a_compute(self):
a = self.getInputFromPort("a")
i = 0
if self.hasInputFromPort("i"):
i = self.getInputFromPort("i")
if a == "abc":
i += 100
self.setResult("b", i)

module_descs = [("ModuleA", {"_inputs": [("a", "basic:String")],
"_outputs": [("b", "basic:Integer")],
"_doc": "ModuleA documentation",
"_compute": a_compute,
"namespace": "Test"}),
("ModuleB", {"_inputs": [("a", "Test|ModuleA")],
"_outputs": [("b", "Test|ModuleA")],
"_doc": "ModuleB documentation"})
]

if configuration.use_b:
_modules = build_modules(module_descs)
else:
_modules = build_modules(module_descs[:1])

_modules = []

== Help! This documentation wasn't good enough! ==

Sorry, it's our fault! If you need help, join the vistrails-users list and post your question there.

FAQ

2011-03-28T16:45:29Z

Ctuttle: /* How do I access the information in the execution log? */

Also check our [[known_issues|Known Issues]] page for troubleshooting.

----
== Running workflows ==

=== How can I run a workflow using the command line? ===

('''Updated for version 1.2''') Call vistrails using the following options:

<code>python vistrails.py -b path_to_vistrails_file:pipeline </code>

where pipeline can be a version '''tag name''' or version '''id'''

'''NOTE:''' If you downloaded the MacOS X bundle, you can run vistrails from the command line via the following commands in the Terminal. Change the current directory to wherever VisTrails was installed (often <code>/Applications</code>), and then type:

<code>Vistrails.app/Contents/MacOS/vistrails [<cmd_line_options>]</code> [http://www.vistrails.org/usersguide/batch.html#running-a-specific-workflow-in-batch-mode Running a Specific Workflow in Batch Mode]

=== Using the command line, we'd like to execute a workflow multiple times, with slightly different parameters, and create a series of output files. Is this possible?===

('''Updated for version 1.2''') We can change parameters that have an alias through the command line.

For example, offscreen pipeline in offscreen.vt always creates the file called image.png. If you want generate it with a different filename:

<code>python vistrails.py -b ../examples/offscreen.vt:offscreen -a"filename=other.png"</code>

<code>filename</code> in the example above is the alias name assigned to the parameter in the <code>value</code> method inside the String module. When running a pipeline from the command line, VisTrails will try to start the spreadsheet automatically if the pipeline requires it. For example, this other execution will also start the spreadsheet (attention to how $ characters are escaped when running on bash):

<code>python vistrails.py -b ../examples/head.vt:aliases -a"isovalue=30\$&\$diffuse_color=0.8,0.4,0.2"</code>

You can also execute more than one pipeline on the command line:

<code>python vistrails.py -b ../examples/head.vt:aliases ../examples/spx.vt:spx \
-a"isovalue=30"</code>

Use the -a parameter only once regardless the number of pipelines. [http://www.vistrails.org/usersguide/batch.html#running-a-workflow-with-specific-parameters Running a Workflow with Specific Parameters]

=== I can load a vistrail, and the version tree shows up fine. However, no pipelines appear when I click on a version. What gives? ===

The most likely reason is that the vistrail uses a package that is not registered with VisTrails. You need to identify the needed package and add it to your <code>.vistrails/startup.py</code>. A single line like the following should be enough:

addPackage('enter_package_name_here')

Some packages might need more information. For example:

addPackage('afront', executable_path='/path/to/afront')

Refer to the package documentation for details. The one inconvenient step is that currently there's no automated way to describe what is the missing package. We're working on this feature for future releases.

=== I have a workflow that reads a file and then does some processing. The first time it runs, it executes correctly. But in subsequent, nothing happens. ===

VisTrails caches by default, so after a workflow is executed, if none of its parameters change, it won't be executed again.

If a workflow reads a file using the basic module File, VisTrails does check whether the file was modified since the last run. It does so by keeping a signature that is based on the modification time of the file. And if the file was modified, the File module and all downstream modules (the ones which depend on File) will be executed.

''Note: If you would like your input and output data to be versioned, you can use the Persistence package.''

If you do not want VisTrails to cache executions, you can turn off caching: go to Menu Edit -> Preferences and in the General Configuration tab, change Cache execution results to Never. [http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

=== Can VisTrails execute workflows in parallel? ===

The VisTrails server can only execute pipelines in parallel if there's more than one instance of VisTrails running. The command

<code> self.rpcserver = ThreadedXMLRPCServer((self.temp_xml_rpc_options.server, self.temp_xml_rpc_options.port))</code>

starts a multithreaded version of the XML-RPC server, so it will create a thread for each request received by the server. The problem is that Qt/PyQT doesn't allow these multiple threads create GUI objects, only in the main thread. To overcome this limitation, the multithreaded version can instantiate other single threaded versions of VisTrails and put them in a queue, so workflow executions and other GUI-related requests, such as generating workflow graphs and history trees can be forwarded to this queue, and each instance takes turns in answering the request. If the results are in the cache, the multithreaded version answers the requests directly.

Note that this infrastructure works on Linux only. To make this work on Windows, you have to create a script similar to start_vistrails_xvfb.sh (located in the scripts folder) where you can send the number of other instances via command-line options to VisTrails. The command line options are:

python vistrails_server.py -T <ADDRESS> -R <PORT> -O<NUMBER_OF_OTHER_VISTRAILS_INSTANCES> [-M]&

If you want the main vistrails instance to be multithreaded, use the -M at the end.

After creating this script, update function start_other_instances in vistrails/gui/application_server.py lines 1007-1023 and set the script variable to point to your script. You may also have to change the arguments sent to your script (line 1016: for example, you don't need to set a virtual display). You will need to change the path to the stop_vistrails_server.py script (on line 1026) according to your installation path. [http://www.vistrails.org/usersguide/batch.html#executing-workflows-in-parallel Executing Workflows in Parallel]

=== When a workflow is executed, what do the colors mean? ===

- lilac: module was notexecuted

- yellow: module is currently being executed

- green: module was successfully executed

- orange: module was cached

- red: the execution of the module failed

[http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

== Building workflows ==

=== Is there a way to give each widget a "display name" in addition to the module name at the center of the widget? ===

Yes, a "display name" can be assigned to a module by selecting the triangle in its top right corner to open a popup menu and selecting the Set Module Label... menu item. You will then be prompted to enter the "display name". [http://www.vistrails.org/usersguide/creating.html#changing-module-labels Changing Module Labels]

=== Is there a way to re-center the picture-in-picture (PiP) view? ===

Yes. If you click on the PIP window to bring it to focus, you can press Ctrl-R (or Command-R on Mac) to re-center the PiP window. [http://www.vistrails.org/usersguide/getting_started.html#vistrails-interaction Vistrails Interaction]

=== How do I search for a literal "?" (question mark) in the search box in the Property panel? ===

Since we allow regular expressions in our search box, question marks are treated as meta-characters. Thus, searching for "?" returns everything and "abc?" will return everything containing "abc". You need to use "\?" instead to search for "?". So the search for "??" would be "\?\?". [http://www.vistrails.org/usersguide/querying.html#textual-queries Textual Queries]

== Using VisTrails as a server ==

=== What is the VisTrails server-mode? ===

Using the VisTrails server mode, it is possible to ''execute workflows and control VisTrails through another application''. For example, the CrowdLabs Web portal (http://www.crowdlabs.org) accesses a VisTrails sever to execute workflows, retrieve and display vistrail trees and workflows. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

=== How do I execute workflows and control VisTrails through another application? ===

The way you access the server is by doing XML-RPC calls. In the current VisTrails release, we include a set of PHP scripts that can talk to a VisTrails server instance. They are in "extensions/http" folder. The files are reasonably well documented. Also, it should be not difficult to create python scripts to access the server (just use xmlrpclib module).

Note that the VisTrails server requires the provenance and workflows to be in a database. More detailed instructions on how to setup the server and the database
are available here:

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_server_setup.html

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_database_setup.html

If what you want is just to execute a series of workflows in batch mode, a simpler solution would be to ''use the VisTrails client in batch mode''. Chapter 12 of the user's guide contains detailed information and examples on that. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

== Control Flow ==

=== Note: using map ===

When using 'map', the module (or subworkflow) used as function port in the map module MUST be a function, i.e., it can only define 1 output port. [http://www.vistrails.org/usersguide/controlflow.html#the-map-operator The Map Operator]

== Spreadsheet ==

Where ''pipeline'' is a version number or a tag.

=== How can I save an image from the spreadsheet? ===

While having the focus on a spreadsheet cell and select the camera on the toolbar to take a snapshot. The system will prompt you for the location and file name where it should be saved. The other icons can be used for saving multiple images that can be used for generating an animation on demand. A whole sheet can also be saved by selecting Export (either from the menu or from the toolbar). [http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet-image Saving a Spreadsheet Image]

===Is it possible to save the complete state of the spreadsheet?===
[http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet Saving a Spreadsheet]

=== Can I view multiple sheets at the same time? ===

Yes. Each sheet on the spreadsheet can be displayed as a dock widget separated from the main spreadsheet window by dragging its tab name out of the tab bar at the bottom of the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Then, how can I put back a separated sheet? ===

A sheet can be docked back to the main window by dragging it back to the tab bar or double-click on its title bar. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== How can I order sheets on the spreadsheet? ===

This can be done by dragging the sheet name on the bottom top bar and drop it to the right place. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Can I control where a cell will be placed on the spreadsheet window? ===

By default, an unoccupied cell on the active sheet will be chosen to display the result. However, you can specify exactly in the pipeline where a spreadsheet cell will be placed by using CellLocation and SheetReference. CellLocation specifies the location (row and column) of a cell when connecting to a spreadsheet cell (VTKCell, ImageViewerCell, ...). Similarly, a SheetReference module (when connecting to a CellLocation) will specify which sheet the cell will be put on given its name, minimum row size and minimum column size. There is an example of this in examples/vtk.xml (select the version below Double Renderer). [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

=== How do I output results to the spreadsheet? ===

By inspecting the VisTrails Spreadsheet package (in the list of packages, to the left of the pipeline builder), you
can see there are built-in cells for different kinds of data, e.g., RichTextCell to display HTML and plain text. op
You (the user) can also define new cell types to display application-specific data. For example, we have developed
VtkCell, MplFigureCell, and OpenGLCell. It is possible to display pretty much anything on the Spreadsheet! [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

Examples of writing cell modules can be found in:
RichTextCell: packages/spreadsheet/widgets/richtext/richtext.py
VTK: packages/vtk/vtkcell.py

Here is the summary of some requirements on a cell widget:

(1) It must be a Qt widget. It should inherit from spreadsheet_cell.QCellWidget in the spreadsheet package. Although any Qt Widget would work, certain features such as animation will not be available (without rewriting it).

(2) It must re-implement the updateContents() function to take a set of inputs (usually coming from input ports of a wrapper Module) and display on the cells. VisTrails uses this function to update/reuse cells on the spreadsheet when new data comes in.

(3) It needs a wrapper VisTrails Module (inherited from basic_widgets.SpreadsheetCell of the spreadsheet package). Inside the compute() method of this module, it may call self.display(CellWidgetType, (inputs)) to trigger the display event on the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#advanced-cell-options Advanced Cell Options]

=== How do I control the default number of cells in the spreadsheet? ===

You can configure the rowCount and colCount using the preferences dialog. Just go to the Module Packages tab, select spreadsheet in the "Enabled packages" and press the Configure button. Then a list of all the configuration options for the spreadsheet will show up. [http://www.vistrails.org/usersguide/spreadsheet.html#custom-layout-options Custom Layout Options]

=== Is it possible to launch a web browser from the vistrails spreadsheet? We would like to output several urls from a parameter sweep and then have the option to click on each one to view the resulting page. I can view the page within the spreadsheet, but it is really too crowded.===

Currently, there isn't a widget that provides exactly this functionality, but I can think of a few solutions that may work for you:

(1) You can use parameter exploration to generate multiple sheets so you might have an exploration that opens each page in a new sheet. Use the third column/dimension in the exploration interface to have a parameter span sheets.

(2) The spreadsheet is extensible so you can write a custom spreadsheet cell widget that has a button or label with the desired link (a QLabel with openExternalLinks set to True, for example).

(3) You can tweak the existing RichTextCell be adding the line "self.browser.setOpenExternalLinks(True)" at line 63 of the source file "vistrails/packages/spreadsheet/widgets/richtext/richtext.py". Then, if your workflow creates a file with html markup text like "<a href="http://www.vistrails.org">VisTrails</a>" connected to a RichTextCell, clicking on the rendered link in the cell will open it in a web browser. You need to add the aforementioned line to the source to let Qt know that you want the link opened externally; by default, it will just issue an event that isn't processed. [http://www.vistrails.org/usersguide/spreadsheet.html#launching-a-web-browser Launching a Web Browser]

== Integrating your software into VisTrails ==

===How can I integrate my own program into VisTrails?===

The easiest way is to create a package. Writing a package is often very simple, here are instructions on how to do it: [[UsersGuideVisTrailsPackages]]

You can also dynamically generate modules. For an example see:

http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#Packages_that_generate_modules_dynamically

In particular, see the new_module call which uses python's type() function to generate new classes dynamically.

=== How do modules deal with multiple inputs in a same port? ===

(And should that even be allowed?)

For compatibility reasons, we do need to allow multiple connections to an input port. However, most package developers should never have to use this, and so we do our best to hide it. the default behavior for getting inputs from a port, then, is to always return a single input.

If on your module you need multiple inputs connected to a single port, use the 'forceGetInputListFromPort' method. It will return a list of all the data items coming through the port. The spreadsheet package uses this feature, so look there for usage examples (vistrails/packages/spreadsheet/basic_widgets.py)

=== Are there mechanisms for attaching widgets to different modules/parameters? ===

Right now, we have a mechanism for putting a specific widget for an input port. For example, if a port is SetColor(red, green, blue), we can put a color wheel widget there. Or we can also replace the SetFileName port with a File Widget. However, this is not per parameter (only per port). We are currently working on this problem.

=== Can I organize my package so it appears hierarchical in the module palette? ===

Yes. Use the namespace keyword argument when adding the module to the registry. For example,

registry.add_module(MyModule, namespace='MyNamespace')

=== Can I nest namespaces? ===

Yes. Use the '|' character to separate different the hierarchy. For example,

registry.add_module(MyModule, namespace='ParentNamespace|ChildNamespace')

=== Are there shortcuts for registry initialization? ===

Yes. If you define _modules as a list of classes in the __init__.py file of your package, VisTrails will attempt to load all classes specified as modules. You can provide add_module options as keyword arguments by specifying a tuple (class, kwargs) in the list. For example:

_modules = [MyModule1, (MyModule2, {'namespace': 'MyNamespace'})]

In addition, you need to identify the ports of your modules as a field in your class by defining _input_ports and _output_ports lists. Here, the items in each list must be tuples of the form (portName, portSignature, optional=False, sort_key=-1). For example:

class MyModule(Module):
def compute(self):
pass

_input_ports = [('firstInput', String), ('secondInput', Integer, True)]
_output_ports = [('firstOutput', String), ('secondOutput', String)]

=== Can I define ports to be of types that I do not import into my package? ===

Yes. You can pass an identifier string as the portSignature instead. The port_signature string is defined by:

<module_string> := <package_identifier>:[<namespace>|]<module_name>,
<port_signature> := (<module_string>*)

For example,

registry.add_input_port(MyModule, 'myInputPort', '(edu.utah.sci.vistrails.basic:String)')

or

_input_ports = [('myInputPort', '(edu.utah.sci.vistrails.basic:String)')]

=== What do I need to change in my package to make it reloadable (new in v1.4.2)? ===
See [http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#How_to_make_you_package_reloadable UsersGuideVisTrailsPackages] for an explanation.

=== Can I add default values or labels for parameters? ===

Yes. Versions 1.4 and greater support these features. See [[UsersGuideVisTrailsPackages#Adding_default_values_and.2For_labels_for_parameters | UsersGuideVisTrailsPacakges]] for more details.

=== I want to write a module to load HDF data whose output (e.g., data, string) varies according to the input I give it. Is is possible to do this in VisTrails, and if yes, how can I do that? Ideally, I would like to avoid having to change the connection of my output every time I change the input. ===

There are a few ways to tackle this - each has it's own benefits and pitfalls. Firstly, module connections do respect class hierarchies as we're familiar with in object oriented languages. For instance, A module can output a Constant of which String, Float, Integer, etc are specifications. In this way, you can have a subclass of something like HDFData be passed out of the module and the connections will be established regardless of the sub-type. This is a bit dangerous though. Modules downstream of such a class may not really know how to operate on certain types derived from the super-class. Extreme care must be taken both when creating the modules as well as connecting them to prevent things like this from happening.

A second method that I employ in several different packages is the idea of a container class. For instance, the NumSciPy package uses a relatively generic container "Numpy Array" to encapsulate the data. Of course, these encapsulating objects can store dictionaries that other modules can easily access and understand how to operate on. Although this method is slightly more work, the benefits of a stricter typing of ports is beneficial - particularly upon interfacing with other packages that may depend on strongly typed constants (for example).

=== I need to determine, at run-time, whether or not a "child" module is attached to the output port of a "parent" module. (I do not specifically need to know which child; just if there is one). ===

The outputPorts dictionary of the base Module stores this information. Thus, you should be able to check

("myPortName" in self.outputPorts)

on the parent module to check if there are any downstream connections from the port "myPortName". This might be used, for example, to only set results for output ports that will be used. ***Note***, however, that the caching algorithm assumes that all outputs are set so adding a new connection to a previously unconnected output port will not work as desired if that module is cached. For this reason, I would currently recommend making such a module not cacheable. Another possibility is overriding the update() method to check the outputPorts and set the upToDate flag if they are not equal. In a single, limited test, this seemed to work, but be warned that it is not fully tested. Here is an example:

class TestModule(Module):
_output_ports = [('a1', '(edu.utah.sci.vistrails.basic:String)'),
('a2', '(edu.utah.sci.vistrails.basic:String)')]
def __init__(self):
Module.__init__(self)
self._cached_output_ports = set()

def update(self):
if len(set(self.outputPorts) - self._cached_output_ports) > 0:
self.upToDate = False
Module.update(self)

def compute(self):
if "a1" in self.outputPorts:
self.setResult("a1", "test")
if "a2" in self.outputPorts:
self.setResult("a2", "test2")
self._cached_output_ports = set(self.outputPorts)

=== How can I make a module not display in the modules list? ===

You should set the abstract parameter to True when adding the module to the registry. Using the original syntax, this looks like:

def initialize():
reg = core.modules.module_registry.get_module_registry()
reg.add_module(InvisibleModule, abstract=True)
# ...

With the _modules dictionary shortcut (for more details, see [http://www.vistrails.org/index.php/FAQ#Are_there_shortucts_for_registry_initialization.3F the FAQ section on this]), you include it in a kwargs dict as part of a module tuple:

_modules = [AnotherModule, (InvisibleModule, {'abstract': True})]

There is also a 'hide_descriptor' parameter that prevents the module from appearing in the module palette without declaring it to be abstract.

The technical difference between the two is that 'abstract' will not add the item to the module palette while 'hide_descriptor' does add the item but immediately hides it. If the module should never be instantiated in a workflow, declare it abstract. If you don't want users to be able to add the module to a pipeline, but you have code that may add it programmatically, declare it with hide_descriptor=True.

== The Console ==

=== Where should I go to find out what I can call from the console and how to import it? ===

We have tried to make some methods more accessible in the console via an api. You can import the api via <code>import api</code> in the console and see the available methods with <code>dir(api)</code>. To open a vistrail:

import api
api.open_vistrail_from_file('/Applications/VisTrails/examples/terminator.vt')

To execute a version of a workflow, you currently have to go through the controller:

api.select_version('Histogram')
api.get_current_controller().execute_current_workflow()

Currently, only a subset of VisTrails functionality is directly available from the api. However, since VisTrails is written in python, you can dig down starting with the VistrailsApplication or controller object to expose most of our internal methods. If you have suggestions for calls to be added to the api, please let us know.

One other feature that we're working on, but is still in progress is the ability to construct workflows via the console. For example:

vtk = load_package('edu.utah.sci.vistrails.vtk')
vtk.vtkDataSetReader() # adds a vtkDataSetReader module to the pipeline
# click on the new module
a = selected_modules()[0] # get the one currently selected module
a.SetFile('/vistrails/examples/data/head120.vtk') # sets the SetFile parmaeter for the data set reader
b = vtk.vtkContourFilter() # adds a vtkContourFilter module to the pipeline and saves to var b
b.SetInputConnection0(a.GetOutputPort0()) # connects a's GetOutputPort0 port to b's SetInputConnection0

== Persistence Package ==

=== How do I use the output of one workflow as the input for another using the persistence package? ===

You need to configure the persistence modules using the module's configuration dialog. After adding a <code>PersistentOutputFile</code> to the workflow, click on the triangle in the upper-right corner of the <code>PersistentOutputFile</code>, and select "Edit Configuration" from the menu that appears. In this dialog, select "Create New Reference" and give the reference a name (and any space-delimited tags). Upon running that workflow, the data will be written to the persistent store. In the second workflow where you wish to use that file, add a <code>PersistentInputFile</code> and go to its configuration dialog in the same manner as with the output file. In that dialog, select "Use Existing Reference" and select the data that you just added in the first workflow in the list of files below. Now, when you run that workflow, it will grab the data from the persistent store.

Here is an example: [[Media:offscreen_persistent.vt | offscreen_persistent.vt]]. Run the "persistent offscreen" workflow first, and then run the "display persistent output" to use the output of the first workflow as the input for the second.

== VTK ==

=== Given a VTK visualization, how can I generate a webpage from it? ===

Check out the html pipeline in offscreen.xml.

=== I'm trying to use VTK, but there doesn't seem to be any output. What is wrong? ===

To use VTK on VisTrails, you need a slightly different way of connecting the renderer modules. Instead of using the standard RenderWindow/RenderWindowInteractor infrastructure, you simply connect the renderer to a VTKCell. The examples directory in the distribution has several VTK examples that illustrate.

=== I am trying to add a module to the workflow via Python, but how can I access vtk modules? ===

Here's an example:

import api

vtvtk = 'edu.utah.sci.vistrails.vtk'

module = api.add_module(0, 0, vtvtk, 'vtkContourFilter', '')

The third argument in add_module is the package identifier. You can find this in the "Module Packages" panel of the Preferences; just click on the package you're interested in and it will appear in the information on the right.

== matplotlib ==

=== I'm experiencing a problem with Latex labels and the matplotlib that comes with VisTrails 1.5. The script below entered to the interpreter that comes with VT is sufficient to reproduce it. ===

import matplotlib.pyplot as plt
plt.plot([1,2,3],[1,2,3])
plt.xlabel("$foo$")

Remove your ~/.matplotlib folder and re-start VisTrails

== VisTrails Development ==

=== I would like to build VisTrails from source. Are there instructions on how to do this? ===

Yes! Take a look at http://www.vistrails.org/index.php/Mac_Intel_Instructions
[http://www.vistrails.org/usersguide/getting_started.html#installing-vistrails-from-source Installing VisTrails from source]

== Accessing Provenance Information ==

=== How do I access the information in the execution log? ===

The code responsible for storing execution information is located in the "core/log" directories, and the code that generates much of that information is in "core/interpreter/cached.py". Modules can add execution-specific annotations to provenance via annotate() calls during execution, but much of the data (like timing and errors) is captured by the LogController and CachedInterpreter (the execution engine) objects. To analyze the log from a vistrail (.vt) file, you might have something like the following:

<code>
import core.log.log
import db.services.io

def run(fname):
# open the .vt bundle specified by the filename "fname"
bundle = db.services.io.open_vistrail_bundle_from_zip_xml(fname)[0]
# get the log filename
log_fname = bundle.vistrail.db_log_filename
if log_fname is not None:
# open the log
log = db.services.io.open_log_from_xml(log_fname, True)
# convert the log from a db object
core.log.log.Log.convert(log)
for workflow_exec in log.workflow_execs:
print 'workflow version:', workflow_exec.parent_version
print 'time started:', workflow_exec.ts_start
print 'time ended:', workflow_exec.ts_end
print 'modules executed:', [i.module_id
for i in workflow_exec.item_execs]
if __name__ == '__main__':
run("some_vistrail.vt")
</code>

You should be able to see what information is available by looking at the "core/log" classes. [http://www.vistrails.org/usersguide/log.html Accessing the Execution Log]

FAQ

2011-03-28T14:35:48Z

Ctuttle: /* How do modules deal with multiple inputs in a same port? */

Also check our [[known_issues|Known Issues]] page for troubleshooting.

----
== Running workflows ==

=== How can I run a workflow using the command line? ===

('''Updated for version 1.2''') Call vistrails using the following options:

<code>python vistrails.py -b path_to_vistrails_file:pipeline </code>

where pipeline can be a version '''tag name''' or version '''id'''

'''NOTE:''' If you downloaded the MacOS X bundle, you can run vistrails from the command line via the following commands in the Terminal. Change the current directory to wherever VisTrails was installed (often <code>/Applications</code>), and then type:

<code>Vistrails.app/Contents/MacOS/vistrails [<cmd_line_options>]</code> [http://www.vistrails.org/usersguide/batch.html#running-a-specific-workflow-in-batch-mode Running a Specific Workflow in Batch Mode]

=== Using the command line, we'd like to execute a workflow multiple times, with slightly different parameters, and create a series of output files. Is this possible?===

('''Updated for version 1.2''') We can change parameters that have an alias through the command line.

For example, offscreen pipeline in offscreen.vt always creates the file called image.png. If you want generate it with a different filename:

<code>python vistrails.py -b ../examples/offscreen.vt:offscreen -a"filename=other.png"</code>

<code>filename</code> in the example above is the alias name assigned to the parameter in the <code>value</code> method inside the String module. When running a pipeline from the command line, VisTrails will try to start the spreadsheet automatically if the pipeline requires it. For example, this other execution will also start the spreadsheet (attention to how $ characters are escaped when running on bash):

<code>python vistrails.py -b ../examples/head.vt:aliases -a"isovalue=30\$&\$diffuse_color=0.8,0.4,0.2"</code>

You can also execute more than one pipeline on the command line:

<code>python vistrails.py -b ../examples/head.vt:aliases ../examples/spx.vt:spx \
-a"isovalue=30"</code>

Use the -a parameter only once regardless the number of pipelines. [http://www.vistrails.org/usersguide/batch.html#running-a-workflow-with-specific-parameters Running a Workflow with Specific Parameters]

=== I can load a vistrail, and the version tree shows up fine. However, no pipelines appear when I click on a version. What gives? ===

The most likely reason is that the vistrail uses a package that is not registered with VisTrails. You need to identify the needed package and add it to your <code>.vistrails/startup.py</code>. A single line like the following should be enough:

addPackage('enter_package_name_here')

Some packages might need more information. For example:

addPackage('afront', executable_path='/path/to/afront')

Refer to the package documentation for details. The one inconvenient step is that currently there's no automated way to describe what is the missing package. We're working on this feature for future releases.

=== I have a workflow that reads a file and then does some processing. The first time it runs, it executes correctly. But in subsequent, nothing happens. ===

VisTrails caches by default, so after a workflow is executed, if none of its parameters change, it won't be executed again.

If a workflow reads a file using the basic module File, VisTrails does check whether the file was modified since the last run. It does so by keeping a signature that is based on the modification time of the file. And if the file was modified, the File module and all downstream modules (the ones which depend on File) will be executed.

''Note: If you would like your input and output data to be versioned, you can use the Persistence package.''

If you do not want VisTrails to cache executions, you can turn off caching: go to Menu Edit -> Preferences and in the General Configuration tab, change Cache execution results to Never. [http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

=== Can VisTrails execute workflows in parallel? ===

The VisTrails server can only execute pipelines in parallel if there's more than one instance of VisTrails running. The command

<code> self.rpcserver = ThreadedXMLRPCServer((self.temp_xml_rpc_options.server, self.temp_xml_rpc_options.port))</code>

starts a multithreaded version of the XML-RPC server, so it will create a thread for each request received by the server. The problem is that Qt/PyQT doesn't allow these multiple threads create GUI objects, only in the main thread. To overcome this limitation, the multithreaded version can instantiate other single threaded versions of VisTrails and put them in a queue, so workflow executions and other GUI-related requests, such as generating workflow graphs and history trees can be forwarded to this queue, and each instance takes turns in answering the request. If the results are in the cache, the multithreaded version answers the requests directly.

Note that this infrastructure works on Linux only. To make this work on Windows, you have to create a script similar to start_vistrails_xvfb.sh (located in the scripts folder) where you can send the number of other instances via command-line options to VisTrails. The command line options are:

python vistrails_server.py -T <ADDRESS> -R <PORT> -O<NUMBER_OF_OTHER_VISTRAILS_INSTANCES> [-M]&

If you want the main vistrails instance to be multithreaded, use the -M at the end.

After creating this script, update function start_other_instances in vistrails/gui/application_server.py lines 1007-1023 and set the script variable to point to your script. You may also have to change the arguments sent to your script (line 1016: for example, you don't need to set a virtual display). You will need to change the path to the stop_vistrails_server.py script (on line 1026) according to your installation path. [http://www.vistrails.org/usersguide/batch.html#executing-workflows-in-parallel Executing Workflows in Parallel]

=== When a workflow is executed, what do the colors mean? ===

- lilac: module was notexecuted

- yellow: module is currently being executed

- green: module was successfully executed

- orange: module was cached

- red: the execution of the module failed

[http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

== Building workflows ==

=== Is there a way to give each widget a "display name" in addition to the module name at the center of the widget? ===

Yes, a "display name" can be assigned to a module by selecting the triangle in its top right corner to open a popup menu and selecting the Set Module Label... menu item. You will then be prompted to enter the "display name". [http://www.vistrails.org/usersguide/creating.html#changing-module-labels Changing Module Labels]

=== Is there a way to re-center the picture-in-picture (PiP) view? ===

Yes. If you click on the PIP window to bring it to focus, you can press Ctrl-R (or Command-R on Mac) to re-center the PiP window. [http://www.vistrails.org/usersguide/getting_started.html#vistrails-interaction Vistrails Interaction]

=== How do I search for a literal "?" (question mark) in the search box in the Property panel? ===

Since we allow regular expressions in our search box, question marks are treated as meta-characters. Thus, searching for "?" returns everything and "abc?" will return everything containing "abc". You need to use "\?" instead to search for "?". So the search for "??" would be "\?\?". [http://www.vistrails.org/usersguide/querying.html#textual-queries Textual Queries]

== Using VisTrails as a server ==

=== What is the VisTrails server-mode? ===

Using the VisTrails server mode, it is possible to ''execute workflows and control VisTrails through another application''. For example, the CrowdLabs Web portal (http://www.crowdlabs.org) accesses a VisTrails sever to execute workflows, retrieve and display vistrail trees and workflows. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

=== How do I execute workflows and control VisTrails through another application? ===

The way you access the server is by doing XML-RPC calls. In the current VisTrails release, we include a set of PHP scripts that can talk to a VisTrails server instance. They are in "extensions/http" folder. The files are reasonably well documented. Also, it should be not difficult to create python scripts to access the server (just use xmlrpclib module).

Note that the VisTrails server requires the provenance and workflows to be in a database. More detailed instructions on how to setup the server and the database
are available here:

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_server_setup.html

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_database_setup.html

If what you want is just to execute a series of workflows in batch mode, a simpler solution would be to ''use the VisTrails client in batch mode''. Chapter 12 of the user's guide contains detailed information and examples on that. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

== Control Flow ==

=== Note: using map ===

When using 'map', the module (or subworkflow) used as function port in the map module MUST be a function, i.e., it can only define 1 output port. [http://www.vistrails.org/usersguide/controlflow.html#the-map-operator The Map Operator]

== Spreadsheet ==

Where ''pipeline'' is a version number or a tag.

=== How can I save an image from the spreadsheet? ===

While having the focus on a spreadsheet cell and select the camera on the toolbar to take a snapshot. The system will prompt you for the location and file name where it should be saved. The other icons can be used for saving multiple images that can be used for generating an animation on demand. A whole sheet can also be saved by selecting Export (either from the menu or from the toolbar). [http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet-image Saving a Spreadsheet Image]

===Is it possible to save the complete state of the spreadsheet?===
[http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet Saving a Spreadsheet]

=== Can I view multiple sheets at the same time? ===

Yes. Each sheet on the spreadsheet can be displayed as a dock widget separated from the main spreadsheet window by dragging its tab name out of the tab bar at the bottom of the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Then, how can I put back a separated sheet? ===

A sheet can be docked back to the main window by dragging it back to the tab bar or double-click on its title bar. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== How can I order sheets on the spreadsheet? ===

This can be done by dragging the sheet name on the bottom top bar and drop it to the right place. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Can I control where a cell will be placed on the spreadsheet window? ===

By default, an unoccupied cell on the active sheet will be chosen to display the result. However, you can specify exactly in the pipeline where a spreadsheet cell will be placed by using CellLocation and SheetReference. CellLocation specifies the location (row and column) of a cell when connecting to a spreadsheet cell (VTKCell, ImageViewerCell, ...). Similarly, a SheetReference module (when connecting to a CellLocation) will specify which sheet the cell will be put on given its name, minimum row size and minimum column size. There is an example of this in examples/vtk.xml (select the version below Double Renderer). [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

=== How do I output results to the spreadsheet? ===

By inspecting the VisTrails Spreadsheet package (in the list of packages, to the left of the pipeline builder), you
can see there are built-in cells for different kinds of data, e.g., RichTextCell to display HTML and plain text. op
You (the user) can also define new cell types to display application-specific data. For example, we have developed
VtkCell, MplFigureCell, and OpenGLCell. It is possible to display pretty much anything on the Spreadsheet! [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

Examples of writing cell modules can be found in:
RichTextCell: packages/spreadsheet/widgets/richtext/richtext.py
VTK: packages/vtk/vtkcell.py

Here is the summary of some requirements on a cell widget:

(1) It must be a Qt widget. It should inherit from spreadsheet_cell.QCellWidget in the spreadsheet package. Although any Qt Widget would work, certain features such as animation will not be available (without rewriting it).

(2) It must re-implement the updateContents() function to take a set of inputs (usually coming from input ports of a wrapper Module) and display on the cells. VisTrails uses this function to update/reuse cells on the spreadsheet when new data comes in.

(3) It needs a wrapper VisTrails Module (inherited from basic_widgets.SpreadsheetCell of the spreadsheet package). Inside the compute() method of this module, it may call self.display(CellWidgetType, (inputs)) to trigger the display event on the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#advanced-cell-options Advanced Cell Options]

=== How do I control the default number of cells in the spreadsheet? ===

You can configure the rowCount and colCount using the preferences dialog. Just go to the Module Packages tab, select spreadsheet in the "Enabled packages" and press the Configure button. Then a list of all the configuration options for the spreadsheet will show up. [http://www.vistrails.org/usersguide/spreadsheet.html#custom-layout-options Custom Layout Options]

=== Is it possible to launch a web browser from the vistrails spreadsheet? We would like to output several urls from a parameter sweep and then have the option to click on each one to view the resulting page. I can view the page within the spreadsheet, but it is really too crowded.===

Currently, there isn't a widget that provides exactly this functionality, but I can think of a few solutions that may work for you:

(1) You can use parameter exploration to generate multiple sheets so you might have an exploration that opens each page in a new sheet. Use the third column/dimension in the exploration interface to have a parameter span sheets.

(2) The spreadsheet is extensible so you can write a custom spreadsheet cell widget that has a button or label with the desired link (a QLabel with openExternalLinks set to True, for example).

(3) You can tweak the existing RichTextCell be adding the line "self.browser.setOpenExternalLinks(True)" at line 63 of the source file "vistrails/packages/spreadsheet/widgets/richtext/richtext.py". Then, if your workflow creates a file with html markup text like "<a href="http://www.vistrails.org">VisTrails</a>" connected to a RichTextCell, clicking on the rendered link in the cell will open it in a web browser. You need to add the aforementioned line to the source to let Qt know that you want the link opened externally; by default, it will just issue an event that isn't processed. [http://www.vistrails.org/usersguide/spreadsheet.html#launching-a-web-browser Launching a Web Browser]

== Integrating your software into VisTrails ==

===How can I integrate my own program into VisTrails?===

The easiest way is to create a package. Writing a package is often very simple, here are instructions on how to do it: [[UsersGuideVisTrailsPackages]]

You can also dynamically generate modules. For an example see:

http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#Packages_that_generate_modules_dynamically

In particular, see the new_module call which uses python's type() function to generate new classes dynamically.

=== How do modules deal with multiple inputs in a same port? ===

(And should that even be allowed?)

For compatibility reasons, we do need to allow multiple connections to an input port. However, most package developers should never have to use this, and so we do our best to hide it. the default behavior for getting inputs from a port, then, is to always return a single input.

If on your module you need multiple inputs connected to a single port, use the 'forceGetInputListFromPort' method. It will return a list of all the data items coming through the port. The spreadsheet package uses this feature, so look there for usage examples (vistrails/packages/spreadsheet/basic_widgets.py)

=== Are there mechanisms for attaching widgets to different modules/parameters? ===

Right now, we have a mechanism for putting a specific widget for an input port. For example, if a port is SetColor(red, green, blue), we can put a color wheel widget there. Or we can also replace the SetFileName port with a File Widget. However, this is not per parameter (only per port). We are currently working on this problem.

=== Can I organize my package so it appears hierarchical in the module palette? ===

Yes. Use the namespace keyword argument when adding the module to the registry. For example,

registry.add_module(MyModule, namespace='MyNamespace')

=== Can I nest namespaces? ===

Yes. Use the '|' character to separate different the hierarchy. For example,

registry.add_module(MyModule, namespace='ParentNamespace|ChildNamespace')

=== Are there shortcuts for registry initialization? ===

Yes. If you define _modules as a list of classes in the __init__.py file of your package, VisTrails will attempt to load all classes specified as modules. You can provide add_module options as keyword arguments by specifying a tuple (class, kwargs) in the list. For example:

_modules = [MyModule1, (MyModule2, {'namespace': 'MyNamespace'})]

In addition, you need to identify the ports of your modules as a field in your class by defining _input_ports and _output_ports lists. Here, the items in each list must be tuples of the form (portName, portSignature, optional=False, sort_key=-1). For example:

class MyModule(Module):
def compute(self):
pass

_input_ports = [('firstInput', String), ('secondInput', Integer, True)]
_output_ports = [('firstOutput', String), ('secondOutput', String)]

=== Can I define ports to be of types that I do not import into my package? ===

Yes. You can pass an identifier string as the portSignature instead. The port_signature string is defined by:

<module_string> := <package_identifier>:[<namespace>|]<module_name>,
<port_signature> := (<module_string>*)

For example,

registry.add_input_port(MyModule, 'myInputPort', '(edu.utah.sci.vistrails.basic:String)')

or

_input_ports = [('myInputPort', '(edu.utah.sci.vistrails.basic:String)')]

=== What do I need to change in my package to make it reloadable (new in v1.4.2)? ===
See [http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#How_to_make_you_package_reloadable UsersGuideVisTrailsPackages] for an explanation.

=== Can I add default values or labels for parameters? ===

Yes. Versions 1.4 and greater support these features. See [[UsersGuideVisTrailsPackages#Adding_default_values_and.2For_labels_for_parameters | UsersGuideVisTrailsPacakges]] for more details.

=== I want to write a module to load HDF data whose output (e.g., data, string) varies according to the input I give it. Is is possible to do this in VisTrails, and if yes, how can I do that? Ideally, I would like to avoid having to change the connection of my output every time I change the input. ===

There are a few ways to tackle this - each has it's own benefits and pitfalls. Firstly, module connections do respect class hierarchies as we're familiar with in object oriented languages. For instance, A module can output a Constant of which String, Float, Integer, etc are specifications. In this way, you can have a subclass of something like HDFData be passed out of the module and the connections will be established regardless of the sub-type. This is a bit dangerous though. Modules downstream of such a class may not really know how to operate on certain types derived from the super-class. Extreme care must be taken both when creating the modules as well as connecting them to prevent things like this from happening.

A second method that I employ in several different packages is the idea of a container class. For instance, the NumSciPy package uses a relatively generic container "Numpy Array" to encapsulate the data. Of course, these encapsulating objects can store dictionaries that other modules can easily access and understand how to operate on. Although this method is slightly more work, the benefits of a stricter typing of ports is beneficial - particularly upon interfacing with other packages that may depend on strongly typed constants (for example).

=== I need to determine, at run-time, whether or not a "child" module is attached to the output port of a "parent" module. (I do not specifically need to know which child; just if there is one). ===

The outputPorts dictionary of the base Module stores this information. Thus, you should be able to check

("myPortName" in self.outputPorts)

on the parent module to check if there are any downstream connections from the port "myPortName". This might be used, for example, to only set results for output ports that will be used. ***Note***, however, that the caching algorithm assumes that all outputs are set so adding a new connection to a previously unconnected output port will not work as desired if that module is cached. For this reason, I would currently recommend making such a module not cacheable. Another possibility is overriding the update() method to check the outputPorts and set the upToDate flag if they are not equal. In a single, limited test, this seemed to work, but be warned that it is not fully tested. Here is an example:

class TestModule(Module):
_output_ports = [('a1', '(edu.utah.sci.vistrails.basic:String)'),
('a2', '(edu.utah.sci.vistrails.basic:String)')]
def __init__(self):
Module.__init__(self)
self._cached_output_ports = set()

def update(self):
if len(set(self.outputPorts) - self._cached_output_ports) > 0:
self.upToDate = False
Module.update(self)

def compute(self):
if "a1" in self.outputPorts:
self.setResult("a1", "test")
if "a2" in self.outputPorts:
self.setResult("a2", "test2")
self._cached_output_ports = set(self.outputPorts)

=== How can I make a module not display in the modules list? ===

You should set the abstract parameter to True when adding the module to the registry. Using the original syntax, this looks like:

def initialize():
reg = core.modules.module_registry.get_module_registry()
reg.add_module(InvisibleModule, abstract=True)
# ...

With the _modules dictionary shortcut (for more details, see [http://www.vistrails.org/index.php/FAQ#Are_there_shortucts_for_registry_initialization.3F the FAQ section on this]), you include it in a kwargs dict as part of a module tuple:

_modules = [AnotherModule, (InvisibleModule, {'abstract': True})]

There is also a 'hide_descriptor' parameter that prevents the module from appearing in the module palette without declaring it to be abstract.

The technical difference between the two is that 'abstract' will not add the item to the module palette while 'hide_descriptor' does add the item but immediately hides it. If the module should never be instantiated in a workflow, declare it abstract. If you don't want users to be able to add the module to a pipeline, but you have code that may add it programmatically, declare it with hide_descriptor=True.

== The Console ==

=== Where should I go to find out what I can call from the console and how to import it? ===

We have tried to make some methods more accessible in the console via an api. You can import the api via <code>import api</code> in the console and see the available methods with <code>dir(api)</code>. To open a vistrail:

import api
api.open_vistrail_from_file('/Applications/VisTrails/examples/terminator.vt')

To execute a version of a workflow, you currently have to go through the controller:

api.select_version('Histogram')
api.get_current_controller().execute_current_workflow()

Currently, only a subset of VisTrails functionality is directly available from the api. However, since VisTrails is written in python, you can dig down starting with the VistrailsApplication or controller object to expose most of our internal methods. If you have suggestions for calls to be added to the api, please let us know.

One other feature that we're working on, but is still in progress is the ability to construct workflows via the console. For example:

vtk = load_package('edu.utah.sci.vistrails.vtk')
vtk.vtkDataSetReader() # adds a vtkDataSetReader module to the pipeline
# click on the new module
a = selected_modules()[0] # get the one currently selected module
a.SetFile('/vistrails/examples/data/head120.vtk') # sets the SetFile parmaeter for the data set reader
b = vtk.vtkContourFilter() # adds a vtkContourFilter module to the pipeline and saves to var b
b.SetInputConnection0(a.GetOutputPort0()) # connects a's GetOutputPort0 port to b's SetInputConnection0

== Persistence Package ==

=== How do I use the output of one workflow as the input for another using the persistence package? ===

You need to configure the persistence modules using the module's configuration dialog. After adding a <code>PersistentOutputFile</code> to the workflow, click on the triangle in the upper-right corner of the <code>PersistentOutputFile</code>, and select "Edit Configuration" from the menu that appears. In this dialog, select "Create New Reference" and give the reference a name (and any space-delimited tags). Upon running that workflow, the data will be written to the persistent store. In the second workflow where you wish to use that file, add a <code>PersistentInputFile</code> and go to its configuration dialog in the same manner as with the output file. In that dialog, select "Use Existing Reference" and select the data that you just added in the first workflow in the list of files below. Now, when you run that workflow, it will grab the data from the persistent store.

Here is an example: [[Media:offscreen_persistent.vt | offscreen_persistent.vt]]. Run the "persistent offscreen" workflow first, and then run the "display persistent output" to use the output of the first workflow as the input for the second.

== VTK ==

=== Given a VTK visualization, how can I generate a webpage from it? ===

Check out the html pipeline in offscreen.xml.

=== I'm trying to use VTK, but there doesn't seem to be any output. What is wrong? ===

To use VTK on VisTrails, you need a slightly different way of connecting the renderer modules. Instead of using the standard RenderWindow/RenderWindowInteractor infrastructure, you simply connect the renderer to a VTKCell. The examples directory in the distribution has several VTK examples that illustrate.

=== I am trying to add a module to the workflow via Python, but how can I access vtk modules? ===

Here's an example:

import api

vtvtk = 'edu.utah.sci.vistrails.vtk'

module = api.add_module(0, 0, vtvtk, 'vtkContourFilter', '')

The third argument in add_module is the package identifier. You can find this in the "Module Packages" panel of the Preferences; just click on the package you're interested in and it will appear in the information on the right.

== matplotlib ==

=== I'm experiencing a problem with Latex labels and the matplotlib that comes with VisTrails 1.5. The script below entered to the interpreter that comes with VT is sufficient to reproduce it. ===

import matplotlib.pyplot as plt
plt.plot([1,2,3],[1,2,3])
plt.xlabel("$foo$")

Remove your ~/.matplotlib folder and re-start VisTrails

== VisTrails Development ==

=== I would like to build VisTrails from source. Are there instructions on how to do this? ===

Yes! Take a look at http://www.vistrails.org/index.php/Mac_Intel_Instructions
[http://www.vistrails.org/usersguide/getting_started.html#installing-vistrails-from-source Installing VisTrails from source]

== Accessing Provenance Information ==

=== How do I access the information in the execution log? ===

The code responsible for storing execution information is located in the "core/log" directories, and the code that generates much of that information is in "core/interpreter/cached.py". Modules can add execution-specific annotations to provenance via annotate() calls during execution, but much of the data (like timing and errors) is captured by the LogController and CachedInterpreter (the execution engine) objects. To analyze the log from a vistrail (.vt) file, you might have something like the following:

<code>
import core.log.log
import db.services.io

def run(fname):
# open the .vt bundle specified by the filename "fname"
bundle = db.services.io.open_vistrail_bundle_from_zip_xml(fname)[0]
# get the log filename
log_fname = bundle.vistrail.db_log_filename
if log_fname is not None:
# open the log
log = db.services.io.open_log_from_xml(log_fname, True)
# convert the log from a db object
core.log.log.Log.convert(log)
for workflow_exec in log.workflow_execs:
print 'workflow version:', workflow_exec.parent_version
print 'time started:', workflow_exec.ts_start
print 'time ended:', workflow_exec.ts_end
print 'modules executed:', [i.module_id
for i in workflow_exec.item_execs]
if __name__ == '__main__':
run("some_vistrail.vt")
</code>

You should be able to see what information is available by looking at the "core/log" classes.

FAQ

2011-03-26T18:29:59Z

Ctuttle: /* Using the command line, we'd like to execute a workflow multiple times, with slightly different parameters, and create a series of output files. Is this possible? */

Also check our [[known_issues|Known Issues]] page for troubleshooting.

----
== Running workflows ==

=== How can I run a workflow using the command line? ===

('''Updated for version 1.2''') Call vistrails using the following options:

<code>python vistrails.py -b path_to_vistrails_file:pipeline </code>

where pipeline can be a version '''tag name''' or version '''id'''

'''NOTE:''' If you downloaded the MacOS X bundle, you can run vistrails from the command line via the following commands in the Terminal. Change the current directory to wherever VisTrails was installed (often <code>/Applications</code>), and then type:

<code>Vistrails.app/Contents/MacOS/vistrails [<cmd_line_options>]</code> [http://www.vistrails.org/usersguide/batch.html#running-a-specific-workflow-in-batch-mode Running a Specific Workflow in Batch Mode]

=== Using the command line, we'd like to execute a workflow multiple times, with slightly different parameters, and create a series of output files. Is this possible?===

('''Updated for version 1.2''') We can change parameters that have an alias through the command line.

For example, offscreen pipeline in offscreen.vt always creates the file called image.png. If you want generate it with a different filename:

<code>python vistrails.py -b ../examples/offscreen.vt:offscreen -a"filename=other.png"</code>

<code>filename</code> in the example above is the alias name assigned to the parameter in the <code>value</code> method inside the String module. When running a pipeline from the command line, VisTrails will try to start the spreadsheet automatically if the pipeline requires it. For example, this other execution will also start the spreadsheet (attention to how $ characters are escaped when running on bash):

<code>python vistrails.py -b ../examples/head.vt:aliases -a"isovalue=30\$&\$diffuse_color=0.8,0.4,0.2"</code>

You can also execute more than one pipeline on the command line:

<code>python vistrails.py -b ../examples/head.vt:aliases ../examples/spx.vt:spx \
-a"isovalue=30"</code>

Use the -a parameter only once regardless the number of pipelines. [http://www.vistrails.org/usersguide/batch.html#running-a-workflow-with-specific-parameters Running a Workflow with Specific Parameters]

=== I can load a vistrail, and the version tree shows up fine. However, no pipelines appear when I click on a version. What gives? ===

The most likely reason is that the vistrail uses a package that is not registered with VisTrails. You need to identify the needed package and add it to your <code>.vistrails/startup.py</code>. A single line like the following should be enough:

addPackage('enter_package_name_here')

Some packages might need more information. For example:

addPackage('afront', executable_path='/path/to/afront')

Refer to the package documentation for details. The one inconvenient step is that currently there's no automated way to describe what is the missing package. We're working on this feature for future releases.

=== I have a workflow that reads a file and then does some processing. The first time it runs, it executes correctly. But in subsequent, nothing happens. ===

VisTrails caches by default, so after a workflow is executed, if none of its parameters change, it won't be executed again.

If a workflow reads a file using the basic module File, VisTrails does check whether the file was modified since the last run. It does so by keeping a signature that is based on the modification time of the file. And if the file was modified, the File module and all downstream modules (the ones which depend on File) will be executed.

''Note: If you would like your input and output data to be versioned, you can use the Persistence package.''

If you do not want VisTrails to cache executions, you can turn off caching: go to Menu Edit -> Preferences and in the General Configuration tab, change Cache execution results to Never. [http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

=== Can VisTrails execute workflows in parallel? ===

The VisTrails server can only execute pipelines in parallel if there's more than one instance of VisTrails running. The command

<code> self.rpcserver = ThreadedXMLRPCServer((self.temp_xml_rpc_options.server, self.temp_xml_rpc_options.port))</code>

starts a multithreaded version of the XML-RPC server, so it will create a thread for each request received by the server. The problem is that Qt/PyQT doesn't allow these multiple threads create GUI objects, only in the main thread. To overcome this limitation, the multithreaded version can instantiate other single threaded versions of VisTrails and put them in a queue, so workflow executions and other GUI-related requests, such as generating workflow graphs and history trees can be forwarded to this queue, and each instance takes turns in answering the request. If the results are in the cache, the multithreaded version answers the requests directly.

Note that this infrastructure works on Linux only. To make this work on Windows, you have to create a script similar to start_vistrails_xvfb.sh (located in the scripts folder) where you can send the number of other instances via command-line options to VisTrails. The command line options are:

python vistrails_server.py -T <ADDRESS> -R <PORT> -O<NUMBER_OF_OTHER_VISTRAILS_INSTANCES> [-M]&

If you want the main vistrails instance to be multithreaded, use the -M at the end.

After creating this script, update function start_other_instances in vistrails/gui/application_server.py lines 1007-1023 and set the script variable to point to your script. You may also have to change the arguments sent to your script (line 1016: for example, you don't need to set a virtual display). You will need to change the path to the stop_vistrails_server.py script (on line 1026) according to your installation path. [http://www.vistrails.org/usersguide/batch.html#executing-workflows-in-parallel Executing Workflows in Parallel]

=== When a workflow is executed, what do the colors mean? ===

- lilac: module was notexecuted

- yellow: module is currently being executed

- green: module was successfully executed

- orange: module was cached

- red: the execution of the module failed

[http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

== Building workflows ==

=== Is there a way to give each widget a "display name" in addition to the module name at the center of the widget? ===

Yes, a "display name" can be assigned to a module by selecting the triangle in its top right corner to open a popup menu and selecting the Set Module Label... menu item. You will then be prompted to enter the "display name". [http://www.vistrails.org/usersguide/creating.html#changing-module-labels Changing Module Labels]

=== Is there a way to re-center the picture-in-picture (PiP) view? ===

Yes. If you click on the PIP window to bring it to focus, you can press Ctrl-R (or Command-R on Mac) to re-center the PiP window. [http://www.vistrails.org/usersguide/getting_started.html#vistrails-interaction Vistrails Interaction]

=== How do I search for a literal "?" (question mark) in the search box in the Property panel? ===

Since we allow regular expressions in our search box, question marks are treated as meta-characters. Thus, searching for "?" returns everything and "abc?" will return everything containing "abc". You need to use "\?" instead to search for "?". So the search for "??" would be "\?\?". [http://www.vistrails.org/usersguide/querying.html#textual-queries Textual Queries]

== Using VisTrails as a server ==

=== What is the VisTrails server-mode? ===

Using the VisTrails server mode, it is possible to ''execute workflows and control VisTrails through another application''. For example, the CrowdLabs Web portal (http://www.crowdlabs.org) accesses a VisTrails sever to execute workflows, retrieve and display vistrail trees and workflows. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

=== How do I execute workflows and control VisTrails through another application? ===

The way you access the server is by doing XML-RPC calls. In the current VisTrails release, we include a set of PHP scripts that can talk to a VisTrails server instance. They are in "extensions/http" folder. The files are reasonably well documented. Also, it should be not difficult to create python scripts to access the server (just use xmlrpclib module).

Note that the VisTrails server requires the provenance and workflows to be in a database. More detailed instructions on how to setup the server and the database
are available here:

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_server_setup.html

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_database_setup.html

If what you want is just to execute a series of workflows in batch mode, a simpler solution would be to ''use the VisTrails client in batch mode''. Chapter 12 of the user's guide contains detailed information and examples on that. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

== Control Flow ==

=== Note: using map ===

When using 'map', the module (or subworkflow) used as function port in the map module MUST be a function, i.e., it can only define 1 output port. [http://www.vistrails.org/usersguide/controlflow.html#the-map-operator The Map Operator]

== Spreadsheet ==

Where ''pipeline'' is a version number or a tag.

=== How can I save an image from the spreadsheet? ===

While having the focus on a spreadsheet cell and select the camera on the toolbar to take a snapshot. The system will prompt you for the location and file name where it should be saved. The other icons can be used for saving multiple images that can be used for generating an animation on demand. A whole sheet can also be saved by selecting Export (either from the menu or from the toolbar). [http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet-image Saving a Spreadsheet Image]

===Is it possible to save the complete state of the spreadsheet?===
[http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet Saving a Spreadsheet]

=== Can I view multiple sheets at the same time? ===

Yes. Each sheet on the spreadsheet can be displayed as a dock widget separated from the main spreadsheet window by dragging its tab name out of the tab bar at the bottom of the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Then, how can I put back a separated sheet? ===

A sheet can be docked back to the main window by dragging it back to the tab bar or double-click on its title bar. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== How can I order sheets on the spreadsheet? ===

This can be done by dragging the sheet name on the bottom top bar and drop it to the right place. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Can I control where a cell will be placed on the spreadsheet window? ===

By default, an unoccupied cell on the active sheet will be chosen to display the result. However, you can specify exactly in the pipeline where a spreadsheet cell will be placed by using CellLocation and SheetReference. CellLocation specifies the location (row and column) of a cell when connecting to a spreadsheet cell (VTKCell, ImageViewerCell, ...). Similarly, a SheetReference module (when connecting to a CellLocation) will specify which sheet the cell will be put on given its name, minimum row size and minimum column size. There is an example of this in examples/vtk.xml (select the version below Double Renderer). [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

=== How do I output results to the spreadsheet? ===

By inspecting the VisTrails Spreadsheet package (in the list of packages, to the left of the pipeline builder), you
can see there are built-in cells for different kinds of data, e.g., RichTextCell to display HTML and plain text. op
You (the user) can also define new cell types to display application-specific data. For example, we have developed
VtkCell, MplFigureCell, and OpenGLCell. It is possible to display pretty much anything on the Spreadsheet! [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

Examples of writing cell modules can be found in:
RichTextCell: packages/spreadsheet/widgets/richtext/richtext.py
VTK: packages/vtk/vtkcell.py

Here is the summary of some requirements on a cell widget:

(1) It must be a Qt widget. It should inherit from spreadsheet_cell.QCellWidget in the spreadsheet package. Although any Qt Widget would work, certain features such as animation will not be available (without rewriting it).

(2) It must re-implement the updateContents() function to take a set of inputs (usually coming from input ports of a wrapper Module) and display on the cells. VisTrails uses this function to update/reuse cells on the spreadsheet when new data comes in.

(3) It needs a wrapper VisTrails Module (inherited from basic_widgets.SpreadsheetCell of the spreadsheet package). Inside the compute() method of this module, it may call self.display(CellWidgetType, (inputs)) to trigger the display event on the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#advanced-cell-options Advanced Cell Options]

=== How do I control the default number of cells in the spreadsheet? ===

You can configure the rowCount and colCount using the preferences dialog. Just go to the Module Packages tab, select spreadsheet in the "Enabled packages" and press the Configure button. Then a list of all the configuration options for the spreadsheet will show up. [http://www.vistrails.org/usersguide/spreadsheet.html#custom-layout-options Custom Layout Options]

=== Is it possible to launch a web browser from the vistrails spreadsheet? We would like to output several urls from a parameter sweep and then have the option to click on each one to view the resulting page. I can view the page within the spreadsheet, but it is really too crowded.===

Currently, there isn't a widget that provides exactly this functionality, but I can think of a few solutions that may work for you:

(1) You can use parameter exploration to generate multiple sheets so you might have an exploration that opens each page in a new sheet. Use the third column/dimension in the exploration interface to have a parameter span sheets.

(2) The spreadsheet is extensible so you can write a custom spreadsheet cell widget that has a button or label with the desired link (a QLabel with openExternalLinks set to True, for example).

(3) You can tweak the existing RichTextCell be adding the line "self.browser.setOpenExternalLinks(True)" at line 63 of the source file "vistrails/packages/spreadsheet/widgets/richtext/richtext.py". Then, if your workflow creates a file with html markup text like "<a href="http://www.vistrails.org">VisTrails</a>" connected to a RichTextCell, clicking on the rendered link in the cell will open it in a web browser. You need to add the aforementioned line to the source to let Qt know that you want the link opened externally; by default, it will just issue an event that isn't processed. [http://www.vistrails.org/usersguide/spreadsheet.html#launching-a-web-browser Launching a Web Browser]

== Integrating your software into VisTrails ==

===How can I integrate my own program into VisTrails?===

The easiest way is to create a package. Writing a package is often very simple, here are instructions on how to do it: [[UsersGuideVisTrailsPackages]]

You can also dynamically generate modules. For an example see:

http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#Packages_that_generate_modules_dynamically

In particular, see the new_module call which uses python's type() function to generate new classes dynamically.

=== How do modules deal with multiple inputs in a same port? ===

(And should that even be allowed?)

For compatibility reasons, we do need to allow multiple connections to an input port. However, most package developers should never have to use this, and so we do our best to hide it. the default behavior for getting inputs from a port, then, is to always return a single input.

If on your module you need multiple inputs connected to a single port, use the 'forceGetInputListFromPort' method. It will return a list of all the data items coming through the port. The VTK package uses this feature, so look there for usage examples (packages/vtk/base_widget.py)

=== Are there mechanisms for attaching widgets to different modules/parameters? ===

Right now, we have a mechanism for putting a specific widget for an input port. For example, if a port is SetColor(red, green, blue), we can put a color wheel widget there. Or we can also replace the SetFileName port with a File Widget. However, this is not per parameter (only per port). We are currently working on this problem.

=== Can I organize my package so it appears hierarchical in the module palette? ===

Yes. Use the namespace keyword argument when adding the module to the registry. For example,

registry.add_module(MyModule, namespace='MyNamespace')

=== Can I nest namespaces? ===

Yes. Use the '|' character to separate different the hierarchy. For example,

registry.add_module(MyModule, namespace='ParentNamespace|ChildNamespace')

=== Are there shortcuts for registry initialization? ===

Yes. If you define _modules as a list of classes in the __init__.py file of your package, VisTrails will attempt to load all classes specified as modules. You can provide add_module options as keyword arguments by specifying a tuple (class, kwargs) in the list. For example:

_modules = [MyModule1, (MyModule2, {'namespace': 'MyNamespace'})]

In addition, you need to identify the ports of your modules as a field in your class by defining _input_ports and _output_ports lists. Here, the items in each list must be tuples of the form (portName, portSignature, optional=False, sort_key=-1). For example:

class MyModule(Module):
def compute(self):
pass

_input_ports = [('firstInput', String), ('secondInput', Integer, True)]
_output_ports = [('firstOutput', String), ('secondOutput', String)]

=== Can I define ports to be of types that I do not import into my package? ===

Yes. You can pass an identifier string as the portSignature instead. The port_signature string is defined by:

<module_string> := <package_identifier>:[<namespace>|]<module_name>,
<port_signature> := (<module_string>*)

For example,

registry.add_input_port(MyModule, 'myInputPort', '(edu.utah.sci.vistrails.basic:String)')

or

_input_ports = [('myInputPort', '(edu.utah.sci.vistrails.basic:String)')]

=== What do I need to change in my package to make it reloadable (new in v1.4.2)? ===
See [http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#How_to_make_you_package_reloadable UsersGuideVisTrailsPackages] for an explanation.

=== Can I add default values or labels for parameters? ===

Yes. Versions 1.4 and greater support these features. See [[UsersGuideVisTrailsPackages#Adding_default_values_and.2For_labels_for_parameters | UsersGuideVisTrailsPacakges]] for more details.

=== I want to write a module to load HDF data whose output (e.g., data, string) varies according to the input I give it. Is is possible to do this in VisTrails, and if yes, how can I do that? Ideally, I would like to avoid having to change the connection of my output every time I change the input. ===

There are a few ways to tackle this - each has it's own benefits and pitfalls. Firstly, module connections do respect class hierarchies as we're familiar with in object oriented languages. For instance, A module can output a Constant of which String, Float, Integer, etc are specifications. In this way, you can have a subclass of something like HDFData be passed out of the module and the connections will be established regardless of the sub-type. This is a bit dangerous though. Modules downstream of such a class may not really know how to operate on certain types derived from the super-class. Extreme care must be taken both when creating the modules as well as connecting them to prevent things like this from happening.

A second method that I employ in several different packages is the idea of a container class. For instance, the NumSciPy package uses a relatively generic container "Numpy Array" to encapsulate the data. Of course, these encapsulating objects can store dictionaries that other modules can easily access and understand how to operate on. Although this method is slightly more work, the benefits of a stricter typing of ports is beneficial - particularly upon interfacing with other packages that may depend on strongly typed constants (for example).

=== I need to determine, at run-time, whether or not a "child" module is attached to the output port of a "parent" module. (I do not specifically need to know which child; just if there is one). ===

The outputPorts dictionary of the base Module stores this information. Thus, you should be able to check

("myPortName" in self.outputPorts)

on the parent module to check if there are any downstream connections from the port "myPortName". This might be used, for example, to only set results for output ports that will be used. ***Note***, however, that the caching algorithm assumes that all outputs are set so adding a new connection to a previously unconnected output port will not work as desired if that module is cached. For this reason, I would currently recommend making such a module not cacheable. Another possibility is overriding the update() method to check the outputPorts and set the upToDate flag if they are not equal. In a single, limited test, this seemed to work, but be warned that it is not fully tested. Here is an example:

class TestModule(Module):
_output_ports = [('a1', '(edu.utah.sci.vistrails.basic:String)'),
('a2', '(edu.utah.sci.vistrails.basic:String)')]
def __init__(self):
Module.__init__(self)
self._cached_output_ports = set()

def update(self):
if len(set(self.outputPorts) - self._cached_output_ports) > 0:
self.upToDate = False
Module.update(self)

def compute(self):
if "a1" in self.outputPorts:
self.setResult("a1", "test")
if "a2" in self.outputPorts:
self.setResult("a2", "test2")
self._cached_output_ports = set(self.outputPorts)

=== How can I make a module not display in the modules list? ===

You should set the abstract parameter to True when adding the module to the registry. Using the original syntax, this looks like:

def initialize():
reg = core.modules.module_registry.get_module_registry()
reg.add_module(InvisibleModule, abstract=True)
# ...

With the _modules dictionary shortcut (for more details, see [http://www.vistrails.org/index.php/FAQ#Are_there_shortucts_for_registry_initialization.3F the FAQ section on this]), you include it in a kwargs dict as part of a module tuple:

_modules = [AnotherModule, (InvisibleModule, {'abstract': True})]

There is also a 'hide_descriptor' parameter that prevents the module from appearing in the module palette without declaring it to be abstract.

The technical difference between the two is that 'abstract' will not add the item to the module palette while 'hide_descriptor' does add the item but immediately hides it. If the module should never be instantiated in a workflow, declare it abstract. If you don't want users to be able to add the module to a pipeline, but you have code that may add it programmatically, declare it with hide_descriptor=True.

== The Console ==

=== Where should I go to find out what I can call from the console and how to import it? ===

We have tried to make some methods more accessible in the console via an api. You can import the api via <code>import api</code> in the console and see the available methods with <code>dir(api)</code>. To open a vistrail:

import api
api.open_vistrail_from_file('/Applications/VisTrails/examples/terminator.vt')

To execute a version of a workflow, you currently have to go through the controller:

api.select_version('Histogram')
api.get_current_controller().execute_current_workflow()

Currently, only a subset of VisTrails functionality is directly available from the api. However, since VisTrails is written in python, you can dig down starting with the VistrailsApplication or controller object to expose most of our internal methods. If you have suggestions for calls to be added to the api, please let us know.

One other feature that we're working on, but is still in progress is the ability to construct workflows via the console. For example:

vtk = load_package('edu.utah.sci.vistrails.vtk')
vtk.vtkDataSetReader() # adds a vtkDataSetReader module to the pipeline
# click on the new module
a = selected_modules()[0] # get the one currently selected module
a.SetFile('/vistrails/examples/data/head120.vtk') # sets the SetFile parmaeter for the data set reader
b = vtk.vtkContourFilter() # adds a vtkContourFilter module to the pipeline and saves to var b
b.SetInputConnection0(a.GetOutputPort0()) # connects a's GetOutputPort0 port to b's SetInputConnection0

== Persistence Package ==

=== How do I use the output of one workflow as the input for another using the persistence package? ===

You need to configure the persistence modules using the module's configuration dialog. After adding a <code>PersistentOutputFile</code> to the workflow, click on the triangle in the upper-right corner of the <code>PersistentOutputFile</code>, and select "Edit Configuration" from the menu that appears. In this dialog, select "Create New Reference" and give the reference a name (and any space-delimited tags). Upon running that workflow, the data will be written to the persistent store. In the second workflow where you wish to use that file, add a <code>PersistentInputFile</code> and go to its configuration dialog in the same manner as with the output file. In that dialog, select "Use Existing Reference" and select the data that you just added in the first workflow in the list of files below. Now, when you run that workflow, it will grab the data from the persistent store.

Here is an example: [[Media:offscreen_persistent.vt | offscreen_persistent.vt]]. Run the "persistent offscreen" workflow first, and then run the "display persistent output" to use the output of the first workflow as the input for the second.

== VTK ==

=== Given a VTK visualization, how can I generate a webpage from it? ===

Check out the html pipeline in offscreen.xml.

=== I'm trying to use VTK, but there doesn't seem to be any output. What is wrong? ===

To use VTK on VisTrails, you need a slightly different way of connecting the renderer modules. Instead of using the standard RenderWindow/RenderWindowInteractor infrastructure, you simply connect the renderer to a VTKCell. The examples directory in the distribution has several VTK examples that illustrate.

=== I am trying to add a module to the workflow via Python, but how can I access vtk modules? ===

Here's an example:

import api

vtvtk = 'edu.utah.sci.vistrails.vtk'

module = api.add_module(0, 0, vtvtk, 'vtkContourFilter', '')

The third argument in add_module is the package identifier. You can find this in the "Module Packages" panel of the Preferences; just click on the package you're interested in and it will appear in the information on the right.

== matplotlib ==

=== I'm experiencing a problem with Latex labels and the matplotlib that comes with VisTrails 1.5. The script below entered to the interpreter that comes with VT is sufficient to reproduce it. ===

import matplotlib.pyplot as plt
plt.plot([1,2,3],[1,2,3])
plt.xlabel("$foo$")

Remove your ~/.matplotlib folder and re-start VisTrails

== VisTrails Development ==

=== I would like to build VisTrails from source. Are there instructions on how to do this? ===

Yes! Take a look at http://www.vistrails.org/index.php/Mac_Intel_Instructions
[http://www.vistrails.org/usersguide/getting_started.html#installing-vistrails-from-source Installing VisTrails from source]

== Accessing Provenance Information ==

=== How do I access the information in the execution log? ===

The code responsible for storing execution information is located in the "core/log" directories, and the code that generates much of that information is in "core/interpreter/cached.py". Modules can add execution-specific annotations to provenance via annotate() calls during execution, but much of the data (like timing and errors) is captured by the LogController and CachedInterpreter (the execution engine) objects. To analyze the log from a vistrail (.vt) file, you might have something like the following:

<code>
import core.log.log
import db.services.io

def run(fname):
# open the .vt bundle specified by the filename "fname"
bundle = db.services.io.open_vistrail_bundle_from_zip_xml(fname)[0]
# get the log filename
log_fname = bundle.vistrail.db_log_filename
if log_fname is not None:
# open the log
log = db.services.io.open_log_from_xml(log_fname, True)
# convert the log from a db object
core.log.log.Log.convert(log)
for workflow_exec in log.workflow_execs:
print 'workflow version:', workflow_exec.parent_version
print 'time started:', workflow_exec.ts_start
print 'time ended:', workflow_exec.ts_end
print 'modules executed:', [i.module_id
for i in workflow_exec.item_execs]
if __name__ == '__main__':
run("some_vistrail.vt")
</code>

You should be able to see what information is available by looking at the "core/log" classes.

FAQ

2011-03-26T18:27:59Z

Ctuttle: /* How can I run a workflow using the command line? */

Also check our [[known_issues|Known Issues]] page for troubleshooting.

----
== Running workflows ==

=== How can I run a workflow using the command line? ===

('''Updated for version 1.2''') Call vistrails using the following options:

<code>python vistrails.py -b path_to_vistrails_file:pipeline </code>

where pipeline can be a version '''tag name''' or version '''id'''

'''NOTE:''' If you downloaded the MacOS X bundle, you can run vistrails from the command line via the following commands in the Terminal. Change the current directory to wherever VisTrails was installed (often <code>/Applications</code>), and then type:

<code>Vistrails.app/Contents/MacOS/vistrails [<cmd_line_options>]</code> [http://www.vistrails.org/usersguide/batch.html#running-a-specific-workflow-in-batch-mode Running a Specific Workflow in Batch Mode]

=== Using the command line, we'd like to execute a workflow multiple times, with slightly different parameters, and create a series of output files. Is this possible?===

('''Updated for version 1.2''') We can change parameters that have an alias through the command line.

For example, offscreen pipeline in offscreen.vt always creates the file called image.png. If you want generate it with a different filename:

<code>python vistrails.py -b ../examples/offscreen.vt:offscreen -a"filename=other.png"</code>

<code>filename</code> in the example above is the alias name assigned to the parameter in the <code>value</code> method inside the String module. When running a pipeline from the command line, VisTrails will try to start the spreadsheet automatically if the pipeline requires it. For example, this other execution will also start the spreadsheet (attention to how $ characters are escaped when running on bash):

<code>python vistrails.py -b ../examples/head.vt:aliases -a"isovalue=30\$&\$diffuse_color=0.8,0.4,0.2"</code>

You can also execute more than one pipeline on the command line:

<code>python vistrails.py -b ../examples/head.vt:aliases ../examples/spx.vt:spx \
-a"isovalue=30"</code>

Use the -a parameter only once regardless the number of pipelines. [http://www.vistrails.org/batch.html#running-a-workflow-with-specific-parameters Running a Workflow with Specific Parameters]

=== I can load a vistrail, and the version tree shows up fine. However, no pipelines appear when I click on a version. What gives? ===

The most likely reason is that the vistrail uses a package that is not registered with VisTrails. You need to identify the needed package and add it to your <code>.vistrails/startup.py</code>. A single line like the following should be enough:

addPackage('enter_package_name_here')

Some packages might need more information. For example:

addPackage('afront', executable_path='/path/to/afront')

Refer to the package documentation for details. The one inconvenient step is that currently there's no automated way to describe what is the missing package. We're working on this feature for future releases.

=== I have a workflow that reads a file and then does some processing. The first time it runs, it executes correctly. But in subsequent, nothing happens. ===

VisTrails caches by default, so after a workflow is executed, if none of its parameters change, it won't be executed again.

If a workflow reads a file using the basic module File, VisTrails does check whether the file was modified since the last run. It does so by keeping a signature that is based on the modification time of the file. And if the file was modified, the File module and all downstream modules (the ones which depend on File) will be executed.

''Note: If you would like your input and output data to be versioned, you can use the Persistence package.''

If you do not want VisTrails to cache executions, you can turn off caching: go to Menu Edit -> Preferences and in the General Configuration tab, change Cache execution results to Never. [http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

=== Can VisTrails execute workflows in parallel? ===

The VisTrails server can only execute pipelines in parallel if there's more than one instance of VisTrails running. The command

<code> self.rpcserver = ThreadedXMLRPCServer((self.temp_xml_rpc_options.server, self.temp_xml_rpc_options.port))</code>

starts a multithreaded version of the XML-RPC server, so it will create a thread for each request received by the server. The problem is that Qt/PyQT doesn't allow these multiple threads create GUI objects, only in the main thread. To overcome this limitation, the multithreaded version can instantiate other single threaded versions of VisTrails and put them in a queue, so workflow executions and other GUI-related requests, such as generating workflow graphs and history trees can be forwarded to this queue, and each instance takes turns in answering the request. If the results are in the cache, the multithreaded version answers the requests directly.

Note that this infrastructure works on Linux only. To make this work on Windows, you have to create a script similar to start_vistrails_xvfb.sh (located in the scripts folder) where you can send the number of other instances via command-line options to VisTrails. The command line options are:

python vistrails_server.py -T <ADDRESS> -R <PORT> -O<NUMBER_OF_OTHER_VISTRAILS_INSTANCES> [-M]&

If you want the main vistrails instance to be multithreaded, use the -M at the end.

After creating this script, update function start_other_instances in vistrails/gui/application_server.py lines 1007-1023 and set the script variable to point to your script. You may also have to change the arguments sent to your script (line 1016: for example, you don't need to set a virtual display). You will need to change the path to the stop_vistrails_server.py script (on line 1026) according to your installation path. [http://www.vistrails.org/usersguide/batch.html#executing-workflows-in-parallel Executing Workflows in Parallel]

=== When a workflow is executed, what do the colors mean? ===

- lilac: module was notexecuted

- yellow: module is currently being executed

- green: module was successfully executed

- orange: module was cached

- red: the execution of the module failed

[http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

== Building workflows ==

=== Is there a way to give each widget a "display name" in addition to the module name at the center of the widget? ===

Yes, a "display name" can be assigned to a module by selecting the triangle in its top right corner to open a popup menu and selecting the Set Module Label... menu item. You will then be prompted to enter the "display name". [http://www.vistrails.org/usersguide/creating.html#changing-module-labels Changing Module Labels]

=== Is there a way to re-center the picture-in-picture (PiP) view? ===

Yes. If you click on the PIP window to bring it to focus, you can press Ctrl-R (or Command-R on Mac) to re-center the PiP window. [http://www.vistrails.org/usersguide/getting_started.html#vistrails-interaction Vistrails Interaction]

=== How do I search for a literal "?" (question mark) in the search box in the Property panel? ===

Since we allow regular expressions in our search box, question marks are treated as meta-characters. Thus, searching for "?" returns everything and "abc?" will return everything containing "abc". You need to use "\?" instead to search for "?". So the search for "??" would be "\?\?". [http://www.vistrails.org/usersguide/querying.html#textual-queries Textual Queries]

== Using VisTrails as a server ==

=== What is the VisTrails server-mode? ===

Using the VisTrails server mode, it is possible to ''execute workflows and control VisTrails through another application''. For example, the CrowdLabs Web portal (http://www.crowdlabs.org) accesses a VisTrails sever to execute workflows, retrieve and display vistrail trees and workflows. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

=== How do I execute workflows and control VisTrails through another application? ===

The way you access the server is by doing XML-RPC calls. In the current VisTrails release, we include a set of PHP scripts that can talk to a VisTrails server instance. They are in "extensions/http" folder. The files are reasonably well documented. Also, it should be not difficult to create python scripts to access the server (just use xmlrpclib module).

Note that the VisTrails server requires the provenance and workflows to be in a database. More detailed instructions on how to setup the server and the database
are available here:

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_server_setup.html

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_database_setup.html

If what you want is just to execute a series of workflows in batch mode, a simpler solution would be to ''use the VisTrails client in batch mode''. Chapter 12 of the user's guide contains detailed information and examples on that. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

== Control Flow ==

=== Note: using map ===

When using 'map', the module (or subworkflow) used as function port in the map module MUST be a function, i.e., it can only define 1 output port. [http://www.vistrails.org/usersguide/controlflow.html#the-map-operator The Map Operator]

== Spreadsheet ==

Where ''pipeline'' is a version number or a tag.

=== How can I save an image from the spreadsheet? ===

While having the focus on a spreadsheet cell and select the camera on the toolbar to take a snapshot. The system will prompt you for the location and file name where it should be saved. The other icons can be used for saving multiple images that can be used for generating an animation on demand. A whole sheet can also be saved by selecting Export (either from the menu or from the toolbar). [http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet-image Saving a Spreadsheet Image]

===Is it possible to save the complete state of the spreadsheet?===
[http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet Saving a Spreadsheet]

=== Can I view multiple sheets at the same time? ===

Yes. Each sheet on the spreadsheet can be displayed as a dock widget separated from the main spreadsheet window by dragging its tab name out of the tab bar at the bottom of the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Then, how can I put back a separated sheet? ===

A sheet can be docked back to the main window by dragging it back to the tab bar or double-click on its title bar. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== How can I order sheets on the spreadsheet? ===

This can be done by dragging the sheet name on the bottom top bar and drop it to the right place. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Can I control where a cell will be placed on the spreadsheet window? ===

By default, an unoccupied cell on the active sheet will be chosen to display the result. However, you can specify exactly in the pipeline where a spreadsheet cell will be placed by using CellLocation and SheetReference. CellLocation specifies the location (row and column) of a cell when connecting to a spreadsheet cell (VTKCell, ImageViewerCell, ...). Similarly, a SheetReference module (when connecting to a CellLocation) will specify which sheet the cell will be put on given its name, minimum row size and minimum column size. There is an example of this in examples/vtk.xml (select the version below Double Renderer). [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

=== How do I output results to the spreadsheet? ===

By inspecting the VisTrails Spreadsheet package (in the list of packages, to the left of the pipeline builder), you
can see there are built-in cells for different kinds of data, e.g., RichTextCell to display HTML and plain text. op
You (the user) can also define new cell types to display application-specific data. For example, we have developed
VtkCell, MplFigureCell, and OpenGLCell. It is possible to display pretty much anything on the Spreadsheet! [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

Examples of writing cell modules can be found in:
RichTextCell: packages/spreadsheet/widgets/richtext/richtext.py
VTK: packages/vtk/vtkcell.py

Here is the summary of some requirements on a cell widget:

(1) It must be a Qt widget. It should inherit from spreadsheet_cell.QCellWidget in the spreadsheet package. Although any Qt Widget would work, certain features such as animation will not be available (without rewriting it).

(2) It must re-implement the updateContents() function to take a set of inputs (usually coming from input ports of a wrapper Module) and display on the cells. VisTrails uses this function to update/reuse cells on the spreadsheet when new data comes in.

(3) It needs a wrapper VisTrails Module (inherited from basic_widgets.SpreadsheetCell of the spreadsheet package). Inside the compute() method of this module, it may call self.display(CellWidgetType, (inputs)) to trigger the display event on the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#advanced-cell-options Advanced Cell Options]

=== How do I control the default number of cells in the spreadsheet? ===

You can configure the rowCount and colCount using the preferences dialog. Just go to the Module Packages tab, select spreadsheet in the "Enabled packages" and press the Configure button. Then a list of all the configuration options for the spreadsheet will show up. [http://www.vistrails.org/usersguide/spreadsheet.html#custom-layout-options Custom Layout Options]

=== Is it possible to launch a web browser from the vistrails spreadsheet? We would like to output several urls from a parameter sweep and then have the option to click on each one to view the resulting page. I can view the page within the spreadsheet, but it is really too crowded.===

Currently, there isn't a widget that provides exactly this functionality, but I can think of a few solutions that may work for you:

(1) You can use parameter exploration to generate multiple sheets so you might have an exploration that opens each page in a new sheet. Use the third column/dimension in the exploration interface to have a parameter span sheets.

(2) The spreadsheet is extensible so you can write a custom spreadsheet cell widget that has a button or label with the desired link (a QLabel with openExternalLinks set to True, for example).

(3) You can tweak the existing RichTextCell be adding the line "self.browser.setOpenExternalLinks(True)" at line 63 of the source file "vistrails/packages/spreadsheet/widgets/richtext/richtext.py". Then, if your workflow creates a file with html markup text like "<a href="http://www.vistrails.org">VisTrails</a>" connected to a RichTextCell, clicking on the rendered link in the cell will open it in a web browser. You need to add the aforementioned line to the source to let Qt know that you want the link opened externally; by default, it will just issue an event that isn't processed. [http://www.vistrails.org/usersguide/spreadsheet.html#launching-a-web-browser Launching a Web Browser]

== Integrating your software into VisTrails ==

===How can I integrate my own program into VisTrails?===

The easiest way is to create a package. Writing a package is often very simple, here are instructions on how to do it: [[UsersGuideVisTrailsPackages]]

You can also dynamically generate modules. For an example see:

http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#Packages_that_generate_modules_dynamically

In particular, see the new_module call which uses python's type() function to generate new classes dynamically.

=== How do modules deal with multiple inputs in a same port? ===

(And should that even be allowed?)

For compatibility reasons, we do need to allow multiple connections to an input port. However, most package developers should never have to use this, and so we do our best to hide it. the default behavior for getting inputs from a port, then, is to always return a single input.

If on your module you need multiple inputs connected to a single port, use the 'forceGetInputListFromPort' method. It will return a list of all the data items coming through the port. The VTK package uses this feature, so look there for usage examples (packages/vtk/base_widget.py)

=== Are there mechanisms for attaching widgets to different modules/parameters? ===

Right now, we have a mechanism for putting a specific widget for an input port. For example, if a port is SetColor(red, green, blue), we can put a color wheel widget there. Or we can also replace the SetFileName port with a File Widget. However, this is not per parameter (only per port). We are currently working on this problem.

=== Can I organize my package so it appears hierarchical in the module palette? ===

Yes. Use the namespace keyword argument when adding the module to the registry. For example,

registry.add_module(MyModule, namespace='MyNamespace')

=== Can I nest namespaces? ===

Yes. Use the '|' character to separate different the hierarchy. For example,

registry.add_module(MyModule, namespace='ParentNamespace|ChildNamespace')

=== Are there shortcuts for registry initialization? ===

Yes. If you define _modules as a list of classes in the __init__.py file of your package, VisTrails will attempt to load all classes specified as modules. You can provide add_module options as keyword arguments by specifying a tuple (class, kwargs) in the list. For example:

_modules = [MyModule1, (MyModule2, {'namespace': 'MyNamespace'})]

In addition, you need to identify the ports of your modules as a field in your class by defining _input_ports and _output_ports lists. Here, the items in each list must be tuples of the form (portName, portSignature, optional=False, sort_key=-1). For example:

class MyModule(Module):
def compute(self):
pass

_input_ports = [('firstInput', String), ('secondInput', Integer, True)]
_output_ports = [('firstOutput', String), ('secondOutput', String)]

=== Can I define ports to be of types that I do not import into my package? ===

Yes. You can pass an identifier string as the portSignature instead. The port_signature string is defined by:

<module_string> := <package_identifier>:[<namespace>|]<module_name>,
<port_signature> := (<module_string>*)

For example,

registry.add_input_port(MyModule, 'myInputPort', '(edu.utah.sci.vistrails.basic:String)')

or

_input_ports = [('myInputPort', '(edu.utah.sci.vistrails.basic:String)')]

=== What do I need to change in my package to make it reloadable (new in v1.4.2)? ===
See [http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#How_to_make_you_package_reloadable UsersGuideVisTrailsPackages] for an explanation.

=== Can I add default values or labels for parameters? ===

Yes. Versions 1.4 and greater support these features. See [[UsersGuideVisTrailsPackages#Adding_default_values_and.2For_labels_for_parameters | UsersGuideVisTrailsPacakges]] for more details.

=== I want to write a module to load HDF data whose output (e.g., data, string) varies according to the input I give it. Is is possible to do this in VisTrails, and if yes, how can I do that? Ideally, I would like to avoid having to change the connection of my output every time I change the input. ===

There are a few ways to tackle this - each has it's own benefits and pitfalls. Firstly, module connections do respect class hierarchies as we're familiar with in object oriented languages. For instance, A module can output a Constant of which String, Float, Integer, etc are specifications. In this way, you can have a subclass of something like HDFData be passed out of the module and the connections will be established regardless of the sub-type. This is a bit dangerous though. Modules downstream of such a class may not really know how to operate on certain types derived from the super-class. Extreme care must be taken both when creating the modules as well as connecting them to prevent things like this from happening.

A second method that I employ in several different packages is the idea of a container class. For instance, the NumSciPy package uses a relatively generic container "Numpy Array" to encapsulate the data. Of course, these encapsulating objects can store dictionaries that other modules can easily access and understand how to operate on. Although this method is slightly more work, the benefits of a stricter typing of ports is beneficial - particularly upon interfacing with other packages that may depend on strongly typed constants (for example).

=== I need to determine, at run-time, whether or not a "child" module is attached to the output port of a "parent" module. (I do not specifically need to know which child; just if there is one). ===

The outputPorts dictionary of the base Module stores this information. Thus, you should be able to check

("myPortName" in self.outputPorts)

on the parent module to check if there are any downstream connections from the port "myPortName". This might be used, for example, to only set results for output ports that will be used. ***Note***, however, that the caching algorithm assumes that all outputs are set so adding a new connection to a previously unconnected output port will not work as desired if that module is cached. For this reason, I would currently recommend making such a module not cacheable. Another possibility is overriding the update() method to check the outputPorts and set the upToDate flag if they are not equal. In a single, limited test, this seemed to work, but be warned that it is not fully tested. Here is an example:

class TestModule(Module):
_output_ports = [('a1', '(edu.utah.sci.vistrails.basic:String)'),
('a2', '(edu.utah.sci.vistrails.basic:String)')]
def __init__(self):
Module.__init__(self)
self._cached_output_ports = set()

def update(self):
if len(set(self.outputPorts) - self._cached_output_ports) > 0:
self.upToDate = False
Module.update(self)

def compute(self):
if "a1" in self.outputPorts:
self.setResult("a1", "test")
if "a2" in self.outputPorts:
self.setResult("a2", "test2")
self._cached_output_ports = set(self.outputPorts)

=== How can I make a module not display in the modules list? ===

You should set the abstract parameter to True when adding the module to the registry. Using the original syntax, this looks like:

def initialize():
reg = core.modules.module_registry.get_module_registry()
reg.add_module(InvisibleModule, abstract=True)
# ...

With the _modules dictionary shortcut (for more details, see [http://www.vistrails.org/index.php/FAQ#Are_there_shortucts_for_registry_initialization.3F the FAQ section on this]), you include it in a kwargs dict as part of a module tuple:

_modules = [AnotherModule, (InvisibleModule, {'abstract': True})]

There is also a 'hide_descriptor' parameter that prevents the module from appearing in the module palette without declaring it to be abstract.

The technical difference between the two is that 'abstract' will not add the item to the module palette while 'hide_descriptor' does add the item but immediately hides it. If the module should never be instantiated in a workflow, declare it abstract. If you don't want users to be able to add the module to a pipeline, but you have code that may add it programmatically, declare it with hide_descriptor=True.

== The Console ==

=== Where should I go to find out what I can call from the console and how to import it? ===

We have tried to make some methods more accessible in the console via an api. You can import the api via <code>import api</code> in the console and see the available methods with <code>dir(api)</code>. To open a vistrail:

import api
api.open_vistrail_from_file('/Applications/VisTrails/examples/terminator.vt')

To execute a version of a workflow, you currently have to go through the controller:

api.select_version('Histogram')
api.get_current_controller().execute_current_workflow()

Currently, only a subset of VisTrails functionality is directly available from the api. However, since VisTrails is written in python, you can dig down starting with the VistrailsApplication or controller object to expose most of our internal methods. If you have suggestions for calls to be added to the api, please let us know.

One other feature that we're working on, but is still in progress is the ability to construct workflows via the console. For example:

vtk = load_package('edu.utah.sci.vistrails.vtk')
vtk.vtkDataSetReader() # adds a vtkDataSetReader module to the pipeline
# click on the new module
a = selected_modules()[0] # get the one currently selected module
a.SetFile('/vistrails/examples/data/head120.vtk') # sets the SetFile parmaeter for the data set reader
b = vtk.vtkContourFilter() # adds a vtkContourFilter module to the pipeline and saves to var b
b.SetInputConnection0(a.GetOutputPort0()) # connects a's GetOutputPort0 port to b's SetInputConnection0

== Persistence Package ==

=== How do I use the output of one workflow as the input for another using the persistence package? ===

You need to configure the persistence modules using the module's configuration dialog. After adding a <code>PersistentOutputFile</code> to the workflow, click on the triangle in the upper-right corner of the <code>PersistentOutputFile</code>, and select "Edit Configuration" from the menu that appears. In this dialog, select "Create New Reference" and give the reference a name (and any space-delimited tags). Upon running that workflow, the data will be written to the persistent store. In the second workflow where you wish to use that file, add a <code>PersistentInputFile</code> and go to its configuration dialog in the same manner as with the output file. In that dialog, select "Use Existing Reference" and select the data that you just added in the first workflow in the list of files below. Now, when you run that workflow, it will grab the data from the persistent store.

Here is an example: [[Media:offscreen_persistent.vt | offscreen_persistent.vt]]. Run the "persistent offscreen" workflow first, and then run the "display persistent output" to use the output of the first workflow as the input for the second.

== VTK ==

=== Given a VTK visualization, how can I generate a webpage from it? ===

Check out the html pipeline in offscreen.xml.

=== I'm trying to use VTK, but there doesn't seem to be any output. What is wrong? ===

To use VTK on VisTrails, you need a slightly different way of connecting the renderer modules. Instead of using the standard RenderWindow/RenderWindowInteractor infrastructure, you simply connect the renderer to a VTKCell. The examples directory in the distribution has several VTK examples that illustrate.

=== I am trying to add a module to the workflow via Python, but how can I access vtk modules? ===

Here's an example:

import api

vtvtk = 'edu.utah.sci.vistrails.vtk'

module = api.add_module(0, 0, vtvtk, 'vtkContourFilter', '')

The third argument in add_module is the package identifier. You can find this in the "Module Packages" panel of the Preferences; just click on the package you're interested in and it will appear in the information on the right.

== matplotlib ==

=== I'm experiencing a problem with Latex labels and the matplotlib that comes with VisTrails 1.5. The script below entered to the interpreter that comes with VT is sufficient to reproduce it. ===

import matplotlib.pyplot as plt
plt.plot([1,2,3],[1,2,3])
plt.xlabel("$foo$")

Remove your ~/.matplotlib folder and re-start VisTrails

== VisTrails Development ==

=== I would like to build VisTrails from source. Are there instructions on how to do this? ===

Yes! Take a look at http://www.vistrails.org/index.php/Mac_Intel_Instructions
[http://www.vistrails.org/usersguide/getting_started.html#installing-vistrails-from-source Installing VisTrails from source]

== Accessing Provenance Information ==

=== How do I access the information in the execution log? ===

The code responsible for storing execution information is located in the "core/log" directories, and the code that generates much of that information is in "core/interpreter/cached.py". Modules can add execution-specific annotations to provenance via annotate() calls during execution, but much of the data (like timing and errors) is captured by the LogController and CachedInterpreter (the execution engine) objects. To analyze the log from a vistrail (.vt) file, you might have something like the following:

<code>
import core.log.log
import db.services.io

def run(fname):
# open the .vt bundle specified by the filename "fname"
bundle = db.services.io.open_vistrail_bundle_from_zip_xml(fname)[0]
# get the log filename
log_fname = bundle.vistrail.db_log_filename
if log_fname is not None:
# open the log
log = db.services.io.open_log_from_xml(log_fname, True)
# convert the log from a db object
core.log.log.Log.convert(log)
for workflow_exec in log.workflow_execs:
print 'workflow version:', workflow_exec.parent_version
print 'time started:', workflow_exec.ts_start
print 'time ended:', workflow_exec.ts_end
print 'modules executed:', [i.module_id
for i in workflow_exec.item_execs]
if __name__ == '__main__':
run("some_vistrail.vt")
</code>

You should be able to see what information is available by looking at the "core/log" classes.

FAQ

2011-03-25T23:40:45Z

Ctuttle: /* Are there shortucts for registry initialization? */

Also check our [[known_issues|Known Issues]] page for troubleshooting.

----
== Running workflows ==

=== How can I run a workflow using the command line? ===

('''Updated for version 1.2''') Call vistrails using the following options:

<code>python vistrails.py -b path_to_vistrails_file:pipeline </code>

where pipeline can be a version '''tag name''' or version '''id'''

'''NOTE:''' If you downloaded the MacOS X bundle, you can run vistrails from the command line via the following commands in the Terminal. Change the current directory to wherever VisTrails was installed (often <code>/Applications</code>), and then type:

<code>Vistrails.app/Contents/MacOS/vistrails [<cmd_line_options>]</code> [http://www.vistrails.org/usersguidebatch.html#running-a-specific-workflow-in-batch-mode Running a Specific Workflow in Batch Mode]

=== Using the command line, we'd like to execute a workflow multiple times, with slightly different parameters, and create a series of output files. Is this possible?===

('''Updated for version 1.2''') We can change parameters that have an alias through the command line.

For example, offscreen pipeline in offscreen.vt always creates the file called image.png. If you want generate it with a different filename:

<code>python vistrails.py -b ../examples/offscreen.vt:offscreen -a"filename=other.png"</code>

<code>filename</code> in the example above is the alias name assigned to the parameter in the <code>value</code> method inside the String module. When running a pipeline from the command line, VisTrails will try to start the spreadsheet automatically if the pipeline requires it. For example, this other execution will also start the spreadsheet (attention to how $ characters are escaped when running on bash):

<code>python vistrails.py -b ../examples/head.vt:aliases -a"isovalue=30\$&\$diffuse_color=0.8,0.4,0.2"</code>

You can also execute more than one pipeline on the command line:

<code>python vistrails.py -b ../examples/head.vt:aliases ../examples/spx.vt:spx \
-a"isovalue=30"</code>

Use the -a parameter only once regardless the number of pipelines. [http://www.vistrails.org/batch.html#running-a-workflow-with-specific-parameters Running a Workflow with Specific Parameters]

=== I can load a vistrail, and the version tree shows up fine. However, no pipelines appear when I click on a version. What gives? ===

The most likely reason is that the vistrail uses a package that is not registered with VisTrails. You need to identify the needed package and add it to your <code>.vistrails/startup.py</code>. A single line like the following should be enough:

addPackage('enter_package_name_here')

Some packages might need more information. For example:

addPackage('afront', executable_path='/path/to/afront')

Refer to the package documentation for details. The one inconvenient step is that currently there's no automated way to describe what is the missing package. We're working on this feature for future releases.

=== I have a workflow that reads a file and then does some processing. The first time it runs, it executes correctly. But in subsequent, nothing happens. ===

VisTrails caches by default, so after a workflow is executed, if none of its parameters change, it won't be executed again.

If a workflow reads a file using the basic module File, VisTrails does check whether the file was modified since the last run. It does so by keeping a signature that is based on the modification time of the file. And if the file was modified, the File module and all downstream modules (the ones which depend on File) will be executed.

''Note: If you would like your input and output data to be versioned, you can use the Persistence package.''

If you do not want VisTrails to cache executions, you can turn off caching: go to Menu Edit -> Preferences and in the General Configuration tab, change Cache execution results to Never. [http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

=== Can VisTrails execute workflows in parallel? ===

The VisTrails server can only execute pipelines in parallel if there's more than one instance of VisTrails running. The command

<code> self.rpcserver = ThreadedXMLRPCServer((self.temp_xml_rpc_options.server, self.temp_xml_rpc_options.port))</code>

starts a multithreaded version of the XML-RPC server, so it will create a thread for each request received by the server. The problem is that Qt/PyQT doesn't allow these multiple threads create GUI objects, only in the main thread. To overcome this limitation, the multithreaded version can instantiate other single threaded versions of VisTrails and put them in a queue, so workflow executions and other GUI-related requests, such as generating workflow graphs and history trees can be forwarded to this queue, and each instance takes turns in answering the request. If the results are in the cache, the multithreaded version answers the requests directly.

Note that this infrastructure works on Linux only. To make this work on Windows, you have to create a script similar to start_vistrails_xvfb.sh (located in the scripts folder) where you can send the number of other instances via command-line options to VisTrails. The command line options are:

python vistrails_server.py -T <ADDRESS> -R <PORT> -O<NUMBER_OF_OTHER_VISTRAILS_INSTANCES> [-M]&

If you want the main vistrails instance to be multithreaded, use the -M at the end.

After creating this script, update function start_other_instances in vistrails/gui/application_server.py lines 1007-1023 and set the script variable to point to your script. You may also have to change the arguments sent to your script (line 1016: for example, you don't need to set a virtual display). You will need to change the path to the stop_vistrails_server.py script (on line 1026) according to your installation path. [http://www.vistrails.org/usersguide/batch.html#executing-workflows-in-parallel Executing Workflows in Parallel]

=== When a workflow is executed, what do the colors mean? ===

- lilac: module was notexecuted

- yellow: module is currently being executed

- green: module was successfully executed

- orange: module was cached

- red: the execution of the module failed

[http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

== Building workflows ==

=== Is there a way to give each widget a "display name" in addition to the module name at the center of the widget? ===

Yes, a "display name" can be assigned to a module by selecting the triangle in its top right corner to open a popup menu and selecting the Set Module Label... menu item. You will then be prompted to enter the "display name". [http://www.vistrails.org/usersguide/creating.html#changing-module-labels Changing Module Labels]

=== Is there a way to re-center the picture-in-picture (PiP) view? ===

Yes. If you click on the PIP window to bring it to focus, you can press Ctrl-R (or Command-R on Mac) to re-center the PiP window. [http://www.vistrails.org/usersguide/getting_started.html#vistrails-interaction Vistrails Interaction]

=== How do I search for a literal "?" (question mark) in the search box in the Property panel? ===

Since we allow regular expressions in our search box, question marks are treated as meta-characters. Thus, searching for "?" returns everything and "abc?" will return everything containing "abc". You need to use "\?" instead to search for "?". So the search for "??" would be "\?\?". [http://www.vistrails.org/usersguide/querying.html#textual-queries Textual Queries]

== Using VisTrails as a server ==

=== What is the VisTrails server-mode? ===

Using the VisTrails server mode, it is possible to ''execute workflows and control VisTrails through another application''. For example, the CrowdLabs Web portal (http://www.crowdlabs.org) accesses a VisTrails sever to execute workflows, retrieve and display vistrail trees and workflows. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

=== How do I execute workflows and control VisTrails through another application? ===

The way you access the server is by doing XML-RPC calls. In the current VisTrails release, we include a set of PHP scripts that can talk to a VisTrails server instance. They are in "extensions/http" folder. The files are reasonably well documented. Also, it should be not difficult to create python scripts to access the server (just use xmlrpclib module).

Note that the VisTrails server requires the provenance and workflows to be in a database. More detailed instructions on how to setup the server and the database
are available here:

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_server_setup.html

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_database_setup.html

If what you want is just to execute a series of workflows in batch mode, a simpler solution would be to ''use the VisTrails client in batch mode''. Chapter 12 of the user's guide contains detailed information and examples on that. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

== Control Flow ==

=== Note: using map ===

When using 'map', the module (or subworkflow) used as function port in the map module MUST be a function, i.e., it can only define 1 output port. [http://www.vistrails.org/usersguide/controlflow.html#the-map-operator The Map Operator]

== Spreadsheet ==

Where ''pipeline'' is a version number or a tag.

=== How can I save an image from the spreadsheet? ===

While having the focus on a spreadsheet cell and select the camera on the toolbar to take a snapshot. The system will prompt you for the location and file name where it should be saved. The other icons can be used for saving multiple images that can be used for generating an animation on demand. A whole sheet can also be saved by selecting Export (either from the menu or from the toolbar). [http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet-image Saving a Spreadsheet Image]

===Is it possible to save the complete state of the spreadsheet?===
[http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet Saving a Spreadsheet]

=== Can I view multiple sheets at the same time? ===

Yes. Each sheet on the spreadsheet can be displayed as a dock widget separated from the main spreadsheet window by dragging its tab name out of the tab bar at the bottom of the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Then, how can I put back a separated sheet? ===

A sheet can be docked back to the main window by dragging it back to the tab bar or double-click on its title bar. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== How can I order sheets on the spreadsheet? ===

This can be done by dragging the sheet name on the bottom top bar and drop it to the right place. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Can I control where a cell will be placed on the spreadsheet window? ===

By default, an unoccupied cell on the active sheet will be chosen to display the result. However, you can specify exactly in the pipeline where a spreadsheet cell will be placed by using CellLocation and SheetReference. CellLocation specifies the location (row and column) of a cell when connecting to a spreadsheet cell (VTKCell, ImageViewerCell, ...). Similarly, a SheetReference module (when connecting to a CellLocation) will specify which sheet the cell will be put on given its name, minimum row size and minimum column size. There is an example of this in examples/vtk.xml (select the version below Double Renderer). [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

=== How do I output results to the spreadsheet? ===

By inspecting the VisTrails Spreadsheet package (in the list of packages, to the left of the pipeline builder), you
can see there are built-in cells for different kinds of data, e.g., RichTextCell to display HTML and plain text. op
You (the user) can also define new cell types to display application-specific data. For example, we have developed
VtkCell, MplFigureCell, and OpenGLCell. It is possible to display pretty much anything on the Spreadsheet! [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

Examples of writing cell modules can be found in:
RichTextCell: packages/spreadsheet/widgets/richtext/richtext.py
VTK: packages/vtk/vtkcell.py

Here is the summary of some requirements on a cell widget:

(1) It must be a Qt widget. It should inherit from spreadsheet_cell.QCellWidget in the spreadsheet package. Although any Qt Widget would work, certain features such as animation will not be available (without rewriting it).

(2) It must re-implement the updateContents() function to take a set of inputs (usually coming from input ports of a wrapper Module) and display on the cells. VisTrails uses this function to update/reuse cells on the spreadsheet when new data comes in.

(3) It needs a wrapper VisTrails Module (inherited from basic_widgets.SpreadsheetCell of the spreadsheet package). Inside the compute() method of this module, it may call self.display(CellWidgetType, (inputs)) to trigger the display event on the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#advanced-cell-options Advanced Cell Options]

=== How do I control the default number of cells in the spreadsheet? ===

You can configure the rowCount and colCount using the preferences dialog. Just go to the Module Packages tab, select spreadsheet in the "Enabled packages" and press the Configure button. Then a list of all the configuration options for the spreadsheet will show up. [http://www.vistrails.org/usersguide/spreadsheet.html#custom-layout-options Custom Layout Options]

=== Is it possible to launch a web browser from the vistrails spreadsheet? We would like to output several urls from a parameter sweep and then have the option to click on each one to view the resulting page. I can view the page within the spreadsheet, but it is really too crowded.===

Currently, there isn't a widget that provides exactly this functionality, but I can think of a few solutions that may work for you:

(1) You can use parameter exploration to generate multiple sheets so you might have an exploration that opens each page in a new sheet. Use the third column/dimension in the exploration interface to have a parameter span sheets.

(2) The spreadsheet is extensible so you can write a custom spreadsheet cell widget that has a button or label with the desired link (a QLabel with openExternalLinks set to True, for example).

(3) You can tweak the existing RichTextCell be adding the line "self.browser.setOpenExternalLinks(True)" at line 63 of the source file "vistrails/packages/spreadsheet/widgets/richtext/richtext.py". Then, if your workflow creates a file with html markup text like "<a href="http://www.vistrails.org">VisTrails</a>" connected to a RichTextCell, clicking on the rendered link in the cell will open it in a web browser. You need to add the aforementioned line to the source to let Qt know that you want the link opened externally; by default, it will just issue an event that isn't processed. [http://www.vistrails.org/usersguide/spreadsheet.html#launching-a-web-browser Launching a Web Browser]

== Integrating your software into VisTrails ==

===How can I integrate my own program into VisTrails?===

The easiest way is to create a package. Writing a package is often very simple, here are instructions on how to do it: [[UsersGuideVisTrailsPackages]]

You can also dynamically generate modules. For an example see:

http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#Packages_that_generate_modules_dynamically

In particular, see the new_module call which uses python's type() function to generate new classes dynamically.

=== How do modules deal with multiple inputs in a same port? ===

(And should that even be allowed?)

For compatibility reasons, we do need to allow multiple connections to an input port. However, most package developers should never have to use this, and so we do our best to hide it. the default behavior for getting inputs from a port, then, is to always return a single input.

If on your module you need multiple inputs connected to a single port, use the 'forceGetInputListFromPort' method. It will return a list of all the data items coming through the port. The VTK package uses this feature, so look there for usage examples (packages/vtk/base_widget.py)

=== Are there mechanisms for attaching widgets to different modules/parameters? ===

Right now, we have a mechanism for putting a specific widget for an input port. For example, if a port is SetColor(red, green, blue), we can put a color wheel widget there. Or we can also replace the SetFileName port with a File Widget. However, this is not per parameter (only per port). We are currently working on this problem.

=== Can I organize my package so it appears hierarchical in the module palette? ===

Yes. Use the namespace keyword argument when adding the module to the registry. For example,

registry.add_module(MyModule, namespace='MyNamespace')

=== Can I nest namespaces? ===

Yes. Use the '|' character to separate different the hierarchy. For example,

registry.add_module(MyModule, namespace='ParentNamespace|ChildNamespace')

=== Are there shortcuts for registry initialization? ===

Yes. If you define _modules as a list of classes in the __init__.py file of your package, VisTrails will attempt to load all classes specified as modules. You can provide add_module options as keyword arguments by specifying a tuple (class, kwargs) in the list. For example:

_modules = [MyModule1, (MyModule2, {'namespace': 'MyNamespace'})]

In addition, you need to identify the ports of your modules as a field in your class by defining _input_ports and _output_ports lists. Here, the items in each list must be tuples of the form (portName, portSignature, optional=False, sort_key=-1). For example:

class MyModule(Module):
def compute(self):
pass

_input_ports = [('firstInput', String), ('secondInput', Integer, True)]
_output_ports = [('firstOutput', String), ('secondOutput', String)]

=== Can I define ports to be of types that I do not import into my package? ===

Yes. You can pass an identifier string as the portSignature instead. The port_signature string is defined by:

<module_string> := <package_identifier>:[<namespace>|]<module_name>,
<port_signature> := (<module_string>*)

For example,

registry.add_input_port(MyModule, 'myInputPort', '(edu.utah.sci.vistrails.basic:String)')

or

_input_ports = [('myInputPort', '(edu.utah.sci.vistrails.basic:String)')]

=== What do I need to change in my package to make it reloadable (new in v1.4.2)? ===
See [http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#How_to_make_you_package_reloadable UsersGuideVisTrailsPackages] for an explanation.

=== Can I add default values or labels for parameters? ===

Yes. Versions 1.4 and greater support these features. See [[UsersGuideVisTrailsPackages#Adding_default_values_and.2For_labels_for_parameters | UsersGuideVisTrailsPacakges]] for more details.

=== I want to write a module to load HDF data whose output (e.g., data, string) varies according to the input I give it. Is is possible to do this in VisTrails, and if yes, how can I do that? Ideally, I would like to avoid having to change the connection of my output every time I change the input. ===

There are a few ways to tackle this - each has it's own benefits and pitfalls. Firstly, module connections do respect class hierarchies as we're familiar with in object oriented languages. For instance, A module can output a Constant of which String, Float, Integer, etc are specifications. In this way, you can have a subclass of something like HDFData be passed out of the module and the connections will be established regardless of the sub-type. This is a bit dangerous though. Modules downstream of such a class may not really know how to operate on certain types derived from the super-class. Extreme care must be taken both when creating the modules as well as connecting them to prevent things like this from happening.

A second method that I employ in several different packages is the idea of a container class. For instance, the NumSciPy package uses a relatively generic container "Numpy Array" to encapsulate the data. Of course, these encapsulating objects can store dictionaries that other modules can easily access and understand how to operate on. Although this method is slightly more work, the benefits of a stricter typing of ports is beneficial - particularly upon interfacing with other packages that may depend on strongly typed constants (for example).

== The Console ==

=== Where should I go to find out what I can call from the console and how to import it? ===

We have tried to make some methods more accessible in the console via an api. You can import the api via <code>import api</code> in the console and see the available methods with <code>dir(api)</code>. To open a vistrail:

import api
api.open_vistrail_from_file('/Applications/VisTrails/examples/terminator.vt')

To execute a version of a workflow, you currently have to go through the controller:

api.select_version('Histogram')
api.get_current_controller().execute_current_workflow()

Currently, only a subset of VisTrails functionality is directly available from the api. However, since VisTrails is written in python, you can dig down starting with the VistrailsApplication or controller object to expose most of our internal methods. If you have suggestions for calls to be added to the api, please let us know.

One other feature that we're working on, but is still in progress is the ability to construct workflows via the console. For example:

vtk = load_package('edu.utah.sci.vistrails.vtk')
vtk.vtkDataSetReader() # adds a vtkDataSetReader module to the pipeline
# click on the new module
a = selected_modules()[0] # get the one currently selected module
a.SetFile('/vistrails/examples/data/head120.vtk') # sets the SetFile parmaeter for the data set reader
b = vtk.vtkContourFilter() # adds a vtkContourFilter module to the pipeline and saves to var b
b.SetInputConnection0(a.GetOutputPort0()) # connects a's GetOutputPort0 port to b's SetInputConnection0

== Persistence Package ==

=== How do I use the output of one workflow as the input for another using the persistence package? ===

You need to configure the persistence modules using the module's configuration dialog. After adding a <code>PersistentOutputFile</code> to the workflow, click on the triangle in the upper-right corner of the <code>PersistentOutputFile</code>, and select "Edit Configuration" from the menu that appears. In this dialog, select "Create New Reference" and give the reference a name (and any space-delimited tags). Upon running that workflow, the data will be written to the persistent store. In the second workflow where you wish to use that file, add a <code>PersistentInputFile</code> and go to its configuration dialog in the same manner as with the output file. In that dialog, select "Use Existing Reference" and select the data that you just added in the first workflow in the list of files below. Now, when you run that workflow, it will grab the data from the persistent store.

Here is an example: [[Media:offscreen_persistent.vt | offscreen_persistent.vt]]. Run the "persistent offscreen" workflow first, and then run the "display persistent output" to use the output of the first workflow as the input for the second.

== VTK ==

=== Given a VTK visualization, how can I generate a webpage from it? ===

Check out the html pipeline in offscreen.xml.

=== I'm trying to use VTK, but there doesn't seem to be any output. What is wrong? ===

To use VTK on VisTrails, you need a slightly different way of connecting the renderer modules. Instead of using the standard RenderWindow/RenderWindowInteractor infrastructure, you simply connect the renderer to a VTKCell. The examples directory in the distribution has several VTK examples that illustrate.

=== I am trying to add a module to the workflow via Python, but how can I access vtk modules? ===

Here's an example:

import api

vtvtk = 'edu.utah.sci.vistrails.vtk'

module = api.add_module(0, 0, vtvtk, 'vtkContourFilter', '')

The third argument in add_module is the package identifier. You can find this in the "Module Packages" panel of the Preferences; just click on the package you're interested in and it will appear in the information on the right.

== matplotlib ==

=== I'm experiencing a problem with Latex labels and the matplotlib that comes with VisTrails 1.5. The script below entered to the interpreter that comes with VT is sufficient to reproduce it. ===

import matplotlib.pyplot as plt
plt.plot([1,2,3],[1,2,3])
plt.xlabel("$foo$")

Remove your ~/.matplotlib folder and re-start VisTrails

== VisTrails Development ==

=== I would like to build VisTrails from source. Are there instructions on how to do this? ===

Yes! Take a look at http://www.vistrails.org/index.php/Mac_Intel_Instructions
[http://www.vistrails.org/usersguide/getting_started.html#installing-vistrails-from-source Installing VisTrails from source]

== Accessing Provenance Information ==

=== How do I access the information in the execution log? ===

The code responsible for storing execution information is located in the "core/log" directories, and the code that generates much of that information is in "core/interpreter/cached.py". Modules can add execution-specific annotations to provenance via annotate() calls during execution, but much of the data (like timing and errors) is captured by the LogController and CachedInterpreter (the execution engine) objects. To analyze the log from a vistrail (.vt) file, you might have something like the following:

<code>
import core.log.log
import db.services.io

def run(fname):
# open the .vt bundle specified by the filename "fname"
bundle = db.services.io.open_vistrail_bundle_from_zip_xml(fname)[0]
# get the log filename
log_fname = bundle.vistrail.db_log_filename
if log_fname is not None:
# open the log
log = db.services.io.open_log_from_xml(log_fname, True)
# convert the log from a db object
core.log.log.Log.convert(log)
for workflow_exec in log.workflow_execs:
print 'workflow version:', workflow_exec.parent_version
print 'time started:', workflow_exec.ts_start
print 'time ended:', workflow_exec.ts_end
print 'modules executed:', [i.module_id
for i in workflow_exec.item_execs]
if __name__ == '__main__':
run("some_vistrail.vt")
</code>

You should be able to see what information is available by looking at the "core/log" classes.

FAQ

2011-03-25T22:47:36Z

Ctuttle: /* Running workflows */

Also check our [[known_issues|Known Issues]] page for troubleshooting.

----
== Running workflows ==

=== How can I run a workflow using the command line? ===

('''Updated for version 1.2''') Call vistrails using the following options:

<code>python vistrails.py -b path_to_vistrails_file:pipeline </code>

where pipeline can be a version '''tag name''' or version '''id'''

'''NOTE:''' If you downloaded the MacOS X bundle, you can run vistrails from the command line via the following commands in the Terminal. Change the current directory to wherever VisTrails was installed (often <code>/Applications</code>), and then type:

<code>Vistrails.app/Contents/MacOS/vistrails [<cmd_line_options>]</code> [http://www.vistrails.org/usersguidebatch.html#running-a-specific-workflow-in-batch-mode Running a Specific Workflow in Batch Mode]

=== Using the command line, we'd like to execute a workflow multiple times, with slightly different parameters, and create a series of output files. Is this possible?===

('''Updated for version 1.2''') We can change parameters that have an alias through the command line.

For example, offscreen pipeline in offscreen.vt always creates the file called image.png. If you want generate it with a different filename:

<code>python vistrails.py -b ../examples/offscreen.vt:offscreen -a"filename=other.png"</code>

<code>filename</code> in the example above is the alias name assigned to the parameter in the <code>value</code> method inside the String module. When running a pipeline from the command line, VisTrails will try to start the spreadsheet automatically if the pipeline requires it. For example, this other execution will also start the spreadsheet (attention to how $ characters are escaped when running on bash):

<code>python vistrails.py -b ../examples/head.vt:aliases -a"isovalue=30\$&\$diffuse_color=0.8,0.4,0.2"</code>

You can also execute more than one pipeline on the command line:

<code>python vistrails.py -b ../examples/head.vt:aliases ../examples/spx.vt:spx \
-a"isovalue=30"</code>

Use the -a parameter only once regardless the number of pipelines. [http://www.vistrails.org/batch.html#running-a-workflow-with-specific-parameters Running a Workflow with Specific Parameters]

=== I can load a vistrail, and the version tree shows up fine. However, no pipelines appear when I click on a version. What gives? ===

The most likely reason is that the vistrail uses a package that is not registered with VisTrails. You need to identify the needed package and add it to your <code>.vistrails/startup.py</code>. A single line like the following should be enough:

addPackage('enter_package_name_here')

Some packages might need more information. For example:

addPackage('afront', executable_path='/path/to/afront')

Refer to the package documentation for details. The one inconvenient step is that currently there's no automated way to describe what is the missing package. We're working on this feature for future releases.

=== I have a workflow that reads a file and then does some processing. The first time it runs, it executes correctly. But in subsequent, nothing happens. ===

VisTrails caches by default, so after a workflow is executed, if none of its parameters change, it won't be executed again.

If a workflow reads a file using the basic module File, VisTrails does check whether the file was modified since the last run. It does so by keeping a signature that is based on the modification time of the file. And if the file was modified, the File module and all downstream modules (the ones which depend on File) will be executed.

''Note: If you would like your input and output data to be versioned, you can use the Persistence package.''

If you do not want VisTrails to cache executions, you can turn off caching: go to Menu Edit -> Preferences and in the General Configuration tab, change Cache execution results to Never. [http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

=== Can VisTrails execute workflows in parallel? ===

The VisTrails server can only execute pipelines in parallel if there's more than one instance of VisTrails running. The command

<code> self.rpcserver = ThreadedXMLRPCServer((self.temp_xml_rpc_options.server, self.temp_xml_rpc_options.port))</code>

starts a multithreaded version of the XML-RPC server, so it will create a thread for each request received by the server. The problem is that Qt/PyQT doesn't allow these multiple threads create GUI objects, only in the main thread. To overcome this limitation, the multithreaded version can instantiate other single threaded versions of VisTrails and put them in a queue, so workflow executions and other GUI-related requests, such as generating workflow graphs and history trees can be forwarded to this queue, and each instance takes turns in answering the request. If the results are in the cache, the multithreaded version answers the requests directly.

Note that this infrastructure works on Linux only. To make this work on Windows, you have to create a script similar to start_vistrails_xvfb.sh (located in the scripts folder) where you can send the number of other instances via command-line options to VisTrails. The command line options are:

python vistrails_server.py -T <ADDRESS> -R <PORT> -O<NUMBER_OF_OTHER_VISTRAILS_INSTANCES> [-M]&

If you want the main vistrails instance to be multithreaded, use the -M at the end.

After creating this script, update function start_other_instances in vistrails/gui/application_server.py lines 1007-1023 and set the script variable to point to your script. You may also have to change the arguments sent to your script (line 1016: for example, you don't need to set a virtual display). You will need to change the path to the stop_vistrails_server.py script (on line 1026) according to your installation path. [http://www.vistrails.org/usersguide/batch.html#executing-workflows-in-parallel Executing Workflows in Parallel]

=== When a workflow is executed, what do the colors mean? ===

- lilac: module was notexecuted

- yellow: module is currently being executed

- green: module was successfully executed

- orange: module was cached

- red: the execution of the module failed

[http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

== Building workflows ==

=== Is there a way to give each widget a "display name" in addition to the module name at the center of the widget? ===

Yes, a "display name" can be assigned to a module by selecting the triangle in its top right corner to open a popup menu and selecting the Set Module Label... menu item. You will then be prompted to enter the "display name". [http://www.vistrails.org/usersguide/creating.html#changing-module-labels Changing Module Labels]

=== Is there a way to re-center the picture-in-picture (PiP) view? ===

Yes. If you click on the PIP window to bring it to focus, you can press Ctrl-R (or Command-R on Mac) to re-center the PiP window. [http://www.vistrails.org/usersguide/getting_started.html#vistrails-interaction Vistrails Interaction]

=== How do I search for a literal "?" (question mark) in the search box in the Property panel? ===

Since we allow regular expressions in our search box, question marks are treated as meta-characters. Thus, searching for "?" returns everything and "abc?" will return everything containing "abc". You need to use "\?" instead to search for "?". So the search for "??" would be "\?\?". [http://www.vistrails.org/usersguide/querying.html#textual-queries Textual Queries]

== Using VisTrails as a server ==

=== What is the VisTrails server-mode? ===

Using the VisTrails server mode, it is possible to ''execute workflows and control VisTrails through another application''. For example, the CrowdLabs Web portal (http://www.crowdlabs.org) accesses a VisTrails sever to execute workflows, retrieve and display vistrail trees and workflows. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

=== How do I execute workflows and control VisTrails through another application? ===

The way you access the server is by doing XML-RPC calls. In the current VisTrails release, we include a set of PHP scripts that can talk to a VisTrails server instance. They are in "extensions/http" folder. The files are reasonably well documented. Also, it should be not difficult to create python scripts to access the server (just use xmlrpclib module).

Note that the VisTrails server requires the provenance and workflows to be in a database. More detailed instructions on how to setup the server and the database
are available here:

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_server_setup.html

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_database_setup.html

If what you want is just to execute a series of workflows in batch mode, a simpler solution would be to ''use the VisTrails client in batch mode''. Chapter 12 of the user's guide contains detailed information and examples on that. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

== Control Flow ==

=== Note: using map ===

When using 'map', the module (or subworkflow) used as function port in the map module MUST be a function, i.e., it can only define 1 output port. [http://www.vistrails.org/usersguide/controlflow.html#the-map-operator The Map Operator]

== Spreadsheet ==

Where ''pipeline'' is a version number or a tag.

=== How can I save an image from the spreadsheet? ===

While having the focus on a spreadsheet cell and select the camera on the toolbar to take a snapshot. The system will prompt you for the location and file name where it should be saved. The other icons can be used for saving multiple images that can be used for generating an animation on demand. A whole sheet can also be saved by selecting Export (either from the menu or from the toolbar). [http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet-image Saving a Spreadsheet Image]

===Is it possible to save the complete state of the spreadsheet?===
[http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet Saving a Spreadsheet]

=== Can I view multiple sheets at the same time? ===

Yes. Each sheet on the spreadsheet can be displayed as a dock widget separated from the main spreadsheet window by dragging its tab name out of the tab bar at the bottom of the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Then, how can I put back a separated sheet? ===

A sheet can be docked back to the main window by dragging it back to the tab bar or double-click on its title bar. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== How can I order sheets on the spreadsheet? ===

This can be done by dragging the sheet name on the bottom top bar and drop it to the right place. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Can I control where a cell will be placed on the spreadsheet window? ===

By default, an unoccupied cell on the active sheet will be chosen to display the result. However, you can specify exactly in the pipeline where a spreadsheet cell will be placed by using CellLocation and SheetReference. CellLocation specifies the location (row and column) of a cell when connecting to a spreadsheet cell (VTKCell, ImageViewerCell, ...). Similarly, a SheetReference module (when connecting to a CellLocation) will specify which sheet the cell will be put on given its name, minimum row size and minimum column size. There is an example of this in examples/vtk.xml (select the version below Double Renderer). [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

=== How do I output results to the spreadsheet? ===

By inspecting the VisTrails Spreadsheet package (in the list of packages, to the left of the pipeline builder), you
can see there are built-in cells for different kinds of data, e.g., RichTextCell to display HTML and plain text. op
You (the user) can also define new cell types to display application-specific data. For example, we have developed
VtkCell, MplFigureCell, and OpenGLCell. It is possible to display pretty much anything on the Spreadsheet! [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

Examples of writing cell modules can be found in:
RichTextCell: packages/spreadsheet/widgets/richtext/richtext.py
VTK: packages/vtk/vtkcell.py

Here is the summary of some requirements on a cell widget:

(1) It must be a Qt widget. It should inherit from spreadsheet_cell.QCellWidget in the spreadsheet package. Although any Qt Widget would work, certain features such as animation will not be available (without rewriting it).

(2) It must re-implement the updateContents() function to take a set of inputs (usually coming from input ports of a wrapper Module) and display on the cells. VisTrails uses this function to update/reuse cells on the spreadsheet when new data comes in.

(3) It needs a wrapper VisTrails Module (inherited from basic_widgets.SpreadsheetCell of the spreadsheet package). Inside the compute() method of this module, it may call self.display(CellWidgetType, (inputs)) to trigger the display event on the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#advanced-cell-options Advanced Cell Options]

=== How do I control the default number of cells in the spreadsheet? ===

You can configure the rowCount and colCount using the preferences dialog. Just go to the Module Packages tab, select spreadsheet in the "Enabled packages" and press the Configure button. Then a list of all the configuration options for the spreadsheet will show up. [http://www.vistrails.org/usersguide/spreadsheet.html#custom-layout-options Custom Layout Options]

=== Is it possible to launch a web browser from the vistrails spreadsheet? We would like to output several urls from a parameter sweep and then have the option to click on each one to view the resulting page. I can view the page within the spreadsheet, but it is really too crowded.===

Currently, there isn't a widget that provides exactly this functionality, but I can think of a few solutions that may work for you:

(1) You can use parameter exploration to generate multiple sheets so you might have an exploration that opens each page in a new sheet. Use the third column/dimension in the exploration interface to have a parameter span sheets.

(2) The spreadsheet is extensible so you can write a custom spreadsheet cell widget that has a button or label with the desired link (a QLabel with openExternalLinks set to True, for example).

(3) You can tweak the existing RichTextCell be adding the line "self.browser.setOpenExternalLinks(True)" at line 63 of the source file "vistrails/packages/spreadsheet/widgets/richtext/richtext.py". Then, if your workflow creates a file with html markup text like "<a href="http://www.vistrails.org">VisTrails</a>" connected to a RichTextCell, clicking on the rendered link in the cell will open it in a web browser. You need to add the aforementioned line to the source to let Qt know that you want the link opened externally; by default, it will just issue an event that isn't processed. [http://www.vistrails.org/usersguide/spreadsheet.html#launching-a-web-browser Launching a Web Browser]

== Integrating your software into VisTrails ==

===How can I integrate my own program into VisTrails?===

The easiest way is to create a package. Writing a package is often very simple, here are instructions on how to do it: [[UsersGuideVisTrailsPackages]]

You can also dynamically generate modules. For an example see:

http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#Packages_that_generate_modules_dynamically

In particular, see the new_module call which uses python's type() function to generate new classes dynamically.

=== How do modules deal with multiple inputs in a same port? ===

(And should that even be allowed?)

For compatibility reasons, we do need to allow multiple connections to an input port. However, most package developers should never have to use this, and so we do our best to hide it. the default behavior for getting inputs from a port, then, is to always return a single input.

If on your module you need multiple inputs connected to a single port, use the 'forceGetInputListFromPort' method. It will return a list of all the data items coming through the port. The VTK package uses this feature, so look there for usage examples (packages/vtk/base_widget.py)

=== Are there mechanisms for attaching widgets to different modules/parameters? ===

Right now, we have a mechanism for putting a specific widget for an input port. For example, if a port is SetColor(red, green, blue), we can put a color wheel widget there. Or we can also replace the SetFileName port with a File Widget. However, this is not per parameter (only per port). We are currently working on this problem.

=== Can I organize my package so it appears hierarchical in the module palette? ===

Yes. Use the namespace keyword argument when adding the module to the registry. For example,

registry.add_module(MyModule, namespace='MyNamespace')

=== Can I nest namespaces? ===

Yes. Use the '|' character to separate different the hierarchy. For example,

registry.add_module(MyModule, namespace='ParentNamespace|ChildNamespace')

=== Are there shortucts for registry initialization? ===

Yes. If you define _modules as a list of classes in the __init__.py file of your package, VisTrails will attempt to load all classes specified as modules. You can provide add_module options as keyword arguments by specifying a tuple (class, kwargs) in the list. For example:

_modules = [MyModule1, (MyModule2, {'namespace': 'MyNamespace'})]

In addition, you need to identify the ports of your modules as a field in your class by defining _input_ports and _output_ports lists. Here, the items in each list must be tuples of the form (portName, portSignature, optional=False, sort_key=-1). For example:

class MyModule(Module):
def compute(self):
pass

_input_ports = [('firstInput', String), ('secondInput', Integer, True)]
_output_ports = [('firstOutput', String), ('secondOutput', String)]

=== Can I define ports to be of types that I do not import into my package? ===

Yes. You can pass an identifier string as the portSignature instead. The port_signature string is defined by:

<module_string> := <package_identifier>:[<namespace>|]<module_name>,
<port_signature> := (<module_string>*)

For example,

registry.add_input_port(MyModule, 'myInputPort', '(edu.utah.sci.vistrails.basic:String)')

or

_input_ports = [('myInputPort', '(edu.utah.sci.vistrails.basic:String)')]

=== What do I need to change in my package to make it reloadable (new in v1.4.2)? ===
See [http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#How_to_make_you_package_reloadable UsersGuideVisTrailsPackages] for an explanation.

=== Can I add default values or labels for parameters? ===

Yes. Versions 1.4 and greater support these features. See [[UsersGuideVisTrailsPackages#Adding_default_values_and.2For_labels_for_parameters | UsersGuideVisTrailsPacakges]] for more details.

=== I want to write a module to load HDF data whose output (e.g., data, string) varies according to the input I give it. Is is possible to do this in VisTrails, and if yes, how can I do that? Ideally, I would like to avoid having to change the connection of my output every time I change the input. ===

There are a few ways to tackle this - each has it's own benefits and pitfalls. Firstly, module connections do respect class hierarchies as we're familiar with in object oriented languages. For instance, A module can output a Constant of which String, Float, Integer, etc are specifications. In this way, you can have a subclass of something like HDFData be passed out of the module and the connections will be established regardless of the sub-type. This is a bit dangerous though. Modules downstream of such a class may not really know how to operate on certain types derived from the super-class. Extreme care must be taken both when creating the modules as well as connecting them to prevent things like this from happening.

A second method that I employ in several different packages is the idea of a container class. For instance, the NumSciPy package uses a relatively generic container "Numpy Array" to encapsulate the data. Of course, these encapsulating objects can store dictionaries that other modules can easily access and understand how to operate on. Although this method is slightly more work, the benefits of a stricter typing of ports is beneficial - particularly upon interfacing with other packages that may depend on strongly typed constants (for example).

== The Console ==

=== Where should I go to find out what I can call from the console and how to import it? ===

We have tried to make some methods more accessible in the console via an api. You can import the api via <code>import api</code> in the console and see the available methods with <code>dir(api)</code>. To open a vistrail:

import api
api.open_vistrail_from_file('/Applications/VisTrails/examples/terminator.vt')

To execute a version of a workflow, you currently have to go through the controller:

api.select_version('Histogram')
api.get_current_controller().execute_current_workflow()

Currently, only a subset of VisTrails functionality is directly available from the api. However, since VisTrails is written in python, you can dig down starting with the VistrailsApplication or controller object to expose most of our internal methods. If you have suggestions for calls to be added to the api, please let us know.

One other feature that we're working on, but is still in progress is the ability to construct workflows via the console. For example:

vtk = load_package('edu.utah.sci.vistrails.vtk')
vtk.vtkDataSetReader() # adds a vtkDataSetReader module to the pipeline
# click on the new module
a = selected_modules()[0] # get the one currently selected module
a.SetFile('/vistrails/examples/data/head120.vtk') # sets the SetFile parmaeter for the data set reader
b = vtk.vtkContourFilter() # adds a vtkContourFilter module to the pipeline and saves to var b
b.SetInputConnection0(a.GetOutputPort0()) # connects a's GetOutputPort0 port to b's SetInputConnection0

== Persistence Package ==

=== How do I use the output of one workflow as the input for another using the persistence package? ===

You need to configure the persistence modules using the module's configuration dialog. After adding a <code>PersistentOutputFile</code> to the workflow, click on the triangle in the upper-right corner of the <code>PersistentOutputFile</code>, and select "Edit Configuration" from the menu that appears. In this dialog, select "Create New Reference" and give the reference a name (and any space-delimited tags). Upon running that workflow, the data will be written to the persistent store. In the second workflow where you wish to use that file, add a <code>PersistentInputFile</code> and go to its configuration dialog in the same manner as with the output file. In that dialog, select "Use Existing Reference" and select the data that you just added in the first workflow in the list of files below. Now, when you run that workflow, it will grab the data from the persistent store.

Here is an example: [[Media:offscreen_persistent.vt | offscreen_persistent.vt]]. Run the "persistent offscreen" workflow first, and then run the "display persistent output" to use the output of the first workflow as the input for the second.

== VTK ==

=== Given a VTK visualization, how can I generate a webpage from it? ===

Check out the html pipeline in offscreen.xml.

=== I'm trying to use VTK, but there doesn't seem to be any output. What is wrong? ===

To use VTK on VisTrails, you need a slightly different way of connecting the renderer modules. Instead of using the standard RenderWindow/RenderWindowInteractor infrastructure, you simply connect the renderer to a VTKCell. The examples directory in the distribution has several VTK examples that illustrate.

=== I am trying to add a module to the workflow via Python, but how can I access vtk modules? ===

Here's an example:

import api

vtvtk = 'edu.utah.sci.vistrails.vtk'

module = api.add_module(0, 0, vtvtk, 'vtkContourFilter', '')

The third argument in add_module is the package identifier. You can find this in the "Module Packages" panel of the Preferences; just click on the package you're interested in and it will appear in the information on the right.

== matplotlib ==

=== I'm experiencing a problem with Latex labels and the matplotlib that comes with VisTrails 1.5. The script below entered to the interpreter that comes with VT is sufficient to reproduce it. ===

import matplotlib.pyplot as plt
plt.plot([1,2,3],[1,2,3])
plt.xlabel("$foo$")

Remove your ~/.matplotlib folder and re-start VisTrails

== VisTrails Development ==

=== I would like to build VisTrails from source. Are there instructions on how to do this? ===

Yes! Take a look at http://www.vistrails.org/index.php/Mac_Intel_Instructions
[http://www.vistrails.org/usersguide/getting_started.html#installing-vistrails-from-source Installing VisTrails from source]

== Accessing Provenance Information ==

=== How do I access the information in the execution log? ===

The code responsible for storing execution information is located in the "core/log" directories, and the code that generates much of that information is in "core/interpreter/cached.py". Modules can add execution-specific annotations to provenance via annotate() calls during execution, but much of the data (like timing and errors) is captured by the LogController and CachedInterpreter (the execution engine) objects. To analyze the log from a vistrail (.vt) file, you might have something like the following:

<code>
import core.log.log
import db.services.io

def run(fname):
# open the .vt bundle specified by the filename "fname"
bundle = db.services.io.open_vistrail_bundle_from_zip_xml(fname)[0]
# get the log filename
log_fname = bundle.vistrail.db_log_filename
if log_fname is not None:
# open the log
log = db.services.io.open_log_from_xml(log_fname, True)
# convert the log from a db object
core.log.log.Log.convert(log)
for workflow_exec in log.workflow_execs:
print 'workflow version:', workflow_exec.parent_version
print 'time started:', workflow_exec.ts_start
print 'time ended:', workflow_exec.ts_end
print 'modules executed:', [i.module_id
for i in workflow_exec.item_execs]
if __name__ == '__main__':
run("some_vistrail.vt")
</code>

You should be able to see what information is available by looking at the "core/log" classes.

FAQ

2011-03-25T22:34:31Z

Ctuttle: /* Running workflows */

Also check our [[known_issues|Known Issues]] page for troubleshooting.

----
== Running workflows ==

=== How can I run a workflow using the command line? ===

('''Updated for version 1.2''') Call vistrails using the following options:

<code>python vistrails.py -b path_to_vistrails_file:pipeline </code>

where pipeline can be a version '''tag name''' or version '''id'''

'''NOTE:''' If you downloaded the MacOS X bundle, you can run vistrails from the command line via the following commands in the Terminal. Change the current directory to wherever VisTrails was installed (often <code>/Applications</code>), and then type:

<code>Vistrails.app/Contents/MacOS/vistrails [<cmd_line_options>]</code> [http://www.vistrails.org/usersguidebatch.html#running-a-specific-workflow-in-batch-mode Running a Specific Workflow in Batch Mode]

=== Using the command line, we'd like to execute a workflow multiple times, with slightly different parameters, and create a series of output files. Is this possible?===

('''Updated for version 1.2''') We can change parameters that have an alias through the command line.

For example, offscreen pipeline in offscreen.vt always creates the file called image.png. If you want generate it with a different filename:

<code>python vistrails.py -b ../examples/offscreen.vt:offscreen -a"filename=other.png"</code>

<code>filename</code> in the example above is the alias name assigned to the parameter in the <code>value</code> method inside the String module. When running a pipeline from the command line, VisTrails will try to start the spreadsheet automatically if the pipeline requires it. For example, this other execution will also start the spreadsheet (attention to how $ characters are escaped when running on bash):

<code>python vistrails.py -b ../examples/head.vt:aliases -a"isovalue=30\$&\$diffuse_color=0.8,0.4,0.2"</code>

You can also execute more than one pipeline on the command line:

<code>python vistrails.py -b ../examples/head.vt:aliases ../examples/spx.vt:spx \
-a"isovalue=30"</code>

Use the -a parameter only once regardless the number of pipelines. [http://www.vistrails.org/batch.html#running-a-workflow-with-specific-parameters Running a Workflow with Specific Parameters]

=== I can load a vistrail, and the version tree shows up fine. However, no pipelines appear when I click on a version. What gives? ===

The most likely reason is that the vistrail uses a package that is not registered with VisTrails. You need to identify the needed package and add it to your <code>.vistrails/startup.py</code>. A single line like the following should be enough:

addPackage('enter_package_name_here')

Some packages might need more information. For example:

addPackage('afront', executable_path='/path/to/afront')

Refer to the package documentation for details. The one inconvenient step is that currently there's no automated way to describe what is the missing package. We're working on this feature for future releases.

=== I have a workflow that reads a file and then does some processing. The first time it runs, it executes correctly. But in subsequent, nothing happens. ===

VisTrails caches by default, so after a workflow is executed, if none of its parameters change, it won't be executed again.

If a workflow reads a file using the basic module File, VisTrails does check whether the file was modified since the last run. It does so by keeping a signature that is based on the modification time of the file. And if the file was modified, the File module and all downstream modules (the ones which depend on File) will be executed.

''Note: If you would like your input and output data to be versioned, you can use the Persistence package.''

If you do not want VisTrails to cache executions, you can turn off caching: go to Menu Edit -> Preferences and in the General Configuration tab, change Cache execution results to Never. [http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

=== Can VisTrails execute workflows in parallel? ===

The VisTrails server can only execute pipelines in parallel if there's more than one instance of VisTrails running. The command

<code> self.rpcserver = ThreadedXMLRPCServer((self.temp_xml_rpc_options.server, self.temp_xml_rpc_options.port))</code>

starts a multithreaded version of the XML-RPC server, so it will create a thread for each request received by the server. The problem is that Qt/PyQT doesn't allow these multiple threads create GUI objects, only in the main thread. To overcome this limitation, the multithreaded version can instantiate other single threaded versions of VisTrails and put them in a queue, so workflow executions and other GUI-related requests, such as generating workflow graphs and history trees can be forwarded to this queue, and each instance takes turns in answering the request. If the results are in the cache, the multithreaded version answers the requests directly.

Note that this infrastructure works on Linux only. To make this work on Windows, you have to create a script similar to start_vistrails_xvfb.sh (located in the scripts folder) where you can send the number of other instances via command-line options to VisTrails. The command line options are:

python vistrails_server.py -T <ADDRESS> -R <PORT> -O<NUMBER_OF_OTHER_VISTRAILS_INSTANCES> [-M]&

If you want the main vistrails instance to be multithreaded, use the -M at the end.

After creating this script, update function start_other_instances in vistrails/gui/application_server.py lines 1007-1023 and set the script variable to point to your script. You may also have to change the arguments sent to your script (line 1016: for example, you don't need to set a virtual display). You will need to change the path to the stop_vistrails_server.py script (on line 1026) according to your installation path.

=== When a workflow is executed, what do the colors mean? ===

- lilac: module was notexecuted

- yellow: module is currently being executed

- green: module was successfully executed

- orange: module was cached

- red: the execution of the module failed

[http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

== Building workflows ==

=== Is there a way to give each widget a "display name" in addition to the module name at the center of the widget? ===

Yes, a "display name" can be assigned to a module by selecting the triangle in its top right corner to open a popup menu and selecting the Set Module Label... menu item. You will then be prompted to enter the "display name". [http://www.vistrails.org/usersguide/creating.html#changing-module-labels Changing Module Labels]

=== Is there a way to re-center the picture-in-picture (PiP) view? ===

Yes. If you click on the PIP window to bring it to focus, you can press Ctrl-R (or Command-R on Mac) to re-center the PiP window. [http://www.vistrails.org/usersguide/getting_started.html#vistrails-interaction Vistrails Interaction]

=== How do I search for a literal "?" (question mark) in the search box in the Property panel? ===

Since we allow regular expressions in our search box, question marks are treated as meta-characters. Thus, searching for "?" returns everything and "abc?" will return everything containing "abc". You need to use "\?" instead to search for "?". So the search for "??" would be "\?\?". [http://www.vistrails.org/usersguide/querying.html#textual-queries Textual Queries]

== Using VisTrails as a server ==

=== What is the VisTrails server-mode? ===

Using the VisTrails server mode, it is possible to ''execute workflows and control VisTrails through another application''. For example, the CrowdLabs Web portal (http://www.crowdlabs.org) accesses a VisTrails sever to execute workflows, retrieve and display vistrail trees and workflows. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

=== How do I execute workflows and control VisTrails through another application? ===

The way you access the server is by doing XML-RPC calls. In the current VisTrails release, we include a set of PHP scripts that can talk to a VisTrails server instance. They are in "extensions/http" folder. The files are reasonably well documented. Also, it should be not difficult to create python scripts to access the server (just use xmlrpclib module).

Note that the VisTrails server requires the provenance and workflows to be in a database. More detailed instructions on how to setup the server and the database
are available here:

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_server_setup.html

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_database_setup.html

If what you want is just to execute a series of workflows in batch mode, a simpler solution would be to ''use the VisTrails client in batch mode''. Chapter 12 of the user's guide contains detailed information and examples on that. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

== Control Flow ==

=== Note: using map ===

When using 'map', the module (or subworkflow) used as function port in the map module MUST be a function, i.e., it can only define 1 output port. [http://www.vistrails.org/usersguide/controlflow.html#the-map-operator The Map Operator]

== Spreadsheet ==

Where ''pipeline'' is a version number or a tag.

=== How can I save an image from the spreadsheet? ===

While having the focus on a spreadsheet cell and select the camera on the toolbar to take a snapshot. The system will prompt you for the location and file name where it should be saved. The other icons can be used for saving multiple images that can be used for generating an animation on demand. A whole sheet can also be saved by selecting Export (either from the menu or from the toolbar). [http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet-image Saving a Spreadsheet Image]

===Is it possible to save the complete state of the spreadsheet?===
[http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet Saving a Spreadsheet]

=== Can I view multiple sheets at the same time? ===

Yes. Each sheet on the spreadsheet can be displayed as a dock widget separated from the main spreadsheet window by dragging its tab name out of the tab bar at the bottom of the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Then, how can I put back a separated sheet? ===

A sheet can be docked back to the main window by dragging it back to the tab bar or double-click on its title bar. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== How can I order sheets on the spreadsheet? ===

This can be done by dragging the sheet name on the bottom top bar and drop it to the right place. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Can I control where a cell will be placed on the spreadsheet window? ===

By default, an unoccupied cell on the active sheet will be chosen to display the result. However, you can specify exactly in the pipeline where a spreadsheet cell will be placed by using CellLocation and SheetReference. CellLocation specifies the location (row and column) of a cell when connecting to a spreadsheet cell (VTKCell, ImageViewerCell, ...). Similarly, a SheetReference module (when connecting to a CellLocation) will specify which sheet the cell will be put on given its name, minimum row size and minimum column size. There is an example of this in examples/vtk.xml (select the version below Double Renderer). [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

=== How do I output results to the spreadsheet? ===

By inspecting the VisTrails Spreadsheet package (in the list of packages, to the left of the pipeline builder), you
can see there are built-in cells for different kinds of data, e.g., RichTextCell to display HTML and plain text. op
You (the user) can also define new cell types to display application-specific data. For example, we have developed
VtkCell, MplFigureCell, and OpenGLCell. It is possible to display pretty much anything on the Spreadsheet! [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

Examples of writing cell modules can be found in:
RichTextCell: packages/spreadsheet/widgets/richtext/richtext.py
VTK: packages/vtk/vtkcell.py

Here is the summary of some requirements on a cell widget:

(1) It must be a Qt widget. It should inherit from spreadsheet_cell.QCellWidget in the spreadsheet package. Although any Qt Widget would work, certain features such as animation will not be available (without rewriting it).

(2) It must re-implement the updateContents() function to take a set of inputs (usually coming from input ports of a wrapper Module) and display on the cells. VisTrails uses this function to update/reuse cells on the spreadsheet when new data comes in.

(3) It needs a wrapper VisTrails Module (inherited from basic_widgets.SpreadsheetCell of the spreadsheet package). Inside the compute() method of this module, it may call self.display(CellWidgetType, (inputs)) to trigger the display event on the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#advanced-cell-options Advanced Cell Options]

=== How do I control the default number of cells in the spreadsheet? ===

You can configure the rowCount and colCount using the preferences dialog. Just go to the Module Packages tab, select spreadsheet in the "Enabled packages" and press the Configure button. Then a list of all the configuration options for the spreadsheet will show up. [http://www.vistrails.org/usersguide/spreadsheet.html#custom-layout-options Custom Layout Options]

=== Is it possible to launch a web browser from the vistrails spreadsheet? We would like to output several urls from a parameter sweep and then have the option to click on each one to view the resulting page. I can view the page within the spreadsheet, but it is really too crowded.===

Currently, there isn't a widget that provides exactly this functionality, but I can think of a few solutions that may work for you:

(1) You can use parameter exploration to generate multiple sheets so you might have an exploration that opens each page in a new sheet. Use the third column/dimension in the exploration interface to have a parameter span sheets.

(2) The spreadsheet is extensible so you can write a custom spreadsheet cell widget that has a button or label with the desired link (a QLabel with openExternalLinks set to True, for example).

(3) You can tweak the existing RichTextCell be adding the line "self.browser.setOpenExternalLinks(True)" at line 63 of the source file "vistrails/packages/spreadsheet/widgets/richtext/richtext.py". Then, if your workflow creates a file with html markup text like "<a href="http://www.vistrails.org">VisTrails</a>" connected to a RichTextCell, clicking on the rendered link in the cell will open it in a web browser. You need to add the aforementioned line to the source to let Qt know that you want the link opened externally; by default, it will just issue an event that isn't processed. [http://www.vistrails.org/usersguide/spreadsheet.html#launching-a-web-browser Launching a Web Browser]

== Integrating your software into VisTrails ==

===How can I integrate my own program into VisTrails?===

The easiest way is to create a package. Writing a package is often very simple, here are instructions on how to do it: [[UsersGuideVisTrailsPackages]]

You can also dynamically generate modules. For an example see:

http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#Packages_that_generate_modules_dynamically

In particular, see the new_module call which uses python's type() function to generate new classes dynamically.

=== How do modules deal with multiple inputs in a same port? ===

(And should that even be allowed?)

For compatibility reasons, we do need to allow multiple connections to an input port. However, most package developers should never have to use this, and so we do our best to hide it. the default behavior for getting inputs from a port, then, is to always return a single input.

If on your module you need multiple inputs connected to a single port, use the 'forceGetInputListFromPort' method. It will return a list of all the data items coming through the port. The VTK package uses this feature, so look there for usage examples (packages/vtk/base_widget.py)

=== Are there mechanisms for attaching widgets to different modules/parameters? ===

Right now, we have a mechanism for putting a specific widget for an input port. For example, if a port is SetColor(red, green, blue), we can put a color wheel widget there. Or we can also replace the SetFileName port with a File Widget. However, this is not per parameter (only per port). We are currently working on this problem.

=== Can I organize my package so it appears hierarchical in the module palette? ===

Yes. Use the namespace keyword argument when adding the module to the registry. For example,

registry.add_module(MyModule, namespace='MyNamespace')

=== Can I nest namespaces? ===

Yes. Use the '|' character to separate different the hierarchy. For example,

registry.add_module(MyModule, namespace='ParentNamespace|ChildNamespace')

=== Are there shortucts for registry initialization? ===

Yes. If you define _modules as a list of classes in the __init__.py file of your package, VisTrails will attempt to load all classes specified as modules. You can provide add_module options as keyword arguments by specifying a tuple (class, kwargs) in the list. For example:

_modules = [MyModule1, (MyModule2, {'namespace': 'MyNamespace'})]

In addition, you need to identify the ports of your modules as a field in your class by defining _input_ports and _output_ports lists. Here, the items in each list must be tuples of the form (portName, portSignature, optional=False, sort_key=-1). For example:

class MyModule(Module):
def compute(self):
pass

_input_ports = [('firstInput', String), ('secondInput', Integer, True)]
_output_ports = [('firstOutput', String), ('secondOutput', String)]

=== Can I define ports to be of types that I do not import into my package? ===

Yes. You can pass an identifier string as the portSignature instead. The port_signature string is defined by:

<module_string> := <package_identifier>:[<namespace>|]<module_name>,
<port_signature> := (<module_string>*)

For example,

registry.add_input_port(MyModule, 'myInputPort', '(edu.utah.sci.vistrails.basic:String)')

or

_input_ports = [('myInputPort', '(edu.utah.sci.vistrails.basic:String)')]

=== What do I need to change in my package to make it reloadable (new in v1.4.2)? ===
See [http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#How_to_make_you_package_reloadable UsersGuideVisTrailsPackages] for an explanation.

=== Can I add default values or labels for parameters? ===

Yes. Versions 1.4 and greater support these features. See [[UsersGuideVisTrailsPackages#Adding_default_values_and.2For_labels_for_parameters | UsersGuideVisTrailsPacakges]] for more details.

=== I want to write a module to load HDF data whose output (e.g., data, string) varies according to the input I give it. Is is possible to do this in VisTrails, and if yes, how can I do that? Ideally, I would like to avoid having to change the connection of my output every time I change the input. ===

There are a few ways to tackle this - each has it's own benefits and pitfalls. Firstly, module connections do respect class hierarchies as we're familiar with in object oriented languages. For instance, A module can output a Constant of which String, Float, Integer, etc are specifications. In this way, you can have a subclass of something like HDFData be passed out of the module and the connections will be established regardless of the sub-type. This is a bit dangerous though. Modules downstream of such a class may not really know how to operate on certain types derived from the super-class. Extreme care must be taken both when creating the modules as well as connecting them to prevent things like this from happening.

A second method that I employ in several different packages is the idea of a container class. For instance, the NumSciPy package uses a relatively generic container "Numpy Array" to encapsulate the data. Of course, these encapsulating objects can store dictionaries that other modules can easily access and understand how to operate on. Although this method is slightly more work, the benefits of a stricter typing of ports is beneficial - particularly upon interfacing with other packages that may depend on strongly typed constants (for example).

== The Console ==

=== Where should I go to find out what I can call from the console and how to import it? ===

We have tried to make some methods more accessible in the console via an api. You can import the api via <code>import api</code> in the console and see the available methods with <code>dir(api)</code>. To open a vistrail:

import api
api.open_vistrail_from_file('/Applications/VisTrails/examples/terminator.vt')

To execute a version of a workflow, you currently have to go through the controller:

api.select_version('Histogram')
api.get_current_controller().execute_current_workflow()

Currently, only a subset of VisTrails functionality is directly available from the api. However, since VisTrails is written in python, you can dig down starting with the VistrailsApplication or controller object to expose most of our internal methods. If you have suggestions for calls to be added to the api, please let us know.

One other feature that we're working on, but is still in progress is the ability to construct workflows via the console. For example:

vtk = load_package('edu.utah.sci.vistrails.vtk')
vtk.vtkDataSetReader() # adds a vtkDataSetReader module to the pipeline
# click on the new module
a = selected_modules()[0] # get the one currently selected module
a.SetFile('/vistrails/examples/data/head120.vtk') # sets the SetFile parmaeter for the data set reader
b = vtk.vtkContourFilter() # adds a vtkContourFilter module to the pipeline and saves to var b
b.SetInputConnection0(a.GetOutputPort0()) # connects a's GetOutputPort0 port to b's SetInputConnection0

== Persistence Package ==

=== How do I use the output of one workflow as the input for another using the persistence package? ===

You need to configure the persistence modules using the module's configuration dialog. After adding a <code>PersistentOutputFile</code> to the workflow, click on the triangle in the upper-right corner of the <code>PersistentOutputFile</code>, and select "Edit Configuration" from the menu that appears. In this dialog, select "Create New Reference" and give the reference a name (and any space-delimited tags). Upon running that workflow, the data will be written to the persistent store. In the second workflow where you wish to use that file, add a <code>PersistentInputFile</code> and go to its configuration dialog in the same manner as with the output file. In that dialog, select "Use Existing Reference" and select the data that you just added in the first workflow in the list of files below. Now, when you run that workflow, it will grab the data from the persistent store.

Here is an example: [[Media:offscreen_persistent.vt | offscreen_persistent.vt]]. Run the "persistent offscreen" workflow first, and then run the "display persistent output" to use the output of the first workflow as the input for the second.

== VTK ==

=== Given a VTK visualization, how can I generate a webpage from it? ===

Check out the html pipeline in offscreen.xml.

=== I'm trying to use VTK, but there doesn't seem to be any output. What is wrong? ===

To use VTK on VisTrails, you need a slightly different way of connecting the renderer modules. Instead of using the standard RenderWindow/RenderWindowInteractor infrastructure, you simply connect the renderer to a VTKCell. The examples directory in the distribution has several VTK examples that illustrate.

=== I am trying to add a module to the workflow via Python, but how can I access vtk modules? ===

Here's an example:

import api

vtvtk = 'edu.utah.sci.vistrails.vtk'

module = api.add_module(0, 0, vtvtk, 'vtkContourFilter', '')

The third argument in add_module is the package identifier. You can find this in the "Module Packages" panel of the Preferences; just click on the package you're interested in and it will appear in the information on the right.

== matplotlib ==

=== I'm experiencing a problem with Latex labels and the matplotlib that comes with VisTrails 1.5. The script below entered to the interpreter that comes with VT is sufficient to reproduce it. ===

import matplotlib.pyplot as plt
plt.plot([1,2,3],[1,2,3])
plt.xlabel("$foo$")

Remove your ~/.matplotlib folder and re-start VisTrails

== VisTrails Development ==

=== I would like to build VisTrails from source. Are there instructions on how to do this? ===

Yes! Take a look at http://www.vistrails.org/index.php/Mac_Intel_Instructions
[http://www.vistrails.org/usersguide/getting_started.html#installing-vistrails-from-source Installing VisTrails from source]

== Accessing Provenance Information ==

=== How do I access the information in the execution log? ===

The code responsible for storing execution information is located in the "core/log" directories, and the code that generates much of that information is in "core/interpreter/cached.py". Modules can add execution-specific annotations to provenance via annotate() calls during execution, but much of the data (like timing and errors) is captured by the LogController and CachedInterpreter (the execution engine) objects. To analyze the log from a vistrail (.vt) file, you might have something like the following:

<code>
import core.log.log
import db.services.io

def run(fname):
# open the .vt bundle specified by the filename "fname"
bundle = db.services.io.open_vistrail_bundle_from_zip_xml(fname)[0]
# get the log filename
log_fname = bundle.vistrail.db_log_filename
if log_fname is not None:
# open the log
log = db.services.io.open_log_from_xml(log_fname, True)
# convert the log from a db object
core.log.log.Log.convert(log)
for workflow_exec in log.workflow_execs:
print 'workflow version:', workflow_exec.parent_version
print 'time started:', workflow_exec.ts_start
print 'time ended:', workflow_exec.ts_end
print 'modules executed:', [i.module_id
for i in workflow_exec.item_execs]
if __name__ == '__main__':
run("some_vistrail.vt")
</code>

You should be able to see what information is available by looking at the "core/log" classes.

FAQ

2011-03-25T22:21:58Z

Ctuttle: /* Running workflows */

Also check our [[known_issues|Known Issues]] page for troubleshooting.

----
== Running workflows ==

=== How can I run a workflow using the command line? ===

('''Updated for version 1.2''') Call vistrails using the following options:

<code>python vistrails.py -b path_to_vistrails_file:pipeline </code>

where pipeline can be a version '''tag name''' or version '''id'''

'''NOTE:''' If you downloaded the MacOS X bundle, you can run vistrails from the command line via the following commands in the Terminal. Change the current directory to wherever VisTrails was installed (often <code>/Applications</code>), and then type:

<code>Vistrails.app/Contents/MacOS/vistrails [<cmd_line_options>]</code> [http://www.vistrails.org/usersguide/batch.html#running-vistrails-in-batch-mode Running Vistrails in Batch Mode]

=== Using the command line, we'd like to execute a workflow multiple times, with slightly different parameters, and create a series of output files. Is this possible?===

('''Updated for version 1.2''') We can change parameters that have an alias through the command line.

For example, offscreen pipeline in offscreen.vt always creates the file called image.png. If you want generate it with a different filename:

<code>python vistrails.py -b ../examples/offscreen.vt:offscreen -a"filename=other.png"</code>

<code>filename</code> in the example above is the alias name assigned to the parameter in the <code>value</code> method inside the String module. When running a pipeline from the command line, VisTrails will try to start the spreadsheet automatically if the pipeline requires it. For example, this other execution will also start the spreadsheet (attention to how $ characters are escaped when running on bash):

<code>python vistrails.py -b ../examples/head.vt:aliases -a"isovalue=30\$&\$diffuse_color=0.8,0.4,0.2"</code>

You can also execute more than one pipeline on the command line:

<code>python vistrails.py -b ../examples/head.vt:aliases ../examples/spx.vt:spx \
-a"isovalue=30"</code>

Use the -a parameter only once regardless the number of pipelines.

=== I can load a vistrail, and the version tree shows up fine. However, no pipelines appear when I click on a version. What gives? ===

The most likely reason is that the vistrail uses a package that is not registered with VisTrails. You need to identify the needed package and add it to your <code>.vistrails/startup.py</code>. A single line like the following should be enough:

addPackage('enter_package_name_here')

Some packages might need more information. For example:

addPackage('afront', executable_path='/path/to/afront')

Refer to the package documentation for details. The one inconvenient step is that currently there's no automated way to describe what is the missing package. We're working on this feature for future releases.

=== I have a workflow that reads a file and then does some processing. The first time it runs, it executes correctly. But in subsequent, nothing happens. ===

VisTrails caches by default, so after a workflow is executed, if none of its parameters change, it won't be executed again.

If a workflow reads a file using the basic module File, VisTrails does check whether the file was modified since the last run. It does so by keeping a signature that is based on the modification time of the file. And if the file was modified, the File module and all downstream modules (the ones which depend on File) will be executed.

''Note: If you would like your input and output data to be versioned, you can use the Persistence package.''

If you do not want VisTrails to cache executions, you can turn off caching: go to Menu Edit -> Preferences and in the General Configuration tab, change Cache execution results to Never. [http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

=== Can VisTrails execute workflows in parallel? ===

The VisTrails server can only execute pipelines in parallel if there's more than one instance of VisTrails running. The command

<code> self.rpcserver = ThreadedXMLRPCServer((self.temp_xml_rpc_options.server, self.temp_xml_rpc_options.port))</code>

starts a multithreaded version of the XML-RPC server, so it will create a thread for each request received by the server. The problem is that Qt/PyQT doesn't allow these multiple threads create GUI objects, only in the main thread. To overcome this limitation, the multithreaded version can instantiate other single threaded versions of VisTrails and put them in a queue, so workflow executions and other GUI-related requests, such as generating workflow graphs and history trees can be forwarded to this queue, and each instance takes turns in answering the request. If the results are in the cache, the multithreaded version answers the requests directly.

Note that this infrastructure works on Linux only. To make this work on Windows, you have to create a script similar to start_vistrails_xvfb.sh (located in the scripts folder) where you can send the number of other instances via command-line options to VisTrails. The command line options are:

python vistrails_server.py -T <ADDRESS> -R <PORT> -O<NUMBER_OF_OTHER_VISTRAILS_INSTANCES> [-M]&

If you want the main vistrails instance to be multithreaded, use the -M at the end.

After creating this script, update function start_other_instances in vistrails/gui/application_server.py lines 1007-1023 and set the script variable to point to your script. You may also have to change the arguments sent to your script (line 1016: for example, you don't need to set a virtual display). You will need to change the path to the stop_vistrails_server.py script (on line 1026) according to your installation path.

=== When a workflow is executed, what do the colors mean? ===

- lilac: module was notexecuted

- yellow: module is currently being executed

- green: module was successfully executed

- orange: module was cached

- red: the execution of the module failed

[http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

== Building workflows ==

=== Is there a way to give each widget a "display name" in addition to the module name at the center of the widget? ===

Yes, a "display name" can be assigned to a module by selecting the triangle in its top right corner to open a popup menu and selecting the Set Module Label... menu item. You will then be prompted to enter the "display name". [http://www.vistrails.org/usersguide/creating.html#changing-module-labels Changing Module Labels]

=== Is there a way to re-center the picture-in-picture (PiP) view? ===

Yes. If you click on the PIP window to bring it to focus, you can press Ctrl-R (or Command-R on Mac) to re-center the PiP window. [http://www.vistrails.org/usersguide/getting_started.html#vistrails-interaction Vistrails Interaction]

=== How do I search for a literal "?" (question mark) in the search box in the Property panel? ===

Since we allow regular expressions in our search box, question marks are treated as meta-characters. Thus, searching for "?" returns everything and "abc?" will return everything containing "abc". You need to use "\?" instead to search for "?". So the search for "??" would be "\?\?". [http://www.vistrails.org/usersguide/querying.html#textual-queries Textual Queries]

== Using VisTrails as a server ==

=== What is the VisTrails server-mode? ===

Using the VisTrails server mode, it is possible to ''execute workflows and control VisTrails through another application''. For example, the CrowdLabs Web portal (http://www.crowdlabs.org) accesses a VisTrails sever to execute workflows, retrieve and display vistrail trees and workflows. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

=== How do I execute workflows and control VisTrails through another application? ===

The way you access the server is by doing XML-RPC calls. In the current VisTrails release, we include a set of PHP scripts that can talk to a VisTrails server instance. They are in "extensions/http" folder. The files are reasonably well documented. Also, it should be not difficult to create python scripts to access the server (just use xmlrpclib module).

Note that the VisTrails server requires the provenance and workflows to be in a database. More detailed instructions on how to setup the server and the database
are available here:

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_server_setup.html

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_database_setup.html

If what you want is just to execute a series of workflows in batch mode, a simpler solution would be to ''use the VisTrails client in batch mode''. Chapter 12 of the user's guide contains detailed information and examples on that. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

== Control Flow ==

=== Note: using map ===

When using 'map', the module (or subworkflow) used as function port in the map module MUST be a function, i.e., it can only define 1 output port. [http://www.vistrails.org/usersguide/controlflow.html#the-map-operator The Map Operator]

== Spreadsheet ==

Where ''pipeline'' is a version number or a tag.

=== How can I save an image from the spreadsheet? ===

While having the focus on a spreadsheet cell and select the camera on the toolbar to take a snapshot. The system will prompt you for the location and file name where it should be saved. The other icons can be used for saving multiple images that can be used for generating an animation on demand. A whole sheet can also be saved by selecting Export (either from the menu or from the toolbar). [http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet-image Saving a Spreadsheet Image]

===Is it possible to save the complete state of the spreadsheet?===
[http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet Saving a Spreadsheet]

=== Can I view multiple sheets at the same time? ===

Yes. Each sheet on the spreadsheet can be displayed as a dock widget separated from the main spreadsheet window by dragging its tab name out of the tab bar at the bottom of the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Then, how can I put back a separated sheet? ===

A sheet can be docked back to the main window by dragging it back to the tab bar or double-click on its title bar. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== How can I order sheets on the spreadsheet? ===

This can be done by dragging the sheet name on the bottom top bar and drop it to the right place. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Can I control where a cell will be placed on the spreadsheet window? ===

By default, an unoccupied cell on the active sheet will be chosen to display the result. However, you can specify exactly in the pipeline where a spreadsheet cell will be placed by using CellLocation and SheetReference. CellLocation specifies the location (row and column) of a cell when connecting to a spreadsheet cell (VTKCell, ImageViewerCell, ...). Similarly, a SheetReference module (when connecting to a CellLocation) will specify which sheet the cell will be put on given its name, minimum row size and minimum column size. There is an example of this in examples/vtk.xml (select the version below Double Renderer). [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

=== How do I output results to the spreadsheet? ===

By inspecting the VisTrails Spreadsheet package (in the list of packages, to the left of the pipeline builder), you
can see there are built-in cells for different kinds of data, e.g., RichTextCell to display HTML and plain text. op
You (the user) can also define new cell types to display application-specific data. For example, we have developed
VtkCell, MplFigureCell, and OpenGLCell. It is possible to display pretty much anything on the Spreadsheet! [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

Examples of writing cell modules can be found in:
RichTextCell: packages/spreadsheet/widgets/richtext/richtext.py
VTK: packages/vtk/vtkcell.py

Here is the summary of some requirements on a cell widget:

(1) It must be a Qt widget. It should inherit from spreadsheet_cell.QCellWidget in the spreadsheet package. Although any Qt Widget would work, certain features such as animation will not be available (without rewriting it).

(2) It must re-implement the updateContents() function to take a set of inputs (usually coming from input ports of a wrapper Module) and display on the cells. VisTrails uses this function to update/reuse cells on the spreadsheet when new data comes in.

(3) It needs a wrapper VisTrails Module (inherited from basic_widgets.SpreadsheetCell of the spreadsheet package). Inside the compute() method of this module, it may call self.display(CellWidgetType, (inputs)) to trigger the display event on the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#advanced-cell-options Advanced Cell Options]

=== How do I control the default number of cells in the spreadsheet? ===

You can configure the rowCount and colCount using the preferences dialog. Just go to the Module Packages tab, select spreadsheet in the "Enabled packages" and press the Configure button. Then a list of all the configuration options for the spreadsheet will show up. [http://www.vistrails.org/usersguide/spreadsheet.html#custom-layout-options Custom Layout Options]

=== Is it possible to launch a web browser from the vistrails spreadsheet? We would like to output several urls from a parameter sweep and then have the option to click on each one to view the resulting page. I can view the page within the spreadsheet, but it is really too crowded.===

Currently, there isn't a widget that provides exactly this functionality, but I can think of a few solutions that may work for you:

(1) You can use parameter exploration to generate multiple sheets so you might have an exploration that opens each page in a new sheet. Use the third column/dimension in the exploration interface to have a parameter span sheets.

(2) The spreadsheet is extensible so you can write a custom spreadsheet cell widget that has a button or label with the desired link (a QLabel with openExternalLinks set to True, for example).

(3) You can tweak the existing RichTextCell be adding the line "self.browser.setOpenExternalLinks(True)" at line 63 of the source file "vistrails/packages/spreadsheet/widgets/richtext/richtext.py". Then, if your workflow creates a file with html markup text like "<a href="http://www.vistrails.org">VisTrails</a>" connected to a RichTextCell, clicking on the rendered link in the cell will open it in a web browser. You need to add the aforementioned line to the source to let Qt know that you want the link opened externally; by default, it will just issue an event that isn't processed. [http://www.vistrails.org/usersguide/spreadsheet.html#launching-a-web-browser Launching a Web Browser]

== Integrating your software into VisTrails ==

===How can I integrate my own program into VisTrails?===

The easiest way is to create a package. Writing a package is often very simple, here are instructions on how to do it: [[UsersGuideVisTrailsPackages]]

You can also dynamically generate modules. For an example see:

http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#Packages_that_generate_modules_dynamically

In particular, see the new_module call which uses python's type() function to generate new classes dynamically.

=== How do modules deal with multiple inputs in a same port? ===

(And should that even be allowed?)

For compatibility reasons, we do need to allow multiple connections to an input port. However, most package developers should never have to use this, and so we do our best to hide it. the default behavior for getting inputs from a port, then, is to always return a single input.

If on your module you need multiple inputs connected to a single port, use the 'forceGetInputListFromPort' method. It will return a list of all the data items coming through the port. The VTK package uses this feature, so look there for usage examples (packages/vtk/base_widget.py)

=== Are there mechanisms for attaching widgets to different modules/parameters? ===

Right now, we have a mechanism for putting a specific widget for an input port. For example, if a port is SetColor(red, green, blue), we can put a color wheel widget there. Or we can also replace the SetFileName port with a File Widget. However, this is not per parameter (only per port). We are currently working on this problem.

=== Can I organize my package so it appears hierarchical in the module palette? ===

Yes. Use the namespace keyword argument when adding the module to the registry. For example,

registry.add_module(MyModule, namespace='MyNamespace')

=== Can I nest namespaces? ===

Yes. Use the '|' character to separate different the hierarchy. For example,

registry.add_module(MyModule, namespace='ParentNamespace|ChildNamespace')

=== Are there shortucts for registry initialization? ===

Yes. If you define _modules as a list of classes in the __init__.py file of your package, VisTrails will attempt to load all classes specified as modules. You can provide add_module options as keyword arguments by specifying a tuple (class, kwargs) in the list. For example:

_modules = [MyModule1, (MyModule2, {'namespace': 'MyNamespace'})]

In addition, you need to identify the ports of your modules as a field in your class by defining _input_ports and _output_ports lists. Here, the items in each list must be tuples of the form (portName, portSignature, optional=False, sort_key=-1). For example:

class MyModule(Module):
def compute(self):
pass

_input_ports = [('firstInput', String), ('secondInput', Integer, True)]
_output_ports = [('firstOutput', String), ('secondOutput', String)]

=== Can I define ports to be of types that I do not import into my package? ===

Yes. You can pass an identifier string as the portSignature instead. The port_signature string is defined by:

<module_string> := <package_identifier>:[<namespace>|]<module_name>,
<port_signature> := (<module_string>*)

For example,

registry.add_input_port(MyModule, 'myInputPort', '(edu.utah.sci.vistrails.basic:String)')

or

_input_ports = [('myInputPort', '(edu.utah.sci.vistrails.basic:String)')]

=== What do I need to change in my package to make it reloadable (new in v1.4.2)? ===
See [http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#How_to_make_you_package_reloadable UsersGuideVisTrailsPackages] for an explanation.

=== Can I add default values or labels for parameters? ===

Yes. Versions 1.4 and greater support these features. See [[UsersGuideVisTrailsPackages#Adding_default_values_and.2For_labels_for_parameters | UsersGuideVisTrailsPacakges]] for more details.

=== I want to write a module to load HDF data whose output (e.g., data, string) varies according to the input I give it. Is is possible to do this in VisTrails, and if yes, how can I do that? Ideally, I would like to avoid having to change the connection of my output every time I change the input. ===

There are a few ways to tackle this - each has it's own benefits and pitfalls. Firstly, module connections do respect class hierarchies as we're familiar with in object oriented languages. For instance, A module can output a Constant of which String, Float, Integer, etc are specifications. In this way, you can have a subclass of something like HDFData be passed out of the module and the connections will be established regardless of the sub-type. This is a bit dangerous though. Modules downstream of such a class may not really know how to operate on certain types derived from the super-class. Extreme care must be taken both when creating the modules as well as connecting them to prevent things like this from happening.

A second method that I employ in several different packages is the idea of a container class. For instance, the NumSciPy package uses a relatively generic container "Numpy Array" to encapsulate the data. Of course, these encapsulating objects can store dictionaries that other modules can easily access and understand how to operate on. Although this method is slightly more work, the benefits of a stricter typing of ports is beneficial - particularly upon interfacing with other packages that may depend on strongly typed constants (for example).

== The Console ==

=== Where should I go to find out what I can call from the console and how to import it? ===

We have tried to make some methods more accessible in the console via an api. You can import the api via <code>import api</code> in the console and see the available methods with <code>dir(api)</code>. To open a vistrail:

import api
api.open_vistrail_from_file('/Applications/VisTrails/examples/terminator.vt')

To execute a version of a workflow, you currently have to go through the controller:

api.select_version('Histogram')
api.get_current_controller().execute_current_workflow()

Currently, only a subset of VisTrails functionality is directly available from the api. However, since VisTrails is written in python, you can dig down starting with the VistrailsApplication or controller object to expose most of our internal methods. If you have suggestions for calls to be added to the api, please let us know.

One other feature that we're working on, but is still in progress is the ability to construct workflows via the console. For example:

vtk = load_package('edu.utah.sci.vistrails.vtk')
vtk.vtkDataSetReader() # adds a vtkDataSetReader module to the pipeline
# click on the new module
a = selected_modules()[0] # get the one currently selected module
a.SetFile('/vistrails/examples/data/head120.vtk') # sets the SetFile parmaeter for the data set reader
b = vtk.vtkContourFilter() # adds a vtkContourFilter module to the pipeline and saves to var b
b.SetInputConnection0(a.GetOutputPort0()) # connects a's GetOutputPort0 port to b's SetInputConnection0

== Persistence Package ==

=== How do I use the output of one workflow as the input for another using the persistence package? ===

You need to configure the persistence modules using the module's configuration dialog. After adding a <code>PersistentOutputFile</code> to the workflow, click on the triangle in the upper-right corner of the <code>PersistentOutputFile</code>, and select "Edit Configuration" from the menu that appears. In this dialog, select "Create New Reference" and give the reference a name (and any space-delimited tags). Upon running that workflow, the data will be written to the persistent store. In the second workflow where you wish to use that file, add a <code>PersistentInputFile</code> and go to its configuration dialog in the same manner as with the output file. In that dialog, select "Use Existing Reference" and select the data that you just added in the first workflow in the list of files below. Now, when you run that workflow, it will grab the data from the persistent store.

Here is an example: [[Media:offscreen_persistent.vt | offscreen_persistent.vt]]. Run the "persistent offscreen" workflow first, and then run the "display persistent output" to use the output of the first workflow as the input for the second.

== VTK ==

=== Given a VTK visualization, how can I generate a webpage from it? ===

Check out the html pipeline in offscreen.xml.

=== I'm trying to use VTK, but there doesn't seem to be any output. What is wrong? ===

To use VTK on VisTrails, you need a slightly different way of connecting the renderer modules. Instead of using the standard RenderWindow/RenderWindowInteractor infrastructure, you simply connect the renderer to a VTKCell. The examples directory in the distribution has several VTK examples that illustrate.

=== I am trying to add a module to the workflow via Python, but how can I access vtk modules? ===

Here's an example:

import api

vtvtk = 'edu.utah.sci.vistrails.vtk'

module = api.add_module(0, 0, vtvtk, 'vtkContourFilter', '')

The third argument in add_module is the package identifier. You can find this in the "Module Packages" panel of the Preferences; just click on the package you're interested in and it will appear in the information on the right.

== matplotlib ==

=== I'm experiencing a problem with Latex labels and the matplotlib that comes with VisTrails 1.5. The script below entered to the interpreter that comes with VT is sufficient to reproduce it. ===

import matplotlib.pyplot as plt
plt.plot([1,2,3],[1,2,3])
plt.xlabel("$foo$")

Remove your ~/.matplotlib folder and re-start VisTrails

== VisTrails Development ==

=== I would like to build VisTrails from source. Are there instructions on how to do this? ===

Yes! Take a look at http://www.vistrails.org/index.php/Mac_Intel_Instructions
[http://www.vistrails.org/usersguide/getting_started.html#installing-vistrails-from-source Installing VisTrails from source]

== Accessing Provenance Information ==

=== How do I access the information in the execution log? ===

The code responsible for storing execution information is located in the "core/log" directories, and the code that generates much of that information is in "core/interpreter/cached.py". Modules can add execution-specific annotations to provenance via annotate() calls during execution, but much of the data (like timing and errors) is captured by the LogController and CachedInterpreter (the execution engine) objects. To analyze the log from a vistrail (.vt) file, you might have something like the following:

<code>
import core.log.log
import db.services.io

def run(fname):
# open the .vt bundle specified by the filename "fname"
bundle = db.services.io.open_vistrail_bundle_from_zip_xml(fname)[0]
# get the log filename
log_fname = bundle.vistrail.db_log_filename
if log_fname is not None:
# open the log
log = db.services.io.open_log_from_xml(log_fname, True)
# convert the log from a db object
core.log.log.Log.convert(log)
for workflow_exec in log.workflow_execs:
print 'workflow version:', workflow_exec.parent_version
print 'time started:', workflow_exec.ts_start
print 'time ended:', workflow_exec.ts_end
print 'modules executed:', [i.module_id
for i in workflow_exec.item_execs]
if __name__ == '__main__':
run("some_vistrail.vt")
</code>

You should be able to see what information is available by looking at the "core/log" classes.

FAQ

2011-03-25T17:39:21Z

Ctuttle: /* Running workflows */

Also check our [[known_issues|Known Issues]] page for troubleshooting.

----
== Running workflows ==

=== How can I run a workflow using the command line? ===

('''Updated for version 1.2''') Call vistrails using the following options:

<code>python vistrails.py -b path_to_vistrails_file:pipeline </code>

where pipeline can be a version '''tag name''' or version '''id'''

'''NOTE:''' If you downloaded the MacOS X bundle, you can run vistrails from the command line via the following commands in the Terminal. Change the current directory to wherever VisTrails was installed (often <code>/Applications</code>), and then type:

<code>Vistrails.app/Contents/MacOS/vistrails [<cmd_line_options>]</code>

=== Using the command line, we'd like to execute a workflow multiple times, with slightly different parameters, and create a series of output files. Is this possible?===

('''Updated for version 1.2''') We can change parameters that have an alias through the command line.

For example, offscreen pipeline in offscreen.vt always creates the file called image.png. If you want generate it with a different filename:

<code>python vistrails.py -b ../examples/offscreen.vt:offscreen -a"filename=other.png"</code>

<code>filename</code> in the example above is the alias name assigned to the parameter in the <code>value</code> method inside the String module. When running a pipeline from the command line, VisTrails will try to start the spreadsheet automatically if the pipeline requires it. For example, this other execution will also start the spreadsheet (attention to how $ characters are escaped when running on bash):

<code>python vistrails.py -b ../examples/head.vt:aliases -a"isovalue=30\$&\$diffuse_color=0.8,0.4,0.2"</code>

You can also execute more than one pipeline on the command line:

<code>python vistrails.py -b ../examples/head.vt:aliases ../examples/spx.vt:spx \
-a"isovalue=30"</code>

Use the -a parameter only once regardless the number of pipelines.

=== I can load a vistrail, and the version tree shows up fine. However, no pipelines appear when I click on a version. What gives? ===

The most likely reason is that the vistrail uses a package that is not registered with VisTrails. You need to identify the needed package and add it to your <code>.vistrails/startup.py</code>. A single line like the following should be enough:

addPackage('enter_package_name_here')

Some packages might need more information. For example:

addPackage('afront', executable_path='/path/to/afront')

Refer to the package documentation for details. The one inconvenient step is that currently there's no automated way to describe what is the missing package. We're working on this feature for future releases.

=== I have a workflow that reads a file and then does some processing. The first time it runs, it executes correctly. But in subsequent, nothing happens. ===

VisTrails caches by default, so after a workflow is executed, if none of its parameters change, it won't be executed again.

If a workflow reads a file using the basic module File, VisTrails does check whether the file was modified since the last run. It does so by keeping a signature that is based on the modification time of the file. And if the file was modified, the File module and all downstream modules (the ones which depend on File) will be executed.

''Note: If you would like your input and output data to be versioned, you can use the Persistence package.''

If you do not want VisTrails to cache executions, you can turn off caching: go to Menu Edit -> Preferences and in the General Configuration tab, change Cache execution results to Never. [http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

=== Can VisTrails execute workflows in parallel? ===

The VisTrails server can only execute pipelines in parallel if there's more than one instance of VisTrails running. The command

<code> self.rpcserver = ThreadedXMLRPCServer((self.temp_xml_rpc_options.server, self.temp_xml_rpc_options.port))</code>

starts a multithreaded version of the XML-RPC server, so it will create a thread for each request received by the server. The problem is that Qt/PyQT doesn't allow these multiple threads create GUI objects, only in the main thread. To overcome this limitation, the multithreaded version can instantiate other single threaded versions of VisTrails and put them in a queue, so workflow executions and other GUI-related requests, such as generating workflow graphs and history trees can be forwarded to this queue, and each instance takes turns in answering the request. If the results are in the cache, the multithreaded version answers the requests directly.

Note that this infrastructure works on Linux only. To make this work on Windows, you have to create a script similar to start_vistrails_xvfb.sh (located in the scripts folder) where you can send the number of other instances via command-line options to VisTrails. The command line options are:

python vistrails_server.py -T <ADDRESS> -R <PORT> -O<NUMBER_OF_OTHER_VISTRAILS_INSTANCES> [-M]&

If you want the main vistrails instance to be multithreaded, use the -M at the end.

After creating this script, update function start_other_instances in vistrails/gui/application_server.py lines 1007-1023 and set the script variable to point to your script. You may also have to change the arguments sent to your script (line 1016: for example, you don't need to set a virtual display). You will need to change the path to the stop_vistrails_server.py script (on line 1026) according to your installation path.

=== When a workflow is executed, what do the colors mean? ===

- lilac: module was notexecuted

- yellow: module is currently being executed

- green: module was successfully executed

- orange: module was cached

- red: the execution of the module failed

[http://www.vistrails.org/usersguide/getting_started.html#workflow-execution Workflow Execution]

== Building workflows ==

=== Is there a way to give each widget a "display name" in addition to the module name at the center of the widget? ===

Yes, a "display name" can be assigned to a module by selecting the triangle in its top right corner to open a popup menu and selecting the Set Module Label... menu item. You will then be prompted to enter the "display name". [http://www.vistrails.org/usersguide/creating.html#changing-module-labels Changing Module Labels]

=== Is there a way to re-center the picture-in-picture (PiP) view? ===

Yes. If you click on the PIP window to bring it to focus, you can press Ctrl-R (or Command-R on Mac) to re-center the PiP window. [http://www.vistrails.org/usersguide/getting_started.html#vistrails-interaction Vistrails Interaction]

=== How do I search for a literal "?" (question mark) in the search box in the Property panel? ===

Since we allow regular expressions in our search box, question marks are treated as meta-characters. Thus, searching for "?" returns everything and "abc?" will return everything containing "abc". You need to use "\?" instead to search for "?". So the search for "??" would be "\?\?". [http://www.vistrails.org/usersguide/querying.html#textual-queries Textual Queries]

== Using VisTrails as a server ==

=== What is the VisTrails server-mode? ===

Using the VisTrails server mode, it is possible to ''execute workflows and control VisTrails through another application''. For example, the CrowdLabs Web portal (http://www.crowdlabs.org) accesses a VisTrails sever to execute workflows, retrieve and display vistrail trees and workflows. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

=== How do I execute workflows and control VisTrails through another application? ===

The way you access the server is by doing XML-RPC calls. In the current VisTrails release, we include a set of PHP scripts that can talk to a VisTrails server instance. They are in "extensions/http" folder. The files are reasonably well documented. Also, it should be not difficult to create python scripts to access the server (just use xmlrpclib module).

Note that the VisTrails server requires the provenance and workflows to be in a database. More detailed instructions on how to setup the server and the database
are available here:

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_server_setup.html

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_database_setup.html

If what you want is just to execute a series of workflows in batch mode, a simpler solution would be to ''use the VisTrails client in batch mode''. Chapter 12 of the user's guide contains detailed information and examples on that. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

== Control Flow ==

=== Note: using map ===

When using 'map', the module (or subworkflow) used as function port in the map module MUST be a function, i.e., it can only define 1 output port. [http://www.vistrails.org/usersguide/controlflow.html#the-map-operator The Map Operator]

== Spreadsheet ==

Where ''pipeline'' is a version number or a tag.

=== How can I save an image from the spreadsheet? ===

While having the focus on a spreadsheet cell and select the camera on the toolbar to take a snapshot. The system will prompt you for the location and file name where it should be saved. The other icons can be used for saving multiple images that can be used for generating an animation on demand. A whole sheet can also be saved by selecting Export (either from the menu or from the toolbar). [http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet-image Saving a Spreadsheet Image]

===Is it possible to save the complete state of the spreadsheet?===
[http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet Saving a Spreadsheet]

=== Can I view multiple sheets at the same time? ===

Yes. Each sheet on the spreadsheet can be displayed as a dock widget separated from the main spreadsheet window by dragging its tab name out of the tab bar at the bottom of the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Then, how can I put back a separated sheet? ===

A sheet can be docked back to the main window by dragging it back to the tab bar or double-click on its title bar. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== How can I order sheets on the spreadsheet? ===

This can be done by dragging the sheet name on the bottom top bar and drop it to the right place. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Can I control where a cell will be placed on the spreadsheet window? ===

By default, an unoccupied cell on the active sheet will be chosen to display the result. However, you can specify exactly in the pipeline where a spreadsheet cell will be placed by using CellLocation and SheetReference. CellLocation specifies the location (row and column) of a cell when connecting to a spreadsheet cell (VTKCell, ImageViewerCell, ...). Similarly, a SheetReference module (when connecting to a CellLocation) will specify which sheet the cell will be put on given its name, minimum row size and minimum column size. There is an example of this in examples/vtk.xml (select the version below Double Renderer). [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

=== How do I output results to the spreadsheet? ===

By inspecting the VisTrails Spreadsheet package (in the list of packages, to the left of the pipeline builder), you
can see there are built-in cells for different kinds of data, e.g., RichTextCell to display HTML and plain text. op
You (the user) can also define new cell types to display application-specific data. For example, we have developed
VtkCell, MplFigureCell, and OpenGLCell. It is possible to display pretty much anything on the Spreadsheet! [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

Examples of writing cell modules can be found in:
RichTextCell: packages/spreadsheet/widgets/richtext/richtext.py
VTK: packages/vtk/vtkcell.py

Here is the summary of some requirements on a cell widget:

(1) It must be a Qt widget. It should inherit from spreadsheet_cell.QCellWidget in the spreadsheet package. Although any Qt Widget would work, certain features such as animation will not be available (without rewriting it).

(2) It must re-implement the updateContents() function to take a set of inputs (usually coming from input ports of a wrapper Module) and display on the cells. VisTrails uses this function to update/reuse cells on the spreadsheet when new data comes in.

(3) It needs a wrapper VisTrails Module (inherited from basic_widgets.SpreadsheetCell of the spreadsheet package). Inside the compute() method of this module, it may call self.display(CellWidgetType, (inputs)) to trigger the display event on the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#advanced-cell-options Advanced Cell Options]

=== How do I control the default number of cells in the spreadsheet? ===

You can configure the rowCount and colCount using the preferences dialog. Just go to the Module Packages tab, select spreadsheet in the "Enabled packages" and press the Configure button. Then a list of all the configuration options for the spreadsheet will show up. [http://www.vistrails.org/usersguide/spreadsheet.html#custom-layout-options Custom Layout Options]

=== Is it possible to launch a web browser from the vistrails spreadsheet? We would like to output several urls from a parameter sweep and then have the option to click on each one to view the resulting page. I can view the page within the spreadsheet, but it is really too crowded.===

Currently, there isn't a widget that provides exactly this functionality, but I can think of a few solutions that may work for you:

(1) You can use parameter exploration to generate multiple sheets so you might have an exploration that opens each page in a new sheet. Use the third column/dimension in the exploration interface to have a parameter span sheets.

(2) The spreadsheet is extensible so you can write a custom spreadsheet cell widget that has a button or label with the desired link (a QLabel with openExternalLinks set to True, for example).

(3) You can tweak the existing RichTextCell be adding the line "self.browser.setOpenExternalLinks(True)" at line 63 of the source file "vistrails/packages/spreadsheet/widgets/richtext/richtext.py". Then, if your workflow creates a file with html markup text like "<a href="http://www.vistrails.org">VisTrails</a>" connected to a RichTextCell, clicking on the rendered link in the cell will open it in a web browser. You need to add the aforementioned line to the source to let Qt know that you want the link opened externally; by default, it will just issue an event that isn't processed. [http://www.vistrails.org/usersguide/spreadsheet.html#launching-a-web-browser Launching a Web Browser]

== Integrating your software into VisTrails ==

===How can I integrate my own program into VisTrails?===

The easiest way is to create a package. Writing a package is often very simple, here are instructions on how to do it: [[UsersGuideVisTrailsPackages]]

You can also dynamically generate modules. For an example see:

http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#Packages_that_generate_modules_dynamically

In particular, see the new_module call which uses python's type() function to generate new classes dynamically.

=== How do modules deal with multiple inputs in a same port? ===

(And should that even be allowed?)

For compatibility reasons, we do need to allow multiple connections to an input port. However, most package developers should never have to use this, and so we do our best to hide it. the default behavior for getting inputs from a port, then, is to always return a single input.

If on your module you need multiple inputs connected to a single port, use the 'forceGetInputListFromPort' method. It will return a list of all the data items coming through the port. The VTK package uses this feature, so look there for usage examples (packages/vtk/base_widget.py)

=== Are there mechanisms for attaching widgets to different modules/parameters? ===

Right now, we have a mechanism for putting a specific widget for an input port. For example, if a port is SetColor(red, green, blue), we can put a color wheel widget there. Or we can also replace the SetFileName port with a File Widget. However, this is not per parameter (only per port). We are currently working on this problem.

=== Can I organize my package so it appears hierarchical in the module palette? ===

Yes. Use the namespace keyword argument when adding the module to the registry. For example,

registry.add_module(MyModule, namespace='MyNamespace')

=== Can I nest namespaces? ===

Yes. Use the '|' character to separate different the hierarchy. For example,

registry.add_module(MyModule, namespace='ParentNamespace|ChildNamespace')

=== Are there shortucts for registry initialization? ===

Yes. If you define _modules as a list of classes in the __init__.py file of your package, VisTrails will attempt to load all classes specified as modules. You can provide add_module options as keyword arguments by specifying a tuple (class, kwargs) in the list. For example:

_modules = [MyModule1, (MyModule2, {'namespace': 'MyNamespace'})]

In addition, you need to identify the ports of your modules as a field in your class by defining _input_ports and _output_ports lists. Here, the items in each list must be tuples of the form (portName, portSignature, optional=False, sort_key=-1). For example:

class MyModule(Module):
def compute(self):
pass

_input_ports = [('firstInput', String), ('secondInput', Integer, True)]
_output_ports = [('firstOutput', String), ('secondOutput', String)]

=== Can I define ports to be of types that I do not import into my package? ===

Yes. You can pass an identifier string as the portSignature instead. The port_signature string is defined by:

<module_string> := <package_identifier>:[<namespace>|]<module_name>,
<port_signature> := (<module_string>*)

For example,

registry.add_input_port(MyModule, 'myInputPort', '(edu.utah.sci.vistrails.basic:String)')

or

_input_ports = [('myInputPort', '(edu.utah.sci.vistrails.basic:String)')]

=== What do I need to change in my package to make it reloadable (new in v1.4.2)? ===
See [http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#How_to_make_you_package_reloadable UsersGuideVisTrailsPackages] for an explanation.

=== Can I add default values or labels for parameters? ===

Yes. Versions 1.4 and greater support these features. See [[UsersGuideVisTrailsPackages#Adding_default_values_and.2For_labels_for_parameters | UsersGuideVisTrailsPacakges]] for more details.

=== I want to write a module to load HDF data whose output (e.g., data, string) varies according to the input I give it. Is is possible to do this in VisTrails, and if yes, how can I do that? Ideally, I would like to avoid having to change the connection of my output every time I change the input. ===

There are a few ways to tackle this - each has it's own benefits and pitfalls. Firstly, module connections do respect class hierarchies as we're familiar with in object oriented languages. For instance, A module can output a Constant of which String, Float, Integer, etc are specifications. In this way, you can have a subclass of something like HDFData be passed out of the module and the connections will be established regardless of the sub-type. This is a bit dangerous though. Modules downstream of such a class may not really know how to operate on certain types derived from the super-class. Extreme care must be taken both when creating the modules as well as connecting them to prevent things like this from happening.

A second method that I employ in several different packages is the idea of a container class. For instance, the NumSciPy package uses a relatively generic container "Numpy Array" to encapsulate the data. Of course, these encapsulating objects can store dictionaries that other modules can easily access and understand how to operate on. Although this method is slightly more work, the benefits of a stricter typing of ports is beneficial - particularly upon interfacing with other packages that may depend on strongly typed constants (for example).

== The Console ==

=== Where should I go to find out what I can call from the console and how to import it? ===

We have tried to make some methods more accessible in the console via an api. You can import the api via <code>import api</code> in the console and see the available methods with <code>dir(api)</code>. To open a vistrail:

import api
api.open_vistrail_from_file('/Applications/VisTrails/examples/terminator.vt')

To execute a version of a workflow, you currently have to go through the controller:

api.select_version('Histogram')
api.get_current_controller().execute_current_workflow()

Currently, only a subset of VisTrails functionality is directly available from the api. However, since VisTrails is written in python, you can dig down starting with the VistrailsApplication or controller object to expose most of our internal methods. If you have suggestions for calls to be added to the api, please let us know.

One other feature that we're working on, but is still in progress is the ability to construct workflows via the console. For example:

vtk = load_package('edu.utah.sci.vistrails.vtk')
vtk.vtkDataSetReader() # adds a vtkDataSetReader module to the pipeline
# click on the new module
a = selected_modules()[0] # get the one currently selected module
a.SetFile('/vistrails/examples/data/head120.vtk') # sets the SetFile parmaeter for the data set reader
b = vtk.vtkContourFilter() # adds a vtkContourFilter module to the pipeline and saves to var b
b.SetInputConnection0(a.GetOutputPort0()) # connects a's GetOutputPort0 port to b's SetInputConnection0

== Persistence Package ==

=== How do I use the output of one workflow as the input for another using the persistence package? ===

You need to configure the persistence modules using the module's configuration dialog. After adding a <code>PersistentOutputFile</code> to the workflow, click on the triangle in the upper-right corner of the <code>PersistentOutputFile</code>, and select "Edit Configuration" from the menu that appears. In this dialog, select "Create New Reference" and give the reference a name (and any space-delimited tags). Upon running that workflow, the data will be written to the persistent store. In the second workflow where you wish to use that file, add a <code>PersistentInputFile</code> and go to its configuration dialog in the same manner as with the output file. In that dialog, select "Use Existing Reference" and select the data that you just added in the first workflow in the list of files below. Now, when you run that workflow, it will grab the data from the persistent store.

Here is an example: [[Media:offscreen_persistent.vt | offscreen_persistent.vt]]. Run the "persistent offscreen" workflow first, and then run the "display persistent output" to use the output of the first workflow as the input for the second.

== VTK ==

=== Given a VTK visualization, how can I generate a webpage from it? ===

Check out the html pipeline in offscreen.xml.

=== I'm trying to use VTK, but there doesn't seem to be any output. What is wrong? ===

To use VTK on VisTrails, you need a slightly different way of connecting the renderer modules. Instead of using the standard RenderWindow/RenderWindowInteractor infrastructure, you simply connect the renderer to a VTKCell. The examples directory in the distribution has several VTK examples that illustrate.

=== I am trying to add a module to the workflow via Python, but how can I access vtk modules? ===

Here's an example:

import api

vtvtk = 'edu.utah.sci.vistrails.vtk'

module = api.add_module(0, 0, vtvtk, 'vtkContourFilter', '')

The third argument in add_module is the package identifier. You can find this in the "Module Packages" panel of the Preferences; just click on the package you're interested in and it will appear in the information on the right.

== matplotlib ==

=== I'm experiencing a problem with Latex labels and the matplotlib that comes with VisTrails 1.5. The script below entered to the interpreter that comes with VT is sufficient to reproduce it. ===

import matplotlib.pyplot as plt
plt.plot([1,2,3],[1,2,3])
plt.xlabel("$foo$")

Remove your ~/.matplotlib folder and re-start VisTrails

== VisTrails Development ==

=== I would like to build VisTrails from source. Are there instructions on how to do this? ===

Yes! Take a look at http://www.vistrails.org/index.php/Mac_Intel_Instructions
[http://www.vistrails.org/usersguide/getting_started.html#installing-vistrails-from-source Installing VisTrails from source]

== Accessing Provenance Information ==

=== How do I access the information in the execution log? ===

The code responsible for storing execution information is located in the "core/log" directories, and the code that generates much of that information is in "core/interpreter/cached.py". Modules can add execution-specific annotations to provenance via annotate() calls during execution, but much of the data (like timing and errors) is captured by the LogController and CachedInterpreter (the execution engine) objects. To analyze the log from a vistrail (.vt) file, you might have something like the following:

<code>
import core.log.log
import db.services.io

def run(fname):
# open the .vt bundle specified by the filename "fname"
bundle = db.services.io.open_vistrail_bundle_from_zip_xml(fname)[0]
# get the log filename
log_fname = bundle.vistrail.db_log_filename
if log_fname is not None:
# open the log
log = db.services.io.open_log_from_xml(log_fname, True)
# convert the log from a db object
core.log.log.Log.convert(log)
for workflow_exec in log.workflow_execs:
print 'workflow version:', workflow_exec.parent_version
print 'time started:', workflow_exec.ts_start
print 'time ended:', workflow_exec.ts_end
print 'modules executed:', [i.module_id
for i in workflow_exec.item_execs]
if __name__ == '__main__':
run("some_vistrail.vt")
</code>

You should be able to see what information is available by looking at the "core/log" classes.

FAQ

2011-03-25T17:25:23Z

Ctuttle: /* When a workflow is executed, what do the colors mean? */

Also check our [[known_issues|Known Issues]] page for troubleshooting.

----
== Running workflows ==

=== How can I run a workflow using the command line? ===

('''Updated for version 1.2''') Call vistrails using the following options:

<code>python vistrails.py -b path_to_vistrails_file:pipeline </code>

where pipeline can be a version '''tag name''' or version '''id'''

'''NOTE:''' If you downloaded the MacOS X bundle, you can run vistrails from the command line via the following commands in the Terminal. Change the current directory to wherever VisTrails was installed (often <code>/Applications</code>), and then type:

<code>Vistrails.app/Contents/MacOS/vistrails [<cmd_line_options>]</code>

=== Using the command line, we'd like to execute a workflow multiple times, with slightly different parameters, and create a series of output files. Is this possible?===

('''Updated for version 1.2''') We can change parameters that have an alias through the command line.

For example, offscreen pipeline in offscreen.vt always creates the file called image.png. If you want generate it with a different filename:

<code>python vistrails.py -b ../examples/offscreen.vt:offscreen -a"filename=other.png"</code>

<code>filename</code> in the example above is the alias name assigned to the parameter in the <code>value</code> method inside the String module. When running a pipeline from the command line, VisTrails will try to start the spreadsheet automatically if the pipeline requires it. For example, this other execution will also start the spreadsheet (attention to how $ characters are escaped when running on bash):

<code>python vistrails.py -b ../examples/head.vt:aliases -a"isovalue=30\$&\$diffuse_color=0.8,0.4,0.2"</code>

You can also execute more than one pipeline on the command line:

<code>python vistrails.py -b ../examples/head.vt:aliases ../examples/spx.vt:spx \
-a"isovalue=30"</code>

Use the -a parameter only once regardless the number of pipelines.

=== I can load a vistrail, and the version tree shows up fine. However, no pipelines appear when I click on a version. What gives? ===

The most likely reason is that the vistrail uses a package that is not registered with VisTrails. You need to identify the needed package and add it to your <code>.vistrails/startup.py</code>. A single line like the following should be enough:

addPackage('enter_package_name_here')

Some packages might need more information. For example:

addPackage('afront', executable_path='/path/to/afront')

Refer to the package documentation for details. The one inconvenient step is that currently there's no automated way to describe what is the missing package. We're working on this feature for future releases.

=== I have a workflow that reads a file and then does some processing. The first time it runs, it executes correctly. But in subsequent, nothing happens. ===

VisTrails caches by default, so after a workflow is executed, if none of its parameters change, it won't be executed again.

If a workflow reads a file using the basic module File, VisTrails does check whether the file was modified since the last run. It does so by keeping a signature that is based on the modification time of the file. And if the file was modified, the File module and all downstream modules (the ones which depend on File) will be executed.

''Note: If you would like your input and output data to be versioned, you can use the Persistence package.''

If you do not want VisTrails to cache executions, you can turn off caching: go to Menu Edit -> Preferences and in the General Configuration tab, change Cache execution results to Never.

=== Can VisTrails execute workflows in parallel? ===

The VisTrails server can only execute pipelines in parallel if there's more than one instance of VisTrails running. The command

<code> self.rpcserver = ThreadedXMLRPCServer((self.temp_xml_rpc_options.server, self.temp_xml_rpc_options.port))</code>

starts a multithreaded version of the XML-RPC server, so it will create a thread for each request received by the server. The problem is that Qt/PyQT doesn't allow these multiple threads create GUI objects, only in the main thread. To overcome this limitation, the multithreaded version can instantiate other single threaded versions of VisTrails and put them in a queue, so workflow executions and other GUI-related requests, such as generating workflow graphs and history trees can be forwarded to this queue, and each instance takes turns in answering the request. If the results are in the cache, the multithreaded version answers the requests directly.

Note that this infrastructure works on Linux only. To make this work on Windows, you have to create a script similar to start_vistrails_xvfb.sh (located in the scripts folder) where you can send the number of other instances via command-line options to VisTrails. The command line options are:

python vistrails_server.py -T <ADDRESS> -R <PORT> -O<NUMBER_OF_OTHER_VISTRAILS_INSTANCES> [-M]&

If you want the main vistrails instance to be multithreaded, use the -M at the end.

After creating this script, update function start_other_instances in vistrails/gui/application_server.py lines 1007-1023 and set the script variable to point to your script. You may also have to change the arguments sent to your script (line 1016: for example, you don't need to set a virtual display). You will need to change the path to the stop_vistrails_server.py script (on line 1026) according to your installation path.

=== When a workflow is executed, what do the colors mean? ===

- lilac: module was notexecuted

- yellow: module is currently being executed

- green: module was successfully executed

- orange: module was cached

- red: the execution of the module failed

[http://www.vistrails.org/usersguide/getting_started.html#vistrails-interaction Vistrails Interaction]

== Building workflows ==

=== Is there a way to give each widget a "display name" in addition to the module name at the center of the widget? ===

Yes, a "display name" can be assigned to a module by selecting the triangle in its top right corner to open a popup menu and selecting the Set Module Label... menu item. You will then be prompted to enter the "display name". [http://www.vistrails.org/usersguide/creating.html#changing-module-labels Changing Module Labels]

=== Is there a way to re-center the picture-in-picture (PiP) view? ===

Yes. If you click on the PIP window to bring it to focus, you can press Ctrl-R (or Command-R on Mac) to re-center the PiP window. [http://www.vistrails.org/usersguide/getting_started.html#vistrails-interaction Vistrails Interaction]

=== How do I search for a literal "?" (question mark) in the search box in the Property panel? ===

Since we allow regular expressions in our search box, question marks are treated as meta-characters. Thus, searching for "?" returns everything and "abc?" will return everything containing "abc". You need to use "\?" instead to search for "?". So the search for "??" would be "\?\?". [http://www.vistrails.org/usersguide/querying.html#textual-queries Textual Queries]

== Using VisTrails as a server ==

=== What is the VisTrails server-mode? ===

Using the VisTrails server mode, it is possible to ''execute workflows and control VisTrails through another application''. For example, the CrowdLabs Web portal (http://www.crowdlabs.org) accesses a VisTrails sever to execute workflows, retrieve and display vistrail trees and workflows. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

=== How do I execute workflows and control VisTrails through another application? ===

The way you access the server is by doing XML-RPC calls. In the current VisTrails release, we include a set of PHP scripts that can talk to a VisTrails server instance. They are in "extensions/http" folder. The files are reasonably well documented. Also, it should be not difficult to create python scripts to access the server (just use xmlrpclib module).

Note that the VisTrails server requires the provenance and workflows to be in a database. More detailed instructions on how to setup the server and the database
are available here:

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_server_setup.html

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_database_setup.html

If what you want is just to execute a series of workflows in batch mode, a simpler solution would be to ''use the VisTrails client in batch mode''. Chapter 12 of the user's guide contains detailed information and examples on that. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

== Control Flow ==

=== Note: using map ===

When using 'map', the module (or subworkflow) used as function port in the map module MUST be a function, i.e., it can only define 1 output port. [http://www.vistrails.org/usersguide/controlflow.html#the-map-operator The Map Operator]

== Spreadsheet ==

Where ''pipeline'' is a version number or a tag.

=== How can I save an image from the spreadsheet? ===

While having the focus on a spreadsheet cell and select the camera on the toolbar to take a snapshot. The system will prompt you for the location and file name where it should be saved. The other icons can be used for saving multiple images that can be used for generating an animation on demand. A whole sheet can also be saved by selecting Export (either from the menu or from the toolbar). [http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet-image Saving a Spreadsheet Image]

===Is it possible to save the complete state of the spreadsheet?===
[http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet Saving a Spreadsheet]

=== Can I view multiple sheets at the same time? ===

Yes. Each sheet on the spreadsheet can be displayed as a dock widget separated from the main spreadsheet window by dragging its tab name out of the tab bar at the bottom of the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Then, how can I put back a separated sheet? ===

A sheet can be docked back to the main window by dragging it back to the tab bar or double-click on its title bar. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== How can I order sheets on the spreadsheet? ===

This can be done by dragging the sheet name on the bottom top bar and drop it to the right place. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Can I control where a cell will be placed on the spreadsheet window? ===

By default, an unoccupied cell on the active sheet will be chosen to display the result. However, you can specify exactly in the pipeline where a spreadsheet cell will be placed by using CellLocation and SheetReference. CellLocation specifies the location (row and column) of a cell when connecting to a spreadsheet cell (VTKCell, ImageViewerCell, ...). Similarly, a SheetReference module (when connecting to a CellLocation) will specify which sheet the cell will be put on given its name, minimum row size and minimum column size. There is an example of this in examples/vtk.xml (select the version below Double Renderer). [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

=== How do I output results to the spreadsheet? ===

By inspecting the VisTrails Spreadsheet package (in the list of packages, to the left of the pipeline builder), you
can see there are built-in cells for different kinds of data, e.g., RichTextCell to display HTML and plain text. op
You (the user) can also define new cell types to display application-specific data. For example, we have developed
VtkCell, MplFigureCell, and OpenGLCell. It is possible to display pretty much anything on the Spreadsheet! [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

Examples of writing cell modules can be found in:
RichTextCell: packages/spreadsheet/widgets/richtext/richtext.py
VTK: packages/vtk/vtkcell.py

Here is the summary of some requirements on a cell widget:

(1) It must be a Qt widget. It should inherit from spreadsheet_cell.QCellWidget in the spreadsheet package. Although any Qt Widget would work, certain features such as animation will not be available (without rewriting it).

(2) It must re-implement the updateContents() function to take a set of inputs (usually coming from input ports of a wrapper Module) and display on the cells. VisTrails uses this function to update/reuse cells on the spreadsheet when new data comes in.

(3) It needs a wrapper VisTrails Module (inherited from basic_widgets.SpreadsheetCell of the spreadsheet package). Inside the compute() method of this module, it may call self.display(CellWidgetType, (inputs)) to trigger the display event on the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#advanced-cell-options Advanced Cell Options]

=== How do I control the default number of cells in the spreadsheet? ===

You can configure the rowCount and colCount using the preferences dialog. Just go to the Module Packages tab, select spreadsheet in the "Enabled packages" and press the Configure button. Then a list of all the configuration options for the spreadsheet will show up. [http://www.vistrails.org/usersguide/spreadsheet.html#custom-layout-options Custom Layout Options]

=== Is it possible to launch a web browser from the vistrails spreadsheet? We would like to output several urls from a parameter sweep and then have the option to click on each one to view the resulting page. I can view the page within the spreadsheet, but it is really too crowded.===

Currently, there isn't a widget that provides exactly this functionality, but I can think of a few solutions that may work for you:

(1) You can use parameter exploration to generate multiple sheets so you might have an exploration that opens each page in a new sheet. Use the third column/dimension in the exploration interface to have a parameter span sheets.

(2) The spreadsheet is extensible so you can write a custom spreadsheet cell widget that has a button or label with the desired link (a QLabel with openExternalLinks set to True, for example).

(3) You can tweak the existing RichTextCell be adding the line "self.browser.setOpenExternalLinks(True)" at line 63 of the source file "vistrails/packages/spreadsheet/widgets/richtext/richtext.py". Then, if your workflow creates a file with html markup text like "<a href="http://www.vistrails.org">VisTrails</a>" connected to a RichTextCell, clicking on the rendered link in the cell will open it in a web browser. You need to add the aforementioned line to the source to let Qt know that you want the link opened externally; by default, it will just issue an event that isn't processed. [http://www.vistrails.org/usersguide/spreadsheet.html#launching-a-web-browser Launching a Web Browser]

== Integrating your software into VisTrails ==

===How can I integrate my own program into VisTrails?===

The easiest way is to create a package. Writing a package is often very simple, here are instructions on how to do it: [[UsersGuideVisTrailsPackages]]

You can also dynamically generate modules. For an example see:

http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#Packages_that_generate_modules_dynamically

In particular, see the new_module call which uses python's type() function to generate new classes dynamically.

=== How do modules deal with multiple inputs in a same port? ===

(And should that even be allowed?)

For compatibility reasons, we do need to allow multiple connections to an input port. However, most package developers should never have to use this, and so we do our best to hide it. the default behavior for getting inputs from a port, then, is to always return a single input.

If on your module you need multiple inputs connected to a single port, use the 'forceGetInputListFromPort' method. It will return a list of all the data items coming through the port. The VTK package uses this feature, so look there for usage examples (packages/vtk/base_widget.py)

=== Are there mechanisms for attaching widgets to different modules/parameters? ===

Right now, we have a mechanism for putting a specific widget for an input port. For example, if a port is SetColor(red, green, blue), we can put a color wheel widget there. Or we can also replace the SetFileName port with a File Widget. However, this is not per parameter (only per port). We are currently working on this problem.

=== Can I organize my package so it appears hierarchical in the module palette? ===

Yes. Use the namespace keyword argument when adding the module to the registry. For example,

registry.add_module(MyModule, namespace='MyNamespace')

=== Can I nest namespaces? ===

Yes. Use the '|' character to separate different the hierarchy. For example,

registry.add_module(MyModule, namespace='ParentNamespace|ChildNamespace')

=== Are there shortucts for registry initialization? ===

Yes. If you define _modules as a list of classes in the __init__.py file of your package, VisTrails will attempt to load all classes specified as modules. You can provide add_module options as keyword arguments by specifying a tuple (class, kwargs) in the list. For example:

_modules = [MyModule1, (MyModule2, {'namespace': 'MyNamespace'})]

In addition, you need to identify the ports of your modules as a field in your class by defining _input_ports and _output_ports lists. Here, the items in each list must be tuples of the form (portName, portSignature, optional=False, sort_key=-1). For example:

class MyModule(Module):
def compute(self):
pass

_input_ports = [('firstInput', String), ('secondInput', Integer, True)]
_output_ports = [('firstOutput', String), ('secondOutput', String)]

=== Can I define ports to be of types that I do not import into my package? ===

Yes. You can pass an identifier string as the portSignature instead. The port_signature string is defined by:

<module_string> := <package_identifier>:[<namespace>|]<module_name>,
<port_signature> := (<module_string>*)

For example,

registry.add_input_port(MyModule, 'myInputPort', '(edu.utah.sci.vistrails.basic:String)')

or

_input_ports = [('myInputPort', '(edu.utah.sci.vistrails.basic:String)')]

=== What do I need to change in my package to make it reloadable (new in v1.4.2)? ===
See [http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#How_to_make_you_package_reloadable UsersGuideVisTrailsPackages] for an explanation.

=== Can I add default values or labels for parameters? ===

Yes. Versions 1.4 and greater support these features. See [[UsersGuideVisTrailsPackages#Adding_default_values_and.2For_labels_for_parameters | UsersGuideVisTrailsPacakges]] for more details.

=== I want to write a module to load HDF data whose output (e.g., data, string) varies according to the input I give it. Is is possible to do this in VisTrails, and if yes, how can I do that? Ideally, I would like to avoid having to change the connection of my output every time I change the input. ===

There are a few ways to tackle this - each has it's own benefits and pitfalls. Firstly, module connections do respect class hierarchies as we're familiar with in object oriented languages. For instance, A module can output a Constant of which String, Float, Integer, etc are specifications. In this way, you can have a subclass of something like HDFData be passed out of the module and the connections will be established regardless of the sub-type. This is a bit dangerous though. Modules downstream of such a class may not really know how to operate on certain types derived from the super-class. Extreme care must be taken both when creating the modules as well as connecting them to prevent things like this from happening.

A second method that I employ in several different packages is the idea of a container class. For instance, the NumSciPy package uses a relatively generic container "Numpy Array" to encapsulate the data. Of course, these encapsulating objects can store dictionaries that other modules can easily access and understand how to operate on. Although this method is slightly more work, the benefits of a stricter typing of ports is beneficial - particularly upon interfacing with other packages that may depend on strongly typed constants (for example).

== The Console ==

=== Where should I go to find out what I can call from the console and how to import it? ===

We have tried to make some methods more accessible in the console via an api. You can import the api via <code>import api</code> in the console and see the available methods with <code>dir(api)</code>. To open a vistrail:

import api
api.open_vistrail_from_file('/Applications/VisTrails/examples/terminator.vt')

To execute a version of a workflow, you currently have to go through the controller:

api.select_version('Histogram')
api.get_current_controller().execute_current_workflow()

Currently, only a subset of VisTrails functionality is directly available from the api. However, since VisTrails is written in python, you can dig down starting with the VistrailsApplication or controller object to expose most of our internal methods. If you have suggestions for calls to be added to the api, please let us know.

One other feature that we're working on, but is still in progress is the ability to construct workflows via the console. For example:

vtk = load_package('edu.utah.sci.vistrails.vtk')
vtk.vtkDataSetReader() # adds a vtkDataSetReader module to the pipeline
# click on the new module
a = selected_modules()[0] # get the one currently selected module
a.SetFile('/vistrails/examples/data/head120.vtk') # sets the SetFile parmaeter for the data set reader
b = vtk.vtkContourFilter() # adds a vtkContourFilter module to the pipeline and saves to var b
b.SetInputConnection0(a.GetOutputPort0()) # connects a's GetOutputPort0 port to b's SetInputConnection0

== Persistence Package ==

=== How do I use the output of one workflow as the input for another using the persistence package? ===

You need to configure the persistence modules using the module's configuration dialog. After adding a <code>PersistentOutputFile</code> to the workflow, click on the triangle in the upper-right corner of the <code>PersistentOutputFile</code>, and select "Edit Configuration" from the menu that appears. In this dialog, select "Create New Reference" and give the reference a name (and any space-delimited tags). Upon running that workflow, the data will be written to the persistent store. In the second workflow where you wish to use that file, add a <code>PersistentInputFile</code> and go to its configuration dialog in the same manner as with the output file. In that dialog, select "Use Existing Reference" and select the data that you just added in the first workflow in the list of files below. Now, when you run that workflow, it will grab the data from the persistent store.

Here is an example: [[Media:offscreen_persistent.vt | offscreen_persistent.vt]]. Run the "persistent offscreen" workflow first, and then run the "display persistent output" to use the output of the first workflow as the input for the second.

== VTK ==

=== Given a VTK visualization, how can I generate a webpage from it? ===

Check out the html pipeline in offscreen.xml.

=== I'm trying to use VTK, but there doesn't seem to be any output. What is wrong? ===

To use VTK on VisTrails, you need a slightly different way of connecting the renderer modules. Instead of using the standard RenderWindow/RenderWindowInteractor infrastructure, you simply connect the renderer to a VTKCell. The examples directory in the distribution has several VTK examples that illustrate.

=== I am trying to add a module to the workflow via Python, but how can I access vtk modules? ===

Here's an example:

import api

vtvtk = 'edu.utah.sci.vistrails.vtk'

module = api.add_module(0, 0, vtvtk, 'vtkContourFilter', '')

The third argument in add_module is the package identifier. You can find this in the "Module Packages" panel of the Preferences; just click on the package you're interested in and it will appear in the information on the right.

== matplotlib ==

=== I'm experiencing a problem with Latex labels and the matplotlib that comes with VisTrails 1.5. The script below entered to the interpreter that comes with VT is sufficient to reproduce it. ===

import matplotlib.pyplot as plt
plt.plot([1,2,3],[1,2,3])
plt.xlabel("$foo$")

Remove your ~/.matplotlib folder and re-start VisTrails

== VisTrails Development ==

=== I would like to build VisTrails from source. Are there instructions on how to do this? ===

Yes! Take a look at http://www.vistrails.org/index.php/Mac_Intel_Instructions
[http://www.vistrails.org/usersguide/getting_started.html#installing-vistrails-from-source Installing VisTrails from source]

== Accessing Provenance Information ==

=== How do I access the information in the execution log? ===

The code responsible for storing execution information is located in the "core/log" directories, and the code that generates much of that information is in "core/interpreter/cached.py". Modules can add execution-specific annotations to provenance via annotate() calls during execution, but much of the data (like timing and errors) is captured by the LogController and CachedInterpreter (the execution engine) objects. To analyze the log from a vistrail (.vt) file, you might have something like the following:

<code>
import core.log.log
import db.services.io

def run(fname):
# open the .vt bundle specified by the filename "fname"
bundle = db.services.io.open_vistrail_bundle_from_zip_xml(fname)[0]
# get the log filename
log_fname = bundle.vistrail.db_log_filename
if log_fname is not None:
# open the log
log = db.services.io.open_log_from_xml(log_fname, True)
# convert the log from a db object
core.log.log.Log.convert(log)
for workflow_exec in log.workflow_execs:
print 'workflow version:', workflow_exec.parent_version
print 'time started:', workflow_exec.ts_start
print 'time ended:', workflow_exec.ts_end
print 'modules executed:', [i.module_id
for i in workflow_exec.item_execs]
if __name__ == '__main__':
run("some_vistrail.vt")
</code>

You should be able to see what information is available by looking at the "core/log" classes.

FAQ

2011-03-25T17:24:18Z

Ctuttle: /* When a workflow is executed, what do the colors mean? */

Also check our [[known_issues|Known Issues]] page for troubleshooting.

----
== Running workflows ==

=== How can I run a workflow using the command line? ===

('''Updated for version 1.2''') Call vistrails using the following options:

<code>python vistrails.py -b path_to_vistrails_file:pipeline </code>

where pipeline can be a version '''tag name''' or version '''id'''

'''NOTE:''' If you downloaded the MacOS X bundle, you can run vistrails from the command line via the following commands in the Terminal. Change the current directory to wherever VisTrails was installed (often <code>/Applications</code>), and then type:

<code>Vistrails.app/Contents/MacOS/vistrails [<cmd_line_options>]</code>

=== Using the command line, we'd like to execute a workflow multiple times, with slightly different parameters, and create a series of output files. Is this possible?===

('''Updated for version 1.2''') We can change parameters that have an alias through the command line.

For example, offscreen pipeline in offscreen.vt always creates the file called image.png. If you want generate it with a different filename:

<code>python vistrails.py -b ../examples/offscreen.vt:offscreen -a"filename=other.png"</code>

<code>filename</code> in the example above is the alias name assigned to the parameter in the <code>value</code> method inside the String module. When running a pipeline from the command line, VisTrails will try to start the spreadsheet automatically if the pipeline requires it. For example, this other execution will also start the spreadsheet (attention to how $ characters are escaped when running on bash):

<code>python vistrails.py -b ../examples/head.vt:aliases -a"isovalue=30\$&\$diffuse_color=0.8,0.4,0.2"</code>

You can also execute more than one pipeline on the command line:

<code>python vistrails.py -b ../examples/head.vt:aliases ../examples/spx.vt:spx \
-a"isovalue=30"</code>

Use the -a parameter only once regardless the number of pipelines.

=== I can load a vistrail, and the version tree shows up fine. However, no pipelines appear when I click on a version. What gives? ===

The most likely reason is that the vistrail uses a package that is not registered with VisTrails. You need to identify the needed package and add it to your <code>.vistrails/startup.py</code>. A single line like the following should be enough:

addPackage('enter_package_name_here')

Some packages might need more information. For example:

addPackage('afront', executable_path='/path/to/afront')

Refer to the package documentation for details. The one inconvenient step is that currently there's no automated way to describe what is the missing package. We're working on this feature for future releases.

=== I have a workflow that reads a file and then does some processing. The first time it runs, it executes correctly. But in subsequent, nothing happens. ===

VisTrails caches by default, so after a workflow is executed, if none of its parameters change, it won't be executed again.

If a workflow reads a file using the basic module File, VisTrails does check whether the file was modified since the last run. It does so by keeping a signature that is based on the modification time of the file. And if the file was modified, the File module and all downstream modules (the ones which depend on File) will be executed.

''Note: If you would like your input and output data to be versioned, you can use the Persistence package.''

If you do not want VisTrails to cache executions, you can turn off caching: go to Menu Edit -> Preferences and in the General Configuration tab, change Cache execution results to Never.

=== Can VisTrails execute workflows in parallel? ===

The VisTrails server can only execute pipelines in parallel if there's more than one instance of VisTrails running. The command

<code> self.rpcserver = ThreadedXMLRPCServer((self.temp_xml_rpc_options.server, self.temp_xml_rpc_options.port))</code>

starts a multithreaded version of the XML-RPC server, so it will create a thread for each request received by the server. The problem is that Qt/PyQT doesn't allow these multiple threads create GUI objects, only in the main thread. To overcome this limitation, the multithreaded version can instantiate other single threaded versions of VisTrails and put them in a queue, so workflow executions and other GUI-related requests, such as generating workflow graphs and history trees can be forwarded to this queue, and each instance takes turns in answering the request. If the results are in the cache, the multithreaded version answers the requests directly.

Note that this infrastructure works on Linux only. To make this work on Windows, you have to create a script similar to start_vistrails_xvfb.sh (located in the scripts folder) where you can send the number of other instances via command-line options to VisTrails. The command line options are:

python vistrails_server.py -T <ADDRESS> -R <PORT> -O<NUMBER_OF_OTHER_VISTRAILS_INSTANCES> [-M]&

If you want the main vistrails instance to be multithreaded, use the -M at the end.

After creating this script, update function start_other_instances in vistrails/gui/application_server.py lines 1007-1023 and set the script variable to point to your script. You may also have to change the arguments sent to your script (line 1016: for example, you don't need to set a virtual display). You will need to change the path to the stop_vistrails_server.py script (on line 1026) according to your installation path.

=== When a workflow is executed, what do the colors mean? ===

- lilac: module was notexecuted

- yellow: module is currently being executed

- green: module was successfully executed

- orange: module was cached

- red: the execution of the module failed

getting_started.html#vistrails-interaction

== Building workflows ==

=== Is there a way to give each widget a "display name" in addition to the module name at the center of the widget? ===

Yes, a "display name" can be assigned to a module by selecting the triangle in its top right corner to open a popup menu and selecting the Set Module Label... menu item. You will then be prompted to enter the "display name". [http://www.vistrails.org/usersguide/creating.html#changing-module-labels Changing Module Labels]

=== Is there a way to re-center the picture-in-picture (PiP) view? ===

Yes. If you click on the PIP window to bring it to focus, you can press Ctrl-R (or Command-R on Mac) to re-center the PiP window. [http://www.vistrails.org/usersguide/getting_started.html#vistrails-interaction Vistrails Interaction]

=== How do I search for a literal "?" (question mark) in the search box in the Property panel? ===

Since we allow regular expressions in our search box, question marks are treated as meta-characters. Thus, searching for "?" returns everything and "abc?" will return everything containing "abc". You need to use "\?" instead to search for "?". So the search for "??" would be "\?\?". [http://www.vistrails.org/usersguide/querying.html#textual-queries Textual Queries]

== Using VisTrails as a server ==

=== What is the VisTrails server-mode? ===

Using the VisTrails server mode, it is possible to ''execute workflows and control VisTrails through another application''. For example, the CrowdLabs Web portal (http://www.crowdlabs.org) accesses a VisTrails sever to execute workflows, retrieve and display vistrail trees and workflows. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

=== How do I execute workflows and control VisTrails through another application? ===

The way you access the server is by doing XML-RPC calls. In the current VisTrails release, we include a set of PHP scripts that can talk to a VisTrails server instance. They are in "extensions/http" folder. The files are reasonably well documented. Also, it should be not difficult to create python scripts to access the server (just use xmlrpclib module).

Note that the VisTrails server requires the provenance and workflows to be in a database. More detailed instructions on how to setup the server and the database
are available here:

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_server_setup.html

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_database_setup.html

If what you want is just to execute a series of workflows in batch mode, a simpler solution would be to ''use the VisTrails client in batch mode''. Chapter 12 of the user's guide contains detailed information and examples on that. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

== Control Flow ==

=== Note: using map ===

When using 'map', the module (or subworkflow) used as function port in the map module MUST be a function, i.e., it can only define 1 output port. [http://www.vistrails.org/usersguide/controlflow.html#the-map-operator The Map Operator]

== Spreadsheet ==

Where ''pipeline'' is a version number or a tag.

=== How can I save an image from the spreadsheet? ===

While having the focus on a spreadsheet cell and select the camera on the toolbar to take a snapshot. The system will prompt you for the location and file name where it should be saved. The other icons can be used for saving multiple images that can be used for generating an animation on demand. A whole sheet can also be saved by selecting Export (either from the menu or from the toolbar). [http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet-image Saving a Spreadsheet Image]

===Is it possible to save the complete state of the spreadsheet?===
[http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet Saving a Spreadsheet]

=== Can I view multiple sheets at the same time? ===

Yes. Each sheet on the spreadsheet can be displayed as a dock widget separated from the main spreadsheet window by dragging its tab name out of the tab bar at the bottom of the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Then, how can I put back a separated sheet? ===

A sheet can be docked back to the main window by dragging it back to the tab bar or double-click on its title bar. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== How can I order sheets on the spreadsheet? ===

This can be done by dragging the sheet name on the bottom top bar and drop it to the right place. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Can I control where a cell will be placed on the spreadsheet window? ===

By default, an unoccupied cell on the active sheet will be chosen to display the result. However, you can specify exactly in the pipeline where a spreadsheet cell will be placed by using CellLocation and SheetReference. CellLocation specifies the location (row and column) of a cell when connecting to a spreadsheet cell (VTKCell, ImageViewerCell, ...). Similarly, a SheetReference module (when connecting to a CellLocation) will specify which sheet the cell will be put on given its name, minimum row size and minimum column size. There is an example of this in examples/vtk.xml (select the version below Double Renderer). [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

=== How do I output results to the spreadsheet? ===

By inspecting the VisTrails Spreadsheet package (in the list of packages, to the left of the pipeline builder), you
can see there are built-in cells for different kinds of data, e.g., RichTextCell to display HTML and plain text. op
You (the user) can also define new cell types to display application-specific data. For example, we have developed
VtkCell, MplFigureCell, and OpenGLCell. It is possible to display pretty much anything on the Spreadsheet! [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

Examples of writing cell modules can be found in:
RichTextCell: packages/spreadsheet/widgets/richtext/richtext.py
VTK: packages/vtk/vtkcell.py

Here is the summary of some requirements on a cell widget:

(1) It must be a Qt widget. It should inherit from spreadsheet_cell.QCellWidget in the spreadsheet package. Although any Qt Widget would work, certain features such as animation will not be available (without rewriting it).

(2) It must re-implement the updateContents() function to take a set of inputs (usually coming from input ports of a wrapper Module) and display on the cells. VisTrails uses this function to update/reuse cells on the spreadsheet when new data comes in.

(3) It needs a wrapper VisTrails Module (inherited from basic_widgets.SpreadsheetCell of the spreadsheet package). Inside the compute() method of this module, it may call self.display(CellWidgetType, (inputs)) to trigger the display event on the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#advanced-cell-options Advanced Cell Options]

=== How do I control the default number of cells in the spreadsheet? ===

You can configure the rowCount and colCount using the preferences dialog. Just go to the Module Packages tab, select spreadsheet in the "Enabled packages" and press the Configure button. Then a list of all the configuration options for the spreadsheet will show up. [http://www.vistrails.org/usersguide/spreadsheet.html#custom-layout-options Custom Layout Options]

=== Is it possible to launch a web browser from the vistrails spreadsheet? We would like to output several urls from a parameter sweep and then have the option to click on each one to view the resulting page. I can view the page within the spreadsheet, but it is really too crowded.===

Currently, there isn't a widget that provides exactly this functionality, but I can think of a few solutions that may work for you:

(1) You can use parameter exploration to generate multiple sheets so you might have an exploration that opens each page in a new sheet. Use the third column/dimension in the exploration interface to have a parameter span sheets.

(2) The spreadsheet is extensible so you can write a custom spreadsheet cell widget that has a button or label with the desired link (a QLabel with openExternalLinks set to True, for example).

(3) You can tweak the existing RichTextCell be adding the line "self.browser.setOpenExternalLinks(True)" at line 63 of the source file "vistrails/packages/spreadsheet/widgets/richtext/richtext.py". Then, if your workflow creates a file with html markup text like "<a href="http://www.vistrails.org">VisTrails</a>" connected to a RichTextCell, clicking on the rendered link in the cell will open it in a web browser. You need to add the aforementioned line to the source to let Qt know that you want the link opened externally; by default, it will just issue an event that isn't processed. [http://www.vistrails.org/usersguide/spreadsheet.html#launching-a-web-browser Launching a Web Browser]

== Integrating your software into VisTrails ==

===How can I integrate my own program into VisTrails?===

The easiest way is to create a package. Writing a package is often very simple, here are instructions on how to do it: [[UsersGuideVisTrailsPackages]]

You can also dynamically generate modules. For an example see:

http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#Packages_that_generate_modules_dynamically

In particular, see the new_module call which uses python's type() function to generate new classes dynamically.

=== How do modules deal with multiple inputs in a same port? ===

(And should that even be allowed?)

For compatibility reasons, we do need to allow multiple connections to an input port. However, most package developers should never have to use this, and so we do our best to hide it. the default behavior for getting inputs from a port, then, is to always return a single input.

If on your module you need multiple inputs connected to a single port, use the 'forceGetInputListFromPort' method. It will return a list of all the data items coming through the port. The VTK package uses this feature, so look there for usage examples (packages/vtk/base_widget.py)

=== Are there mechanisms for attaching widgets to different modules/parameters? ===

Right now, we have a mechanism for putting a specific widget for an input port. For example, if a port is SetColor(red, green, blue), we can put a color wheel widget there. Or we can also replace the SetFileName port with a File Widget. However, this is not per parameter (only per port). We are currently working on this problem.

=== Can I organize my package so it appears hierarchical in the module palette? ===

Yes. Use the namespace keyword argument when adding the module to the registry. For example,

registry.add_module(MyModule, namespace='MyNamespace')

=== Can I nest namespaces? ===

Yes. Use the '|' character to separate different the hierarchy. For example,

registry.add_module(MyModule, namespace='ParentNamespace|ChildNamespace')

=== Are there shortucts for registry initialization? ===

Yes. If you define _modules as a list of classes in the __init__.py file of your package, VisTrails will attempt to load all classes specified as modules. You can provide add_module options as keyword arguments by specifying a tuple (class, kwargs) in the list. For example:

_modules = [MyModule1, (MyModule2, {'namespace': 'MyNamespace'})]

In addition, you need to identify the ports of your modules as a field in your class by defining _input_ports and _output_ports lists. Here, the items in each list must be tuples of the form (portName, portSignature, optional=False, sort_key=-1). For example:

class MyModule(Module):
def compute(self):
pass

_input_ports = [('firstInput', String), ('secondInput', Integer, True)]
_output_ports = [('firstOutput', String), ('secondOutput', String)]

=== Can I define ports to be of types that I do not import into my package? ===

Yes. You can pass an identifier string as the portSignature instead. The port_signature string is defined by:

<module_string> := <package_identifier>:[<namespace>|]<module_name>,
<port_signature> := (<module_string>*)

For example,

registry.add_input_port(MyModule, 'myInputPort', '(edu.utah.sci.vistrails.basic:String)')

or

_input_ports = [('myInputPort', '(edu.utah.sci.vistrails.basic:String)')]

=== What do I need to change in my package to make it reloadable (new in v1.4.2)? ===
See [http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#How_to_make_you_package_reloadable UsersGuideVisTrailsPackages] for an explanation.

=== Can I add default values or labels for parameters? ===

Yes. Versions 1.4 and greater support these features. See [[UsersGuideVisTrailsPackages#Adding_default_values_and.2For_labels_for_parameters | UsersGuideVisTrailsPacakges]] for more details.

=== I want to write a module to load HDF data whose output (e.g., data, string) varies according to the input I give it. Is is possible to do this in VisTrails, and if yes, how can I do that? Ideally, I would like to avoid having to change the connection of my output every time I change the input. ===

There are a few ways to tackle this - each has it's own benefits and pitfalls. Firstly, module connections do respect class hierarchies as we're familiar with in object oriented languages. For instance, A module can output a Constant of which String, Float, Integer, etc are specifications. In this way, you can have a subclass of something like HDFData be passed out of the module and the connections will be established regardless of the sub-type. This is a bit dangerous though. Modules downstream of such a class may not really know how to operate on certain types derived from the super-class. Extreme care must be taken both when creating the modules as well as connecting them to prevent things like this from happening.

A second method that I employ in several different packages is the idea of a container class. For instance, the NumSciPy package uses a relatively generic container "Numpy Array" to encapsulate the data. Of course, these encapsulating objects can store dictionaries that other modules can easily access and understand how to operate on. Although this method is slightly more work, the benefits of a stricter typing of ports is beneficial - particularly upon interfacing with other packages that may depend on strongly typed constants (for example).

== The Console ==

=== Where should I go to find out what I can call from the console and how to import it? ===

We have tried to make some methods more accessible in the console via an api. You can import the api via <code>import api</code> in the console and see the available methods with <code>dir(api)</code>. To open a vistrail:

import api
api.open_vistrail_from_file('/Applications/VisTrails/examples/terminator.vt')

To execute a version of a workflow, you currently have to go through the controller:

api.select_version('Histogram')
api.get_current_controller().execute_current_workflow()

Currently, only a subset of VisTrails functionality is directly available from the api. However, since VisTrails is written in python, you can dig down starting with the VistrailsApplication or controller object to expose most of our internal methods. If you have suggestions for calls to be added to the api, please let us know.

One other feature that we're working on, but is still in progress is the ability to construct workflows via the console. For example:

vtk = load_package('edu.utah.sci.vistrails.vtk')
vtk.vtkDataSetReader() # adds a vtkDataSetReader module to the pipeline
# click on the new module
a = selected_modules()[0] # get the one currently selected module
a.SetFile('/vistrails/examples/data/head120.vtk') # sets the SetFile parmaeter for the data set reader
b = vtk.vtkContourFilter() # adds a vtkContourFilter module to the pipeline and saves to var b
b.SetInputConnection0(a.GetOutputPort0()) # connects a's GetOutputPort0 port to b's SetInputConnection0

== Persistence Package ==

=== How do I use the output of one workflow as the input for another using the persistence package? ===

You need to configure the persistence modules using the module's configuration dialog. After adding a <code>PersistentOutputFile</code> to the workflow, click on the triangle in the upper-right corner of the <code>PersistentOutputFile</code>, and select "Edit Configuration" from the menu that appears. In this dialog, select "Create New Reference" and give the reference a name (and any space-delimited tags). Upon running that workflow, the data will be written to the persistent store. In the second workflow where you wish to use that file, add a <code>PersistentInputFile</code> and go to its configuration dialog in the same manner as with the output file. In that dialog, select "Use Existing Reference" and select the data that you just added in the first workflow in the list of files below. Now, when you run that workflow, it will grab the data from the persistent store.

Here is an example: [[Media:offscreen_persistent.vt | offscreen_persistent.vt]]. Run the "persistent offscreen" workflow first, and then run the "display persistent output" to use the output of the first workflow as the input for the second.

== VTK ==

=== Given a VTK visualization, how can I generate a webpage from it? ===

Check out the html pipeline in offscreen.xml.

=== I'm trying to use VTK, but there doesn't seem to be any output. What is wrong? ===

To use VTK on VisTrails, you need a slightly different way of connecting the renderer modules. Instead of using the standard RenderWindow/RenderWindowInteractor infrastructure, you simply connect the renderer to a VTKCell. The examples directory in the distribution has several VTK examples that illustrate.

=== I am trying to add a module to the workflow via Python, but how can I access vtk modules? ===

Here's an example:

import api

vtvtk = 'edu.utah.sci.vistrails.vtk'

module = api.add_module(0, 0, vtvtk, 'vtkContourFilter', '')

The third argument in add_module is the package identifier. You can find this in the "Module Packages" panel of the Preferences; just click on the package you're interested in and it will appear in the information on the right.

== matplotlib ==

=== I'm experiencing a problem with Latex labels and the matplotlib that comes with VisTrails 1.5. The script below entered to the interpreter that comes with VT is sufficient to reproduce it. ===

import matplotlib.pyplot as plt
plt.plot([1,2,3],[1,2,3])
plt.xlabel("$foo$")

Remove your ~/.matplotlib folder and re-start VisTrails

== VisTrails Development ==

=== I would like to build VisTrails from source. Are there instructions on how to do this? ===

Yes! Take a look at http://www.vistrails.org/index.php/Mac_Intel_Instructions
[http://www.vistrails.org/usersguide/getting_started.html#installing-vistrails-from-source Installing VisTrails from source]

== Accessing Provenance Information ==

=== How do I access the information in the execution log? ===

The code responsible for storing execution information is located in the "core/log" directories, and the code that generates much of that information is in "core/interpreter/cached.py". Modules can add execution-specific annotations to provenance via annotate() calls during execution, but much of the data (like timing and errors) is captured by the LogController and CachedInterpreter (the execution engine) objects. To analyze the log from a vistrail (.vt) file, you might have something like the following:

<code>
import core.log.log
import db.services.io

def run(fname):
# open the .vt bundle specified by the filename "fname"
bundle = db.services.io.open_vistrail_bundle_from_zip_xml(fname)[0]
# get the log filename
log_fname = bundle.vistrail.db_log_filename
if log_fname is not None:
# open the log
log = db.services.io.open_log_from_xml(log_fname, True)
# convert the log from a db object
core.log.log.Log.convert(log)
for workflow_exec in log.workflow_execs:
print 'workflow version:', workflow_exec.parent_version
print 'time started:', workflow_exec.ts_start
print 'time ended:', workflow_exec.ts_end
print 'modules executed:', [i.module_id
for i in workflow_exec.item_execs]
if __name__ == '__main__':
run("some_vistrail.vt")
</code>

You should be able to see what information is available by looking at the "core/log" classes.

FAQ

2011-03-25T17:07:54Z

Ctuttle: /* Building workflows */

Also check our [[known_issues|Known Issues]] page for troubleshooting.

----
== Running workflows ==

=== How can I run a workflow using the command line? ===

('''Updated for version 1.2''') Call vistrails using the following options:

<code>python vistrails.py -b path_to_vistrails_file:pipeline </code>

where pipeline can be a version '''tag name''' or version '''id'''

'''NOTE:''' If you downloaded the MacOS X bundle, you can run vistrails from the command line via the following commands in the Terminal. Change the current directory to wherever VisTrails was installed (often <code>/Applications</code>), and then type:

<code>Vistrails.app/Contents/MacOS/vistrails [<cmd_line_options>]</code>

=== Using the command line, we'd like to execute a workflow multiple times, with slightly different parameters, and create a series of output files. Is this possible?===

('''Updated for version 1.2''') We can change parameters that have an alias through the command line.

For example, offscreen pipeline in offscreen.vt always creates the file called image.png. If you want generate it with a different filename:

<code>python vistrails.py -b ../examples/offscreen.vt:offscreen -a"filename=other.png"</code>

<code>filename</code> in the example above is the alias name assigned to the parameter in the <code>value</code> method inside the String module. When running a pipeline from the command line, VisTrails will try to start the spreadsheet automatically if the pipeline requires it. For example, this other execution will also start the spreadsheet (attention to how $ characters are escaped when running on bash):

<code>python vistrails.py -b ../examples/head.vt:aliases -a"isovalue=30\$&\$diffuse_color=0.8,0.4,0.2"</code>

You can also execute more than one pipeline on the command line:

<code>python vistrails.py -b ../examples/head.vt:aliases ../examples/spx.vt:spx \
-a"isovalue=30"</code>

Use the -a parameter only once regardless the number of pipelines.

=== I can load a vistrail, and the version tree shows up fine. However, no pipelines appear when I click on a version. What gives? ===

The most likely reason is that the vistrail uses a package that is not registered with VisTrails. You need to identify the needed package and add it to your <code>.vistrails/startup.py</code>. A single line like the following should be enough:

addPackage('enter_package_name_here')

Some packages might need more information. For example:

addPackage('afront', executable_path='/path/to/afront')

Refer to the package documentation for details. The one inconvenient step is that currently there's no automated way to describe what is the missing package. We're working on this feature for future releases.

=== I have a workflow that reads a file and then does some processing. The first time it runs, it executes correctly. But in subsequent, nothing happens. ===

VisTrails caches by default, so after a workflow is executed, if none of its parameters change, it won't be executed again.

If a workflow reads a file using the basic module File, VisTrails does check whether the file was modified since the last run. It does so by keeping a signature that is based on the modification time of the file. And if the file was modified, the File module and all downstream modules (the ones which depend on File) will be executed.

''Note: If you would like your input and output data to be versioned, you can use the Persistence package.''

If you do not want VisTrails to cache executions, you can turn off caching: go to Menu Edit -> Preferences and in the General Configuration tab, change Cache execution results to Never.

=== Can VisTrails execute workflows in parallel? ===

The VisTrails server can only execute pipelines in parallel if there's more than one instance of VisTrails running. The command

<code> self.rpcserver = ThreadedXMLRPCServer((self.temp_xml_rpc_options.server, self.temp_xml_rpc_options.port))</code>

starts a multithreaded version of the XML-RPC server, so it will create a thread for each request received by the server. The problem is that Qt/PyQT doesn't allow these multiple threads create GUI objects, only in the main thread. To overcome this limitation, the multithreaded version can instantiate other single threaded versions of VisTrails and put them in a queue, so workflow executions and other GUI-related requests, such as generating workflow graphs and history trees can be forwarded to this queue, and each instance takes turns in answering the request. If the results are in the cache, the multithreaded version answers the requests directly.

Note that this infrastructure works on Linux only. To make this work on Windows, you have to create a script similar to start_vistrails_xvfb.sh (located in the scripts folder) where you can send the number of other instances via command-line options to VisTrails. The command line options are:

python vistrails_server.py -T <ADDRESS> -R <PORT> -O<NUMBER_OF_OTHER_VISTRAILS_INSTANCES> [-M]&

If you want the main vistrails instance to be multithreaded, use the -M at the end.

After creating this script, update function start_other_instances in vistrails/gui/application_server.py lines 1007-1023 and set the script variable to point to your script. You may also have to change the arguments sent to your script (line 1016: for example, you don't need to set a virtual display). You will need to change the path to the stop_vistrails_server.py script (on line 1026) according to your installation path.

=== When a workflow is executed, what do the colors mean? ===

- lilac: module was notexecuted

- yellow: module is currently being executed

- green: module was successfully executed

- orange: module was cached

- red: the execution of the module failed

== Building workflows ==

=== Is there a way to give each widget a "display name" in addition to the module name at the center of the widget? ===

Yes, a "display name" can be assigned to a module by selecting the triangle in its top right corner to open a popup menu and selecting the Set Module Label... menu item. You will then be prompted to enter the "display name". [http://www.vistrails.org/usersguide/creating.html#changing-module-labels Changing Module Labels]

=== Is there a way to re-center the picture-in-picture (PiP) view? ===

Yes. If you click on the PIP window to bring it to focus, you can press Ctrl-R (or Command-R on Mac) to re-center the PiP window. [http://www.vistrails.org/usersguide/getting_started.html#vistrails-interaction Vistrails Interaction]

=== How do I search for a literal "?" (question mark) in the search box in the Property panel? ===

Since we allow regular expressions in our search box, question marks are treated as meta-characters. Thus, searching for "?" returns everything and "abc?" will return everything containing "abc". You need to use "\?" instead to search for "?". So the search for "??" would be "\?\?". [http://www.vistrails.org/usersguide/querying.html#textual-queries Textual Queries]

== Using VisTrails as a server ==

=== What is the VisTrails server-mode? ===

Using the VisTrails server mode, it is possible to ''execute workflows and control VisTrails through another application''. For example, the CrowdLabs Web portal (http://www.crowdlabs.org) accesses a VisTrails sever to execute workflows, retrieve and display vistrail trees and workflows. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

=== How do I execute workflows and control VisTrails through another application? ===

The way you access the server is by doing XML-RPC calls. In the current VisTrails release, we include a set of PHP scripts that can talk to a VisTrails server instance. They are in "extensions/http" folder. The files are reasonably well documented. Also, it should be not difficult to create python scripts to access the server (just use xmlrpclib module).

Note that the VisTrails server requires the provenance and workflows to be in a database. More detailed instructions on how to setup the server and the database
are available here:

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_server_setup.html

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_database_setup.html

If what you want is just to execute a series of workflows in batch mode, a simpler solution would be to ''use the VisTrails client in batch mode''. Chapter 12 of the user's guide contains detailed information and examples on that. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

== Control Flow ==

=== Note: using map ===

When using 'map', the module (or subworkflow) used as function port in the map module MUST be a function, i.e., it can only define 1 output port. [http://www.vistrails.org/usersguide/controlflow.html#the-map-operator The Map Operator]

== Spreadsheet ==

Where ''pipeline'' is a version number or a tag.

=== How can I save an image from the spreadsheet? ===

While having the focus on a spreadsheet cell and select the camera on the toolbar to take a snapshot. The system will prompt you for the location and file name where it should be saved. The other icons can be used for saving multiple images that can be used for generating an animation on demand. A whole sheet can also be saved by selecting Export (either from the menu or from the toolbar). [http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet-image Saving a Spreadsheet Image]

===Is it possible to save the complete state of the spreadsheet?===
[http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet Saving a Spreadsheet]

=== Can I view multiple sheets at the same time? ===

Yes. Each sheet on the spreadsheet can be displayed as a dock widget separated from the main spreadsheet window by dragging its tab name out of the tab bar at the bottom of the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Then, how can I put back a separated sheet? ===

A sheet can be docked back to the main window by dragging it back to the tab bar or double-click on its title bar. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== How can I order sheets on the spreadsheet? ===

This can be done by dragging the sheet name on the bottom top bar and drop it to the right place. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Can I control where a cell will be placed on the spreadsheet window? ===

By default, an unoccupied cell on the active sheet will be chosen to display the result. However, you can specify exactly in the pipeline where a spreadsheet cell will be placed by using CellLocation and SheetReference. CellLocation specifies the location (row and column) of a cell when connecting to a spreadsheet cell (VTKCell, ImageViewerCell, ...). Similarly, a SheetReference module (when connecting to a CellLocation) will specify which sheet the cell will be put on given its name, minimum row size and minimum column size. There is an example of this in examples/vtk.xml (select the version below Double Renderer). [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

=== How do I output results to the spreadsheet? ===

By inspecting the VisTrails Spreadsheet package (in the list of packages, to the left of the pipeline builder), you
can see there are built-in cells for different kinds of data, e.g., RichTextCell to display HTML and plain text. op
You (the user) can also define new cell types to display application-specific data. For example, we have developed
VtkCell, MplFigureCell, and OpenGLCell. It is possible to display pretty much anything on the Spreadsheet! [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

Examples of writing cell modules can be found in:
RichTextCell: packages/spreadsheet/widgets/richtext/richtext.py
VTK: packages/vtk/vtkcell.py

Here is the summary of some requirements on a cell widget:

(1) It must be a Qt widget. It should inherit from spreadsheet_cell.QCellWidget in the spreadsheet package. Although any Qt Widget would work, certain features such as animation will not be available (without rewriting it).

(2) It must re-implement the updateContents() function to take a set of inputs (usually coming from input ports of a wrapper Module) and display on the cells. VisTrails uses this function to update/reuse cells on the spreadsheet when new data comes in.

(3) It needs a wrapper VisTrails Module (inherited from basic_widgets.SpreadsheetCell of the spreadsheet package). Inside the compute() method of this module, it may call self.display(CellWidgetType, (inputs)) to trigger the display event on the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#advanced-cell-options Advanced Cell Options]

=== How do I control the default number of cells in the spreadsheet? ===

You can configure the rowCount and colCount using the preferences dialog. Just go to the Module Packages tab, select spreadsheet in the "Enabled packages" and press the Configure button. Then a list of all the configuration options for the spreadsheet will show up. [http://www.vistrails.org/usersguide/spreadsheet.html#custom-layout-options Custom Layout Options]

=== Is it possible to launch a web browser from the vistrails spreadsheet? We would like to output several urls from a parameter sweep and then have the option to click on each one to view the resulting page. I can view the page within the spreadsheet, but it is really too crowded.===

Currently, there isn't a widget that provides exactly this functionality, but I can think of a few solutions that may work for you:

(1) You can use parameter exploration to generate multiple sheets so you might have an exploration that opens each page in a new sheet. Use the third column/dimension in the exploration interface to have a parameter span sheets.

(2) The spreadsheet is extensible so you can write a custom spreadsheet cell widget that has a button or label with the desired link (a QLabel with openExternalLinks set to True, for example).

(3) You can tweak the existing RichTextCell be adding the line "self.browser.setOpenExternalLinks(True)" at line 63 of the source file "vistrails/packages/spreadsheet/widgets/richtext/richtext.py". Then, if your workflow creates a file with html markup text like "<a href="http://www.vistrails.org">VisTrails</a>" connected to a RichTextCell, clicking on the rendered link in the cell will open it in a web browser. You need to add the aforementioned line to the source to let Qt know that you want the link opened externally; by default, it will just issue an event that isn't processed. [http://www.vistrails.org/usersguide/spreadsheet.html#launching-a-web-browser Launching a Web Browser]

== Integrating your software into VisTrails ==

===How can I integrate my own program into VisTrails?===

The easiest way is to create a package. Writing a package is often very simple, here are instructions on how to do it: [[UsersGuideVisTrailsPackages]]

You can also dynamically generate modules. For an example see:

http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#Packages_that_generate_modules_dynamically

In particular, see the new_module call which uses python's type() function to generate new classes dynamically.

=== How do modules deal with multiple inputs in a same port? ===

(And should that even be allowed?)

For compatibility reasons, we do need to allow multiple connections to an input port. However, most package developers should never have to use this, and so we do our best to hide it. the default behavior for getting inputs from a port, then, is to always return a single input.

If on your module you need multiple inputs connected to a single port, use the 'forceGetInputListFromPort' method. It will return a list of all the data items coming through the port. The VTK package uses this feature, so look there for usage examples (packages/vtk/base_widget.py)

=== Are there mechanisms for attaching widgets to different modules/parameters? ===

Right now, we have a mechanism for putting a specific widget for an input port. For example, if a port is SetColor(red, green, blue), we can put a color wheel widget there. Or we can also replace the SetFileName port with a File Widget. However, this is not per parameter (only per port). We are currently working on this problem.

=== Can I organize my package so it appears hierarchical in the module palette? ===

Yes. Use the namespace keyword argument when adding the module to the registry. For example,

registry.add_module(MyModule, namespace='MyNamespace')

=== Can I nest namespaces? ===

Yes. Use the '|' character to separate different the hierarchy. For example,

registry.add_module(MyModule, namespace='ParentNamespace|ChildNamespace')

=== Are there shortucts for registry initialization? ===

Yes. If you define _modules as a list of classes in the __init__.py file of your package, VisTrails will attempt to load all classes specified as modules. You can provide add_module options as keyword arguments by specifying a tuple (class, kwargs) in the list. For example:

_modules = [MyModule1, (MyModule2, {'namespace': 'MyNamespace'})]

In addition, you need to identify the ports of your modules as a field in your class by defining _input_ports and _output_ports lists. Here, the items in each list must be tuples of the form (portName, portSignature, optional=False, sort_key=-1). For example:

class MyModule(Module):
def compute(self):
pass

_input_ports = [('firstInput', String), ('secondInput', Integer, True)]
_output_ports = [('firstOutput', String), ('secondOutput', String)]

=== Can I define ports to be of types that I do not import into my package? ===

Yes. You can pass an identifier string as the portSignature instead. The port_signature string is defined by:

<module_string> := <package_identifier>:[<namespace>|]<module_name>,
<port_signature> := (<module_string>*)

For example,

registry.add_input_port(MyModule, 'myInputPort', '(edu.utah.sci.vistrails.basic:String)')

or

_input_ports = [('myInputPort', '(edu.utah.sci.vistrails.basic:String)')]

=== What do I need to change in my package to make it reloadable (new in v1.4.2)? ===
See [http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#How_to_make_you_package_reloadable UsersGuideVisTrailsPackages] for an explanation.

=== Can I add default values or labels for parameters? ===

Yes. Versions 1.4 and greater support these features. See [[UsersGuideVisTrailsPackages#Adding_default_values_and.2For_labels_for_parameters | UsersGuideVisTrailsPacakges]] for more details.

=== I want to write a module to load HDF data whose output (e.g., data, string) varies according to the input I give it. Is is possible to do this in VisTrails, and if yes, how can I do that? Ideally, I would like to avoid having to change the connection of my output every time I change the input. ===

There are a few ways to tackle this - each has it's own benefits and pitfalls. Firstly, module connections do respect class hierarchies as we're familiar with in object oriented languages. For instance, A module can output a Constant of which String, Float, Integer, etc are specifications. In this way, you can have a subclass of something like HDFData be passed out of the module and the connections will be established regardless of the sub-type. This is a bit dangerous though. Modules downstream of such a class may not really know how to operate on certain types derived from the super-class. Extreme care must be taken both when creating the modules as well as connecting them to prevent things like this from happening.

A second method that I employ in several different packages is the idea of a container class. For instance, the NumSciPy package uses a relatively generic container "Numpy Array" to encapsulate the data. Of course, these encapsulating objects can store dictionaries that other modules can easily access and understand how to operate on. Although this method is slightly more work, the benefits of a stricter typing of ports is beneficial - particularly upon interfacing with other packages that may depend on strongly typed constants (for example).

== The Console ==

=== Where should I go to find out what I can call from the console and how to import it? ===

We have tried to make some methods more accessible in the console via an api. You can import the api via <code>import api</code> in the console and see the available methods with <code>dir(api)</code>. To open a vistrail:

import api
api.open_vistrail_from_file('/Applications/VisTrails/examples/terminator.vt')

To execute a version of a workflow, you currently have to go through the controller:

api.select_version('Histogram')
api.get_current_controller().execute_current_workflow()

Currently, only a subset of VisTrails functionality is directly available from the api. However, since VisTrails is written in python, you can dig down starting with the VistrailsApplication or controller object to expose most of our internal methods. If you have suggestions for calls to be added to the api, please let us know.

One other feature that we're working on, but is still in progress is the ability to construct workflows via the console. For example:

vtk = load_package('edu.utah.sci.vistrails.vtk')
vtk.vtkDataSetReader() # adds a vtkDataSetReader module to the pipeline
# click on the new module
a = selected_modules()[0] # get the one currently selected module
a.SetFile('/vistrails/examples/data/head120.vtk') # sets the SetFile parmaeter for the data set reader
b = vtk.vtkContourFilter() # adds a vtkContourFilter module to the pipeline and saves to var b
b.SetInputConnection0(a.GetOutputPort0()) # connects a's GetOutputPort0 port to b's SetInputConnection0

== Persistence Package ==

=== How do I use the output of one workflow as the input for another using the persistence package? ===

You need to configure the persistence modules using the module's configuration dialog. After adding a <code>PersistentOutputFile</code> to the workflow, click on the triangle in the upper-right corner of the <code>PersistentOutputFile</code>, and select "Edit Configuration" from the menu that appears. In this dialog, select "Create New Reference" and give the reference a name (and any space-delimited tags). Upon running that workflow, the data will be written to the persistent store. In the second workflow where you wish to use that file, add a <code>PersistentInputFile</code> and go to its configuration dialog in the same manner as with the output file. In that dialog, select "Use Existing Reference" and select the data that you just added in the first workflow in the list of files below. Now, when you run that workflow, it will grab the data from the persistent store.

Here is an example: [[Media:offscreen_persistent.vt | offscreen_persistent.vt]]. Run the "persistent offscreen" workflow first, and then run the "display persistent output" to use the output of the first workflow as the input for the second.

== VTK ==

=== Given a VTK visualization, how can I generate a webpage from it? ===

Check out the html pipeline in offscreen.xml.

=== I'm trying to use VTK, but there doesn't seem to be any output. What is wrong? ===

To use VTK on VisTrails, you need a slightly different way of connecting the renderer modules. Instead of using the standard RenderWindow/RenderWindowInteractor infrastructure, you simply connect the renderer to a VTKCell. The examples directory in the distribution has several VTK examples that illustrate.

=== I am trying to add a module to the workflow via Python, but how can I access vtk modules? ===

Here's an example:

import api

vtvtk = 'edu.utah.sci.vistrails.vtk'

module = api.add_module(0, 0, vtvtk, 'vtkContourFilter', '')

The third argument in add_module is the package identifier. You can find this in the "Module Packages" panel of the Preferences; just click on the package you're interested in and it will appear in the information on the right.

== matplotlib ==

=== I'm experiencing a problem with Latex labels and the matplotlib that comes with VisTrails 1.5. The script below entered to the interpreter that comes with VT is sufficient to reproduce it. ===

import matplotlib.pyplot as plt
plt.plot([1,2,3],[1,2,3])
plt.xlabel("$foo$")

Remove your ~/.matplotlib folder and re-start VisTrails

== VisTrails Development ==

=== I would like to build VisTrails from source. Are there instructions on how to do this? ===

Yes! Take a look at http://www.vistrails.org/index.php/Mac_Intel_Instructions
[http://www.vistrails.org/usersguide/getting_started.html#installing-vistrails-from-source Installing VisTrails from source]

== Accessing Provenance Information ==

=== How do I access the information in the execution log? ===

The code responsible for storing execution information is located in the "core/log" directories, and the code that generates much of that information is in "core/interpreter/cached.py". Modules can add execution-specific annotations to provenance via annotate() calls during execution, but much of the data (like timing and errors) is captured by the LogController and CachedInterpreter (the execution engine) objects. To analyze the log from a vistrail (.vt) file, you might have something like the following:

<code>
import core.log.log
import db.services.io

def run(fname):
# open the .vt bundle specified by the filename "fname"
bundle = db.services.io.open_vistrail_bundle_from_zip_xml(fname)[0]
# get the log filename
log_fname = bundle.vistrail.db_log_filename
if log_fname is not None:
# open the log
log = db.services.io.open_log_from_xml(log_fname, True)
# convert the log from a db object
core.log.log.Log.convert(log)
for workflow_exec in log.workflow_execs:
print 'workflow version:', workflow_exec.parent_version
print 'time started:', workflow_exec.ts_start
print 'time ended:', workflow_exec.ts_end
print 'modules executed:', [i.module_id
for i in workflow_exec.item_execs]
if __name__ == '__main__':
run("some_vistrail.vt")
</code>

You should be able to see what information is available by looking at the "core/log" classes.

FAQ

2011-03-25T16:51:21Z

Ctuttle: /* Building workflows */

Also check our [[known_issues|Known Issues]] page for troubleshooting.

----
== Running workflows ==

=== How can I run a workflow using the command line? ===

('''Updated for version 1.2''') Call vistrails using the following options:

<code>python vistrails.py -b path_to_vistrails_file:pipeline </code>

where pipeline can be a version '''tag name''' or version '''id'''

'''NOTE:''' If you downloaded the MacOS X bundle, you can run vistrails from the command line via the following commands in the Terminal. Change the current directory to wherever VisTrails was installed (often <code>/Applications</code>), and then type:

<code>Vistrails.app/Contents/MacOS/vistrails [<cmd_line_options>]</code>

=== Using the command line, we'd like to execute a workflow multiple times, with slightly different parameters, and create a series of output files. Is this possible?===

('''Updated for version 1.2''') We can change parameters that have an alias through the command line.

For example, offscreen pipeline in offscreen.vt always creates the file called image.png. If you want generate it with a different filename:

<code>python vistrails.py -b ../examples/offscreen.vt:offscreen -a"filename=other.png"</code>

<code>filename</code> in the example above is the alias name assigned to the parameter in the <code>value</code> method inside the String module. When running a pipeline from the command line, VisTrails will try to start the spreadsheet automatically if the pipeline requires it. For example, this other execution will also start the spreadsheet (attention to how $ characters are escaped when running on bash):

<code>python vistrails.py -b ../examples/head.vt:aliases -a"isovalue=30\$&\$diffuse_color=0.8,0.4,0.2"</code>

You can also execute more than one pipeline on the command line:

<code>python vistrails.py -b ../examples/head.vt:aliases ../examples/spx.vt:spx \
-a"isovalue=30"</code>

Use the -a parameter only once regardless the number of pipelines.

=== I can load a vistrail, and the version tree shows up fine. However, no pipelines appear when I click on a version. What gives? ===

The most likely reason is that the vistrail uses a package that is not registered with VisTrails. You need to identify the needed package and add it to your <code>.vistrails/startup.py</code>. A single line like the following should be enough:

addPackage('enter_package_name_here')

Some packages might need more information. For example:

addPackage('afront', executable_path='/path/to/afront')

Refer to the package documentation for details. The one inconvenient step is that currently there's no automated way to describe what is the missing package. We're working on this feature for future releases.

=== I have a workflow that reads a file and then does some processing. The first time it runs, it executes correctly. But in subsequent, nothing happens. ===

VisTrails caches by default, so after a workflow is executed, if none of its parameters change, it won't be executed again.

If a workflow reads a file using the basic module File, VisTrails does check whether the file was modified since the last run. It does so by keeping a signature that is based on the modification time of the file. And if the file was modified, the File module and all downstream modules (the ones which depend on File) will be executed.

''Note: If you would like your input and output data to be versioned, you can use the Persistence package.''

If you do not want VisTrails to cache executions, you can turn off caching: go to Menu Edit -> Preferences and in the General Configuration tab, change Cache execution results to Never.

=== Can VisTrails execute workflows in parallel? ===

The VisTrails server can only execute pipelines in parallel if there's more than one instance of VisTrails running. The command

<code> self.rpcserver = ThreadedXMLRPCServer((self.temp_xml_rpc_options.server, self.temp_xml_rpc_options.port))</code>

starts a multithreaded version of the XML-RPC server, so it will create a thread for each request received by the server. The problem is that Qt/PyQT doesn't allow these multiple threads create GUI objects, only in the main thread. To overcome this limitation, the multithreaded version can instantiate other single threaded versions of VisTrails and put them in a queue, so workflow executions and other GUI-related requests, such as generating workflow graphs and history trees can be forwarded to this queue, and each instance takes turns in answering the request. If the results are in the cache, the multithreaded version answers the requests directly.

Note that this infrastructure works on Linux only. To make this work on Windows, you have to create a script similar to start_vistrails_xvfb.sh (located in the scripts folder) where you can send the number of other instances via command-line options to VisTrails. The command line options are:

python vistrails_server.py -T <ADDRESS> -R <PORT> -O<NUMBER_OF_OTHER_VISTRAILS_INSTANCES> [-M]&

If you want the main vistrails instance to be multithreaded, use the -M at the end.

After creating this script, update function start_other_instances in vistrails/gui/application_server.py lines 1007-1023 and set the script variable to point to your script. You may also have to change the arguments sent to your script (line 1016: for example, you don't need to set a virtual display). You will need to change the path to the stop_vistrails_server.py script (on line 1026) according to your installation path.

=== When a workflow is executed, what do the colors mean? ===

- lilac: module was notexecuted

- yellow: module is currently being executed

- green: module was successfully executed

- orange: module was cached

- red: the execution of the module failed

== Building workflows ==

=== Is there a way to give each widget a "display name" in addition to the module name at the center of the widget? ===

Yes, a "display name" can be assigned to a module by selecting the triangle in its top right corner to open a popup menu and selecting the Set Module Label... menu item. You will then be prompted to enter the "display name". [http://www.vistrails.org/usersguide/creating.html#changing-module-labels Changing Module Labels]

=== Is there a way to re-center the picture-in-picture (PiP) view? ===

Yes. If you click on the PIP window to bring it to focus, you can press Ctrl-R (or Command-R on Mac) to re-center the PiP window.

=== How do I search for a literal "?" (question mark) in the search box in the Property panel? ===

Since we allow regular expressions in our search box, question marks are treated as meta-characters. Thus, searching for "?" returns everything and "abc?" will return everything containing "abc". You need to use "\?" instead to search for "?". So the search for "??" would be "\?\?". [http://www.vistrails.org/usersguide/querying.html#textual-queries Textual Queries]

== Using VisTrails as a server ==

=== What is the VisTrails server-mode? ===

Using the VisTrails server mode, it is possible to ''execute workflows and control VisTrails through another application''. For example, the CrowdLabs Web portal (http://www.crowdlabs.org) accesses a VisTrails sever to execute workflows, retrieve and display vistrail trees and workflows. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

=== How do I execute workflows and control VisTrails through another application? ===

The way you access the server is by doing XML-RPC calls. In the current VisTrails release, we include a set of PHP scripts that can talk to a VisTrails server instance. They are in "extensions/http" folder. The files are reasonably well documented. Also, it should be not difficult to create python scripts to access the server (just use xmlrpclib module).

Note that the VisTrails server requires the provenance and workflows to be in a database. More detailed instructions on how to setup the server and the database
are available here:

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_server_setup.html

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_database_setup.html

If what you want is just to execute a series of workflows in batch mode, a simpler solution would be to ''use the VisTrails client in batch mode''. Chapter 12 of the user's guide contains detailed information and examples on that. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

== Control Flow ==

=== Note: using map ===

When using 'map', the module (or subworkflow) used as function port in the map module MUST be a function, i.e., it can only define 1 output port. [http://www.vistrails.org/usersguide/controlflow.html#the-map-operator The Map Operator]

== Spreadsheet ==

Where ''pipeline'' is a version number or a tag.

=== How can I save an image from the spreadsheet? ===

While having the focus on a spreadsheet cell and select the camera on the toolbar to take a snapshot. The system will prompt you for the location and file name where it should be saved. The other icons can be used for saving multiple images that can be used for generating an animation on demand. A whole sheet can also be saved by selecting Export (either from the menu or from the toolbar). [http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet-image Saving a Spreadsheet Image]

===Is it possible to save the complete state of the spreadsheet?===
[http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet Saving a Spreadsheet]

=== Can I view multiple sheets at the same time? ===

Yes. Each sheet on the spreadsheet can be displayed as a dock widget separated from the main spreadsheet window by dragging its tab name out of the tab bar at the bottom of the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Then, how can I put back a separated sheet? ===

A sheet can be docked back to the main window by dragging it back to the tab bar or double-click on its title bar. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== How can I order sheets on the spreadsheet? ===

This can be done by dragging the sheet name on the bottom top bar and drop it to the right place. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Can I control where a cell will be placed on the spreadsheet window? ===

By default, an unoccupied cell on the active sheet will be chosen to display the result. However, you can specify exactly in the pipeline where a spreadsheet cell will be placed by using CellLocation and SheetReference. CellLocation specifies the location (row and column) of a cell when connecting to a spreadsheet cell (VTKCell, ImageViewerCell, ...). Similarly, a SheetReference module (when connecting to a CellLocation) will specify which sheet the cell will be put on given its name, minimum row size and minimum column size. There is an example of this in examples/vtk.xml (select the version below Double Renderer). [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

=== How do I output results to the spreadsheet? ===

By inspecting the VisTrails Spreadsheet package (in the list of packages, to the left of the pipeline builder), you
can see there are built-in cells for different kinds of data, e.g., RichTextCell to display HTML and plain text. op
You (the user) can also define new cell types to display application-specific data. For example, we have developed
VtkCell, MplFigureCell, and OpenGLCell. It is possible to display pretty much anything on the Spreadsheet! [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

Examples of writing cell modules can be found in:
RichTextCell: packages/spreadsheet/widgets/richtext/richtext.py
VTK: packages/vtk/vtkcell.py

Here is the summary of some requirements on a cell widget:

(1) It must be a Qt widget. It should inherit from spreadsheet_cell.QCellWidget in the spreadsheet package. Although any Qt Widget would work, certain features such as animation will not be available (without rewriting it).

(2) It must re-implement the updateContents() function to take a set of inputs (usually coming from input ports of a wrapper Module) and display on the cells. VisTrails uses this function to update/reuse cells on the spreadsheet when new data comes in.

(3) It needs a wrapper VisTrails Module (inherited from basic_widgets.SpreadsheetCell of the spreadsheet package). Inside the compute() method of this module, it may call self.display(CellWidgetType, (inputs)) to trigger the display event on the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#advanced-cell-options Advanced Cell Options]

=== How do I control the default number of cells in the spreadsheet? ===

You can configure the rowCount and colCount using the preferences dialog. Just go to the Module Packages tab, select spreadsheet in the "Enabled packages" and press the Configure button. Then a list of all the configuration options for the spreadsheet will show up. [http://www.vistrails.org/usersguide/spreadsheet.html#custom-layout-options Custom Layout Options]

=== Is it possible to launch a web browser from the vistrails spreadsheet? We would like to output several urls from a parameter sweep and then have the option to click on each one to view the resulting page. I can view the page within the spreadsheet, but it is really too crowded.===

Currently, there isn't a widget that provides exactly this functionality, but I can think of a few solutions that may work for you:

(1) You can use parameter exploration to generate multiple sheets so you might have an exploration that opens each page in a new sheet. Use the third column/dimension in the exploration interface to have a parameter span sheets.

(2) The spreadsheet is extensible so you can write a custom spreadsheet cell widget that has a button or label with the desired link (a QLabel with openExternalLinks set to True, for example).

(3) You can tweak the existing RichTextCell be adding the line "self.browser.setOpenExternalLinks(True)" at line 63 of the source file "vistrails/packages/spreadsheet/widgets/richtext/richtext.py". Then, if your workflow creates a file with html markup text like "<a href="http://www.vistrails.org">VisTrails</a>" connected to a RichTextCell, clicking on the rendered link in the cell will open it in a web browser. You need to add the aforementioned line to the source to let Qt know that you want the link opened externally; by default, it will just issue an event that isn't processed. [http://www.vistrails.org/usersguide/spreadsheet.html#launching-a-web-browser Launching a Web Browser]

== Integrating your software into VisTrails ==

===How can I integrate my own program into VisTrails?===

The easiest way is to create a package. Writing a package is often very simple, here are instructions on how to do it: [[UsersGuideVisTrailsPackages]]

You can also dynamically generate modules. For an example see:

http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#Packages_that_generate_modules_dynamically

In particular, see the new_module call which uses python's type() function to generate new classes dynamically.

=== How do modules deal with multiple inputs in a same port? ===

(And should that even be allowed?)

For compatibility reasons, we do need to allow multiple connections to an input port. However, most package developers should never have to use this, and so we do our best to hide it. the default behavior for getting inputs from a port, then, is to always return a single input.

If on your module you need multiple inputs connected to a single port, use the 'forceGetInputListFromPort' method. It will return a list of all the data items coming through the port. The VTK package uses this feature, so look there for usage examples (packages/vtk/base_widget.py)

=== Are there mechanisms for attaching widgets to different modules/parameters? ===

Right now, we have a mechanism for putting a specific widget for an input port. For example, if a port is SetColor(red, green, blue), we can put a color wheel widget there. Or we can also replace the SetFileName port with a File Widget. However, this is not per parameter (only per port). We are currently working on this problem.

=== Can I organize my package so it appears hierarchical in the module palette? ===

Yes. Use the namespace keyword argument when adding the module to the registry. For example,

registry.add_module(MyModule, namespace='MyNamespace')

=== Can I nest namespaces? ===

Yes. Use the '|' character to separate different the hierarchy. For example,

registry.add_module(MyModule, namespace='ParentNamespace|ChildNamespace')

=== Are there shortucts for registry initialization? ===

Yes. If you define _modules as a list of classes in the __init__.py file of your package, VisTrails will attempt to load all classes specified as modules. You can provide add_module options as keyword arguments by specifying a tuple (class, kwargs) in the list. For example:

_modules = [MyModule1, (MyModule2, {'namespace': 'MyNamespace'})]

In addition, you need to identify the ports of your modules as a field in your class by defining _input_ports and _output_ports lists. Here, the items in each list must be tuples of the form (portName, portSignature, optional=False, sort_key=-1). For example:

class MyModule(Module):
def compute(self):
pass

_input_ports = [('firstInput', String), ('secondInput', Integer, True)]
_output_ports = [('firstOutput', String), ('secondOutput', String)]

=== Can I define ports to be of types that I do not import into my package? ===

Yes. You can pass an identifier string as the portSignature instead. The port_signature string is defined by:

<module_string> := <package_identifier>:[<namespace>|]<module_name>,
<port_signature> := (<module_string>*)

For example,

registry.add_input_port(MyModule, 'myInputPort', '(edu.utah.sci.vistrails.basic:String)')

or

_input_ports = [('myInputPort', '(edu.utah.sci.vistrails.basic:String)')]

=== What do I need to change in my package to make it reloadable (new in v1.4.2)? ===
See [http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#How_to_make_you_package_reloadable UsersGuideVisTrailsPackages] for an explanation.

=== Can I add default values or labels for parameters? ===

Yes. Versions 1.4 and greater support these features. See [[UsersGuideVisTrailsPackages#Adding_default_values_and.2For_labels_for_parameters | UsersGuideVisTrailsPacakges]] for more details.

=== I want to write a module to load HDF data whose output (e.g., data, string) varies according to the input I give it. Is is possible to do this in VisTrails, and if yes, how can I do that? Ideally, I would like to avoid having to change the connection of my output every time I change the input. ===

There are a few ways to tackle this - each has it's own benefits and pitfalls. Firstly, module connections do respect class hierarchies as we're familiar with in object oriented languages. For instance, A module can output a Constant of which String, Float, Integer, etc are specifications. In this way, you can have a subclass of something like HDFData be passed out of the module and the connections will be established regardless of the sub-type. This is a bit dangerous though. Modules downstream of such a class may not really know how to operate on certain types derived from the super-class. Extreme care must be taken both when creating the modules as well as connecting them to prevent things like this from happening.

A second method that I employ in several different packages is the idea of a container class. For instance, the NumSciPy package uses a relatively generic container "Numpy Array" to encapsulate the data. Of course, these encapsulating objects can store dictionaries that other modules can easily access and understand how to operate on. Although this method is slightly more work, the benefits of a stricter typing of ports is beneficial - particularly upon interfacing with other packages that may depend on strongly typed constants (for example).

== The Console ==

=== Where should I go to find out what I can call from the console and how to import it? ===

We have tried to make some methods more accessible in the console via an api. You can import the api via <code>import api</code> in the console and see the available methods with <code>dir(api)</code>. To open a vistrail:

import api
api.open_vistrail_from_file('/Applications/VisTrails/examples/terminator.vt')

To execute a version of a workflow, you currently have to go through the controller:

api.select_version('Histogram')
api.get_current_controller().execute_current_workflow()

Currently, only a subset of VisTrails functionality is directly available from the api. However, since VisTrails is written in python, you can dig down starting with the VistrailsApplication or controller object to expose most of our internal methods. If you have suggestions for calls to be added to the api, please let us know.

One other feature that we're working on, but is still in progress is the ability to construct workflows via the console. For example:

vtk = load_package('edu.utah.sci.vistrails.vtk')
vtk.vtkDataSetReader() # adds a vtkDataSetReader module to the pipeline
# click on the new module
a = selected_modules()[0] # get the one currently selected module
a.SetFile('/vistrails/examples/data/head120.vtk') # sets the SetFile parmaeter for the data set reader
b = vtk.vtkContourFilter() # adds a vtkContourFilter module to the pipeline and saves to var b
b.SetInputConnection0(a.GetOutputPort0()) # connects a's GetOutputPort0 port to b's SetInputConnection0

== Persistence Package ==

=== How do I use the output of one workflow as the input for another using the persistence package? ===

You need to configure the persistence modules using the module's configuration dialog. After adding a <code>PersistentOutputFile</code> to the workflow, click on the triangle in the upper-right corner of the <code>PersistentOutputFile</code>, and select "Edit Configuration" from the menu that appears. In this dialog, select "Create New Reference" and give the reference a name (and any space-delimited tags). Upon running that workflow, the data will be written to the persistent store. In the second workflow where you wish to use that file, add a <code>PersistentInputFile</code> and go to its configuration dialog in the same manner as with the output file. In that dialog, select "Use Existing Reference" and select the data that you just added in the first workflow in the list of files below. Now, when you run that workflow, it will grab the data from the persistent store.

Here is an example: [[Media:offscreen_persistent.vt | offscreen_persistent.vt]]. Run the "persistent offscreen" workflow first, and then run the "display persistent output" to use the output of the first workflow as the input for the second.

== VTK ==

=== Given a VTK visualization, how can I generate a webpage from it? ===

Check out the html pipeline in offscreen.xml.

=== I'm trying to use VTK, but there doesn't seem to be any output. What is wrong? ===

To use VTK on VisTrails, you need a slightly different way of connecting the renderer modules. Instead of using the standard RenderWindow/RenderWindowInteractor infrastructure, you simply connect the renderer to a VTKCell. The examples directory in the distribution has several VTK examples that illustrate.

=== I am trying to add a module to the workflow via Python, but how can I access vtk modules? ===

Here's an example:

import api

vtvtk = 'edu.utah.sci.vistrails.vtk'

module = api.add_module(0, 0, vtvtk, 'vtkContourFilter', '')

The third argument in add_module is the package identifier. You can find this in the "Module Packages" panel of the Preferences; just click on the package you're interested in and it will appear in the information on the right.

== matplotlib ==

=== I'm experiencing a problem with Latex labels and the matplotlib that comes with VisTrails 1.5. The script below entered to the interpreter that comes with VT is sufficient to reproduce it. ===

import matplotlib.pyplot as plt
plt.plot([1,2,3],[1,2,3])
plt.xlabel("$foo$")

Remove your ~/.matplotlib folder and re-start VisTrails

== VisTrails Development ==

=== I would like to build VisTrails from source. Are there instructions on how to do this? ===

Yes! Take a look at http://www.vistrails.org/index.php/Mac_Intel_Instructions
[http://www.vistrails.org/usersguide/getting_started.html#installing-vistrails-from-source Installing VisTrails from source]

== Accessing Provenance Information ==

=== How do I access the information in the execution log? ===

The code responsible for storing execution information is located in the "core/log" directories, and the code that generates much of that information is in "core/interpreter/cached.py". Modules can add execution-specific annotations to provenance via annotate() calls during execution, but much of the data (like timing and errors) is captured by the LogController and CachedInterpreter (the execution engine) objects. To analyze the log from a vistrail (.vt) file, you might have something like the following:

<code>
import core.log.log
import db.services.io

def run(fname):
# open the .vt bundle specified by the filename "fname"
bundle = db.services.io.open_vistrail_bundle_from_zip_xml(fname)[0]
# get the log filename
log_fname = bundle.vistrail.db_log_filename
if log_fname is not None:
# open the log
log = db.services.io.open_log_from_xml(log_fname, True)
# convert the log from a db object
core.log.log.Log.convert(log)
for workflow_exec in log.workflow_execs:
print 'workflow version:', workflow_exec.parent_version
print 'time started:', workflow_exec.ts_start
print 'time ended:', workflow_exec.ts_end
print 'modules executed:', [i.module_id
for i in workflow_exec.item_execs]
if __name__ == '__main__':
run("some_vistrail.vt")
</code>

You should be able to see what information is available by looking at the "core/log" classes.

FAQ

2011-03-25T01:43:57Z

Ctuttle: /* Building workflows */

Also check our [[known_issues|Known Issues]] page for troubleshooting.

----
== Running workflows ==

=== How can I run a workflow using the command line? ===

('''Updated for version 1.2''') Call vistrails using the following options:

<code>python vistrails.py -b path_to_vistrails_file:pipeline </code>

where pipeline can be a version '''tag name''' or version '''id'''

'''NOTE:''' If you downloaded the MacOS X bundle, you can run vistrails from the command line via the following commands in the Terminal. Change the current directory to wherever VisTrails was installed (often <code>/Applications</code>), and then type:

<code>Vistrails.app/Contents/MacOS/vistrails [<cmd_line_options>]</code>

=== Using the command line, we'd like to execute a workflow multiple times, with slightly different parameters, and create a series of output files. Is this possible?===

('''Updated for version 1.2''') We can change parameters that have an alias through the command line.

For example, offscreen pipeline in offscreen.vt always creates the file called image.png. If you want generate it with a different filename:

<code>python vistrails.py -b ../examples/offscreen.vt:offscreen -a"filename=other.png"</code>

<code>filename</code> in the example above is the alias name assigned to the parameter in the <code>value</code> method inside the String module. When running a pipeline from the command line, VisTrails will try to start the spreadsheet automatically if the pipeline requires it. For example, this other execution will also start the spreadsheet (attention to how $ characters are escaped when running on bash):

<code>python vistrails.py -b ../examples/head.vt:aliases -a"isovalue=30\$&\$diffuse_color=0.8,0.4,0.2"</code>

You can also execute more than one pipeline on the command line:

<code>python vistrails.py -b ../examples/head.vt:aliases ../examples/spx.vt:spx \
-a"isovalue=30"</code>

Use the -a parameter only once regardless the number of pipelines.

=== I can load a vistrail, and the version tree shows up fine. However, no pipelines appear when I click on a version. What gives? ===

The most likely reason is that the vistrail uses a package that is not registered with VisTrails. You need to identify the needed package and add it to your <code>.vistrails/startup.py</code>. A single line like the following should be enough:

addPackage('enter_package_name_here')

Some packages might need more information. For example:

addPackage('afront', executable_path='/path/to/afront')

Refer to the package documentation for details. The one inconvenient step is that currently there's no automated way to describe what is the missing package. We're working on this feature for future releases.

=== I have a workflow that reads a file and then does some processing. The first time it runs, it executes correctly. But in subsequent, nothing happens. ===

VisTrails caches by default, so after a workflow is executed, if none of its parameters change, it won't be executed again.

If a workflow reads a file using the basic module File, VisTrails does check whether the file was modified since the last run. It does so by keeping a signature that is based on the modification time of the file. And if the file was modified, the File module and all downstream modules (the ones which depend on File) will be executed.

''Note: If you would like your input and output data to be versioned, you can use the Persistence package.''

If you do not want VisTrails to cache executions, you can turn off caching: go to Menu Edit -> Preferences and in the General Configuration tab, change Cache execution results to Never.

=== Can VisTrails execute workflows in parallel? ===

The VisTrails server can only execute pipelines in parallel if there's more than one instance of VisTrails running. The command

<code> self.rpcserver = ThreadedXMLRPCServer((self.temp_xml_rpc_options.server, self.temp_xml_rpc_options.port))</code>

starts a multithreaded version of the XML-RPC server, so it will create a thread for each request received by the server. The problem is that Qt/PyQT doesn't allow these multiple threads create GUI objects, only in the main thread. To overcome this limitation, the multithreaded version can instantiate other single threaded versions of VisTrails and put them in a queue, so workflow executions and other GUI-related requests, such as generating workflow graphs and history trees can be forwarded to this queue, and each instance takes turns in answering the request. If the results are in the cache, the multithreaded version answers the requests directly.

Note that this infrastructure works on Linux only. To make this work on Windows, you have to create a script similar to start_vistrails_xvfb.sh (located in the scripts folder) where you can send the number of other instances via command-line options to VisTrails. The command line options are:

python vistrails_server.py -T <ADDRESS> -R <PORT> -O<NUMBER_OF_OTHER_VISTRAILS_INSTANCES> [-M]&

If you want the main vistrails instance to be multithreaded, use the -M at the end.

After creating this script, update function start_other_instances in vistrails/gui/application_server.py lines 1007-1023 and set the script variable to point to your script. You may also have to change the arguments sent to your script (line 1016: for example, you don't need to set a virtual display). You will need to change the path to the stop_vistrails_server.py script (on line 1026) according to your installation path.

=== When a workflow is executed, what do the colors mean? ===

- lilac: module was notexecuted

- yellow: module is currently being executed

- green: module was successfully executed

- orange: module was cached

- red: the execution of the module failed

== Building workflows ==

=== Is there a way to give each widget a "display name" in addition to the module name at the center of the widget? ===

Yes, a "display name" can be assigned to a module by selecting the triangle in its top right corner to open a popup menu and selecting the Set Module Label... menu item. You will then be prompted to enter the "display name". [http://www.vistrails.org/usersguide/creating.html#changing-module-labels Changing Module Labels]

=== Is there a way to re-center the picture-in-picture (PiP) view? ===

Yes. If you click on the PIP window to bring it to focus, you can press Ctrl-R (or Command-R on Mac) to re-center the PiP window.

=== How do I search for a literal "?" (question mark) in the search box in the Property panel? ===

Since we allow regular expressions in our search box, question marks are treated as meta-characters. Thus, searching for "?" returns everything and "abc?" will return everything containing "abc". You need to use "\?" instead to search for "?". So the search for "??" would be "\?\?".

== Using VisTrails as a server ==

=== What is the VisTrails server-mode? ===

Using the VisTrails server mode, it is possible to ''execute workflows and control VisTrails through another application''. For example, the CrowdLabs Web portal (http://www.crowdlabs.org) accesses a VisTrails sever to execute workflows, retrieve and display vistrail trees and workflows. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

=== How do I execute workflows and control VisTrails through another application? ===

The way you access the server is by doing XML-RPC calls. In the current VisTrails release, we include a set of PHP scripts that can talk to a VisTrails server instance. They are in "extensions/http" folder. The files are reasonably well documented. Also, it should be not difficult to create python scripts to access the server (just use xmlrpclib module).

Note that the VisTrails server requires the provenance and workflows to be in a database. More detailed instructions on how to setup the server and the database
are available here:

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_server_setup.html

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_database_setup.html

If what you want is just to execute a series of workflows in batch mode, a simpler solution would be to ''use the VisTrails client in batch mode''. Chapter 12 of the user's guide contains detailed information and examples on that. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

== Control Flow ==

=== Note: using map ===

When using 'map', the module (or subworkflow) used as function port in the map module MUST be a function, i.e., it can only define 1 output port. [http://www.vistrails.org/usersguide/controlflow.html#the-map-operator The Map Operator]

== Spreadsheet ==

Where ''pipeline'' is a version number or a tag.

=== How can I save an image from the spreadsheet? ===

While having the focus on a spreadsheet cell and select the camera on the toolbar to take a snapshot. The system will prompt you for the location and file name where it should be saved. The other icons can be used for saving multiple images that can be used for generating an animation on demand. A whole sheet can also be saved by selecting Export (either from the menu or from the toolbar). [http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet-image Saving a Spreadsheet Image]

===Is it possible to save the complete state of the spreadsheet?===
[http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet Saving a Spreadsheet]

=== Can I view multiple sheets at the same time? ===

Yes. Each sheet on the spreadsheet can be displayed as a dock widget separated from the main spreadsheet window by dragging its tab name out of the tab bar at the bottom of the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Then, how can I put back a separated sheet? ===

A sheet can be docked back to the main window by dragging it back to the tab bar or double-click on its title bar. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== How can I order sheets on the spreadsheet? ===

This can be done by dragging the sheet name on the bottom top bar and drop it to the right place. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Can I control where a cell will be placed on the spreadsheet window? ===

By default, an unoccupied cell on the active sheet will be chosen to display the result. However, you can specify exactly in the pipeline where a spreadsheet cell will be placed by using CellLocation and SheetReference. CellLocation specifies the location (row and column) of a cell when connecting to a spreadsheet cell (VTKCell, ImageViewerCell, ...). Similarly, a SheetReference module (when connecting to a CellLocation) will specify which sheet the cell will be put on given its name, minimum row size and minimum column size. There is an example of this in examples/vtk.xml (select the version below Double Renderer). [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

=== How do I output results to the spreadsheet? ===

By inspecting the VisTrails Spreadsheet package (in the list of packages, to the left of the pipeline builder), you
can see there are built-in cells for different kinds of data, e.g., RichTextCell to display HTML and plain text. op
You (the user) can also define new cell types to display application-specific data. For example, we have developed
VtkCell, MplFigureCell, and OpenGLCell. It is possible to display pretty much anything on the Spreadsheet! [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

Examples of writing cell modules can be found in:
RichTextCell: packages/spreadsheet/widgets/richtext/richtext.py
VTK: packages/vtk/vtkcell.py

Here is the summary of some requirements on a cell widget:

(1) It must be a Qt widget. It should inherit from spreadsheet_cell.QCellWidget in the spreadsheet package. Although any Qt Widget would work, certain features such as animation will not be available (without rewriting it).

(2) It must re-implement the updateContents() function to take a set of inputs (usually coming from input ports of a wrapper Module) and display on the cells. VisTrails uses this function to update/reuse cells on the spreadsheet when new data comes in.

(3) It needs a wrapper VisTrails Module (inherited from basic_widgets.SpreadsheetCell of the spreadsheet package). Inside the compute() method of this module, it may call self.display(CellWidgetType, (inputs)) to trigger the display event on the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#advanced-cell-options Advanced Cell Options]

=== How do I control the default number of cells in the spreadsheet? ===

You can configure the rowCount and colCount using the preferences dialog. Just go to the Module Packages tab, select spreadsheet in the "Enabled packages" and press the Configure button. Then a list of all the configuration options for the spreadsheet will show up. [http://www.vistrails.org/usersguide/spreadsheet.html#custom-layout-options Custom Layout Options]

=== Is it possible to launch a web browser from the vistrails spreadsheet? We would like to output several urls from a parameter sweep and then have the option to click on each one to view the resulting page. I can view the page within the spreadsheet, but it is really too crowded.===

Currently, there isn't a widget that provides exactly this functionality, but I can think of a few solutions that may work for you:

(1) You can use parameter exploration to generate multiple sheets so you might have an exploration that opens each page in a new sheet. Use the third column/dimension in the exploration interface to have a parameter span sheets.

(2) The spreadsheet is extensible so you can write a custom spreadsheet cell widget that has a button or label with the desired link (a QLabel with openExternalLinks set to True, for example).

(3) You can tweak the existing RichTextCell be adding the line "self.browser.setOpenExternalLinks(True)" at line 63 of the source file "vistrails/packages/spreadsheet/widgets/richtext/richtext.py". Then, if your workflow creates a file with html markup text like "<a href="http://www.vistrails.org">VisTrails</a>" connected to a RichTextCell, clicking on the rendered link in the cell will open it in a web browser. You need to add the aforementioned line to the source to let Qt know that you want the link opened externally; by default, it will just issue an event that isn't processed. [http://www.vistrails.org/usersguide/spreadsheet.html#launching-a-web-browser Launching a Web Browser]

== Integrating your software into VisTrails ==

===How can I integrate my own program into VisTrails?===

The easiest way is to create a package. Writing a package is often very simple, here are instructions on how to do it: [[UsersGuideVisTrailsPackages]]

You can also dynamically generate modules. For an example see:

http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#Packages_that_generate_modules_dynamically

In particular, see the new_module call which uses python's type() function to generate new classes dynamically.

=== How do modules deal with multiple inputs in a same port? ===

(And should that even be allowed?)

For compatibility reasons, we do need to allow multiple connections to an input port. However, most package developers should never have to use this, and so we do our best to hide it. the default behavior for getting inputs from a port, then, is to always return a single input.

If on your module you need multiple inputs connected to a single port, use the 'forceGetInputListFromPort' method. It will return a list of all the data items coming through the port. The VTK package uses this feature, so look there for usage examples (packages/vtk/base_widget.py)

=== Are there mechanisms for attaching widgets to different modules/parameters? ===

Right now, we have a mechanism for putting a specific widget for an input port. For example, if a port is SetColor(red, green, blue), we can put a color wheel widget there. Or we can also replace the SetFileName port with a File Widget. However, this is not per parameter (only per port). We are currently working on this problem.

=== Can I organize my package so it appears hierarchical in the module palette? ===

Yes. Use the namespace keyword argument when adding the module to the registry. For example,

registry.add_module(MyModule, namespace='MyNamespace')

=== Can I nest namespaces? ===

Yes. Use the '|' character to separate different the hierarchy. For example,

registry.add_module(MyModule, namespace='ParentNamespace|ChildNamespace')

=== Are there shortucts for registry initialization? ===

Yes. If you define _modules as a list of classes in the __init__.py file of your package, VisTrails will attempt to load all classes specified as modules. You can provide add_module options as keyword arguments by specifying a tuple (class, kwargs) in the list. For example:

_modules = [MyModule1, (MyModule2, {'namespace': 'MyNamespace'})]

In addition, you need to identify the ports of your modules as a field in your class by defining _input_ports and _output_ports lists. Here, the items in each list must be tuples of the form (portName, portSignature, optional=False, sort_key=-1). For example:

class MyModule(Module):
def compute(self):
pass

_input_ports = [('firstInput', String), ('secondInput', Integer, True)]
_output_ports = [('firstOutput', String), ('secondOutput', String)]

=== Can I define ports to be of types that I do not import into my package? ===

Yes. You can pass an identifier string as the portSignature instead. The port_signature string is defined by:

<module_string> := <package_identifier>:[<namespace>|]<module_name>,
<port_signature> := (<module_string>*)

For example,

registry.add_input_port(MyModule, 'myInputPort', '(edu.utah.sci.vistrails.basic:String)')

or

_input_ports = [('myInputPort', '(edu.utah.sci.vistrails.basic:String)')]

=== What do I need to change in my package to make it reloadable (new in v1.4.2)? ===
See [http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#How_to_make_you_package_reloadable UsersGuideVisTrailsPackages] for an explanation.

=== Can I add default values or labels for parameters? ===

Yes. Versions 1.4 and greater support these features. See [[UsersGuideVisTrailsPackages#Adding_default_values_and.2For_labels_for_parameters | UsersGuideVisTrailsPacakges]] for more details.

=== I want to write a module to load HDF data whose output (e.g., data, string) varies according to the input I give it. Is is possible to do this in VisTrails, and if yes, how can I do that? Ideally, I would like to avoid having to change the connection of my output every time I change the input. ===

There are a few ways to tackle this - each has it's own benefits and pitfalls. Firstly, module connections do respect class hierarchies as we're familiar with in object oriented languages. For instance, A module can output a Constant of which String, Float, Integer, etc are specifications. In this way, you can have a subclass of something like HDFData be passed out of the module and the connections will be established regardless of the sub-type. This is a bit dangerous though. Modules downstream of such a class may not really know how to operate on certain types derived from the super-class. Extreme care must be taken both when creating the modules as well as connecting them to prevent things like this from happening.

A second method that I employ in several different packages is the idea of a container class. For instance, the NumSciPy package uses a relatively generic container "Numpy Array" to encapsulate the data. Of course, these encapsulating objects can store dictionaries that other modules can easily access and understand how to operate on. Although this method is slightly more work, the benefits of a stricter typing of ports is beneficial - particularly upon interfacing with other packages that may depend on strongly typed constants (for example).

== The Console ==

=== Where should I go to find out what I can call from the console and how to import it? ===

We have tried to make some methods more accessible in the console via an api. You can import the api via <code>import api</code> in the console and see the available methods with <code>dir(api)</code>. To open a vistrail:

import api
api.open_vistrail_from_file('/Applications/VisTrails/examples/terminator.vt')

To execute a version of a workflow, you currently have to go through the controller:

api.select_version('Histogram')
api.get_current_controller().execute_current_workflow()

Currently, only a subset of VisTrails functionality is directly available from the api. However, since VisTrails is written in python, you can dig down starting with the VistrailsApplication or controller object to expose most of our internal methods. If you have suggestions for calls to be added to the api, please let us know.

One other feature that we're working on, but is still in progress is the ability to construct workflows via the console. For example:

vtk = load_package('edu.utah.sci.vistrails.vtk')
vtk.vtkDataSetReader() # adds a vtkDataSetReader module to the pipeline
# click on the new module
a = selected_modules()[0] # get the one currently selected module
a.SetFile('/vistrails/examples/data/head120.vtk') # sets the SetFile parmaeter for the data set reader
b = vtk.vtkContourFilter() # adds a vtkContourFilter module to the pipeline and saves to var b
b.SetInputConnection0(a.GetOutputPort0()) # connects a's GetOutputPort0 port to b's SetInputConnection0

== Persistence Package ==

=== How do I use the output of one workflow as the input for another using the persistence package? ===

You need to configure the persistence modules using the module's configuration dialog. After adding a <code>PersistentOutputFile</code> to the workflow, click on the triangle in the upper-right corner of the <code>PersistentOutputFile</code>, and select "Edit Configuration" from the menu that appears. In this dialog, select "Create New Reference" and give the reference a name (and any space-delimited tags). Upon running that workflow, the data will be written to the persistent store. In the second workflow where you wish to use that file, add a <code>PersistentInputFile</code> and go to its configuration dialog in the same manner as with the output file. In that dialog, select "Use Existing Reference" and select the data that you just added in the first workflow in the list of files below. Now, when you run that workflow, it will grab the data from the persistent store.

Here is an example: [[Media:offscreen_persistent.vt | offscreen_persistent.vt]]. Run the "persistent offscreen" workflow first, and then run the "display persistent output" to use the output of the first workflow as the input for the second.

== VTK ==

=== Given a VTK visualization, how can I generate a webpage from it? ===

Check out the html pipeline in offscreen.xml.

=== I'm trying to use VTK, but there doesn't seem to be any output. What is wrong? ===

To use VTK on VisTrails, you need a slightly different way of connecting the renderer modules. Instead of using the standard RenderWindow/RenderWindowInteractor infrastructure, you simply connect the renderer to a VTKCell. The examples directory in the distribution has several VTK examples that illustrate.

=== I am trying to add a module to the workflow via Python, but how can I access vtk modules? ===

Here's an example:

import api

vtvtk = 'edu.utah.sci.vistrails.vtk'

module = api.add_module(0, 0, vtvtk, 'vtkContourFilter', '')

The third argument in add_module is the package identifier. You can find this in the "Module Packages" panel of the Preferences; just click on the package you're interested in and it will appear in the information on the right.

== matplotlib ==

=== I'm experiencing a problem with Latex labels and the matplotlib that comes with VisTrails 1.5. The script below entered to the interpreter that comes with VT is sufficient to reproduce it. ===

import matplotlib.pyplot as plt
plt.plot([1,2,3],[1,2,3])
plt.xlabel("$foo$")

Remove your ~/.matplotlib folder and re-start VisTrails

== VisTrails Development ==

=== I would like to build VisTrails from source. Are there instructions on how to do this? ===

Yes! Take a look at http://www.vistrails.org/index.php/Mac_Intel_Instructions
[http://www.vistrails.org/usersguide/getting_started.html#installing-vistrails-from-source Installing VisTrails from source]

== Accessing Provenance Information ==

=== How do I access the information in the execution log? ===

The code responsible for storing execution information is located in the "core/log" directories, and the code that generates much of that information is in "core/interpreter/cached.py". Modules can add execution-specific annotations to provenance via annotate() calls during execution, but much of the data (like timing and errors) is captured by the LogController and CachedInterpreter (the execution engine) objects. To analyze the log from a vistrail (.vt) file, you might have something like the following:

<code>
import core.log.log
import db.services.io

def run(fname):
# open the .vt bundle specified by the filename "fname"
bundle = db.services.io.open_vistrail_bundle_from_zip_xml(fname)[0]
# get the log filename
log_fname = bundle.vistrail.db_log_filename
if log_fname is not None:
# open the log
log = db.services.io.open_log_from_xml(log_fname, True)
# convert the log from a db object
core.log.log.Log.convert(log)
for workflow_exec in log.workflow_execs:
print 'workflow version:', workflow_exec.parent_version
print 'time started:', workflow_exec.ts_start
print 'time ended:', workflow_exec.ts_end
print 'modules executed:', [i.module_id
for i in workflow_exec.item_execs]
if __name__ == '__main__':
run("some_vistrail.vt")
</code>

You should be able to see what information is available by looking at the "core/log" classes.

FAQ

2011-03-25T01:25:41Z

Ctuttle: /* Is there a way to give each widget a "display name" in addition to the module name at the center of the widget? */

Also check our [[known_issues|Known Issues]] page for troubleshooting.

----
== Running workflows ==

=== How can I run a workflow using the command line? ===

('''Updated for version 1.2''') Call vistrails using the following options:

<code>python vistrails.py -b path_to_vistrails_file:pipeline </code>

where pipeline can be a version '''tag name''' or version '''id'''

'''NOTE:''' If you downloaded the MacOS X bundle, you can run vistrails from the command line via the following commands in the Terminal. Change the current directory to wherever VisTrails was installed (often <code>/Applications</code>), and then type:

<code>Vistrails.app/Contents/MacOS/vistrails [<cmd_line_options>]</code>

=== Using the command line, we'd like to execute a workflow multiple times, with slightly different parameters, and create a series of output files. Is this possible?===

('''Updated for version 1.2''') We can change parameters that have an alias through the command line.

For example, offscreen pipeline in offscreen.vt always creates the file called image.png. If you want generate it with a different filename:

<code>python vistrails.py -b ../examples/offscreen.vt:offscreen -a"filename=other.png"</code>

<code>filename</code> in the example above is the alias name assigned to the parameter in the <code>value</code> method inside the String module. When running a pipeline from the command line, VisTrails will try to start the spreadsheet automatically if the pipeline requires it. For example, this other execution will also start the spreadsheet (attention to how $ characters are escaped when running on bash):

<code>python vistrails.py -b ../examples/head.vt:aliases -a"isovalue=30\$&\$diffuse_color=0.8,0.4,0.2"</code>

You can also execute more than one pipeline on the command line:

<code>python vistrails.py -b ../examples/head.vt:aliases ../examples/spx.vt:spx \
-a"isovalue=30"</code>

Use the -a parameter only once regardless the number of pipelines.

=== I can load a vistrail, and the version tree shows up fine. However, no pipelines appear when I click on a version. What gives? ===

The most likely reason is that the vistrail uses a package that is not registered with VisTrails. You need to identify the needed package and add it to your <code>.vistrails/startup.py</code>. A single line like the following should be enough:

addPackage('enter_package_name_here')

Some packages might need more information. For example:

addPackage('afront', executable_path='/path/to/afront')

Refer to the package documentation for details. The one inconvenient step is that currently there's no automated way to describe what is the missing package. We're working on this feature for future releases.

=== I have a workflow that reads a file and then does some processing. The first time it runs, it executes correctly. But in subsequent, nothing happens. ===

VisTrails caches by default, so after a workflow is executed, if none of its parameters change, it won't be executed again.

If a workflow reads a file using the basic module File, VisTrails does check whether the file was modified since the last run. It does so by keeping a signature that is based on the modification time of the file. And if the file was modified, the File module and all downstream modules (the ones which depend on File) will be executed.

''Note: If you would like your input and output data to be versioned, you can use the Persistence package.''

If you do not want VisTrails to cache executions, you can turn off caching: go to Menu Edit -> Preferences and in the General Configuration tab, change Cache execution results to Never.

=== Can VisTrails execute workflows in parallel? ===

The VisTrails server can only execute pipelines in parallel if there's more than one instance of VisTrails running. The command

<code> self.rpcserver = ThreadedXMLRPCServer((self.temp_xml_rpc_options.server, self.temp_xml_rpc_options.port))</code>

starts a multithreaded version of the XML-RPC server, so it will create a thread for each request received by the server. The problem is that Qt/PyQT doesn't allow these multiple threads create GUI objects, only in the main thread. To overcome this limitation, the multithreaded version can instantiate other single threaded versions of VisTrails and put them in a queue, so workflow executions and other GUI-related requests, such as generating workflow graphs and history trees can be forwarded to this queue, and each instance takes turns in answering the request. If the results are in the cache, the multithreaded version answers the requests directly.

Note that this infrastructure works on Linux only. To make this work on Windows, you have to create a script similar to start_vistrails_xvfb.sh (located in the scripts folder) where you can send the number of other instances via command-line options to VisTrails. The command line options are:

python vistrails_server.py -T <ADDRESS> -R <PORT> -O<NUMBER_OF_OTHER_VISTRAILS_INSTANCES> [-M]&

If you want the main vistrails instance to be multithreaded, use the -M at the end.

After creating this script, update function start_other_instances in vistrails/gui/application_server.py lines 1007-1023 and set the script variable to point to your script. You may also have to change the arguments sent to your script (line 1016: for example, you don't need to set a virtual display). You will need to change the path to the stop_vistrails_server.py script (on line 1026) according to your installation path.

=== When a workflow is executed, what do the colors mean? ===

- lilac: module was notexecuted

- yellow: module is currently being executed

- green: module was successfully executed

- orange: module was cached

- red: the execution of the module failed

== Building workflows ==

=== Is there a way to give each widget a "display name" in addition to the module name at the center of the widget? ===

Yes, a "display name" can be assigned to a module by selecting the triangle in its top right corner to open a popup menu and selecting the Set Module Label... menu item. You will then be prompted to enter the "display name".

=== Is there a way to re-center the picture-in-picture (PiP) view? ===

Yes. If you click on the PIP window to bring it to focus, you can press Ctrl-R (or Command-R on Mac) to re-center the PiP window.

=== How do I search for a literal "?" (question mark) in the search box in the Property panel? ===

Since we allow regular expressions in our search box, question marks are treated as meta-characters. Thus, searching for "?" returns everything and "abc?" will return everything containing "abc". You need to use "\?" instead to search for "?". So the search for "??" would be "\?\?".

== Using VisTrails as a server ==

=== What is the VisTrails server-mode? ===

Using the VisTrails server mode, it is possible to ''execute workflows and control VisTrails through another application''. For example, the CrowdLabs Web portal (http://www.crowdlabs.org) accesses a VisTrails sever to execute workflows, retrieve and display vistrail trees and workflows. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

=== How do I execute workflows and control VisTrails through another application? ===

The way you access the server is by doing XML-RPC calls. In the current VisTrails release, we include a set of PHP scripts that can talk to a VisTrails server instance. They are in "extensions/http" folder. The files are reasonably well documented. Also, it should be not difficult to create python scripts to access the server (just use xmlrpclib module).

Note that the VisTrails server requires the provenance and workflows to be in a database. More detailed instructions on how to setup the server and the database
are available here:

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_server_setup.html

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_database_setup.html

If what you want is just to execute a series of workflows in batch mode, a simpler solution would be to ''use the VisTrails client in batch mode''. Chapter 12 of the user's guide contains detailed information and examples on that. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

== Control Flow ==

=== Note: using map ===

When using 'map', the module (or subworkflow) used as function port in the map module MUST be a function, i.e., it can only define 1 output port. [http://www.vistrails.org/usersguide/controlflow.html#the-map-operator The Map Operator]

== Spreadsheet ==

Where ''pipeline'' is a version number or a tag.

=== How can I save an image from the spreadsheet? ===

While having the focus on a spreadsheet cell and select the camera on the toolbar to take a snapshot. The system will prompt you for the location and file name where it should be saved. The other icons can be used for saving multiple images that can be used for generating an animation on demand. A whole sheet can also be saved by selecting Export (either from the menu or from the toolbar). [http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet-image Saving a Spreadsheet Image]

===Is it possible to save the complete state of the spreadsheet?===
[http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet Saving a Spreadsheet]

=== Can I view multiple sheets at the same time? ===

Yes. Each sheet on the spreadsheet can be displayed as a dock widget separated from the main spreadsheet window by dragging its tab name out of the tab bar at the bottom of the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Then, how can I put back a separated sheet? ===

A sheet can be docked back to the main window by dragging it back to the tab bar or double-click on its title bar. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== How can I order sheets on the spreadsheet? ===

This can be done by dragging the sheet name on the bottom top bar and drop it to the right place. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Can I control where a cell will be placed on the spreadsheet window? ===

By default, an unoccupied cell on the active sheet will be chosen to display the result. However, you can specify exactly in the pipeline where a spreadsheet cell will be placed by using CellLocation and SheetReference. CellLocation specifies the location (row and column) of a cell when connecting to a spreadsheet cell (VTKCell, ImageViewerCell, ...). Similarly, a SheetReference module (when connecting to a CellLocation) will specify which sheet the cell will be put on given its name, minimum row size and minimum column size. There is an example of this in examples/vtk.xml (select the version below Double Renderer). [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

=== How do I output results to the spreadsheet? ===

By inspecting the VisTrails Spreadsheet package (in the list of packages, to the left of the pipeline builder), you
can see there are built-in cells for different kinds of data, e.g., RichTextCell to display HTML and plain text. op
You (the user) can also define new cell types to display application-specific data. For example, we have developed
VtkCell, MplFigureCell, and OpenGLCell. It is possible to display pretty much anything on the Spreadsheet! [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

Examples of writing cell modules can be found in:
RichTextCell: packages/spreadsheet/widgets/richtext/richtext.py
VTK: packages/vtk/vtkcell.py

Here is the summary of some requirements on a cell widget:

(1) It must be a Qt widget. It should inherit from spreadsheet_cell.QCellWidget in the spreadsheet package. Although any Qt Widget would work, certain features such as animation will not be available (without rewriting it).

(2) It must re-implement the updateContents() function to take a set of inputs (usually coming from input ports of a wrapper Module) and display on the cells. VisTrails uses this function to update/reuse cells on the spreadsheet when new data comes in.

(3) It needs a wrapper VisTrails Module (inherited from basic_widgets.SpreadsheetCell of the spreadsheet package). Inside the compute() method of this module, it may call self.display(CellWidgetType, (inputs)) to trigger the display event on the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#advanced-cell-options Advanced Cell Options]

=== How do I control the default number of cells in the spreadsheet? ===

You can configure the rowCount and colCount using the preferences dialog. Just go to the Module Packages tab, select spreadsheet in the "Enabled packages" and press the Configure button. Then a list of all the configuration options for the spreadsheet will show up. [http://www.vistrails.org/usersguide/spreadsheet.html#custom-layout-options Custom Layout Options]

=== Is it possible to launch a web browser from the vistrails spreadsheet? We would like to output several urls from a parameter sweep and then have the option to click on each one to view the resulting page. I can view the page within the spreadsheet, but it is really too crowded.===

Currently, there isn't a widget that provides exactly this functionality, but I can think of a few solutions that may work for you:

(1) You can use parameter exploration to generate multiple sheets so you might have an exploration that opens each page in a new sheet. Use the third column/dimension in the exploration interface to have a parameter span sheets.

(2) The spreadsheet is extensible so you can write a custom spreadsheet cell widget that has a button or label with the desired link (a QLabel with openExternalLinks set to True, for example).

(3) You can tweak the existing RichTextCell be adding the line "self.browser.setOpenExternalLinks(True)" at line 63 of the source file "vistrails/packages/spreadsheet/widgets/richtext/richtext.py". Then, if your workflow creates a file with html markup text like "<a href="http://www.vistrails.org">VisTrails</a>" connected to a RichTextCell, clicking on the rendered link in the cell will open it in a web browser. You need to add the aforementioned line to the source to let Qt know that you want the link opened externally; by default, it will just issue an event that isn't processed. [http://www.vistrails.org/usersguide/spreadsheet.html#launching-a-web-browser Launching a Web Browser]

== Integrating your software into VisTrails ==

===How can I integrate my own program into VisTrails?===

The easiest way is to create a package. Writing a package is often very simple, here are instructions on how to do it: [[UsersGuideVisTrailsPackages]]

You can also dynamically generate modules. For an example see:

http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#Packages_that_generate_modules_dynamically

In particular, see the new_module call which uses python's type() function to generate new classes dynamically.

=== How do modules deal with multiple inputs in a same port? ===

(And should that even be allowed?)

For compatibility reasons, we do need to allow multiple connections to an input port. However, most package developers should never have to use this, and so we do our best to hide it. the default behavior for getting inputs from a port, then, is to always return a single input.

If on your module you need multiple inputs connected to a single port, use the 'forceGetInputListFromPort' method. It will return a list of all the data items coming through the port. The VTK package uses this feature, so look there for usage examples (packages/vtk/base_widget.py)

=== Are there mechanisms for attaching widgets to different modules/parameters? ===

Right now, we have a mechanism for putting a specific widget for an input port. For example, if a port is SetColor(red, green, blue), we can put a color wheel widget there. Or we can also replace the SetFileName port with a File Widget. However, this is not per parameter (only per port). We are currently working on this problem.

=== Can I organize my package so it appears hierarchical in the module palette? ===

Yes. Use the namespace keyword argument when adding the module to the registry. For example,

registry.add_module(MyModule, namespace='MyNamespace')

=== Can I nest namespaces? ===

Yes. Use the '|' character to separate different the hierarchy. For example,

registry.add_module(MyModule, namespace='ParentNamespace|ChildNamespace')

=== Are there shortucts for registry initialization? ===

Yes. If you define _modules as a list of classes in the __init__.py file of your package, VisTrails will attempt to load all classes specified as modules. You can provide add_module options as keyword arguments by specifying a tuple (class, kwargs) in the list. For example:

_modules = [MyModule1, (MyModule2, {'namespace': 'MyNamespace'})]

In addition, you need to identify the ports of your modules as a field in your class by defining _input_ports and _output_ports lists. Here, the items in each list must be tuples of the form (portName, portSignature, optional=False, sort_key=-1). For example:

class MyModule(Module):
def compute(self):
pass

_input_ports = [('firstInput', String), ('secondInput', Integer, True)]
_output_ports = [('firstOutput', String), ('secondOutput', String)]

=== Can I define ports to be of types that I do not import into my package? ===

Yes. You can pass an identifier string as the portSignature instead. The port_signature string is defined by:

<module_string> := <package_identifier>:[<namespace>|]<module_name>,
<port_signature> := (<module_string>*)

For example,

registry.add_input_port(MyModule, 'myInputPort', '(edu.utah.sci.vistrails.basic:String)')

or

_input_ports = [('myInputPort', '(edu.utah.sci.vistrails.basic:String)')]

=== What do I need to change in my package to make it reloadable (new in v1.4.2)? ===
See [http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#How_to_make_you_package_reloadable UsersGuideVisTrailsPackages] for an explanation.

=== Can I add default values or labels for parameters? ===

Yes. Versions 1.4 and greater support these features. See [[UsersGuideVisTrailsPackages#Adding_default_values_and.2For_labels_for_parameters | UsersGuideVisTrailsPacakges]] for more details.

=== I want to write a module to load HDF data whose output (e.g., data, string) varies according to the input I give it. Is is possible to do this in VisTrails, and if yes, how can I do that? Ideally, I would like to avoid having to change the connection of my output every time I change the input. ===

There are a few ways to tackle this - each has it's own benefits and pitfalls. Firstly, module connections do respect class hierarchies as we're familiar with in object oriented languages. For instance, A module can output a Constant of which String, Float, Integer, etc are specifications. In this way, you can have a subclass of something like HDFData be passed out of the module and the connections will be established regardless of the sub-type. This is a bit dangerous though. Modules downstream of such a class may not really know how to operate on certain types derived from the super-class. Extreme care must be taken both when creating the modules as well as connecting them to prevent things like this from happening.

A second method that I employ in several different packages is the idea of a container class. For instance, the NumSciPy package uses a relatively generic container "Numpy Array" to encapsulate the data. Of course, these encapsulating objects can store dictionaries that other modules can easily access and understand how to operate on. Although this method is slightly more work, the benefits of a stricter typing of ports is beneficial - particularly upon interfacing with other packages that may depend on strongly typed constants (for example).

== The Console ==

=== Where should I go to find out what I can call from the console and how to import it? ===

We have tried to make some methods more accessible in the console via an api. You can import the api via <code>import api</code> in the console and see the available methods with <code>dir(api)</code>. To open a vistrail:

import api
api.open_vistrail_from_file('/Applications/VisTrails/examples/terminator.vt')

To execute a version of a workflow, you currently have to go through the controller:

api.select_version('Histogram')
api.get_current_controller().execute_current_workflow()

Currently, only a subset of VisTrails functionality is directly available from the api. However, since VisTrails is written in python, you can dig down starting with the VistrailsApplication or controller object to expose most of our internal methods. If you have suggestions for calls to be added to the api, please let us know.

One other feature that we're working on, but is still in progress is the ability to construct workflows via the console. For example:

vtk = load_package('edu.utah.sci.vistrails.vtk')
vtk.vtkDataSetReader() # adds a vtkDataSetReader module to the pipeline
# click on the new module
a = selected_modules()[0] # get the one currently selected module
a.SetFile('/vistrails/examples/data/head120.vtk') # sets the SetFile parmaeter for the data set reader
b = vtk.vtkContourFilter() # adds a vtkContourFilter module to the pipeline and saves to var b
b.SetInputConnection0(a.GetOutputPort0()) # connects a's GetOutputPort0 port to b's SetInputConnection0

== Persistence Package ==

=== How do I use the output of one workflow as the input for another using the persistence package? ===

You need to configure the persistence modules using the module's configuration dialog. After adding a <code>PersistentOutputFile</code> to the workflow, click on the triangle in the upper-right corner of the <code>PersistentOutputFile</code>, and select "Edit Configuration" from the menu that appears. In this dialog, select "Create New Reference" and give the reference a name (and any space-delimited tags). Upon running that workflow, the data will be written to the persistent store. In the second workflow where you wish to use that file, add a <code>PersistentInputFile</code> and go to its configuration dialog in the same manner as with the output file. In that dialog, select "Use Existing Reference" and select the data that you just added in the first workflow in the list of files below. Now, when you run that workflow, it will grab the data from the persistent store.

Here is an example: [[Media:offscreen_persistent.vt | offscreen_persistent.vt]]. Run the "persistent offscreen" workflow first, and then run the "display persistent output" to use the output of the first workflow as the input for the second.

== VTK ==

=== Given a VTK visualization, how can I generate a webpage from it? ===

Check out the html pipeline in offscreen.xml.

=== I'm trying to use VTK, but there doesn't seem to be any output. What is wrong? ===

To use VTK on VisTrails, you need a slightly different way of connecting the renderer modules. Instead of using the standard RenderWindow/RenderWindowInteractor infrastructure, you simply connect the renderer to a VTKCell. The examples directory in the distribution has several VTK examples that illustrate.

=== I am trying to add a module to the workflow via Python, but how can I access vtk modules? ===

Here's an example:

import api

vtvtk = 'edu.utah.sci.vistrails.vtk'

module = api.add_module(0, 0, vtvtk, 'vtkContourFilter', '')

The third argument in add_module is the package identifier. You can find this in the "Module Packages" panel of the Preferences; just click on the package you're interested in and it will appear in the information on the right.

== matplotlib ==

=== I'm experiencing a problem with Latex labels and the matplotlib that comes with VisTrails 1.5. The script below entered to the interpreter that comes with VT is sufficient to reproduce it. ===

import matplotlib.pyplot as plt
plt.plot([1,2,3],[1,2,3])
plt.xlabel("$foo$")

Remove your ~/.matplotlib folder and re-start VisTrails

== VisTrails Development ==

=== I would like to build VisTrails from source. Are there instructions on how to do this? ===

Yes! Take a look at http://www.vistrails.org/index.php/Mac_Intel_Instructions
[http://www.vistrails.org/usersguide/getting_started.html#installing-vistrails-from-source Installing VisTrails from source]

== Accessing Provenance Information ==

=== How do I access the information in the execution log? ===

The code responsible for storing execution information is located in the "core/log" directories, and the code that generates much of that information is in "core/interpreter/cached.py". Modules can add execution-specific annotations to provenance via annotate() calls during execution, but much of the data (like timing and errors) is captured by the LogController and CachedInterpreter (the execution engine) objects. To analyze the log from a vistrail (.vt) file, you might have something like the following:

<code>
import core.log.log
import db.services.io

def run(fname):
# open the .vt bundle specified by the filename "fname"
bundle = db.services.io.open_vistrail_bundle_from_zip_xml(fname)[0]
# get the log filename
log_fname = bundle.vistrail.db_log_filename
if log_fname is not None:
# open the log
log = db.services.io.open_log_from_xml(log_fname, True)
# convert the log from a db object
core.log.log.Log.convert(log)
for workflow_exec in log.workflow_execs:
print 'workflow version:', workflow_exec.parent_version
print 'time started:', workflow_exec.ts_start
print 'time ended:', workflow_exec.ts_end
print 'modules executed:', [i.module_id
for i in workflow_exec.item_execs]
if __name__ == '__main__':
run("some_vistrail.vt")
</code>

You should be able to see what information is available by looking at the "core/log" classes.

FAQ

2011-03-25T01:01:25Z

Ctuttle: /* Using VisTrails as a server */

Also check our [[known_issues|Known Issues]] page for troubleshooting.

----
== Running workflows ==

=== How can I run a workflow using the command line? ===

('''Updated for version 1.2''') Call vistrails using the following options:

<code>python vistrails.py -b path_to_vistrails_file:pipeline </code>

where pipeline can be a version '''tag name''' or version '''id'''

'''NOTE:''' If you downloaded the MacOS X bundle, you can run vistrails from the command line via the following commands in the Terminal. Change the current directory to wherever VisTrails was installed (often <code>/Applications</code>), and then type:

<code>Vistrails.app/Contents/MacOS/vistrails [<cmd_line_options>]</code>

=== Using the command line, we'd like to execute a workflow multiple times, with slightly different parameters, and create a series of output files. Is this possible?===

('''Updated for version 1.2''') We can change parameters that have an alias through the command line.

For example, offscreen pipeline in offscreen.vt always creates the file called image.png. If you want generate it with a different filename:

<code>python vistrails.py -b ../examples/offscreen.vt:offscreen -a"filename=other.png"</code>

<code>filename</code> in the example above is the alias name assigned to the parameter in the <code>value</code> method inside the String module. When running a pipeline from the command line, VisTrails will try to start the spreadsheet automatically if the pipeline requires it. For example, this other execution will also start the spreadsheet (attention to how $ characters are escaped when running on bash):

<code>python vistrails.py -b ../examples/head.vt:aliases -a"isovalue=30\$&\$diffuse_color=0.8,0.4,0.2"</code>

You can also execute more than one pipeline on the command line:

<code>python vistrails.py -b ../examples/head.vt:aliases ../examples/spx.vt:spx \
-a"isovalue=30"</code>

Use the -a parameter only once regardless the number of pipelines.

=== I can load a vistrail, and the version tree shows up fine. However, no pipelines appear when I click on a version. What gives? ===

The most likely reason is that the vistrail uses a package that is not registered with VisTrails. You need to identify the needed package and add it to your <code>.vistrails/startup.py</code>. A single line like the following should be enough:

addPackage('enter_package_name_here')

Some packages might need more information. For example:

addPackage('afront', executable_path='/path/to/afront')

Refer to the package documentation for details. The one inconvenient step is that currently there's no automated way to describe what is the missing package. We're working on this feature for future releases.

=== I have a workflow that reads a file and then does some processing. The first time it runs, it executes correctly. But in subsequent, nothing happens. ===

VisTrails caches by default, so after a workflow is executed, if none of its parameters change, it won't be executed again.

If a workflow reads a file using the basic module File, VisTrails does check whether the file was modified since the last run. It does so by keeping a signature that is based on the modification time of the file. And if the file was modified, the File module and all downstream modules (the ones which depend on File) will be executed.

''Note: If you would like your input and output data to be versioned, you can use the Persistence package.''

If you do not want VisTrails to cache executions, you can turn off caching: go to Menu Edit -> Preferences and in the General Configuration tab, change Cache execution results to Never.

=== Can VisTrails execute workflows in parallel? ===

The VisTrails server can only execute pipelines in parallel if there's more than one instance of VisTrails running. The command

<code> self.rpcserver = ThreadedXMLRPCServer((self.temp_xml_rpc_options.server, self.temp_xml_rpc_options.port))</code>

starts a multithreaded version of the XML-RPC server, so it will create a thread for each request received by the server. The problem is that Qt/PyQT doesn't allow these multiple threads create GUI objects, only in the main thread. To overcome this limitation, the multithreaded version can instantiate other single threaded versions of VisTrails and put them in a queue, so workflow executions and other GUI-related requests, such as generating workflow graphs and history trees can be forwarded to this queue, and each instance takes turns in answering the request. If the results are in the cache, the multithreaded version answers the requests directly.

Note that this infrastructure works on Linux only. To make this work on Windows, you have to create a script similar to start_vistrails_xvfb.sh (located in the scripts folder) where you can send the number of other instances via command-line options to VisTrails. The command line options are:

python vistrails_server.py -T <ADDRESS> -R <PORT> -O<NUMBER_OF_OTHER_VISTRAILS_INSTANCES> [-M]&

If you want the main vistrails instance to be multithreaded, use the -M at the end.

After creating this script, update function start_other_instances in vistrails/gui/application_server.py lines 1007-1023 and set the script variable to point to your script. You may also have to change the arguments sent to your script (line 1016: for example, you don't need to set a virtual display). You will need to change the path to the stop_vistrails_server.py script (on line 1026) according to your installation path.

=== When a workflow is executed, what do the colors mean? ===

- lilac: module was notexecuted

- yellow: module is currently being executed

- green: module was successfully executed

- orange: module was cached

- red: the execution of the module failed

== Building workflows ==

=== Is there a way to give each widget a "display name" in addition to the module name at the center of the widget? ===

Yes, but it is not easily accessible from the GUI and it definitely needs to be more intuitive. For now, we use the annotation value of key "__desc__" as a module label. If you want to set a PythonSource label, you have to select the module. Then click on the Annotation tab, and add a key named "__desc__", whatever value you set to this key will be the label. We are currently working on a new interface for this functionality.

=== Is there a way to re-center the picture-in-picture (PiP) view? ===

Yes. If you click on the PIP window to bring it to focus, you can press Ctrl-R (or Command-R on Mac) to re-center the PiP window.

=== How do I search for a literal "?" (question mark) in the search box in the Property panel? ===

Since we allow regular expressions in our search box, question marks are treated as meta-characters. Thus, searching for "?" returns everything and "abc?" will return everything containing "abc". You need to use "\?" instead to search for "?". So the search for "??" would be "\?\?".

== Using VisTrails as a server ==

=== What is the VisTrails server-mode? ===

Using the VisTrails server mode, it is possible to ''execute workflows and control VisTrails through another application''. For example, the CrowdLabs Web portal (http://www.crowdlabs.org) accesses a VisTrails sever to execute workflows, retrieve and display vistrail trees and workflows. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

=== How do I execute workflows and control VisTrails through another application? ===

The way you access the server is by doing XML-RPC calls. In the current VisTrails release, we include a set of PHP scripts that can talk to a VisTrails server instance. They are in "extensions/http" folder. The files are reasonably well documented. Also, it should be not difficult to create python scripts to access the server (just use xmlrpclib module).

Note that the VisTrails server requires the provenance and workflows to be in a database. More detailed instructions on how to setup the server and the database
are available here:

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_server_setup.html

http://www.crowdlabs.org/site_media/static/dev_docs/vistrails_database_setup.html

If what you want is just to execute a series of workflows in batch mode, a simpler solution would be to ''use the VisTrails client in batch mode''. Chapter 12 of the user's guide contains detailed information and examples on that. [http://www.vistrails.org/usersguide/batch.html#using-vistrails-as-a-server Using VisTrails as a Server]

== Control Flow ==

=== Note: using map ===

When using 'map', the module (or subworkflow) used as function port in the map module MUST be a function, i.e., it can only define 1 output port. [http://www.vistrails.org/usersguide/controlflow.html#the-map-operator The Map Operator]

== Spreadsheet ==

Where ''pipeline'' is a version number or a tag.

=== How can I save an image from the spreadsheet? ===

While having the focus on a spreadsheet cell and select the camera on the toolbar to take a snapshot. The system will prompt you for the location and file name where it should be saved. The other icons can be used for saving multiple images that can be used for generating an animation on demand. A whole sheet can also be saved by selecting Export (either from the menu or from the toolbar). [http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet-image Saving a Spreadsheet Image]

===Is it possible to save the complete state of the spreadsheet?===
[http://www.vistrails.org/usersguide/spreadsheet.html#saving-a-spreadsheet Saving a Spreadsheet]

=== Can I view multiple sheets at the same time? ===

Yes. Each sheet on the spreadsheet can be displayed as a dock widget separated from the main spreadsheet window by dragging its tab name out of the tab bar at the bottom of the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Then, how can I put back a separated sheet? ===

A sheet can be docked back to the main window by dragging it back to the tab bar or double-click on its title bar. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== How can I order sheets on the spreadsheet? ===

This can be done by dragging the sheet name on the bottom top bar and drop it to the right place. [http://www.vistrails.org/usersguide/spreadsheet.html#multiple-spreadsheets Multiple Spreadsheets]

=== Can I control where a cell will be placed on the spreadsheet window? ===

By default, an unoccupied cell on the active sheet will be chosen to display the result. However, you can specify exactly in the pipeline where a spreadsheet cell will be placed by using CellLocation and SheetReference. CellLocation specifies the location (row and column) of a cell when connecting to a spreadsheet cell (VTKCell, ImageViewerCell, ...). Similarly, a SheetReference module (when connecting to a CellLocation) will specify which sheet the cell will be put on given its name, minimum row size and minimum column size. There is an example of this in examples/vtk.xml (select the version below Double Renderer). [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

=== How do I output results to the spreadsheet? ===

By inspecting the VisTrails Spreadsheet package (in the list of packages, to the left of the pipeline builder), you
can see there are built-in cells for different kinds of data, e.g., RichTextCell to display HTML and plain text. op
You (the user) can also define new cell types to display application-specific data. For example, we have developed
VtkCell, MplFigureCell, and OpenGLCell. It is possible to display pretty much anything on the Spreadsheet! [http://www.vistrails.org/usersguide/spreadsheet.html#sending-output-to-the-spreadsheet Sending Output to the Spreadsheet]

Examples of writing cell modules can be found in:
RichTextCell: packages/spreadsheet/widgets/richtext/richtext.py
VTK: packages/vtk/vtkcell.py

Here is the summary of some requirements on a cell widget:

(1) It must be a Qt widget. It should inherit from spreadsheet_cell.QCellWidget in the spreadsheet package. Although any Qt Widget would work, certain features such as animation will not be available (without rewriting it).

(2) It must re-implement the updateContents() function to take a set of inputs (usually coming from input ports of a wrapper Module) and display on the cells. VisTrails uses this function to update/reuse cells on the spreadsheet when new data comes in.

(3) It needs a wrapper VisTrails Module (inherited from basic_widgets.SpreadsheetCell of the spreadsheet package). Inside the compute() method of this module, it may call self.display(CellWidgetType, (inputs)) to trigger the display event on the spreadsheet. [http://www.vistrails.org/usersguide/spreadsheet.html#advanced-cell-options Advanced Cell Options]

=== How do I control the default number of cells in the spreadsheet? ===

You can configure the rowCount and colCount using the preferences dialog. Just go to the Module Packages tab, select spreadsheet in the "Enabled packages" and press the Configure button. Then a list of all the configuration options for the spreadsheet will show up. [http://www.vistrails.org/usersguide/spreadsheet.html#custom-layout-options Custom Layout Options]

=== Is it possible to launch a web browser from the vistrails spreadsheet? We would like to output several urls from a parameter sweep and then have the option to click on each one to view the resulting page. I can view the page within the spreadsheet, but it is really too crowded.===

Currently, there isn't a widget that provides exactly this functionality, but I can think of a few solutions that may work for you:

(1) You can use parameter exploration to generate multiple sheets so you might have an exploration that opens each page in a new sheet. Use the third column/dimension in the exploration interface to have a parameter span sheets.

(2) The spreadsheet is extensible so you can write a custom spreadsheet cell widget that has a button or label with the desired link (a QLabel with openExternalLinks set to True, for example).

(3) You can tweak the existing RichTextCell be adding the line "self.browser.setOpenExternalLinks(True)" at line 63 of the source file "vistrails/packages/spreadsheet/widgets/richtext/richtext.py". Then, if your workflow creates a file with html markup text like "<a href="http://www.vistrails.org">VisTrails</a>" connected to a RichTextCell, clicking on the rendered link in the cell will open it in a web browser. You need to add the aforementioned line to the source to let Qt know that you want the link opened externally; by default, it will just issue an event that isn't processed. [http://www.vistrails.org/usersguide/spreadsheet.html#launching-a-web-browser Launching a Web Browser]

== Integrating your software into VisTrails ==

===How can I integrate my own program into VisTrails?===

The easiest way is to create a package. Writing a package is often very simple, here are instructions on how to do it: [[UsersGuideVisTrailsPackages]]

You can also dynamically generate modules. For an example see:

http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#Packages_that_generate_modules_dynamically

In particular, see the new_module call which uses python's type() function to generate new classes dynamically.

=== How do modules deal with multiple inputs in a same port? ===

(And should that even be allowed?)

For compatibility reasons, we do need to allow multiple connections to an input port. However, most package developers should never have to use this, and so we do our best to hide it. the default behavior for getting inputs from a port, then, is to always return a single input.

If on your module you need multiple inputs connected to a single port, use the 'forceGetInputListFromPort' method. It will return a list of all the data items coming through the port. The VTK package uses this feature, so look there for usage examples (packages/vtk/base_widget.py)

=== Are there mechanisms for attaching widgets to different modules/parameters? ===

Right now, we have a mechanism for putting a specific widget for an input port. For example, if a port is SetColor(red, green, blue), we can put a color wheel widget there. Or we can also replace the SetFileName port with a File Widget. However, this is not per parameter (only per port). We are currently working on this problem.

=== Can I organize my package so it appears hierarchical in the module palette? ===

Yes. Use the namespace keyword argument when adding the module to the registry. For example,

registry.add_module(MyModule, namespace='MyNamespace')

=== Can I nest namespaces? ===

Yes. Use the '|' character to separate different the hierarchy. For example,

registry.add_module(MyModule, namespace='ParentNamespace|ChildNamespace')

=== Are there shortucts for registry initialization? ===

Yes. If you define _modules as a list of classes in the __init__.py file of your package, VisTrails will attempt to load all classes specified as modules. You can provide add_module options as keyword arguments by specifying a tuple (class, kwargs) in the list. For example:

_modules = [MyModule1, (MyModule2, {'namespace': 'MyNamespace'})]

In addition, you need to identify the ports of your modules as a field in your class by defining _input_ports and _output_ports lists. Here, the items in each list must be tuples of the form (portName, portSignature, optional=False, sort_key=-1). For example:

class MyModule(Module):
def compute(self):
pass

_input_ports = [('firstInput', String), ('secondInput', Integer, True)]
_output_ports = [('firstOutput', String), ('secondOutput', String)]

=== Can I define ports to be of types that I do not import into my package? ===

Yes. You can pass an identifier string as the portSignature instead. The port_signature string is defined by:

<module_string> := <package_identifier>:[<namespace>|]<module_name>,
<port_signature> := (<module_string>*)

For example,

registry.add_input_port(MyModule, 'myInputPort', '(edu.utah.sci.vistrails.basic:String)')

or

_input_ports = [('myInputPort', '(edu.utah.sci.vistrails.basic:String)')]

=== What do I need to change in my package to make it reloadable (new in v1.4.2)? ===
See [http://www.vistrails.org/index.php/UsersGuideVisTrailsPackages#How_to_make_you_package_reloadable UsersGuideVisTrailsPackages] for an explanation.

=== Can I add default values or labels for parameters? ===

Yes. Versions 1.4 and greater support these features. See [[UsersGuideVisTrailsPackages#Adding_default_values_and.2For_labels_for_parameters | UsersGuideVisTrailsPacakges]] for more details.

=== I want to write a module to load HDF data whose output (e.g., data, string) varies according to the input I give it. Is is possible to do this in VisTrails, and if yes, how can I do that? Ideally, I would like to avoid having to change the connection of my output every time I change the input. ===

There are a few ways to tackle this - each has it's own benefits and pitfalls. Firstly, module connections do respect class hierarchies as we're familiar with in object oriented languages. For instance, A module can output a Constant of which String, Float, Integer, etc are specifications. In this way, you can have a subclass of something like HDFData be passed out of the module and the connections will be established regardless of the sub-type. This is a bit dangerous though. Modules downstream of such a class may not really know how to operate on certain types derived from the super-class. Extreme care must be taken both when creating the modules as well as connecting them to prevent things like this from happening.

A second method that I employ in several different packages is the idea of a container class. For instance, the NumSciPy package uses a relatively generic container "Numpy Array" to encapsulate the data. Of course, these encapsulating objects can store dictionaries that other modules can easily access and understand how to operate on. Although this method is slightly more work, the benefits of a stricter typing of ports is beneficial - particularly upon interfacing with other packages that may depend on strongly typed constants (for example).

== The Console ==

=== Where should I go to find out what I can call from the console and how to import it? ===

We have tried to make some methods more accessible in the console via an api. You can import the api via <code>import api</code> in the console and see the available methods with <code>dir(api)</code>. To open a vistrail:

import api
api.open_vistrail_from_file('/Applications/VisTrails/examples/terminator.vt')

To execute a version of a workflow, you currently have to go through the controller:

api.select_version('Histogram')
api.get_current_controller().execute_current_workflow()

Currently, only a subset of VisTrails functionality is directly available from the api. However, since VisTrails is written in python, you can dig down starting with the VistrailsApplication or controller object to expose most of our internal methods. If you have suggestions for calls to be added to the api, please let us know.

One other feature that we're working on, but is still in progress is the ability to construct workflows via the console. For example:

vtk = load_package('edu.utah.sci.vistrails.vtk')
vtk.vtkDataSetReader() # adds a vtkDataSetReader module to the pipeline
# click on the new module
a = selected_modules()[0] # get the one currently selected module
a.SetFile('/vistrails/examples/data/head120.vtk') # sets the SetFile parmaeter for the data set reader
b = vtk.vtkContourFilter() # adds a vtkContourFilter module to the pipeline and saves to var b
b.SetInputConnection0(a.GetOutputPort0()) # connects a's GetOutputPort0 port to b's SetInputConnection0

== Persistence Package ==

=== How do I use the output of one workflow as the input for another using the persistence package? ===

You need to configure the persistence modules using the module's configuration dialog. After adding a <code>PersistentOutputFile</code> to the workflow, click on the triangle in the upper-right corner of the <code>PersistentOutputFile</code>, and select "Edit Configuration" from the menu that appears. In this dialog, select "Create New Reference" and give the reference a name (and any space-delimited tags). Upon running that workflow, the data will be written to the persistent store. In the second workflow where you wish to use that file, add a <code>PersistentInputFile</code> and go to its configuration dialog in the same manner as with the output file. In that dialog, select "Use Existing Reference" and select the data that you just added in the first workflow in the list of files below. Now, when you run that workflow, it will grab the data from the persistent store.

Here is an example: [[Media:offscreen_persistent.vt | offscreen_persistent.vt]]. Run the "persistent offscreen" workflow first, and then run the "display persistent output" to use the output of the first workflow as the input for the second.

== VTK ==

=== Given a VTK visualization, how can I generate a webpage from it? ===

Check out the html pipeline in offscreen.xml.

=== I'm trying to use VTK, but there doesn't seem to be any output. What is wrong? ===

To use VTK on VisTrails, you need a slightly different way of connecting the renderer modules. Instead of using the standard RenderWindow/RenderWindowInteractor infrastructure, you simply connect the renderer to a VTKCell. The examples directory in the distribution has several VTK examples that illustrate.

=== I am trying to add a module to the workflow via Python, but how can I access vtk modules? ===

Here's an example:

import api

vtvtk = 'edu.utah.sci.vistrails.vtk'

module = api.add_module(0, 0, vtvtk, 'vtkContourFilter', '')

The third argument in add_module is the package identifier. You can find this in the "Module Packages" panel of the Preferences; just click on the package you're interested in and it will appear in the information on the right.

== matplotlib ==

=== I'm experiencing a problem with Latex labels and the matplotlib that comes with VisTrails 1.5. The script below entered to the interpreter that comes with VT is sufficient to reproduce it. ===

import matplotlib.pyplot as plt
plt.plot([1,2,3],[1,2,3])
plt.xlabel("$foo$")

Remove your ~/.matplotlib folder and re-start VisTrails

== VisTrails Development ==

=== I would like to build VisTrails from source. Are there instructions on how to do this? ===

Yes! Take a look at http://www.vistrails.org/index.php/Mac_Intel_Instructions
[http://www.vistrails.org/usersguide/getting_started.html#installing-vistrails-from-source Installing VisTrails from source]

== Accessing Provenance Information ==

=== How do I access the information in the execution log? ===

The code responsible for storing execution information is located in the "core/log" directories, and the code that generates much of that information is in "core/interpreter/cached.py". Modules can add execution-specific annotations to provenance via annotate() calls during execution, but much of the data (like timing and errors) is captured by the LogController and CachedInterpreter (the execution engine) objects. To analyze the log from a vistrail (.vt) file, you might have something like the following:

<code>
import core.log.log
import db.services.io

def run(fname):
# open the .vt bundle specified by the filename "fname"
bundle = db.services.io.open_vistrail_bundle_from_zip_xml(fname)[0]
# get the log filename
log_fname = bundle.vistrail.db_log_filename
if log_fname is not None:
# open the log
log = db.services.io.open_log_from_xml(log_fname, True)
# convert the log from a db object
core.log.log.Log.convert(log)
for workflow_exec in log.workflow_execs:
print 'workflow version:', workflow_exec.parent_version
print 'time started:', workflow_exec.ts_start
print 'time ended:', workflow_exec.ts_end
print 'modules executed:', [i.module_id
for i in workflow_exec.item_execs]
if __name__ == '__main__':
run("some_vistrail.vt")
</code>

You should be able to see what information is available by looking at the "core/log" classes.