Difference between revisions of "Main Page"

From VistrailsWiki
Jump to navigation Jump to search
 
(176 intermediate revisions by 15 users not shown)
Line 1: Line 1:
Hello, and welcome to the VisTrails Wiki.
This is the official website for VisTrails, an open-source scientific workflow and provenance management system that supports data exploration and visualization.


== What is it? ==
VisTrails provides a general infrastructure that can be combined with existing systems and libraries. Official releases come with several packages, including VTK, matplotlib, and ImageMagick. The system is written in Python/Qt and runs on Mac, Unix and Windows.


Recently, workflows have been emerging as a paradigm for representing and managing complex
[[#Downloading VisTrails|Read more >]]
computations. Workflows can capture complex analyses processes at various levels of detail and
provide the provenance information necessary for reproducibility, result publication and result
sharing among collaborators. Because of the formalism they provide and the automation they
support, workflows have the potential to accelerate and transform the information analysis
process. Workflows are rapidly replacing primitive shell scripts as evidenced by the release of
Apple’s Mac OS X Automator, Microsoft Windows Workflow Foundation, and the SGI
Scientific Workflow Solution.
VisTrails is a new scientific workflow management system developed at the University of Utah that
provides support for data exploration and visualization. Whereas workflows have been
traditionally used to automate repetitive tasks, for applications that are exploratory in nature, very
little is repeated---change is the norm. As an engineer or scientist generates and evaluates
hypotheses about data under study, a series of different, albeit related, workflows are created
while a workflow is adjusted in an interactive process. VisTrails was designed to manage these
rapidly-evolving workflows. VisTrails streamlines the creation, execution and sharing of complex
visualizations, data mining or other large-scale data analysis applications. By automatically
managing the data, metadata, and the data exploration process, VisTrails allows users to focus on
the task at hand and relieves them from tedious and time-consuming tasks involved in organizing
the vast volumes of data they manipulate. VisTrails provides infrastructure that can be combined
with and enhance existing visualization and workflow systems.
Although VisTrails was originally built to address the needs of exploratory scientific applications,
the infrastructure it provides is very general. This became clear as the system was demoed to
people from different domains, both from industry and academia. VisTrails has the potential to
reduce the time to insight in virtually any exploratory task.


== Downloading ==
'''Please note that as of 2018, we are no longer maintaining VisTrails.'''


Get the open source version of VisTrails [[downloads|here]].
== Downloading VisTrails ==
Current and past releases are available on our [[downloads|download page]].


== Publications and Presentations ==
== News ==
* Unfortunately we can no longer maintain VisTrails
* [2016-05-03] VisTrails 2.2.4 has been released and is available on the [[Downloads]] page.
* [2015-09-11] SAHM VisTrails training course at USGS: http://pubs.usgs.gov/fs/2014/3007/pdf/fs2014-3007.pdf
* [2015-07-10] VisTrails 2.2.2 has been released and is available on the [[Downloads]] page.
* [2015-04-22] VisTrails 2.2 has been released and is available on the [[Downloads]] page.
* [2015-03-21] VisTrails 2.1.5 has been released and is available on the [[Downloads]] page.
* [2014-08-25] VisTrails 2.1.4 has been released and is available on the [[Downloads]] page.
* VisTrails 2.0.3 has been released and is available on the [[Downloads]] page.
* To learn more about how to explore climate data using DV3D and ParaView in UV-CDAT, check out http://www.kitware.com/source/home/post/105
* Comparison of two forecasts of Hurricane Sandy using the DV3D slicer in UV-CDAT http://www.youtube.com/watch?v=BFN0RzN1hSE&feature=em-share_video_user
* Presentation on the use of VisTrails at NASA: VisTrails: Workflow Management System on NEX - https://c3.nasa.gov/nex/static/media/other/Vistrails-Votava.pdf
* VisTrails 2.0 has been released and it is available for download on our [[Downloads]] page.  VisTrails is now distributed under the BSD license and its GUI was completely re-designed to make it easier to manage and interact with multiple workflows. It also includes a much improved [https://vistrails.org/usersguide Users' Manual]. See the [http://sourceforge.net/projects/vistrails/files/vistrails/v2.0/README/view release notes] for a complete list of changes.  Some notable features include:
** the ability to create mashups;
** support for the creation of reproducible papers;
** a provenance viewer;
** support for visual diffs as well as queries over workflows in different vistrails;
** a Control Flow Assistant to simplify the creation of workflows with loops and conditional constructs;
** support for global variables, so that modules deriving from Constant can be used to create variables that are shared among multiple workflows.


[http://www.vistrails.org/download/download.php?type=PUB&id=ipaw2006.pdf Managing Rapidly-Evolving Scientific Workflows]  (by Juliana Freire, Claudio T. Silva, Steven P. Callahan, Emanuele Santos, Carlos E. Scheidegger and Huy T. Vo)  Invited paper, in the proceedings of the International Provenance and Annotation Workshop (IPAW), 2006. [http://www.vistrails.org/download/download.php?type=PUB&id=ipaw2006-slides.pdf presentation]
[[Image:Screen Shot 2012-01-12 at 2.50.19 PM.png|200px|right]]
* UV-CDAT 1.0.0 has been released. More detailed information and binaries are available at http://uvcdat.llnl.gov
** UV-CDAT was demoed at [http://ipaw2012.bren.ucsb.edu IPAW 2012]
** Video tutorials are available at http://uv-cdat.llnl.gov/wiki/VideoTutorials
** Binaries (Mac/RedHAT6/Ubuntu) or build-from-sources options can be found here: http://uv-cdat.llnl.gov/install/
** Please report any issues you encounter via bugzilla at: http://uv-cdat.llnl.gov/bugzilla/


[http://www.sci.utah.edu/publications/SCITechReports/UUSCI-2006-017.pdf Visualization in Radiation Oncology: Towards Replacing the Laboratory Notebook] (by Erik W. Anderson, Steven P. Callahan, George T. Y. Chen, Juliana Freire, Emanuele Santos, Carlos E. Scheidegger, Claudio T. Silva and Huy T. Vo) SCI Institute Technical Report, No. UUSCI-2006-17, University of Utah, 2006.


[http://www.sci.utah.edu/publications/SCITechReports/UUSCI-2006-016.pdf Using Provenance to Streamline Data Exploration through Visualization] (by Steven P. Callahan, Juliana Freire, Emanuele Santos, Carlos E. Scheidegger, Claudio T. Silva and Huy T. Vo) SCI Institute Technical Report, No. UUSCI-2006-016, University of Utah, 2006.
* [[Older News]]


[http://www.vistrails.org/download/download.php?type=PUB&id=sciflow2006.pdf Managing the Evolution of Dataflows with VisTrails] (by Steven P. Callahan, Juliana Freire, Emanuele Santos, Carlos E. Scheidegger, Claudio T. Silva and Huy T. Vo) IEEE Workshop on Workflow and Data Flow for Scientific Applications (SciFlow) 2006. [http://www.vistrails.org/download/download.php?type=PUB&id=sciflow2006-slides.pdf presentation]
== VisTrails Overview ==
[[Image:Cosmology_example.png|480px|right]]
VisTrails is an open-source scientific workflow and provenance management system that provides support for simulations, data exploration and visualization. Whereas workflows have been  traditionally used to automate repetitive tasks, for applications that are exploratory in nature, such as simulations, data analysis and visualization, very little is repeated---change is the norm. As an engineer or scientist generates and evaluates  hypotheses about data under study, a series of different, albeit related, workflows are created while a workflow is adjusted in an interactive process. VisTrails was designed to manage these rapidly-evolving workflows.  


[http://www.vistrails.org/download/download.php?type=PUB&id=sigmod2006.pdf VisTrails: Visualization meets Data Management] (by Steven P. Callahan, Juliana Freire, Emanuele Santos, Carlos E. Scheidegger, Claudio T. Silva and Huy T. Vo) In Proceedings of ACM SIGMOD 2006. [http://www.vistrails.org/download/download.php?type=PUB&id=sigmod2006-slides.pdf presentation]
A key distinguishing feature of VisTrails is a comprehensive provenance infrastructure that maintains detailed history information about the steps followed and data derived in the course of an exploratory task: VisTrails maintains provenance of data products, of the workflows that derive these products and their executions. This information is persisted as XML files or in a relational database, and it allows users to navigate workflow versions in an intuitive way, to undo changes but not lose any results, to visually compare different workflows and their results, and to examine the actions that led to a result. It also enables a series operations and user interfaces that simplify workflow design and use, including the ability to create and refine workflows by analogy and to query workflows by example.


[http://www.vistrails.org/download/download.php?type=PUB&id=vistrails-tutorial.pdf VisTrails: A Short Tutorial] (by Steven P. Callahan, Juliana Freire, Emanuele Santos, Carlos E. Scheidegger, Claudio T. Silva and Huy T. Vo) Technical Report. University of Utah, 2005.
VisTrails supports the creation and execution of workflows. It allows the combination of loosely-coupled resources, specialized libraries, grid and Web services. The released version comes with support for several packages including, [http://www.vtk.org/ VTK], [http://www.imagemagick.org/script/index.php Image Magick], Web Services, and [http://www.scipy.org/PyLab pylab]. You can also download [[UserContributedPackages|packages contributed by our users]], as well as [http://vistrails.org/usersguide/v2.1/html/packages.html create your own packages/modules]. There are also a number of [[Projects using VisTrails|projects extending or building on top of VisTrails]].
Workflows can be run interactively, through the VisTrails GUI, or in batch using a VisTrails server.


[http://www.vistrails.org/download/download.php?type=PUB&id=vis2005.pdf VisTrails: Enabling Interactive Multiple-View Visualizations] (by Louis Bavoil, Steven P. Callahan, Patricia J. Crossno, Juliana Freire, Carlos E. Scheidegger, Claudio T. Silva and Huy T. Vo) In Proceedings of IEEE Visualization, 2005. [http://www.vistrails.org/download/download.php?type=PUB&id=vis2005-slides.pdf presentation]
VisTrails 1.0 was released in October 2007. Since then, the system has been downloaded thousands of times. VisTrails is written in Python and it uses the multi-platform Qt library for its user interface. It runs on Mac, Linux and Windows. You can download VisTrails from https://vistrails.org/index.php/Downloads


[http://www.vistrails.org/download/download.php?type=PUB&id=poster.pdf A poster on VisTrails applications]
=== Publishing Reproducible Results ===
VisTrails 2.0 allows the inclusion of reproducible results in LaTeX/PDF documents. We provide a LaTeX package that allows users to add links to their results in the LaTeX source. For example:


== Documentation ==
<code>
\usepackage{vistrails}


=== [[Users_Guide|User's Guide]] ===
...
On the link above is an (always evolving) User's Guide.


=== Video Tutorial  ===
\begin{figure}


We are in the process of creating a video tutorial that can be used to guide users through the process of learning to use VisTrails. The videos that follow below are a preliminary version of this tutorial, and will keep evolving towards a complete tutorial.
\begin{center}
{|{{Prettytable}}
 
|-
\subfigure[a=0.9]{\vistrail[filename=alps.vt, version=2, pdf]{width=8cm}}
|{{qt|link=http://www.vistrails.org/download/download.php?type=MEDIA&id=Tutorial_001_AddModule-H.264_LAN_960x540.mov|text=Adding a Module}}
{{qt|link=http://www.vistrails.org/download/download.php?type=MEDIA&id=Tutorial_002_AddParameter-H.264_LAN_960x540.mov|text=Adding a Parameter}}
{{qt|link=http://www.vistrails.org/download/download.php?type=MEDIA&id=Tutorial_003_ConnectingModule-H.264_LAN_960x540.mov|text=Connecting Two Modules}}
{{qt|link=http://www.vistrails.org/download/download.php?type=MEDIA&id=Tutorial_004_Add_ParameterModule-H.264_LAN_960x540.mov|text=Adding a Parameter via Modules}}
|'''The Pipeline Builder'''
These videos demonstrate the basic usage of the Pipeline Builder, including how to add modules, connect modules, and modify module parameters.


|-
\subfigure[a=0.9]{\vistrail[filename=alps.vt, version=11, pdf,buildalways]{width=8cm}}
|{{qt|link=http://www.vistrails.org/download/download.php?type=MEDIA&id=Tutorial_005_ParameterExploration-H.264_LAN_960x540.mov|text=Parameter Space Exploration}}
|'''Parameter Space Exploration'''
These videos demonstrate more advanced use of the Pipeline Builder window.  Parameter exploration allows easy mutli-dimensional comparative visualization generation to quickly and thoroughly explore data.


|-
\caption{A figure produced by an ALPS VisTrails workflow. Clicking the figure retrieves the workflow used to
|{{qt|link=http://www.vistrails.org/download/download.php?type=MEDIA&id=Tutorial_006_VersionTreeInteraction-H.264_LAN_960x540.mov|text=Version Tree Interaction}}
create it. Opening that workflow on a machine with VisTrails and ALPS installed lets the reader execute the full calculation.}
{{qt|link=http://www.vistrails.org/download/download.php?type=MEDIA&id=Tutorial_007_VisualDiff-H.264_LAN_960x540.mov|text=The Visual Diff}}
{{qt|link=http://www.vistrails.org/download/download.php?type=MEDIA&id=Tutorial_008_QueryByExample-H.264_LAN_960x540.mov|text=Querying the Version Tree and Pipelines}}
|'''The Version Tree'''
These videos show interaction with the Version Tree. Emphasis here is on interaction with the widgets, searching and refining the versions displayed, and performing diffs between versions to highlight changes.
|}


=== Videos  ===
\end{center}


Please note that some of these videos were generated using versions of VisTrails, and the interface might look different than the one available in the most up-to-date version.
\end{figure}
</code>


{| {{Prettytable}}
Once the LaTeX document is compiled, the figure in the PDF becomes active, and when clicked, it will invoke VisTrails and reproduce the result.
|-
You can also upload your results to [http://www.crowdlabs.org CrowdLabs] and export them to Web sites or Wikis, where users can interact with them through a Web browser. See e.g., http://www.crowdlabs.org/vistrails/medleys/details/26/
|http://www.vistrails.org/images/Demonstration_thumb.png
|'''Demonstration'''
This video shows basic usage of VisTrails by going through three examples.
Some of the features shown are:
* Manipulating a pipeline by adding modules and connections and setting parameters
* Performing a parameter exploration
As this is an old version of the system, some of the features are not currently available.


{{qt|link=http://www.vistrails.org/download/download.php?type=MEDIA&id=demo_high_res.mov|text=High Resolution(81 MB)}}
=== Building Workflows ===
{{qt|link=http://www.vistrails.org/download/download.php?type=MEDIA&id=demo_med_res.mov|text=Medium Resolution(29 MB)}}
VisTrails supports workflows expressed as dataflows. It can also support functional loops and conditional statements (new in VisTrails 1.3!). In addition, VisTrails is easily extensible; users can quickly create new modules or wrap existing code or libraries for use as modules in VisTrails. Also, the <code>PythonSource</code> module can be used to write one-time translations or test code in a workflow. See  the [http://sourceforge.net/projects/vistrails/files/vistrails/vistrails-usersguide-1.3-rev198.pdf/download Users' Guide] for more details on all of these features.
{{qt|link=http://www.vistrails.org/download/download.php?type=MEDIA&id=demo_low_res.mov|text=Low Resolution(16 MB)}}
|-
|http://www.vistrails.org/images/Radiation_thumb.png
|'''Radiation Oncology'''
This video shows how some of VisTrails features were used on a Radiation Oncology study.  


As this is an old version of the system, some of the features are not currently available.
=== Combining VisTrails Provenance with other Tools ===
[[Image:DisplayWall_ipod_screenshot.png|180px|right]]
The VisTrails provenance infrastructure can also be combined with interactive tools. We are developing a series of provenance plugins for different tools, including [https://wci.llnl.gov/codes/visit VisIt] and [http://www.paraview.org/ ParaView]. The [https://vistrails.org/index.php/ParaView_Plugin open-source ParaView plugin has been released].


{{qt|link=http://www.vistrails.org/download/download.php?type=MEDIA&id=vt-rad-onc.mov|text=High Resolution(70 MB)}}
=== Visualizing and Comparing Workflow Results ===
|}
VisTrails provides a ''spreadsheet'' where users can compare the results of multiple workflows, or multiple workflow runs. Besides using a desktop, you can display the results of VisTrails workflows on a display wall or on an iPod. Users can also compare different workflows using the visual difference interface.


== People ==
=== Querying and Refining Workflows by Example ===
Users can quickly construct expressive queries over a collection of workflows by example, using the same familiar interface they use to build workflows. VisTrails also provides an analogy mechanism whereby users can perform complex modifications to workflows by analogy, without having to directly modify the workflow specifications.


=== Principal Investigators ===
==[[Documentation|System Documentation]]==
If you want to get more information about VisTrails you can consult the Users' Manual or the FAQ (see links below). If you have questions that are not answered in these resources, you can post it to the users' mailing list: https://vistrails.org/index.php/MailingLists


* Dr. [http://www.cs.utah.edu/~juliana Juliana Freire]
To report bugs, please use our bug tracker: https://github.com/VisTrails/VisTrails/issues
* Dr. [http://www.cs.utah.edu/~csilva Claudio Silva]


=== Current Team ===
=== [[Documentation#Screen Shots|Screen Shots]]===
=== [[Users_Guide|User's Guide]]===
A Web version of the Users Guide is currently under development and can be accessed at https://vistrails.org/usersguide.
=== [[FAQ]]===


* [http://www.sci.utah.edu/~eranders Erik Anderson]
==[[MailingLists|Mailing Lists]]==
* [http://www.sci.utah.edu/~stevec Steven P. Callahan]
==[[Publications, Tutorials and Presentations]]==
* [http://www.cs.utah.edu/~dakoop David Koop]
==[[People]]==
* [http://www.sci.utah.edu/~emanuele Emanuele Santos]
* [http://www.sci.utah.edu/~cscheid Carlos E. Scheidegger]
* [http://www.sci.utah.edu/~natesmith Nathan Smith]
* [http://www.sci.utah.edu/~hvo Huy T. Vo]


=== Previous Members ===
== [https://vistrails.org/index.php/Projects_using_VisTrails Some Projects using VisTrails] ==


* Louis Bavoil
{| class="wikitable"
* Marcelo Nery dos Santos
|-
* Wayne Tyler
! USGS Habitat modeling
|-
| [[Image:usgs.png|200px|left]]
|
|}
 
{| class="wikitable"
|-
! NASA Climate Data Analysis
! DOE CDAT
! ALPS Simulations
|-
| [[Image:nasa.png|200px|left]]
| [[Image:cdat.png|200px|left]]
| [[Image:alps-shot.png|200px|left]]
|
|}
 
{| class="wikitable"
|-
! NSF STC CMOP
! NSF CDI Wildfire
! NSF DataONE-EVA
|-
| [[Image:cmop-ss.png|200px|left]]
| [[Image:wildfire.png|200px|center]]
| [[Image:eva.png|200px|left]]
|
|}
[https://vistrails.org/index.php/Projects_using_VisTrails See other projects]
 
== [[Vistrails and Teaching]] ==
 
The VisTrails system has been used a tool to teach different courses, including Scientific Visualization and Digital Media.
For more information on the educational uses of VisTrails see [http://vistrails.org/index.php/Vistrails_and_Teaching VisTrails and Teaching]. Our [http://www.cs.utah.edu/~juliana/pub/vistrails-teaching-eurographics2010.pdf paper] describing a provenance-rich teaching methodology received the Best Paper Award at the Eurographics 2010 Education program.


== Sponsors ==
== Sponsors ==


This work has been partially supported by the National Science Foundation under grants  
This work has been partially supported by the National Science Foundation under grants  
[http://www.nsf.gov/awardsearch/showAward.do?AwardNumber=0905385 IIS-0905385],
[http://www.nsf.gov/awardsearch/showAward.do?AwardNumber=0844572 IIS-0844572],
[http://www.nsf.gov/awardsearch/showAward.do?AwardNumber=0746500 IIS CAREER-0746500],
[http://www.nsf.gov/awardsearch/showAward.do?AwardNumber=0751152 CNS-0751152],
[http://www.nsf.gov/awardsearch/showAward.do?AwardNumber=0513692 IIS-0513692],  
[http://www.nsf.gov/awardsearch/showAward.do?AwardNumber=0513692 IIS-0513692],  
[http://www.nsf.gov/awardsearch/showAward.do?AwardNumber=0401498 CCF-0401498],  
[http://www.nsf.gov/awardsearch/showAward.do?AwardNumber=0401498 CCF-0401498],  
[http://www.nsf.gov/awardsearch/showAward.do?AwardNumber=0541560 CNS-0541560], [http://www.nsf.gov/awardsearch/showAward.do?AwardNumber=0405402 OISE-0405402],  
[http://www.nsf.gov/awardsearch/showAward.do?AwardNumber=0541560 CNS-0541560], [http://www.nsf.gov/awardsearch/showAward.do?AwardNumber=0405402 OISE-0405402],  
[http://www.nsf.gov/awardsearch/showAward.do?AwardNumber=0424602 OCE-0424602],  
[http://www.nsf.gov/awardsearch/showAward.do?AwardNumber=0424602 OCE-0424602],  
[http://www.nsf.gov/awardsearch/showAward.do?AwardNumber=0524096 CNS-0524096],
[http://www.nsf.gov/awardsearch/showAward.do?AwardNumber=0534628 IIS-0534628],
the Department of Energy under the SciDAC program
the Department of Energy under the SciDAC program
([http://www.scidac.org/compsci/SDM.html SDM]
([http://www.scidac.org/compsci/SDM.html SDM]
and
and
[http://www.scidac.org/viz/VACET.html VACET]),
[http://www.scidac.org/viz/VACET.html VACET]) and UV-CDAT,
IBM Faculty Awards (2005 and 2006) and a University of Utah Seed Grant.
IBM Faculty Awards (2005, 2006, 2007, and 2008) and a University of Utah Seed Grant.
 
== Related Sites ==
 
[[BirdVis]]
 
[http://www.crowdlabs.org CrowdLabs]
 
[[RepeatabilityCentral]]
 
[[ProvenanceAnalytics]]
 
[[Provenance: potpourri]]

Latest revision as of 17:30, 25 April 2022

This is the official website for VisTrails, an open-source scientific workflow and provenance management system that supports data exploration and visualization.

VisTrails provides a general infrastructure that can be combined with existing systems and libraries. Official releases come with several packages, including VTK, matplotlib, and ImageMagick. The system is written in Python/Qt and runs on Mac, Unix and Windows.

Read more >

Please note that as of 2018, we are no longer maintaining VisTrails.

Downloading VisTrails

Current and past releases are available on our download page.

News

  • Unfortunately we can no longer maintain VisTrails
  • [2016-05-03] VisTrails 2.2.4 has been released and is available on the Downloads page.
  • [2015-09-11] SAHM VisTrails training course at USGS: http://pubs.usgs.gov/fs/2014/3007/pdf/fs2014-3007.pdf
  • [2015-07-10] VisTrails 2.2.2 has been released and is available on the Downloads page.
  • [2015-04-22] VisTrails 2.2 has been released and is available on the Downloads page.
  • [2015-03-21] VisTrails 2.1.5 has been released and is available on the Downloads page.
  • [2014-08-25] VisTrails 2.1.4 has been released and is available on the Downloads page.
  • VisTrails 2.0.3 has been released and is available on the Downloads page.
  • To learn more about how to explore climate data using DV3D and ParaView in UV-CDAT, check out http://www.kitware.com/source/home/post/105
  • Comparison of two forecasts of Hurricane Sandy using the DV3D slicer in UV-CDAT http://www.youtube.com/watch?v=BFN0RzN1hSE&feature=em-share_video_user
  • Presentation on the use of VisTrails at NASA: VisTrails: Workflow Management System on NEX - https://c3.nasa.gov/nex/static/media/other/Vistrails-Votava.pdf
  • VisTrails 2.0 has been released and it is available for download on our Downloads page. VisTrails is now distributed under the BSD license and its GUI was completely re-designed to make it easier to manage and interact with multiple workflows. It also includes a much improved Users' Manual. See the release notes for a complete list of changes. Some notable features include:
    • the ability to create mashups;
    • support for the creation of reproducible papers;
    • a provenance viewer;
    • support for visual diffs as well as queries over workflows in different vistrails;
    • a Control Flow Assistant to simplify the creation of workflows with loops and conditional constructs;
    • support for global variables, so that modules deriving from Constant can be used to create variables that are shared among multiple workflows.
Screen Shot 2012-01-12 at 2.50.19 PM.png


VisTrails Overview

Cosmology example.png

VisTrails is an open-source scientific workflow and provenance management system that provides support for simulations, data exploration and visualization. Whereas workflows have been traditionally used to automate repetitive tasks, for applications that are exploratory in nature, such as simulations, data analysis and visualization, very little is repeated---change is the norm. As an engineer or scientist generates and evaluates hypotheses about data under study, a series of different, albeit related, workflows are created while a workflow is adjusted in an interactive process. VisTrails was designed to manage these rapidly-evolving workflows.

A key distinguishing feature of VisTrails is a comprehensive provenance infrastructure that maintains detailed history information about the steps followed and data derived in the course of an exploratory task: VisTrails maintains provenance of data products, of the workflows that derive these products and their executions. This information is persisted as XML files or in a relational database, and it allows users to navigate workflow versions in an intuitive way, to undo changes but not lose any results, to visually compare different workflows and their results, and to examine the actions that led to a result. It also enables a series operations and user interfaces that simplify workflow design and use, including the ability to create and refine workflows by analogy and to query workflows by example.

VisTrails supports the creation and execution of workflows. It allows the combination of loosely-coupled resources, specialized libraries, grid and Web services. The released version comes with support for several packages including, VTK, Image Magick, Web Services, and pylab. You can also download packages contributed by our users, as well as create your own packages/modules. There are also a number of projects extending or building on top of VisTrails. Workflows can be run interactively, through the VisTrails GUI, or in batch using a VisTrails server.

VisTrails 1.0 was released in October 2007. Since then, the system has been downloaded thousands of times. VisTrails is written in Python and it uses the multi-platform Qt library for its user interface. It runs on Mac, Linux and Windows. You can download VisTrails from https://vistrails.org/index.php/Downloads

Publishing Reproducible Results

VisTrails 2.0 allows the inclusion of reproducible results in LaTeX/PDF documents. We provide a LaTeX package that allows users to add links to their results in the LaTeX source. For example:

\usepackage{vistrails}

...

\begin{figure}

\begin{center}

\subfigure[a=0.9]{\vistrail[filename=alps.vt, version=2, pdf]{width=8cm}}

\subfigure[a=0.9]{\vistrail[filename=alps.vt, version=11, pdf,buildalways]{width=8cm}}

\caption{A figure produced by an ALPS VisTrails workflow. Clicking the figure retrieves the workflow used to create it. Opening that workflow on a machine with VisTrails and ALPS installed lets the reader execute the full calculation.}

\end{center}

\end{figure}

Once the LaTeX document is compiled, the figure in the PDF becomes active, and when clicked, it will invoke VisTrails and reproduce the result. You can also upload your results to CrowdLabs and export them to Web sites or Wikis, where users can interact with them through a Web browser. See e.g., http://www.crowdlabs.org/vistrails/medleys/details/26/

Building Workflows

VisTrails supports workflows expressed as dataflows. It can also support functional loops and conditional statements (new in VisTrails 1.3!). In addition, VisTrails is easily extensible; users can quickly create new modules or wrap existing code or libraries for use as modules in VisTrails. Also, the PythonSource module can be used to write one-time translations or test code in a workflow. See the Users' Guide for more details on all of these features.

Combining VisTrails Provenance with other Tools

DisplayWall ipod screenshot.png

The VisTrails provenance infrastructure can also be combined with interactive tools. We are developing a series of provenance plugins for different tools, including VisIt and ParaView. The open-source ParaView plugin has been released.

Visualizing and Comparing Workflow Results

VisTrails provides a spreadsheet where users can compare the results of multiple workflows, or multiple workflow runs. Besides using a desktop, you can display the results of VisTrails workflows on a display wall or on an iPod. Users can also compare different workflows using the visual difference interface.

Querying and Refining Workflows by Example

Users can quickly construct expressive queries over a collection of workflows by example, using the same familiar interface they use to build workflows. VisTrails also provides an analogy mechanism whereby users can perform complex modifications to workflows by analogy, without having to directly modify the workflow specifications.

System Documentation

If you want to get more information about VisTrails you can consult the Users' Manual or the FAQ (see links below). If you have questions that are not answered in these resources, you can post it to the users' mailing list: https://vistrails.org/index.php/MailingLists

To report bugs, please use our bug tracker: https://github.com/VisTrails/VisTrails/issues

Screen Shots

User's Guide

A Web version of the Users Guide is currently under development and can be accessed at https://vistrails.org/usersguide.

FAQ

Mailing Lists

Publications, Tutorials and Presentations

People

Some Projects using VisTrails

USGS Habitat modeling
Usgs.png
NASA Climate Data Analysis DOE CDAT ALPS Simulations
Nasa.png
Cdat.png
Alps-shot.png
NSF STC CMOP NSF CDI Wildfire NSF DataONE-EVA
Cmop-ss.png
Wildfire.png
Eva.png

See other projects

Vistrails and Teaching

The VisTrails system has been used a tool to teach different courses, including Scientific Visualization and Digital Media. For more information on the educational uses of VisTrails see VisTrails and Teaching. Our paper describing a provenance-rich teaching methodology received the Best Paper Award at the Eurographics 2010 Education program.

Sponsors

This work has been partially supported by the National Science Foundation under grants IIS-0905385, IIS-0844572, IIS CAREER-0746500, CNS-0751152, IIS-0513692, CCF-0401498, CNS-0541560, OISE-0405402, OCE-0424602, CNS-0524096, IIS-0534628, the Department of Energy under the SciDAC program (SDM and VACET) and UV-CDAT, IBM Faculty Awards (2005, 2006, 2007, and 2008) and a University of Utah Seed Grant.

Related Sites

BirdVis

CrowdLabs

RepeatabilityCentral

ProvenanceAnalytics

Provenance: potpourri