Latest revision as of 22:15, 23 April 2026

VisTrails

VisTrails is an open-source scientific workflow and provenance management system developed at the VIDA Center at New York University. It supports computational science by capturing and managing the complete history of the exploratory process: the workflows, their executions, and the results they produce.

VisTrails is actively developed again. The new version, VisTrailsJL, is a complete reimplementation in Julia that brings modern performance, notebook-based workflow authoring, and native compatibility with existing .vt files. See the GitHub repository to get started.

What's New

After a hiatus since 2018, VisTrails is back. VisTrailsJL (v2.2) is a ground-up reimplementation in Julia that preserves everything that made the original system valuable — comprehensive provenance, visual workflow management, and support for real scientific use cases — while modernizing the foundation:

Julia reimplementation — Julia's JIT compilation brings performance suitable for demanding scientific workflows, and its rich ecosystem (DataFrames.jl, DifferentialEquations.jl, Plots.jl) is a natural fit.
Notebook-based workflow authoring — Workflows can now be defined directly in Jupyter notebooks using simple #| directives, with no GUI required.
Full .vt compatibility — Existing workflows created with the Python version can be loaded, replayed, and visualized without modification.
Git-native version control — Standard git replaces the custom versioning infrastructure for workflow history.
Python interoperability — Existing Python modules and libraries remain accessible via PyCall.jl.

The original Python codebase (v2.2) is preserved in the repository for reference and compatibility testing.

Quick links: GitHub (VisTrailsJL) | Documentation | Publications, Tutorials and Presentations | Mailing Lists

Core Features

Provenance and Workflow History

A defining feature of VisTrails is its comprehensive provenance infrastructure. Unlike systems that track only the current state of a workflow, VisTrails maintains the full history of every step taken during an exploratory analysis — what was tried, what was changed, and what results each version produced. This enables users to:

Navigate and compare workflow versions in an intuitive tree interface
Undo changes without losing intermediate results
Visually diff two workflows and their outputs side by side
Reproduce any prior result exactly, long after it was first computed

Provenance information is stored as XML or in a relational database (Python version), or managed via standard git (Julia version).

Building and Running Workflows

VisTrails supports workflows expressed as dataflows, with support for functional loops and conditional branching. Workflows can be run interactively through the GUI or in batch mode via a server. The system is designed to connect loosely coupled resources — specialized libraries, web services, and grid computing infrastructure.

In VisTrailsJL, workflows can also be defined declaratively in Jupyter notebooks:

#| workflow: my_analysis

#| module-id: input
#| module-type: basic:Integer
#| params:
#|   - value: 42

#| module-id: process
#| module-type: mypackage:Transform
#| inputs:
#|   - value: input.value

#| execute

Packages and modules are easy to add. The JuliaSource and PythonSource module types allow custom code to be embedded directly in a workflow without creating a full package.

Publishing Reproducible Results

VisTrails 2.0 introduced support for embedding reproducible results directly in LaTeX/PDF documents via a companion LaTeX package. A figure in a compiled PDF becomes active: clicking it invokes VisTrails and re-executes the workflow that produced it on any machine with the software installed.

\usepackage{vistrails}

\begin{figure}
\begin{center}
\subfigure[a=0.9]{\vistrail[filename=alps.vt, version=2, pdf]{width=8cm}}
\caption{Clicking this figure retrieves and re-runs the workflow that produced it.}
\end{center}
\end{figure}

Querying and Refining Workflows

Users can construct expressive queries over a collection of workflows using the same interface used to build them. An analogy mechanism allows complex modifications to be applied to one workflow by example from another, without manually editing workflow specifications — useful when a family of related analyses needs to evolve together.

Visualizing and Comparing Results

VisTrails provides a spreadsheet view for comparing the results of multiple workflows or multiple parameterizations of the same workflow side by side. The visual diff interface highlights structural differences between two workflow versions. Workflows and their version trees can be rendered as SVG (VisTrailsJL) or displayed on large-format display walls.

Getting Started

VisTrailsJL (Julia — current)

# Clone the repository
git clone https://github.com/VIDA-NYU/VisTrailsJL.git
cd VisTrailsJL/julia

# Install dependencies
julia --project=. -e 'using Pkg; Pkg.instantiate()'

# Load and render an existing workflow
julia --project=. -e '
using VisTrailsJL
vt = load_vistrail("../examples/gcd.vt")
workflow = get_pipeline(vt)
render_pipeline_svg(workflow, "workflow.svg")
'

See the Quickstart Guide for a full walkthrough.

Python VisTrails (legacy reference)

The original Python version (v2.2, requires Python 2 / PyQt4) is preserved in the repository for reference and for loading existing .vt files in legacy environments.

# GUI mode
python vistrails/run.py

# Batch mode
python vistrails/run.py --batch [options]

Projects Using VisTrails

VisTrails has supported real scientific workflows across a wide range of domains. The following projects reflect the breadth of communities that have relied on the system.

USGS Habitat Modeling	NASA Climate Data Analysis	DOE CDAT

ALPS Simulations	NSF STC CMOP	NSF CDI Wildfire

NSF DataONE-EVA

See all projects using VisTrails

VisTrails in Teaching

VisTrails has been used as a teaching tool in courses on Scientific Visualization and Digital Media. Its provenance infrastructure makes it particularly effective in educational settings, where capturing and comparing student workflows provides rich feedback for instructors and learners alike.

Our paper describing a provenance-rich teaching methodology received the Best Paper Award at Eurographics 2010 Education.

System Documentation

To report bugs or request features, please use the issue tracker.

For questions not covered by the documentation, post to the mailing list.

Citing VisTrails

If you use VisTrails or VisTrailsJL in your research, please cite the relevant work:

Original VisTrails system:

@inproceedings{vistrails2006,
  title     = {VisTrails: visualization meets data management},
  author    = {Callahan, Steven P and Freire, Juliana and Scheidegger,
               Carlos E and Silva, Cl{\'a}udio T and Vo, Huy T},
  booktitle = {Proceedings of the 2006 ACM SIGMOD International Conference
               on Management of Data},
  pages     = {745--747},
  year      = {2006},
  doi       = {10.1145/1142473.1142574}
}

VisTrailsJL (Julia reimplementation):

@software{vistrailsjl2025,
  title  = {VisTrailsJL: A Julia Implementation of VisTrails},
  author = {Silva, Claudio T},
  year   = {2025},
  url    = {https://github.com/VIDA-NYU/VisTrailsJL}
}

Full publication list

People

Difference between revisions of "Main Page"