Data/provenance browser

Dave's ideas

• We need a better mechanism that allows users to search/query for workflows and provenance, for example search by project, parameter names and values
• One way to view this is to allow the creation of mappings between the vistrails/projects into different structures---like "smart folders", e.g., /projectN/vt1/wf3/exec23/O1....
• Support editing and synchronization in both ways (if edit smart folder, should update vt)
• Should be able to script the use of the browser, e.g., to test a new version of a workflow against a saved test suite
• Scientists like a directory structure and can go browse there like they browse files with finder/explorer
• Keep mappings from our datatypes to file structures
• Need an exportable archive (to send with a paper, allow browsing outside of VisTrails)

Juliana's suggestions

As for the steps, here's what I suggest:

1. try and get multiple vistrails accessible through the browser
2. allow users to sort the information in different ways, based on the metadata stored in SQLLite.
3. support for (simple) queries over the actual provenance info (version tree, workflows and logs), e.g., we could support the layered queries we described in the First Provenance Challenge.
4. design and implement the map-to-directory functionality.

As we implement these, we should make sure we have a clean API that can be used without a GUI.

Tommy's notes

• How to add project annotations?
• Keep a synchronized index
  • "Monitor" a specified directory for changes?
  • "add" vistrail or directory to index
  • Index vistrail when opened/saved
    • How to efficiently index the execution log?
• Browsing the index
  • Create smart directory structure using project/vistrail/workflow
  • If we allow a vistrail to belong to multiple projects, the directory structure will be a DAG and not a tree.
• Querying the index
  • Keyword searches
  • Search on specific types e.g. "vistrail:proj3*"
• Export functionality
  • Export all vistrails in a specific project in a directory structure
• Also handle workflow data?