Archive

From VistrailsWiki
Revision as of 19:24, 24 September 2013 by Remi (talk | contribs) (Summary from meeting with Matthias)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

Matthias Troyer came to Poly to discuss his use of VisTrails and the problems he was facing with the persistence package.

Fernando, Juliana, Matthias and Remi met on 2013-09-24.

Summary

Persistence only used as a cache

  • Can’t delete stuff; he deletes and recreates the whole store
  • He wants to use it to archive correct result, without the other intermediate files that resulted from bogus workflow OR module code
  • He wants to be able to find his files afterwards. Git revision hash + file reference = impracticable
  • He doesn’t mind filenames being unreadable if he has some way of finding these from metadata (workflow name, vistrail query, or custom metadata from module code)

Conclusions:

  • Drop git. If we are only going to use it for storage, and keep a separate database to map from ref uuid/upstream hash to object hash, commits and branches are useless (and are a nuisance because we can’t rewrite history)
  • Use a flat object store with hashes (like git’s)
  • Use a database to associate hash filename (upstream modules hash?) to metadata: vistrails parameter, (execution info), custom metadata
  • Request: make this separate from VisTrails (and used by the new archive package) so that it can be used directly by other code, and used to find files in the store