Practical Provenance
This is an archive of the blog Practical Provenance, previously hosted at https://practicalprovenance.wordpress.com/
This blog is about how to practically create and use provenance data, in particular using the W3C PROV specifications.
Students of the course Understanding Data and their Environment (UDatE) at The University of Manchester may prefer this reading order:
- Validating and visualising PROV
- PROV-N Cheat sheet
- Installing ProvToolbox in Windows
- Installing ProvToolbox on MacOS
- FAQ: Multiple agents sharing roles
- FAQ: What are good PROV-N prefixes
- FAQ: Attribution vs Association
List of content (2023–2013)
-
PROV-N Cheat Sheet This is a quick “cheat sheet” for the PROV-N syntax.
-
Installing ProvToolbox on macOS ProvToolbox is a useful command line tool for validating and visualizing PROV documents, but unfortunately it can be a bit of a challenge to install on Windows and on macOS because of its dependency requirements. This post suggests three step-by-step methods of installing ProvToolbox on your Mac – you should follow the method you feel most comfortable with, but can try the other methods in case of problems. Table of content Overview of requirements Software packaging for macOS Conda Installing Graphviz and OpenJDK with Conda HomeBrew Installing Graphviz with HomeBrew Installing OpenJDK with HomeBrew Installing manually Installing AdoptOpenJDK manually Installing Graphviz manually Installing ProvToolbox Using ProvToolbox from VSCode Overview of requirements As of 2020-08, ProvToolbox 0.
-
Installing ProvToolbox in Windows While there are several tools available for validating and visualizing PROV, the ProvToolbox is perhaps the most useful for validating PROV-N syntax. However, the normal releases does not run in Windows due to a operating system restriction for command line and folder path length. We have suggested a fix, but while we wait for that, here we describe a patch build that should work on Windows. We also show how to install dependencies: Java for executing ProvToolbox, and Graphviz for visualization.
-
Attribution vs association A valid question when writing provenance in responsibility view and process view is. Should we attribute contributors from entities, isn’t that what the activities are showing? In this blog post we explore the different options. Specially with roles it may seem unnecessary to also declare wasAttributedTo statements. It is true that you can conclude from: wasAttributed(ex:entity, ex:agent) then there was some activity X such that: wasGeneratedBy(ex:entity, X) wasAssociatedWith(X, ex:entity) This conclusion follows from the constraint on agents and the definition of wasAttributedTo.
-
Multiple agents sharing roles Assuming the task of writing provenance for a student group exercise, consider the question: Do we need to assign everyone in the group a specific role since in our group we found that for many of the tasks, everyone worked together to complete it? MSc Student in Understanding Data and their Environment, University of Manchester, 2020 This blog post explores the different PROV patterns that could describe this scenario.
-
What are good PROV-N prefixes? In this blog post we explore the role of PROV-N prefixes and how to decide on a good namespace to use your own custom provenance terms. Most examples of PROV-N use example prefixes like: prefix ex <http://example.com/> prefix exg <http://example.org/government> These example domains are explicitly reserved globally for all kinds of examples and training material, and deliberately do not have any content, advertisement or affiliations. Assume you are writing the provenance of a student group exercise, should you be using the prefix/namespace ex and example.
-
Validating and visualising PROV This blog post gives a gentle PROV-N introdction and then explores tools for validating and visualising PROV. One of the advantages of W3C PROV having a common data model is that it can be serialized, or written out, in multiple file formats. The PROV family of W3C specifications describe mappings PROV-XML and PROV-O (which, being based on OWL2 itself has multiple serializations, for Linked Data including RDF formats Turtle and JSON-LD.
-
Tracking versions with PAV The PAV ontology specializes the W3C PROV-O standard to give a lightweight approach to recording details about a resource, giving its Provenance, Authorship and Versioning. Our paper on PAV explores all of these aspects in details. In this blog post we discuss Versioning as modelled by PAV, including their hierarchical organization. Version numbers {#versionnumbers} Semantic versioning {#semver} Making versions retrievable {#retrievable} Ordering previous versions {#ordering} Providing provenance for each version {#each-version} Related work {#relatedwork} PROV-O revisions {#provo} Qualified revisions {#qualified} DC Terms {#dcterms} schema.
-
PAV Ontology paper highly accessed Our recent paper about the PAV ontology has been classified as highly accessed by Journal of Biomedical Semantics, with more than 1097 views since it was published two months ago, with an Altmetric score of 12. The PAV ontology provides a lightweight approach to record typical Provenance, Authorship and Versioning information, and builds upon existing standards like PROV-O and DC Terms. Our previous Practical Provenance post gives a brief overview of PAV, but you might also want to explore these links for more details:
-
Resources that change state The PROV working group received a question from Mike: My understanding is that an entity referenced in a PROV bundle (e.g. via wasGeneratedBy) must be in the bundle…but I do not wish to duplicate entity definitions through out my bundles. My entities are long lived and will exist in multiple bundles. So lets say I have a resource for alarms which contains a list of all alarms my company monitors.
-
PROV released as W3C Recommendations The Provenance Working Group was chartered to develop a framework for interchanging provenance on the Web. The Working Group has now published the PROV Family of Documents as W3C Recommendations, along with corresponding supporting notes. You can find a complete list of the documents in the PROV Overview Note. PROV enables one to represent and interchange provenance information using widely available formats such as RDF and XML. In addition, it provides definitions for accessing provenance information, validating it, and mapping to Dublin Core.
-
Locating provenance for a RESTful web service This blog post shows how RESTful web services can provide, and link to, provenance data for their exposed resources by using the PROV-AQ mechanism of HTTP Link headers. This is demonstrated by showing how to update a hello world REST service implemented with Java and JAX-RS 2.0 to provide these links. The PROV-AQ HTTP mechanism is easiest explained by an example: GET http://example.com/resource.html HTTP/1.1 Accept: text/html HTTP/1.1 200 OK Content-type: text/html Link: <http://example.
-
W3C PROV Implementations: Preliminary Analysis By Khalid Belhajjame, syndicated from https://khalidbelhajjame.wordpress.com/2013/04/04/w3c-prov-implementations/ In the beginning of December 2012, the W3C Provenance Working Group issued a call for implementations. As of February the 25th 2013, 64 PROV implementations were reported to the W3C Provenance Working Group. These implementations took different forms ranging from stand alone applications (30), to reusable frameworks and libraries (10), to services hosted by third parties (9), to vocabularies (21), and constraints validation modules (3).
-
Recording authorship, curation and digital creation with the PAV ontology PAV is a lightweight ontology for tracking Provenance, Authoring and Versioning. PAV supplies terms for distinguishing between the different roles of the agents contributing content in current web based systems: contributors, authors, curators and digital artifact creators. The ontology also provides terms for tracking provenance of digital entities that are published on the web and then accessed, transformed and consumed.
-
Tutorial on the W3C PROV family of specifications Posted by Khalid Belhajjame Provenance, a form of structured metadata designed to record the origin or source of information, can be instrumental in deciding whether information is to be trusted, how it can be integrated with other diverse information sources, and how to establish attribution of information to authors throughout its history. The PROV set of specifications, produced by the World Wide Web Consortium (W3C), is designed to promote the publication of provenance information on the Web, and offers a basis for interoperability across diverse provenance management systems.
-
What can provenance do for me? 2013-03-21 What can provenance do for me? from Stian Soiland-Reyes Also available on Slideshare, pdf and as pptx. The above presentation was originally given at the Metagenomics, metagenetics and Pylogenetic workflows for Ocean Sampling Day Workshop at Max Planck Institute for Marine Microbiology on 2013-03-21 by Stian Soiland-Reyes. Reuse allowed under the Creative Commons Attribution license 3.0.