recipy
Effortless provenance tracking in Python
Description
- Keep track of what code you ran to generate results (e.g., graphs or data)
- Add a single statement to enable provenance tracking in your Python script
- Search your runs using a command line interface or GUI
- Customize provenance tracking for each project
Imagine the situation: You’ve written some wonderful Python code which produces a beautiful graph as an output. You save that graph, naturally enough, as graph.png. You run the code a couple of times, each time making minor modifications. You come back to it the next week/month/year. Do you remember how you created that graph? What input data? What version of your code? Frustratingly, the answer will often be 'no'. Of course, you then waste lots of time trying to work out how you created it, or even give up and never use it in that journal paper that will win you a Nobel Prize…
ReciPy (from recipe and python) is a Python module that will save you from this situation! (Although it can’t guarantee that your paper will win a Nobel Prize!) With the addition of a single line of code to the top of your Python files, ReciPy will log each run of your code to a database, keeping track of the input files, output files and the version of your code, and then let you query this database to find out how you actually did create graph.png.
Participating organisations
Mentions
- 1.Author(s): Sonia Natalie Mitchell, Andrew Lahiff, Nathan Cummings, Jonathan Hollocombe, Bram Boskamp, Ryan Field, Dennis Reddyhoff, Kristian Zarebski, Antony Wilson, Bruno Viola, Martin Burke, Blair Archibald, Paul Bessell, Richard Blackwell, Lisa A. Boden, Alys Brett, Sam Brett, Ruth Dundas, Jessica Enright, Alejandra N. Gonzalez-Beltran, Claire Harris, Ian Hinder, Christopher David Hughes, Martin Knight, Vino Mano, Ciaran McMonagle, Dominic Mellor, Sibylle Mohr, Glenn Marion, Louise Matthews, Iain J. McKendrick, Christopher Mark Pooley, Thibaud Porphyre, Aaron Reeves, Edward Townsend, Robert Turner, Jeremy Walton, Richard ReevePublished in Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences by The Royal Society in 202210.1098/rsta.2021.0300
- 2.Author(s): George Datseris, Jonas Isensee, Sebastian Pech, Tamás GálPublished in Journal of Open Source Software by The Open Journal in 2020, page: 267310.21105/joss.02673
- 3.Author(s): João Felipe Pimentel, Juliana Freire, Leonardo Murta, Vanessa BraganholoPublished in ACM Computing Surveys by Association for Computing Machinery (ACM) in 2019, page: 1-3810.1145/3311955