git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Git as electronic lab notebook

From
TJThomas Johnson <thomas.j.johnson@gmail.com>
Date
Dec 19, 2009, 12:23 UTC
Message-ID
<loom.20091219T130946-844@post.gmane.org>
Hello group,
I've been using git on a few different projects over the last couple of months,
and as a former svn user I really like it. Recently, I've been using it as an
'electronic lab notebook' for an empirical project. My workflow looks like this:
1. Start with the stable code base on head
2. Create  and change to branch 'Experiment123'
3. Make some changes
4. Run the program, which generates a giant (10MB-4G) output text file,
Experiment123.log. Update my LabNotebook.txt file.
5. Were the new changes helpful?
5.yes: Bzip Experiment123.log, and commit it on the branch. Merge the
Experiment123 branch to head and goto 1.
5.no: Bzip Experiment123.log, and commit it on the branch. Merge LabNotebook.txt
and Experiment123.log back to head. Switch back to head and goto 1.

The thing is, Experiment123.log is going to be very similar to Experiment122.log and Experiment124.log except for a few details. My understanding is that git is great at compressing groups of files like this, is that correct? Should I not be bzipping them myself? On the other hand, I don't want HEAD to contain hundreds of gigs of uncompressed files that bzip down to only a few hundred megs.

Any thoughts on the workflow itself would also be very welcome.
Next: Ciprian Dorin, Craciun
Message 1 of 6 in “Git as electronic lab notebook”
  1. Thomas JohnsonDec 19, 2009
  2. Ciprian Dorin, CraciunDec 19, 2009
  3. Johan 't HartDec 20, 2009
  4. Nicolas PitreDec 20, 2009
  5. Bill LearDec 20, 2009
  6. Nicolas PitreDec 20, 2009

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.