# Dumb "continuous" commit dumb question 9 messages from 2008-08-19 to 2008-08-19. Participants: Pat LeSmithe, Marcus Griep, David Tweed, Shawn O. Pearce, Avery Pennarun, Jeff King. Thread: https://gitlist.dev/t/15079 ## Pat LeSmithe, 2008-08-19 03:47 Subject: Dumb "continuous" commit dumb question Message-ID: <48AA4263.8090606@gmail.com> URL: https://gitlist.dev/e/48AA4263.8090606%40gmail.com ``` Hello, Is it possible to enable git to automatically and continuously "softly" commit or stage *all* changes to a [subset of] files in a repository, without my intervention, as they happen? Perhaps via a daemon which monitors the disk for explicit file-saving? Of course, I would still be able to perform explicit commits (with descriptive comments) and other git commands, in which case there probably should be smart handling of the recent soft history. For example, it could simply be discarded. I understand that I could simply remember to commit and/or branch early and often. But given that changes by an individual on a given branch are well-ordered by time, and that the "continuous" operation may be cheap in many situations, a "live" journal could be useful. Perhaps a better term is branch-aware undo or git with microstructure. Sorry if this is old stuff or plain crap. Thanks. Sincerely, Pat LeSmith ``` ## Marcus Griep, 2008-08-19 04:13 Subject: Re: Dumb "continuous" commit dumb question Message-ID: <48AA4873.4040107@gmail.com> URL: https://gitlist.dev/e/48AA4873.4040107%40gmail.com In-Reply-To: <48AA4263.8090606@gmail.com> ``` Pat LeSmithe wrote: > Hello, > > Is it possible to enable git to automatically and continuously "softly" > commit or stage *all* changes to a [subset of] files in a repository, > without my intervention, as they happen? Perhaps via a daemon which > monitors the disk for explicit file-saving? Perhaps with a cron job running a script every minute or so. You'd want to verify that changes actually took place before attempting a commit. Also, what log messages would you be providing? Log messages can be very important to the context of a change and the ability to later find them. If you're using a specific application, such as a Wiki, it may work nicely, as log messages can be easily gleaned, and many offer post-edit hooks that you could use to invoke git. > Of course, I would still be able to perform explicit commits (with > descriptive comments) and other git commands, in which case there > probably should be smart handling of the recent soft history. For > example, it could simply be discarded. You may want to use StGit as that may allow you to make such lightweight commits easily while still allowing reordering, etc., though I am not familiar with that application porcelain. > I understand that I could simply remember to commit and/or branch early > and often. But given that changes by an individual on a given branch > are well-ordered by time, and that the "continuous" operation may be > cheap in many situations, a "live" journal could be useful. > > Perhaps a better term is branch-aware undo or git with microstructure. Obviously, my final caution is that while this is possible with some configuration, you may lose valuable "signal" information in the "noise" of commits that such a continuous operation might incur. This low SNR may make it more difficult to find and revert changes you didn't actually want to happen or to glean a meaningful history of your working tree. While in the operable borders of Git, it's outside the normal usage patterns. If you have success, let us know how it worked for you. -- Marcus Griep GPG Key ID: 0x5E968152 —— http://www.boohaunt.net את.ψο´ ``` ## David Tweed, 2008-08-19 14:32 Subject: Re: Dumb "continuous" commit dumb question Message-ID: URL: https://gitlist.dev/e/e1dab3980808190732i303f06ach50e36e13a624bd23%40mail.gmail.com In-Reply-To: <48AA4263.8090606@gmail.com> ``` On Tue, Aug 19, 2008 at 4:47 AM, Pat LeSmithe wrote: > > Hello, > > Is it possible to enable git to automatically and continuously "softly" > commit or stage *all* changes to a [subset of] files in a repository, > without my intervention, as they happen? Perhaps via a daemon which > monitors the disk for explicit file-saving? > > Of course, I would still be able to perform explicit commits (with > descriptive comments) and other git commands, in which case there > probably should be smart handling of the recent soft history. For > example, it could simply be discarded. > > I understand that I could simply remember to commit and/or branch early > and often. But given that changes by an individual on a given branch > are well-ordered by time, and that the "continuous" operation may be > cheap in many situations, a "live" journal could be useful. What I do is have a script that runs every 10 minutes that stages files to the index and then, using the low-level git plumbing, creates tree and commit objects on a side branch "temp". With this you can easily commit to the main branch "main" PROVIDING you are commiting a superset of the changes you're storing to the side-branch. If you wanted to specify exactly the things in your "main" commit I suspect you'd have to do some kind of "git-reset --mixed" to rollback the side-branch state. One thing to be aware of if you do this is that git expects your index file to be describing what you intend to do on the current branch you are on (in, eg, git status), but by doing it this way you'll get output that acts as if you've staged things for your "main" branch. This isn't a problem for me as my (idiosyncratic) usage is to have commits on my main branch every hour via cron anyway. If you're particularly interested in this approach I can either try and explain the commands I use in email or you can try and extract them from my python script chronoversion: http://www.personal.rdg.ac.uk/~sis05dst/chronoversion.tgz > Perhaps a better term is branch-aware undo or git with microstructure. I actually find I don't use the temp branch to actually undo stuff (partly because I'm not even trying to keep a neat history so I do modifications primarily via new commits). Instead, I sometimes modify and extend my research code in a way that I do maybe an hour or so of new code and refactoring before it's in a state to actually run again. If when it finally runs something's broken I find it very helpful to be able to look backwards to see what changes I've made as the problem either jumps out or I know where to start experimenting. One problem with chronological backups is that they often don't compile/run so you can't bisect on them. One thing I have been trying to figure out is if there's an easy way to modify my build system so that it makes a commit approximately every 10 minutes but just after a successful compile. However, that looks to be a bit too complex and error prone for the moment. -- cheers, dave tweed__________________________ david.tweed@gmail.com Rm 124, School of Systems Engineering, University of Reading. "while having code so boring anyone can maintain it, use Python." -- attempted insult seen on slashdot ``` ## Shawn O. Pearce, 2008-08-19 14:48 Subject: Re: Dumb "continuous" commit dumb question Message-ID: <20080819144853.GD20947@spearce.org> URL: https://gitlist.dev/e/20080819144853.GD20947%40spearce.org In-Reply-To: ``` David Tweed wrote: > What I do is have a script that runs every 10 minutes that stages > files to the index and then, using the low-level git plumbing, creates > tree and commit objects on a side branch "temp". With this you can > easily commit to the main branch "main" PROVIDING you are commiting a > superset of the changes you're storing to the side-branch. Have the script use "export GIT_INDEX_FILE=.git/temp-branch-index" so that it stages changes into an index which isn't the main one, and thus has no impact on the main branch. Thus you can still commit a subset of the temp branch at any time. Actually you can do something like this: export GIT_INDEX_FILE=.git/temp-branch-index && cp .git/index $GIT_INDEX_FILE && git add . && git add -u && git update-ref refs/heads/temp $(date | git commit-tree $(git write-tree) -p temp) ;-) -- Shawn. ``` ## Avery Pennarun, 2008-08-19 14:54 Subject: Re: Dumb "continuous" commit dumb question Message-ID: <32541b130808190754l43f053abnc4e3c5c064d6ade7@mail.gmail.com> URL: https://gitlist.dev/e/32541b130808190754l43f053abnc4e3c5c064d6ade7%40mail.gmail.com In-Reply-To: ``` On Tue, Aug 19, 2008 at 10:32 AM, David Tweed wrote: > One problem with chronological backups is that they often don't > compile/run so you can't bisect on them. One thing I have been trying > to figure out is if there's an easy way to modify my build system so > that it makes a commit approximately every 10 minutes but just after a > successful compile. However, that looks to be a bit too complex and > error prone for the moment. You could just have a makefile rule or bash alias that does something like "make && git commit -a -m temp". Then remember to always run that instead of 'make' when you're building. Of course, committing on a side branch and not screwing with the index would be much less intrusive. Avery ``` ## David Tweed, 2008-08-19 15:02 Subject: Re: Dumb "continuous" commit dumb question Message-ID: URL: https://gitlist.dev/e/e1dab3980808190802r202aadc0p2cf8431f645354e3%40mail.gmail.com In-Reply-To: <32541b130808190754l43f053abnc4e3c5c064d6ade7@mail.gmail.com> ``` On Tue, Aug 19, 2008 at 3:54 PM, Avery Pennarun wrote: > You could just have a makefile rule or bash alias that does something > like "make && git commit -a -m temp". Then remember to always run > that instead of 'make' when you're building. As ever, I wanna do something more deviant than that :-) . The idea is to take a snapshot (if any tracked file has changed) roughly every ten minutes. If there happens to have been a successful compile around that time (+/- 1 minute say), grab the snapshot (including detecting potential newly created files) then. But if there hasn't, I still want a snapshot roughly on that 10 minute interval. I could try doing something like "git reset --soft HEAD~1 && git commit -a" if a make succeeds within 1 minute, on a strictly chronological snapshot but scripted resets make me a bit nervous. It's not hyper-important, just something I'm thinking about. -- cheers, dave tweed__________________________ david.tweed@gmail.com Rm 124, School of Systems Engineering, University of Reading. "while having code so boring anyone can maintain it, use Python." -- attempted insult seen on slashdot ``` ## Avery Pennarun, 2008-08-19 15:08 Subject: Re: Dumb "continuous" commit dumb question Message-ID: <32541b130808190808g79bb53a1l9ea7f2ea4c1e5ed3@mail.gmail.com> URL: https://gitlist.dev/e/32541b130808190808g79bb53a1l9ea7f2ea4c1e5ed3%40mail.gmail.com In-Reply-To: ``` On Tue, Aug 19, 2008 at 11:02 AM, David Tweed wrote: > On Tue, Aug 19, 2008 at 3:54 PM, Avery Pennarun wrote: >> You could just have a makefile rule or bash alias that does something >> like "make && git commit -a -m temp". Then remember to always run >> that instead of 'make' when you're building. > > As ever, I wanna do something more deviant than that :-) . The idea is > to take a snapshot (if any tracked file has changed) roughly every ten > minutes. If there happens to have been a successful compile around > that time (+/- 1 minute say), grab the snapshot (including detecting > potential newly created files) then. But if there hasn't, I still want > a snapshot roughly on that 10 minute interval. I could try doing > something like "git reset --soft HEAD~1 && git commit -a" if a make > succeeds within 1 minute, on a strictly chronological snapshot but > scripted resets make me a bit nervous. > > It's not hyper-important, just something I'm thinking about. Doing the 10-minute snapshot doesn't preclude the on-make snapshot. Just commit at *both* times. But commiting "around the time there was a successful build" is kind of pointless since you might change a file two seconds later. (Or maybe only I'm that idiosyncratic. :)) Shawn's GIT_INDEX_FILE script seems like a good place to start. If it were me, I'd use *two* branches here: one for every time I build, and one for the periodic commits. Then the build branch would always be bisectable, and the periodic branch would always have up-to-date data. Commits on the periodic branch would use *both* branch heads as parents, so you'd be able to easily see and diff the full history. Have fun, Avery ``` ## David Tweed, 2008-08-19 15:21 Subject: Re: Dumb "continuous" commit dumb question Message-ID: URL: https://gitlist.dev/e/e1dab3980808190821l2d859bf0p4986260b1fe1ab65%40mail.gmail.com In-Reply-To: <32541b130808190808g79bb53a1l9ea7f2ea4c1e5ed3@mail.gmail.com> ``` On Tue, Aug 19, 2008 at 4:08 PM, Avery Pennarun wrote: > Doing the 10-minute snapshot doesn't preclude the on-make snapshot. > Just commit at *both* times. But commiting "around the time there was > a successful build" is kind of pointless since you might change a file > two seconds later. (Or maybe only I'm that idiosyncratic. :)) I think I was unclear: the only point about the "roughly" was to emphasize that being dead-on 10 minutes line doesn't matter from the chronological aspect. Clearly you want to start the commit processes as soon as a successful compile you decide is worth keeping completes. (Actually, looking at the commit times on my cron-initiated commits the time is mostly 1 or 2 seconds past the 10 minute boundary already, so there's an already up to 2 seconds for my "detect new files" script to scan the directories.) > Shawn's GIT_INDEX_FILE script seems like a good place to start. If it > were me, I'd use *two* branches here: one for every time I build, and > one for the periodic commits. Then the build branch would always be > bisectable, and the periodic branch would always have up-to-date data. > Commits on the periodic branch would use *both* branch heads as > parents, so you'd be able to easily see and diff the full history. I think you mean 3 branches: successful compile, short term history (every 10 minutes) and long-term archival history (every hour). I already have the last two, although not in as optimal a manner as Shawn suggested, and wipe the short term history every week or so. Interesting idea. I'll ponder it. -- cheers, dave tweed__________________________ david.tweed@gmail.com Rm 124, School of Systems Engineering, University of Reading. "while having code so boring anyone can maintain it, use Python." -- attempted insult seen on slashdot ``` ## Jeff King, 2008-08-19 17:55 Subject: Re: Dumb "continuous" commit dumb question Message-ID: <20080819175520.GB10142@coredump.intra.peff.net> URL: https://gitlist.dev/e/20080819175520.GB10142%40coredump.intra.peff.net In-Reply-To: <20080819144853.GD20947@spearce.org> ``` On Tue, Aug 19, 2008 at 07:48:53AM -0700, Shawn O. Pearce wrote: > Actually you can do something like this: > > export GIT_INDEX_FILE=.git/temp-branch-index && > cp .git/index $GIT_INDEX_FILE && > git add . && > git add -u && > git update-ref refs/heads/temp $(date | git commit-tree $(git write-tree) -p temp) Get with the times, Shawn. It's "git add -A" now. ;) -Peff ```