{"thread":{"id":"37941","subject":"smudge filters during checkout & crash consistency","startedAt":"2014-11-12T17:46:19Z","lastAt":"2014-11-12T20:51:16Z","messageCount":5,"participants":["Derek Moore","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"251760","messageId":"CAMsgyKbox7e2pv4+_=jG6Ywh3Km2gPsw+Qf6qj-28GWrVg7RZQ@mail.gmail.com","threadId":"37941","inReplyTo":null,"subject":"smudge filters during checkout & crash consistency","fromName":"Derek Moore","fromEmail":"derek.p.moore@gmail.com","sentAt":"2014-11-12T17:46:19Z","receivedAt":"2014-11-12T17:46:19Z","isPatch":false,"sender":{"key":"derek.p.moore@gmail.com","avatar":"https://gravatar.com/avatar/4bf86633cdd04eb5d07180791de5ae0ece9f3d04b34f6e13fdb81d563fc62c23?d=mp&s=160"},"body":"I have a case where I would like to smudge files according to the\nreflog information of the switching-to branch.\n\nThis is difficult to achieve because updating HEAD to the new\nswitched-to refname or commit hash is the last step performed in a\ncheckout prior to calling the post-checkout hook, and smudge filters\nprocess content during the rewriting of the index and work-tree before\nHEAD is updated.\n\nI believe this weakness of checkout & filters also exposes a crash\nconsistency concern. Suppose power is lost during a long-running\ncheckout while the index/worktree is being updated but before the new\nHEAD file is written.\n\nUpon coming back up, your git status will show edits against your\nswitching-from branch, and possibilities of recovery would rely on\nyour memory of what you were doing (instead of git-status reporting\n\"Incomplete checkout to {branch,commit}, 'git checkout --continue' to\ncontinue\").\n\nMaybe git could record a CHECKOUT_HEAD at the start of a checkout,\nthen at the end of the commit update_refs_for_switch() would move\nCHECKOUT_HEAD over top HEAD instead of rewriting HEAD (but,\npresumably, a lot of logic in update_refs_for_switch() would have to\nbe relocated to when CHECKOUT_HEAD is written, other implications\nnotwithstanding).\n\nCrash consistency aside, my workaround for filtering will probably be\nto use a fake smudge filter that records the file paths of all\nto-be-smudged files to a file under .git/, and then use a post-commit\nhook that will process those files from within the newly checked-out\nbranch (where I'll be using git-archive to overwrite files).\n\nSeems git could fix these two concerns in one fell swoop.\n\nThanks,\n\nDerek\n"},{"id":"251765","messageId":"xmqqk33046ha.fsf@gitster.dls.corp.google.com","threadId":"37941","inReplyTo":"CAMsgyKbox7e2pv4+_=jG6Ywh3Km2gPsw+Qf6qj-28GWrVg7RZQ@mail.gmail.com","subject":"Re: smudge filters during checkout & crash consistency","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2014-11-12T18:30:09Z","receivedAt":"2014-11-12T18:30:09Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Derek Moore writes:\n\n> I have a case where I would like to smudge files according to the\n> reflog information of the switching-to branch.\n\nDon't do that.\n\nWhen you have branches A, B and C, and a path F is the same between\nbranches A and B but different in branch C, if you start from branch\nC and switch to branch A, F will be updated and obtain your smudge\ntailored for \"branch A's instance of F\".\n\nBut if you then switch to B from that state, F will not even be\nmodified (i.e. it will keep the contents you prepared for \"branch\nA's instance of F\").\n\nIn short, do not make clean/smudge depend on anything but blob\ncontents.\n"},{"id":"251767","messageId":"CAMsgyKa7zSqYFHVt9VGz2zarBwCcJRxjGtkAoaxbtrnMJCZxpg@mail.gmail.com","threadId":"37941","inReplyTo":"xmqqk33046ha.fsf@gitster.dls.corp.google.com","subject":"Re: smudge filters during checkout & crash consistency","fromName":"Derek Moore","fromEmail":"derek.p.moore@gmail.com","sentAt":"2014-11-12T19:41:54Z","receivedAt":"2014-11-12T19:41:54Z","isPatch":false,"sender":{"key":"derek.p.moore@gmail.com","avatar":"https://gravatar.com/avatar/4bf86633cdd04eb5d07180791de5ae0ece9f3d04b34f6e13fdb81d563fc62c23?d=mp&s=160"},"body":"Here's a solution that depends only/mostly on blob contents:\n\n1) construct the ident of the blob via an `(echo -e -n \"blob \\0\"\n; cat file) | sha1sum` equivalent if an $Id$ string is not found in\nits contents,\n\n2) look up the earliest commit with that blob hash at that path, and\n\n3) use the reflog metadata from that earliest commit.\n\nThen when switching from C-to-A or C-to-B, F will have the same\ncontents as a noop switch when switching A-to-B from C-to-A (although,\nconceivably, you may get a commit that is in neither A nor B, but you\nwill have the earliest introduction of that file at that state).\n\nIn other words, always use the earliest occurrence of a specific\ncontent at a given path, earliest commit wins irrespective of\nbranches. Not the most elegant solution.\n\nI may have to go back and let these people know that outside of build\nscripts they can't get what they think they want.\n\nThanks!\n\n:D\n\nOn Wed, Nov 12, 2014 at 12:30 PM, Junio C Hamano wrote:\n> Derek Moore writes:\n>\n>> I have a case where I would like to smudge files according to the\n>> reflog information of the switching-to branch.\n>\n> Don't do that.\n>\n> When you have branches A, B and C, and a path F is the same between\n> branches A and B but different in branch C, if you start from branch\n> C and switch to branch A, F will be updated and obtain your smudge\n> tailored for \"branch A's instance of F\".\n>\n> But if you then switch to B from that state, F will not even be\n> modified (i.e. it will keep the contents you prepared for \"branch\n> A's instance of F\").\n>\n> In short, do not make clean/smudge depend on anything but blob\n> contents.\n"},{"id":"251771","messageId":"CAMsgyKagoz7NU7cGuwvq61aiKc6Wq-z+w0_Fep7t9tYy90pB6w@mail.gmail.com","threadId":"37941","inReplyTo":"xmqqk33046ha.fsf@gitster.dls.corp.google.com","subject":"Re: smudge filters during checkout & crash consistency","fromName":"Derek Moore","fromEmail":"derek.p.moore@gmail.com","sentAt":"2014-11-12T20:30:12Z","receivedAt":"2014-11-12T20:30:12Z","isPatch":false,"sender":{"key":"derek.p.moore@gmail.com","avatar":"https://gravatar.com/avatar/4bf86633cdd04eb5d07180791de5ae0ece9f3d04b34f6e13fdb81d563fc62c23?d=mp&s=160"},"body":"> But if you then switch to B from that state, F will not even be\n> modified (i.e. it will keep the contents you prepared for \"branch\n> A's instance of F\").\n\nOr: the post-commit hook used in the workaround looks up the prior\nbranch via @{-1}, finds all files common between @ & @{-1} that don't\nshare a latest commit, deletes those files and replaces them singly\nwith the results of git-archive using the latest commits of those\nfiles relative to @. (\"All files common between @ & @{-1}\" would need\nto be either all non-locally-modified files or making use of git-stash\n{save,pop} to preserve local modifications.) All this assumes having\nreversible $Format$ strings, so the clean filter can restore the\nproper $Format$ string.\n\nMight be worth doing just so there's at least 1 accurate and\nmaybe-fast \"git rcs keywords substitution using smudge/clean filters\"\nproject on github. ;) Otherwise, users of \"git-keyword-substitution\"\nand \"git-rcs-keywords\" are being led astray.\n"},{"id":"251773","messageId":"xmqqfvdo3zy3.fsf@gitster.dls.corp.google.com","threadId":"37941","inReplyTo":"CAMsgyKagoz7NU7cGuwvq61aiKc6Wq-z+w0_Fep7t9tYy90pB6w@mail.gmail.com","subject":"Re: smudge filters during checkout & crash consistency","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2014-11-12T20:51:16Z","receivedAt":"2014-11-12T20:51:16Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Derek Moore writes:\n\n>> But if you then switch to B from that state, F will not even be\n>> modified (i.e. it will keep the contents you prepared for \"branch\n>> A's instance of F\").\n>\n> Or: the post-commit hook used in the workaround looks up the prior\n> branch via @{-1}, finds all files common between @ & @{-1} that don't\n> share a latest commit, deletes those files and replaces them singly\n> with the results of git-archive using the latest commits of those\n> files relative to @. (\"All files common between @ & @{-1}\" would need\n> to be either all non-locally-modified files or making use of git-stash\n> {save,pop} to preserve local modifications.) All this assumes having\n> reversible $Format$ strings, so the clean filter can restore the\n> proper $Format$ string.\n>\n> Might be worth doing...\n\nI still do not see what you are trying to record in the checked out\nsource files with your smudge filter, so I won't comment if it might\nbe \"worth\" doing.\n\nYour use of reflog suggests me that whatever you are recording\ndepends on how you acquired your history in your specific repository\nyou work in, and your result is not reproducible by other people who\nwork with you by fetching from a repository that is different from\nthe repository you work in. E.g. perhaps you have a repository at\nGitHub and push into there, and others fetch from there into their\nrepository. What is in their reflog has no relation to what you\nhave in your reflog.\n\nThat's the nature of distrubuted life. More generally, in a\ndistributred world with merges, even between two people who agree\nthat the tip of the 'master' branch of the project is at a certain\ncommit, there is no single sensible answer to the question \"which\ncommit changed this path last?\" We wouldn't mind anything you may\ndo to emulate RCS $Id$, but it would be futile.\n"}]}