{"thread":{"id":"16749","subject":"Git Notes idea.","startedAt":"2008-12-16T08:15:47Z","lastAt":"2008-12-20T20:09:24Z","messageCount":41,"participants":["Govind Salinas","Jeff King","Johannes Schindelin","Johan Herland","Petr Baudis","Stephan Beyer","Junio C Hamano","Boyd Stephen Smith Jr.","Robin Rosenberg"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"98050","messageId":"5d46db230812160015t55b4ff2fubbf1e2f826a97b98@mail.gmail.com","threadId":"16749","inReplyTo":null,"subject":"Git Notes idea.","fromName":"Govind Salinas","fromEmail":"govind@sophiasuchtig.com","sentAt":"2008-12-16T08:15:47Z","receivedAt":"2008-12-16T08:15:47Z","isPatch":false,"sender":{"key":"govind@sophiasuchtig.com","avatar":null},"body":"Hi All,\n\nI was thinking about possible ideas for my little pet project and I\nhad and idea for way to tack on notes to a commit, or any object\nreally. I know that the idea has been flying around for a long time\nbut there has never been any implementation or a concept that people\nliked enough to use (unless I have missed something).\n\nHere is my idea.\n\n.git/refs/notes contains a tree-id (assuming that using a tree-id\nwill not cause any problems, otherwise a commit object can be used.\nit does not *need* a history, but it *could* have one).\n\nThat tree has a structure similar to the layout of .git/objects, where\nit is 2 letter subdirectories for the notes objects.\n\nGiven a git object (commit, tree, blob, tag), use its sha as the\npath/filename in this tree.\n If I have a commit 1234567890123456789012345678901234567890 then\nthe notes tree will have a file\n12/34567890123456789012345678901234567890\n\nThat file has a list of sha1s (one per line). These shas are object\nIDs for blobs that have the notes or whatever that you want attached\nto the item.\n\nI think you get the idea. When looking up an item, it should be\nfairly easy to have the notes tree and subtrees available for doing\nlookups. And as far as I know stuff under .git/refs can be\npushed/pulled even if its not under heads or remotes or tags using\nalready existing machinery. I am not sure, but I think that would\nsatisfy gc operations as well. Also, these trees and blobs never have\nto be put in the working directory.\n\nDoes this sound like something that is workable? I thought it might\nappeal since it uses only features that are already present.\n\nThis could be extended so that you have different sets of notes under\n.git/refs/notes/ or whatever. So that you can have some\nnotes you keep private and some that you publish or whatever.\n\nOK, hopefully this isn't a off the wall,\nthats-what-you-get-for-being-up-at-2-AM idea.\n\nThanks,\nGovind.\n"},{"id":"98052","messageId":"20081216085108.GA3031@coredump.intra.peff.net","threadId":"16749","inReplyTo":"5d46db230812160015t55b4ff2fubbf1e2f826a97b98@mail.gmail.com","subject":"Re: Git Notes idea.","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2008-12-16T08:51:08Z","receivedAt":"2008-12-16T08:51:08Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Dec 16, 2008 at 02:15:47AM -0600, Govind Salinas wrote:\n\n> I was thinking about possible ideas for my little pet project and I\n> had and idea for way to tack on notes to a commit, or any object\n> really. I know that the idea has been flying around for a long time\n> but there has never been any implementation or a concept that people\n> liked enough to use (unless I have missed something).\n\nI think you look at the previous suggestions, because yours is very\nsimilar. Which is good, I think, because the current status is that the\ndesign is good, but nobody has gotten around to working on it yet. So\nmaybe you can fix that. :)\n\n> .git/refs/notes contains a tree-id (assuming that using a tree-id\n> will not cause any problems, otherwise a commit object can be used.\n> it does not *need* a history, but it *could* have one).\n\nThat is the same as the current proposal, except:\n\n - the proposal is to use a commit, so your notes are version-controlled\n\n - I have suggested supporting multiple note \"bases\" in the refs/notes\n namespace. This would allow you to share some notes but not others\n (e.g., if you had some automated notes related to a build/test\n system, you might not want to mix those with your human-written\n notes).\n\n> That tree has a structure similar to the layout of .git/objects, where\n> it is 2 letter subdirectories for the notes objects.\n\nI don't think this has been suggested yet, but I'm not sure it is a good\nidea. The usual reason for this split is that many filesystems handle\nlarge directories badly; that isn't a problem here.\n\nIt does reduce the size of the resulting tree objects when a note is\nmodified (we make updates to two smaller trees instead of one big tree).\nI don't know if this really matters all that much, since the trees\nwill end up deltified in a pack anyway.\n\nAnd it does make the implementation slightly less simple, since we have\nto deal with two levels of trees.\n\n> Given a git object (commit, tree, blob, tag), use its sha as the\n> path/filename in this tree.\n> If I have a commit 1234567890123456789012345678901234567890 then\n> the notes tree will have a file\n> 12/34567890123456789012345678901234567890\n> \n> That file has a list of sha1s (one per line). These shas are object\n> IDs for blobs that have the notes or whatever that you want attached\n> to the item.\n\nThis is slightly different than the current proposal. You are proposing\nthat each item have a \"list of notes\". My thinking was to have \"named\nnotes\" using a tree for each entry full of blobs. So you could look up\nthe \"foo\" note for a given commit, but that note would be a single\nscalar (which could, of course, be interpreted according to its name).\n\n> I think you get the idea. When looking up an item, it should be\n> fairly easy to have the notes tree and subtrees available for doing\n> lookups. And as far as I know stuff under .git/refs can be\n\nIt is easy, but it's slow because we have to do a linear search in the\n(potentially huge) notes tree. And that's what held up the initial\nimplementation. I did a proof-of-concept a month or so ago that\npre-seeded an in-memory hash using the tree contents and got pretty\nreasonable performance.\n\n> pushed/pulled even if its not under heads or remotes or tags using\n> already existing machinery. I am not sure, but I think that would\n> satisfy gc operations as well. Also, these trees and blobs never have\n> to be put in the working directory.\n\nRight, though I think one of the benefits of this approach is that you\n_could_ do a checkout on the notes tree if you wanted to do very\nflexible editing.\n\n> Does this sound like something that is workable? I thought it might\n> appeal since it uses only features that are already present.\n\nYes, it sounds workable, though if you diverge from what has already\nbeen discussed, I think you should make an argument about why your\napproach is better.\n\n> This could be extended so that you have different sets of notes under\n> .git/refs/notes/ or whatever. So that you can have some\n> notes you keep private and some that you publish or whatever.\n\nOops, I should have read your whole mail. Yes, that is a good idea. :)\n\nFor reference, here are the previous discussions that I think are\nrelevant:\n\n Johan Herland's original notes proposal (which I think is largely\n dead, replaced by the one below):\n http://thread.gmane.org/gmane.comp.version-control.git/46770\n\n Johannes Schindelin's notes proposal (which is more or less the\n current proposal, but I think the on-disk notes index was not\n well liked):\n http://thread.gmane.org/gmane.comp.version-control.git/52598\n\n My test with using a hash to speed it up:\n http://article.gmane.org/gmane.comp.version-control.git/99415\n\n Some discussion of the interaction of notes and rebase:\n http://thread.gmane.org/gmane.comp.version-control.git/100533\n\n Some thoughts from me on naming issues:\n http://article.gmane.org/gmane.comp.version-control.git/100402\n\n Some thoughts from me on the tree speedup:\n http://article.gmane.org/gmane.comp.version-control.git/101460\n\nwhich I think should bring you up to speed. :)\n\n-Peff\n"},{"id":"98054","messageId":"20081216085322.GC3031@coredump.intra.peff.net","threadId":"16749","inReplyTo":"20081216085108.GA3031@coredump.intra.peff.net","subject":"Re: Git Notes idea.","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2008-12-16T08:53:22Z","receivedAt":"2008-12-16T08:53:22Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Dec 16, 2008 at 03:51:08AM -0500, Jeff King wrote:\n\n> I think you look at the previous suggestions, because yours is very\n\nSorry, there is a typo there. I meant \"I think you _should_ look at the\nprevious suggestions.\" Not saying in broken English that you already\nhave looked at them.\n\n-Peff\n"},{"id":"98080","messageId":"5d46db230812161043m4a5873a8w4c323d634b639ba0@mail.gmail.com","threadId":"16749","inReplyTo":"20081216085108.GA3031@coredump.intra.peff.net","subject":"Re: Git Notes idea.","fromName":"Govind Salinas","fromEmail":"govind@sophiasuchtig.com","sentAt":"2008-12-16T18:43:55Z","receivedAt":"2008-12-16T18:43:55Z","isPatch":false,"sender":{"key":"govind@sophiasuchtig.com","avatar":null},"body":"On Tue, Dec 16, 2008 at 2:51 AM, Jeff King wrote:\n> On Tue, Dec 16, 2008 at 02:15:47AM -0600, Govind Salinas wrote:\n>\n>> I was thinking about possible ideas for my little pet project and I\n>> had and idea for way to tack on notes to a commit, or any object\n>> really. I know that the idea has been flying around for a long time\n>> but there has never been any implementation or a concept that people\n>> liked enough to use (unless I have missed something).\n>\n> I think you look at the previous suggestions, because yours is very\n> similar. Which is good, I think, because the current status is that the\n> design is good, but nobody has gotten around to working on it yet. So\n> maybe you can fix that. :)\n>\n\nI was thinking I would do my first implementation in pyrite and if I find\nthat it works well I will port it.\n\n>> .git/refs/notes contains a tree-id (assuming that using a tree-id\n>> will not cause any problems, otherwise a commit object can be used.\n>> it does not *need* a history, but it *could* have one).\n>\n> That is the same as the current proposal, except:\n>\n> - the proposal is to use a commit, so your notes are version-controlled\n>\n> - I have suggested supporting multiple note \"bases\" in the refs/notes\n> namespace. This would allow you to share some notes but not others\n> (e.g., if you had some automated notes related to a build/test\n> system, you might not want to mix those with your human-written\n> notes).\n>\n>> That tree has a structure similar to the layout of .git/objects, where\n>> it is 2 letter subdirectories for the notes objects.\n>\n> I don't think this has been suggested yet, but I'm not sure it is a good\n> idea. The usual reason for this split is that many filesystems handle\n> large directories badly; that isn't a problem here.\n>\n\nI just read the proposal from Johannes, he seems to want to use a\nsimilar layout. However, I would like to modify my proposal slightly\nto make it work better when a gc is run. I would modify the tree to\nlook like this...\n\nlet 1234567890123456789012345678901234567890 be the\nid of the item that is annotated.\n\nlet abcdef7890123456789012345678901234567890 be the\nid of the note to be attached\n\nroot/\n 12/\n 34567890123456789012345678901234567890/\n abcdef7890123456789012345678901234567890\n\nThis way all the notes are attached to a tree, so that gc won't\nthink they are unreferenced objects.\n\n> It does reduce the size of the resulting tree objects when a note is\n> modified (we make updates to two smaller trees instead of one big tree).\n> I don't know if this really matters all that much, since the trees\n> will end up deltified in a pack anyway.\n>\n> And it does make the implementation slightly less simple, since we have\n> to deal with two levels of trees.\n>\n>> Given a git object (commit, tree, blob, tag), use its sha as the\n>> path/filename in this tree.\n>> If I have a commit 1234567890123456789012345678901234567890 then\n>> the notes tree will have a file\n>> 12/34567890123456789012345678901234567890\n>>\n>> That file has a list of sha1s (one per line). These shas are object\n>> IDs for blobs that have the notes or whatever that you want attached\n>> to the item.\n>\n> This is slightly different than the current proposal. You are proposing\n> that each item have a \"list of notes\". My thinking was to have \"named\n> notes\" using a tree for each entry full of blobs. So you could look up\n> the \"foo\" note for a given commit, but that note would be a single\n> scalar (which could, of course, be interpreted according to its name).\n>\n\n\n>> I think you get the idea. When looking up an item, it should be\n>> fairly easy to have the notes tree and subtrees available for doing\n>> lookups. And as far as I know stuff under .git/refs can be\n>\n> It is easy, but it's slow because we have to do a linear search in the\n> (potentially huge) notes tree. And that's what held up the initial\n> implementation. I did a proof-of-concept a month or so ago that\n> pre-seeded an in-memory hash using the tree contents and got pretty\n> reasonable performance.\n>\n\nPerhaps I am missing something, how is it a linear search?. Since we\nare keying off of the sha of the annotated object, using a hashtable for\na cache should be a fairly quick binary search. If you just wanted to\nuse the tree objects, that should work almost as well since the first tree\nwill split them up nicely for you.\n\nAlso, how large do you expect the list to be under reasonable\ncircumstances.\n\n>> pushed/pulled even if its not under heads or remotes or tags using\n>> already existing machinery. I am not sure, but I think that would\n>> satisfy gc operations as well. Also, these trees and blobs never have\n>> to be put in the working directory.\n>\n> Right, though I think one of the benefits of this approach is that you\n> _could_ do a checkout on the notes tree if you wanted to do very\n> flexible editing.\n>\n\nSure, why not.\n\n>> Does this sound like something that is workable? I thought it might\n>> appeal since it uses only features that are already present.\n>\n> Yes, it sounds workable, though if you diverge from what has already\n> been discussed, I think you should make an argument about why your\n> approach is better.\n>\n\nWell after reading Johannes proposal, I find it to be surprisingly similar\nsince I had not seen it before. However I think mine is a win in a few ways.\nOne, it allows multiple notes per object. Two, it plays well with gc. From\nwhat I could follow, the files in his layout just have a ref to the note object\nbut gc would be required to know about this feature and not remove those\nnote blobs. Third it allows multiple sets of notes. Although it seems that\nat least 1 and 3 have been discussed at some length.\n\n>> This could be extended so that you have different sets of notes under\n>> .git/refs/notes/ or whatever. So that you can have some\n>> notes you keep private and some that you publish or whatever.\n>\n> Oops, I should have read your whole mail. Yes, that is a good idea. :)\n>\n> For reference, here are the previous discussions that I think are\n> relevant:\n>\n\nThanks for the pointers, a couple quick thoughts...\n\n> Some discussion of the interaction of notes and rebase:\n> http://thread.gmane.org/gmane.comp.version-control.git/100533\n>\n\nOn rebasing, I have a couple thoughts. 1) I think it only really makes\nsense to make a public annotation to a commit that is public, and\nonce a commit is public it should not be rebased. 2) We could also\nannotate the commit's tree instead of the commit. That would make\nit somewhat resistant to rebases, cherry-picks and amends. And\nonce a tree has changed, the notes are probably less reliable\nalthough the user should be able to force a note or notes to be\ncarried along.\n\n> Some thoughts from me on naming issues:\n> http://article.gmane.org/gmane.comp.version-control.git/100402\n>\n\nOn naming. I strongly support a ref/notes/sha1/sha1 approach. If\nhaving a type to the note is important, then perhaps the first line of\na note could be considered a type or a set of \"tags\". This way you\nhave both naming/typing and one lookup per sha. The only\ndrawback is that you have to open the blob to see the type. A hybrid\napproach that uses refs/notes/acked/sha/sha which is one lookup if\nyou know the type and the sha of the annotated object before hand\nmight be worth considering. This would be similar to the public or\nprivate notes that i mentioned before.\n\n> Some thoughts from me on the tree speedup:\n> http://article.gmane.org/gmane.comp.version-control.git/101460\n>\n\nI guess I must be missing something. I have seen several references\nto this not being a binary search several times in the links that you have\nhere. But I fail to see why a binary search cannot be done. That said, I\nwould still think that the existing hash table would be the way to go.\n\n> which I think should bring you up to speed. :)\n\nThanks again.\n\n-Govind\n"},{"id":"98089","messageId":"alpine.DEB.1.00.0812170003540.14632@racer","threadId":"16749","inReplyTo":"5d46db230812161043m4a5873a8w4c323d634b639ba0@mail.gmail.com","subject":"Re: Git Notes idea.","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-12-16T23:48:02Z","receivedAt":"2008-12-16T23:48:02Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Tue, 16 Dec 2008, Govind Salinas wrote:\n\n> I was thinking I would do my first implementation in pyrite and if I \n> find that it works well I will port it.\n\nGiven that there are a lot of building blocks in C already, I think it \nwould be a waste of your time.\n\n> I just read the proposal from Johannes, he seems to want to use a \n> similar layout. However, I would like to modify my proposal slightly to \n> make it work better when a gc is run. I would modify the tree to look \n> like this...\n> \n> let 1234567890123456789012345678901234567890 be the\n> id of the item that is annotated.\n> \n> let abcdef7890123456789012345678901234567890 be the\n> id of the note to be attached\n> \n> root/\n> 12/\n> 34567890123456789012345678901234567890/\n> abcdef7890123456789012345678901234567890\n> \n> This way all the notes are attached to a tree, so that gc won't\n> think they are unreferenced objects.\n\nIn my proposal back then, your root/12/345... would be a blob, in Peff's, \nit would be a tree, and in both cases the blobs/trees would be referenced \nby refs/notes, so git gc would not kill them either.\n\nThe bigger issue is that commit objects can be gc'ed, and then their notes \nshould be gc'ed, too.\n\nAnd of course, there is the rebase issue (which I completely missed; I \nwill read the mail Jeff referenced tomorrow).\n\nSpeaking about the blobs vs trees issue, I think it is no issue, as Peff \nand me already discussed: the notes could check if it is a tree or a \nblob, and handle both easily.\n\n> Peff wrote:\n>\n> > It is easy, but it's slow because we have to do a linear search in the \n> > (potentially huge) notes tree. And that's what held up the initial \n> > implementation. I did a proof-of-concept a month or so ago that \n> > pre-seeded an in-memory hash using the tree contents and got pretty \n> > reasonable performance.\n> \n> Perhaps I am missing something, how is it a linear search?\n\nYes, you are missing what I wrote in the original thread: tree objects \nmust be read in a forward direction, one by one.\n\nIIRC back then, Junio and/or Linus suggested that you could backward \nsearch with heuristics finding the beginning of a tree entry, and thus you \ncould kind of bisect the tree to search for a specific tree entry (since \ntree objects have the contents sorted), but I presented a case where this \nbreaks down at the GitTogether:\n\nTree entries consist of a mode (as 6-byte ASCII representing the octal \nvalue), then SPC, then a NUL-terminated path, and then a 20-byte SHA-1. \n(Just hexdump the output of \"git cat-file tree HEAD:\" in any repository to \nsee it).\n\nThe only heuristic you could apply to find your way backward (or forward) \nto find the beginning of a tree entry would be to find the NUL character \nand verify that exactly 20 bytes after it, either the tree object ends, or \nthere is a valid octal number followed by a SPC.\n\nThe only thing you would need for this heuristic to break down is a SHA-1 \nwhich contains a \\x00 (which is then mistaken for a string termination), \nand part of a filename that could be mistaken to be an octal number.\n\nTake for example the SHA-1 of git-gui in v1.6.0.5, which has a NUL its \n14th byte, and just assume that the next tree entry has mode 100644 \nand name \"4040000 Some financial record.txt\".\n\nThen, the NUL could be mistaken for the end of the previous tree entry, \nand \"040000 \" as the mode for the next one, whose name would be assumed to \nbe \"Some financial record.txt\".\n\nGranted, if you find two NULs in 21 bytes, one of them must be the \ntermination of the path, but which one?\n\nWorse, even if you would find a method (complicated, and therefore \nnecessarily fragile) to find the boundary of the tree entries reliably, \nyou would _still_ have a linear time unpacking the darned tree object in \nthe first place.\n\nSo you cannot do that for every commit you encounter and expect not to die \nof boredom in the process.\n\nPeff's very cute idea was to decouple that process from the per-commit \nprocedure, and basically make it a one-time cost (per Git call, and only \nwhen notes were asked for).\n\nIt will still be linear in the number of notes, but it would then be in a \nhashmap, with an expected linear cost per commit.\n\n> Also, how large do you expect the list to be under reasonable \n> circumstances.\n\nWe did not intend Git to be used as a backup tool, did we?\n\nOne of the _worst_ design decisions is to limit yourself by expectations.\n\n> > Some discussion of the interaction of notes and rebase:\n> > http://thread.gmane.org/gmane.comp.version-control.git/100533\n> \n> On rebasing, I have a couple thoughts.\n>\n> 1) I think it only really makes sense to make a public annotation to a \n> commit that is public, and once a commit is public it should not be \n> rebased.\n\nAgain, do not limit your design by your expectations. People already talk \nabout having cover letters for their patch series as notes, and Pasky \nseems to discuss tracking explicit renames with notes when he does not \nplay Go, instead of maintaining repo.or.cz and git.or.cz.\n\n> 2) We could also annotate the commit's tree instead of the commit. \n> That would make it somewhat resistant to rebases, cherry-picks and \n> amends.\n\nTo the contrary. When I rebase, the tree _does_ change, otherwise I would \nhave rebased onto something that had the same original tree as my \nrebase-base to begin with, which would make the rebase rather pointless.\n\n> > Some thoughts from me on naming issues: \n> > http://article.gmane.org/gmane.comp.version-control.git/100402\n> \n> On naming. I strongly support a ref/notes/sha1/sha1 approach.\n\nI think you meant refs/notes://?\n\nI am rather supporting refs/nots:/ being \neither a blob, or a tree containing human readable tags, such as \"bugfix\" \nor \"review\" or some such.\n\n> If having a type to the note is important, then perhaps the first line \n> of a note could be considered a type or a set of \"tags\".\n\nThat would be horrible! Just to know if you need to unpack the blob, \nyou'd have to unpack it!\n\nHth,\nDscho\n"},{"id":"98093","messageId":"alpine.DEB.1.00.0812170110160.14632@racer","threadId":"16749","inReplyTo":"20081216085108.GA3031@coredump.intra.peff.net","subject":"rebasing commits that have notes, was Re: Git Notes idea.","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-12-17T00:12:25Z","receivedAt":"2008-12-17T00:12:25Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Tue, 16 Dec 2008, Jeff King wrote:\n\n> Some discussion of the interaction of notes and rebase:\n> http://thread.gmane.org/gmane.comp.version-control.git/100533\n\nOh, I misinterpreted that label... of course you can track rebases in \nnotes, but some issue that we did not look into yet (I think) is the issue \nthat you can cherry-pick and rebase commits and lose notes in the process.\n\nIt seems that the notes idea is not that unintrusive as I thought...\n\nCiao,\nDscho\n"},{"id":"98119","messageId":"200812171015.45303.johan@herland.net","threadId":"16749","inReplyTo":"alpine.DEB.1.00.0812170110160.14632@racer","subject":"Re: rebasing commits that have notes, was Re: Git Notes idea.","fromName":"Johan Herland","fromEmail":"johan@herland.net","sentAt":"2008-12-17T09:15:45Z","receivedAt":"2008-12-17T09:15:45Z","isPatch":false,"sender":{"key":"johan@herland.net","avatar":"https://avatars.githubusercontent.com/u/547031?v=4"},"body":"On Wednesday 17 December 2008, Johannes Schindelin wrote:\n> Hi,\n>\n> On Tue, 16 Dec 2008, Jeff King wrote:\n> > Some discussion of the interaction of notes and rebase:\n> > http://thread.gmane.org/gmane.comp.version-control.git/100533\n>\n> Oh, I misinterpreted that label... of course you can track rebases in\n> notes, but some issue that we did not look into yet (I think) is the\n> issue that you can cherry-pick and rebase commits and lose notes in the\n> process.\n>\n> It seems that the notes idea is not that unintrusive as I thought...\n\nSo we have two issues here:\n\n1. Using notes to annotate the rebase/cherry-pick action itself.\n\n2. Preserving (or at least giving the user the option of preserving) notes \nacross a rebase/cherry-pick.\n\n\nI think issue #1 has already been discussed, and is largely resolved: People \ncan do this if they want to; it probably only makes sense when \nrebasing/cherry-picking public branches; etc... AFAICS there are no \nremaining problems here that needs an intrusive solution (see below for one \nsuch unintrusive alternative).\n\nIssue #2, however, is a little more involved. We can discuss the merits of \nwanting to preserve notes across a rebase/cherry-pick itself; e.g. when it \nmakes sense to preserve notes, and when it doesn't make sense, but I think \nthis is orthogonal to the issue of HOW to preserve them, so instead of \nfocusing on WHY, I'll focus on HOW:\n\nIf notes are named according to the \"refs/notes://\" scheme (and AFAICS this is still being \ndiscussed, so it's indeed a big IF), then rebase/cherry-pick of the \nreferenced object simply translates to a rename/copy of the corresponding \nnote (this is of course assuming that the note itself does not contain the \nSHA-1 of the referenced object). This could probably be solved fairly \nunintrusively in the current code, but there are (as always) complications:\n\n- The user may want to amend the note after the rebase/cherry-pick (just as \n(s)he may want to amend the commit message).\n\n- In some cases it may even make sense to fold (parts of) the note _into_ \nthe commit message.\n\n- probably more reasons...\n\nSo what about the following proposal: Add hooks that are invoked by \nrebase/cherry-pick with the and as arguments. A \ntypical hook script can then use this information to look up notes \nreferencing and update these to reference instead, \nand in the process, prompt the user to do whatever changes (s)he wants to. \nThe hook scripts can do other things as well, e.g. implementing issue #1 \nabove (adding notes for annotating the rebase/cherry-pick itself.)\n\n\nHave fun! :)\n\n...Johan\n\n\nPS: What's the current status on git-sequencer? It's probably the best place \nto invoke these hooks.\n\n-- \nJohan Herland, \nwww.herland.net\n"},{"id":"98120","messageId":"20081217093843.GA18265@coredump.intra.peff.net","threadId":"16749","inReplyTo":"5d46db230812161043m4a5873a8w4c323d634b639ba0@mail.gmail.com","subject":"Re: Git Notes idea.","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2008-12-17T09:38:44Z","receivedAt":"2008-12-17T09:38:44Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Dec 16, 2008 at 12:43:55PM -0600, Govind Salinas wrote:\n\n> I was thinking I would do my first implementation in pyrite and if I find\n> that it works well I will port it.\n\nOK, though your performance will probably suck unless you dump the notes\ntree into a local hash at the beginning of your program. And looking up\nevery commit's note during revision traversal is one of the intended\nuses (e.g., decorating git-log output, or filtering commits based on a\nparticular note).\n\nAnd as Dscho mentioned, most of what you need is already there in C.\nYou are welcome to implement whatever you want in pyrite, of course, but\nthere is a desire to have this accessible to the revision traversal\nmachinery. And that means if you want your version in pyrite to be\ncompatible with what ends up in git, the data structure design needs to\nbe suitable for both.\n\n> I just read the proposal from Johannes, he seems to want to use a\n> similar layout. However, I would like to modify my proposal slightly\n> to make it work better when a gc is run. I would modify the tree to\n> look like this...\n> \n> let 1234567890123456789012345678901234567890 be the\n> id of the item that is annotated.\n> \n> let abcdef7890123456789012345678901234567890 be the\n> id of the note to be attached\n> \n> root/\n> 12/\n> 34567890123456789012345678901234567890/\n> abcdef7890123456789012345678901234567890\n> \n> This way all the notes are attached to a tree, so that gc won't\n> think they are unreferenced objects.\n\nBut you have lost the ordering in your list, then, since they will not\nbe ordered by sha1 of the note contents. I don't know if you care. The\nsecond sha1 is pointless, anyway, since nobody will know that number as\na reference; why not just name them monotonically starting at 1?\n\nOne of the things I don't like about having several notes is that it\nintroduces an extra level of indirection that every user has to pay for,\nwhether they want it or not. If a note can be a blob _or_ a tree, then\nthose who want to use blobs can reap the performance benefit. Those who\nwant multiple named notes in a hierarchy can pay the extra indirection\ncost.\n\nI haven't measured how big a cost that is (but bearing in mind that we\nmight want to do this lookup once per revision in a traversal, even one\nextra object lookup can have an impact).\n\nI'm also still not convinced the fan-out is worthwhile, but I can see\nhow it might be. It would be nice to see numbers for both.\n\n> Perhaps I am missing something, how is it a linear search?. Since we\n\nI think Johannes explained in detail in another message, but it is a\nlinear search to look up directly in a tree object. Of course you can\nbuild a hash or a sorted fixed-size list as an index.\n\n> Also, how large do you expect the list to be under reasonable\n> circumstances.\n\nAs many notes as there are commits is my goal (e.g., it is not hard to\nimagine an automated process to add notes on build status). Ideally, we\ncould handle as many notes as there are objects; I see no reason not to\nallow annotating arbitrary sha1's (I don't know if there is a use for\nthat, but the more scalable the implementation, the better).\n\n> > Some thoughts from me on naming issues:\n> > http://article.gmane.org/gmane.comp.version-control.git/100402\n> \n> On naming. I strongly support a ref/notes/sha1/sha1 approach. If\n> having a type to the note is important, then perhaps the first line of\n> a note could be considered a type or a set of \"tags\". This way you\n\nI don't think we are talking about the same thing. What I mean by naming\nis \"here is a shorthand for referring to notes\" that is not necessarily\ncoupled with the implementation. That is, I would like to do something\nlike:\n\n git log --notes-filter=\"foo:bar == 1\"\n\nand have that \"foo:bar\" as a shorthand on each commit for:\n\n refs/notes/foo:$COMMIT/bar\n\nWithout a left-hand side (e.g., \"bar\"), we get:\n\n refs/notes/default:$COMMIT/bar\n\nOr without a right-hand side (e.g., \"foo:\"), we get:\n\n refs/notes/foo:$COMMIT\n\nSo you can group related notes in the same tree (which gives you fast\nlookup if you are looking at multiple ones, since you only have to do\nthe tree lookup once), or you can keep notes in separate trees (which\nmeans you can distribute some but not others).\n\nI think your \"list of notes\" proposal on top of that would be for any\nnote resolution to provide a tree instead of a blob, with sequenced\nelements. I.e., foo:bar might have multiple notes, like:\n\n refs/notes/foo:$COMMIT/bar/1\n refs/notes/foo:$COMMIT/bar/2\n refs/notes/foo:$COMMIT/bar/3\n\n> drawback is that you have to open the blob to see the type. A hybrid\n> approach that uses refs/notes/acked/sha/sha which is one lookup if\n\nRight, that is possible, but implementing only half of what I suggested\nabove. I think you should be flexible enough to have grouped notes for\nfast lookup, or ungrouped notes for more flexibility.\n\n-Peff\n"},{"id":"98121","messageId":"20081217094528.GA18492@coredump.intra.peff.net","threadId":"16749","inReplyTo":"alpine.DEB.1.00.0812170003540.14632@racer","subject":"Re: Git Notes idea.","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2008-12-17T09:45:29Z","receivedAt":"2008-12-17T09:45:29Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Dec 17, 2008 at 12:48:02AM +0100, Johannes Schindelin wrote:\n\n> > Perhaps I am missing something, how is it a linear search?\n> \n> Yes, you are missing what I wrote in the original thread: tree objects \n> must be read in a forward direction, one by one.\n\nThanks, that was a nice writeup of what was discussed at the\nGitTogether.\n\n> Peff's very cute idea was to decouple that process from the per-commit \n> procedure, and basically make it a one-time cost (per Git call, and only \n> when notes were asked for).\n\nTo be fair, it was not my cute idea, but somebody else's (I think David\nReiss). I just coded it quickly because somebody was talking about\niPhones or something. :)\n\n> To the contrary. When I rebase, the tree _does_ change, otherwise I\n> would have rebased onto something that had the same original tree as\n> my rebase-base to begin with, which would make the rebase rather\n> pointless.\n\nAnother fun option would be to put notes on patch-ids. That would of\ncourse be horrifically slow to look up, but would survive many\ncherry-picks and rebases (but not all, of course).\n\nI don't know if that is useful or not, but I don't see any reason why\nthe discussed implementation would forbid it (somebody would just have\nto implement the lookups at a useful spot).\n\n> > On naming. I strongly support a ref/notes/sha1/sha1 approach.\n> I think you meant refs/notes:// arbitrary SHA-1>?\n\nYou seem to be in favor of the fan-out. Out of curiosity, did you ever\ndo numbers on whether the fan-out is actually helpful?\n\n> I am rather supporting refs/nots:/\n> being either a blob, or a tree containing human readable tags, such as\n> \"bugfix\" or \"review\" or some such.\n\nYes, I am in favor of the \"tree or blob\" idea, too. I want this to not\njust be about \"I am a person writing a note\" but \"arbitrary data\nattached to an object after it has been created\". And that means\nthinking up front about managing the namespace.\n\n-Peff\n"},{"id":"98122","messageId":"20081217101110.GC18265@coredump.intra.peff.net","threadId":"16749","inReplyTo":"alpine.DEB.1.00.0812170420560.14632@racer","subject":"Re: Git Notes idea.","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2008-12-17T10:11:10Z","receivedAt":"2008-12-17T10:11:10Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Dec 17, 2008 at 04:43:57AM +0100, Johannes Schindelin wrote:\n\n> > I agree, I haven't thought of any fix along these lines other than to \n> > make gc do the clean up.\n> \n> I have, and IIRC I briefly mentioned it back then. Basically, you will \n> have to add a \"git notes gc\" or some such, which basically reads in the \n> whole notes, traverses all reachable commits, marking the corresponding \n> notes, and then writes out all marked notes (leaving the other notes \n> behind).\n\nI was thinking something similar, but I think it is even easier. Make\nthe rule \"if we still have the object, then we still have the note\".\nThat has three benefits:\n\n - implementation is simple: for each note $n, delete it unless\n has_sha1_file($n).\n\n - it handles notes on non-commit objects\n\n - it kills off notes when an object is _pruned_, not when it stops\n being _reachable_. So if I delete a branch with a commit found\n nowhere else, its notes will hang around until it is actually pruned.\n If I pull it from lost+found, I still keep the notes.\n\nNote that all of this garbage collection of notes is really just\nremoving them from the most current notes _tree_. If the notes structure\nis actually composed of commits, then old notes that are \"deleted\" will\nstill be available historically.\n\n> I wonder why you speak as if none of that had been implemented yet. From \n> my work, it is obvious that hashtable is better than sorted list (except \n> for the fan-out, which obviously wants to be an array), and from Peff's it \n> is obvious that we want to keep the hashtables in memory.\n\nIf he is planning on doing a separate pyrite implementation, then it\n_hasn't_ been implemented yet. And I don't care there if he uses\nhash tables or sorted lists or whatever. I think the most important\nthing is getting down the design of the _data structure_, so that we can\nhave a compatible implementation inside git itself.\n\n> > IMO notes are just a generallized tag.\n> \n> IMO notes have nothing to do with a tag. Tags point to commits (or other \n> objects, but that's beside the point here). Notes are pointed _to_ by \n> commits.\n\nI think maybe we are just talking about semantics, but I think notes are\nnot pointed to by commits. There is an external mapping of commits to\nnotes, which is very different. I can give you the commit without you\nknowing the notes, or that the notes even exist.\n\nBut in practice, I don't know if this distinction is going to influence\nany of the design or use.\n\n> Has the tree changed? Sure it has. Because Junio committed and pushed \n> some changes.\n\nI think it is safe to say the tree generally changes for rebase, but not\nnecessarily for something like an amended commit, or a pull of a patch\nsent by mail. So there are times when it changes, and times when it\ndoesn't.\n\nAnd if there were some simple way of handling the times when it didn't\nchange at no general cost, I think going that way would be fine. But:\n\n> And the worst part about your idea to attach notes to trees rather than \n> commits: For things like Acked-by:... you very much want to annotate the \n> commit, _not_ the tree. The tree is useless here. It says nothing about \n> the patch, nothing about the explanation, and nothing about the history.\n\nThis is a huge cost, IMHO. I think you generally want to annotate\ncommits, not trees, and the semantic difference is important (but again,\nI think all of the proposals are capable of doing either -- but if you\nwant a \"show me the notes on this commit\" feature to interoperate, it\nneeds to pick one).\n\n> > root/\n> > 12/\n> > 34567890123456789012345678901234567890/\n> > \n> \n> Funny. That is Peff's proposal.\n\nClearly we have independent verification that it's a good idea. ;)\n\n-Peff\n"},{"id":"98123","messageId":"alpine.DEB.1.00.0812171233270.28560@intel-tinevez-2-302","threadId":"16749","inReplyTo":"20081217101110.GC18265@coredump.intra.peff.net","subject":"Re: Git Notes idea.","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-12-17T11:38:53Z","receivedAt":"2008-12-17T11:38:53Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Wed, 17 Dec 2008, Jeff King wrote:\n\n> On Wed, Dec 17, 2008 at 04:43:57AM +0100, Johannes Schindelin wrote:\n> \n> > > I agree, I haven't thought of any fix along these lines other than \n> > > to make gc do the clean up.\n> > \n> > I have, and IIRC I briefly mentioned it back then. Basically, you \n> > will have to add a \"git notes gc\" or some such, which basically reads \n> > in the whole notes, traverses all reachable commits, marking the \n> > corresponding notes, and then writes out all marked notes (leaving the \n> > other notes behind).\n> \n> I was thinking something similar, but I think it is even easier. Make\n> the rule \"if we still have the object, then we still have the note\".\n> That has three benefits:\n> \n> - implementation is simple: for each note $n, delete it unless\n> has_sha1_file($n).\n> \n> - it handles notes on non-commit objects\n> \n> - it kills off notes when an object is _pruned_, not when it stops\n> being _reachable_. So if I delete a branch with a commit found\n> nowhere else, its notes will hang around until it is actually pruned.\n> If I pull it from lost+found, I still keep the notes.\n> \n> Note that all of this garbage collection of notes is really just \n> removing them from the most current notes _tree_. If the notes structure \n> is actually composed of commits, then old notes that are \"deleted\" will \n> still be available historically.\n\nRight. So my original proposal to use separate refs for separate purposes \nmight make sense again, so you can have private as well as public notes.\n\n> > I wonder why you speak as if none of that had been implemented yet. \n> > From my work, it is obvious that hashtable is better than sorted list \n> > (except for the fan-out, which obviously wants to be an array), and \n> > from Peff's it is obvious that we want to keep the hashtables in \n> > memory.\n> \n> If he is planning on doing a separate pyrite implementation, then it \n> _hasn't_ been implemented yet. And I don't care there if he uses hash \n> tables or sorted lists or whatever. I think the most important thing is \n> getting down the design of the _data structure_, so that we can have a \n> compatible implementation inside git itself.\n\nWell, I don't care about pyrite. As far as I am concerned, it might as \nwell use an incompatible version. I really don't care.\n\n> > > IMO notes are just a generallized tag.\n> > \n> > IMO notes have nothing to do with a tag. Tags point to commits (or \n> > other objects, but that's beside the point here). Notes are pointed \n> > _to_ by commits.\n> \n> I think maybe we are just talking about semantics, but I think notes are\n> not pointed to by commits. There is an external mapping of commits to\n> notes, which is very different. I can give you the commit without you\n> knowing the notes, or that the notes even exist.\n> \n> But in practice, I don't know if this distinction is going to influence \n> any of the design or use.\n\nYou are correct, of course, that the commit does not point to the notes \nexplicitely, by having a SHA-1 _in_ the commit object. But the main point \nstill stands: you go from commit to note, not from note to commit. And \nthis is in stark contrast to tags, where you go from tag to commit, _not_ \nfrom commit to tag.\n\nThat is a fundamental _difference_ between tags and notes, so that I \nrefuse to accept the notion of notes being a generalized form of tags.\n\nCiao,\nDscho\n"},{"id":"98135","messageId":"20081217122158.GD3640@machine.or.cz","threadId":"16749","inReplyTo":"alpine.DEB.1.00.0812170003540.14632@racer","subject":"Re: Git Notes idea.","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2008-12-17T12:21:58Z","receivedAt":"2008-12-17T12:21:58Z","isPatch":false,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"On Wed, Dec 17, 2008 at 12:48:02AM +0100, Johannes Schindelin wrote:\n> Again, do not limit your design by your expectations. People already talk \n> about having cover letters for their patch series as notes, and Pasky \n> seems to discuss tracking explicit renames with notes when he does not \n> play Go, instead of maintaining repo.or.cz and git.or.cz.\n\nI don't really play Go that much anymore! ;-)\n\n\t\t\t\tPetr \"Pasky\" Baudis\n"},{"id":"98155","messageId":"5d46db230812170906h7fdcac03o60386504c8df1083@mail.gmail.com","threadId":"16749","inReplyTo":"20081217093843.GA18265@coredump.intra.peff.net","subject":"Re: Git Notes idea.","fromName":"Govind Salinas","fromEmail":"govind@sophiasuchtig.com","sentAt":"2008-12-17T17:06:15Z","receivedAt":"2008-12-17T17:06:15Z","isPatch":false,"sender":{"key":"govind@sophiasuchtig.com","avatar":null},"body":"On Wed, Dec 17, 2008 at 3:38 AM, Jeff King wrote:\n> On Tue, Dec 16, 2008 at 12:43:55PM -0600, Govind Salinas wrote:\n>\n>> I was thinking I would do my first implementation in pyrite and if I find\n>> that it works well I will port it.\n>\n> OK, though your performance will probably suck unless you dump the notes\n> tree into a local hash at the beginning of your program. And looking up\n> every commit's note during revision traversal is one of the intended\n> uses (e.g., decorating git-log output, or filtering commits based on a\n> particular note).\n\nYes, I was thinking that this is the natural way to do things, save that I\nwould be lazy loading the trees into a cache instead of caching them\nall up front. This is one of the reasons that I think the fan out will\nhelp.\n\n> And as Dscho mentioned, most of what you need is already there in C.\n> You are welcome to implement whatever you want in pyrite, of course, but\n> there is a desire to have this accessible to the revision traversal\n> machinery. And that means if you want your version in pyrite to be\n> compatible with what ends up in git, the data structure design needs to\n> be suitable for both.\n\nYes, I completely agree that I want it to have the same scheme as what\ngit will use. That is the reason I posted this here. Since no method\nhas been formally accepted (checked into master) I wanted to see if\nI could nudge things along. I wasn't aware that you and Dscho had\na (very similar) plan. Please, if you guys are decided on the format\nthen I can just go off and start working on it. But it sounds like there\nisn't consensus yet.\n\n\n>> root/\n>> 12/\n>> 34567890123456789012345678901234567890/\n>> abcdef7890123456789012345678901234567890\n>>\n>> This way all the notes are attached to a tree, so that gc won't\n>> think they are unreferenced objects.\n>\n> But you have lost the ordering in your list, then, since they will not\n> be ordered by sha1 of the note contents. I don't know if you care. The\n> second sha1 is pointless, anyway, since nobody will know that number as\n> a reference; why not just name them monotonically starting at 1?\n\nIn a later mail I suggested that this be the type or name of the note. Which\nI hear is similar to what you suggested.\n\n> One of the things I don't like about having several notes is that it\n> introduces an extra level of indirection that every user has to pay for,\n> whether they want it or not. If a note can be a blob _or_ a tree, then\n> those who want to use blobs can reap the performance benefit. Those who\n> want multiple named notes in a hierarchy can pay the extra indirection\n> cost.\n>\n> I haven't measured how big a cost that is (but bearing in mind that we\n> might want to do this lookup once per revision in a traversal, even one\n> extra object lookup can have an impact).\n\nThat seems reasonable.\n\n> I'm also still not convinced the fan-out is worthwhile, but I can see\n> how it might be. It would be nice to see numbers for both.\n>\n\n> Also, how large do you expect the list to be under reasonable\n>> circumstances.\n>\n> As many notes as there are commits is my goal (e.g., it is not hard to\n> imagine an automated process to add notes on build status). Ideally, we\n> could handle as many notes as there are objects; I see no reason not to\n> allow annotating arbitrary sha1's (I don't know if there is a use for\n> that, but the more scalable the implementation, the better).\n\nAh, that is in line with what I was thinking as well.\n\n>> > Some thoughts from me on naming issues:\n>> > http://article.gmane.org/gmane.comp.version-control.git/100402\n>>\n>> On naming. I strongly support a ref/notes/sha1/sha1 approach. If\n>> having a type to the note is important, then perhaps the first line of\n>> a note could be considered a type or a set of \"tags\". This way you\n>\n> I don't think we are talking about the same thing. What I mean by naming\n> is \"here is a shorthand for referring to notes\" that is not necessarily\n> coupled with the implementation. That is, I would like to do something\n> like:\n>\n> git log --notes-filter=\"foo:bar == 1\"\n>\n> and have that \"foo:bar\" as a shorthand on each commit for:\n>\n> refs/notes/foo:$COMMIT/bar\n>\n> Without a left-hand side (e.g., \"bar\"), we get:\n>\n> refs/notes/default:$COMMIT/bar\n>\n> Or without a right-hand side (e.g., \"foo:\"), we get:\n>\n> refs/notes/foo:$COMMIT\n>\n\nI like the overall plan, but I would suggest that --notes[=default] and\n--note-type=whatever would be a little friendlier and less error prone.\n\nThanks for helping me think through this.\n\n-Govind\n"},{"id":"98156","messageId":"20081217175542.GA14600@leksak.fem-net","threadId":"16749","inReplyTo":"200812171015.45303.johan@herland.net","subject":"Re: rebasing commits that have notes, was Re: Git Notes idea.","fromName":"Stephan Beyer","fromEmail":"s-beyer@gmx.net","sentAt":"2008-12-17T17:55:42Z","receivedAt":"2008-12-17T17:55:42Z","isPatch":false,"sender":{"key":"s-beyer@gmx.net","avatar":"https://avatars.githubusercontent.com/u/143889?v=4"},"body":"Hi,\n\nJohan Herland wrote:\n> PS: What's the current status on git-sequencer?\n\nUsable, but I still have a small todo list of things to add\nor fix. (And the next days some more time again. I hope it's\nenough. Sorry, I'm too lame.)\n\n> It's probably the best place to invoke these hooks.\n\nIf you want to do stuff on top of git sequencer it's currently\nthe best to fetch\n\tgit://repo.or.cz/git/sbeyer.git seq-builtin-dev\nor just to wait. :-\\\n\nRegards,\n Stephan\n\n-- \nStephan Beyer , PGP 0x6EDDD207FCC5040F\n"},{"id":"98164","messageId":"7voczaobhb.fsf@gitster.siamese.dyndns.org","threadId":"16749","inReplyTo":"alpine.DEB.1.00.0812171233270.28560@intel-tinevez-2-302","subject":"Re: Git Notes idea.","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-12-17T19:20:00Z","receivedAt":"2008-12-17T19:20:00Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin writes:\n\n> ... But the main point \n> still stands: you go from commit to note, not from note to commit. And \n> this is in stark contrast to tags, where you go from tag to commit, _not_ \n> from commit to tag.\n>\n> That is a fundamental _difference_ between tags and notes, so that I \n> refuse to accept the notion of notes being a generalized form of tags.\n\nHmm, how would you explain things like \"git describe\" (and \"name-rev\")?\n"},{"id":"98214","messageId":"alpine.DEB.1.00.0812180407580.14632@racer","threadId":"16749","inReplyTo":"7voczaobhb.fsf@gitster.siamese.dyndns.org","subject":"Re: Git Notes idea.","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-12-18T03:08:20Z","receivedAt":"2008-12-18T03:08:20Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Wed, 17 Dec 2008, Junio C Hamano wrote:\n\n> Johannes Schindelin writes:\n> \n> > ... But the main point still stands: you go from commit to note, not \n> > from note to commit. And this is in stark contrast to tags, where you \n> > go from tag to commit, _not_ from commit to tag.\n> >\n> > That is a fundamental _difference_ between tags and notes, so that I \n> > refuse to accept the notion of notes being a generalized form of tags.\n> \n> Hmm, how would you explain things like \"git describe\" (and \"name-rev\")?\n\nPrograms?\n\nCiao,\nDscho\n"},{"id":"98253","messageId":"20081218135433.GA6706@coredump.intra.peff.net","threadId":"16749","inReplyTo":"5d46db230812170906h7fdcac03o60386504c8df1083@mail.gmail.com","subject":"Re: Git Notes idea.","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2008-12-18T13:54:34Z","receivedAt":"2008-12-18T13:54:34Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Dec 17, 2008 at 11:06:15AM -0600, Govind Salinas wrote:\n\n> Yes, I was thinking that this is the natural way to do things, save that I\n> would be lazy loading the trees into a cache instead of caching them\n> all up front. This is one of the reasons that I think the fan out will\n> help.\n\nI was working under the assumption that you are going to do multiple\nnote lookups. If you are, then the fan-out isn't really going to help,\nas you're going to end up pulling in all of the subtrees anyway. It\nhelps some if you're only doing a single lookup, but I don't know if\nthat is measurable.\n\n> Yes, I completely agree that I want it to have the same scheme as what\n> git will use. That is the reason I posted this here. Since no method\n> has been formally accepted (checked into master) I wanted to see if\n> I could nudge things along. I wasn't aware that you and Dscho had\n> a (very similar) plan. Please, if you guys are decided on the format\n> then I can just go off and start working on it. But it sounds like there\n> isn't consensus yet.\n\nThis is probably not the answer you want, but I think the final design\ndepends on some C experiments. For example, whether or not there should\nbe fan-out depends on how it affects performance, which means we need to\ndo at least partial implementations to compare. So it really is just\nwaiting for somebody to sit down and do it.\n\n> I like the overall plan, but I would suggest that --notes[=default] and\n> --note-type=whatever would be a little friendlier and less error prone.\n\nBut keeping it as a single string means there is no ambiguity when you\nspecify multiple notes at once. For example:\n\n git log \\\n --note-filter='test:status == \"fail\" && importance > 3' \\\n --pretty=format:%h%n%N(test:errors)\n\nwould do something like:\n\n foreach commit $C\n compare refs/notes/test:$C/status against the string \"fail\"\n compare refs/notes/default:$C/importance against the number 3\n if either don't match, skip the commit\n show the hash and the contents of refs/notes/test:$C/errors\n\nand obviously that filter language is totally made up and we may or may\nnot want to do something that complex. But my point is that we are\ndefining a namespace of notes, and we want to be able to refer to a\nmultiple fully qualified names.\n\n-Peff\n"},{"id":"98361","messageId":"5d46db230812190918qf22b874n8d8aeea557083df8@mail.gmail.com","threadId":"16749","inReplyTo":"20081217101110.GC18265@coredump.intra.peff.net","subject":"Re: Git Notes idea.","fromName":"Govind Salinas","fromEmail":"govind@sophiasuchtig.com","sentAt":"2008-12-19T17:18:17Z","receivedAt":"2008-12-19T17:18:17Z","isPatch":false,"sender":{"key":"govind@sophiasuchtig.com","avatar":null},"body":"On Wed, Dec 17, 2008 at 4:11 AM, Jeff King wrote:\n> On Wed, Dec 17, 2008 at 04:43:57AM +0100, Johannes Schindelin wrote:\n>\n>> > I agree, I haven't thought of any fix along these lines other than to\n>> > make gc do the clean up.\n>>\n>> I have, and IIRC I briefly mentioned it back then. Basically, you will\n>> have to add a \"git notes gc\" or some such, which basically reads in the\n>> whole notes, traverses all reachable commits, marking the corresponding\n>> notes, and then writes out all marked notes (leaving the other notes\n>> behind).\n>\n> I was thinking something similar, but I think it is even easier. Make\n> the rule \"if we still have the object, then we still have the note\".\n> That has three benefits:\n>\n> - implementation is simple: for each note $n, delete it unless\n> has_sha1_file($n).\n>\n> - it handles notes on non-commit objects\n>\n> - it kills off notes when an object is _pruned_, not when it stops\n> being _reachable_. So if I delete a branch with a commit found\n> nowhere else, its notes will hang around until it is actually pruned.\n> If I pull it from lost+found, I still keep the notes.\n>\n> Note that all of this garbage collection of notes is really just\n> removing them from the most current notes _tree_. If the notes structure\n> is actually composed of commits, then old notes that are \"deleted\" will\n> still be available historically.\n>\n\nThis is my concern with keeping a history of the notes pseudo-branch. Let\nus say that I do the following\n\n1) on branch A commit a\n2) add note a`\n3) on branch B commit b\n4) add note b`\n5) on branch B commit c\n6) add note c`\n7) delete branch A\n8) gc after a time such that a is pruned\n\nNow either I will always have anote a`\n"},{"id":"98362","messageId":"5d46db230812190938r4e8ff994gfcb616c750be0f22@mail.gmail.com","threadId":"16749","inReplyTo":"5d46db230812190918qf22b874n8d8aeea557083df8@mail.gmail.com","subject":"Re: Git Notes idea.","fromName":"Govind Salinas","fromEmail":"govind@sophiasuchtig.com","sentAt":"2008-12-19T17:38:55Z","receivedAt":"2008-12-19T17:38:55Z","isPatch":false,"sender":{"key":"govind@sophiasuchtig.com","avatar":null},"body":"Sorry, hit the send key accidentally.\n\nOn Wed, Dec 17, 2008 at 4:11 AM, Jeff King wrote:\n> On Wed, Dec 17, 2008 at 04:43:57AM +0100, Johannes Schindelin wrote:\n>\n> > I agree, I haven't thought of any fix along these lines other than to\n> > make gc do the clean up.\n>\n> I have, and IIRC I briefly mentioned it back then. Basically, you will\n>> have to add a \"git notes gc\" or some such, which basically reads in the\n>> whole notes, traverses all reachable commits, marking the corresponding\n>> notes, and then writes out all marked notes (leaving the other notes\n>> behind).\n>\n> I was thinking something similar, but I think it is even easier. Make\n> the rule \"if we still have the object, then we still have the note\".\n> That has three benefits:\n>\n> - implementation is simple: for each note $n, delete it unless\n> has_sha1_file($n).\n>\n> - it handles notes on non-commit objects\n>\n> - it kills off notes when an object is _pruned_, not when it stops\n> being _reachable_. So if I delete a branch with a commit found\n> nowhere else, its notes will hang around until it is actually pruned.\n> If I pull it from lost+found, I still keep the notes.\n>\n> Note that all of this garbage collection of notes is really just\n> removing them from the most current notes _tree_. If the notes structure\n> is actually composed of commits, then old notes that are \"deleted\" will\n> still be available historically.\n>\n\nThis is my concern with keeping a history of the notes pseudo-branch. Let\nme restate what you are saying with an example\n\n1) on branch A commit a\n2) add note a`\n3) on branch B commit b\n4) add note b`\n5) on branch B commit c\n6) add note c`\n7) delete branch A\n8) gc after a time such that a is pruned\n\nNow either I will always have a note a` as an object forever even though\nthe only commit that points to it is gone or I have to re-write the history of\nthe notes branch from the point that it was added.\n\nGiven this problem, is it really such a good idea to keep the history?\n\nOf course the other side of this conversation is that the merge operation\nwill be more complex since the following can also happen\n\n9) push notes\n10) user 2 pulls notes but still has commit a and note a`\n\nOn the other, other hand, pushing and pulling notes if a history is kept\nwill have to involve a lot of rebasing/merging.\n\nJust to throw an idea out...\n\nA possible solution is that notes are per-branch,\n\nrefs/notes/heads/master\nrefs/notes/heads/foo/bar\nrefs/notes/remotes/baz/bang\n\nand then it is easier to deal with. A published branch's notes are isolated\nfrom the changes in unpublished branches. And since published branches\naren't *supposed* to change, then the notes should also always be fast\nforwards. Similarly, if a branch is not considered stable, like pu or even\nnext, then the associated notes branch could be forced in the same way.\n\nRebase, cherry-pick and merge (and possibly branch/checkout) would have\nto be updated to handle notes, which is the down side. It also doesn't solve\nthe issue of a history causing us to keep notes after the aren't useful anymore.\n\nSo perhaps we could use the above layout with no history?\n\nThanks,\nGovind.\n"},{"id":"98363","messageId":"5d46db230812190942j2670b120ga7ec49798a5dd49d@mail.gmail.com","threadId":"16749","inReplyTo":"alpine.DEB.1.00.0812171233270.28560@intel-tinevez-2-302","subject":"Re: Git Notes idea.","fromName":"Govind Salinas","fromEmail":"govind@sophiasuchtig.com","sentAt":"2008-12-19T17:42:42Z","receivedAt":"2008-12-19T17:42:42Z","isPatch":false,"sender":{"key":"govind@sophiasuchtig.com","avatar":null},"body":"On Wed, Dec 17, 2008 at 5:38 AM, Johannes Schindelin\n wrote:\n> Hi,\n>\n> On Wed, 17 Dec 2008, Jeff King wrote:\n\n>> If he is planning on doing a separate pyrite implementation, then it\n>> _hasn't_ been implemented yet. And I don't care there if he uses hash\n>> tables or sorted lists or whatever. I think the most important thing is\n>> getting down the design of the _data structure_, so that we can have a\n>> compatible implementation inside git itself.\n>\n> Well, I don't care about pyrite. As far as I am concerned, it might as\n> well use an incompatible version. I really don't care.\n\nWell I do care. It would not be a good thing for anyone to have 2 separate\nsystems for notes. Let us say that someone who you work with uses pyrite\nand you don't. They will add notes which you can't see and vice versa.\n\nThanks,\nGovind.\n"},{"id":"98375","messageId":"20081219212536.GA27168@coredump.intra.peff.net","threadId":"16749","inReplyTo":"5d46db230812190938r4e8ff994gfcb616c750be0f22@mail.gmail.com","subject":"Re: Git Notes idea.","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2008-12-19T21:25:36Z","receivedAt":"2008-12-19T21:25:36Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Dec 19, 2008 at 11:38:55AM -0600, Govind Salinas wrote:\n\n> This is my concern with keeping a history of the notes pseudo-branch. Let\n> me restate what you are saying with an example\n> \n> 1) on branch A commit a\n> 2) add note a`\n> 3) on branch B commit b\n> 4) add note b`\n> 5) on branch B commit c\n> 6) add note c`\n> 7) delete branch A\n> 8) gc after a time such that a is pruned\n> \n> Now either I will always have a note a` as an object forever even though\n> the only commit that points to it is gone or I have to re-write the history of\n> the notes branch from the point that it was added.\n\nYes, that's correct.\n\n> Given this problem, is it really such a good idea to keep the history?\n\nI think so. Otherwise how will you push and pull notes? You won't even\nknow which one is the more recent tree, let alone handle any merges\ncaused by editing notes in two places.\n\n> On the other, other hand, pushing and pulling notes if a history is kept\n> will have to involve a lot of rebasing/merging.\n\nDepending on your workflow. It might just involve a lot of fast forwards\nif the note writer is in one place.\n\n> A possible solution is that notes are per-branch,\n> \n> refs/notes/heads/master\n> refs/notes/heads/foo/bar\n> refs/notes/remotes/baz/bang\n\nSorry, I don't quite get it. You are asking for per-branch notes that\nkeep history, or per-branch notes that don't keep history?\n\nIf the former, then you haven't solved the cruft accumulation problem.\nYou can get obsolete notes in your note history by rebasing on a branch\nthat is long-running (which is OK as long as you haven't published\n_those particular_ commits). Or are you proposing to rebase and cleanup\nthe notes history every time you do a destructive operation?\n\nIf the latter, then I don't see how you've solved the push-pull and\nmerge problem (which you need history for).\n\nBut in either case, I think the solution is non-intuitive. If I annotate\na commit, and then merge the commit from one branch to another,\nshouldn't the annotation stay?\n\n\nReally, I am not sure this is worth getting too concerned about. Since\nwe are talking about cruft in the _history_ of the notes branch, it\nwon't impact actual notes usage (which will always just deal with the\nmost recent tree). So really we are talking about some uninteresting\nobjects in the db, which wastes some space. In practice, I suspect this\nwon't be that large because notes themselves are going to be relatively\nshort and in many cases, repetitive (i.e., many annotations may have the\nsame blob hash for several commits). And if it is a space problem, then\nthe right solution is to periodically truncate the notes history by\nrewriting.\n\n-Peff\n"},{"id":"98380","messageId":"5d46db230812191424m14e82c5fx1c1c12027db901ed@mail.gmail.com","threadId":"16749","inReplyTo":"20081219212536.GA27168@coredump.intra.peff.net","subject":"Re: Git Notes idea.","fromName":"Govind Salinas","fromEmail":"govind@sophiasuchtig.com","sentAt":"2008-12-19T22:24:01Z","receivedAt":"2008-12-19T22:24:01Z","isPatch":false,"sender":{"key":"govind@sophiasuchtig.com","avatar":null},"body":"On Fri, Dec 19, 2008 at 3:25 PM, Jeff King wrote:\n> On Fri, Dec 19, 2008 at 11:38:55AM -0600, Govind Salinas wrote:\n>\n>> This is my concern with keeping a history of the notes pseudo-branch. Let\n>> me restate what you are saying with an example\n>>\n>> 1) on branch A commit a\n>> 2) add note a`\n>> 3) on branch B commit b\n>> 4) add note b`\n>> 5) on branch B commit c\n>> 6) add note c`\n>> 7) delete branch A\n>> 8) gc after a time such that a is pruned\n>>\n>> Now either I will always have a note a` as an object forever even though\n>> the only commit that points to it is gone or I have to re-write the history of\n>> the notes branch from the point that it was added.\n>\n> Yes, that's correct.\n>\n>> Given this problem, is it really such a good idea to keep the history?\n>\n> I think so. Otherwise how will you push and pull notes? You won't even\n> know which one is the more recent tree, let alone handle any merges\n> caused by editing notes in two places.\n\nCouldn't you simply merge your tree and theirs even if there is no\nhistory. You would have to find a way to handle merges in any event\nsince they could just as easily happen if you have a history.\n\n>> On the other, other hand, pushing and pulling notes if a history is kept\n>> will have to involve a lot of rebasing/merging.\n>\n> Depending on your workflow. It might just involve a lot of fast forwards\n> if the note writer is in one place.\n>\n>> A possible solution is that notes are per-branch,\n>>\n>> refs/notes/heads/master\n>> refs/notes/heads/foo/bar\n>> refs/notes/remotes/baz/bang\n>\n> Sorry, I don't quite get it. You are asking for per-branch notes that\n> keep history, or per-branch notes that don't keep history?\n\nBoth, at the end of my previous mail I said...\n\n\"So perhaps we could use the above layout with no history?\"\n\nBut they are two separate fixes to 2 different problems.\n\n> If the former, then you haven't solved the cruft accumulation problem.\n> You can get obsolete notes in your note history by rebasing on a branch\n> that is long-running (which is OK as long as you haven't published\n> _those particular_ commits). Or are you proposing to rebase and cleanup\n> the notes history every time you do a destructive operation?\n\nYes, it does not solve that problem. But it does solve things like\n\nDev1 and Dev2 both have branches A and topic branch B. and they\nare in refs/notes/public (or refs/notes or something not branch specific).\n\nDev1 adds 100 notes to topic B, lets say half of them are obsolete due\nto rebases or whatever. Dev2 pulls A and updates their notes\nas well. Now Dev2 has acquired all the notes from Dev1 including the\nobsolete ones. So you have 100 commits, 100 blobs and all the new\ntrees that go with them that the user was not interested in.\n\nRun this across 1000 users and you have a lot of cruft.\n\nNow, if instead we have a per-branch notes scheme, then you only get\nthe cruft from the branches you were interested in. If you remove the\nhistory you could end up with no cruft because gc should handle it.\n\n> If the latter, then I don't see how you've solved the push-pull and\n> merge problem (which you need history for).\n\nWhat git-fetch would have to do is say. This is a note. The remote\nsha is not the same as mine, i will treat this as a force and fetch the\nobjects without checking history and then run a merge on the 2\ncommits. The notes merge could have its own strategy that checked\nif an object exists before deciding to add a new item or delete a\nremoved one. Then the user would only have to intervene if the\nnotes where edited.\n\n> But in either case, I think the solution is non-intuitive. If I annotate\n> a commit, and then merge the commit from one branch to another,\n> shouldn't the annotation stay?\n\nSure, either the merge command could run 2 merges, one for the\nreal branch and one for the notes pseudo branch or the user\ncould be required to do that manually. I would think that doing\nit automatically would be good. Especially if you use a special\nmerge strategy.\n\n> Really, I am not sure this is worth getting too concerned about. Since\n> we are talking about cruft in the _history_ of the notes branch, it\n> won't impact actual notes usage (which will always just deal with the\n> most recent tree). So really we are talking about some uninteresting\n> objects in the db, which wastes some space. In practice, I suspect this\n> won't be that large because notes themselves are going to be relatively\n> short and in many cases, repetitive (i.e., many annotations may have the\n> same blob hash for several commits). And if it is a space problem, then\n> the right solution is to periodically truncate the notes history by\n> rewriting.\n\nYou are correct of course that it will just be wasted space. But I am\nconcerned that it could end up being a lot of wasted space. I mean, what\nif every person who contributed to the kernel contributed note cruft. Users\nhave branches that they consider public, so they might go into the a public\nnote store if there is no per-branch store. Or errant users could use the\npublic store without understanding how they are affecting the central repo,\nincluding the obsolete ones.\n\nIf you *really* don't think its something to be worried about then I am OK\nwith that since you have a lot more experience with this, but it sounds hairy\nto me.\n\nThanks,\nGovind.\n"},{"id":"98390","messageId":"alpine.DEB.1.00.0812192347261.30769@pacific.mpi-cbg.de","threadId":"16749","inReplyTo":"20081216085108.GA3031@coredump.intra.peff.net","subject":"[PATCH 0/4] Notes reloaded","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-12-19T23:34:20Z","receivedAt":"2008-12-19T23:34:20Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Tue, 16 Dec 2008, Jeff King wrote:\n\n> Johannes Schindelin's notes proposal (which is more or less \n> the current proposal, but I think the on-disk notes index was not \n> well liked): \n> http://thread.gmane.org/gmane.comp.version-control.git/52598\n\nI redid the benchmark (this time with a bit beefier machine), just \ncomparing no notes with David's/Peff's idea:\n\n\n-- snip --\n$ GIT_NOTES_TIMING_TESTS=1 sh t3302-notes-index-expensive.sh -i -v\nInitialized empty Git repository in /home/gitte/git/t/trash directory.t3302-notes-index-expensive/.git/\n* expecting success: create_repo 10\nInitialized empty Git repository in /home/gitte/git/t/trash directory.t3302-notes-index-expensive/10/.git/\n* ok 1: setup 10\n\n* expecting success: test_notes 10\n* ok 2: notes work\n\n* expecting success: time_notes 100\nno-notes\n0.08user 0.10system 0:00.18elapsed 95%CPU (0avgtext+0avgdata 0maxresident)k\n0inputs+0outputs (0major+58926minor)pagefaults 0swaps\nnotes\n0.14user 0.07system 0:00.54elapsed 38%CPU (0avgtext+0avgdata 0maxresident)k\n0inputs+0outputs (0major+60319minor)pagefaults 0swaps\n* ok 3: notes timing\n\n* expecting success: create_repo 100\nInitialized empty Git repository in /home/gitte/git/t/trash directory.t3302-notes-index-expensive/100/.git/\n* ok 1: setup 100\n\n* expecting success: test_notes 100\n* ok 2: notes work\n\n* expecting success: time_notes 100\nno-notes\n0.23user 0.21system 0:00.45elapsed 96%CPU (0avgtext+0avgdata 0maxresident)k\n0inputs+0outputs (0major+68043minor)pagefaults 0swaps\nnotes\n0.38user 0.21system 0:00.59elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k\n0inputs+0outputs (0major+78829minor)pagefaults 0swaps\n* ok 3: notes timing\n\n* expecting success: create_repo 1000\nInitialized empty Git repository in /home/gitte/git/t/trash directory.t3302-notes-index-expensive/1000/.git/\n* ok 1: setup 1000\n\n* expecting success: test_notes 1000\n* ok 2: notes work\n\n* expecting success: time_notes 100\nno-notes\n2.06user 0.95system 0:04.26elapsed 70%CPU (0avgtext+0avgdata 0maxresident)k\n0inputs+0outputs (0major+159115minor)pagefaults 0swaps\nnotes\n2.83user 1.54system 0:04.38elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k\n0inputs+0outputs (0major+267416minor)pagefaults 0swaps\n* ok 3: notes timing\n\n* expecting success: create_repo 10000\nInitialized empty Git repository in /home/gitte/git/t/trash directory.t3302-notes-index-expensive/10000/.git/\n* ok 1: setup 10000\n\n* expecting success: test_notes 10000\n* ok 2: notes work\n\n* expecting success: time_notes 100\nno-notes\n20.46user 7.63system 0:28.30elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k\n0inputs+0outputs (0major+1083378minor)pagefaults 0swaps\nnotes\n28.78user 13.74system 0:42.85elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k\n0inputs+0outputs (0major+2240296minor)pagefaults 0swaps\n* ok 3: notes timing\n\n* passed all 0 test(s)\n-- snap --\n\n\nKeep in mind that the tests run \"git log\" 99 times, and show the \naccumulated time.\n\nSo it seems that an increase of roughly 40% in the user time, and roughly \n70% in the system time is the price to have notes associated with every \nsingle commit.\n\nNote that in that very same repository, a single \"git show\" goes from\n\n0.00user 0.00system 0:00.00elapsed 0%CPU (0avgtext+0avgdata \n0maxresident)k\n0inputs+0outputs (0major+561minor)pagefaults 0swaps\n\nto this:\n\n0.03user 0.02system 0:00.04elapsed 113%CPU (0avgtext+0avgdata \n0maxresident)k\n0inputs+0outputs (0major+2294minor)pagefaults 0swaps\n\n(In another run, it only used 90%CPU)\n\nThat's not too shabby, given that Git needs to unpack double the number of \nobjects in this test when using notes vs. no notes.\n\nFor comparison, the numbers back then were something like 10% in user time \nwith a penalty of an extraordinary magnitude everytime the notes are \nupdated: around 800%.\n\nNote: all these numbers are worst-case numbers, i.e. every commit has one \nnote.\n\nTo be frank, I do not completely understand why the numbers are that high. \nI would have understood an increase roughly 4 seconds for reading the \nquite large tree 99 times, and then the same ~0.20 seconds back then. \nMaybe I made a huge mistake when implementing the thing.\n\nAnd BTW, my code does not yet handle the case when \nrefs/notes/commits:$commit is a tree instead of a blob. That is left as \nan exercise to the reader.\n\n\n\nJohannes Schindelin (4):\n Introduce commit notes\n Add a script to edit/inspect notes\n Speed up git notes lookup\n Add an expensive test for git-notes\n\n .gitignore | 1 +\n Documentation/config.txt | 15 ++++\n Documentation/git-notes.txt | 46 +++++++++++\n Makefile | 3 +\n cache.h | 3 +\n command-list.txt | 1 +\n commit.c | 1 +\n config.c | 5 +\n environment.c | 1 +\n git-notes.sh | 65 +++++++++++++++\n notes.c | 159 ++++++++++++++++++++++++++++++++++++++\n notes.h | 7 ++\n pretty.c | 5 +\n t/t3301-notes.sh | 65 +++++++++++++++\n t/t3302-notes-index-expensive.sh | 98 +++++++++++++++++++++++\n 15 files changed, 475 insertions(+), 0 deletions(-)\n create mode 100644 Documentation/git-notes.txt\n create mode 100755 git-notes.sh\n create mode 100644 notes.c\n create mode 100644 notes.h\n create mode 100755 t/t3301-notes.sh\n create mode 100755 t/t3302-notes-index-expensive.sh\n"},{"id":"98391","messageId":"alpine.DEB.1.00.0812200034450.30769@pacific.mpi-cbg.de","threadId":"16749","inReplyTo":"alpine.DEB.1.00.0812192347261.30769@pacific.mpi-cbg.de","subject":"[PATCH 1/4] Introduce commit notes","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-12-19T23:35:06Z","receivedAt":"2008-12-19T23:35:06Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"\nCommit notes are blobs which are shown together with the commit\nmessage. These blobs are taken from the notes ref, which you can\nconfigure by the config variable core.notesRef, which in turn can\nbe overridden by the environment variable GIT_NOTES_REF.\n\nThe notes ref is a branch which contains \"files\" whose names are\nthe names of the corresponding commits (i.e. the SHA-1).\n\nThe rationale for putting this information into a ref is this: we\nwant to be able to fetch and possibly union-merge the notes,\nmaybe even look at the date when a note was introduced, and we\nwant to store them efficiently together with the other objects.\n\nSigned-off-by: Johannes Schindelin \n---\n Documentation/config.txt | 15 ++++++++++\n Makefile | 2 +\n cache.h | 3 ++\n commit.c | 1 +\n config.c | 5 +++\n environment.c | 1 +\n notes.c | 68 ++++++++++++++++++++++++++++++++++++++++++++++\n notes.h | 7 +++++\n pretty.c | 5 +++\n 9 files changed, 107 insertions(+), 0 deletions(-)\n create mode 100644 notes.c\n create mode 100644 notes.h\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 4089362..3248524 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -431,6 +431,21 @@ core.inithook::\n \tlinkgit:git-init[1]. The hook is called with the argument\n \t\"reinit\" if an existing repository is re-initialized.\n \n+core.notesRef::\n+\tWhen showing commit messages, also show notes which are stored in\n+\tthe given ref. This ref is expected to contain paths of the form\n+\t??/*, where the directory name consists of the first two\n+\tcharacters of the commit name, and the base name consists of\n+\tthe remaining 38 characters.\n++\n+If such a path exists in the given ref, the referenced blob is read, and\n+appended to the commit message, separated by a \"Notes:\" line. If the\n+given ref itself does not exist, it is not an error, but means that no\n+notes should be print.\n++\n+This setting defaults to \"refs/notes/commits\", and can be overridden by\n+the `GIT_NOTES_REF` environment variable.\n+\n alias.*::\n \tCommand aliases for the linkgit:git[1] command wrapper - e.g.\n \tafter defining \"alias.last = cat-file commit HEAD\", the invocation\ndiff --git a/Makefile b/Makefile\nindex 5e293fe..7871f52 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -371,6 +371,7 @@ LIB_H += ll-merge.h\n LIB_H += log-tree.h\n LIB_H += mailmap.h\n LIB_H += merge-recursive.h\n+LIB_H += notes.h\n LIB_H += object.h\n LIB_H += pack.h\n LIB_H += pack-refs.h\n@@ -454,6 +455,7 @@ LIB_OBJS += match-trees.o\n LIB_OBJS += merge-file.o\n LIB_OBJS += merge-recursive.o\n LIB_OBJS += name-hash.o\n+LIB_OBJS += notes.o\n LIB_OBJS += object.o\n LIB_OBJS += pack-check.o\n LIB_OBJS += pack-refs.o\ndiff --git a/cache.h b/cache.h\nindex b393c2d..30981de 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -375,6 +375,8 @@ static inline enum object_type object_type(unsigned int mode)\n #define GITATTRIBUTES_FILE \".gitattributes\"\n #define INFOATTRIBUTES_FILE \"info/attributes\"\n #define ATTRIBUTE_MACRO_PREFIX \"[attr]\"\n+#define GIT_NOTES_REF_ENVIRONMENT \"GIT_NOTES_REF\"\n+#define GIT_NOTES_DEFAULT_REF \"refs/notes/commits\"\n \n extern int is_bare_repository_cfg;\n extern int is_bare_repository(void);\n@@ -548,6 +550,7 @@ enum rebase_setup_type {\n extern enum branch_track git_branch_track;\n extern enum rebase_setup_type autorebase;\n extern int keep_hard_links;\n+extern char *notes_ref_name;\n \n #define GIT_REPO_VERSION 0\n extern int repository_format_version;\ndiff --git a/commit.c b/commit.c\nindex 00f4774..547b88f 100644\n--- a/commit.c\n+++ b/commit.c\n@@ -5,6 +5,7 @@\n #include \"utf8.h\"\n #include \"diff.h\"\n #include \"revision.h\"\n+#include \"notes.h\"\n \n int save_commit_buffer = 1;\n \ndiff --git a/config.c b/config.c\nindex 8ff2b4b..9ab9d21 100644\n--- a/config.c\n+++ b/config.c\n@@ -469,6 +469,11 @@ static int git_default_core_config(const char *var, const char *value)\n \t\treturn 0;\n \t}\n \n+\tif (!strcmp(var, \"core.notesref\")) {\n+\t\tnotes_ref_name = xstrdup(value);\n+\t\treturn 0;\n+\t}\n+\n \tif (!strcmp(var, \"core.pager\"))\n \t\treturn git_config_string(&pager_program, var, value);\n \ndiff --git a/environment.c b/environment.c\nindex fc91809..5724ad2 100644\n--- a/environment.c\n+++ b/environment.c\n@@ -46,6 +46,7 @@ int keep_hard_links = 0;\n \n /* Parallel index stat data preload? */\n int core_preload_index = 0;\n+char *notes_ref_name;\n \n /* This is set by setup_git_dir_gently() and/or git_default_config() */\n char *git_work_tree_cfg;\ndiff --git a/notes.c b/notes.c\nnew file mode 100644\nindex 0000000..91ec77f\n--- /dev/null\n+++ b/notes.c\n@@ -0,0 +1,68 @@\n+#include \"cache.h\"\n+#include \"commit.h\"\n+#include \"notes.h\"\n+#include \"refs.h\"\n+#include \"utf8.h\"\n+#include \"strbuf.h\"\n+\n+static int initialized;\n+\n+void get_commit_notes(const struct commit *commit, struct strbuf *sb,\n+\t\tconst char *output_encoding)\n+{\n+\tstatic const char *utf8 = \"utf-8\";\n+\tstruct strbuf name = STRBUF_INIT;\n+\tconst char *hex;\n+\tunsigned char sha1[20];\n+\tchar *msg;\n+\tunsigned long msgoffset, msglen;\n+\tenum object_type type;\n+\n+\tif (!initialized) {\n+\t\tconst char *env = getenv(GIT_NOTES_REF_ENVIRONMENT);\n+\t\tif (env)\n+\t\t\tnotes_ref_name = getenv(GIT_NOTES_REF_ENVIRONMENT);\n+\t\telse if (!notes_ref_name)\n+\t\t\tnotes_ref_name = GIT_NOTES_DEFAULT_REF;\n+\t\tif (notes_ref_name && read_ref(notes_ref_name, sha1))\n+\t\t\tnotes_ref_name = NULL;\n+\t\tinitialized = 1;\n+\t}\n+\n+\tif (!notes_ref_name)\n+\t\treturn;\n+\n+\tstrbuf_addf(&name, \"%s:%s\", notes_ref_name,\n+\t\t\tsha1_to_hex(commit->object.sha1));\n+\tif (get_sha1(name.buf, sha1))\n+\t\treturn;\n+\n+\tif (!(msg = read_sha1_file(sha1, &type, &msglen)) || !msglen ||\n+\t\t\ttype != OBJ_BLOB)\n+\t\treturn;\n+\n+\tif (output_encoding && *output_encoding &&\n+\t\t\tstrcmp(utf8, output_encoding)) {\n+\t\tchar *reencoded = reencode_string(msg, output_encoding, utf8);\n+\t\tif (reencoded) {\n+\t\t\tfree(msg);\n+\t\t\tmsg = reencoded;\n+\t\t\tmsglen = strlen(msg);\n+\t\t}\n+\t}\n+\n+\t/* we will end the annotation by a newline anyway */\n+\tif (msglen && msg[msglen - 1] == '\\n')\n+\t\tmsglen--;\n+\n+\tstrbuf_addstr(sb, \"\\nNotes:\\n\");\n+\n+\tfor (msgoffset = 0; msgoffset < msglen;) {\n+\t\tint linelen = strchrnul(msg, '\\n') - msg;\n+\n+\t\tstrbuf_addstr(sb, \" \");\n+\t\tstrbuf_add(sb, msg + msgoffset, linelen);\n+\t\tmsgoffset += linelen;\n+\t}\n+\tfree(msg);\n+}\ndiff --git a/notes.h b/notes.h\nnew file mode 100644\nindex 0000000..79d21b6\n--- /dev/null\n+++ b/notes.h\n@@ -0,0 +1,7 @@\n+#ifndef NOTES_H\n+#define NOTES_H\n+\n+void get_commit_notes(const struct commit *commit, struct strbuf *sb,\n+\t\tconst char *output_encoding);\n+\n+#endif\ndiff --git a/pretty.c b/pretty.c\nindex 5f9a0c7..c2bf451 100644\n--- a/pretty.c\n+++ b/pretty.c\n@@ -6,6 +6,7 @@\n #include \"string-list.h\"\n #include \"mailmap.h\"\n #include \"log-tree.h\"\n+#include \"notes.h\"\n \n static char *user_format;\n \n@@ -911,5 +912,9 @@ void pretty_print_commit(enum cmit_fmt fmt, const struct commit *commit,\n \t */\n \tif (fmt == CMIT_FMT_EMAIL && sb->len <= beginning_of_body)\n \t\tstrbuf_addch(sb, '\\n');\n+\n+\tif (fmt != CMIT_FMT_ONELINE)\n+\t\tget_commit_notes(commit, sb, encoding);\n+\n \tfree(reencoded);\n }\n-- \n1.6.1.rc3.368.g63acc\n"},{"id":"98392","messageId":"alpine.DEB.1.00.0812200035180.30769@pacific.mpi-cbg.de","threadId":"16749","inReplyTo":"alpine.DEB.1.00.0812192347261.30769@pacific.mpi-cbg.de","subject":"[PATCH 2/4] Add a script to edit/inspect notes","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-12-19T23:35:28Z","receivedAt":"2008-12-19T23:35:28Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"\nThe script 'git notes' allows you to edit and show commit notes, by\ncalling either\n\n\tgit notes show \n\nor\n\n\tgit notes edit \n\nSigned-off-by: Johannes Schindelin \n---\n .gitignore | 1 +\n Documentation/git-notes.txt | 46 ++++++++++++++++++++++++++++++\n Makefile | 1 +\n command-list.txt | 1 +\n git-notes.sh | 65 +++++++++++++++++++++++++++++++++++++++++++\n t/t3301-notes.sh | 65 +++++++++++++++++++++++++++++++++++++++++++\n 6 files changed, 179 insertions(+), 0 deletions(-)\n create mode 100644 Documentation/git-notes.txt\n create mode 100755 git-notes.sh\n create mode 100755 t/t3301-notes.sh\n\ndiff --git a/.gitignore b/.gitignore\nindex 90bbb2a..8b90af1 100644\n--- a/.gitignore\n+++ b/.gitignore\n@@ -83,6 +83,7 @@ git-mktag\n git-mktree\n git-name-rev\n git-mv\n+git-notes\n git-pack-redundant\n git-pack-objects\n git-pack-refs\ndiff --git a/Documentation/git-notes.txt b/Documentation/git-notes.txt\nnew file mode 100644\nindex 0000000..3d93625\n--- /dev/null\n+++ b/Documentation/git-notes.txt\n@@ -0,0 +1,46 @@\n+git-notes(1)\n+============\n+\n+NAME\n+----\n+git-notes - Add/inspect commit notes\n+\n+SYNOPSIS\n+--------\n+[verse]\n+'git-notes' (edit | show) [commit]\n+\n+DESCRIPTION\n+-----------\n+This command allows you to add notes to commit messages, without\n+changing the commit. To discern these notes from the message stored\n+in the commit object, the notes are indented like the message, after\n+an unindented line saying \"Notes:\".\n+\n+To disable commit notes, you have to set the config variable\n+core.notesRef to the empty string. Alternatively, you can set it\n+to a different ref, something like \"refs/notes/bugzilla\". This setting\n+can be overridden by the environment variable \"GIT_NOTES_REF\".\n+\n+\n+SUBCOMMANDS\n+-----------\n+\n+edit::\n+\tEdit the notes for a given commit (defaults to HEAD).\n+\n+show::\n+\tShow the notes for a given commit (defaults to HEAD).\n+\n+\n+Author\n+------\n+Written by Johannes Schindelin \n+\n+Documentation\n+-------------\n+Documentation by Johannes Schindelin\n+\n+GIT\n+---\n+Part of the gitlink:git[7] suite\ndiff --git a/Makefile b/Makefile\nindex 7871f52..4bdc86e 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -259,6 +259,7 @@ SCRIPT_SH += git-merge-octopus.sh\n SCRIPT_SH += git-merge-one-file.sh\n SCRIPT_SH += git-merge-resolve.sh\n SCRIPT_SH += git-mergetool.sh\n+SCRIPT_SH += git-notes.sh\n SCRIPT_SH += git-parse-remote.sh\n SCRIPT_SH += git-pull.sh\n SCRIPT_SH += git-quiltimport.sh\ndiff --git a/command-list.txt b/command-list.txt\nindex 3583a33..2dc2c33 100644\n--- a/command-list.txt\n+++ b/command-list.txt\n@@ -73,6 +73,7 @@ git-mktag plumbingmanipulators\n git-mktree plumbingmanipulators\n git-mv mainporcelain common\n git-name-rev plumbinginterrogators\n+git-notes mainporcelain\n git-pack-objects plumbingmanipulators\n git-pack-redundant plumbinginterrogators\n git-pack-refs ancillarymanipulators\ndiff --git a/git-notes.sh b/git-notes.sh\nnew file mode 100755\nindex 0000000..bfdbaa8\n--- /dev/null\n+++ b/git-notes.sh\n@@ -0,0 +1,65 @@\n+#!/bin/sh\n+\n+USAGE=\"(edit | show) [commit]\"\n+. git-sh-setup\n+\n+test -n \"$3\" && usage\n+\n+test -z \"$1\" && usage\n+ACTION=\"$1\"; shift\n+\n+test -z \"$GIT_NOTES_REF\" && GIT_NOTES_REF=\"$(git config core.notesref)\"\n+test -z \"$GIT_NOTES_REF\" && GIT_NOTES_REF=\"refs/notes/commits\"\n+\n+COMMIT=$(git rev-parse --verify --default HEAD \"$@\") ||\n+die \"Invalid commit: $@\"\n+\n+MESSAGE=\"$GIT_DIR\"/new-notes-$COMMIT\n+trap '\n+\ttest -f \"$MESSAGE\" && rm \"$MESSAGE\"\n+' 0\n+\n+case \"$ACTION\" in\n+edit)\n+\tGIT_NOTES_REF= git log -1 $COMMIT | sed \"s/^/#/\" > \"$MESSAGE\"\n+\n+\tGIT_INDEX_FILE=\"$MESSAGE\".idx\n+\texport GIT_INDEX_FILE\n+\n+\tCURRENT_HEAD=$(git show-ref \"$GIT_NOTES_REF\" | cut -f 1 -d ' ')\n+\tif [ -z \"$CURRENT_HEAD\" ]; then\n+\t\tPARENT=\n+\telse\n+\t\tPARENT=\"-p $CURRENT_HEAD\"\n+\t\tgit read-tree \"$GIT_NOTES_REF\" || die \"Could not read index\"\n+\t\tgit cat-file blob :$COMMIT >> \"$MESSAGE\" 2> /dev/null\n+\tfi\n+\n+\t${VISUAL:-${EDITOR:-vi}} \"$MESSAGE\"\n+\n+\tgrep -v ^# < \"$MESSAGE\" | git stripspace > \"$MESSAGE\".processed\n+\tmv \"$MESSAGE\".processed \"$MESSAGE\"\n+\tif [ -s \"$MESSAGE\" ]; then\n+\t\tBLOB=$(git hash-object -w \"$MESSAGE\") ||\n+\t\t\tdie \"Could not write into object database\"\n+\t\tgit update-index --add --cacheinfo 0644 $BLOB $COMMIT ||\n+\t\t\tdie \"Could not write index\"\n+\telse\n+\t\ttest -z \"$CURRENT_HEAD\" &&\n+\t\t\tdie \"Will not initialise with empty tree\"\n+\t\tgit update-index --force-remove $COMMIT ||\n+\t\t\tdie \"Could not update index\"\n+\tfi\n+\n+\tTREE=$(git write-tree) || die \"Could not write tree\"\n+\tNEW_HEAD=$(echo Annotate $COMMIT | git commit-tree $TREE $PARENT) ||\n+\t\tdie \"Could not annotate\"\n+\tgit update-ref -m \"Annotate $COMMIT\" \\\n+\t\t\"$GIT_NOTES_REF\" $NEW_HEAD $CURRENT_HEAD\n+;;\n+show)\n+\tgit show \"$GIT_NOTES_REF\":$COMMIT\n+;;\n+*)\n+\tusage\n+esac\ndiff --git a/t/t3301-notes.sh b/t/t3301-notes.sh\nnew file mode 100755\nindex 0000000..ba42c45\n--- /dev/null\n+++ b/t/t3301-notes.sh\n@@ -0,0 +1,65 @@\n+#!/bin/sh\n+#\n+# Copyright (c) 2007 Johannes E. Schindelin\n+#\n+\n+test_description='Test commit notes'\n+\n+. ./test-lib.sh\n+\n+cat > fake_editor.sh << \\EOF\n+echo \"$MSG\" > \"$1\"\n+echo \"$MSG\" >& 2\n+EOF\n+chmod a+x fake_editor.sh\n+VISUAL=./fake_editor.sh\n+export VISUAL\n+\n+test_expect_success 'cannot annotate non-existing HEAD' '\n+\t! MSG=3 git notes edit\n+'\n+\n+test_expect_success setup '\n+\t: > a1 &&\n+\tgit add a1 &&\n+\ttest_tick &&\n+\tgit commit -m 1st &&\n+\t: > a2 &&\n+\tgit add a2 &&\n+\ttest_tick &&\n+\tgit commit -m 2nd\n+'\n+\n+test_expect_success 'need valid notes ref' '\n+\t! MSG=1 GIT_NOTES_REF='/' git notes edit &&\n+\t! MSG=2 GIT_NOTES_REF='/' git notes show\n+'\n+\n+test_expect_success 'create notes' '\n+\tgit config core.notesRef refs/notes/commits &&\n+\tMSG=b1 git notes edit &&\n+\ttest ! -f .git/new-notes &&\n+\ttest 1 = $(git ls-tree refs/notes/commits | wc -l) &&\n+\ttest b1 = $(git notes show) &&\n+\tgit show HEAD^ &&\n+\t! git notes show HEAD^\n+'\n+\n+cat > expect << EOF\n+commit 268048bfb8a1fb38e703baceb8ab235421bf80c5\n+Author: A U Thor \n+Date: Thu Apr 7 15:14:13 2005 -0700\n+\n+ 2nd\n+\n+Notes:\n+ b1\n+EOF\n+\n+test_expect_success 'show notes' '\n+\t! (git cat-file commit HEAD | grep b1) &&\n+\tgit log -1 > output &&\n+\tgit diff expect output\n+'\n+\n+test_done\n-- \n1.6.1.rc3.368.g63acc\n"},{"id":"98393","messageId":"alpine.DEB.1.00.0812200035360.30769@pacific.mpi-cbg.de","threadId":"16749","inReplyTo":"alpine.DEB.1.00.0812192347261.30769@pacific.mpi-cbg.de","subject":"[PATCH 3/4] Speed up git notes lookup","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-12-19T23:35:45Z","receivedAt":"2008-12-19T23:35:45Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"\nTo avoid looking up each and every commit in the notes ref's tree\nobject, which is very expensive, speed things up by slurping the tree\nobject's contents into a hash_map.\n\nThe idea fo the hashmap singleton is from David Reiss, initial\nbenchmarking by Jeff King.\n\nNote: the implementation allows for arbitrary entries in the notes\ntree object, ignoring those that do not reference a valid object. This\nallows you to annotate arbitrary branches, or objects.\n\nSigned-off-by: Johannes Schindelin \n---\n notes.c | 113 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++------\n 1 files changed, 102 insertions(+), 11 deletions(-)\n\ndiff --git a/notes.c b/notes.c\nindex 91ec77f..68bcb24 100644\n--- a/notes.c\n+++ b/notes.c\n@@ -4,16 +4,112 @@\n #include \"refs.h\"\n #include \"utf8.h\"\n #include \"strbuf.h\"\n+#include \"tree-walk.h\"\n+\n+struct entry {\n+\tunsigned char commit_sha1[20];\n+\tunsigned char notes_sha1[20];\n+};\n+\n+struct hash_map {\n+\tstruct entry *entries;\n+\toff_t count, size;\n+};\n \n static int initialized;\n+static struct hash_map hash_map;\n+\n+static int hash_index(struct hash_map *map, const unsigned char *sha1)\n+{\n+\tint i = ((*(unsigned int *)sha1) % map->size);\n+\n+\tfor (;;) {\n+\t\tunsigned char *current = map->entries[i].commit_sha1;\n+\n+\t\tif (!hashcmp(sha1, current))\n+\t\t\treturn i;\n+\n+\t\tif (is_null_sha1(current))\n+\t\t\treturn -1 - i;\n+\n+\t\tif (++i == map->size)\n+\t\t\ti = 0;\n+\t}\n+}\n+\n+static void add_entry(const unsigned char *commit_sha1,\n+\t\tconst unsigned char *notes_sha1)\n+{\n+\tint index;\n+\n+\tif (hash_map.count + 1 > hash_map.size >> 1) {\n+\t\tint i, old_size = hash_map.size;\n+\t\tstruct entry *old = hash_map.entries;\n+\n+\t\thash_map.size = old_size ? old_size << 1 : 64;\n+\t\thash_map.entries = (struct entry *)\n+\t\t\txcalloc(sizeof(struct entry), hash_map.size);\n+\n+\t\tfor (i = 0; i < old_size; i++)\n+\t\t\tif (!is_null_sha1(old[i].commit_sha1)) {\n+\t\t\t\tindex = -1 - hash_index(&hash_map,\n+\t\t\t\t\t\told[i].commit_sha1);\n+\t\t\t\tmemcpy(hash_map.entries + index, old + i,\n+\t\t\t\t\tsizeof(struct entry));\n+\t\t\t}\n+\t\tfree(old);\n+\t}\n+\n+\tindex = hash_index(&hash_map, commit_sha1);\n+\tif (index < 0) {\n+\t\tindex = -1 - index;\n+\t\thash_map.count++;\n+\t}\n+\n+\thashcpy(hash_map.entries[index].commit_sha1, commit_sha1);\n+\thashcpy(hash_map.entries[index].notes_sha1, notes_sha1);\n+}\n+\n+static void initialize_hash_map(const char *notes_ref_name)\n+{\n+\tunsigned char sha1[20], commit_sha1[20];\n+\tunsigned *mode;\n+\tstruct tree_desc desc;\n+\tstruct name_entry entry;\n+\tvoid *buf;\n+\n+\tif (!notes_ref_name || read_ref(notes_ref_name, commit_sha1) ||\n+\t\t\tget_tree_entry(commit_sha1, \"\", sha1, mode))\n+\t\treturn;\n+\n+\tbuf = fill_tree_descriptor(&desc, sha1);\n+\tif (!buf)\n+\t\tdie (\"Could not read %s for notes-index\", sha1_to_hex(sha1));\n+\n+\twhile (tree_entry(&desc, &entry))\n+\t\tif (!get_sha1(entry.path, commit_sha1))\n+\t\t\tadd_entry(commit_sha1, entry.sha1);\n+\tfree(buf);\n+}\n+\n+static unsigned char *lookup_notes(const unsigned char *commit_sha1)\n+{\n+\tint index;\n+\n+\tif (!hash_map.size)\n+\t\treturn NULL;\n+\n+\tindex = hash_index(&hash_map, commit_sha1);\n+\tif (index < 0)\n+\t\treturn NULL;\n+\treturn hash_map.entries[index].notes_sha1;\n+}\n \n void get_commit_notes(const struct commit *commit, struct strbuf *sb,\n \t\tconst char *output_encoding)\n {\n \tstatic const char *utf8 = \"utf-8\";\n-\tstruct strbuf name = STRBUF_INIT;\n-\tconst char *hex;\n-\tunsigned char sha1[20];\n+\tunsigned char *sha1;\n \tchar *msg;\n \tunsigned long msgoffset, msglen;\n \tenum object_type type;\n@@ -24,17 +120,12 @@ void get_commit_notes(const struct commit *commit, struct strbuf *sb,\n \t\t\tnotes_ref_name = getenv(GIT_NOTES_REF_ENVIRONMENT);\n \t\telse if (!notes_ref_name)\n \t\t\tnotes_ref_name = GIT_NOTES_DEFAULT_REF;\n-\t\tif (notes_ref_name && read_ref(notes_ref_name, sha1))\n-\t\t\tnotes_ref_name = NULL;\n+\t\tinitialize_hash_map(notes_ref_name);\n \t\tinitialized = 1;\n \t}\n \n-\tif (!notes_ref_name)\n-\t\treturn;\n-\n-\tstrbuf_addf(&name, \"%s:%s\", notes_ref_name,\n-\t\t\tsha1_to_hex(commit->object.sha1));\n-\tif (get_sha1(name.buf, sha1))\n+\tsha1 = lookup_notes(commit->object.sha1);\n+\tif (!sha1)\n \t\treturn;\n \n \tif (!(msg = read_sha1_file(sha1, &type, &msglen)) || !msglen ||\n-- \n1.6.1.rc3.368.g63acc\n"},{"id":"98394","messageId":"alpine.DEB.1.00.0812200035590.30769@pacific.mpi-cbg.de","threadId":"16749","inReplyTo":"alpine.DEB.1.00.0812192347261.30769@pacific.mpi-cbg.de","subject":"[PATCH 4/4] Add an expensive test for git-notes","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-12-19T23:37:15Z","receivedAt":"2008-12-19T23:37:15Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"\ngit-notes have the potential of being pretty expensive, so test with\na lot of commits. A lot. So to make things cheaper, you have to\nopt-in explicitely, by setting the environment variable\nGIT_NOTES_TIMING_TESTS.\n\nSigned-off-by: Johannes Schindelin \n---\n\n\tI would appreciate other people running the tests, and maybe \n\tprofiling the code.\n\n\tHowever, I will not be really online the next two weeks, so if you \n\tfeel like working on this series, go ahead.\n\n\tMerry Christmas.\n\n t/t3302-notes-index-expensive.sh | 98 ++++++++++++++++++++++++++++++++++++++\n 1 files changed, 98 insertions(+), 0 deletions(-)\n create mode 100755 t/t3302-notes-index-expensive.sh\n\ndiff --git a/t/t3302-notes-index-expensive.sh b/t/t3302-notes-index-expensive.sh\nnew file mode 100755\nindex 0000000..00d27bf\n--- /dev/null\n+++ b/t/t3302-notes-index-expensive.sh\n@@ -0,0 +1,98 @@\n+#!/bin/sh\n+#\n+# Copyright (c) 2007 Johannes E. Schindelin\n+#\n+\n+test_description='Test commit notes index (expensive!)'\n+\n+. ./test-lib.sh\n+\n+test -z \"$GIT_NOTES_TIMING_TESTS\" && {\n+\tsay Skipping timing tests\n+\ttest_done\n+\texit\n+}\n+\n+create_repo () {\n+\tnumber_of_commits=$1\n+\tnr=0\n+\tparent=\n+\ttest -d .git || {\n+\tgit init &&\n+\ttree=$(git write-tree) &&\n+\twhile [ $nr -lt $number_of_commits ]; do\n+\t\ttest_tick &&\n+\t\tcommit=$(echo $nr | git commit-tree $tree $parent) ||\n+\t\t\treturn\n+\t\tparent=\"-p $commit\"\n+\t\tnr=$(($nr+1))\n+\tdone &&\n+\tgit update-ref refs/heads/master $commit &&\n+\t{\n+\t\texport GIT_INDEX_FILE=.git/temp;\n+\t\tgit rev-list HEAD | cat -n | sed \"s/^[ \t][ \t]*/ /g\" |\n+\t\twhile read nr sha1; do\n+\t\t\tblob=$(echo note $nr | git hash-object -w --stdin) &&\n+\t\t\techo $sha1 | sed \"s/^/0644 $blob 0\t/\"\n+\t\tdone | git update-index --index-info &&\n+\t\ttree=$(git write-tree) &&\n+\t\ttest_tick &&\n+\t\tcommit=$(echo notes | git commit-tree $tree) &&\n+\t\tgit update-ref refs/notes/commits $commit\n+\t} &&\n+\tgit config core.notesRef refs/notes/commits\n+\t}\n+}\n+\n+test_notes () {\n+\tcount=$1 &&\n+\tgit config core.notesRef refs/notes/commits &&\n+\tgit log | grep \"^ \" > output &&\n+\ti=1 &&\n+\twhile [ $i -le $count ]; do\n+\t\techo \" $(($count-$i))\" &&\n+\t\techo \" note $i\" &&\n+\t\ti=$(($i+1));\n+\tdone > expect &&\n+\tgit diff expect output\n+}\n+\n+cat > time_notes << \\EOF\n+\tmode=$1\n+\ti=1\n+\twhile [ $i -lt $2 ]; do\n+\t\tcase $1 in\n+\t\tno-notes)\n+\t\t\texport GIT_NOTES_REF=non-existing\n+\t\t;;\n+\t\tnotes)\n+\t\t\tunset GIT_NOTES_REF\n+\t\t;;\n+\t\tesac\n+\t\tgit log >/dev/null\n+\t\ti=$(($i+1))\n+\tdone\n+EOF\n+\n+time_notes () {\n+\tfor mode in no-notes notes\n+\tdo\n+\t\techo $mode\n+\t\t/usr/bin/time sh ../time_notes $mode $1\n+\tdone\n+}\n+\n+for count in 10 100 1000 10000; do\n+\n+\tmkdir $count\n+\t(cd $count;\n+\n+\ttest_expect_success \"setup $count\" \"create_repo $count\"\n+\n+\ttest_expect_success 'notes work' \"test_notes $count\"\n+\n+\ttest_expect_success 'notes timing' \"time_notes 100\"\n+\t)\n+done\n+\n+test_done\n-- \n1.6.1.rc3.368.g63acc\n"},{"id":"98408","messageId":"200812191749.43512.bss@iguanasuicide.net","threadId":"16749","inReplyTo":"alpine.DEB.1.00.0812200035590.30769@pacific.mpi-cbg.de","subject":"Re: [PATCH 4/4] Add an expensive test for git-notes","fromName":"Boyd Stephen Smith Jr.","fromEmail":"bss@iguanasuicide.net","sentAt":"2008-12-19T23:49:39Z","receivedAt":"2008-12-19T23:49:39Z","isPatch":true,"sender":{"key":"bss@iguanasuicide.net","avatar":"https://gravatar.com/avatar/84b95eeff194b816c1568b1339e63e4b229825298664a9037b9f1ec713ead1e3?d=mp&s=160"},"body":"On Friday 2008 December 19 17:37:15 Johannes Schindelin wrote:\n> +       test_expect_success 'notes timing' \"time_notes 100\"\n ^^^\nProbably should be ${count}.\n-- \nBoyd Stephen Smith Jr. ,= ,-_-. =. \nbss@iguanasuicide.net ((_/)o o(\\_))\nICQ: 514984 YM/AIM: DaTwinkDaddy `-'(. .)`-' \nhttp://iguanasuicide.net/ \\_/ \n"},{"id":"98411","messageId":"20081220045437.GA27341@coredump.intra.peff.net","threadId":"16749","inReplyTo":"5d46db230812191424m14e82c5fx1c1c12027db901ed@mail.gmail.com","subject":"Re: Git Notes idea.","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2008-12-20T04:54:37Z","receivedAt":"2008-12-20T04:54:37Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Dec 19, 2008 at 04:24:01PM -0600, Govind Salinas wrote:\n\n> > I think so. Otherwise how will you push and pull notes? You won't even\n> > know which one is the more recent tree, let alone handle any merges\n> > caused by editing notes in two places.\n> \n> Couldn't you simply merge your tree and theirs even if there is no\n> history. You would have to find a way to handle merges in any event\n> since they could just as easily happen if you have a history.\n\nLet's say I have a tree T1 like this:\n\n $COMMIT_A -> $BLOB_A\n $COMMIT_B -> $BLOB_B1\n\nand a tree T2 like this:\n\n $COMMIT_B -> $BLOB_B2\n $COMMIT_C -> $BLOB_C\n\nwhat is the correct merge? Was $COMMIT_A added in T1, or deleted in T2?\nHow about $COMMIT_C? Even if you went with a strategy like \"always add\nfrom both\" (which I don't think is a good idea, because deleted notes\nwill keep popping back up) you have a conflict with $COMMIT_B. Should\nit be B1 or B2? You can't tell if B1 became B2, vice versa, or if there\nis a true merge conflict.\n\n> > If the former, then you haven't solved the cruft accumulation problem.\n> > You can get obsolete notes in your note history by rebasing on a branch\n> > that is long-running (which is OK as long as you haven't published\n> > _those particular_ commits). Or are you proposing to rebase and cleanup\n> > the notes history every time you do a destructive operation?\n> \n> Yes, it does not solve that problem. But it does solve things like\n> \n> Dev1 and Dev2 both have branches A and topic branch B. and they\n> are in refs/notes/public (or refs/notes or something not branch specific).\n> \n> Dev1 adds 100 notes to topic B, lets say half of them are obsolete due\n> to rebases or whatever. Dev2 pulls A and updates their notes\n> as well. Now Dev2 has acquired all the notes from Dev1 including the\n> obsolete ones. So you have 100 commits, 100 blobs and all the new\n> trees that go with them that the user was not interested in.\n> \n> Run this across 1000 users and you have a lot of cruft.\n> \n> Now, if instead we have a per-branch notes scheme, then you only get\n> the cruft from the branches you were interested in. If you remove the\n> history you could end up with no cruft because gc should handle it.\n\nOK. But my point is that this is an incomplete solution. You can _still_\nget cruft, and you _still_ have to deal with that cruft some other way.\nSo we will still end up having to implement something else. And I might\neven be fine with a partial solution that helped some if it didn't come\nwith a cost, but I think the \"notes stick to branches\" behavior is\nstrictly worse.\n\n> > If the latter, then I don't see how you've solved the push-pull and\n> > merge problem (which you need history for).\n> \n> What git-fetch would have to do is say. This is a note. The remote\n> sha is not the same as mine, i will treat this as a force and fetch the\n> objects without checking history and then run a merge on the 2\n> commits. The notes merge could have its own strategy that checked\n> if an object exists before deciding to add a new item or delete a\n> removed one. Then the user would only have to intervene if the\n> notes where edited.\n\nI don't like that because:\n\n - the user is going to end up manually resolving merge conflicts for\n things that _should_ have been fast forwards. But much worse, it's\n going to be on content they may never even have seen before. How\n will they decide which is which?\n\n - how do you push notes? There's no opportunity to handle the merge\n on the remote side. And you can't just pull, merge locally, and push\n what is now a fast-forward, because there is no concept of\n fast-forward without history.\n\n - Suddenly pulling and pushing notes isn't just taken care of by the\n usual ref transfer mechanisms. We have to implement a whole new\n system.\n\n> You are correct of course that it will just be wasted space. But I am\n> concerned that it could end up being a lot of wasted space. I mean, what\n> if every person who contributed to the kernel contributed note cruft. Users\n\nWhat if every person who contributed to the kernel contributed history\ncruft? It's really the same problem, and it is solved by people keeping\ntheir trees clean (via rebase) and being picky about how data comes into\nyour tree (i.e., don't pull from people with cruft). I suspect Linus\nwouldn't pull notes at all (and they wouldn't make it over patch\ntransmission anyway). But in a workflow that is pulling the notes, the\nright time to clean up history is probably before publishing. That is,\nyou can rebase and clean up your notes history just before you push it\nto somewhere public, just like you might clean up messy history.\n\n> If you *really* don't think its something to be worried about then I\n> am OK with that since you have a lot more experience with this, but it\n> sounds hairy to me.\n\nIt is hairy, and I wish there were a better solution. But I think every\nother option is much worse.\n\n-Peff\n"},{"id":"98417","messageId":"20081220065337.GA2581@coredump.intra.peff.net","threadId":"16749","inReplyTo":"alpine.DEB.1.00.0812200034450.30769@pacific.mpi-cbg.de","subject":"Re: [PATCH 1/4] Introduce commit notes","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2008-12-20T06:53:38Z","receivedAt":"2008-12-20T06:53:38Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sat, Dec 20, 2008 at 12:35:06AM +0100, Johannes Schindelin wrote:\n\n> Commit notes are blobs which are shown together with the commit\n> message. These blobs are taken from the notes ref, which you can\n> configure by the config variable core.notesRef, which in turn can\n> be overridden by the environment variable GIT_NOTES_REF.\n\nHmm. I wanted to try some performance comparisons based on this\nimplementation, but I can't get your 1/4 to apply. Conflicts in\nconfig.txt and cache.h when applying to master, and \"sha1 information is\nlacking or useless\" for a 3-way merge. What did you base this on?\n\n-Peff\n"},{"id":"98419","messageId":"200812200855.14915.robin.rosenberg.lists@dewire.com","threadId":"16749","inReplyTo":"20081220065337.GA2581@coredump.intra.peff.net","subject":"Re: [PATCH 1/4] Introduce commit notes","fromName":"Robin Rosenberg","fromEmail":"robin.rosenberg.lists@dewire.com","sentAt":"2008-12-20T07:55:14Z","receivedAt":"2008-12-20T07:55:14Z","isPatch":true,"sender":{"key":"robin.rosenberg@dewire.com","avatar":"https://avatars.githubusercontent.com/u/46357?v=4"},"body":"lördag 20 december 2008 07:53:38 skrev Jeff King:\n> On Sat, Dec 20, 2008 at 12:35:06AM +0100, Johannes Schindelin wrote:\n> \n> > Commit notes are blobs which are shown together with the commit\n> > message. These blobs are taken from the notes ref, which you can\n> > configure by the config variable core.notesRef, which in turn can\n> > be overridden by the environment variable GIT_NOTES_REF.\n> \n> Hmm. I wanted to try some performance comparisons based on this\n> implementation, but I can't get your 1/4 to apply. Conflicts in\n> config.txt and cache.h when applying to master, and \"sha1 information is\n> lacking or useless\" for a 3-way merge. What did you base this on?\n\npatch(1) however can crunch it, with the exception of cache.h. Shouldn't\ngit am/appy and patch agree on git generated patches (without binary diffs)?\n\n-- robin\n"},{"id":"98420","messageId":"20081220080546.GA4580@coredump.intra.peff.net","threadId":"16749","inReplyTo":"200812200855.14915.robin.rosenberg.lists@dewire.com","subject":"Re: [PATCH 1/4] Introduce commit notes","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2008-12-20T08:05:47Z","receivedAt":"2008-12-20T08:05:47Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sat, Dec 20, 2008 at 08:55:14AM +0100, Robin Rosenberg wrote:\n\n> > Hmm. I wanted to try some performance comparisons based on this\n> > implementation, but I can't get your 1/4 to apply. Conflicts in\n> > config.txt and cache.h when applying to master, and \"sha1 information is\n> > lacking or useless\" for a 3-way merge. What did you base this on?\n> \n> patch(1) however can crunch it, with the exception of cache.h. Shouldn't\n> git am/appy and patch agree on git generated patches (without binary diffs)?\n\nNo. git apply is intentionally much more strict about applying under the\nassumption that it is better to force a conflict than to silently apply\nsomething that has a reasonable chance of being completely wrong.\n\nAnd usually it is not a big deal because falling back to the 3-way merge\nis a much nicer way of handling any conflicts _anyway_ (I find .rej\nfiles so much more useless than conflict markers, personally).\n\nIn this case I was able to:\n\n 1. git am /the/patch\n 2. patch -p1 <.git/rebase-apply/patch\n 3. manually inspect the results for sanity, and fix up the cache.h\n bit that failed totally\n 4. git add -u && git add notes.[ch]\n 5. git am --resolved\n\n-Peff\n"},{"id":"98422","messageId":"7vk59vz2dx.fsf@gitster.siamese.dyndns.org","threadId":"16749","inReplyTo":"20081220080546.GA4580@coredump.intra.peff.net","subject":"Re: [PATCH 1/4] Introduce commit notes","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-12-20T08:17:46Z","receivedAt":"2008-12-20T08:17:46Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> No. git apply is intentionally much more strict about applying under the\n> assumption that it is better to force a conflict than to silently apply\n> something that has a reasonable chance of being completely wrong.\n>\n> And usually it is not a big deal because falling back to the 3-way merge\n> is a much nicer way of handling any conflicts _anyway_ (I find .rej\n> files so much more useless than conflict markers, personally).\n>\n> In this case I was able to:\n>\n> 1. git am /the/patch\n> 2. patch -p1 <.git/rebase-apply/patch\n> 3. manually inspect the results for sanity, and fix up the cache.h\n> bit that failed totally\n> 4. git add -u && git add notes.[ch]\n> 5. git am --resolved\n\nI usually skip 2-4 and edit .git/rebase-apply/patch in place instead, and\nrun \"git am\" instead of step 5.\n\nWhy was the patch, that was based on something that is clearly different\nfrom what other people work on, sent to the list in the first place? IOW,\nwhat good does it do to show your patch if other people (plural) need to\nspend a lot of time and head-scratching?\n"},{"id":"98424","messageId":"20081220082304.GA5693@coredump.intra.peff.net","threadId":"16749","inReplyTo":"7vk59vz2dx.fsf@gitster.siamese.dyndns.org","subject":"Re: [PATCH 1/4] Introduce commit notes","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2008-12-20T08:23:04Z","receivedAt":"2008-12-20T08:23:04Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sat, Dec 20, 2008 at 12:17:46AM -0800, Junio C Hamano wrote:\n\n> > 1. git am /the/patch\n> > 2. patch -p1 <.git/rebase-apply/patch\n> > 3. manually inspect the results for sanity, and fix up the cache.h\n> > bit that failed totally\n> > 4. git add -u && git add notes.[ch]\n> > 5. git am --resolved\n> \n> I usually skip 2-4 and edit .git/rebase-apply/patch in place instead, and\n> run \"git am\" instead of step 5.\n\nHow do you track down the source of the conflict to do the patch fixup?\nIn this case, it was a context line in the patch that had been deleted\nin my version. Do you just find the appropriate chunk in what you have\nalready and visually compare?\n\n-Peff\n"},{"id":"98431","messageId":"alpine.DEB.1.00.0812201250570.30769@pacific.mpi-cbg.de","threadId":"16749","inReplyTo":"200812191749.43512.bss@iguanasuicide.net","subject":"Re: [PATCH 4/4] Add an expensive test for git-notes","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-12-20T11:51:48Z","receivedAt":"2008-12-20T11:51:48Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Fri, 19 Dec 2008, Boyd Stephen Smith Jr. wrote:\n\n> On Friday 2008 December 19 17:37:15 Johannes Schindelin wrote:\n> > +       test_expect_success 'notes timing' \"time_notes 100\"\n> ^^^\n> Probably should be ${count}.\n\nNo. It times a git log 100 times (actually, 99 times due to a thinko). \nThis is only to protect against jitter, otherwise I'd do it only once.\n\nHth,\nDscho"},{"id":"98432","messageId":"alpine.DEB.1.00.0812201304210.30769@pacific.mpi-cbg.de","threadId":"16749","inReplyTo":"20081220065337.GA2581@coredump.intra.peff.net","subject":"[PATCH v2 0/4] Notes, reloaded","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-12-20T12:04:34Z","receivedAt":"2008-12-20T12:04:34Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"\nMy apologies; I forgot to rebase the series from my personal fork to Junio's\nnext. This is the version that should apply cleanly.\n\nJohannes Schindelin (4):\n Introduce commit notes\n Add a script to edit/inspect notes\n Speed up git notes lookup\n Add an expensive test for git-notes\n\n .gitignore | 1 +\n Documentation/config.txt | 15 ++++\n Documentation/git-notes.txt | 46 +++++++++++\n Makefile | 3 +\n cache.h | 3 +\n command-list.txt | 1 +\n commit.c | 1 +\n config.c | 5 +\n environment.c | 1 +\n git-notes.sh | 65 +++++++++++++++\n notes.c | 159 ++++++++++++++++++++++++++++++++++++++\n notes.h | 7 ++\n pretty.c | 5 +\n t/t3301-notes.sh | 65 +++++++++++++++\n t/t3302-notes-index-expensive.sh | 98 +++++++++++++++++++++++\n 15 files changed, 475 insertions(+), 0 deletions(-)\n create mode 100644 Documentation/git-notes.txt\n create mode 100755 git-notes.sh\n create mode 100644 notes.c\n create mode 100644 notes.h\n create mode 100755 t/t3301-notes.sh\n create mode 100755 t/t3302-notes-index-expensive.sh\n"},{"id":"98433","messageId":"alpine.DEB.1.00.0812201304580.30769@pacific.mpi-cbg.de","threadId":"16749","inReplyTo":"alpine.DEB.1.00.0812201304210.30769@pacific.mpi-cbg.de","subject":"[PATCH v2 1/4] Introduce commit notes","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-12-20T12:05:14Z","receivedAt":"2008-12-20T12:05:14Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"\nCommit notes are blobs which are shown together with the commit\nmessage. These blobs are taken from the notes ref, which you can\nconfigure by the config variable core.notesRef, which in turn can\nbe overridden by the environment variable GIT_NOTES_REF.\n\nThe notes ref is a branch which contains \"files\" whose names are\nthe names of the corresponding commits (i.e. the SHA-1).\n\nThe rationale for putting this information into a ref is this: we\nwant to be able to fetch and possibly union-merge the notes,\nmaybe even look at the date when a note was introduced, and we\nwant to store them efficiently together with the other objects.\n\nSigned-off-by: Johannes Schindelin \n---\n Documentation/config.txt | 15 ++++++++++\n Makefile | 2 +\n cache.h | 3 ++\n commit.c | 1 +\n config.c | 5 +++\n environment.c | 1 +\n notes.c | 68 ++++++++++++++++++++++++++++++++++++++++++++++\n notes.h | 7 +++++\n pretty.c | 5 +++\n 9 files changed, 107 insertions(+), 0 deletions(-)\n create mode 100644 notes.c\n create mode 100644 notes.h\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 21ea165..b35a32a 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -422,6 +422,21 @@ relatively high IO latencies. With this set to 'true', git will do the\n index comparison to the filesystem data in parallel, allowing\n overlapping IO's.\n \n+core.notesRef::\n+\tWhen showing commit messages, also show notes which are stored in\n+\tthe given ref. This ref is expected to contain paths of the form\n+\t??/*, where the directory name consists of the first two\n+\tcharacters of the commit name, and the base name consists of\n+\tthe remaining 38 characters.\n++\n+If such a path exists in the given ref, the referenced blob is read, and\n+appended to the commit message, separated by a \"Notes:\" line. If the\n+given ref itself does not exist, it is not an error, but means that no\n+notes should be print.\n++\n+This setting defaults to \"refs/notes/commits\", and can be overridden by\n+the `GIT_NOTES_REF` environment variable.\n+\n alias.*::\n \tCommand aliases for the linkgit:git[1] command wrapper - e.g.\n \tafter defining \"alias.last = cat-file commit HEAD\", the invocation\ndiff --git a/Makefile b/Makefile\nindex 5fcff9a..dc0a324 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -370,6 +370,7 @@ LIB_H += ll-merge.h\n LIB_H += log-tree.h\n LIB_H += mailmap.h\n LIB_H += merge-recursive.h\n+LIB_H += notes.h\n LIB_H += object.h\n LIB_H += pack.h\n LIB_H += pack-refs.h\n@@ -451,6 +452,7 @@ LIB_OBJS += match-trees.o\n LIB_OBJS += merge-file.o\n LIB_OBJS += merge-recursive.o\n LIB_OBJS += name-hash.o\n+LIB_OBJS += notes.o\n LIB_OBJS += object.o\n LIB_OBJS += pack-check.o\n LIB_OBJS += pack-refs.o\ndiff --git a/cache.h b/cache.h\nindex 9dabcc7..9192853 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -375,6 +375,8 @@ static inline enum object_type object_type(unsigned int mode)\n #define GITATTRIBUTES_FILE \".gitattributes\"\n #define INFOATTRIBUTES_FILE \"info/attributes\"\n #define ATTRIBUTE_MACRO_PREFIX \"[attr]\"\n+#define GIT_NOTES_REF_ENVIRONMENT \"GIT_NOTES_REF\"\n+#define GIT_NOTES_DEFAULT_REF \"refs/notes/commits\"\n \n extern int is_bare_repository_cfg;\n extern int is_bare_repository(void);\n@@ -546,6 +548,7 @@ enum rebase_setup_type {\n \n extern enum branch_track git_branch_track;\n extern enum rebase_setup_type autorebase;\n+extern char *notes_ref_name;\n \n #define GIT_REPO_VERSION 0\n extern int repository_format_version;\ndiff --git a/commit.c b/commit.c\nindex c99db16..10e532a 100644\n--- a/commit.c\n+++ b/commit.c\n@@ -5,6 +5,7 @@\n #include \"utf8.h\"\n #include \"diff.h\"\n #include \"revision.h\"\n+#include \"notes.h\"\n \n int save_commit_buffer = 1;\n \ndiff --git a/config.c b/config.c\nindex 790405a..e5d5b4b 100644\n--- a/config.c\n+++ b/config.c\n@@ -469,6 +469,11 @@ static int git_default_core_config(const char *var, const char *value)\n \t\treturn 0;\n \t}\n \n+\tif (!strcmp(var, \"core.notesref\")) {\n+\t\tnotes_ref_name = xstrdup(value);\n+\t\treturn 0;\n+\t}\n+\n \tif (!strcmp(var, \"core.pager\"))\n \t\treturn git_config_string(&pager_program, var, value);\n \ndiff --git a/environment.c b/environment.c\nindex e278bce..0edae21 100644\n--- a/environment.c\n+++ b/environment.c\n@@ -45,6 +45,7 @@ enum rebase_setup_type autorebase = AUTOREBASE_NEVER;\n \n /* Parallel index stat data preload? */\n int core_preload_index = 0;\n+char *notes_ref_name;\n \n /* This is set by setup_git_dir_gently() and/or git_default_config() */\n char *git_work_tree_cfg;\ndiff --git a/notes.c b/notes.c\nnew file mode 100644\nindex 0000000..91ec77f\n--- /dev/null\n+++ b/notes.c\n@@ -0,0 +1,68 @@\n+#include \"cache.h\"\n+#include \"commit.h\"\n+#include \"notes.h\"\n+#include \"refs.h\"\n+#include \"utf8.h\"\n+#include \"strbuf.h\"\n+\n+static int initialized;\n+\n+void get_commit_notes(const struct commit *commit, struct strbuf *sb,\n+\t\tconst char *output_encoding)\n+{\n+\tstatic const char *utf8 = \"utf-8\";\n+\tstruct strbuf name = STRBUF_INIT;\n+\tconst char *hex;\n+\tunsigned char sha1[20];\n+\tchar *msg;\n+\tunsigned long msgoffset, msglen;\n+\tenum object_type type;\n+\n+\tif (!initialized) {\n+\t\tconst char *env = getenv(GIT_NOTES_REF_ENVIRONMENT);\n+\t\tif (env)\n+\t\t\tnotes_ref_name = getenv(GIT_NOTES_REF_ENVIRONMENT);\n+\t\telse if (!notes_ref_name)\n+\t\t\tnotes_ref_name = GIT_NOTES_DEFAULT_REF;\n+\t\tif (notes_ref_name && read_ref(notes_ref_name, sha1))\n+\t\t\tnotes_ref_name = NULL;\n+\t\tinitialized = 1;\n+\t}\n+\n+\tif (!notes_ref_name)\n+\t\treturn;\n+\n+\tstrbuf_addf(&name, \"%s:%s\", notes_ref_name,\n+\t\t\tsha1_to_hex(commit->object.sha1));\n+\tif (get_sha1(name.buf, sha1))\n+\t\treturn;\n+\n+\tif (!(msg = read_sha1_file(sha1, &type, &msglen)) || !msglen ||\n+\t\t\ttype != OBJ_BLOB)\n+\t\treturn;\n+\n+\tif (output_encoding && *output_encoding &&\n+\t\t\tstrcmp(utf8, output_encoding)) {\n+\t\tchar *reencoded = reencode_string(msg, output_encoding, utf8);\n+\t\tif (reencoded) {\n+\t\t\tfree(msg);\n+\t\t\tmsg = reencoded;\n+\t\t\tmsglen = strlen(msg);\n+\t\t}\n+\t}\n+\n+\t/* we will end the annotation by a newline anyway */\n+\tif (msglen && msg[msglen - 1] == '\\n')\n+\t\tmsglen--;\n+\n+\tstrbuf_addstr(sb, \"\\nNotes:\\n\");\n+\n+\tfor (msgoffset = 0; msgoffset < msglen;) {\n+\t\tint linelen = strchrnul(msg, '\\n') - msg;\n+\n+\t\tstrbuf_addstr(sb, \" \");\n+\t\tstrbuf_add(sb, msg + msgoffset, linelen);\n+\t\tmsgoffset += linelen;\n+\t}\n+\tfree(msg);\n+}\ndiff --git a/notes.h b/notes.h\nnew file mode 100644\nindex 0000000..79d21b6\n--- /dev/null\n+++ b/notes.h\n@@ -0,0 +1,7 @@\n+#ifndef NOTES_H\n+#define NOTES_H\n+\n+void get_commit_notes(const struct commit *commit, struct strbuf *sb,\n+\t\tconst char *output_encoding);\n+\n+#endif\ndiff --git a/pretty.c b/pretty.c\nindex f6ff312..2d2872f 100644\n--- a/pretty.c\n+++ b/pretty.c\n@@ -6,6 +6,7 @@\n #include \"string-list.h\"\n #include \"mailmap.h\"\n #include \"log-tree.h\"\n+#include \"notes.h\"\n \n static char *user_format;\n \n@@ -881,5 +882,9 @@ void pretty_print_commit(enum cmit_fmt fmt, const struct commit *commit,\n \t */\n \tif (fmt == CMIT_FMT_EMAIL && sb->len <= beginning_of_body)\n \t\tstrbuf_addch(sb, '\\n');\n+\n+\tif (fmt != CMIT_FMT_ONELINE)\n+\t\tget_commit_notes(commit, sb, encoding);\n+\n \tfree(reencoded);\n }\n-- \n1.6.1.rc3.412.ga72b\n"},{"id":"98434","messageId":"alpine.DEB.1.00.0812201305230.30769@pacific.mpi-cbg.de","threadId":"16749","inReplyTo":"alpine.DEB.1.00.0812201304210.30769@pacific.mpi-cbg.de","subject":"[PATCH v2 2/4] Add a script to edit/inspect notes","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-12-20T12:05:33Z","receivedAt":"2008-12-20T12:05:33Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"\nThe script 'git notes' allows you to edit and show commit notes, by\ncalling either\n\n\tgit notes show \n\nor\n\n\tgit notes edit \n\nSigned-off-by: Johannes Schindelin \n---\n .gitignore | 1 +\n Documentation/git-notes.txt | 46 ++++++++++++++++++++++++++++++\n Makefile | 1 +\n command-list.txt | 1 +\n git-notes.sh | 65 +++++++++++++++++++++++++++++++++++++++++++\n t/t3301-notes.sh | 65 +++++++++++++++++++++++++++++++++++++++++++\n 6 files changed, 179 insertions(+), 0 deletions(-)\n create mode 100644 Documentation/git-notes.txt\n create mode 100755 git-notes.sh\n create mode 100755 t/t3301-notes.sh\n\ndiff --git a/.gitignore b/.gitignore\nindex 327e660..ac5fbf4 100644\n--- a/.gitignore\n+++ b/.gitignore\n@@ -82,6 +82,7 @@ git-mktag\n git-mktree\n git-name-rev\n git-mv\n+git-notes\n git-pack-redundant\n git-pack-objects\n git-pack-refs\ndiff --git a/Documentation/git-notes.txt b/Documentation/git-notes.txt\nnew file mode 100644\nindex 0000000..3d93625\n--- /dev/null\n+++ b/Documentation/git-notes.txt\n@@ -0,0 +1,46 @@\n+git-notes(1)\n+============\n+\n+NAME\n+----\n+git-notes - Add/inspect commit notes\n+\n+SYNOPSIS\n+--------\n+[verse]\n+'git-notes' (edit | show) [commit]\n+\n+DESCRIPTION\n+-----------\n+This command allows you to add notes to commit messages, without\n+changing the commit. To discern these notes from the message stored\n+in the commit object, the notes are indented like the message, after\n+an unindented line saying \"Notes:\".\n+\n+To disable commit notes, you have to set the config variable\n+core.notesRef to the empty string. Alternatively, you can set it\n+to a different ref, something like \"refs/notes/bugzilla\". This setting\n+can be overridden by the environment variable \"GIT_NOTES_REF\".\n+\n+\n+SUBCOMMANDS\n+-----------\n+\n+edit::\n+\tEdit the notes for a given commit (defaults to HEAD).\n+\n+show::\n+\tShow the notes for a given commit (defaults to HEAD).\n+\n+\n+Author\n+------\n+Written by Johannes Schindelin \n+\n+Documentation\n+-------------\n+Documentation by Johannes Schindelin\n+\n+GIT\n+---\n+Part of the gitlink:git[7] suite\ndiff --git a/Makefile b/Makefile\nindex dc0a324..5082f0f 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -258,6 +258,7 @@ SCRIPT_SH += git-merge-octopus.sh\n SCRIPT_SH += git-merge-one-file.sh\n SCRIPT_SH += git-merge-resolve.sh\n SCRIPT_SH += git-mergetool.sh\n+SCRIPT_SH += git-notes.sh\n SCRIPT_SH += git-parse-remote.sh\n SCRIPT_SH += git-pull.sh\n SCRIPT_SH += git-quiltimport.sh\ndiff --git a/command-list.txt b/command-list.txt\nindex 3583a33..2dc2c33 100644\n--- a/command-list.txt\n+++ b/command-list.txt\n@@ -73,6 +73,7 @@ git-mktag plumbingmanipulators\n git-mktree plumbingmanipulators\n git-mv mainporcelain common\n git-name-rev plumbinginterrogators\n+git-notes mainporcelain\n git-pack-objects plumbingmanipulators\n git-pack-redundant plumbinginterrogators\n git-pack-refs ancillarymanipulators\ndiff --git a/git-notes.sh b/git-notes.sh\nnew file mode 100755\nindex 0000000..bfdbaa8\n--- /dev/null\n+++ b/git-notes.sh\n@@ -0,0 +1,65 @@\n+#!/bin/sh\n+\n+USAGE=\"(edit | show) [commit]\"\n+. git-sh-setup\n+\n+test -n \"$3\" && usage\n+\n+test -z \"$1\" && usage\n+ACTION=\"$1\"; shift\n+\n+test -z \"$GIT_NOTES_REF\" && GIT_NOTES_REF=\"$(git config core.notesref)\"\n+test -z \"$GIT_NOTES_REF\" && GIT_NOTES_REF=\"refs/notes/commits\"\n+\n+COMMIT=$(git rev-parse --verify --default HEAD \"$@\") ||\n+die \"Invalid commit: $@\"\n+\n+MESSAGE=\"$GIT_DIR\"/new-notes-$COMMIT\n+trap '\n+\ttest -f \"$MESSAGE\" && rm \"$MESSAGE\"\n+' 0\n+\n+case \"$ACTION\" in\n+edit)\n+\tGIT_NOTES_REF= git log -1 $COMMIT | sed \"s/^/#/\" > \"$MESSAGE\"\n+\n+\tGIT_INDEX_FILE=\"$MESSAGE\".idx\n+\texport GIT_INDEX_FILE\n+\n+\tCURRENT_HEAD=$(git show-ref \"$GIT_NOTES_REF\" | cut -f 1 -d ' ')\n+\tif [ -z \"$CURRENT_HEAD\" ]; then\n+\t\tPARENT=\n+\telse\n+\t\tPARENT=\"-p $CURRENT_HEAD\"\n+\t\tgit read-tree \"$GIT_NOTES_REF\" || die \"Could not read index\"\n+\t\tgit cat-file blob :$COMMIT >> \"$MESSAGE\" 2> /dev/null\n+\tfi\n+\n+\t${VISUAL:-${EDITOR:-vi}} \"$MESSAGE\"\n+\n+\tgrep -v ^# < \"$MESSAGE\" | git stripspace > \"$MESSAGE\".processed\n+\tmv \"$MESSAGE\".processed \"$MESSAGE\"\n+\tif [ -s \"$MESSAGE\" ]; then\n+\t\tBLOB=$(git hash-object -w \"$MESSAGE\") ||\n+\t\t\tdie \"Could not write into object database\"\n+\t\tgit update-index --add --cacheinfo 0644 $BLOB $COMMIT ||\n+\t\t\tdie \"Could not write index\"\n+\telse\n+\t\ttest -z \"$CURRENT_HEAD\" &&\n+\t\t\tdie \"Will not initialise with empty tree\"\n+\t\tgit update-index --force-remove $COMMIT ||\n+\t\t\tdie \"Could not update index\"\n+\tfi\n+\n+\tTREE=$(git write-tree) || die \"Could not write tree\"\n+\tNEW_HEAD=$(echo Annotate $COMMIT | git commit-tree $TREE $PARENT) ||\n+\t\tdie \"Could not annotate\"\n+\tgit update-ref -m \"Annotate $COMMIT\" \\\n+\t\t\"$GIT_NOTES_REF\" $NEW_HEAD $CURRENT_HEAD\n+;;\n+show)\n+\tgit show \"$GIT_NOTES_REF\":$COMMIT\n+;;\n+*)\n+\tusage\n+esac\ndiff --git a/t/t3301-notes.sh b/t/t3301-notes.sh\nnew file mode 100755\nindex 0000000..ba42c45\n--- /dev/null\n+++ b/t/t3301-notes.sh\n@@ -0,0 +1,65 @@\n+#!/bin/sh\n+#\n+# Copyright (c) 2007 Johannes E. Schindelin\n+#\n+\n+test_description='Test commit notes'\n+\n+. ./test-lib.sh\n+\n+cat > fake_editor.sh << \\EOF\n+echo \"$MSG\" > \"$1\"\n+echo \"$MSG\" >& 2\n+EOF\n+chmod a+x fake_editor.sh\n+VISUAL=./fake_editor.sh\n+export VISUAL\n+\n+test_expect_success 'cannot annotate non-existing HEAD' '\n+\t! MSG=3 git notes edit\n+'\n+\n+test_expect_success setup '\n+\t: > a1 &&\n+\tgit add a1 &&\n+\ttest_tick &&\n+\tgit commit -m 1st &&\n+\t: > a2 &&\n+\tgit add a2 &&\n+\ttest_tick &&\n+\tgit commit -m 2nd\n+'\n+\n+test_expect_success 'need valid notes ref' '\n+\t! MSG=1 GIT_NOTES_REF='/' git notes edit &&\n+\t! MSG=2 GIT_NOTES_REF='/' git notes show\n+'\n+\n+test_expect_success 'create notes' '\n+\tgit config core.notesRef refs/notes/commits &&\n+\tMSG=b1 git notes edit &&\n+\ttest ! -f .git/new-notes &&\n+\ttest 1 = $(git ls-tree refs/notes/commits | wc -l) &&\n+\ttest b1 = $(git notes show) &&\n+\tgit show HEAD^ &&\n+\t! git notes show HEAD^\n+'\n+\n+cat > expect << EOF\n+commit 268048bfb8a1fb38e703baceb8ab235421bf80c5\n+Author: A U Thor \n+Date: Thu Apr 7 15:14:13 2005 -0700\n+\n+ 2nd\n+\n+Notes:\n+ b1\n+EOF\n+\n+test_expect_success 'show notes' '\n+\t! (git cat-file commit HEAD | grep b1) &&\n+\tgit log -1 > output &&\n+\tgit diff expect output\n+'\n+\n+test_done\n-- \n1.6.1.rc3.412.ga72b\n"},{"id":"98436","messageId":"alpine.DEB.1.00.0812201305390.30769@pacific.mpi-cbg.de","threadId":"16749","inReplyTo":"alpine.DEB.1.00.0812201304210.30769@pacific.mpi-cbg.de","subject":"[PATCH v2 3/4] Speed up git notes lookup","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-12-20T12:05:47Z","receivedAt":"2008-12-20T12:05:47Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"\nTo avoid looking up each and every commit in the notes ref's tree\nobject, which is very expensive, speed things up by slurping the tree\nobject's contents into a hash_map.\n\nThe idea fo the hashmap singleton is from David Reiss, initial\nbenchmarking by Jeff King.\n\nNote: the implementation allows for arbitrary entries in the notes\ntree object, ignoring those that do not reference a valid object. This\nallows you to annotate arbitrary branches, or objects.\n\nSigned-off-by: Johannes Schindelin \n---\n notes.c | 113 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++------\n 1 files changed, 102 insertions(+), 11 deletions(-)\n\ndiff --git a/notes.c b/notes.c\nindex 91ec77f..68bcb24 100644\n--- a/notes.c\n+++ b/notes.c\n@@ -4,16 +4,112 @@\n #include \"refs.h\"\n #include \"utf8.h\"\n #include \"strbuf.h\"\n+#include \"tree-walk.h\"\n+\n+struct entry {\n+\tunsigned char commit_sha1[20];\n+\tunsigned char notes_sha1[20];\n+};\n+\n+struct hash_map {\n+\tstruct entry *entries;\n+\toff_t count, size;\n+};\n \n static int initialized;\n+static struct hash_map hash_map;\n+\n+static int hash_index(struct hash_map *map, const unsigned char *sha1)\n+{\n+\tint i = ((*(unsigned int *)sha1) % map->size);\n+\n+\tfor (;;) {\n+\t\tunsigned char *current = map->entries[i].commit_sha1;\n+\n+\t\tif (!hashcmp(sha1, current))\n+\t\t\treturn i;\n+\n+\t\tif (is_null_sha1(current))\n+\t\t\treturn -1 - i;\n+\n+\t\tif (++i == map->size)\n+\t\t\ti = 0;\n+\t}\n+}\n+\n+static void add_entry(const unsigned char *commit_sha1,\n+\t\tconst unsigned char *notes_sha1)\n+{\n+\tint index;\n+\n+\tif (hash_map.count + 1 > hash_map.size >> 1) {\n+\t\tint i, old_size = hash_map.size;\n+\t\tstruct entry *old = hash_map.entries;\n+\n+\t\thash_map.size = old_size ? old_size << 1 : 64;\n+\t\thash_map.entries = (struct entry *)\n+\t\t\txcalloc(sizeof(struct entry), hash_map.size);\n+\n+\t\tfor (i = 0; i < old_size; i++)\n+\t\t\tif (!is_null_sha1(old[i].commit_sha1)) {\n+\t\t\t\tindex = -1 - hash_index(&hash_map,\n+\t\t\t\t\t\told[i].commit_sha1);\n+\t\t\t\tmemcpy(hash_map.entries + index, old + i,\n+\t\t\t\t\tsizeof(struct entry));\n+\t\t\t}\n+\t\tfree(old);\n+\t}\n+\n+\tindex = hash_index(&hash_map, commit_sha1);\n+\tif (index < 0) {\n+\t\tindex = -1 - index;\n+\t\thash_map.count++;\n+\t}\n+\n+\thashcpy(hash_map.entries[index].commit_sha1, commit_sha1);\n+\thashcpy(hash_map.entries[index].notes_sha1, notes_sha1);\n+}\n+\n+static void initialize_hash_map(const char *notes_ref_name)\n+{\n+\tunsigned char sha1[20], commit_sha1[20];\n+\tunsigned *mode;\n+\tstruct tree_desc desc;\n+\tstruct name_entry entry;\n+\tvoid *buf;\n+\n+\tif (!notes_ref_name || read_ref(notes_ref_name, commit_sha1) ||\n+\t\t\tget_tree_entry(commit_sha1, \"\", sha1, mode))\n+\t\treturn;\n+\n+\tbuf = fill_tree_descriptor(&desc, sha1);\n+\tif (!buf)\n+\t\tdie (\"Could not read %s for notes-index\", sha1_to_hex(sha1));\n+\n+\twhile (tree_entry(&desc, &entry))\n+\t\tif (!get_sha1(entry.path, commit_sha1))\n+\t\t\tadd_entry(commit_sha1, entry.sha1);\n+\tfree(buf);\n+}\n+\n+static unsigned char *lookup_notes(const unsigned char *commit_sha1)\n+{\n+\tint index;\n+\n+\tif (!hash_map.size)\n+\t\treturn NULL;\n+\n+\tindex = hash_index(&hash_map, commit_sha1);\n+\tif (index < 0)\n+\t\treturn NULL;\n+\treturn hash_map.entries[index].notes_sha1;\n+}\n \n void get_commit_notes(const struct commit *commit, struct strbuf *sb,\n \t\tconst char *output_encoding)\n {\n \tstatic const char *utf8 = \"utf-8\";\n-\tstruct strbuf name = STRBUF_INIT;\n-\tconst char *hex;\n-\tunsigned char sha1[20];\n+\tunsigned char *sha1;\n \tchar *msg;\n \tunsigned long msgoffset, msglen;\n \tenum object_type type;\n@@ -24,17 +120,12 @@ void get_commit_notes(const struct commit *commit, struct strbuf *sb,\n \t\t\tnotes_ref_name = getenv(GIT_NOTES_REF_ENVIRONMENT);\n \t\telse if (!notes_ref_name)\n \t\t\tnotes_ref_name = GIT_NOTES_DEFAULT_REF;\n-\t\tif (notes_ref_name && read_ref(notes_ref_name, sha1))\n-\t\t\tnotes_ref_name = NULL;\n+\t\tinitialize_hash_map(notes_ref_name);\n \t\tinitialized = 1;\n \t}\n \n-\tif (!notes_ref_name)\n-\t\treturn;\n-\n-\tstrbuf_addf(&name, \"%s:%s\", notes_ref_name,\n-\t\t\tsha1_to_hex(commit->object.sha1));\n-\tif (get_sha1(name.buf, sha1))\n+\tsha1 = lookup_notes(commit->object.sha1);\n+\tif (!sha1)\n \t\treturn;\n \n \tif (!(msg = read_sha1_file(sha1, &type, &msglen)) || !msglen ||\n-- \n1.6.1.rc3.412.ga72b\n"},{"id":"98435","messageId":"alpine.DEB.1.00.0812201305530.30769@pacific.mpi-cbg.de","threadId":"16749","inReplyTo":"alpine.DEB.1.00.0812201304210.30769@pacific.mpi-cbg.de","subject":"[PATCH v2 4/4] Add an expensive test for git-notes","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-12-20T12:06:03Z","receivedAt":"2008-12-20T12:06:03Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"\ngit-notes have the potential of being pretty expensive, so test with\na lot of commits. A lot. So to make things cheaper, you have to\nopt-in explicitely, by setting the environment variable\nGIT_NOTES_TIMING_TESTS.\n\nSigned-off-by: Johannes Schindelin \n---\n t/t3302-notes-index-expensive.sh | 98 ++++++++++++++++++++++++++++++++++++++\n 1 files changed, 98 insertions(+), 0 deletions(-)\n create mode 100755 t/t3302-notes-index-expensive.sh\n\ndiff --git a/t/t3302-notes-index-expensive.sh b/t/t3302-notes-index-expensive.sh\nnew file mode 100755\nindex 0000000..00d27bf\n--- /dev/null\n+++ b/t/t3302-notes-index-expensive.sh\n@@ -0,0 +1,98 @@\n+#!/bin/sh\n+#\n+# Copyright (c) 2007 Johannes E. Schindelin\n+#\n+\n+test_description='Test commit notes index (expensive!)'\n+\n+. ./test-lib.sh\n+\n+test -z \"$GIT_NOTES_TIMING_TESTS\" && {\n+\tsay Skipping timing tests\n+\ttest_done\n+\texit\n+}\n+\n+create_repo () {\n+\tnumber_of_commits=$1\n+\tnr=0\n+\tparent=\n+\ttest -d .git || {\n+\tgit init &&\n+\ttree=$(git write-tree) &&\n+\twhile [ $nr -lt $number_of_commits ]; do\n+\t\ttest_tick &&\n+\t\tcommit=$(echo $nr | git commit-tree $tree $parent) ||\n+\t\t\treturn\n+\t\tparent=\"-p $commit\"\n+\t\tnr=$(($nr+1))\n+\tdone &&\n+\tgit update-ref refs/heads/master $commit &&\n+\t{\n+\t\texport GIT_INDEX_FILE=.git/temp;\n+\t\tgit rev-list HEAD | cat -n | sed \"s/^[ \t][ \t]*/ /g\" |\n+\t\twhile read nr sha1; do\n+\t\t\tblob=$(echo note $nr | git hash-object -w --stdin) &&\n+\t\t\techo $sha1 | sed \"s/^/0644 $blob 0\t/\"\n+\t\tdone | git update-index --index-info &&\n+\t\ttree=$(git write-tree) &&\n+\t\ttest_tick &&\n+\t\tcommit=$(echo notes | git commit-tree $tree) &&\n+\t\tgit update-ref refs/notes/commits $commit\n+\t} &&\n+\tgit config core.notesRef refs/notes/commits\n+\t}\n+}\n+\n+test_notes () {\n+\tcount=$1 &&\n+\tgit config core.notesRef refs/notes/commits &&\n+\tgit log | grep \"^ \" > output &&\n+\ti=1 &&\n+\twhile [ $i -le $count ]; do\n+\t\techo \" $(($count-$i))\" &&\n+\t\techo \" note $i\" &&\n+\t\ti=$(($i+1));\n+\tdone > expect &&\n+\tgit diff expect output\n+}\n+\n+cat > time_notes << \\EOF\n+\tmode=$1\n+\ti=1\n+\twhile [ $i -lt $2 ]; do\n+\t\tcase $1 in\n+\t\tno-notes)\n+\t\t\texport GIT_NOTES_REF=non-existing\n+\t\t;;\n+\t\tnotes)\n+\t\t\tunset GIT_NOTES_REF\n+\t\t;;\n+\t\tesac\n+\t\tgit log >/dev/null\n+\t\ti=$(($i+1))\n+\tdone\n+EOF\n+\n+time_notes () {\n+\tfor mode in no-notes notes\n+\tdo\n+\t\techo $mode\n+\t\t/usr/bin/time sh ../time_notes $mode $1\n+\tdone\n+}\n+\n+for count in 10 100 1000 10000; do\n+\n+\tmkdir $count\n+\t(cd $count;\n+\n+\ttest_expect_success \"setup $count\" \"create_repo $count\"\n+\n+\ttest_expect_success 'notes work' \"test_notes $count\"\n+\n+\ttest_expect_success 'notes timing' \"time_notes 100\"\n+\t)\n+done\n+\n+test_done\n-- \n1.6.1.rc3.412.ga72b\n"},{"id":"98450","messageId":"7vhc4ywqvf.fsf@gitster.siamese.dyndns.org","threadId":"16749","inReplyTo":"20081220082304.GA5693@coredump.intra.peff.net","subject":"Re: [PATCH 1/4] Introduce commit notes","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-12-20T20:09:24Z","receivedAt":"2008-12-20T20:09:24Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> On Sat, Dec 20, 2008 at 12:17:46AM -0800, Junio C Hamano wrote:\n>\n>> > 1. git am /the/patch\n>> > 2. patch -p1 <.git/rebase-apply/patch\n>> > 3. manually inspect the results for sanity, and fix up the cache.h\n>> > bit that failed totally\n>> > 4. git add -u && git add notes.[ch]\n>> > 5. git am --resolved\n>> \n>> I usually skip 2-4 and edit .git/rebase-apply/patch in place instead, and\n>> run \"git am\" instead of step 5.\n>\n> How do you track down the source of the conflict to do the patch fixup?\n\nOld fashioned way, by looking at the patch and the file that the patch is\nsupposed to apply and reading the contexts.\n"}]}