{"thread":{"id":"22590","subject":"[RFD] Notes are independent: proposal for new notes implementation","startedAt":"2010-02-09T20:05:23Z","lastAt":"2010-02-10T04:51:50Z","messageCount":5,"participants":["Jakub Narebski","Avery Pennarun","Jeff King"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"134071","messageId":"201002092105.25636.jnareb@gmail.com","threadId":"22590","inReplyTo":null,"subject":"[RFD] Notes are independent: proposal for new notes implementation","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2010-02-09T20:05:23Z","receivedAt":"2010-02-09T20:05:23Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Junio have noticed in one of threads about notes implementation in\ngit[*1*] that current notes implementation has (conceptual) problems:\n\n[1] \"Re: A generalization of git notes from blobs to trees - git metadata?\"\n Message-ID: <7v8wb4aj4m.fsf@alter.siamese.dyndns.org>\n http://permalink.gmane.org/gmane.comp.version-control.git/139252\n\n (its one of threads that IIRC started with implementing hand-rolled\n support for notes in gitweb by Giuseppe Bilotta)\n\nJunio C Hamano writes:\nJH>\nJH> It's [current notes implementation] like \"keeping track of /etc\" (or\nJH> \"your home directory\"). It is a misguided thing to do because you\nJH> are throwing in records of the states of totally unrelated things\nJH> into a single history (e.g. \"Why does it matter I added new user \nJH> frotz to /etc/passwd before I futzed with my sendmail configuration?\nJH> ---It shouldn't matter; there shouldn't be ancestry relationships\nJH> between these two changes\"). I somehow feel that keeping track of\nJH> the \"growth of the bag of annotations to any and all commits\" in a\nJH> single history may be making the same mistake.\n\nThe proposed solution was to use custom merge strategy for notes. But\nwhat if the answer was to change implementation, decoupling history of\nnotes from each other, and keeping history of each note separate.\n\nLet's simplify situation, and talk for now about single notes namespace\n(refs/notes/commits), no fanout scheme, and plain blob notes.\n\n\nIn CURRENT notes implementation the notes ref (e.g. refs/notes/commits)\npoint to a commit object: the tip of history of all notes. This commit\nstores information about last change to any note; it's commit message is\n\"Annotate \". It's tree contains mapping between notes and\nannotated object: notes are stored as leafs in the tree, and their\npathnames are (representing) objects they are annotating.\n\nThis means for example that if in repository A somebody annotated\ncommits foo and bar creating notes in this order, and in repository B\nsomebody annotated bar and foo (creating notes in reverse order), then\nmerging those changes would require generating merge commit even if\nthose notes are identical.\n\n \n tree: <-- Annotate bar <-- refs/notes/commits\n |\n | (parent)\n |\n v\n tree: <-- Annotate foo\n (no parent)\n\n\nThe PROPOSED solution (with admittedly larger overhead) is to have notes\nhistory stored in submodule-like fashion. The notes ref would point to\nthe tree object. In this tree each leaf would point to a *commit*\nrepresenting tip of history for a given note (like for submodules).\nEach commit would contain tree, which would map note to annotated object\n(it is extra level of indirection, needed because commit cannot point to\nblob directly... unless multiple notes for the same commit in tree\nstructure got implemented, or tree annotations got implemented.)\n\nThis way history of each note is in kind of a separate branch, and notes\nrefs point to tree object representing branch hierarchy.\n\nMerge conflict would appear only if notes for the same object would have\ndifferent contents or/and different history.\n\n tree: <-- refs/notes/commits\n /------------ \n | /-- \n | |\n | v\n | Annotate bar --> tree:\n | (no parent) \n v\n Annotate foo ----------> tree\n (no parent) \n\nOne thing that would need to be addressed is converting from older notes\nimplementation, but this should be doable. The problem would be in\nsupporting both implementations in one repository; it might be not\npossible. Also this would break compatibility: older git versions\nsupporting notes wouldn't be able, I guess, to access new (proposed)\nformat.\n\nThere are probably numerous issues with proposed implementation, beside\nbreaking backward compatibility...\n\n\nP.S. This shows why git tools (such as gitweb) should not access notes\ndirectly, but use git-notes, %N / %N() format specifier, and proposed\n^@{} / object^@{} or ^{notes} / ^{notes:}\n\n-- \nJakub Narebski\nPoland\n"},{"id":"134073","messageId":"32541b131002091226p58b40237j40d3cd6cfaa91df5@mail.gmail.com","threadId":"22590","inReplyTo":"201002092105.25636.jnareb@gmail.com","subject":"Re: [RFD] Notes are independent: proposal for new notes implementation","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2010-02-09T20:26:27Z","receivedAt":"2010-02-09T20:26:27Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"2010/2/9 Jakub Narebski :\n> But\n> what if the answer was to change implementation, decoupling history of\n> notes from each other, and keeping history of each note separate.\n\nCongratulations, you've re-invented CVS! :)\n\nSeriously though, I'm not sure what problems this solved. Notes that\nare related to each other can (and perhaps should) be in the same\nnotes commit history; notes that are not related to each other can\nexist in separate histories with their own ref.\n\n> This means for example that if in repository A somebody annotated\n> commits foo and bar creating notes in this order, and in repository B\n> somebody annotated bar and foo (creating notes in reverse order), then\n> merging those changes would require generating merge commit even if\n> those notes are identical.\n\nThat's a feature; now you have the true history of your notes, which\nis good for all the same reasons it's good in git.\n\nOf course this whole line of reasoning could lead to questions like\n\"can I rebase my notes history?\" and \"what about rebase -i\" and \"can I\nmaintain a notes patch queue\" and so on.\n\nHave fun,\n\nAvery\n"},{"id":"134074","messageId":"201002092155.29304.jnareb@gmail.com","threadId":"22590","inReplyTo":"32541b131002091226p58b40237j40d3cd6cfaa91df5@mail.gmail.com","subject":"Re: [RFD] Notes are independent: proposal for new notes implementation","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2010-02-09T20:55:28Z","receivedAt":"2010-02-09T20:55:28Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"On Tue, 9 Feb 2010, Avery Pennarun wrote:\n> 2010/2/9 Jakub Narebski :\n\n> > But\n> > what if the answer was to change implementation, decoupling history of\n> > notes from each other, and keeping history of each note separate.\n> \n> Congratulations, you've re-invented CVS! :)\n> \n> Seriously though, I'm not sure what problems this solved. Notes that\n> are related to each other can (and perhaps should) be in the same\n> notes commit history; notes that are not related to each other can\n> exist in separate histories with their own ref.\n\nThe problem is (as I see it) that notes are _not_ (in almost all cases)\nrelated to each other, just like files in $HOME or in /etc are not\nrelated to each other. Separate notes refs for separate histories\nare not IMHO a good solution: refs namespaces are about *kind* (flavor)\nof notes: commits annotations, bisect, git-svn, apply-email, bugs / tickets,\netc. and each flavor (kind) of notes contain many independent notes.\n \nThis is opposed to workspace history, where history (in almost all cases)\nmakes sense only of all files, history of a project as a whole.\n\nAnd of course we would have atomic commits, merge tracking, support for\nrenames etc., something like Zit[1]\n\n[1]: http://git.wiki.kernel.org/index.php/InterfacesFrontendsAndTools#Zit\n\n> > This means for example that if in repository A somebody annotated\n> > commits foo and bar creating notes in this order, and in repository B\n> > somebody annotated bar and foo (creating notes in reverse order), then\n> > merging those changes would require generating merge commit even if\n> > those notes are identical.\n> \n> That's a feature; now you have the true history of your notes, which\n> is good for all the same reasons it's good in git.\n\nNo, you are introducing artificial ordering in something that is a bag,\nunordered collection.\n\n> Of course this whole line of reasoning could lead to questions like\n> \"can I rebase my notes history?\" and \"what about rebase -i\" and \"can I\n> maintain a notes patch queue\" and so on.\n\n-- \nJakub Narebski\nPoland\n"},{"id":"134077","messageId":"32541b131002091337t3848a87fg2d35ec4c198465b1@mail.gmail.com","threadId":"22590","inReplyTo":"201002092155.29304.jnareb@gmail.com","subject":"Re: [RFD] Notes are independent: proposal for new notes implementation","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2010-02-09T21:37:43Z","receivedAt":"2010-02-09T21:37:43Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On Tue, Feb 9, 2010 at 3:55 PM, Jakub Narebski wrote:\n> On Tue, 9 Feb 2010, Avery Pennarun wrote:\n>> 2010/2/9 Jakub Narebski :\n>> > But\n>> > what if the answer was to change implementation, decoupling history of\n>> > notes from each other, and keeping history of each note separate.\n>>\n>> Congratulations, you've re-invented CVS! :)\n>>\n>> Seriously though, I'm not sure what problems this solved.  Notes that\n>> are related to each other can (and perhaps should) be in the same\n>> notes commit history; notes that are not related to each other can\n>> exist in separate histories with their own ref.\n>\n> The problem is (as I see it) that notes are _not_ (in almost all cases)\n> related to each other, just like files in $HOME or in /etc are not\n> related to each other.\n\nAs a side note, I didn't find this example compelling at all. I\n*absolutely* want to manage all my files in /etc as a single repo.\n\"The configuration of my computer\" is an ongoing project where the\nconfiguration of my smtp daemon depends on /etc/hosts and /etc/passwd\nand /etc/group. If I set up another server, I want to be able to fork\nmy basic configuration and apply some patches. If I set up some\nclever aliases in /etc/profile, I want to send that patch \"upstream\"\nto the /etc project on my other servers.\n\nSimilarly with $HOME; the evolution of my home directory over time is\na thing I can talk about as a sensible whole, and of course I want\nrename tracking and deltas and so on.\n\nCombining /etc and $HOME into a single repo would be harder to\njustify. But that sounds to me like the \"kind (flavor)\" distinction\nyou're talking about; system config files and personal files are two\ndifferent kinds of files.\n\n>> > This means for example that if in repository A somebody annotated\n>> > commits foo and bar creating notes in this order, and in repository B\n>> > somebody annotated bar and foo (creating notes in reverse order), then\n>> > merging those changes would require generating merge commit even if\n>> > those notes are identical.\n>>\n>> That's a feature; now you have the true history of your notes, which\n>> is good for all the same reasons it's good in git.\n>\n> No, you are introducing artificial ordering in something that is a bag,\n> unordered collection.\n\nI would put it another way: you're recording the true ordering of\nsomething where the ordering *may* not be important. It is easy to\nignore that ordering.\n\nHowever, it's very hard to unignore ordering that you didn't record in\nthe first place. That's why CVS's model of recording\none-history-per-file is so nasty. Yet it seemed so clever when they\ninvented it.\n\nWhat is a real use case where the \"artificial ordering\" causes a problem?\n\nHere's a use case where having a single history would be a clear\nbenefit: say you're running an autobuilder such as gitbuilder[1].\nSomething goes wrong with your autobuild environment, like the disk\nfills up, and all the build results since yesterday at noon are\ninvalid. If the build results are stored as a single history of git\nnotes[2], I can just rewind the history to yesterday at noon and\ndiscard the entire sequence of bad results across all commits. If\nthey each had their own history, it would be more complex to\nimplement.\n\nHave fun,\n\nAvery\n\n[1] http://github.com/apenwarr/gitbuilder\n\n[2] gitbuilder doesn't currently use git-notes, it just uses plain files.\n"},{"id":"134116","messageId":"20100210045150.GB28526@coredump.intra.peff.net","threadId":"22590","inReplyTo":"201002092105.25636.jnareb@gmail.com","subject":"Re: [RFD] Notes are independent: proposal for new notes implementation","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2010-02-10T04:51:50Z","receivedAt":"2010-02-10T04:51:50Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Feb 09, 2010 at 09:05:23PM +0100, Jakub Narebski wrote:\n\n> The proposed solution was to use custom merge strategy for notes. But\n> what if the answer was to change implementation, decoupling history of\n> notes from each other, and keeping history of each note separate.\n\nIf I am understanding you correctly, instead of keeping a commit history\nof trees of many notes (one per sha1), we will have a tree of commit\nhistories, one history per sha1. What problem is this solving?\n\nIf I modify commit X and you modify commit Y, we avoid making a merge\ncommit. But so what? The merge would be trivial, since we did not modify\nthe same entries. The user never cares that there is a merge commit.\n\nAnd if we both _did_ edit commit X, both cases result in a merge.\n\nIf we both modified X and Y, then you will presumably do the merge for X\nand Y iteratively before you can create a new notes tree. Or you could\nmerge them separately. But why? Why would I want to pull some subset of\nyour notes (and keep in mind this is a subset of the commits you have\nnoted, not a semantically different notes namespace), and how would I\neven specify which notes were of interest and which were not?\n\nSo what is the concrete use case where this helps?\n\n-Peff\n"}]}