{"thread":{"id":"13028","subject":"Is my repository broken?","startedAt":"2008-04-08T22:02:16Z","lastAt":"2008-04-09T10:01:28Z","messageCount":6,"participants":["Julian Phillips","Junio C Hamano","Shawn O. Pearce"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"73884","messageId":"Pine.LNX.4.64.0804082246160.8753@kaos.quantumfyre.co.uk","threadId":"13028","inReplyTo":null,"subject":"Is my repository broken?","fromName":"Julian Phillips","fromEmail":"julian@quantumfyre.co.uk","sentAt":"2008-04-08T22:02:16Z","receivedAt":"2008-04-08T22:02:16Z","isPatch":false,"sender":{"key":"julian@quantumfyre.co.uk","avatar":"https://avatars.githubusercontent.com/u/948888?v=4"},"body":"Having just converted a subversion repository to git using a custom \nfast-import based script, I realised that I had forgotten to add some \nusers to the mapping table - so I thought that filter-branch would save me \nfrom having to redo the 72hr import.\n\nAt which point I discovered that I had a number of commits that git didn't \nlike. In particular I had a number of commits with an empty ident, and a \nnumber with more than 16 parents (actually mostly these are the same \ncommits - but I don't think that's relevant). When filter-branch tries to \nprocess these commits it can't commit them - and so I end up losing a \nnumber of refs.\n\nI can fix the empty ident using the msg-filter in filter-branch, and I can \nwork around the 16 parent limit by building a custom git with MAX_PARENT = \n32. However - will having commits with more than 16 parents break things \nfor un-modified git? The commits in question were originally created in \nCVS, so the parenting is kinda 'real' for some value of real - but we are \ntalking revision 4000/40000, so I don't mind arbitrarily limiting the \ncommits to 16 parents, but I would prefer to avoid redoing the 72hr import \nif possible. (Perhaps I can also 'fix' the parenting in filter-branch?)\n\nAlso, shouldn't fast-import be imposing the same restrictions on what you \nare allowed to commit that the main git tools do? If not, are such \nrestrictions documented so that I can apply them in my conversion script?\n\n(28G Subversion repository to 7.1G git repository (before repacking) ... \nnot bad :D)\n\n-- \nJulian\n\n ---\nAgnes' Law:\n \tAlmost everything in life is easier to get into than out of.\n"},{"id":"73885","messageId":"7vk5j8knas.fsf@gitster.siamese.dyndns.org","threadId":"13028","inReplyTo":"Pine.LNX.4.64.0804082246160.8753@kaos.quantumfyre.co.uk","subject":"Re: Is my repository broken?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-04-08T22:13:31Z","receivedAt":"2008-04-08T22:13:31Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Julian Phillips writes:\n\n> ... However - will having commits with more than 16\n> parents break things for un-modified git?\n\nOffhand, 82d853d (builtin-blame.c: allow more than 16 parents, 2008-04-03)\ncomes to mind. Also I would not be surprised if Porcelain scripts on the\nperiphery had trouble with commits with duplicated parents and such\nesotericos, though.\n"},{"id":"73894","messageId":"20080408225522.GD10274@spearce.org","threadId":"13028","inReplyTo":"Pine.LNX.4.64.0804082246160.8753@kaos.quantumfyre.co.uk","subject":"Re: Is my repository broken?","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2008-04-08T22:55:22Z","receivedAt":"2008-04-08T22:55:22Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Julian Phillips wrote:\n> [...] In particular I had a number of commits with an empty ident [...]\n...\n> Also, shouldn't fast-import be imposing the same restrictions on what you \n> are allowed to commit that the main git tools do? If not, are such \n> restrictions documented so that I can apply them in my conversion script?\n\nHmm, no. fast-import allows what the generalized data model permits\nin the object store, its really plumbing. If you are feeding it\nan input stream that creates data that isn't compliant with what\nthe higher level VCS porcelain wants, well, all I can say is \"don't\ndo that\".\n\nThe fast-import manual specifically warns in the \"merge\" command\ndocumentation that you may not want to use more than 15 merge\ncommands, as it can create a commit that other tools based around\ngit won't like. But we still let you do it.\n\nWe also still let you create a commit with duplicate parents.\nSome tools (gitk) have had issues with that in the past, but many of\nthem have been fixed after a fast-import result was used with them.\nFrom a VCS point of view its silly to list the same ancestor twice.\nBut from an object model point of view, it may make sense if you\nwere building something else on top of the core plumbing.\n\nThe same holds true for the empty ident.\n\n-- \nShawn.\n"},{"id":"73902","messageId":"Pine.LNX.4.64.0804090003480.10813@kaos.quantumfyre.co.uk","threadId":"13028","inReplyTo":"20080408225522.GD10274@spearce.org","subject":"Re: Is my repository broken?","fromName":"Julian Phillips","fromEmail":"julian@quantumfyre.co.uk","sentAt":"2008-04-08T23:21:26Z","receivedAt":"2008-04-08T23:21:26Z","isPatch":false,"sender":{"key":"julian@quantumfyre.co.uk","avatar":"https://avatars.githubusercontent.com/u/948888?v=4"},"body":"On Tue, 8 Apr 2008, Shawn O. Pearce wrote:\n\n> Julian Phillips wrote:\n>> [...] In particular I had a number of commits with an empty ident [...]\n> ...\n>> Also, shouldn't fast-import be imposing the same restrictions on what you\n>> are allowed to commit that the main git tools do? If not, are such\n>> restrictions documented so that I can apply them in my conversion script?\n>\n> Hmm, no. fast-import allows what the generalized data model permits\n> in the object store, its really plumbing. If you are feeding it\n> an input stream that creates data that isn't compliant with what\n> the higher level VCS porcelain wants, well, all I can say is \"don't\n> do that\".\n\nWell, the 16 parent limit is enforced in builtin-commit-tree.c, and the \ncommit-tree command is listed as plumbingmanipulators in command-list.txt \n- and the #define line actually blames all the way back to Linus' original \ncommit. So that's not really a porcelain thing ;), but ... *shrugs*\n\n> The fast-import manual specifically warns in the \"merge\" command\n> documentation that you may not want to use more than 15 merge\n> commands, as it can create a commit that other tools based around\n> git won't like. But we still let you do it.\n\nOk, so the answer is \"read the manpage\" ... that's what I get for spending \nso long playing with fast-import that I don't read the manpage anymore \n(I found the syntax description in fast-import.c a more useful \nreference) ... ho hum.\n\n> We also still let you create a commit with duplicate parents.\n\nWell - that certainly shouldn't be happening - I remebered to check for \nthat one.\n\n> Some tools (gitk) have had issues with that in the past, but many of\n> them have been fixed after a fast-import result was used with them.\n>> From a VCS point of view its silly to list the same ancestor twice.\n> But from an object model point of view, it may make sense if you\n> were building something else on top of the core plumbing.\n\nWould it make sense perhaps for fast-import to warn about (or even error \nout on) such things unless you tell it not to? Most people would probably \nwant to know if they were creating a repository that wasn't going to play \nnice with git's main tool suite? (I didn't realise that I _was_ creating \n>16 way merges until filter-branch told me).\n\n> The same holds true for the empty ident.\n\nOk - but I can't even find a note in the manpage for this one ...\n\n-- \nJulian\n\n ---\nI thought there was chocolate inside ... Well, why was it wrapped in foil?\n\n \t\t-- Homer Simpson\n \t\t Mr. Plow\n"},{"id":"73926","messageId":"7vbq4jcyj7.fsf@gitster.siamese.dyndns.org","threadId":"13028","inReplyTo":"Pine.LNX.4.64.0804090003480.10813@kaos.quantumfyre.co.uk","subject":"Re: Is my repository broken?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-04-09T06:50:20Z","receivedAt":"2008-04-09T06:50:20Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Julian Phillips writes:\n\n> On Tue, 8 Apr 2008, Shawn O. Pearce wrote:\n>\n>> The same holds true for the empty ident.\n>\n> Ok - but I can't even find a note in the manpage for this one ...\n\nThat's not a fair complaint.\n\nIt is often very hard to document that \"we do not do X\", because the line\nto stop at becomes fuzzier as you try to do more thorough job. We do not\nwarn on empty ident, we do not warn on typos in commit log messages, we do\nnot warn on empty blob, we do not warn on ... You get the idea.\n"},{"id":"73942","messageId":"Pine.LNX.4.64.0804091046170.14797@reaper.quantumfyre.co.uk","threadId":"13028","inReplyTo":"7vbq4jcyj7.fsf@gitster.siamese.dyndns.org","subject":"Re: Is my repository broken?","fromName":"Julian Phillips","fromEmail":"julian@quantumfyre.co.uk","sentAt":"2008-04-09T10:01:28Z","receivedAt":"2008-04-09T10:01:28Z","isPatch":false,"sender":{"key":"julian@quantumfyre.co.uk","avatar":"https://avatars.githubusercontent.com/u/948888?v=4"},"body":"On Tue, 8 Apr 2008, Junio C Hamano wrote:\n\n> Julian Phillips writes:\n>\n>> On Tue, 8 Apr 2008, Shawn O. Pearce wrote:\n>>\n>>> The same holds true for the empty ident.\n>>\n>> Ok - but I can't even find a note in the manpage for this one ...\n>\n> That's not a fair complaint.\n\nI didn't mean it as a complaint, but rather was hoping for a response of \nthe form \"no, it's not there\" or \"it's in the ... section\" - sorry for not \nbeing clear. It caught me out, if that was my own fault then fair enough \n- but if this was because the documentation doesn't make it clear then I \ncan submit a documentation patch to try and help others avoid the same \nproblem.\n\nI think that the fast-import tool is extremely useful, and generally very \nwell documented. That doesn't mean that the documentation can't be \nimproved though.\n\n> It is often very hard to document that \"we do not do X\", because the line\n> to stop at becomes fuzzier as you try to do more thorough job. We do not\n> warn on empty ident, we do not warn on typos in commit log messages, we do\n> not warn on empty blob, we do not warn on ... You get the idea.\n\nGit doesn't die when trying to commit typos though ... I think \"creating \na commit that you could not create using git-commit\" is a pretty hard \nline.\n\nI don't think that it is entirely unreasonable to expect that when an \nexisting repository is run through a tool like filter-branch that all your \nexisting commits are preserved - and that you don't lose large chunks \nbecause it turns out that they are actually invalid by the rules of \ngit-commit.\n\nI accept that you may want fast-import to create things that are \ntechnically illegal, but at the very least it ought to be possible to find \nout what restrictions are not being enforced. Otherwise it might be that \nyou manage to destroy a previously functioning repository by accident long \nafter you thought you had successfully converted your respository. After \nall, I would have thought that the majority of people using fast-import \n(either directly, or indirectly by using a fast-import based importer) \nwould actually intend to use the repository created with the normal git \ntools from then on.\n\n-- \nJulian\n\n ---\nI have often regretted my speech, never my silence.\n \t\t-- Publilius Syrus\n"}]}