{"thread":{"id":"25480","subject":"fetch and bundle don't work in (semi-)broken repo","startedAt":"2010-10-19T16:09:57Z","lastAt":"2010-10-22T03:53:37Z","messageCount":12,"participants":["Uwe Kleine-König","Jonathan Nieder","Nicolas Pitre"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"153794","messageId":"20101019160957.GK11713@pengutronix.de","threadId":"25480","inReplyTo":null,"subject":"fetch and bundle don't work in (semi-)broken repo","fromName":"Uwe Kleine-König","fromEmail":"u.kleine-koenig@pengutronix.de","sentAt":"2010-10-19T16:09:57Z","receivedAt":"2010-10-19T16:09:57Z","isPatch":false,"sender":{"key":"u.kleine-koenig@pengutronix.de","avatar":"https://gravatar.com/avatar/354b5e3ceb2806a2f1e1e382ac29ddbdad18288654da62b61eb13583a857eee7?d=mp&s=160"},"body":"Hi,\n\nI have a repo that got broken somehow (don't know the exact details,\nprobably because it is shared with another repo and I rewrote history).\nNow I want to fetch one branch to a different repo (that happens to be\nthe alternative to the first one, but I think this is unrelated.):\n\n\tukl@hostname:~/path1/linux-2.6$ git fetch ~/path2/linux-2.6 sectionmismatches\n\tremote: Counting objects: 118, done.\n\tremote: error: unable to find 40aaeb204dc04d3cf15c060133f65538b43b13b0\n\tremote: Compressing objects: 100% (83/83), done.\n\tremote: fatal: unable to read 40aaeb204dc04d3cf15c060133f65538b43b13b0\n\terror: git upload-pack: git-pack-objects died with error.\n\tfatal: git upload-pack: aborting due to possible repository corruption on the remote side.\n\tremote: aborting due to possible repository corruption on the remote side.\n\tfatal: protocol error: bad pack header\n\nI don't know what 40aaeb204dc04d3cf15c060133f65538b43b13b0 is, but I\nthink it's not necessary for the sectionmismatches branch:\n\n\tukl@hostname:~/path2/linux-2.6$ git format-patch linus/master..sectionmismatches\n\t0001-wip-enable-DEBUG_SECTION_MISMATCH.patch\n\t0002-ARM-sa1111-move-__sa1111_probe-to-.devinit.text.patch\n\t0003-ARM-omap1-nokia770-mark-some-functions-__init.patch\n\t0004-ARM-omap-fb-move-omap_init_fb-to-.init.text.patch\n\t0005-ARM-omap-fb-move-omapfb_reserve_sram-to-.init.text.patch\n\t0006-ARM-omap-fb-move-get_fbmem_region-to-.init.text.patch\n\t0007-ARM-omap-move-omap_get_config-et-al.-to-.init.text.patch\n\t0008-wip-ARM-omap-move-omap_board_config_kernel-to-.init..patch\n\t0009-ARM-omap-ams-delta-move-config-to-.init.data.patch\n\t0010-MTD-pxa2xx-move-pxa2xx_flash_probe-to-.devinit.text.patch\n\t0011-VIDEO-sa1100fb-register-driver-using-platform_driver.patch\n\t0012-ARM-s3c64xx-don-t-put-smartq_bl_init-in-.init.text.patch\n\t0013-ARM-s3c64xx-don-t-put-smartq7_leds-in-.init.data.patch\n\t0014-ARM-s3c64xx-don-t-put-smartq5_leds-in-.init.data.patch\n\t0015-ARM-nomadik-move-nmk_gpio_probe-to-.devinit.text.patch\n\nand linus/master is contained in ~/path1/linux-2.6, too.\n\nBundling doesn't work either:\n\n\tukl@hostname:~/path2/linux-2.6$ git bundle create tralala linus/master..sectionmismatches\n\tCounting objects: 118, done.\n\terror: unable to find 40aaeb204dc04d3cf15c060133f65538b43b13b0\n\tDelta compression using up to 8 threads.\n\tCompressing objects: 100% (83/83), done.\n\tfatal: unable to read 40aaeb204dc04d3cf15c060133f65538b43b13b0\n\terror: pack-objects died\n\ngit gc fails with a different object:\n\n\tukl@hostname:~/path2/linux-2.6$ git gc\n\terror: Could not read f6b6cb2336198913371e66664f28c135df01aea5\n\tfatal: Failed to traverse parents of commit 76d1acb95eef413a2501a63cb7f7f4036b71ed37\n\terror: failed to run repack\n\n(git gc is fine in ~/path1/linux-2.6)\n\n\tukl@hostname:~$ git version\n\tgit version 1.7.2.3\n\nAs a side note I have the general feeling that git recently started to\ndo more things than necessary (e.g. sometimes `git stash pop` takes >1s\n(not mesured) between the status stuff (up to \"no changes added to\ncommit (use ...)\") and \"Dropped refs/stash@{0} ($commitid)\".  I wonder\nwhat git stash is doing during that time.) I don't know if this is the\nproblem here, though.\n\nBest regards\nUwe\n\n-- \nPengutronix e.K.                           | Uwe Kleine-König            |\nIndustrial Linux Solutions                 | http://www.pengutronix.de/  |\n"},{"id":"153808","messageId":"20101019183925.GF25139@burratino","threadId":"25480","inReplyTo":"20101019160957.GK11713@pengutronix.de","subject":"Re: fetch and bundle don't work in (semi-)broken repo","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-10-19T18:39:25Z","receivedAt":"2010-10-19T18:39:25Z","isPatch":false,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Hi,\n\nUwe Kleine-König wrote:\n\n> \tukl@hostname:~/path1/linux-2.6$ git fetch ~/path2/linux-2.6 sectionmismatches\n> \tremote: Counting objects: 118, done.\n> \tremote: error: unable to find 40aaeb204dc04d3cf15c060133f65538b43b13b0\n> \tremote: Compressing objects: 100% (83/83), done.\n> \tremote: fatal: unable to read 40aaeb204dc04d3cf15c060133f65538b43b13b0\n> \terror: git upload-pack: git-pack-objects died with error.\n> \tfatal: git upload-pack: aborting due to possible repository corruption on the remote side.\n\nSounds like alternates or workdir allowed gc to be overzealous, indeed.\n\nCould you:\n\n 1. Make a copy of the corrupted repo, just in case.\n 2. Explode all backs with \"git unpack-objects\"\n 3. Identify the missing object, as explained in\n    Documentation/howto/recover-corrupted-blob-object.txt?\n\nWith that information, it would be easier to examine whether and how\npack-objects could be smarter about fetching the non-missing objects.\n\n> I don't know what 40aaeb204dc04d3cf15c060133f65538b43b13b0 is, but I\n> think it's not necessary for the sectionmismatches branch:\n> \n> \tukl@hostname:~/path2/linux-2.6$ git format-patch linus/master..sectionmismatches\n> \t0001-wip-enable-DEBUG_SECTION_MISMATCH.patch\n[...]\n> and linus/master is contained in ~/path1/linux-2.6, too.\n\nCc-ing Nico, pack-objects wizard.\n\nThanks for reporting.\nJonathan\n"},{"id":"153819","messageId":"20101019201139.GB28166@pengutronix.de","threadId":"25480","inReplyTo":"20101019183925.GF25139@burratino","subject":"Re: fetch and bundle don't work in (semi-)broken repo","fromName":"Uwe Kleine-König","fromEmail":"u.kleine-koenig@pengutronix.de","sentAt":"2010-10-19T20:11:39Z","receivedAt":"2010-10-19T20:11:39Z","isPatch":false,"sender":{"key":"u.kleine-koenig@pengutronix.de","avatar":"https://gravatar.com/avatar/354b5e3ceb2806a2f1e1e382ac29ddbdad18288654da62b61eb13583a857eee7?d=mp&s=160"},"body":"Hi Jonathan,\n\nOn Tue, Oct 19, 2010 at 01:39:25PM -0500, Jonathan Nieder wrote:\n> Uwe Kleine-König wrote:\n> \n> > \tukl@hostname:~/path1/linux-2.6$ git fetch ~/path2/linux-2.6 sectionmismatches\n> > \tremote: Counting objects: 118, done.\n> > \tremote: error: unable to find 40aaeb204dc04d3cf15c060133f65538b43b13b0\n> > \tremote: Compressing objects: 100% (83/83), done.\n> > \tremote: fatal: unable to read 40aaeb204dc04d3cf15c060133f65538b43b13b0\n> > \terror: git upload-pack: git-pack-objects died with error.\n> > \tfatal: git upload-pack: aborting due to possible repository corruption on the remote side.\n> \n> Sounds like alternates or workdir allowed gc to be overzealous, indeed.\n> \n> Could you:\n> \n>  1. Make a copy of the corrupted repo, just in case.\n>  2. Explode all backs with \"git unpack-objects\"\nI did:\n\n\tmv .git/objects/pack .git/objects/pack.bak\n\trm .git/objects/info/alternates\n\tfor p in .git/objects/pack.bak/*.pack ~/path1/linux-2.6/.git/objects/pack/*.pack; do\n\t\tgit unpack-objects < $p\n\tdone\n\n\n>  3. Identify the missing object, as explained in\n>     Documentation/howto/recover-corrupted-blob-object.txt?\n\nand I'm running git-fsck --full now over night as it's bedtime here.\n \n> With that information, it would be easier to examine whether and how\n> pack-objects could be smarter about fetching the non-missing objects.\n> \n> > I don't know what 40aaeb204dc04d3cf15c060133f65538b43b13b0 is, but I\n> > think it's not necessary for the sectionmismatches branch:\n> > \n> > \tukl@hostname:~/path2/linux-2.6$ git format-patch linus/master..sectionmismatches\n> > \t0001-wip-enable-DEBUG_SECTION_MISMATCH.patch\n> [...]\n> > and linus/master is contained in ~/path1/linux-2.6, too.\n> \n> Cc-ing Nico, pack-objects wizard.\n> \n> Thanks for reporting.\nThanks for helping\n\nBest regards\nUwe\n\n-- \nPengutronix e.K.                           | Uwe Kleine-König            |\nIndustrial Linux Solutions                 | http://www.pengutronix.de/  |\n"},{"id":"153820","messageId":"alpine.LFD.2.00.1010191635030.2764@xanadu.home","threadId":"25480","inReplyTo":"20101019201139.GB28166@pengutronix.de","subject":"Re: fetch and bundle don't work in (semi-)broken repo","fromName":"Nicolas Pitre","fromEmail":"nico@fluxnic.net","sentAt":"2010-10-19T20:48:23Z","receivedAt":"2010-10-19T20:48:23Z","isPatch":false,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Tue, 19 Oct 2010, Uwe Kleine-König wrote:\n\n> Hi Jonathan,\n> \n> On Tue, Oct 19, 2010 at 01:39:25PM -0500, Jonathan Nieder wrote:\n> > Uwe Kleine-König wrote:\n> > \n> > > \tukl@hostname:~/path1/linux-2.6$ git fetch ~/path2/linux-2.6 sectionmismatches\n> > > \tremote: Counting objects: 118, done.\n> > > \tremote: error: unable to find 40aaeb204dc04d3cf15c060133f65538b43b13b0\n> > > \tremote: Compressing objects: 100% (83/83), done.\n> > > \tremote: fatal: unable to read 40aaeb204dc04d3cf15c060133f65538b43b13b0\n> > > \terror: git upload-pack: git-pack-objects died with error.\n> > > \tfatal: git upload-pack: aborting due to possible repository corruption on the remote side.\n> > \n> > Sounds like alternates or workdir allowed gc to be overzealous, indeed.\n> > \n> > Could you:\n> > \n> >  1. Make a copy of the corrupted repo, just in case.\n> >  2. Explode all backs with \"git unpack-objects\"\n\nThat's usually unnecessary.  If the pack itself is corrupted, trying to \nunpack it all won't help.\n\n> I did:\n> \n> \tmv .git/objects/pack .git/objects/pack.bak\n> \trm .git/objects/info/alternates\n> \tfor p in .git/objects/pack.bak/*.pack ~/path1/linux-2.6/.git/objects/pack/*.pack; do\n> \t\tgit unpack-objects < $p\n> \tdone\n\nOuch!  You will end up with a multi-gigabyte repository, which will be \nabominably slow.\n\n> >  3. Identify the missing object, as explained in\n> >     Documentation/howto/recover-corrupted-blob-object.txt?\n\nThat's useful when you have only one corrupted object and you want to \nrecreate it from raw material.  But ideally you should simply find a \npack that contains the problematic object in another repository and copy \nit with its index \nfile into the broken repository.\n\n> and I'm running git-fsck --full now over night as it's bedtime here.\n\nGiven that you exploded your repo into loose objects, it'll take _time_.\n\n\nNicolas\n"},{"id":"153825","messageId":"20101019210233.GA32029@burratino","threadId":"25480","inReplyTo":"alpine.LFD.2.00.1010191635030.2764@xanadu.home","subject":"Re: fetch and bundle don't work in (semi-)broken repo","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-10-19T21:02:33Z","receivedAt":"2010-10-19T21:02:33Z","isPatch":false,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Hi,\n\n[out of order for convenience]\nNicolas Pitre wrote:\n> On Tue, 19 Oct 2010, Uwe Kleine-König wrote:\n\n>> and I'm running git-fsck --full now over night as it's bedtime here.\n>\n> Given that you exploded your repo into loose objects, it'll take _time_.\n\nYep, I gave bad advice. :(  Especially because I forgot that a fsck\nwould be useful at all.\n\nBetter advice would be:\n\n 1. Use \"git rev-list --objects\" to find out what 40aaeb204dc was.\n\nAnd if that doesn't work:\n\n 2. Run \"git fsck\", with packs intact.  This will take a while.  The\n    result would include a list of missing objects (like 40aaeb204dc),\n    and, most importantly, their type.\n\nFollowing howto/recover-corrupted-blob-object.txt would be useful for\nidentifying a corrupt loose object, but iiuc no corrupt objects are\ninvolved here, anyway.\n\n>                                 But ideally you should simply find a \n> pack that contains the problematic object in another repository and copy \n> it with its index \n> file into the broken repository.\n\nI assume the object is gone for good, but if you have it in another\nrepo that would be interesting, too.\n\nTo be clear: I think the important data has been recovered from the\nbroken repo already in the form of patches (right?) so the question\nat hand is whether it would be possible to teach git to do better at\nrecovering automatically.  Which might depend on the nature of the\nmissing objects.\n\nCiao,\nJonathan\n"},{"id":"153843","messageId":"alpine.LFD.2.00.1010192154200.2764@xanadu.home","threadId":"25480","inReplyTo":"20101019210233.GA32029@burratino","subject":"Re: fetch and bundle don't work in (semi-)broken repo","fromName":"Nicolas Pitre","fromEmail":"nico@fluxnic.net","sentAt":"2010-10-20T03:06:50Z","receivedAt":"2010-10-20T03:06:50Z","isPatch":false,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Tue, 19 Oct 2010, Jonathan Nieder wrote:\n\n> Nicolas Pitre wrote:\n> \n> >                                 But ideally you should simply find a \n> > pack that contains the problematic object in another repository and \n> > copy it with its index file into the broken repository.\n> \n> I assume the object is gone for good, but if you have it in another\n> repo that would be interesting, too.\n> \n> To be clear: I think the important data has been recovered from the\n> broken repo already in the form of patches (right?) so the question\n> at hand is whether it would be possible to teach git to do better at\n> recovering automatically.  Which might depend on the nature of the\n> missing objects.\n\nSure.  Given that it is possible to create a patch series, that means \nthat all the important objects are still available.  Therefore Git \nshould be able to produce the pack for the equivalent fetch/bundle as \nwell.\n\nSo the following patch should help.  I hope that Uwe still has a copy of \nthe broken repo to test this patch with.\n\ndiff --git a/builtin/pack-objects.c b/builtin/pack-objects.c\nindex f8eba53..691c2f1 100644\n--- a/builtin/pack-objects.c\n+++ b/builtin/pack-objects.c\n@@ -1299,6 +1299,15 @@ static int try_delta(struct unpacked *trg, struct unpacked *src,\n \t\tsrc->data = read_sha1_file(src_entry->idx.sha1, &type, &sz);\n \t\tread_unlock();\n \t\tif (!src->data)\n+\t\t\tif (src_entry->preferred_base) {\n+\t\t\t\t/* \n+\t\t\t\t * Those objects are not included in the\n+\t\t\t\t * resulting pack.  Be resilient and ignore\n+\t\t\t\t * them, in case the pack could be created\n+\t\t\t\t * nevertheless.\n+\t\t\t\t */\n+\t\t\t\treturn 0;\n+\t\t\t}\n \t\t\tdie(\"object %s cannot be read\",\n \t\t\t    sha1_to_hex(src_entry->idx.sha1));\n \t\tif (sz != src_size)\n\n\nNicolas\n"},{"id":"153856","messageId":"20101020074122.GC28166@pengutronix.de","threadId":"25480","inReplyTo":"alpine.LFD.2.00.1010192154200.2764@xanadu.home","subject":"Re: fetch and bundle don't work in (semi-)broken repo","fromName":"Uwe Kleine-König","fromEmail":"u.kleine-koenig@pengutronix.de","sentAt":"2010-10-20T07:41:22Z","receivedAt":"2010-10-20T07:41:22Z","isPatch":false,"sender":{"key":"u.kleine-koenig@pengutronix.de","avatar":"https://gravatar.com/avatar/354b5e3ceb2806a2f1e1e382ac29ddbdad18288654da62b61eb13583a857eee7?d=mp&s=160"},"body":"Hello Nico,\n\nOn Tue, Oct 19, 2010 at 11:06:50PM -0400, Nicolas Pitre wrote:\n> On Tue, 19 Oct 2010, Jonathan Nieder wrote:\n> \n> > Nicolas Pitre wrote:\n> > \n> > >                                 But ideally you should simply find a \n> > > pack that contains the problematic object in another repository and \n> > > copy it with its index file into the broken repository.\n> > \n> > I assume the object is gone for good, but if you have it in another\n> > repo that would be interesting, too.\n> > \n> > To be clear: I think the important data has been recovered from the\n> > broken repo already in the form of patches (right?) so the question\n> > at hand is whether it would be possible to teach git to do better at\n> > recovering automatically.  Which might depend on the nature of the\n> > missing objects.\n> \n> Sure.  Given that it is possible to create a patch series, that means \n> that all the important objects are still available.  Therefore Git \n> should be able to produce the pack for the equivalent fetch/bundle as \n> well.\n> \n> So the following patch should help.  I hope that Uwe still has a copy of \n> the broken repo to test this patch with.\n> \n> diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c\n> index f8eba53..691c2f1 100644\n> --- a/builtin/pack-objects.c\n> +++ b/builtin/pack-objects.c\n> @@ -1299,6 +1299,15 @@ static int try_delta(struct unpacked *trg, struct unpacked *src,\n>  \t\tsrc->data = read_sha1_file(src_entry->idx.sha1, &type, &sz);\n>  \t\tread_unlock();\n>  \t\tif (!src->data)\n> +\t\t\tif (src_entry->preferred_base) {\n> +\t\t\t\t/* \n> +\t\t\t\t * Those objects are not included in the\n> +\t\t\t\t * resulting pack.  Be resilient and ignore\n> +\t\t\t\t * them, in case the pack could be created\n> +\t\t\t\t * nevertheless.\n> +\t\t\t\t */\n> +\t\t\t\treturn 0;\n> +\t\t\t}\n>  \t\t\tdie(\"object %s cannot be read\",\n>  \t\t\t    sha1_to_hex(src_entry->idx.sha1));\n>  \t\tif (sz != src_size)\nDoesn't help :-(  I added a warning(...) before your return 0, and I\ndon't see it.  Probably this means this is not the problematic code\npath.\n\nThe output with your patch applied is:\n\tuser@hostname:~/path/linux-2.6$ ~/gsrc/git/bin-wrappers/git bundle create tra linus/master..sectionmismatches\n\tCounting objects: 118, done.\n\terror: unable to find 40aaeb204dc04d3cf15c060133f65538b43b13b0\n\tDelta compression using up to 8 threads.\n\tfatal: object 3cf4fa25ab3d078a49e9488effaebf571fa128da cannot be read\n\terror: pack-objects died\n\nIf you want I can provide you the broken repo.\n\nBest regards\nUwe\n\n-- \nPengutronix e.K.                           | Uwe Kleine-König            |\nIndustrial Linux Solutions                 | http://www.pengutronix.de/  |\n"},{"id":"153858","messageId":"20101020075945.GD28166@pengutronix.de","threadId":"25480","inReplyTo":"alpine.LFD.2.00.1010191635030.2764@xanadu.home","subject":"Re: fetch and bundle don't work in (semi-)broken repo","fromName":"Uwe Kleine-König","fromEmail":"u.kleine-koenig@pengutronix.de","sentAt":"2010-10-20T07:59:45Z","receivedAt":"2010-10-20T07:59:45Z","isPatch":false,"sender":{"key":"u.kleine-koenig@pengutronix.de","avatar":"https://gravatar.com/avatar/354b5e3ceb2806a2f1e1e382ac29ddbdad18288654da62b61eb13583a857eee7?d=mp&s=160"},"body":"Hello,\n\nOn Tue, Oct 19, 2010 at 04:48:23PM -0400, Nicolas Pitre wrote:\n> On Tue, 19 Oct 2010, Uwe Kleine-König wrote:\n> \n> > Hi Jonathan,\n> > \n> > On Tue, Oct 19, 2010 at 01:39:25PM -0500, Jonathan Nieder wrote:\n> > > Uwe Kleine-König wrote:\n> > > \n> > > > \tukl@hostname:~/path1/linux-2.6$ git fetch ~/path2/linux-2.6 sectionmismatches\n> > > > \tremote: Counting objects: 118, done.\n> > > > \tremote: error: unable to find 40aaeb204dc04d3cf15c060133f65538b43b13b0\n> > > > \tremote: Compressing objects: 100% (83/83), done.\n> > > > \tremote: fatal: unable to read 40aaeb204dc04d3cf15c060133f65538b43b13b0\n> > > > \terror: git upload-pack: git-pack-objects died with error.\n> > > > \tfatal: git upload-pack: aborting due to possible repository corruption on the remote side.\n> > > \n> > > Sounds like alternates or workdir allowed gc to be overzealous, indeed.\n> > > \n> > > Could you:\n> > > \n> > >  1. Make a copy of the corrupted repo, just in case.\n> > >  2. Explode all backs with \"git unpack-objects\"\n> \n> That's usually unnecessary.  If the pack itself is corrupted, trying to \n> unpack it all won't help.\n> \n> > I did:\n> > \n> > \tmv .git/objects/pack .git/objects/pack.bak\n> > \trm .git/objects/info/alternates\n> > \tfor p in .git/objects/pack.bak/*.pack ~/path1/linux-2.6/.git/objects/pack/*.pack; do\n> > \t\tgit unpack-objects < $p\n> > \tdone\n> \n> Ouch!  You will end up with a multi-gigabyte repository, which will be \n> abominably slow.\n> \n> > >  3. Identify the missing object, as explained in\n> > >     Documentation/howto/recover-corrupted-blob-object.txt?\n> \n> That's useful when you have only one corrupted object and you want to \n> recreate it from raw material.  But ideally you should simply find a \n> pack that contains the problematic object in another repository and copy \n> it with its index \n> file into the broken repository.\n> \n> > and I'm running git-fsck --full now over night as it's bedtime here.\n> \n> Given that you exploded your repo into loose objects, it'll take _time_.\nWell it took 34 minutes, which is OK I guess.\n\nI will study the output a bit now.\n\nFor the interested (all lines matching \"dangling\" removed):\n\n\t22:10:57 I: Started git fsck --full\n\t22:44:14 O: broken link from    tree 519af383e181399db929823299bbd14c04b4229a\n\t22:44:14 O: to    tree d58c333c44672cb933df5a353dfb63ac571964e8\n\t22:44:14 O: broken link from  commit e8f7f6a23979c398249a15fb71b3e52dae933fa3\n\t22:44:14 O: to    tree 7f22979d86cf00c8bd3487feb973353ab5a1beee\n\t22:44:14 O: broken link from  commit 3164f6598ae44703a89822ced9746c1876ba7fab\n\t22:44:14 O: to    tree 1017bb1f45b8527ee3c7cfc30288b8098bcf0915\n\t22:44:14 O: broken link from  commit 124dde2ea387dc9509b0a5574c6f44f7d348a65d\n\t22:44:14 O: to    tree e4d0ac236995847e4e1d15c6d0afb47787255703\n\t22:44:14 O: broken link from  commit 60deff2fffd90b217d90284295d5a910f21fe98e\n\t22:44:14 O: to    tree 18bb32cfd08228820f929d62e63933fe2896b424\n\t22:44:14 O: broken link from  commit 0b84e651b84dba73772fda15a8a66de8cc274af0\n\t22:44:14 O: to    tree f8939a09d73b78459381b7991423529592e66324\n\t22:44:14 O: broken link from  commit e0de1d3c3355f9b1e3474417f05657a1041e7c8a\n\t22:44:14 O: to    tree 776ad9ac45dab11f2644151a690e1035789a49b6\n\t22:44:14 O: broken link from  commit 76d1acb95eef413a2501a63cb7f7f4036b71ed37\n\t22:44:14 O: to  commit f6b6cb2336198913371e66664f28c135df01aea5\n\t22:44:14 O: broken link from    tree bb473ad85c260b6a1659aa2059cac23b337842e3\n\t22:44:14 O: to    tree e035bc14698cc3e9abfca1a174feacb25e7e262a\n\t22:44:14 O: broken link from    tree bb473ad85c260b6a1659aa2059cac23b337842e3\n\t22:44:14 O: to    tree 8908b2458c1a2c6a6db81e88d96a01aa9a89abe5\n\t22:44:14 O: broken link from    tree ee35b3a549f45830ed50eb1032836a71ab2b7886\n\t22:44:14 O: to    tree f2f33722af4b5e32ac17f914cf24cc96c6e80077\n\t22:44:14 O: broken link from    tree ee35b3a549f45830ed50eb1032836a71ab2b7886\n\t22:44:14 O: to    tree 70f0188991b8406ec6ec75a504cf50c778fc1001\n\t22:44:14 O: broken link from    tree 1772732da7d4751d3c0febd7b0ceee61a84702f0\n\t22:44:14 O: to    tree dc3c1a018fcf2797a860af0cb899fe6be1173da3\n\t22:44:14 O: broken link from    tree 1772732da7d4751d3c0febd7b0ceee61a84702f0\n\t22:44:14 O: to    tree 2ade2c0731e57fc6b1baddb6faf683847d706179\n\t22:44:14 O: broken link from    tree 1772732da7d4751d3c0febd7b0ceee61a84702f0\n\t22:44:14 O: to    tree 659f4a6280150b7ae7b8f824257ad3cbb7cd1788\n\t22:44:14 O: broken link from    tree f29915a03ea5ba8739aacfcbb349d40b019481ed\n\t22:44:14 O: to    tree b52d608d7743b1e096725d4b8d7c7769b19f6674\n\t22:44:15 O: missing tree 7f22979d86cf00c8bd3487feb973353ab5a1beee\n\t22:44:15 O: missing tree 70f0188991b8406ec6ec75a504cf50c778fc1001\n\t22:44:15 O: missing tree dc3c1a018fcf2797a860af0cb899fe6be1173da3\n\t22:44:15 O: missing tree f8939a09d73b78459381b7991423529592e66324\n\t22:44:15 O: missing tree e4d0ac236995847e4e1d15c6d0afb47787255703\n\t22:44:15 O: missing tree 2ade2c0731e57fc6b1baddb6faf683847d706179\n\t22:44:15 O: missing tree 8908b2458c1a2c6a6db81e88d96a01aa9a89abe5\n\t22:44:15 O: missing tree 18bb32cfd08228820f929d62e63933fe2896b424\n\t22:44:15 O: missing tree d58c333c44672cb933df5a353dfb63ac571964e8\n\t22:44:15 O: missing tree f2f33722af4b5e32ac17f914cf24cc96c6e80077\n\t22:44:16 O: missing tree 1017bb1f45b8527ee3c7cfc30288b8098bcf0915\n\t22:44:16 O: missing tree e035bc14698cc3e9abfca1a174feacb25e7e262a\n\t22:44:16 O: missing tree 659f4a6280150b7ae7b8f824257ad3cbb7cd1788\n\t22:44:16 O: missing commit f6b6cb2336198913371e66664f28c135df01aea5\n\t22:44:16 O: missing tree 776ad9ac45dab11f2644151a690e1035789a49b6\n\t22:44:16 O: missing tree b52d608d7743b1e096725d4b8d7c7769b19f6674\n\t22:44:16 O: missing blob 40aaeb204dc04d3cf15c060133f65538b43b13b0\n\nHere is the output for the object bundle and fetch wail about, but\nnothing about a tree that would need it :-(  Actually it's the only one\nof the missing objects that is not described above (and it's the only\nblob).\n\n\t22:44:17 I: Finished with exitcode 2\n\nNow I'm running\n\n\tgit for-each-ref | while read sha type name; do\n\t\techo $name;\n\t\tgit log --raw --full-history $sha | grep 40aaeb204dc04d3cf15c060133f65538b43b13b0;\n\tdone\n\nBest regards\nUwe\n\n-- \nPengutronix e.K.                           | Uwe Kleine-König            |\nIndustrial Linux Solutions                 | http://www.pengutronix.de/  |\n"},{"id":"153866","messageId":"alpine.LFD.2.00.1010200937200.2764@xanadu.home","threadId":"25480","inReplyTo":"20101020074122.GC28166@pengutronix.de","subject":"Re: fetch and bundle don't work in (semi-)broken repo","fromName":"Nicolas Pitre","fromEmail":"nico@fluxnic.net","sentAt":"2010-10-20T13:38:14Z","receivedAt":"2010-10-20T13:38:14Z","isPatch":false,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Wed, 20 Oct 2010, Uwe Kleine-König wrote:\n\n> If you want I can provide you the broken repo.\n\nYes, please.\n\n\nNicolas\n"},{"id":"153943","messageId":"20101021071106.GG19834@pengutronix.de","threadId":"25480","inReplyTo":"alpine.LFD.2.00.1010201301130.2764@xanadu.home","subject":"format-patch broken [Was: fetch and bundle don't work in (semi-)broken repo]","fromName":"Uwe Kleine-König","fromEmail":"u.kleine-koenig@pengutronix.de","sentAt":"2010-10-21T07:11:06Z","receivedAt":"2010-10-21T07:11:06Z","isPatch":false,"sender":{"key":"u.kleine-koenig@pengutronix.de","avatar":"https://gravatar.com/avatar/354b5e3ceb2806a2f1e1e382ac29ddbdad18288654da62b61eb13583a857eee7?d=mp&s=160"},"body":"On Wed, Oct 20, 2010 at 01:05:26PM -0400, Nicolas Pitre wrote:\n> On Wed, 20 Oct 2010, Uwe Kleine-König wrote:\n> \n> > On Wed, Oct 20, 2010 at 09:38:14AM -0400, Nicolas Pitre wrote:\n> > > On Wed, 20 Oct 2010, Uwe Kleine-König wrote:\n> > > \n> > > > If you want I can provide you the broken repo.\n> > > \n> > > Yes, please.\n> \n> I'm trying to reproduce the exact error you get. But I'm missing some \n> alternates for objects, and none of the trees I have locally provide the \n> needed objects.\n> \n> [nico@xanadu linux-2.6]$ ! git format-patch linus/master..sectionmismatches\n> 0001-wip-enable-DEBUG_SECTION_MISMATCH.patch\n> 0002-ARM-sa1111-move-__sa1111_probe-to-.devinit.text.patch\n> 0003-ARM-omap1-nokia770-mark-some-functions-__init.patch\n> fatal: unable to read destination tree (16edb8381f2f2dabec9cc59f4a3d8c9ead899668)\n> \n> What do you have in your alternate repo?\nHmm, I just unpacked the archive in a seperate directory, removed\n.git/objects/info/alternates and then git format-patch\nlinus/master..sectionmismatches fails in a different way:\n\n\tfatal: Invalid revision range linus/master..sectionmismatches\n\nI guess adding a pristine copy of Linus' tree should do the trick.\n\n[ ... some time later ... creating a fresh clone takes quite some time ... ]\n\nNo, that's not enough, I will handpick some objects from the original.\n\nAh, you only need 16edb8381f2f2dabec9cc59f4a3d8c9ead899668 to make\nformat-patch work, but still 09b3f464a50111071f7740056b98fa8f36133347 is\nmissing for this tree.  This doesn't hurt format-patch as it's enough\nfor it to know that this entry didn't change.  So format-patch needs\nless information than bundle/fetch and it's OK that the former succeeds\nand the latter fails.\n\n[...]\n\nNo, that's not the (only) problem,\n40aaeb204dc04d3cf15c060133f65538b43b13b0 is needed, git format-patch is\njust ignorant enough and invents something different:\n\n\tusername@hostname:~/path/linux-2.6$ git rev-list linus/master..sectionmismatches\n\teb84720860a90769473b42215a4cb67ee5efe7a7\n\t2e14a5c831032fa489384763087f4a03d88607cb\n\t00b18e8058e98927e2e4eae32deae7e58f47467c\n\t1ad328f663128b5c6e6b4af1ac2da1b443dba530\n\t2a0e4c23a34c78891db685b2b4851705fd36d656\n\t089d061c26b00a5b8dbb9e70b81d36a97e1daded\n\tb7ce4ec88f1bdfbe49fa7ef12df8f985d705605a\n\tb40acb01793933cd6baaaf826f3fef6dd734f72b\n\t780e3d47d067b54b17bcac3794d62825e8e60422\n\tce06129cf7bbf85afe4fc127afc957d36ba4e9e4\n\tc2172d687578e7eb037a232802a4a8c6de1b0eea\n\t0c23684f39714a72f54036ca2be36e8894794b66\n\tcea2a0668ee1a9dc3617a810954a41c7701a08e9\n\t2bd6ff604ac3aa4c96636dda1ad80a289205ccba\n\t7591700d538d08f2e8327bb439b6cb0488e13f3e\n\n\tusername@hostname:~/path/linux-2.6$ git diff-tree -r 7591700d538d08f2e8327bb439b6cb0488e13f3e\n\t7591700d538d08f2e8327bb439b6cb0488e13f3e\n\t:100644 100644 1b4afd2e6ca089de0babdacc5781426ef118da5c 40aaeb204dc04d3cf15c060133f65538b43b13b0 M\tlib/Kconfig.debug\n\n\tcommit 7591700d538d08f2e8327bb439b6cb0488e13f3e\n\tAuthor: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>\n\tDate:   Wed Aug 4 08:52:56 2010 +0200\n\n\t    wip: enable DEBUG_SECTION_MISMATCH\n\n\tdiff --git a/lib/Kconfig.debug b/lib/Kconfig.debug\n\tindex 1b4afd2..40aaeb2 100644\n\t--- a/lib/Kconfig.debug\n\t+++ b/lib/Kconfig.debug\n\t@@ -1,1157 +0,0 @@\n\t-\n\t-config PRINTK_TIME\n\t-\tbool \"Show timing information on printks\"\n\t-\tdepends on PRINTK\n\t-\thelp\n\t-\t  Selecting this option causes timing information to be\n\t-\t  included in printk output.  This allows you to measure\n\t-\t  the interval between kernel operations, including bootup\n\t-\t  operations.  This is useful for identifying long delays\n\nSo format-patch assumes that lib/Kconfig.debug became empty because of\nnot knowing what 40aaeb204dc04d3cf15c060133f65538b43b13b0 is.\n\nSo it's not git-bundle/git-fetch that is broken, but format-patch.\n\nI attached an archive with the two objects\n16edb8381f2f2dabec9cc59f4a3d8c9ead899668 and\n09b3f464a50111071f7740056b98fa8f36133347 and will try to recreate\n40aaeb204dc04d3cf15c060133f65538b43b13b0 now.\n\nBest regards\nUwe\n\n-- \nPengutronix e.K.                           | Uwe Kleine-König            |\nIndustrial Linux Solutions                 | http://www.pengutronix.de/  |\n"},{"id":"153944","messageId":"20101021081250.GH19834@pengutronix.de","threadId":"25480","inReplyTo":"20101021071106.GG19834@pengutronix.de","subject":"Re: format-patch broken [Was: fetch and bundle don't work in (semi-)broken repo]","fromName":"Uwe Kleine-König","fromEmail":"u.kleine-koenig@pengutronix.de","sentAt":"2010-10-21T08:12:50Z","receivedAt":"2010-10-21T08:12:50Z","isPatch":false,"sender":{"key":"u.kleine-koenig@pengutronix.de","avatar":"https://gravatar.com/avatar/354b5e3ceb2806a2f1e1e382ac29ddbdad18288654da62b61eb13583a857eee7?d=mp&s=160"},"body":"Hello,\n\nOn Thu, Oct 21, 2010 at 09:11:06AM +0200, Uwe Kleine-König wrote:\n> On Wed, Oct 20, 2010 at 01:05:26PM -0400, Nicolas Pitre wrote:\n> > On Wed, 20 Oct 2010, Uwe Kleine-König wrote:\n> > \n> > > On Wed, Oct 20, 2010 at 09:38:14AM -0400, Nicolas Pitre wrote:\n> > > > On Wed, 20 Oct 2010, Uwe Kleine-König wrote:\n> > > > \n> > > > > If you want I can provide you the broken repo.\n> > > > \n> > > > Yes, please.\n> > \n> > I'm trying to reproduce the exact error you get. But I'm missing some \n> > alternates for objects, and none of the trees I have locally provide the \n> > needed objects.\n> > \n> > [nico@xanadu linux-2.6]$ ! git format-patch linus/master..sectionmismatches\n> > 0001-wip-enable-DEBUG_SECTION_MISMATCH.patch\n> > 0002-ARM-sa1111-move-__sa1111_probe-to-.devinit.text.patch\n> > 0003-ARM-omap1-nokia770-mark-some-functions-__init.patch\n> > fatal: unable to read destination tree (16edb8381f2f2dabec9cc59f4a3d8c9ead899668)\n> > \n> > What do you have in your alternate repo?\n> Hmm, I just unpacked the archive in a seperate directory, removed\n> .git/objects/info/alternates and then git format-patch\n> linus/master..sectionmismatches fails in a different way:\n> \n> \tfatal: Invalid revision range linus/master..sectionmismatches\n> \n> I guess adding a pristine copy of Linus' tree should do the trick.\n> \n> [ ... some time later ... creating a fresh clone takes quite some time ... ]\n> \n> No, that's not enough, I will handpick some objects from the original.\n> \n> Ah, you only need 16edb8381f2f2dabec9cc59f4a3d8c9ead899668 to make\n> format-patch work, but still 09b3f464a50111071f7740056b98fa8f36133347 is\n> missing for this tree.  This doesn't hurt format-patch as it's enough\n> for it to know that this entry didn't change.  So format-patch needs\n> less information than bundle/fetch and it's OK that the former succeeds\n> and the latter fails.\n> \n> [...]\n> \n> No, that's not the (only) problem,\n> 40aaeb204dc04d3cf15c060133f65538b43b13b0 is needed, git format-patch is\n> just ignorant enough and invents something different:\n> \n> \tusername@hostname:~/path/linux-2.6$ git rev-list linus/master..sectionmismatches\n> \teb84720860a90769473b42215a4cb67ee5efe7a7\n> \t2e14a5c831032fa489384763087f4a03d88607cb\n> \t00b18e8058e98927e2e4eae32deae7e58f47467c\n> \t1ad328f663128b5c6e6b4af1ac2da1b443dba530\n> \t2a0e4c23a34c78891db685b2b4851705fd36d656\n> \t089d061c26b00a5b8dbb9e70b81d36a97e1daded\n> \tb7ce4ec88f1bdfbe49fa7ef12df8f985d705605a\n> \tb40acb01793933cd6baaaf826f3fef6dd734f72b\n> \t780e3d47d067b54b17bcac3794d62825e8e60422\n> \tce06129cf7bbf85afe4fc127afc957d36ba4e9e4\n> \tc2172d687578e7eb037a232802a4a8c6de1b0eea\n> \t0c23684f39714a72f54036ca2be36e8894794b66\n> \tcea2a0668ee1a9dc3617a810954a41c7701a08e9\n> \t2bd6ff604ac3aa4c96636dda1ad80a289205ccba\n> \t7591700d538d08f2e8327bb439b6cb0488e13f3e\n> \n> \tusername@hostname:~/path/linux-2.6$ git diff-tree -r 7591700d538d08f2e8327bb439b6cb0488e13f3e\n> \t7591700d538d08f2e8327bb439b6cb0488e13f3e\n> \t:100644 100644 1b4afd2e6ca089de0babdacc5781426ef118da5c 40aaeb204dc04d3cf15c060133f65538b43b13b0 M\tlib/Kconfig.debug\n> \n> \tcommit 7591700d538d08f2e8327bb439b6cb0488e13f3e\n> \tAuthor: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>\n> \tDate:   Wed Aug 4 08:52:56 2010 +0200\n> \n> \t    wip: enable DEBUG_SECTION_MISMATCH\n> \n> \tdiff --git a/lib/Kconfig.debug b/lib/Kconfig.debug\n> \tindex 1b4afd2..40aaeb2 100644\n> \t--- a/lib/Kconfig.debug\n> \t+++ b/lib/Kconfig.debug\n> \t@@ -1,1157 +0,0 @@\n> \t-\n> \t-config PRINTK_TIME\n> \t-\tbool \"Show timing information on printks\"\n> \t-\tdepends on PRINTK\n> \t-\thelp\n> \t-\t  Selecting this option causes timing information to be\n> \t-\t  included in printk output.  This allows you to measure\n> \t-\t  the interval between kernel operations, including bootup\n> \t-\t  operations.  This is useful for identifying long delays\n> \n> So format-patch assumes that lib/Kconfig.debug became empty because of\n> not knowing what 40aaeb204dc04d3cf15c060133f65538b43b13b0 is.\n> \n> So it's not git-bundle/git-fetch that is broken, but format-patch.\n> \n> I attached an archive with the two objects\n> 16edb8381f2f2dabec9cc59f4a3d8c9ead899668 and\n> 09b3f464a50111071f7740056b98fa8f36133347 and will try to recreate\n> 40aaeb204dc04d3cf15c060133f65538b43b13b0 now.\nThat was easy:\n\n\tgit hash-object -w lib/Kconfig.debug\n\nNow git bundle works again.\n\nNicolas: I forgot to say, that I needed a pristine clone of Linus' repo\nas alternate to get it running.\n\nBest regards\nUwe\n\n-- \nPengutronix e.K.                           | Uwe Kleine-König            |\nIndustrial Linux Solutions                 | http://www.pengutronix.de/  |\n"},{"id":"154091","messageId":"alpine.LFD.2.00.1010212235410.2764@xanadu.home","threadId":"25480","inReplyTo":"20101021071106.GG19834@pengutronix.de","subject":"Re: format-patch broken [Was: fetch and bundle don't work in (semi-)broken repo]","fromName":"Nicolas Pitre","fromEmail":"nico@fluxnic.net","sentAt":"2010-10-22T03:53:37Z","receivedAt":"2010-10-22T03:53:37Z","isPatch":false,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Thu, 21 Oct 2010, Uwe Kleine-König wrote:\n\n> Ah, you only need 16edb8381f2f2dabec9cc59f4a3d8c9ead899668 to make\n> format-patch work, but still 09b3f464a50111071f7740056b98fa8f36133347 is\n> missing for this tree.  This doesn't hurt format-patch as it's enough\n> for it to know that this entry didn't change.  So format-patch needs\n> less information than bundle/fetch and it's OK that the former succeeds\n> and the latter fails.\n\nNo it is not.  In theory both format-patch and fetch/bundle should \nrequire the exact same information.\n\n> [...]\n> \n> No, that's not the (only) problem,\n> 40aaeb204dc04d3cf15c060133f65538b43b13b0 is needed, git format-patch is\n> just ignorant enough and invents something different:\n> \n> \tusername@hostname:~/path/linux-2.6$ git rev-list linus/master..sectionmismatches\n> \teb84720860a90769473b42215a4cb67ee5efe7a7\n> \t2e14a5c831032fa489384763087f4a03d88607cb\n> \t00b18e8058e98927e2e4eae32deae7e58f47467c\n> \t1ad328f663128b5c6e6b4af1ac2da1b443dba530\n> \t2a0e4c23a34c78891db685b2b4851705fd36d656\n> \t089d061c26b00a5b8dbb9e70b81d36a97e1daded\n> \tb7ce4ec88f1bdfbe49fa7ef12df8f985d705605a\n> \tb40acb01793933cd6baaaf826f3fef6dd734f72b\n> \t780e3d47d067b54b17bcac3794d62825e8e60422\n> \tce06129cf7bbf85afe4fc127afc957d36ba4e9e4\n> \tc2172d687578e7eb037a232802a4a8c6de1b0eea\n> \t0c23684f39714a72f54036ca2be36e8894794b66\n> \tcea2a0668ee1a9dc3617a810954a41c7701a08e9\n> \t2bd6ff604ac3aa4c96636dda1ad80a289205ccba\n> \t7591700d538d08f2e8327bb439b6cb0488e13f3e\n> \n> \tusername@hostname:~/path/linux-2.6$ git diff-tree -r 7591700d538d08f2e8327bb439b6cb0488e13f3e\n> \t7591700d538d08f2e8327bb439b6cb0488e13f3e\n> \t:100644 100644 1b4afd2e6ca089de0babdacc5781426ef118da5c 40aaeb204dc04d3cf15c060133f65538b43b13b0 M\tlib/Kconfig.debug\n> \n> \tcommit 7591700d538d08f2e8327bb439b6cb0488e13f3e\n> \tAuthor: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>\n> \tDate:   Wed Aug 4 08:52:56 2010 +0200\n> \n> \t    wip: enable DEBUG_SECTION_MISMATCH\n> \n> \tdiff --git a/lib/Kconfig.debug b/lib/Kconfig.debug\n> \tindex 1b4afd2..40aaeb2 100644\n> \t--- a/lib/Kconfig.debug\n> \t+++ b/lib/Kconfig.debug\n> \t@@ -1,1157 +0,0 @@\n> \t-\n> \t-config PRINTK_TIME\n> \t-\tbool \"Show timing information on printks\"\n> \t-\tdepends on PRINTK\n> \t-\thelp\n> \t-\t  Selecting this option causes timing information to be\n> \t-\t  included in printk output.  This allows you to measure\n> \t-\t  the interval between kernel operations, including bootup\n> \t-\t  operations.  This is useful for identifying long delays\n> \n> So format-patch assumes that lib/Kconfig.debug became empty because of\n> not knowing what 40aaeb204dc04d3cf15c060133f65538b43b13b0 is.\n> \n> So it's not git-bundle/git-fetch that is broken, but format-patch.\n\nOr rather the low-level diff code.\n\ndiff --git a/diff.c b/diff.c\nindex 4732b32..b2839f9 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -2386,10 +2386,14 @@ int diff_populate_filespec(struct diff_filespec *s, int size_only)\n \t}\n \telse {\n \t\tenum object_type type;\n-\t\tif (size_only)\n+\t\tif (size_only) {\n \t\t\ttype = sha1_object_info(s->sha1, &s->size);\n-\t\telse {\n+\t\t\tif (type < 0)\n+\t\t\t\tdie(\"unable to read %s\", sha1_to_hex(s->sha1));\n+\t\t} else {\n \t\t\ts->data = read_sha1_file(s->sha1, &type, &s->size);\n+\t\t\tif (!s->data)\n+\t\t\t\tdie(\"unable to read %s\", sha1_to_hex(s->sha1));\n \t\t\ts->should_free = 1;\n \t\t}\n \t}\n\n\nNicolas\n"}]}