{"thread":{"id":"51597","subject":"Feature-request: git-bundle --quiet","startedAt":"2019-08-06T19:19:19Z","lastAt":"2019-08-12T16:08:37Z","messageCount":4,"participants":["Robin H. Johnson","Jeff King","Jacob Vosmaer"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"380039","messageId":"robbat2-20190806T191156-796782357Z@orbis-terrarum.net","threadId":"51597","inReplyTo":null,"subject":"Feature-request: git-bundle --quiet","fromName":"Robin H. Johnson","fromEmail":"robbat2@gentoo.org","sentAt":"2019-08-06T19:19:11Z","receivedAt":"2019-08-06T19:19:19Z","isPatch":false,"sender":{"key":"robbat2@gentoo.org","avatar":"https://avatars.githubusercontent.com/u/373898?v=4"},"body":"I started trying to make a stab at implementing this, but the code\nwasn't standing out for it. Hopefully somebody else has poked at it\nbefore:\n\nI'd like to have a --quiet option for git-bundle, such that only errors\nare sent to stderr, and not the packing progress.\n\nThis is towards a better incremental backup of large Git repos, later\nsteps would be easier selection of ranges (writing out markers e.g. to\nuse in the next day's incremental backup), and more fine-grained control\nof the bundle repack behavior (e.g. don't try hard to repack, backup\nspeed is important)\n\n-- \nRobin Hugh Johnson\nGentoo Linux: Dev, Infra Lead, Foundation Treasurer\nE-Mail   : robbat2@gentoo.org\nGnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85\nGnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136\n"},{"id":"380115","messageId":"20190808104212.GA14189@sigill.intra.peff.net","threadId":"51597","inReplyTo":"robbat2-20190806T191156-796782357Z@orbis-terrarum.net","subject":"Re: Feature-request: git-bundle --quiet","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-08-08T10:42:12Z","receivedAt":"2019-08-08T10:42:15Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Aug 06, 2019 at 07:19:11PM +0000, Robin H. Johnson wrote:\n\n> I started trying to make a stab at implementing this, but the code\n> wasn't standing out for it. Hopefully somebody else has poked at it\n> before:\n> \n> I'd like to have a --quiet option for git-bundle, such that only errors\n> are sent to stderr, and not the packing progress.\n\nThis seems like a reasonable thing to want.\n\nIt looks like you'd have to teach cmd_bundle() to use parse_options(),\nparse a quiet flag, and then pas that down to create_bundle(). Then it\nwould pass it along to write_pack_data(), which would decide whether to\npass \"-q\".\n\nThat would allow:\n\n  git bundle --quiet create foo.bundle ...\n\n-Peff\n"},{"id":"380299","messageId":"CADMWQoP3tH7-e+3FCYkB7b7nf8v-HBQ1toDoje8G_hvD_LaDZw@mail.gmail.com","threadId":"51597","inReplyTo":"20190808104212.GA14189@sigill.intra.peff.net","subject":"Re: Feature-request: git-bundle --quiet","fromName":"Jacob Vosmaer","fromEmail":"jacob@gitlab.com","sentAt":"2019-08-12T10:15:19Z","receivedAt":"2019-08-12T10:15:32Z","isPatch":false,"sender":{"key":"jacob@gitlab.com","avatar":"https://avatars.githubusercontent.com/u/1336311?v=4"},"body":"This is a tangent, but relevant: how do we feel about the fact that\n'git bundle create' does not perform CRC32 checks when copying data\nout of an existing packfile?\n\nSee https://github.com/git/git/blob/v2.22.0/builtin/pack-objects.c#L2614-L2622 .\n\nI understand the rationale of \"skip CRC32 when serving a fetch\",\nalthough I have no clue how much we gain from skipping it. But \"pack\nto stdout means fetch\" isn't quite accurate, as it includes bundles.\n\nBest regards,\n\nJacob Vosmaer\nGitLab, Inc.\n\nBest regards,\n\nJacob Vosmaer\nGitLab, Inc.\n\n\nOn Thu, Aug 8, 2019 at 12:42 PM Jeff King <peff@peff.net> wrote:\n>\n> On Tue, Aug 06, 2019 at 07:19:11PM +0000, Robin H. Johnson wrote:\n>\n> > I started trying to make a stab at implementing this, but the code\n> > wasn't standing out for it. Hopefully somebody else has poked at it\n> > before:\n> >\n> > I'd like to have a --quiet option for git-bundle, such that only errors\n> > are sent to stderr, and not the packing progress.\n>\n> This seems like a reasonable thing to want.\n>\n> It looks like you'd have to teach cmd_bundle() to use parse_options(),\n> parse a quiet flag, and then pas that down to create_bundle(). Then it\n> would pass it along to write_pack_data(), which would decide whether to\n> pass \"-q\".\n>\n> That would allow:\n>\n>   git bundle --quiet create foo.bundle ...\n>\n> -Peff\n"},{"id":"380312","messageId":"20190812160834.GA25388@sigill.intra.peff.net","threadId":"51597","inReplyTo":"CADMWQoP3tH7-e+3FCYkB7b7nf8v-HBQ1toDoje8G_hvD_LaDZw@mail.gmail.com","subject":"Re: Feature-request: git-bundle --quiet","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-08-12T16:08:35Z","receivedAt":"2019-08-12T16:08:37Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Aug 12, 2019 at 12:15:19PM +0200, Jacob Vosmaer wrote:\n\n> This is a tangent, but relevant: how do we feel about the fact that\n> 'git bundle create' does not perform CRC32 checks when copying data\n> out of an existing packfile?\n> \n> See https://github.com/git/git/blob/v2.22.0/builtin/pack-objects.c#L2614-L2622 .\n> \n> I understand the rationale of \"skip CRC32 when serving a fetch\",\n> although I have no clue how much we gain from skipping it. But \"pack\n> to stdout means fetch\" isn't quite accurate, as it includes bundles.\n\nI don't recall it being discussed in the past. I think you could argue\neither way:\n\n  - a bundle is just another form of object transfer, like a fetch, and\n    so we don't need to be careful about bitrot. The receiver would\n    notice it when it indexes the pack (as opposed to an on-disk repack,\n    where we'll immediately delete the old copy, and really want to make\n    sure we haven't just lost data).\n\n  - because a bundle isn't interactive like a regular fetch, any bit\n    errors may not be seen until much later when somebody reads the. At\n    that point it may not be possible to go back to the original repo\n    (in the extreme case of using a bundle as a backup, it may have been\n    deleted entirely!).\n\nDepending on the cost of those checks (and I really doubt they are all\n_that_ expensive), it might make sense for bundles to err on the\nconservative side and do them. And if they are expensive, it should\nperhaps be made an option for people who know they are planning to store\nthe bundle for a long time without reading it[1].\n\nI agree that linking \"skip CRC32\" to \"pack to stdout\" is a bit hacky. It\nshould be easy to add a new --check-crc32 option which defaults to\n\"!pack_to_stdout\" if not specified.\n\n-Peff\n\n[1] Of course bitrot in the original packfile is just one place this can\n    go wrong. Depending how paranoid you want to be, it might be worth\n    reading back the result before considering it a valid backup. That\n    would catch some software bugs, as well as any bit corruption on the\n    writing side. Doing a full index-pack is the most robust way there,\n    but it's quite expensive. Just checking the SHA1 of the packfile\n    itself would give pretty good protection against write errors,\n    though you'd definitely want to couple it with CRC32 checks on the\n    source (since Git would otherwise include the bad bits in its SHA1\n    checksum).\n"}]}