{"thread":{"id":"60882","subject":"Race condition in git-bundle(1) create when ref is updated while running","startedAt":"2024-02-09T13:51:59Z","lastAt":"2024-02-12T07:19:49Z","messageCount":3,"participants":["Toon Claes","Junio C Hamano","Patrick Steinhardt"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"488290","messageId":"87eddlpx5k.fsf@iotcl.com","threadId":"60882","inReplyTo":null,"subject":"Race condition in git-bundle(1) create when ref is updated while running","fromName":"Toon Claes","fromEmail":"toon@iotcl.com","sentAt":"2024-02-09T13:40:30Z","receivedAt":"2024-02-09T13:51:59Z","isPatch":false,"sender":{"key":"toon@iotcl.com","avatar":"https://avatars.githubusercontent.com/u/121621?v=4"},"body":"Hi y'all,\n\nI discovered a bug in git-bundle(1) create. There is a race condition\nhappening when a ref gets updated while the bundle creation process is\nrunning.\n\nTo reproduce, I've been running git-bundle(1) with\n`create my.bndl --all --ignore-missing` in a debugger. I've set a\nbreakpoint at bundle.c:515[1] where setup_revisions() is called. After\nstepping over this line I see in the debugger `revs.pending` is\npopulated.\n\n    (gdb) p *revs.pending.objects\n    $6 = {item = 0x7a2fb0, name = 0x78d7e0 \"refs/heads/master\", path = 0x0, mode = 12288}\n    (gdb) p *revs.pending.objects.item\n    $7 = {parsed = 1, type = 1, flags = 0, oid = {hash = \"R\\026\\370\\365\\304\\b\\236\\302\\234\\344\\232\\372\\024t4\\302>\\017\\001c\\000\\000\\000\\000sS\\344\\367\\377\\177\\000\", algo = 1}}\n\nThe hash value is the binary representation of\n`5216f8f5c4089ec29ce49afa147434c23e0f0163`, the current HEAD of\n`master`. At this point I've updated `master` in another terminal\nwindow:\n\n    git commit --allow-empty -m\"dummy\"\n\nThen in the debugger I continue the process to create the bundle. The\nresulting bundle seems to be missing `refs/heads/master`.\n\nI'm surprised this ref is completely omitted from the bundle. Even\nthough the ref is outdated, I would expect git-bundle(1) to just take\nthe old commit ID.\n\n[1]: https://github.com/git/git/blob/5216f8f5c4089ec29ce49afa147434c23e0f0163/bundle.c#L515\n\n--\nToon\n"},{"id":"488308","messageId":"xmqq5xyxh6zp.fsf@gitster.g","threadId":"60882","inReplyTo":"87eddlpx5k.fsf@iotcl.com","subject":"Re: Race condition in git-bundle(1) create when ref is updated while running","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-02-09T17:39:38Z","receivedAt":"2024-02-09T17:39:42Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Toon Claes <toon@iotcl.com> writes:\n\n> I discovered a bug in git-bundle(1) create. There is a race condition\n> happening when a ref gets updated while the bundle creation process is\n> running.\n\n\"--all\" that tells \"traverse from the tip of all the refs\" to any\nrev-list family of commands (like log and bundle) eventually boils\ndown to opendir(\"refs/...\") followed by readdir(), and if somebody\ncreates or deletes files while you are reading in such a loop,\nreaddir() may appear to skip an entry, which is understandable.\nEven \"git for-each-ref\" would race with a ref update (which involves\nremoving a file and then creating another file at the same path), I\nwould think.  IOW, I do not think this is limited to \"git bundle\".\n\n"},{"id":"488430","messageId":"ZcnGkFI5IiylfGiH@tanuki","threadId":"60882","inReplyTo":"xmqq5xyxh6zp.fsf@gitster.g","subject":"Re: Race condition in git-bundle(1) create when ref is updated while running","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-02-12T07:19:44Z","receivedAt":"2024-02-12T07:19:49Z","isPatch":false,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Fri, Feb 09, 2024 at 09:39:38AM -0800, Junio C Hamano wrote:\n> Toon Claes <toon@iotcl.com> writes:\n> \n> > I discovered a bug in git-bundle(1) create. There is a race condition\n> > happening when a ref gets updated while the bundle creation process is\n> > running.\n> \n> \"--all\" that tells \"traverse from the tip of all the refs\" to any\n> rev-list family of commands (like log and bundle) eventually boils\n> down to opendir(\"refs/...\") followed by readdir(), and if somebody\n> creates or deletes files while you are reading in such a loop,\n> readdir() may appear to skip an entry, which is understandable.\n> Even \"git for-each-ref\" would race with a ref update (which involves\n> removing a file and then creating another file at the same path), I\n> would think.  IOW, I do not think this is limited to \"git bundle\".\n\nYeah, it's unfortunately a general issue with the \"files\" backend and\nnothing that can easily be solved there. The only answer I can provide\nin this context is that the \"reftable\" backend will fix it because it\nenables consistent reads -- sorry for this shameless plug.\n\nPatrick\n"}]}