{"thread":{"id":"66795","subject":"[Outreachy] fast-import: Use repo_for_each_pack() for duplicate checks","startedAt":"2026-11-06T17:44:49Z","lastAt":"2026-11-09T15:25:51Z","messageCount":3,"participants":["Joie Joie","Patrick Steinhardt","Karthik Nayak"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"557720","messageId":"CAH=d44co_ZFA8EJwAoFL4=88ZqWzO65_+GbHDZfjG8amR4cHBw@mail.gmail.com","threadId":"66795","inReplyTo":null,"subject":"[Outreachy] fast-import: Use repo_for_each_pack() for duplicate checks","fromName":"Joie Joie","fromEmail":"soniqjjj@gmail.com","sentAt":"2026-11-06T17:44:49Z","receivedAt":"2026-11-06T17:44:49Z","isPatch":false,"sender":{"key":"soniqjjj@gmail.com","avatar":null},"body":"Hi Patrick,\n\nI was looking through `fast-import.c` and noticed that both\n`store_object()` and `stream_blob()` still loop through `files->dirs`\nto check whether an object is already in a pack.\n\nI saw that Git already has `repo_for_each_pack()`, which is used in\nplaces like `count-objects.c` and `pack-objects.c`.\n\nWould it be worth using that iterator in these two places instead? The\nidea would be to avoid accessing the files backend directly while\nkeeping the current packed-object lookup and duplicate-handling\nbehavior unchanged.\n\nI haven't made any changes yet. I wanted to get your thoughts first\nand check whether this would be a useful follow-up in the current\nrefactoring work.\n\nThanks!\nJoy\n\n"},{"id":"557881","messageId":"avF3yR-PYA3_Ih3v@pks.im","threadId":"66795","inReplyTo":"CAH=d44co_ZFA8EJwAoFL4=88ZqWzO65_+GbHDZfjG8amR4cHBw@mail.gmail.com","subject":"Re: [Outreachy] fast-import: Use repo_for_each_pack() for duplicate checks","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-11-09T07:35:37Z","receivedAt":"2026-11-09T07:35:37Z","isPatch":false,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Hi Joie,\n\nOn Fri, Nov 06, 2026 at 06:44:49PM +0100, Joie Joie wrote:\n> Hi Patrick,\n> \n> I was looking through `fast-import.c` and noticed that both\n> `store_object()` and `stream_blob()` still loop through `files->dirs`\n> to check whether an object is already in a pack.\n> \n> I saw that Git already has `repo_for_each_pack()`, which is used in\n> places like `count-objects.c` and `pack-objects.c`.\n> \n> Would it be worth using that iterator in these two places instead? The\n> idea would be to avoid accessing the files backend directly while\n> keeping the current packed-object lookup and duplicate-handling\n> behavior unchanged.\n\nNo. `repo_for_each_pack()` was introduced as a helper function to make\nthe migration towards pluggable object databases a bit easier. But it is\nultimately still flawed, as it assumes that the backend uses packfiles\nin the first place. So the goal is rather to eventually get rid of this\nmacro altogether in favor of higher-level interfaces that can be\nimplemented by arbitrary backends.\n\nFor git-fast-import(1) this will be done by making use of transactions,\nand Karthik has already started taking a look at it, so I've Cc'd him.\nFor other subsystems we'll need to investigate how to do it properly.\n\nThanks!\n\nPatrick\n\n"},{"id":"557953","messageId":"CAOLa=ZTDwMgOLPTHgkcX=pULFHWSWkwOQoFYVgnL8r1trd7D4Q@mail.gmail.com","threadId":"66795","inReplyTo":"avF3yR-PYA3_Ih3v@pks.im","subject":"Re: [Outreachy] fast-import: Use repo_for_each_pack() for duplicate checks","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2026-11-09T15:25:51Z","receivedAt":"2026-11-09T15:25:51Z","isPatch":false,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":">Patrick Steinhardt <ps@pks.im> writes:\n\n> Hi Joie,\n>\n> On Fri, Nov 06, 2026 at 06:44:49PM +0100, Joie Joie wrote:\n>> Hi Patrick,\n>>\n>> I was looking through `fast-import.c` and noticed that both\n>> `store_object()` and `stream_blob()` still loop through `files->dirs`\n>> to check whether an object is already in a pack.\n>>\n>> I saw that Git already has `repo_for_each_pack()`, which is used in\n>> places like `count-objects.c` and `pack-objects.c`.\n>>\n>> Would it be worth using that iterator in these two places instead? The\n>> idea would be to avoid accessing the files backend directly while\n>> keeping the current packed-object lookup and duplicate-handling\n>> behavior unchanged.\n>\n> No. `repo_for_each_pack()` was introduced as a helper function to make\n> the migration towards pluggable object databases a bit easier. But it is\n> ultimately still flawed, as it assumes that the backend uses packfiles\n> in the first place. So the goal is rather to eventually get rid of this\n> macro altogether in favor of higher-level interfaces that can be\n> implemented by arbitrary backends.\n>\n> For git-fast-import(1) this will be done by making use of transactions,\n> and Karthik has already started taking a look at it, so I've Cc'd him.\n> For other subsystems we'll need to investigate how to do it properly.\n>\n\nYeah, that's the idea. I have started poking around a bit and don't have\nanything concrete yet. I was hoping to send in a series cleaning up the\ntests before getting into the weeds.\n\n> Thanks!\n>\n> Patrick\n"}]}