{"thread":{"id":"50257","subject":"Calculate packfile and idxfile checksums","startedAt":"2019-01-18T07:24:46Z","lastAt":"2019-01-18T23:51:17Z","messageCount":3,"participants":["Farhan Khan","Ævar Arnfjörð Bjarmason"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"367053","messageId":"CAFd4kYAbrUcV2U1u6Fz3VgZDN_YcpLrSymeA5bvvLy1LY98aHA@mail.gmail.com","threadId":"50257","inReplyTo":null,"subject":"Calculate packfile and idxfile checksums","fromName":"Farhan Khan","fromEmail":"khanzf@gmail.com","sentAt":"2019-01-18T07:24:32Z","receivedAt":"2019-01-18T07:24:46Z","isPatch":false,"sender":{"key":"khanzf@gmail.com","avatar":null},"body":"Hi all,\n\nI am trying to understand how git's index-pack works, particularly how\nit calculates the packfile checksum and idxfile checksum. I traced\nback the packfile checksum in the source to the value char *sha1 that\nis utilized in write_idx_file() in pack_file.c:45. However, I cannot\ndetermine where this value is set.\n\nMy printf() debugging has it set at pack-write.c:171 (right before the\nhashwrite call) but it does not seem to be utilized prior to that\npoint. Please assist.\n\nThanks!\n--\nFarhan Khan\nPGP Fingerprint: B28D 2726 E2BC A97E 3854 5ABE 9A9F 00BC D525 16EE\n"},{"id":"367062","messageId":"87lg3i4b15.fsf@evledraar.gmail.com","threadId":"50257","inReplyTo":"CAFd4kYAbrUcV2U1u6Fz3VgZDN_YcpLrSymeA5bvvLy1LY98aHA@mail.gmail.com","subject":"Re: Calculate packfile and idxfile checksums","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-01-18T08:36:54Z","receivedAt":"2019-01-18T08:36:59Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Fri, Jan 18 2019, Farhan Khan wrote:\n\n> Hi all,\n>\n> I am trying to understand how git's index-pack works, particularly how\n> it calculates the packfile checksum and idxfile checksum.\n\nIt's unclear if this is what you're asking, but the pack checksum is\njust a 20 bytes of SHA1 of the preceding contents at the end for *.idx\nand *.pack. As seen with this program:\n\n    $ perl -MFile::Slurp=slurp -MDigest::SHA=sha1 -wE 'my $f = shift; my $c = slurp($f); my $cp = $c; $cp =~ s/.{20}$//s; my $n = $cp . sha1($cp); if ($n eq $c) { say \"Computed checksum trailer for $f\" } else { say \"Failed trailer for new content for $f is different\" }' pack-79c2ccce950e6676452dc9f0473f80003e7ccdef.idx\n    Computed checksum trailer for pack-79c2ccce950e6676452dc9f0473f80003e7ccdef.idx\n\nYou can also feed it *.pack files.\n\n> I traced back the packfile checksum in the source to the value char\n> *sha1 that is utilized in write_idx_file() in pack_file.c:45. However,\n> I cannot\n\nIt seems you mean pack-write.c not pack_file.c\n\n> determine where this value is set.\n\nIt's initialized in cmd_index_pack() and then passed down to that\nfunction.\n\n> My printf() debugging has it set at pack-write.c:171 (right before the\n> hashwrite call) but it does not seem to be utilized prior to that\n> point. Please assist.\n\nI'm happy to help, but still not quite sure what the source of the\nconfusion is, maybe that the variable in index-pack.c has a different\nname and is passed down to pack-write.c's function as a pointer?\n"},{"id":"367161","messageId":"CAFd4kYBo7HCBD56KddWdHH_E=nToQcCuAmBpWw7P6A3O2eYtcA@mail.gmail.com","threadId":"50257","inReplyTo":"87lg3i4b15.fsf@evledraar.gmail.com","subject":"Re: Calculate packfile and idxfile checksums","fromName":"Farhan Khan","fromEmail":"khanzf@gmail.com","sentAt":"2019-01-18T23:51:03Z","receivedAt":"2019-01-18T23:51:17Z","isPatch":false,"sender":{"key":"khanzf@gmail.com","avatar":null},"body":"On Fri, Jan 18, 2019 at 3:36 AM Ævar Arnfjörð Bjarmason\n<avarab@gmail.com> wrote:\n>\n>\n> On Fri, Jan 18 2019, Farhan Khan wrote:\n>\n> > Hi all,\n> >\n> > I am trying to understand how git's index-pack works, particularly how\n> > it calculates the packfile checksum and idxfile checksum.\n>\n> It's unclear if this is what you're asking, but the pack checksum is\n> just a 20 bytes of SHA1 of the preceding contents at the end for *.idx\n> and *.pack. As seen with this program:\n>\n>     $ perl -MFile::Slurp=slurp -MDigest::SHA=sha1 -wE 'my $f = shift; my $c = slurp($f); my $cp = $c; $cp =~ s/.{20}$//s; my $n = $cp . sha1($cp); if ($n eq $c) { say \"Computed checksum trailer for $f\" } else { say \"Failed trailer for new content for $f is different\" }' pack-79c2ccce950e6676452dc9f0473f80003e7ccdef.idx\n>     Computed checksum trailer for pack-79c2ccce950e6676452dc9f0473f80003e7ccdef.idx\n>\n> You can also feed it *.pack files.\n>\n> > I traced back the packfile checksum in the source to the value char\n> > *sha1 that is utilized in write_idx_file() in pack_file.c:45. However,\n> > I cannot\n>\n> It seems you mean pack-write.c not pack_file.c\n>\n> > determine where this value is set.\n>\n> It's initialized in cmd_index_pack() and then passed down to that\n> function.\n>\n> > My printf() debugging has it set at pack-write.c:171 (right before the\n> > hashwrite call) but it does not seem to be utilized prior to that\n> > point. Please assist.\n>\n> I'm happy to help, but still not quite sure what the source of the\n> confusion is, maybe that the variable in index-pack.c has a different\n> name and is passed down to pack-write.c's function as a pointer?\n\nI apologize for my poor email and appreciate your willingness to help.\nLet me try that again.\n\nMy objective is to identify how to calculate the packfile and idxfile\nchecksums and/or where this is performed in the git source code.\n\nYou said that this value is initialized in\nbuiltin/index-pack.c:cmd_index_pack(). Is it correct that this value\nis the 'pack_hash' variable? At a high level, how is this value\ncalculated?\n\nThat seems to be the packfile index, how and where is that idxfile\nhash calculated?\n\nThanks!\n"}]}