git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Calculate packfile and idxfile checksums

From
FKFarhan Khan <khanzf@gmail.com>
Date
Jan 18, 2019, 23:51 UTC
Message-ID
<CAFd4kYBo7HCBD56KddWdHH_E=nToQcCuAmBpWw7P6A3O2eYtcA@mail.gmail.com>
In-Reply-To
<87lg3i4b15.fsf@evledraar.gmail.com>

On Fri, Jan 18, 2019 at 3:36 AM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:

Show 36 quoted lines
>
>
> On Fri, Jan 18 2019, Farhan Khan wrote:
>
> > Hi all,
> >
> > I am trying to understand how git's index-pack works, particularly how
> > it calculates the packfile checksum and idxfile checksum.
>
> It's unclear if this is what you're asking, but the pack checksum is
> just a 20 bytes of SHA1 of the preceding contents at the end for *.idx
> and *.pack. As seen with this program:
>
>     $ perl -MFile::Slurp=slurp -MDigest::SHA=sha1 -wE 'my $f = shift; my $c = slurp($f); my $cp = $c; $cp =~ s/.{20}$//s; my $n = $cp . sha1($cp); if ($n eq $c) { say "Computed checksum trailer for $f" } else { say "Failed trailer for new content for $f is different" }' pack-79c2ccce950e6676452dc9f0473f80003e7ccdef.idx
>     Computed checksum trailer for pack-79c2ccce950e6676452dc9f0473f80003e7ccdef.idx
>
> You can also feed it *.pack files.
>
> > I traced back the packfile checksum in the source to the value char
> > *sha1 that is utilized in write_idx_file() in pack_file.c:45. However,
> > I cannot
>
> It seems you mean pack-write.c not pack_file.c
>
> > determine where this value is set.
>
> It's initialized in cmd_index_pack() and then passed down to that
> function.
>
> > My printf() debugging has it set at pack-write.c:171 (right before the
> > hashwrite call) but it does not seem to be utilized prior to that
> > point. Please assist.
>
> I'm happy to help, but still not quite sure what the source of the
> confusion is, maybe that the variable in index-pack.c has a different
> name and is passed down to pack-write.c's function as a pointer?

I apologize for my poor email and appreciate your willingness to help. Let me try that again.

My objective is to identify how to calculate the packfile and idxfile checksums and/or where this is performed in the git source code.

You said that this value is initialized in builtin/index-pack.c:cmd_index_pack(). Is it correct that this value is the 'pack_hash' variable? At a high level, how is this value calculated?

That seems to be the packfile index, how and where is that idxfile hash calculated?

Thanks!
Previous: Ævar Arnfjörð Bjarmason
Message 3 of 3 in “Calculate packfile and idxfile checksums”
  1. Farhan KhanJan 18, 2019
  2. Ævar Arnfjörð BjarmasonJan 18, 2019
  3. Farhan KhanJan 18, 2019

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.