git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Manually decoding a git object

From
Philip Oakley <philipoakley@iee.org>
Date
Feb 20, 2012, 18:27 UTC
Message-ID
<EF3174E8E5D5433CBC44E912309946A8@PhilipOakley>
In-Reply-To
<871uppbwnu.fsf@thomas.inf.ethz.ch>
From: "Thomas Rast" <trast@inf.ethz.ch> Sent: Monday, February 20, 2012
10:56 AM
Show 53 quoted lines
> Philip Oakley <philipoakley@iee.org> writes:
>
>> From: "Thomas Rast" <trast@inf.ethz.ch> Sent: Monday, February 20,
>> 2012 8:29 AM
>>>
>>> The SHA1 is over the decompressed object contents.  The file simply
>>> holds a zlib-compressed stream of those contents.  (It's pretty much
>>> like gzip without the file header.)
>>>
>>> You can use any bindings to zlib and something that does sha1, e.g. in
>>> python:
>>>
>>>  $ cd g/.git/objects/aa/  # my git.git
>>>  $ ls
>>>  592bda986a8380b64acd8cbb3d5bdfcbc0834d
>>> 6322a757bee31919f54edcc127608a3d724c99
>>>  $ python
>>>  Python 2.7.2 (default, Aug 19 2011, 20:41:43) [GCC] on linux2
>>>  Type "help", "copyright", "credits" or "license" for more information.
>>>  >>> import hashlib
>>>  >>>
>>> hashlib.sha1(open('592bda986a8380b64acd8cbb3d5bdfcbc0834d').read().decode('zlib')).digest().encode('hex')
>>>  'aa592bda986a8380b64acd8cbb3d5bdfcbc0834d'
>>>
>>> Notice that the first byte of the hash goes into the directory name.
>>>
>>
>> At the moment I'm in a Catch 22 situation where I can't make the first
>> step of examining the deflated contents, so I can't do all those next
>> steps to get the sha1 etc.. Have I misunderstood your suggestions?
>
> Huh?  The method I showed does not rely on knowing the SHA1.  The fact
> that I used it on a properly filed away (by its SHA1) object file is
> immaterial, if perhaps confusing.
>
> I can untangle that python expression for you:
>
> hashlib.sha1(foo).digest()      gives the SHA1 digest of the string foo,
> as a (binary) string
> foo.encode('hex')               turns foo from (binary) string into its
> hex representation
> open('filename').read()         opens the file called filename, and
> returns its whole contents
> foo.decode('zlib')              applies the zlib decompressor to foo, and
> returns the resulting data
>
> So that trick works for any file[*], and you can then use its results to
> file it back where it needs to go.
>
>
> [*] that is sufficiently small for Python to hold it in memory, but git
> shares the same problems in that department.
>

I see what you mean now. I'll need to work out how to get Python in msysgit - the 'minimal' part of msys keeps on biting... That is, I didn't see it (Python) in the 1.7.8 full install bash - I didn't see it anyway.

I was hopeful that unzip/gunzip would have an option to simply deflate a (file)stream, rather than it expecting the normal file archive.

Previous: 徐迪
Message 17 of 17 in “Document format of basic Git objects”
  1. Document format of basic Git objectsNguyễn Thái Ngọc Duy, Feb 15, 2012
  2. Jonathan NiederFeb 15, 2012
  3. Junio C HamanoFeb 15, 2012
  4. Junio C HamanoFeb 16, 2012
  5. Document format of basic Git objectsNguyễn Thái Ngọc Duy, Feb 19, 2012
  6. Junio C HamanoFeb 19, 2012
  7. Junio C HamanoFeb 19, 2012
  8. Nguyen Thai Ngoc DuyFeb 20, 2012
  9. Jeff KingFeb 20, 2012
  10. Manually decoding a git objectPhilip Oakley, Feb 19, 2012
  11. 徐迪Feb 20, 2012
  12. Philip OakleyFeb 20, 2012
  13. Thomas RastFeb 20, 2012
  14. Philip OakleyFeb 20, 2012
  15. Thomas RastFeb 20, 2012
  16. 徐迪Feb 20, 2012
  17. Philip OakleyFeb 20, 2012

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.