git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: support for large packs and 64-bit offsets

From
Shawn O. Pearce <spearce@spearce.org>
Date
Apr 9, 2007, 17:43 UTC
Message-ID
<20070409174305.GU5436@spearce.org>
In-Reply-To
<alpine.LFD.0.98.0704091328130.28181@xanadu.home>
Nicolas Pitre <nico@cam.org> wrote:
Show 11 quoted lines
> On Mon, 9 Apr 2007, Shawn O. Pearce wrote:
> 
> > It is unfortunate that we are changing the index file format without
> > also bringing in packv4 support at the same time.  I have just been
> > too swamped in useless bulls**t in day-job work to spend time on
> > Git lately.
> 
> Well... I still did index v2 with pack v4 in mind.  The diference 
> between index v2 and v3 would be minimal.
> 
> Pack v4 is coming along.  Slowly but still coming.

I take it you are working on it alone at this point? I'd love to get back into it, but I don't think I've got the cycles for at least a couple of weeks.

Here's something we didn't think about, but that occurred to me today when reading this series: If we move the SHA-1 table out of the index and into the packfile (like we are planning) dumb commit-walkers (http-fetch) will have problems. Right now they download the indexes of every available packfile to determine if they need to download the corresponding packfile to obtain a needed object.

Moving the SHA-1 table from the index into the packfile will mean the client cannot do this `optimization'. Instead it will need to perform a byte-range request for part of the packfile to decide if it needs to fetch the remainder of that packfile; or it must download the entire packfile. Since not all HTTP servers support byte-range requests the former may not always be viable and the latter is obviously not a good idea.

-- 
Shawn.
Previous: Nicolas PitreNext: Junio C Hamano
Message 16 of 24 in “support for large packs and 64-bit offsets”
  1. Nicolas PitreApr 9, 2007
  2. 01/10 get rid of num_packed_objects()Nicolas Pitre, Apr 9, 2007
  3. 02/10 make overflow test on delta base offset work regardless of variable sizeNicolas Pitre, Apr 9, 2007
  4. 03/10 add overflow tests on pack offset variablesNicolas Pitre, Apr 9, 2007
  5. 04/10 compute a CRC32 for each object as stored in a packNicolas Pitre, Apr 9, 2007
  6. 05/10 compute object CRC32 with index-packNicolas Pitre, Apr 9, 2007
  7. 06/10 pack-objects: learn about pack index version 2Nicolas Pitre, Apr 9, 2007
  8. 07/10 index-pack: learn about pack index version 2Nicolas Pitre, Apr 9, 2007
  9. 08/10 sha1_file.c: learn about index version 2Nicolas Pitre, Apr 9, 2007
  10. 09/10 show-index.c: learn about index v2Nicolas Pitre, Apr 9, 2007
  11. 10/10 pack-redundant.c: learn about index v2Nicolas Pitre, Apr 9, 2007
  12. Junio C HamanoApr 9, 2007
  13. Nicolas PitreApr 9, 2007
  14. Shawn O. PearceApr 9, 2007
  15. Nicolas PitreApr 9, 2007
  16. Shawn O. PearceApr 9, 2007
  17. Junio C HamanoApr 9, 2007
  18. Shawn O. PearceApr 9, 2007
  19. Nicolas PitreApr 9, 2007
  20. Junio C HamanoApr 9, 2007
  21. Linus TorvaldsApr 9, 2007
  22. Nicolas PitreApr 9, 2007
  23. Shawn O. PearceApr 9, 2007
  24. Nicolas PitreApr 9, 2007

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.