git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: git --archive

From
Junio C Hamano <gitster@pobox.com>
Date
Sep 22, 2022, 20:13 UTC
Message-ID
<xmqqleqbxj4h.fsf@gitster.g>
In-Reply-To
<PH0PR06MB763962EB6321F85803C5CE2D864E9@PH0PR06MB7639.namprd06.prod.outlook.com>
"Scheffenegger, Richard" <Richard.Scheffenegger@netapp.com> writes:
Show 5 quoted lines
> Unless I’m mistaken, the procedure to create a zip archive reads like a recursive collection of all relevant objects, and then writing them out sequentially, in a single thread. 
>
> Is this assessment correct?
>
> I was wondering if a highly concurrent fetching phase could be optionally added…

The details matter here, I think. Enumerating and slurping down the contents to be archived out of the repository/object store to the core can indeed be made parallel, but the end result product being a zip archive or a tarball, which is fairly a serialized output format, there is only so much you can hold in core, and it is not clear what your plan is to do this without filling all the memory.

Previous: Scheffenegger, RichardNext: Scheffenegger, Richard
Message 2 of 12 in “git --archive”
  1. Scheffenegger, RichardSep 22, 2022
  2. Junio C HamanoSep 22, 2022
  3. Scheffenegger, RichardSep 22, 2022
  4. brian m. carlsonSep 23, 2022
  5. Junio C HamanoSep 23, 2022
  6. Scheffenegger, RichardSep 23, 2022
  7. René ScharfeSep 24, 2022
  8. Scheffenegger, RichardSep 24, 2022
  9. René ScharfeSep 24, 2022
  10. René ScharfeSep 24, 2022
  11. René ScharfeSep 25, 2022
  12. Scheffenegger, RichardSep 24, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.