git/list[1] front-page[2] threads[3] people[4] search[5] about
 

git-fast-import yields huge packfile

From
RHRichard Hipp <drh@sqlite.org>
Date
Mar 16, 2019, 20:31 UTC
Message-ID
<CALwJ=MzrqPUNw=jc0NRtaJaJG+ErXNb577JNSN66GiGY4UFtRw@mail.gmail.com>

I'm trying to transform a repository from another VCS into a Git repository using "git fast-import". It appears to work, but the resulting Git repository is huge relative to the original - 18 times larger. Most of the space seems to be taken up by a single large packfile. That packfile is about 967 MB which is about 1/4th the total uncompressed size of all 41785 distinct Blobs in the original repository. The source VCS is able to compress this down to 52 MB by comparison.

Maybe I'm doing something wrong with the fast-import stream that is defeating Git's attempts at delta compression....

Are there any utility programs available for analyzing packfiles so that I try to figure out where the inefficiencies are cropping up, so that I can try to address them?

Anybody have any suggestions on what I should be looking for?

If anyone would care to see this oversized packfile and perhaps offer suggestions on how I can make it more space-efficient, it can be cloned from https://github.com/drhsqlite/fossil-mirror.git - at least for now - surely I will delete that repo and regenerate it once I figure out this problem.

-- 
D. Richard Hipp
drh@sqlite.org
Next: Linus Torvalds
Message 1 of 6 in “git-fast-import yields huge packfile”
  1. Richard HippMar 16, 2019
  2. Linus TorvaldsMar 16, 2019
  3. Mike HommeyMar 16, 2019
  4. Richard HippMar 16, 2019
  5. Johannes SchindelinMar 21, 2019
  6. Ævar Arnfjörð BjarmasonMar 21, 2019

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.