git/list[1] front-page[2] threads[3] people[4] search[5] about
 

the 100 mb push

From
Joey Hess <joey@kitenet.net>
Date
Dec 15, 2009, 19:23 UTC
Message-ID
<20091215192338.GA16654@gnu.kitenet.net>

Is it normal for git push to sometimes transfer much more data than seems necessary? Here is a case where that happens:

joey@gnu:~/src/p.t>git branch
* master
  pristine-tar
  testsuite
joey@gnu:~/src/p.t>git remote show origin
* remote origin
  Fetch URL: ssh://joey@git.kitenet.net/srv/git/pristine-tar.test
  Push  URL: ssh://joey@git.kitenet.net/srv/git/pristine-tar.test
  HEAD branch: master
  Remote branches:
    master       tracked
    pristine-tar tracked
    testsuite    tracked
  Local branches configured for 'git pull':
    master       merges with remote master
    pristine-tar merges with remote pristine-tar
    testsuite    merges with remote testsuite
  Local refs configured for 'git push':
    master       pushes to master       (fast forwardable)
    pristine-tar pushes to pristine-tar (up to date)
    testsuite    pushes to testsuite    (local out of date)

Here, master is a typical small project branch. It has a 1 line change made locally.

Meanwhile, the testsuite branch is a 100+ mb monster, containing a lot of big binaries. In it, a small change has been made in the origin repo. In the local repo, a *lot* of *big* files have been deleted from the same branch, about 20 mb of files were removed all told. But the diff for this change should be quite small.

So, testsuite needs to be merged before it can be pushed, but git push doesn't tell me that. Instead, it goes off and does this for 2+ hours:

joey@gnu:~/src/p.t>git push Counting objects: 241, done. Delta compression using up to 2 threads. Compressing objects: 100% (206/206), done. Writing objects: 15% (36/237), 2.16 MiB | 15 KiB/s ^C

It seems to be uploading the entire repo over the wire, and this is a typical asymmetric network connection, so that goes slow. (Took me a while to realize it was not just auto-gcing the repo locally.)

Once I realized what was going on, it was easy to merge it as shown below, and then the push transferred an appropriatly small amount of data. So, my question is, assuming this is not a straight up bug in git, would it make sense to avoid this gotcha in some way?

joey@gnu:~/src/p.t2>git checkout testsuite
Switched to branch 'testsuite'
Your branch is ahead of 'origin/testsuite' by 1 commit.
joey@gnu:~/src/p.t2>git pull
remote: Counting objects: 5, done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 3 (delta 2), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
From ssh://git.kitenet.net/srv/git/pristine-tar.test
   3c16948..fce7ec1  testsuite  -> origin/testsuite
Merge made by recursive.
 Makefile |    5 ++++-
 1 files changed, 4 insertions(+), 1 deletions(-)
joey@gnu:~/src/p.t2>git push
Counting objects: 13, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (8/8), done.
Writing objects: 100% (8/8), 889 bytes, done.
Total 8 (delta 5), reused 0 (delta 0)
To ssh://joey@git.kitenet.net/srv/git/pristine-tar.test
   aab45a1..cc93945  master -> master
   fce7ec1..d82f225  testsuite -> testsuite
git version 1.6.5.3
-- 
see shy jo
Next: Shawn O. Pearce
Message 1 of 3 in “the 100 mb push”
  1. Joey HessDec 15, 2009
  2. Shawn O. PearceDec 15, 2009
  3. Junio C HamanoDec 15, 2009

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.