git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: GSoC - Some questions on the idea of

From
NKNeal Kreitzinger <nkreitzinger@gmail.com>
Date
Apr 15, 2012, 02:33 UTC
Message-ID
<4F8A3381.803@gmail.com>
In-Reply-To
<20120415021550.GA24102@sigill.intra.peff.net>
On 4/14/2012 9:15 PM, Jeff King wrote:
Show 14 quoted lines
> On Sat, Apr 14, 2012 at 09:13:17PM -0500, Neal Kreitzinger wrote:
>
>> Does a file's delta-compression efficiency in the pack-file directly
>> correlate to its efficiency of transmission size/bandwidth in a
>> git-fetch and git-push?  IOW, are big-files also a problem for
>> git-fetch and git-push by taking too long in a remote transfer?
> Yes. The on-the-wire format is a packfile. We create a new packfile on
> the fly, so we may find new deltas (e.g., between objects that were
> stored on disk in two different packs), but we will mostly be reusing
> deltas from the existing packs.
>
> So any time you improve the on-disk representation, you are also
> improving the network bandwidth utilization.
>

We use git to transfer database files from the dev server to qa-servers. Sometimes these barf for some reason and I get called to remediate. I assumed the user closed their session prematurely because it was "taking too long". However, now I'm wondering if the git-pull --ff-only is dying on its own due to the big-files. It could be that on a qa-server that hasn't updated database files in awhile they are pulling way more than another qa-server that does their git-pull more requently. How would I go about troubleshooting this? Is there some log files I would look at? (I'm using git 1.7.1 compiled with git makefile on rhel6.) When I go to remediate do git-reset --hard to clear out the barfed worktree/index and then run git-pull --ff-only manually and it always works. I'm not sure if that proves it wasn't git that barfed the first time. Maybe the first time git brought some stuff over and barfed because it bit off more than it could chew, but the second time its really having to chew less food because it already chewed some of it the first time and therefore works the second time.

v/r, neal

Previous: Jeff KingNext: Jeff King
Message 40 of 43 in “GSoC - Some questions on the idea of "Better big-file support".”
  1. Bo ChenMar 28, 2012
  2. Nguyen Thai Ngoc DuyMar 28, 2012
  3. SergioMar 28, 2012
  4. Bo ChenMar 30, 2012
  5. Bo ChenMar 30, 2012
  6. Jeff KingMar 30, 2012
  7. Bo ChenMar 30, 2012
  8. Sergio CallegariMar 31, 2012
  9. Neal KreitzingerMar 31, 2012
  10. Jeff KingApr 2, 2012
  11. Sergio CallegariApr 3, 2012
  12. Neal KreitzingerApr 11, 2012
  13. Jonathan NiederApr 11, 2012
  14. Neal KreitzingerApr 11, 2012
  15. Jeff KingApr 11, 2012
  16. Neal KreitzingerApr 11, 2012
  17. Neal KreitzingerApr 11, 2012
  18. Jonathan NiederApr 11, 2012
  19. Junio C HamanoApr 11, 2012
  20. Jonathan NiederApr 11, 2012
  21. Neal KreitzingerApr 11, 2012
  22. Jeff KingApr 11, 2012
  23. Neal KreitzingerApr 12, 2012
  24. Jeff KingApr 12, 2012
  25. Neal KreitzingerApr 12, 2012
  26. Bo ChenApr 13, 2012
  27. Neal KreitzingerMar 31, 2012
  28. Jeff KingApr 2, 2012
  29. Junio C HamanoApr 2, 2012
  30. Jeff KingApr 3, 2012
  31. Neal KreitzingerMar 31, 2012
  32. Neal KreitzingerMar 31, 2012
  33. Bo ChenMar 31, 2012
  34. Nguyen Thai Ngoc DuyApr 1, 2012
  35. Bo ChenApr 1, 2012
  36. Nguyen Thai Ngoc DuyApr 2, 2012
  37. Bo ChenMar 30, 2012
  38. Jeff KingMar 30, 2012
  39. Jeff KingApr 15, 2012
  40. Neal KreitzingerApr 15, 2012
  41. Jeff KingApr 16, 2012
  42. Neal KreitzingerMay 10, 2012
  43. Jeff KingMay 10, 2012

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.