git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH] RFC: git lazy clone proof-of-concept

From
HHHarvey Harrison <harvey.harrison@gmail.com>
Date
Feb 8, 2008, 18:14 UTC
Message-ID
<1202494475.31361.45.camel@brick>
In-Reply-To
<200802081828.43849.kendy@suse.cz>
On Fri, 2008-02-08 at 18:28 +0100, Jan Holesovsky wrote:
Show 27 quoted lines
> Hi,
> 
> This is my attempt to implement the 'lazy clone' I've read about a bit in the
> git mailing list archive, but did not see implemented anywhere - the clone
> that fetches a minimal amount of data with the possibility to download the
> rest later (transparently!) when necessary.  I am sorry to send it as a huge
> patch, not as a series of patches, but as I don't know if I chose a way that is
> acceptable for you [I'm new to the git code ;-)], I'd like to hear some
> feedback first, and then I'll split it into smaller pieces for easier
> integration - if OK.
> 
> Background:
> 
> Currently we are evaluating the usage of git for OpenOffice.org as one of the
> candidates (SVN is the other one), see
> 
>   http://wiki.services.openoffice.org/wiki/SCM_Migration
> 
> I've provided a git import of OOo with the entire history; the problem is that
> the pack has 2.5G, so it's not too convenient to download for casual
> developers that just want to try it.  Shallow clone is not a possibility - we
> don't get patches through mailing lists, so we need the pull/push, and also
> thanks to the OOo development cycle, we have too many living heads which
> causes the shallow clone to download about 1.5G even with --depth 1.  Lazy
> clone sounded like the right idea to me.  With this proof-of-concept
> implementation, just about 550M from the 2.5G is downloaded, which is still
> about twice as much in comparison with downloading a tarball, but bearable.

For comparison, how big was the svn repo you're testing? My experience has been about 15-20 times smaller than SVN once a tuned repack has been done.

Cheers,
Harvey
Previous: Johannes SchindelinNext: Jan Holesovsky
Message 12 of 85 in “RFC: git lazy clone proof-of-concept”
  1. RFC: git lazy clone proof-of-conceptJan Holesovsky, Feb 8, 2008
  2. Nicolas PitreFeb 8, 2008
  3. Jan HolesovskyFeb 9, 2008
  4. Mike HommeyFeb 9, 2008
  5. Nicolas PitreFeb 9, 2008
  6. Marco CostalbaFeb 10, 2008
  7. Johannes SchindelinFeb 10, 2008
  8. David SymondsFeb 10, 2008
  9. Johannes SchindelinFeb 10, 2008
  10. Nicolas PitreFeb 10, 2008
  11. Johannes SchindelinFeb 10, 2008
  12. Harvey HarrisonFeb 8, 2008
  13. Jan HolesovskyFeb 9, 2008
  14. Johannes SchindelinFeb 8, 2008
  15. Mike HommeyFeb 8, 2008
  16. Johannes SchindelinFeb 8, 2008
  17. Jan HolesovskyFeb 9, 2008
  18. Jakub NarebskiFeb 8, 2008
  19. Jon SmirlFeb 8, 2008
  20. Nicolas PitreFeb 8, 2008
  21. Andreas EricssonFeb 11, 2008
  22. 1/2 pack-objects: Allow setting the #threads equal to #cpus automaticallyBrandon Casey, Feb 12, 2008
  23. Andreas EricssonFeb 12, 2008
  24. Harvey HarrisonFeb 8, 2008
  25. Jon SmirlFeb 8, 2008
  26. Harvey HarrisonFeb 8, 2008
  27. Jon SmirlFeb 8, 2008
  28. Jan HolesovskyFeb 9, 2008
  29. Nicolas PitreFeb 10, 2008
  30. SeanFeb 10, 2008
  31. Nicolas PitreFeb 10, 2008
  32. SeanFeb 10, 2008
  33. Jakub NarebskiFeb 11, 2008
  34. Nicolas PitreFeb 11, 2008
  35. Jakub NarebskiFeb 11, 2008
  36. Joachim B HagaFeb 10, 2008
  37. Johannes SchindelinFeb 10, 2008
  38. Jon SmirlFeb 10, 2008
  39. Johannes SchindelinFeb 10, 2008
  40. Johannes SchindelinFeb 10, 2008
  41. Nicolas PitreFeb 10, 2008
  42. Jon SmirlFeb 10, 2008
  43. Johannes SchindelinFeb 12, 2008
  44. Nicolas PitreFeb 12, 2008
  45. Linus TorvaldsFeb 12, 2008
  46. Jon SmirlFeb 12, 2008
  47. Linus TorvaldsFeb 12, 2008
  48. Linus TorvaldsFeb 12, 2008
  49. Jon SmirlFeb 12, 2008
  50. Linus TorvaldsFeb 12, 2008
  51. Jon SmirlFeb 12, 2008
  52. Johannes SchindelinFeb 14, 2008
  53. Jakub NarebskiFeb 14, 2008
  54. Nicolas PitreFeb 14, 2008
  55. Johannes SchindelinFeb 14, 2008
  56. Jakub NarebskiFeb 14, 2008
  57. Johannes SchindelinFeb 14, 2008
  58. Brian DowningFeb 14, 2008
  59. Brian DowningFeb 14, 2008
  60. Johannes SchindelinFeb 15, 2008
  61. Nicolas PitreFeb 15, 2008
  62. Shawn O. PearceFeb 17, 2008
  63. Junio C HamanoFeb 17, 2008
  64. Nicolas PitreFeb 17, 2008
  65. Jakub NarebskiFeb 15, 2008
  66. Jan HolesovskyFeb 15, 2008
  67. Brandon CaseyFeb 14, 2008
  68. Jan HolesovskyFeb 15, 2008
  69. Nicolas PitreFeb 10, 2008
  70. Brandon CaseyFeb 14, 2008
  71. Johannes SchindelinFeb 14, 2008
  72. Nicolas PitreFeb 14, 2008
  73. Jakub NarebskiFeb 11, 2008
  74. Johannes SchindelinFeb 8, 2008
  75. Jakub NarebskiFeb 8, 2008
  76. Johannes SchindelinFeb 8, 2008
  77. Mike HommeyFeb 8, 2008
  78. Johannes SchindelinFeb 8, 2008
  79. Mike HommeyFeb 8, 2008
  80. Johannes SchindelinFeb 8, 2008
  81. Mike HommeyFeb 8, 2008
  82. Jan HudecFeb 9, 2008
  83. Jan HolesovskyFeb 9, 2008
  84. 2/2 pack-objects: Default to zero threads, meaning auto-assign to #cpusBrandon Casey, Feb 12, 2008
  85. Nicolas PitreFeb 12, 2008

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.