git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Mercurial 0.4b vs git patchbomb benchmark

From
MMMatt Mackall <mpm@selenic.com>
Date
Apr 30, 2005, 15:20 UTC
Message-ID
<20050430152014.GI21897@waste.org>
In-Reply-To
<20050430025211.GP17379@opteron.random>
On Sat, Apr 30, 2005 at 04:52:11AM +0200, Andrea Arcangeli wrote:
Show 18 quoted lines
> On Fri, Apr 29, 2005 at 01:39:59PM -0700, Matt Mackall wrote:
> > Mercurial is ammenable to rsync provided you devote a read-only
> > repository to it on the client side. In other words, you rsync from
> > kernel.org/mercurial/linus to local/linus and then you merge from
> > local/linus to your own branch. Mercurial's hashing hierarchy is
> > similar to git's (and Monotone's), so you can sign a single hash of
> > the tree as well.
> 
> Ok fine. It's also interesting how you already enabled partial transfers
> through http.
> 
> Please apply this patch so it doesn't fail on my setup ;)
> 
> --- mercurial-0.4b/hg.~1~	2005-04-29 02:52:52.000000000 +0200
> +++ mercurial-0.4b/hg	2005-04-30 00:53:02.000000000 +0200
> @@ -1,4 +1,4 @@
> -#!/usr/bin/python
> +#!/usr/bin/env python
Done.
Show 37 quoted lines
> On a bit more technical side, one thing I'm wondering about is the
> compression. If I change mercurial like this:
> 
> --- revlog.py.~1~	2005-04-29 01:33:14.000000000 +0200
> +++ revlog.py	2005-04-30 03:54:12.000000000 +0200
> @@ -11,9 +11,11 @@
>  import zlib, struct, mdiff, sha, binascii, os, tempfile
>  
>  def compress(text):
> +    return text
>      return zlib.compress(text)
>  
>  def decompress(bin):
> +    return text
>      return zlib.decompress(bin)
>  
>  def hash(text):
> 
> 
> the .hg directory sizes changes from 167M to 302M _BUT_ the _compressed_
> size of the .hg directory (i.e. like in a full network transfer with
> rsync -z or a tar.gz backup) changes from 55M to 38M:
> 
> andrea@opteron:~/devel/kernel> du -sm hg-orig hg-aa hg-orig.tar.bz2 hg-aa.tar.bz2 
> 167     hg-orig
> 302     hg-aa
> 55      hg-orig.tar.bz2
> 38      hg-aa.tar.bz2
> ^^^^^^^^^^^^^^^^^^^^^ 38M backup and network transfer is what I want
> 
> So I don't really see an huge benefit in compression, other than to
> slowdown the checkins measurably [i.e. what Linus doesn't want] (the
> time of compression is a lot higher than the time of python runtime during
> checkin, so it's hard to believe your 100% boost with psyco in the hg file,
> sometime psyco doesn't make any difference infact, I'd rather prefer people to
> work on the real thing of generating native bytecode at compile time, rather
> than at runtime, like some haskell compiler can do).

Most of that psyco speed up is accelerating subsequent diffs in difflib, which you probably didn't hit yet.

Show 9 quoted lines
> mercurial is already good at decreasing the entropy by using an efficient
> storage format, it doesn't need to cheat by putting compression on each blob
> that can only leads to bad ratios when doing backups and while transferring
> more than one blob through the network.
> 
> So I suggest to try disabling compression optionally, perhaps it'll be even
> faster than git in the initial checkin that way! No need of compressing or
> decompressing anything with mercurial (unlike with git that would explode
> without control w/o compression).

I can make it some sort of environment variable, sure. I think the speed is already in a domain where it's not a big deal though. There are other things to do first, like unifying the merge/commit/update code.

> Http is not intended for maximal efficiency, it's there just to make
> life easy. special protocol with zlib is required for maximum
> efficiency.
Yeah, I've got a plan here.
> You also should move the .py into a hg directory, so that they won't
> pollute the site-packages.
Yep, I'm rather new to actually packaging my Python hacks.
-- 
Mathematics is the supreme nostalgia of our time.
Previous: Andrea ArcangeliNext: Andrea Arcangeli
Message 101 of 116 in “Mercurial 0.3 vs git benchmarks”
  1. Matt MackallApr 26, 2005
  2. Daniel PhillipsApr 26, 2005
  3. Linus TorvaldsApr 26, 2005
  4. Mike TahtApr 26, 2005
  5. Linus TorvaldsApr 26, 2005
  6. Linus TorvaldsApr 26, 2005
  7. Chris MasonApr 26, 2005
  8. Magnus DammApr 26, 2005
  9. Chris MasonApr 26, 2005
  10. Magnus DammApr 26, 2005
  11. Chris MasonApr 26, 2005
  12. Andrew MortonApr 26, 2005
  13. Linus TorvaldsApr 26, 2005
  14. H. Peter AnvinApr 26, 2005
  15. Andrew MortonApr 26, 2005
  16. H. Peter AnvinApr 26, 2005
  17. Florian WeimerApr 27, 2005
  18. Thomas GlanzmannApr 27, 2005
  19. H. Peter AnvinApr 27, 2005
  20. Thomas GlanzmannApr 27, 2005
  21. Theodore Ts'oApr 27, 2005
  22. Thomas GlanzmannApr 27, 2005
  23. H. Peter AnvinApr 27, 2005
  24. Thomas GlanzmannApr 27, 2005
  25. Florian WeimerApr 27, 2005
  26. Florian WeimerApr 27, 2005
  27. H. Peter AnvinApr 27, 2005
  28. Florian WeimerApr 27, 2005
  29. Theodore Ts'oApr 27, 2005
  30. Theodore Ts'oApr 27, 2005
  31. Ingo MolnarApr 27, 2005
  32. Bill DavidsenApr 27, 2005
  33. Linus TorvaldsApr 27, 2005
  34. Linus TorvaldsApr 26, 2005
  35. Chris MasonApr 26, 2005
  36. Chris MasonApr 26, 2005
  37. H. Peter AnvinApr 26, 2005
  38. Bill DavidsenApr 26, 2005
  39. Bill DavidsenApr 26, 2005
  40. Matt MackallApr 26, 2005
  41. Linus TorvaldsApr 26, 2005
  42. Chris WedgwoodApr 26, 2005
  43. Andreas GalApr 26, 2005
  44. Linus TorvaldsApr 26, 2005
  45. Mercurial 0.4b vs git patchbomb benchmarkMatt Mackall, Apr 29, 2005
  46. SeanApr 29, 2005
  47. Matt MackallApr 29, 2005
  48. SeanApr 29, 2005
  49. Linus TorvaldsApr 29, 2005
  50. Morten WelinderApr 29, 2005
  51. Matt MackallApr 29, 2005
  52. Bill DavidsenMay 2, 2005
  53. SeanMay 2, 2005
  54. Linus TorvaldsMay 2, 2005
  55. Matt MackallMay 2, 2005
  56. Linus TorvaldsMay 2, 2005
  57. Matt MackallMay 3, 2005
  58. Linus TorvaldsMay 3, 2005
  59. Matt MackallMay 3, 2005
  60. Linus TorvaldsMay 3, 2005
  61. Linus TorvaldsMay 3, 2005
  62. Matt MackallMay 3, 2005
  63. Chris WedgwoodMay 3, 2005
  64. Tom LordApr 29, 2005
  65. Linus TorvaldsApr 29, 2005
  66. Tom LordApr 29, 2005
  67. Linus TorvaldsApr 29, 2005
  68. Tom LordApr 29, 2005
  69. SeanApr 29, 2005
  70. Tom LordApr 29, 2005
  71. SeanApr 29, 2005
  72. Tom LordApr 29, 2005
  73. Tom LordApr 29, 2005
  74. Noel MaddyApr 29, 2005
  75. Tom LordApr 29, 2005
  76. Andrew Timberlake-NewellApr 29, 2005
  77. Tom LordApr 29, 2005
  78. Andrew Timberlake-NewellApr 29, 2005
  79. Morgan SchweersApr 29, 2005
  80. Noel MaddyApr 29, 2005
  81. git network protocolDavid Lang, Apr 29, 2005
  82. Daniel BarkalowApr 29, 2005
  83. Tom LordApr 29, 2005
  84. Denys DuchierApr 29, 2005
  85. Signed commit vulnerabilities? (was: Mercurial 0.4b vs git patchbomb benchmark)Kevin Smith, Apr 29, 2005
  86. Bill DavidsenMay 2, 2005
  87. Matt MackallApr 29, 2005
  88. Linus TorvaldsApr 29, 2005
  89. Matt MackallApr 29, 2005
  90. Linus TorvaldsApr 29, 2005
  91. Matt MackallApr 29, 2005
  92. Linus TorvaldsApr 29, 2005
  93. Matt MackallApr 29, 2005
  94. Bill DavidsenApr 29, 2005
  95. Andrea ArcangeliApr 29, 2005
  96. Olivier GalibertApr 29, 2005
  97. Andrea ArcangeliApr 29, 2005
  98. Andrea ArcangeliApr 29, 2005
  99. Matt MackallApr 29, 2005
  100. Andrea ArcangeliApr 30, 2005
  101. Matt MackallApr 30, 2005
  102. Andrea ArcangeliApr 30, 2005
  103. Bill DavidsenMay 2, 2005
  104. valdis.kletnieks@vt.eduMay 2, 2005
  105. Bill DavidsenMay 3, 2005
  106. David A. WheelerMay 4, 2005
  107. Andrea ArcangeliMay 2, 2005
  108. Linus TorvaldsMay 2, 2005
  109. Daniel JacobowitzMay 2, 2005
  110. Linus TorvaldsMay 2, 2005
  111. Edgar ToernigMay 2, 2005
  112. Sam RavnborgMay 2, 2005
  113. Ryan AndersonMay 2, 2005
  114. Linus TorvaldsMay 2, 2005
  115. Kyle MoffettMay 2, 2005
  116. Bill DavidsenMay 3, 2005

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.