git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 11/16] rev-list: add bitmap mode to speed up lists

From
Thomas Rast <trast@inf.ethz.ch>
Date
Jun 25, 2013, 16:22 UTC
Message-ID
<87mwqdlvsq.fsf@linux-k42r.v.cablecom.net>
In-Reply-To
<1372116193-32762-12-git-send-email-tanoku@gmail.com>
Vicent Marti <tanoku@gmail.com> writes:
> Calling `git rev-list --use-bitmaps [committish]` is the equivalent
> of `git rev-list --objects`, but the rev list is performed based on
> a bitmap result instead of using a manual counting objects phase.

Why would we ever want to not --use-bitmaps, once it actually works? I.e., shouldn't this be the default if pack.usebitmaps is set (or possibly even core.usebitmaps for these things)?

Show 13 quoted lines
> These are some example timings for `torvalds/linux`:
>
> 	$ time ../git/git rev-list --objects master > /dev/null
>
> 	real    0m25.567s
> 	user    0m25.148s
> 	sys     0m0.384s
>
> 	$ time ../git/git rev-list --use-bitmaps master > /dev/null
>
> 	real    0m0.393s
> 	user    0m0.356s
> 	sys     0m0.036s
I see your badass numbers, and raise you a critical issue:
  $ time git rev-list --use-bitmaps --count --left-right origin/pu...origin/next
  Segmentation fault
  real    0m0.408s
  user    0m0.383s
  sys     0m0.022s

It actually seems to be related solely to having negated commits in the walk:

  thomas@linux-k42r:~/g(next u+65)$ time git rev-list --use-bitmaps --count origin/pu
  32315
  real    0m0.041s
  user    0m0.034s
  sys     0m0.006s
  thomas@linux-k42r:~/g(next u+65)$ time git rev-list --use-bitmaps --count origin/pu ^origin/next
  Segmentation fault
  real    0m0.460s
  user    0m0.214s
  sys     0m0.244s

I also can't help noticing that the time spent generating the segfault would have sufficed to generate the answer "the old way" as well:

  $ time git rev-list --count --left-right origin/pu...origin/next
  189     125
  real    0m0.409s
  user    0m0.386s
  sys     0m0.022s

Can we use the same trick to speed up merge base computation and then --left-right? The latter is a component of __git_ps1 and can get somewhat slow in some cases, so it would be nice to make it really fast, too.

-- 
Thomas Rast
trast@{inf,student}.ethz.ch
Previous: Vicent MartiNext: Vicent Martí
Message 53 of 64 in “Speed up Counting Objects with bitmap data”
  1. 00/16 Speed up Counting Objects with bitmap dataVicent Marti, Jun 24, 2013
  2. 01/16 list-objects: mark tree as unparsed when we free its bufferVicent Marti, Jun 24, 2013
  3. 02/16 sha1_file: refactor into `find_pack_object_pos`Vicent Marti, Jun 24, 2013
  4. Thomas RastJun 25, 2013
  5. 03/16 pack-objects: use a faster hash tableVicent Marti, Jun 24, 2013
  6. Thomas RastJun 25, 2013
  7. Jeff KingJun 26, 2013
  8. Jeff KingJun 26, 2013
  9. Ramkumar RamachandraJun 25, 2013
  10. Junio C HamanoJun 25, 2013
  11. Vicent MartíJun 25, 2013
  12. 04/16 pack-objects: make `pack_name_hash` globalVicent Marti, Jun 24, 2013
  13. 05/16 revision: allow setting custom limiter functionVicent Marti, Jun 24, 2013
  14. 06/16 sha1_file: export `git_open_noatime`Vicent Marti, Jun 24, 2013
  15. 07/16 compat: add endinanness helpersVicent Marti, Jun 24, 2013
  16. Peter KreftingJun 25, 2013
  17. Vicent MartíJun 25, 2013
  18. Peter KreftingJun 27, 2013
  19. 08/16 ewah: compressed bitmap implementationVicent Marti, Jun 24, 2013
  20. Junio C HamanoJun 25, 2013
  21. Junio C HamanoJun 25, 2013
  22. Thomas RastJun 25, 2013
  23. 09/16 documentation: add documentation for the bitmap formatVicent Marti, Jun 24, 2013
  24. Shawn PearceJun 25, 2013
  25. Vicent MartíJun 25, 2013
  26. Junio C HamanoJun 25, 2013
  27. Vicent MartíJun 25, 2013
  28. Shawn PearceJun 27, 2013
  29. Vicent MartíJun 27, 2013
  30. Jeff KingJun 27, 2013
  31. Shawn PearceJun 27, 2013
  32. Jeff KingJun 27, 2013
  33. Colby RangerJul 1, 2013
  34. Shawn PearceJul 1, 2013
  35. Jeff KingJul 7, 2013
  36. Shawn PearceJul 7, 2013
  37. Jeff KingJun 26, 2013
  38. Colby RangerJun 26, 2013
  39. Colby RangerJun 26, 2013
  40. Colby RangerJun 27, 2013
  41. Shawn PearceJun 27, 2013
  42. Shawn PearceJun 27, 2013
  43. Thomas RastJun 25, 2013
  44. Vicent MartíJun 25, 2013
  45. Thomas RastJun 26, 2013
  46. Thomas RastJun 26, 2013
  47. 10/16 pack-objects: use bitmaps when packing objectsVicent Marti, Jun 24, 2013
  48. Ramkumar RamachandraJun 25, 2013
  49. Thomas RastJun 25, 2013
  50. Junio C HamanoJun 25, 2013
  51. Vicent MartíJun 25, 2013
  52. 11/16 rev-list: add bitmap mode to speed up listsVicent Marti, Jun 24, 2013
  53. Thomas RastJun 25, 2013
  54. Vicent MartíJun 26, 2013
  55. Thomas RastJun 26, 2013
  56. Jeff KingJun 26, 2013
  57. 12/16 pack-objects: implement bitmap writingVicent Marti, Jun 24, 2013
  58. 13/16 repack: consider bitmaps when performing repacksVicent Marti, Jun 24, 2013
  59. Junio C HamanoJun 25, 2013
  60. Vicent MartíJun 25, 2013
  61. 14/16 sha1_file: implement `nth_packed_object_info`Vicent Marti, Jun 24, 2013
  62. 15/16 write-bitmap: implement new git command to write bitmapsVicent Marti, Jun 24, 2013
  63. 16/16 rev-list: Optimize --count using bitmaps tooVicent Marti, Jun 24, 2013
  64. Thomas RastJun 25, 2013

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.