git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 11/16] rev-list: add bitmap mode to speed up lists

From
Thomas Rast <trast@inf.ethz.ch>
Date
Jun 26, 2013, 23:13 UTC
Message-ID
<87obaslb2g.fsf@linux-k42r.v.cablecom.net>
In-Reply-To
<CAFFjANSYoRGFDx109kMWJtYAO4TaTwSW0NCaemnrERuwakfpGg@mail.gmail.com>
Vicent Martí <tanoku@gmail.com> writes:
> I'm afraid I cannot reproduce the segfault locally (assuming you're
> performing the rev-list on the git/git repository). Could you please
> send me more information, and a core dump if possible?

Sure, but isn't the core dump useless if you don't have the same executable? And since I'm building "custom" git, you won't have that.

Here's a semi-full backtrace (I left out the spammy output in the outermost frames). Some variables in #2 and #3 seem to have gone off the rails.

#0  0x00007ffff72b06fb in __memset_sse2 () from /lib64/libc.so.6
No symbol table info available.
#1  0x000000000054c31c in bitmap_set (self=0x89c360, pos=18446744072278122040) at ewah/bitmap.c:46
        old_size = 7666
        block = 288230376129345656
#2  0x00000000004e6c70 in add_to_include_set (data=0x7fffffffcd00, sha1=0x85c014 "\230\062˝M\311i\373\372\317\321\370\224\017\313\336\301\213\271\060", bitmap_pos=-1431429576) at pack-bitmap.c:428
        hash_pos = 512
#3  0x00000000004e6cd6 in should_include (commit=0x85c010, _data=0x7fffffffcd00) at pack-bitmap.c:443
        data = 0x7fffffffcd00
        bitmap_pos = -1431429576
#4  0x000000000050cf1d in add_parents_to_list (revs=0x7fffffffce30, commit=0x85c010, list=0x7fffffffce30, cache_ptr=0x0) at revision.c:784
        parent = 0x88c260
        left_flag = 32767
        cached_base = 0x0
#5  0x0000000000512b66 in get_revision_1 (revs=0x7fffffffce30) at revision.c:2857
        entry = 0x8f9ce0
        commit = 0x85c010
#6  0x0000000000512dcf in get_revision_internal (revs=0x7fffffffce30) at revision.c:2964
        c = 0x0
        l = 0x1000
#7  0x0000000000512fe1 in get_revision (revs=0x7fffffffce30) at revision.c:3040
        c = 0xb92608
        reversed = 0x89c360
#8  0x00000000004d2a24 in traverse_commit_list (revs=0x7fffffffce30, show_commit=0x4e6b72 <show_commit>, show_object=0x4e6afa <show_object>, data=0x89c360) at list-objects.c:179
        i = -1
        commit = 0xb92608
        base = {
          alloc = 4097, 
          len = 0, 
          buf = 0x87bbe0 ""
        }
#9  0x00000000004e6fa4 in find_objects (revs=0x7fffffffce30, roots=0x0, seen=0x85b760) at pack-bitmap.c:549
        incdata = {
          base = 0x89c360, 
          seen = 0x85b760
        }
        base = 0x89c360
        needs_walk = true
        not_mapped = 0x8f9dc0
#10 0x00000000004e747b in prepare_bitmap_walk (revs=0x7fffffffce30, result_size=0x0) at pack-bitmap.c:679
        i = 2
        pending_nr = 2
        pending_alloc = 64
        pending_e = 0x853e10
        wants = 0x8545b0
        haves = 0x854820
        wants_bitmap = 0x0
        haves_bitmap = 0x85b760
#11 0x0000000000474bb3 in cmd_rev_list (argc=2, argv=0x7fffffffd6e8, prefix=0x0) at builtin/rev-list.c:356
#12 0x0000000000405820 in run_builtin (p=0x7c3ef8 <commands.20770+2040>, argc=4, argv=0x7fffffffd6e8) at git.c:291
#13 0x00000000004059b3 in handle_internal_command (argc=4, argv=0x7fffffffd6e8) at git.c:454
#14 0x0000000000405b87 in main (argc=4, av=0x7fffffffd6e8) at git.c:544
This is with a version of your series that you can find at
  https://github.com/trast/git.git vm/ewah

I am'd your patches on top of Junio's master at the time, except for the parts to the Makefile that did not apply, which I fixed up manually.

-- 
Thomas Rast
trast@{inf,student}.ethz.ch
Previous: Vicent MartíNext: Jeff King
Message 55 of 64 in “Speed up Counting Objects with bitmap data”
  1. 00/16 Speed up Counting Objects with bitmap dataVicent Marti, Jun 24, 2013
  2. 01/16 list-objects: mark tree as unparsed when we free its bufferVicent Marti, Jun 24, 2013
  3. 02/16 sha1_file: refactor into `find_pack_object_pos`Vicent Marti, Jun 24, 2013
  4. Thomas RastJun 25, 2013
  5. 03/16 pack-objects: use a faster hash tableVicent Marti, Jun 24, 2013
  6. Thomas RastJun 25, 2013
  7. Jeff KingJun 26, 2013
  8. Jeff KingJun 26, 2013
  9. Ramkumar RamachandraJun 25, 2013
  10. Junio C HamanoJun 25, 2013
  11. Vicent MartíJun 25, 2013
  12. 04/16 pack-objects: make `pack_name_hash` globalVicent Marti, Jun 24, 2013
  13. 05/16 revision: allow setting custom limiter functionVicent Marti, Jun 24, 2013
  14. 06/16 sha1_file: export `git_open_noatime`Vicent Marti, Jun 24, 2013
  15. 07/16 compat: add endinanness helpersVicent Marti, Jun 24, 2013
  16. Peter KreftingJun 25, 2013
  17. Vicent MartíJun 25, 2013
  18. Peter KreftingJun 27, 2013
  19. 08/16 ewah: compressed bitmap implementationVicent Marti, Jun 24, 2013
  20. Junio C HamanoJun 25, 2013
  21. Junio C HamanoJun 25, 2013
  22. Thomas RastJun 25, 2013
  23. 09/16 documentation: add documentation for the bitmap formatVicent Marti, Jun 24, 2013
  24. Shawn PearceJun 25, 2013
  25. Vicent MartíJun 25, 2013
  26. Junio C HamanoJun 25, 2013
  27. Vicent MartíJun 25, 2013
  28. Shawn PearceJun 27, 2013
  29. Vicent MartíJun 27, 2013
  30. Jeff KingJun 27, 2013
  31. Shawn PearceJun 27, 2013
  32. Jeff KingJun 27, 2013
  33. Colby RangerJul 1, 2013
  34. Shawn PearceJul 1, 2013
  35. Jeff KingJul 7, 2013
  36. Shawn PearceJul 7, 2013
  37. Jeff KingJun 26, 2013
  38. Colby RangerJun 26, 2013
  39. Colby RangerJun 26, 2013
  40. Colby RangerJun 27, 2013
  41. Shawn PearceJun 27, 2013
  42. Shawn PearceJun 27, 2013
  43. Thomas RastJun 25, 2013
  44. Vicent MartíJun 25, 2013
  45. Thomas RastJun 26, 2013
  46. Thomas RastJun 26, 2013
  47. 10/16 pack-objects: use bitmaps when packing objectsVicent Marti, Jun 24, 2013
  48. Ramkumar RamachandraJun 25, 2013
  49. Thomas RastJun 25, 2013
  50. Junio C HamanoJun 25, 2013
  51. Vicent MartíJun 25, 2013
  52. 11/16 rev-list: add bitmap mode to speed up listsVicent Marti, Jun 24, 2013
  53. Thomas RastJun 25, 2013
  54. Vicent MartíJun 26, 2013
  55. Thomas RastJun 26, 2013
  56. Jeff KingJun 26, 2013
  57. 12/16 pack-objects: implement bitmap writingVicent Marti, Jun 24, 2013
  58. 13/16 repack: consider bitmaps when performing repacksVicent Marti, Jun 24, 2013
  59. Junio C HamanoJun 25, 2013
  60. Vicent MartíJun 25, 2013
  61. 14/16 sha1_file: implement `nth_packed_object_info`Vicent Marti, Jun 24, 2013
  62. 15/16 write-bitmap: implement new git command to write bitmapsVicent Marti, Jun 24, 2013
  63. 16/16 rev-list: Optimize --count using bitmaps tooVicent Marti, Jun 24, 2013
  64. Thomas RastJun 25, 2013

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.