git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v3 0/21] pack bitmaps

From
Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Date
Nov 14, 2013, 19:19 UTC
Message-ID
<5285224A.2070606@ramsay1.demon.co.uk>
In-Reply-To
<20131114124157.GA23784@sigill.intra.peff.net>
On 14/11/13 12:41, Jeff King wrote:
Show 6 quoted lines
> Here's another iteration of the pack bitmaps series. Compared to v2, it
> changes:
> 
>  - misc style/typo fixes
> 
>  - portability fixes from Ramsay and Torsten

Unfortunately, I didn't find time this weekend to finish the msvc build fixes. However, after a quick squint at these patches, I think you have almost done it for me! :-D

I must have misunderstood the previous discussion, because my patch was written on the assumption that the ewah directory wouldn't be "git-ified" (e.g. #include git-compat-util.h).

So, most of my patch is no longer necessary, given the use of the git compat header (and removal of system headers). I suspect that you only need to add an '#define PRIx64 "I64x"' definition (Hmm, probably to the compat/mingw.h header).

I won't know for sure until I actually try them out, of course. I will wait until these patches land in pu.

[Note: the msvc build is still broken, but the failure is not caused by these patches. Unfortunately, the tests in t5310-*.sh fail. However, if I include some debug code, the tests pass ... :-P ]

The part of the patch I was still working on was ...
Show 21 quoted lines
> 
>  - count-objects garbage-reporting patch from Duy
> 
>  - disable bitmaps when is_repository_shallow(); this also covers the
>    case where the client is shallow, since we feed pack-objects a
>    --shallow-file in that case. This used to done by checking
>    !internal_rev_list, but that doesn't apply after cdab485.
> 
>  - ewah sources now properly use git-compat-util.h and do not include
>    system headers
> 
>  - the ewah code uses ewah_malloc, ewah_realloc, and so forth to let the
>    project use a particular allocator (and we want to use xmalloc and
>    friends). And we defined those in pack-bitmap.h, but of course that
>    had no effect on the ewah/*.c files that did not include
>    pack-bitmap.h.  Since we are hacking up and git-ifying libewok
>    anyway, we can just set the hardcoded fallback to xmalloc instead of
>    malloc.
> 
>   - the ewah code used gcc's __builtin_ctzll, but did not provide a
>     suitable fallback. We now provide a fallback in C.
... here.

I was messing around with several implementations (including the use of msvc compiler intrinsics) with the intention of doing some timing tests etc. [I suspected my C fallback function (a different implementation to yours) would be slightly faster.]

ATB, Ramsay Jones

Previous: Thomas RastNext: Jeff King
Message 51 of 55 in “pack bitmaps”
  1. 0/21 pack bitmapsJeff King, Nov 14, 2013
  2. 01/21 sha1write: make buffer const-correctJeff King, Nov 14, 2013
  3. 02/21 revindex: Export new APIsJeff King, Nov 14, 2013
  4. 03/21 pack-objects: Refactor the packing listJeff King, Nov 14, 2013
  5. 04/21 pack-objects: factor out name_hashJeff King, Nov 14, 2013
  6. 05/21 revision: allow setting custom limiter functionJeff King, Nov 14, 2013
  7. 06/21 sha1_file: export `git_open_noatime`Jeff King, Nov 14, 2013
  8. 07/21 compat: add endianness helpersJeff King, Nov 14, 2013
  9. 08/21 ewah: compressed bitmap implementationJeff King, Nov 14, 2013
  10. 09/21 documentation: add documentation for the bitmap formatJeff King, Nov 14, 2013
  11. 10/21 pack-bitmap: add support for bitmap indexesJeff King, Nov 14, 2013
  12. Thomas RastNov 24, 2013
  13. Document khashThomas Rast, Nov 25, 2013
  14. Jeff KingNov 28, 2013
  15. Karsten BleesNov 27, 2013
  16. Jeff KingNov 28, 2013
  17. Karsten BleesDec 3, 2013
  18. Jeff KingDec 3, 2013
  19. Karsten BleesDec 7, 2013
  20. Thomas RastNov 29, 2013
  21. Jeff KingDec 2, 2013
  22. Junio C HamanoDec 2, 2013
  23. Jeff KingDec 2, 2013
  24. Junio C HamanoDec 2, 2013
  25. 11/21 pack-objects: use bitmaps when packing objectsJeff King, Nov 14, 2013
  26. Thomas RastDec 7, 2013
  27. Jeff KingDec 21, 2013
  28. 12/21 rev-list: add bitmap mode to speed up object listsJeff King, Nov 14, 2013
  29. Thomas RastDec 7, 2013
  30. 13/21 pack-objects: implement bitmap writingJeff King, Nov 14, 2013
  31. Thomas RastDec 7, 2013
  32. Jeff KingDec 21, 2013
  33. 14/21 repack: stop using magic number for ARRAY_SIZE(exts)Jeff King, Nov 14, 2013
  34. Thomas RastDec 7, 2013
  35. 15/21 repack: turn exts array into array-of-structJeff King, Nov 14, 2013
  36. Thomas RastDec 7, 2013
  37. 16/21 repack: handle optional files created by pack-objectsJeff King, Nov 14, 2013
  38. Thomas RastDec 7, 2013
  39. 17/21 repack: consider bitmaps when performing repacksJeff King, Nov 14, 2013
  40. Thomas RastDec 7, 2013
  41. 18/21 count-objects: recognize .bitmap in garbage-checkingJeff King, Nov 14, 2013
  42. Thomas RastDec 7, 2013
  43. 19/21 t: add basic bitmap functionality testsJeff King, Nov 14, 2013
  44. Thomas RastDec 7, 2013
  45. Jeff KingDec 21, 2013
  46. 20/21 t/perf: add tests for pack bitmapsJeff King, Nov 14, 2013
  47. Thomas RastDec 7, 2013
  48. Jeff KingDec 21, 2013
  49. 21/21 pack-bitmap: implement optional name_hash cacheJeff King, Nov 14, 2013
  50. Thomas RastDec 7, 2013
  51. Ramsay JonesNov 14, 2013
  52. Jeff KingNov 14, 2013
  53. Ramsay JonesNov 14, 2013
  54. Ramsay JonesNov 18, 2013
  55. Thomas RastNov 16, 2013

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.