git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 08/12] add oidset API

From
Ramsay Jones <ramsay@ramsayjones.plus.com>
Date
Jan 24, 2017, 20:26 UTC
Message-ID
<944ea1f8-8f9c-cc17-02a5-a73cb6565b45@ramsayjones.plus.com>
In-Reply-To
<20170124004647.3o26ionfq3td2irf@sigill.intra.peff.net>
On 24/01/17 00:46, Jeff King wrote:
Show 48 quoted lines
> This is similar to many of our uses of sha1-array, but it
> overcomes one limitation of a sha1-array: when you are
> de-duplicating a large input with relatively few unique
> entries, sha1-array uses 20 bytes per non-unique entry.
> Whereas this set will use memory linear in the number of
> unique entries (albeit a few more than 20 bytes due to
> hashmap overhead).
> 
> Signed-off-by: Jeff King <peff@peff.net>
> ---
> This may be overkill. You can get roughly the same thing by making
> actual object structs via lookup_unknown_object(). But see the next
> patch for some comments on that.
> 
>  Makefile |  1 +
>  oidset.c | 49 +++++++++++++++++++++++++++++++++++++++++++++++++
>  oidset.h | 45 +++++++++++++++++++++++++++++++++++++++++++++
>  3 files changed, 95 insertions(+)
>  create mode 100644 oidset.c
>  create mode 100644 oidset.h
> 
> diff --git a/Makefile b/Makefile
> index 27afd0f37..e41efc2d8 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -774,6 +774,7 @@ LIB_OBJS += notes-cache.o
>  LIB_OBJS += notes-merge.o
>  LIB_OBJS += notes-utils.o
>  LIB_OBJS += object.o
> +LIB_OBJS += oidset.o
>  LIB_OBJS += pack-bitmap.o
>  LIB_OBJS += pack-bitmap-write.o
>  LIB_OBJS += pack-check.o
> diff --git a/oidset.c b/oidset.c
> new file mode 100644
> index 000000000..6094cff8c
> --- /dev/null
> +++ b/oidset.c
> @@ -0,0 +1,49 @@
> +#include "cache.h"
> +#include "oidset.h"
> +
> +struct oidset_entry {
> +	struct hashmap_entry hash;
> +	struct object_id oid;
> +};
> +
> +int oidset_hashcmp(const void *va, const void *vb,
static int oidset_hashcmp( ...

ATB, Ramsay Jones

Previous: Jeff KingNext: Jeff King
Message 20 of 48 in “reducing resource usage of for_each_alternate_ref”
  1. 0/12 reducing resource usage of for_each_alternate_refJeff King, Jan 24, 2017
  2. 01/12 for_each_alternate_ref: handle failure from real_pathdup()Jeff King, Jan 24, 2017
  3. Junio C HamanoJan 25, 2017
  4. 02/12 for_each_alternate_ref: stop trimming trailing slashesJeff King, Jan 24, 2017
  5. 03/12 for_each_alternate_ref: use strbuf for path allocationJeff King, Jan 24, 2017
  6. Junio C HamanoJan 25, 2017
  7. Jeff KingJan 25, 2017
  8. 04/12 for_each_alternate_ref: pass name/oid instead of ref structJeff King, Jan 24, 2017
  9. 05/12 for_each_alternate_ref: replace transport code with for-each-refJeff King, Jan 24, 2017
  10. Junio C HamanoJan 25, 2017
  11. 06/12 clone: disable save_commit_bufferJeff King, Jan 24, 2017
  12. Junio C HamanoJan 25, 2017
  13. Jeff KingJan 25, 2017
  14. Jeff KingJan 25, 2017
  15. Jeff KingJan 25, 2017
  16. 07/12 fetch-pack: cache results of for_each_alternate_refJeff King, Jan 24, 2017
  17. Junio C HamanoJan 25, 2017
  18. Jeff KingJan 25, 2017
  19. 08/12 add oidset APIJeff King, Jan 24, 2017
  20. Ramsay JonesJan 24, 2017
  21. Jeff KingJan 24, 2017
  22. 10/12 receive-pack: fix misleading namespace/.have commentJeff King, Jan 24, 2017
  23. 09/12 receive-pack: use oidset to de-duplicate .have linesJeff King, Jan 24, 2017
  24. Junio C HamanoJan 25, 2017
  25. Jeff KingJan 25, 2017
  26. 12/12 receive-pack: avoid duplicates between our refs and alternatesJeff King, Jan 24, 2017
  27. Junio C HamanoJan 25, 2017
  28. Jeff KingJan 25, 2017
  29. 11/12 receive-pack: treat namespace .have lines like alternatesJeff King, Jan 24, 2017
  30. Junio C HamanoJan 25, 2017
  31. Jeff KingJan 25, 2017
  32. Lukas FleischerJan 27, 2017
  33. Jeff KingJan 27, 2017
  34. Junio C HamanoJan 27, 2017
  35. Brandon WilliamsJan 24, 2017
  36. Jeff KingJan 24, 2017
  37. 0/11 reducing resource usage of for_each_alternate_refJeff King, Feb 8, 2017
  38. 01/11 for_each_alternate_ref: handle failure from real_pathdup()Jeff King, Feb 8, 2017
  39. 02/11 for_each_alternate_ref: stop trimming trailing slashesJeff King, Feb 8, 2017
  40. 04/11 for_each_alternate_ref: pass name/oid instead of ref structJeff King, Feb 8, 2017
  41. 03/11 for_each_alternate_ref: use strbuf for path allocationJeff King, Feb 8, 2017
  42. 05/11 for_each_alternate_ref: replace transport code with for-each-refJeff King, Feb 8, 2017
  43. 08/11 receive-pack: use oidset to de-duplicate .have linesJeff King, Feb 8, 2017
  44. 07/11 add oidset APIJeff King, Feb 8, 2017
  45. 06/11 fetch-pack: cache results of for_each_alternate_refJeff King, Feb 8, 2017
  46. 09/11 receive-pack: fix misleading namespace/.have commentJeff King, Feb 8, 2017
  47. 10/11 receive-pack: treat namespace .have lines like alternatesJeff King, Feb 8, 2017
  48. 11/11 receive-pack: avoid duplicates between our refs and alternatesJeff King, Feb 8, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.