git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 0/11] reducing resource usage of for_each_alternate_ref

From
Jeff King <peff@peff.net>
Date
Feb 8, 2017, 20:52 UTC
Message-ID
<20170208205219.twgm5rggovqbepte@sigill.intra.peff.net>
In-Reply-To
<20170124003729.j4ygjcgypdq7hceg@sigill.intra.peff.net>
This is a minor re-roll of the patches from:
  http://public-inbox.org/git/20170124003729.j4ygjcgypdq7hceg@sigill.intra.peff.net/
(which got some review, but I don't think was picked up for even 'pu').

I won't repeat the numbers and background from that message, but the gist of it is that this reduces memory usage significantly when your alternate has a lot of refs in it.

This version makes two minor changes:
  - it drops the save_commit_buffer patch to clone; it's redundant with
    what fetch_pack() is doing internally, and I wasn't able to measure
    any improvement
  - it adds a missing "static" to an internal function

The only other possible change from the review would be sorting the expected output in the test of the final script. I'm on the fence whether it is a feature that we expect a particular ordering. It's not set in stone ,but it _is_ deterministic, and if we change the order, it might be worth somebody actually noticing.

  [01/11]: for_each_alternate_ref: handle failure from real_pathdup()
  [02/11]: for_each_alternate_ref: stop trimming trailing slashes
  [03/11]: for_each_alternate_ref: use strbuf for path allocation
  [04/11]: for_each_alternate_ref: pass name/oid instead of ref struct
  [05/11]: for_each_alternate_ref: replace transport code with for-each-ref
  [06/11]: fetch-pack: cache results of for_each_alternate_ref
  [07/11]: add oidset API
  [08/11]: receive-pack: use oidset to de-duplicate .have lines
  [09/11]: receive-pack: fix misleading namespace/.have comment
  [10/11]: receive-pack: treat namespace .have lines like alternates
  [11/11]: receive-pack: avoid duplicates between our refs and alternates
 Makefile               |  1 +
 builtin/receive-pack.c | 41 +++++++++++++++-------------
 fetch-pack.c           | 48 ++++++++++++++++++++++++++++-----
 object.h               |  2 +-
 oidset.c               | 49 ++++++++++++++++++++++++++++++++++
 oidset.h               | 45 +++++++++++++++++++++++++++++++
 t/t5400-send-pack.sh   | 38 ++++++++++++++++++++++++++
 transport.c            | 72 +++++++++++++++++++++++++++++++++++---------------
 transport.h            |  2 +-
 9 files changed, 249 insertions(+), 49 deletions(-)
 create mode 100644 oidset.c
 create mode 100644 oidset.h
Previous: Jeff KingNext: Jeff King
Message 37 of 48 in “reducing resource usage of for_each_alternate_ref”
  1. 0/12 reducing resource usage of for_each_alternate_refJeff King, Jan 24, 2017
  2. 01/12 for_each_alternate_ref: handle failure from real_pathdup()Jeff King, Jan 24, 2017
  3. Junio C HamanoJan 25, 2017
  4. 02/12 for_each_alternate_ref: stop trimming trailing slashesJeff King, Jan 24, 2017
  5. 03/12 for_each_alternate_ref: use strbuf for path allocationJeff King, Jan 24, 2017
  6. Junio C HamanoJan 25, 2017
  7. Jeff KingJan 25, 2017
  8. 04/12 for_each_alternate_ref: pass name/oid instead of ref structJeff King, Jan 24, 2017
  9. 05/12 for_each_alternate_ref: replace transport code with for-each-refJeff King, Jan 24, 2017
  10. Junio C HamanoJan 25, 2017
  11. 06/12 clone: disable save_commit_bufferJeff King, Jan 24, 2017
  12. Junio C HamanoJan 25, 2017
  13. Jeff KingJan 25, 2017
  14. Jeff KingJan 25, 2017
  15. Jeff KingJan 25, 2017
  16. 07/12 fetch-pack: cache results of for_each_alternate_refJeff King, Jan 24, 2017
  17. Junio C HamanoJan 25, 2017
  18. Jeff KingJan 25, 2017
  19. 08/12 add oidset APIJeff King, Jan 24, 2017
  20. Ramsay JonesJan 24, 2017
  21. Jeff KingJan 24, 2017
  22. 10/12 receive-pack: fix misleading namespace/.have commentJeff King, Jan 24, 2017
  23. 09/12 receive-pack: use oidset to de-duplicate .have linesJeff King, Jan 24, 2017
  24. Junio C HamanoJan 25, 2017
  25. Jeff KingJan 25, 2017
  26. 12/12 receive-pack: avoid duplicates between our refs and alternatesJeff King, Jan 24, 2017
  27. Junio C HamanoJan 25, 2017
  28. Jeff KingJan 25, 2017
  29. 11/12 receive-pack: treat namespace .have lines like alternatesJeff King, Jan 24, 2017
  30. Junio C HamanoJan 25, 2017
  31. Jeff KingJan 25, 2017
  32. Lukas FleischerJan 27, 2017
  33. Jeff KingJan 27, 2017
  34. Junio C HamanoJan 27, 2017
  35. Brandon WilliamsJan 24, 2017
  36. Jeff KingJan 24, 2017
  37. 0/11 reducing resource usage of for_each_alternate_refJeff King, Feb 8, 2017
  38. 01/11 for_each_alternate_ref: handle failure from real_pathdup()Jeff King, Feb 8, 2017
  39. 02/11 for_each_alternate_ref: stop trimming trailing slashesJeff King, Feb 8, 2017
  40. 04/11 for_each_alternate_ref: pass name/oid instead of ref structJeff King, Feb 8, 2017
  41. 03/11 for_each_alternate_ref: use strbuf for path allocationJeff King, Feb 8, 2017
  42. 05/11 for_each_alternate_ref: replace transport code with for-each-refJeff King, Feb 8, 2017
  43. 08/11 receive-pack: use oidset to de-duplicate .have linesJeff King, Feb 8, 2017
  44. 07/11 add oidset APIJeff King, Feb 8, 2017
  45. 06/11 fetch-pack: cache results of for_each_alternate_refJeff King, Feb 8, 2017
  46. 09/11 receive-pack: fix misleading namespace/.have commentJeff King, Feb 8, 2017
  47. 10/11 receive-pack: treat namespace .have lines like alternatesJeff King, Feb 8, 2017
  48. 11/11 receive-pack: avoid duplicates between our refs and alternatesJeff King, Feb 8, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.