git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 0/3] upload-pack: support a missing-action

From
Christian Couder <christian.couder@gmail.com>
Date
May 15, 2024, 13:25 UTC
Message-ID
<20240515132543.851987-1-christian.couder@gmail.com>
In-Reply-To
<20221012135114.294680-1-christian.couder@gmail.com>

`git pack-objects` already supports a `--missing=<missing-action>` option, so that it can avoid erroring out if some objects aren't available.

It is interesting to have `git upload-pack` support a similar way to avoid sending some objects in case they aren't available on the server.

For example, in case both the server and the client are using a separate promisor remote that contain some objects, it can be better if the server doesn't try to send such objects back to the client, but instead let the client get those objects separately from the promisor remote. (The client needs to have the separate promisor remote configured, for that to work.)

This could work better if there was something, like perhaps a capability, for the client to tell the server something like:

  "I know how to fetch missing objects from this and that
  promisor remotes, so if you choose to, you may omit objects
  that you know are available from these promisor remotes when
  sending objects to me."

But that capability could be added later as other similar capabilities in this area could be very useful. For example in case of a client cloning, something like the following is currently needed:

  GIT_NO_LAZY_FETCH=0 git clone
      -c remote.my_promisor.promisor=true \
      -c remote.my_promisor.fetch="+refs/heads/*:refs/remotes/my_promisor/*" \
      -c remote.my_promisor.url=<MY_PROMISOR_URL> \
      --filter="blob:limit=5k" server

But it would be nice if there was a capability for the client to say that it would like the server to give it information about the promisor that it could use, so that the user doesn't have to pass all the "remote.my_promisor.XXX" config options on the command like. (It would then be a bit similar to the bundle-uri feature where all the bundle related information comes from the server.)

Another example use of this feature could be a server where some objects have been corrupted or deleted. It could still be useful for clients who could get those objects from another source, like perhaps a different client, to be able to fetch or clone from the server.

The fact that the new `uploadpack.missingAction` configuration variable has to be set to a non default value on the server means that regular client users cannot hurt themselves with this feature.

As `git rev-list` also supports a `--missing=<missing-action>` option, the first 2 patches in this series are about refactoring related code from both `git rev-list` and `git pack-objects` into new "missing.{c,h}" files. Patch 3/3 then adds a new `uploadpack.missingAction` configuration variable.

The changes since v1 are the following:
  - In patch 1/3 the refactored parse_missing_action_value() function
    doesn't change the fetch_if_missing global variable anymore. The
    clients of this function change it by themselves instead.
  - In patch 1/3 a minor typo was also fixed.
  - Old patch 2/4 that added a 'int print_ok' argument to
    parse_missing_action_value() was removed, and now callers deal
    with only the values they recognise.

Thanks to Junio for his review and for suggesting of the above changes.

Christian Couder (3):
  rev-list: refactor --missing=<missing-action>
  pack-objects: use the missing action API
  upload-pack: allow configuring a missing-action
 Documentation/config/uploadpack.txt |   9 ++
 Makefile                            |   1 +
 builtin/pack-objects.c              |  48 ++++++-----
 builtin/rev-list.c                  |  43 ++--------
 missing.c                           |  36 ++++++++
 missing.h                           |  19 +++++
 t/t5706-upload-pack-missing.sh      | 125 ++++++++++++++++++++++++++++
 upload-pack.c                       |  19 +++++
 8 files changed, 242 insertions(+), 58 deletions(-)
 create mode 100644 missing.c
 create mode 100644 missing.h
 create mode 100755 t/t5706-upload-pack-missing.sh
-- 
2.45.1.148.g0f5efb064b
Previous: Patrick SteinhardtNext: Christian Couder
Message 30 of 46 in “Implement filtering repacks”
  1. 0/3 Implement filtering repacksChristian Couder, Oct 12, 2022
  2. 1/3 pack-objects: allow --filter without --stdoutChristian Couder, Oct 12, 2022
  3. 2/3 repack: add --filter=<filter-spec> optionChristian Couder, Oct 12, 2022
  4. 3/3 repack: introduce --force to force filteringChristian Couder, Oct 12, 2022
  5. Junio C HamanoOct 14, 2022
  6. Christian CouderOct 20, 2022
  7. Taylor BlauOct 28, 2022
  8. Junio C HamanoOct 28, 2022
  9. Christian CouderNov 7, 2022
  10. Christian CouderNov 7, 2022
  11. 0/2 Implement filtering repacksChristian Couder, Oct 25, 2022
  12. 1/2 pack-objects: allow --filter without --stdoutChristian Couder, Oct 25, 2022
  13. 2/2 repack: add --filter=<filter-spec> optionChristian Couder, Oct 25, 2022
  14. Taylor BlauOct 28, 2022
  15. Christian CouderNov 7, 2022
  16. 0/2 Implement filtering repacksChristian Couder, Nov 22, 2022
  17. 1/2 pack-objects: allow --filter without --stdoutChristian Couder, Nov 22, 2022
  18. 2/2 repack: add --filter=<filter-spec> optionChristian Couder, Nov 22, 2022
  19. Junio C HamanoNov 23, 2022
  20. Christian CouderDec 21, 2022
  21. Junio C HamanoNov 23, 2022
  22. 0/3 Implement filtering repacksChristian Couder, Dec 21, 2022
  23. 1/3 pack-objects: allow --filter without --stdoutChristian Couder, Dec 21, 2022
  24. Patrick SteinhardtJan 4, 2023
  25. 2/3 repack: add --filter=<filter-spec> optionChristian Couder, Dec 21, 2022
  26. Patrick SteinhardtJan 4, 2023
  27. Junio C HamanoJan 5, 2023
  28. 3/3 gc: add gc.repackFilter config optionChristian Couder, Dec 21, 2022
  29. Patrick SteinhardtJan 4, 2023
  30. 0/3 upload-pack: support a missing-actionChristian Couder, May 15, 2024
  31. 1/3 rev-list: refactor --missing=<missing-action>Christian Couder, May 15, 2024
  32. Junio C HamanoMay 15, 2024
  33. 2/3 pack-objects: use the missing action APIChristian Couder, May 15, 2024
  34. Junio C HamanoMay 15, 2024
  35. Christian CouderMay 24, 2024
  36. 3/3 upload-pack: allow configuring a missing-actionChristian Couder, May 15, 2024
  37. Junio C HamanoMay 15, 2024
  38. Christian CouderMay 24, 2024
  39. Junio C HamanoMay 24, 2024
  40. Christian CouderMay 28, 2024
  41. Junio C HamanoMay 28, 2024
  42. Christian CouderMay 31, 2024
  43. Junio C HamanoJun 1, 2024
  44. Christian CouderJun 3, 2024
  45. Junio C HamanoJun 3, 2024
  46. Christian CouderMay 15, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.