git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 28/40] pack-objects: don't pack objects in external odbs

From
JHJeff Hostetler <git@jeffhostetler.com>
Date
Jan 4, 2018, 20:54 UTC
Message-ID
<8de3597a-01bd-a721-fffb-20769701d0af@jeffhostetler.com>
In-Reply-To
<20180103163403.11303-29-chriscool@tuxfamily.org>
On 1/3/2018 11:33 AM, Christian Couder wrote:
Show 28 quoted lines
> Objects managed by an external ODB should not be put into
> pack files. They should be transfered using other mechanism
> that can be specific to the external odb.
> 
> Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
> ---
>   builtin/pack-objects.c | 4 ++++
>   1 file changed, 4 insertions(+)
> 
> diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
> index 6c71552cdf..4ed66c7677 100644
> --- a/builtin/pack-objects.c
> +++ b/builtin/pack-objects.c
> @@ -28,6 +28,7 @@
>   #include "argv-array.h"
>   #include "mru.h"
>   #include "packfile.h"
> +#include "external-odb.h"
>   
>   static const char *pack_usage[] = {
>   	N_("git pack-objects --stdout [<options>...] [< <ref-list> | < <object-list>]"),
> @@ -1026,6 +1027,9 @@ static int want_object_in_pack(const struct object_id *oid,
>   			return want;
>   	}
>   
> +	if (external_odb_has_object(oid->hash))
> +		return 0;
> +

I worry about the performance of this in light of my comments earlier in the patch series about the expense of building the "have" sets.

Since we've already checked for a loose object and we are about to walk thru the local packfiles, so if we don't find it in any of them, then we don't have it locally. Only then do we need to worry about external odbs.

If we don't have it locally, does the caller of this function have sufficient "promisor" infomation infer that the object should exist on the promisor remote? Since you're going to omit it from the packfile anyway, you don't really need to know if the remote actually has it.

>   	for (entry = packed_git_mru.head; entry; entry = entry->next) {
>   		struct packed_git *p = entry->item;
>   		off_t offset;
> 
Jeff
Previous: Christian CouderNext: Christian Couder
Message 15 of 51 in “Promisor remotes and external ODB support”
  1. 00/40 Promisor remotes and external ODB supportChristian Couder, Jan 3, 2018
  2. 08/40 index-pack: refactor writing of .keep filesChristian Couder, Jan 3, 2018
  3. 12/40 rev-list: support termination at promisor objectsChristian Couder, Jan 3, 2018
  4. 11/40 sha1_file: support lazily fetching missing objectsChristian Couder, Jan 3, 2018
  5. 09/40 introduce fetch-object: fetch one promisor objectChristian Couder, Jan 3, 2018
  6. 18/40 t0400: add 'put_raw_obj' instruction to odb-helper scriptChristian Couder, Jan 3, 2018
  7. 21/40 t0400: add test for external odb write supportChristian Couder, Jan 3, 2018
  8. 24/40 lib-httpd: add upload.shChristian Couder, Jan 3, 2018
  9. 25/40 lib-httpd: add list.shChristian Couder, Jan 3, 2018
  10. 23/40 lib-httpd: pass config file to start_httpd()Christian Couder, Jan 3, 2018
  11. 27/40 odb-helper: add odb_helper_get_raw_object()Christian Couder, Jan 3, 2018
  12. 29/40 Add t0420 to test transfer to HTTP external odbChristian Couder, Jan 3, 2018
  13. 26/40 lib-httpd: add apache-e-odb.confChristian Couder, Jan 3, 2018
  14. 28/40 pack-objects: don't pack objects in external odbsChristian Couder, Jan 3, 2018
  15. Jeff HostetlerJan 4, 2018
  16. Christian CouderMar 19, 2018
  17. 30/40 external-odb: add 'get_direct' supportChristian Couder, Jan 3, 2018
  18. 31/40 odb-helper: add 'script_mode' to 'struct odb_helper'Christian Couder, Jan 3, 2018
  19. 32/40 odb-helper: add init_object_process()Christian Couder, Jan 3, 2018
  20. 34/40 Add t0460 to test passing git objectsChristian Couder, Jan 3, 2018
  21. 33/40 Add t0450 to test 'get_direct' mechanismChristian Couder, Jan 3, 2018
  22. 40/40 Add Documentation/technical/external-odb.txtChristian Couder, Jan 3, 2018
  23. 35/40 odb-helper: add put_object_process()Christian Couder, Jan 3, 2018
  24. 36/40 Add t0470 to test passing raw objectsChristian Couder, Jan 3, 2018
  25. 37/40 odb-helper: add have_object_process()Christian Couder, Jan 3, 2018
  26. 38/40 Add t0480 to test "have" capability and raw objectsChristian Couder, Jan 3, 2018
  27. 39/40 external-odb: use 'odb=magic' attribute to mark odb blobsChristian Couder, Jan 3, 2018
  28. 10/40 external-odb: implement external_odb_get_directChristian Couder, Jan 3, 2018
  29. Jeff HostetlerJan 4, 2018
  30. Christian CouderJan 15, 2018
  31. 22/40 Add t0410 to test external ODB transferChristian Couder, Jan 3, 2018
  32. 20/40 external-odb: accept only blobs for nowChristian Couder, Jan 3, 2018
  33. 19/40 external odb: add 'put_raw_obj' supportChristian Couder, Jan 3, 2018
  34. 13/40 gc: do not repack promisor packfilesChristian Couder, Jan 3, 2018
  35. 16/40 odb-helper: add 'enum odb_helper_type'Christian Couder, Jan 3, 2018
  36. 17/40 odb-helper: add odb_helper_init() to send 'init' instructionChristian Couder, Jan 3, 2018
  37. 14/40 sha1_file: prepare for external odbsChristian Couder, Jan 3, 2018
  38. Jeff HostetlerJan 4, 2018
  39. Christian CouderJan 16, 2018
  40. 07/40 fsck: support promisor objects as CLI argumentChristian Couder, Jan 3, 2018
  41. 15/40 external-odb: add script mode supportChristian Couder, Jan 3, 2018
  42. Jeff HostetlerJan 4, 2018
  43. Christian CouderMar 19, 2018
  44. 06/40 fsck: support referenced promisor objectsChristian Couder, Jan 3, 2018
  45. 05/40 fsck: support refs pointing to promisor objectsChristian Couder, Jan 3, 2018
  46. 02/40 Add GIT_NO_EXTERNAL_ODB env variableChristian Couder, Jan 3, 2018
  47. 01/40 Add initial external odb supportChristian Couder, Jan 3, 2018
  48. Jeff HostetlerJan 4, 2018
  49. Christian CouderJan 15, 2018
  50. 03/40 external-odb: add has_external_odb()Christian Couder, Jan 3, 2018
  51. 04/40 fsck: introduce promisor objectsChristian Couder, Jan 3, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.