git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 15/36] external-odb: add 'get_direct' support

From
Christian Couder <christian.couder@gmail.com>
Date
Mar 19, 2018, 13:31 UTC
Message-ID
<20180319133147.15413-16-chriscool@tuxfamily.org>
In-Reply-To
<20180319133147.15413-1-chriscool@tuxfamily.org>

This implements the 'get_direct' capability/instruction that makes it possible for external odb helper scripts to pass blobs to Git by directly writing them as loose objects files.

It is better to call this a "direct" mode rather than a "fault-in" mode as we could have the same kind of mechanism to "put" objects into an external odb, where the odb helper would access blobs it wants to send to an external odb directly from files, but it would be strange to call that a fault-in mode too.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
 external-odb.c |  3 ++-
 odb-helper.c   | 20 +++++++++++++++++---
 2 files changed, 19 insertions(+), 4 deletions(-)
diff --git a/external-odb.c b/external-odb.c
index c3d90f39fb..192b09e843 100644
--- a/external-odb.c
+++ b/external-odb.c
@@ -138,7 +138,8 @@ int external_odb_get_object(const unsigned char *sha1)
 		int ret;
 		int fd;
 
-		if (!odb_helper_has_object(o, sha1))
+		if (!(o->supported_capabilities & ODB_HELPER_CAP_GET_RAW_OBJ) &&
+		    !(o->supported_capabilities & ODB_HELPER_CAP_GET_GIT_OBJ))
 			continue;
 
 		fd = create_object_tmpfile(&tmpfile, pathbuf.buf);
diff --git a/odb-helper.c b/odb-helper.c
index 7e5eab94cf..30cfe689de 100644
--- a/odb-helper.c
+++ b/odb-helper.c
@@ -336,14 +336,28 @@ int odb_helper_get_object(struct odb_helper *o, const unsigned char *sha1,
 int odb_helper_get_direct(struct odb_helper *o,
 			  const unsigned char *sha1)
 {
-	int res = 0;
 	uint64_t start = getnanotime();
 
-	fetch_object(o->dealer, sha1);
+	if (o->type == ODB_HELPER_GIT_REMOTE) {
+		fetch_object(o->dealer, sha1);
+	} else {
+		struct odb_helper_object *obj;
+		struct odb_helper_cmd cmd;
+
+		obj = odb_helper_lookup(o, sha1);
+		if (!obj)
+			return -1;
+
+		if (odb_helper_start(o, &cmd, "get_direct %s", sha1_to_hex(sha1)) < 0)
+			return -1;
+
+		if (odb_helper_finish(o, &cmd))
+			return -1;
+	}
 
 	trace_performance_since(start, "odb_helper_get_direct");
 
-	return res;
+	return 0;
 }
 
 int odb_helper_get_many_direct(struct odb_helper *o,
-- 
2.17.0.rc0.37.g8f476fabe9
Previous: Christian CouderNext: Christian Couder
Message 25 of 38 in “Promisor remotes and external ODB support”
  1. 00/36 Promisor remotes and external ODB supportChristian Couder, Mar 19, 2018
  2. 01/36 Add initial external odb supportChristian Couder, Mar 19, 2018
  3. 02/36 sha1_file: add prepare_external_alt_odb()Christian Couder, Mar 19, 2018
  4. 06/36 sha1_file: prepare for external odbsChristian Couder, Mar 19, 2018
  5. 05/36 external-odb: implement external_odb_get_directChristian Couder, Mar 19, 2018
  6. 07/36 odb-helper: add 'enum odb_helper_type'Christian Couder, Mar 19, 2018
  7. 04/36 external-odb: add has_external_odb()Christian Couder, Mar 19, 2018
  8. 03/36 Add GIT_NO_EXTERNAL_ODB env variableChristian Couder, Mar 19, 2018
  9. 12/36 external-odb: add external_odb_get_many_direct()Christian Couder, Mar 19, 2018
  10. 16/36 t0500: add 'put_raw_obj' instruction to odb-helper scriptChristian Couder, Mar 19, 2018
  11. 18/36 external-odb: accept only blobs for nowChristian Couder, Mar 19, 2018
  12. 17/36 external odb: add 'put_raw_obj' supportChristian Couder, Mar 19, 2018
  13. 14/36 Use odb.origin.partialclonefilter instead of core.partialclonefilterChristian Couder, Mar 19, 2018
  14. 13/36 Use external_odb_get_direct() and has_external_odb()Christian Couder, Mar 19, 2018
  15. 22/36 lib-httpd: add upload.shChristian Couder, Mar 19, 2018
  16. 21/36 lib-httpd: pass config file to start_httpd()Christian Couder, Mar 19, 2018
  17. 26/36 pack-objects: don't pack objects in external odbsChristian Couder, Mar 19, 2018
  18. 27/36 Add t0520 to test transfer to HTTP external odbChristian Couder, Mar 19, 2018
  19. 30/36 Add t0560 to test passing git objectsChristian Couder, Mar 19, 2018
  20. 36/36 Add Documentation/technical/external-odb.txtChristian Couder, Mar 19, 2018
  21. 11/36 odb-helper: add odb_helper_init() to send 'init' instructionChristian Couder, Mar 19, 2018
  22. 34/36 Add t0580 to test "have" capability and raw objectsChristian Couder, Mar 19, 2018
  23. 33/36 odb-helper: add have_object_process()Christian Couder, Mar 19, 2018
  24. 35/36 external-odb: use 'odb=magic' attribute to mark odb blobsChristian Couder, Mar 19, 2018
  25. 15/36 external-odb: add 'get_direct' supportChristian Couder, Mar 19, 2018
  26. 32/36 Add t0570 to test passing raw objectsChristian Couder, Mar 19, 2018
  27. 31/36 odb-helper: add put_object_process()Christian Couder, Mar 19, 2018
  28. 29/36 Add t0550 to test 'get_direct' mechanismChristian Couder, Mar 19, 2018
  29. 28/36 odb-helper: add init_object_process()Christian Couder, Mar 19, 2018
  30. 25/36 odb-helper: add odb_helper_get_raw_object()Christian Couder, Mar 19, 2018
  31. 20/36 Add t0510 to test external ODB transferChristian Couder, Mar 19, 2018
  32. 19/36 t0500: add test for external odb write supportChristian Couder, Mar 19, 2018
  33. 24/36 lib-httpd: add apache-e-odb.confChristian Couder, Mar 19, 2018
  34. 23/36 lib-httpd: add list.shChristian Couder, Mar 19, 2018
  35. 10/36 odb-helper: add 'script_mode' to 'struct odb_helper'Christian Couder, Mar 19, 2018
  36. 09/36 external-odb: add script mode supportChristian Couder, Mar 19, 2018
  37. 08/36 external-odb: add external_odb_reinit()Christian Couder, Mar 19, 2018
  38. Junio C HamanoMar 19, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.