git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 29/36] Add t0550 to test 'get_direct' mechanism

From
Christian Couder <christian.couder@gmail.com>
Date
Mar 19, 2018, 13:31 UTC
Message-ID
<20180319133147.15413-30-chriscool@tuxfamily.org>
In-Reply-To
<20180319133147.15413-1-chriscool@tuxfamily.org>
From: Ben Peart <benpeart@microsoft.com>
Signed-off-by: Ben Peart <benpeart@microsoft.com>
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
 t/t0550-read-object.sh | 28 +++++++++++++++++
 t/t0550/read-object    | 68 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 96 insertions(+)
 create mode 100755 t/t0550-read-object.sh
 create mode 100755 t/t0550/read-object
diff --git a/t/t0550-read-object.sh b/t/t0550-read-object.sh
new file mode 100755
index 0000000000..93c8eea3f9
--- /dev/null
+++ b/t/t0550-read-object.sh
@@ -0,0 +1,28 @@
+#!/bin/sh
+
+test_description='tests for long running read-object process'
+
+. ./test-lib.sh
+
+PATH="$PATH:$TEST_DIRECTORY/t0550"
+
+test_expect_success 'setup host repo with a root commit' '
+	test_commit zero &&
+	hash1=$(git ls-tree HEAD | grep zero.t | cut -f1 | cut -d\  -f3)
+'
+
+HELPER="read-object"
+
+test_expect_success 'blobs can be retrieved from the host repo' '
+	git init guest-repo &&
+	(cd guest-repo &&
+	 git config odb.magic.subprocessCommand "$HELPER" &&
+	 git cat-file blob "$hash1" >/dev/null)
+'
+
+test_expect_success 'invalid blobs generate errors' '
+	cd guest-repo &&
+	test_must_fail git cat-file blob "invalid"
+'
+
+test_done
diff --git a/t/t0550/read-object b/t/t0550/read-object
new file mode 100755
index 0000000000..004e9368c9
--- /dev/null
+++ b/t/t0550/read-object
@@ -0,0 +1,68 @@
+#!/usr/bin/perl
+#
+# Example implementation for the Git read-object protocol version 1
+# See Documentation/technical/read-object-protocol.txt
+#
+# Allows you to test the ability for blobs to be pulled from a host git repo
+# "on demand."  Called when git needs a blob it couldn't find locally due to
+# a lazy clone that only cloned the commits and trees.
+#
+# A lazy clone can be simulated via the following commands from the host repo
+# you wish to create a lazy clone of:
+#
+# cd /host_repo
+# git rev-parse HEAD
+# git init /guest_repo
+# git cat-file --batch-check --batch-all-objects | grep -v 'blob' |
+#	cut -d' ' -f1 | git pack-objects /e/guest_repo/.git/objects/pack/noblobs
+# cd /guest_repo
+# git config core.virtualizeobjects true
+# git reset --hard <sha from rev-parse call above>
+#
+# Please note, this sample is a minimal skeleton. No proper error handling 
+# was implemented.
+#
+
+use 5.008;
+use lib (split(/:/, $ENV{GITPERLLIB}));
+use strict;
+use warnings;
+use Git::Packet;
+
+#
+# Point $DIR to the folder where your host git repo is located so we can pull
+# missing objects from it
+#
+my $DIR = "../.git/";
+
+packet_initialize("git-read-object", 1);
+
+my %remote_caps = packet_read_and_check_capabilities("get_direct");
+packet_check_and_write_capabilities(\%remote_caps, "get_direct");
+
+while (1) {
+	my ($res, $command) = packet_txt_read();
+
+	if ( $res == -1 ) {
+		exit 0;
+	}
+
+	$command =~ s/^command=//;
+
+	if ( $command eq "init" ) {
+		packet_bin_read();
+
+		packet_txt_write("status=success");
+		packet_flush();
+	} elsif ( $command eq "get_direct" ) {
+		my ($sha1) = packet_txt_read() =~ /^sha1=([0-9a-f]{40})$/;
+		packet_bin_read();
+
+		system ('git --git-dir="' . $DIR . '" cat-file blob ' . $sha1 . ' | GIT_NO_EXTERNAL_ODB=1 git hash-object -w --stdin >/dev/null 2>&1');
+
+		packet_txt_write(($?) ? "status=error" : "status=success");
+		packet_flush();
+	} else {
+		die "bad command '$command'";
+	}
+}
-- 
2.17.0.rc0.37.g8f476fabe9
Previous: Christian CouderNext: Christian Couder
Message 28 of 38 in “Promisor remotes and external ODB support”
  1. 00/36 Promisor remotes and external ODB supportChristian Couder, Mar 19, 2018
  2. 01/36 Add initial external odb supportChristian Couder, Mar 19, 2018
  3. 02/36 sha1_file: add prepare_external_alt_odb()Christian Couder, Mar 19, 2018
  4. 06/36 sha1_file: prepare for external odbsChristian Couder, Mar 19, 2018
  5. 05/36 external-odb: implement external_odb_get_directChristian Couder, Mar 19, 2018
  6. 07/36 odb-helper: add 'enum odb_helper_type'Christian Couder, Mar 19, 2018
  7. 04/36 external-odb: add has_external_odb()Christian Couder, Mar 19, 2018
  8. 03/36 Add GIT_NO_EXTERNAL_ODB env variableChristian Couder, Mar 19, 2018
  9. 12/36 external-odb: add external_odb_get_many_direct()Christian Couder, Mar 19, 2018
  10. 16/36 t0500: add 'put_raw_obj' instruction to odb-helper scriptChristian Couder, Mar 19, 2018
  11. 18/36 external-odb: accept only blobs for nowChristian Couder, Mar 19, 2018
  12. 17/36 external odb: add 'put_raw_obj' supportChristian Couder, Mar 19, 2018
  13. 14/36 Use odb.origin.partialclonefilter instead of core.partialclonefilterChristian Couder, Mar 19, 2018
  14. 13/36 Use external_odb_get_direct() and has_external_odb()Christian Couder, Mar 19, 2018
  15. 22/36 lib-httpd: add upload.shChristian Couder, Mar 19, 2018
  16. 21/36 lib-httpd: pass config file to start_httpd()Christian Couder, Mar 19, 2018
  17. 26/36 pack-objects: don't pack objects in external odbsChristian Couder, Mar 19, 2018
  18. 27/36 Add t0520 to test transfer to HTTP external odbChristian Couder, Mar 19, 2018
  19. 30/36 Add t0560 to test passing git objectsChristian Couder, Mar 19, 2018
  20. 36/36 Add Documentation/technical/external-odb.txtChristian Couder, Mar 19, 2018
  21. 11/36 odb-helper: add odb_helper_init() to send 'init' instructionChristian Couder, Mar 19, 2018
  22. 34/36 Add t0580 to test "have" capability and raw objectsChristian Couder, Mar 19, 2018
  23. 33/36 odb-helper: add have_object_process()Christian Couder, Mar 19, 2018
  24. 35/36 external-odb: use 'odb=magic' attribute to mark odb blobsChristian Couder, Mar 19, 2018
  25. 15/36 external-odb: add 'get_direct' supportChristian Couder, Mar 19, 2018
  26. 32/36 Add t0570 to test passing raw objectsChristian Couder, Mar 19, 2018
  27. 31/36 odb-helper: add put_object_process()Christian Couder, Mar 19, 2018
  28. 29/36 Add t0550 to test 'get_direct' mechanismChristian Couder, Mar 19, 2018
  29. 28/36 odb-helper: add init_object_process()Christian Couder, Mar 19, 2018
  30. 25/36 odb-helper: add odb_helper_get_raw_object()Christian Couder, Mar 19, 2018
  31. 20/36 Add t0510 to test external ODB transferChristian Couder, Mar 19, 2018
  32. 19/36 t0500: add test for external odb write supportChristian Couder, Mar 19, 2018
  33. 24/36 lib-httpd: add apache-e-odb.confChristian Couder, Mar 19, 2018
  34. 23/36 lib-httpd: add list.shChristian Couder, Mar 19, 2018
  35. 10/36 odb-helper: add 'script_mode' to 'struct odb_helper'Christian Couder, Mar 19, 2018
  36. 09/36 external-odb: add script mode supportChristian Couder, Mar 19, 2018
  37. 08/36 external-odb: add external_odb_reinit()Christian Couder, Mar 19, 2018
  38. Junio C HamanoMar 19, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.