threads / patch / 53944

v5, 39 partst1001: use $ZERO_OID

Subject: [PATCH v5 02/39] t1001: use $ZERO_OID

## tl;dr

116 messages between Jul 28, 2020 and Mar 11, 2021. Diffs are folded; open one to read it.

replies: 115people: 6as markdown or json

brian m. carlson· Jul 28, 2020, 23:34 UTC · lore

[PATCH v5 00/39] SHA-256, part 3/3

This is the final part required for the stage 4 implementation of SHA-256.

Changes from v4:
* Fix several commit message to fix typos and inaccuracies.
* Remove useless touch statement in tests.
* Adjust several tests to introduce less dependence on previous test
  assertions other than the setup ones.
* Loosen the regexp for object IDs in a Perl script in a test.
* Ensure help functionality works for git http-fetch.
* Switch http-fetch.c to use the already existing struct argv_array
  instance.
* Avoid a memory leak, albeit a minor one.
* Improve error message for unknown bundle hash algorithm.
* Hoist a strbuf_rtrim in the bundle parsing code to simplify things.
* Specify a minimum version instead of a default version in the bundle
  code.
* Remove a useless line in tests.
* Avoid invoking git hash-object needlessly for the SHA1 prerequisite.
* Be explicit about which algorithms we're testing in CI.
* Properly capitalize "core.repositoryFormatVersion" in documentation.
Changes from v2:
* Move bundle signature parsing into a function.
* Improve parsing of bundle capabilities.
* Improve t5510 to select only the interesting lines with sed.
* Use cat and a heredoc instead of multiple echos.
* Use test_i18ngrep where appropriate.
* Reject extensions.objectFormat for repo v0.
* Update documentation accordingly.
Changes from v1:
* Add a patch to add a bundle v3 format with capabilities that include
  the hash algorithm.
* Remove bundle detection of hash algorithm using hash algorithm length.
* Fix typo in the extensions.objectFormat documentation.
* Document that extensions.objectFormat is ignored for repo v0.
* Use test instead of brackets.
* Use a command group instead of a subshell for efficiency.
* Drop the bizarre test_might_fail, since it isn't needed (and is
  bizarre and wrong).
* Write "SHA-256" instead of "NewHash" in commit messages.
* Split SHA-256 CI tests to an independent run and avoid running a third
  test run to a single CI job.
Range-diff below.
Johannes Schindelin (1):
  t3404: prepare 'short SHA-1 collision' tests for SHA-256
brian m. carlson (38):
  t: make test-bloom initialize repository
  t1001: use $ZERO_OID
  t3305: make hash agnostic
  t6100: make hash size independent
  t6101: make hash size independent
  t6301: make hash size independent
  t6500: specify test values for SHA-256
  t6501: avoid hard-coded objects
  t7003: compute appropriate length constant
  t7063: make hash size independent
  t7201: abstract away SHA-1-specific constants
  t7102: abstract away SHA-1-specific constants
  t7400: make hash size independent
  t7405: make hash size independent
  t7506: avoid checking for SHA-1-specific constants
  t7508: use $ZERO_OID instead of hard-coded constant
  t8002: make hash size independent
  t8003: make hash size independent
  t8011: make hash size independent
  t9300: abstract away SHA-1-specific constants
  t9300: use $ZERO_OID instead of hard-coded object ID
  t9301: make hash size independent
  t9350: make hash size independent
  t9500: ensure that algorithm info is preserved in config
  t9700: make hash size independent
  t5308: make test work with SHA-256
  t0410: mark test with SHA1 prerequisite
  http-fetch: set up git directory before parsing pack hashes
  builtin/verify-pack: implement an --object-format option
  bundle: add new version for use with SHA-256
  setup: add support for reading extensions.objectformat
  Enable SHA-256 support by default
  t: add test_oid option to select hash algorithm
  t: allow testing different hash algorithms via environment
  t: make SHA1 prerequisite depend on default hash
  ci: run tests with SHA-256
  docs: add documentation for extensions.objectFormat
  t: remove test_oid_init in tests
 Documentation/config.txt                  |   2 +
 Documentation/config/extensions.txt       |   8 ++
 Documentation/git-bundle.txt              |   9 +-
 Documentation/technical/bundle-format.txt |  30 ++++-
 builtin/bundle.c                          |   5 +-
 builtin/init-db.c                         |   5 -
 builtin/verify-pack.c                     |  23 ++--
 bundle.c                                  |  83 +++++++++---
 bundle.h                                  |   4 +-
 ci/run-build-and-tests.sh                 |   6 +
 config.mak.dev                            |   2 -
 http-fetch.c                              |   4 +-
 repository.c                              |   4 -
 setup.c                                   |  16 ++-
 t/helper/test-bloom.c                     |   2 +
 t/lib-pack.sh                             |  11 +-
 t/lib-submodule-update.sh                 |   1 -
 t/t0000-basic.sh                          |  15 ++-
 t/t0001-init.sh                           |  33 +++++
 t/t0410-partial-clone.sh                  |   4 +-
 t/t1006-cat-file.sh                       |   2 -
 t/t1050-large.sh                          |   1 -
 t/t1091-sparse-checkout-builtin.sh        |   4 +-
 t/t1410-reflog.sh                         |   1 -
 t/t1450-fsck.sh                           |   1 -
 t/t1500-rev-parse.sh                      |   1 -
 t/t3305-notes-fanout.sh                   |   2 +-
 t/t3308-notes-merge.sh                    |   1 -
 t/t3404-rebase-interactive.sh             |  49 +++++--
 t/t3600-rm.sh                             |   1 -
 t/t3800-mktag.sh                          |   1 -
 t/t4002-diff-basic.sh                     |   2 -
 t/t4027-diff-submodule.sh                 |   1 -
 t/t4134-apply-submodule.sh                |   1 -
 t/t4200-rerere.sh                         |   1 -
 t/t4211-line-log.sh                       |   1 -
 t/t5300-pack-object.sh                    |   3 +-
 t/t5302-pack-index.sh                     |   1 -
 t/t5308-pack-detect-duplicates.sh         |  20 +--
 t/t5313-pack-bounds-checks.sh             |   1 -
 t/t5318-commit-graph.sh                   |   3 +-
 t/t5319-multi-pack-index.sh               |   1 -
 t/t5324-split-commit-graph.sh             |   1 -
 t/t5504-fetch-receive-strict.sh           |   1 -
 t/t5510-fetch.sh                          |  16 ++-
 t/t5530-upload-pack-error.sh              |   1 -
 t/t5562-http-backend-content-length.sh    |   1 -
 t/t5607-clone-bundle.sh                   |  31 +++++
 t/t5702-protocol-v2.sh                    |   3 +-
 t/t5703-upload-pack-ref-in-want.sh        |   1 -
 t/t6006-rev-list-format.sh                |   1 -
 t/t6100-rev-list-in-order.sh              |   4 +-
 t/t6101-rev-parse-parents.sh              |   2 +-
 t/t6301-for-each-ref-errors.sh            |   2 +-
 t/t6500-gc.sh                             |  27 +++-
 t/t6501-freshen-objects.sh                |  14 +-
 t/t7003-filter-branch.sh                  |   3 +-
 t/t7063-status-untracked-cache.sh         | 150 ++++++++++++----------
 t/t7102-reset.sh                          |  93 ++++++++------
 t/t7201-co.sh                             |   6 +-
 t/t7400-submodule-basic.sh                |  26 ++--
 t/t7405-submodule-merge.sh                |   4 +-
 t/t7506-status-submodule.sh               |  12 +-
 t/t7508-status.sh                         |   2 +-
 t/t8002-blame.sh                          |  18 ++-
 t/t8003-blame-corner-cases.sh             |   3 +-
 t/t8011-blame-split-file.sh               |   2 +-
 t/t9300-fast-import.sh                    | 118 +++++++++--------
 t/t9301-fast-import-notes.sh              |  13 +-
 t/t9350-fast-export.sh                    |  15 +--
 t/t9500-gitweb-standalone-no-errors.sh    |  22 +++-
 t/t9700/test.pl                           |   6 +-
 t/test-lib-functions.sh                   |  16 ++-
 t/test-lib.sh                             |   9 +-
 74 files changed, 635 insertions(+), 354 deletions(-)
 create mode 100644 Documentation/config/extensions.txt
Diff-intervalle contre v4 :
 1:  2d720a6beb =  1:  fb53834bd3 t: make test-bloom initialize repository
 2:  530704736a =  2:  8170f55d65 t1001: use $ZERO_OID
 3:  33e7822d0e =  3:  4b14099f1a t3305: make hash agnostic
 4:  82a6827f45 !  4:  8d3ce7cccd t3404: prepare 'short SHA-1 collision' tests for SHA-256
    @@ Commit message
         commit message "collide2" to find such a collision. To make it easier to
         find such a collision, we reduce the number of digits to 4.
     
    -    The `t/oid-info/oid` file now lists the values found in that way, and
    -    the test cases are modified to use those values.
    -
         As the tests are no longer dependent on SHA-1, we also rename their
         titles to talk about "commit IDs" instead of "SHA-1s".
     
 5:  23cbd25060 =  5:  8329368075 t6100: make hash size independent
 6:  9e2553b1f2 =  6:  63595c30d4 t6101: make hash size independent
 7:  c2c5dd407e =  7:  88c7a501db t6301: make hash size independent
 8:  1ab4ed032a =  8:  4546001417 t6500: specify test values for SHA-256
 9:  54fcc025bb =  9:  4a9304a648 t6501: avoid hard-coded objects
10:  a2f420fcd4 = 10:  7785b7cc1f t7003: compute appropriate length constant
11:  766bba42f6 ! 11:  71db1716f5 t7063: make hash size independent
    @@ Metadata
      ## Commit message ##
         t7063: make hash size independent
     
    -    Use test_oid instead of hard-coding a fixed size all-zeros object ID.
    +    Use test_oid instead of hard-coding algorithm-specific constants and
    +    all-zero values.
     
         Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
     
    @@ t/t7063-status-untracked-cache.sh: EOF
      two
      EOF
      	test_cmp ../expect-from-test-dump ../actual
    -@@ t/t7063-status-untracked-cache.sh: test_expect_success 'using --untracked-cache does not fail when core.untrackedCa
    - '
    - 
    - test_expect_success 'setting core.untrackedCache to keep' '
    -+	touch ../expect-from-test-dump &&
    - 	git config core.untrackedCache keep &&
    - 	git update-index --untracked-cache &&
    - 	test-tool dump-untracked-cache >../actual &&
     @@ t/t7063-status-untracked-cache.sh: test_expect_success '"status" after file replacement should be clean with UC=tru
      	test-tool dump-untracked-cache >../actual &&
      	grep -F "recurse valid" ../actual >../actual.grep &&
12:  66d242ccd9 ! 12:  79dcdc3496 t7201: abstract away SHA-1-specific constants
    @@ t/t7201-co.sh: test_expect_success 'switch to another branch while carrying a de
      	test_line_count = 1 messages &&
      	H=$(git rev-parse --verify HEAD) &&
      	M=$(git show-ref -s --verify refs/heads/master) &&
    -@@ t/t7201-co.sh: test_expect_success 'checkout to detach HEAD' '
    +@@ t/t7201-co.sh: test_expect_success 'checkout to detach HEAD (with advice declined)' '
    + 
    + test_expect_success 'checkout to detach HEAD' '
      	git config advice.detachedHead true &&
    ++	rev=$(git rev-parse --short renamer^) &&
      	git checkout -f renamer && git clean -f &&
      	GIT_TEST_GETTEXT_POISON=false git checkout renamer^ 2>messages &&
     -	grep "HEAD is now at 7329388" messages &&
13:  090041ee54 = 13:  f89560df08 t7102: abstract away SHA-1-specific constants
14:  4a6991d6a1 = 14:  79800ba8a3 t7400: make hash size independent
15:  d39bbeb5a2 = 15:  07d287fe0f t7405: make hash size independent
16:  5b0d52ae0e = 16:  1ee37967f2 t7506: avoid checking for SHA-1-specific constants
17:  54d1706107 = 17:  ba3c997fd7 t7508: use $ZERO_OID instead of hard-coded constant
18:  f3f4239ce9 ! 18:  362e0a9440 t8002: make hash size independent
    @@ Metadata
      ## Commit message ##
         t8002: make hash size independent
     
    -    Compute the length of an object ID instead of of hard-coding 40-based
    +    Compute the length of an object ID instead of hard-coding 40-based
         values.
     
         Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
    @@ t/t8002-blame.sh: test_description='git blame'
      . "$TEST_DIRECTORY"/annotate-tests.sh
      
     +test_expect_success 'setup' '
    -+	test_oid_init
    ++	hexsz=$(test_oid hexsz)
     +'
     +
      test_expect_success 'blame untracked file in empty repo' '
    @@ t/t8002-blame.sh: test_expect_success 'blame --abbrev=<n> works' '
      test_expect_success 'blame -l aligns regular and boundary commits' '
     -	check_abbrev 40 -l HEAD &&
     -	check_abbrev 39 -l ^HEAD
    -+	hexsz=$(test_oid hexsz) &&
     +	check_abbrev $hexsz         -l HEAD &&
     +	check_abbrev $((hexsz - 1)) -l ^HEAD
      '
19:  719db3822e = 19:  eeeba30e75 t8003: make hash size independent
20:  803e5665e9 ! 20:  a71b7fb7f1 t8011: make hash size independent
    @@ t/t8011-blame-split-file.sh: test_expect_success 'setup simulated porcelain' '
      	my $field = shift;
      	while (<>) {
     -		if (/^[0-9a-f]{40} /) {
    -+		if (/^[0-9a-f]{40,64} /) {
    ++		if (/^[0-9a-f]{40,} /) {
      			flush();
      			$hash = $&;
      		} elsif (/^$field (.*)/) {
21:  d1c4351bc9 ! 21:  b9a4ac6547 t9300: abstract away SHA-1-specific constants
    @@ Commit message
         t9300: abstract away SHA-1-specific constants
     
         Adjust the test so that it computes variables for object IDs instead of
    -    using hard-coded hashes.
    +    using hard-coded hashes.  In addition, use cut to filter out the object
    +    IDs and verify only the information that we're really interested in.
     
         Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
     
      ## t/t9300-fast-import.sh ##
     @@ t/t9300-fast-import.sh: test_expect_success 'A: verify pack' '
    + '
      
      test_expect_success 'A: verify diff' '
    ++	copy=$(git rev-parse --verify master:file2) &&
      	cat >expect <<-EOF &&
     -	:000000 100755 0000000000000000000000000000000000000000 7123f7f44e39be127c5eb701e5968176ee9d78b1 A	copy-of-file2
    -+	:000000 100755 0000000000000000000000000000000000000000 $(git rev-parse --verify master:file2) A	copy-of-file2
    ++	:000000 100755 0000000000000000000000000000000000000000 $copy A	copy-of-file2
      	EOF
      	git diff-tree -M -r master verify--import-marks >actual &&
      	compare_diff_raw expect actual &&
22:  db74bf3c8c ! 22:  8d617381c0 t9300: use $ZERO_OID instead of hard-coded object ID
    @@ t/t9300-fast-import.sh: test_expect_success 'A: create pack from stdin' '
      	tag nested
      	mark :7
     @@ t/t9300-fast-import.sh: test_expect_success 'A: verify pack' '
    - 
      test_expect_success 'A: verify diff' '
    + 	copy=$(git rev-parse --verify master:file2) &&
      	cat >expect <<-EOF &&
    --	:000000 100755 0000000000000000000000000000000000000000 $(git rev-parse --verify master:file2) A	copy-of-file2
    -+	:000000 100755 $ZERO_OID $(git rev-parse --verify master:file2) A	copy-of-file2
    +-	:000000 100755 0000000000000000000000000000000000000000 $copy A	copy-of-file2
    ++	:000000 100755 $ZERO_OID $copy A	copy-of-file2
      	EOF
      	git diff-tree -M -r master verify--import-marks >actual &&
      	compare_diff_raw expect actual &&
23:  fa7daed2bf = 23:  d3ddbe3bcf t9301: make hash size independent
24:  84aef86ac0 = 24:  70bfe6c076 t9350: make hash size independent
25:  541108fc64 = 25:  c784e76075 t9500: ensure that algorithm info is preserved in config
26:  2de5d2da58 = 26:  00e82dc23e t9700: make hash size independent
27:  c916058c38 = 27:  2d03d136df t5308: make test work with SHA-256
28:  b0781430ef = 28:  d8941dd81d t0410: mark test with SHA1 prerequisite
29:  230a933e87 ! 29:  39eb2e82b5 http-fetch: set up git directory before parsing pack hashes
    @@ Commit message
         To ensure that we can parse packfile hashes of the right length, let's
         set up the git directory before we start parsing arguments.
     
    +    Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
    +
      ## http-fetch.c ##
     @@ http-fetch.c: int cmd_main(int argc, const char **argv)
      	int packfile = 0;
      	struct object_id packfile_hash;
      
    -+	setup_git_directory();
    ++	setup_git_directory_gently(NULL);
     +
      	while (arg < argc && argv[arg][0] == '-') {
      		const char *p;
30:  d3dbe9e760 ! 30:  7c71a12f67 builtin/verify-pack: implement an --object-format option
    @@ builtin/verify-pack.c
      {
      	struct child_process index_pack = CHILD_PROCESS_INIT;
     -	const char *argv[] = {"index-pack", NULL, NULL, NULL };
    --	struct strbuf arg = STRBUF_INIT;
    -+	struct argv_array argv = ARGV_ARRAY_INIT;
    -+	struct strbuf arg = STRBUF_INIT, hash_arg = STRBUF_INIT;
    ++	struct argv_array *argv = &index_pack.args;
    + 	struct strbuf arg = STRBUF_INIT;
      	int verbose = flags & VERIFY_PACK_VERBOSE;
      	int stat_only = flags & VERIFY_PACK_STAT_ONLY;
      	int err;
      
    -+	argv_array_push(&argv, "index-pack");
    ++	argv_array_push(argv, "index-pack");
     +
      	if (stat_only)
     -		argv[1] = "--verify-stat-only";
    -+		argv_array_push(&argv, "--verify-stat-only");
    ++		argv_array_push(argv, "--verify-stat-only");
      	else if (verbose)
     -		argv[1] = "--verify-stat";
    -+		argv_array_push(&argv, "--verify-stat");
    ++		argv_array_push(argv, "--verify-stat");
      	else
     -		argv[1] = "--verify";
    -+		argv_array_push(&argv, "--verify");
    ++		argv_array_push(argv, "--verify");
     +
    -+	if (hash_algo) {
    -+		strbuf_addf(&hash_arg, "--object-format=%s", hash_algo);
    -+		argv_array_push(&argv, hash_arg.buf);
    -+	}
    ++	if (hash_algo)
    ++		argv_array_pushf(argv, "--object-format=%s", hash_algo);
      
      	/*
      	 * In addition to "foo.pack" we accept "foo.idx" and "foo";
    @@ builtin/verify-pack.c: static int verify_one_pack(const char *path, unsigned int
      	    !ends_with(arg.buf, ".pack"))
      		strbuf_addstr(&arg, ".pack");
     -	argv[2] = arg.buf;
    -+	argv_array_push(&argv, arg.buf);
    ++	argv_array_push(argv, arg.buf);
      
     -	index_pack.argv = argv;
    -+	index_pack.argv = argv.argv;
      	index_pack.git_cmd = 1;
      
      	err = run_command(&index_pack);
    -@@ builtin/verify-pack.c: static int verify_one_pack(const char *path, unsigned int flags)
    - 		}
    - 	}
    - 	strbuf_release(&arg);
    -+	argv_array_clear(&argv);
    - 
    - 	return err;
    - }
     @@ builtin/verify-pack.c: int cmd_verify_pack(int argc, const char **argv, const char *prefix)
      {
      	int err = 0;
31:  e3805101ce ! 31:  9a0345a1f6 bundle: add new version for use with SHA-256
    @@ bundle.c: static void add_to_ref_list(const struct object_id *oid, const char *n
     +	if (skip_prefix(capability, "object-format=", &arg)) {
     +		int algo = hash_algo_by_name(arg);
     +		if (algo == GIT_HASH_UNKNOWN)
    -+			return error(_("unable to detect hash algorithm"));
    ++			return error(_("unrecognized bundle hash algorithm: %s"), arg);
     +		header->hash_algo = &hash_algos[algo];
     +		return 0;
     +	}
    @@ bundle.c: static int parse_bundle_header(int fd, struct bundle_header *header,
      		int is_prereq = 0;
      		const char *p;
      
    -+		if (header->version == 3 && *buf.buf == '@') {
    -+			buf.buf[--buf.len] = '\0';
    -+			if (parse_capability(header, buf.buf + 1)) {
    -+				status = -1;
    -+				break;
    -+			}
    -+			continue;
    -+		}
    -+
    - 		if (*buf.buf == '-') {
    - 			is_prereq = 1;
    - 			strbuf_remove(&buf, 0, 1);
    - 		}
    +-		if (*buf.buf == '-') {
    +-			is_prereq = 1;
    +-			strbuf_remove(&buf, 0, 1);
    +-		}
      		strbuf_rtrim(&buf);
      
     -		if (!header->hash_algo) {
     -			header->hash_algo = detect_hash_algo(&buf);
     -			if (!header->hash_algo) {
     -				error(_("unknown hash algorithm length"));
    --				status = -1;
    --				break;
    --			}
    --		}
    --
    ++		if (header->version == 3 && *buf.buf == '@') {
    ++			if (parse_capability(header, buf.buf + 1)) {
    + 				status = -1;
    + 				break;
    + 			}
    ++			continue;
    ++		}
    ++
    ++		if (*buf.buf == '-') {
    ++			is_prereq = 1;
    ++			strbuf_remove(&buf, 0, 1);
    + 		}
    + 
      		/*
    - 		 * Tip lines have object name, SP, and refname.
    - 		 * Prerequisites have object name that is optionally
     @@ bundle.c: static int write_bundle_refs(int bundle_fd, struct rev_info *revs)
      }
      
    @@ bundle.c: static int write_bundle_refs(int bundle_fd, struct rev_info *revs)
      	int bundle_to_stdout;
      	int ref_count = 0;
      	struct rev_info revs;
    -+	int default_version = the_hash_algo == &hash_algos[GIT_HASH_SHA1] ? 2 : 3;
    ++	int min_version = the_hash_algo == &hash_algos[GIT_HASH_SHA1] ? 2 : 3;
      
      	bundle_to_stdout = !strcmp(path, "-");
      	if (bundle_to_stdout)
    @@ bundle.c: int create_bundle(struct repository *r, const char *path,
     -	/* write signature */
     -	write_or_die(bundle_fd, bundle_signature, strlen(bundle_signature));
     +	if (version == -1)
    -+		version = default_version;
    ++		version = min_version;
     +
     +	if (version < 2 || version > 3) {
     +		die(_("unsupported bundle version %d"), version);
    -+	} else if (version < default_version) {
    ++	} else if (version < min_version) {
     +		die(_("cannot write bundle version %d with algorithm %s"), version, the_hash_algo->name);
     +	} else if (version == 2) {
     +		write_or_die(bundle_fd, v2_bundle_signature, strlen(v2_bundle_signature));
    @@ t/t5510-fetch.sh: test_expect_success 'create bundle 1' '
      '
      
      test_expect_success 'header of bundle looks right' '
    - 	head -n 4 "$D"/bundle1 &&
    +-	head -n 4 "$D"/bundle1 &&
     -	head -n 1 "$D"/bundle1 | grep "^#" &&
     -	head -n 2 "$D"/bundle1 | grep "^-$OID_REGEX " &&
     -	head -n 3 "$D"/bundle1 | grep "^$OID_REGEX " &&
    @@ t/t5510-fetch.sh: test_expect_success 'create bundle 1' '
     +	cat >expect <<-EOF &&
     +	# v3 git bundle
     +	@object-format=$(test_oid algo)
    -+	-OID message
    -+	OID message
    ++	-OID updated by origin
    ++	OID refs/heads/master
     +
     +	EOF
    -+	sed -e "s/$OID_REGEX .*/OID message/g" -e "5q" "$D"/bundle1 >actual &&
    ++	sed -e "s/$OID_REGEX/OID/g" -e "5q" "$D"/bundle1 >actual &&
     +	test_cmp expect actual
      '
      
32:  c941fcf63c = 32:  4b72b98ce7 setup: add support for reading extensions.objectformat
33:  7c6264ac79 ! 33:  73ef3ef66a Enable SHA-256 support by default
    @@ t/t0001-init.sh: test_expect_success 're-init from a linked worktree' '
     +'
     +
     +test_expect_success 'init honors --object-format' '
    -+	git init --object-format=sha256 explicit &&
    -+	git -C explicit rev-parse --show-object-format >actual &&
    ++	git init --object-format=sha1 explicit-sha1 &&
    ++	git -C explicit-sha1 rev-parse --show-object-format >actual &&
    ++	echo sha1 >expected &&
    ++	test_cmp expected actual &&
    ++	git init --object-format=sha256 explicit-sha256 &&
    ++	git -C explicit-sha256 rev-parse --show-object-format >actual &&
     +	echo sha256 >expected &&
     +	test_cmp expected actual
     +'
34:  2719ab94e6 = 34:  2512a1669c t: add test_oid option to select hash algorithm
35:  1c8e5041d1 = 35:  015636559b t: allow testing different hash algorithms via environment
36:  7213d59222 <  -:  ---------- t: make SHA1 prerequisite depend on default hash
 -:  ---------- > 36:  6f2dec51cd t: make SHA1 prerequisite depend on default hash
37:  caf6c026ba ! 37:  32536503f3 ci: run tests with SHA-256
    @@ Commit message
         run as fast as possible, so we wouldn't want to add a third run to the
         linux-gcc job.  To make sure we properly exercise the code, let's run
         the tests in the default mode (SHA-1) first and then run a second time
    -    with SHA-256.
    +    with SHA-256.  We explicitly specify SHA-1 for the first run so that if
    +    we change the default in the future, we make sure to test both cases.
     
         Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
     
    @@ ci/run-build-and-tests.sh: linux-gcc)
      	make test
      	;;
     +linux-clang)
    ++	export GIT_TEST_DEFAULT_HASH=sha1
     +	make test
     +	export GIT_TEST_DEFAULT_HASH=sha256
     +	make test
38:  ec4e6ae166 ! 38:  cc5679d353 docs: add documentation for extensions.objectFormat
    @@ Documentation/config/extensions.txt (new)
     +extensions.objectFormat::
     +	Specify the hash algorithm to use.  The acceptable values are `sha1` and
     +	`sha256`.  If not specified, `sha1` is assumed.  It is an error to specify
    -+	this key unless `core.repositoryformatversion` is 1.
    ++	this key unless `core.repositoryFormatVersion` is 1.
     ++
     +Note that this setting should only be set by linkgit:git-init[1] or
     +linkgit:git-clone[1].  Trying to change it after initialization will not
39:  3185c62cfc ! 39:  307b75c88e t: remove test_oid_init in tests
    @@ t/t7003-filter-branch.sh: test_description='git filter-branch'
      	GIT_COMMITTER_DATE="@0 +0000" GIT_AUTHOR_DATE="@0 +0000" &&
      	test_commit --notick B &&
     
    - ## t/t8002-blame.sh ##
    -@@ t/t8002-blame.sh: test_description='git blame'
    - PROG='git blame -c'
    - . "$TEST_DIRECTORY"/annotate-tests.sh
    - 
    --test_expect_success 'setup' '
    --	test_oid_init
    --'
    --
    - test_expect_success 'blame untracked file in empty repo' '
    - 	>untracked &&
    - 	test_must_fail git blame untracked
    -
      ## t/t8003-blame-corner-cases.sh ##
     @@ t/t8003-blame-corner-cases.sh: test_description='git blame corner cases'
      pick_fc='s/^[0-9a-f^]* *\([^ ]*\) *(\([^ ]*\) .*/\1-\2/'
brian m. carlson· Jul 28, 2020, 23:34 UTC · re: brian m. carlson · lore
Use $ZERO_OID to make the test hash independent.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t1091-sparse-checkout-builtin.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
Show changes to t/t1091-sparse-checkout-builtin.sh +2 −2
diff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh
index 7cd45fc139..84acfc48b6 100755
--- a/t/t1091-sparse-checkout-builtin.sh
+++ b/t/t1091-sparse-checkout-builtin.sh
@@ -369,7 +369,7 @@ test_expect_success 'sparse-checkout (init|set|disable) warns with unmerged stat
 	git clone repo unmerged &&
 
 	cat >input <<-EOF &&
-	0 0000000000000000000000000000000000000000	folder1/a
+	0 $ZERO_OID	folder1/a
 	100644 $(git -C unmerged rev-parse HEAD:folder1/a) 1	folder1/a
 	EOF
 	git -C unmerged update-index --index-info <input &&
@@ -396,7 +396,7 @@ test_expect_success 'sparse-checkout reapply' '
 	echo dirty >tweak/deep/deeper2/a &&
 
 	cat >input <<-EOF &&
-	0 0000000000000000000000000000000000000000	folder1/a
+	0 $ZERO_OID	folder1/a
 	100644 $(git -C tweak rev-parse HEAD:folder1/a) 1	folder1/a
 	EOF
 	git -C tweak update-index --index-info <input &&
brian m. carlson· Jul 28, 2020, 23:34 UTC · re: brian m. carlson · lore

[PATCH v5 03/39] t3305: make hash agnostic

When computing the fanout length, let's use test_oid to look up the hexadecimal size of the hash in question instead of hard-coding a value.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t3305-notes-fanout.sh | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
Show changes to t/t3305-notes-fanout.sh +2 −1
diff --git a/t/t3305-notes-fanout.sh b/t/t3305-notes-fanout.sh
index 3b4753e1b4..d5a94b7174 100755
--- a/t/t3305-notes-fanout.sh
+++ b/t/t3305-notes-fanout.sh
@@ -7,7 +7,7 @@ test_description='Test that adding/removing many notes triggers automatic fanout
 path_has_fanout() {
 	path=$1 &&
 	fanout=$2 &&
-	after_last_slash=$((40 - $fanout * 2)) &&
+	after_last_slash=$(($(test_oid hexsz) - $fanout * 2)) &&
 	echo $path | grep -q "^\([0-9a-f]\{2\}/\)\{$fanout\}[0-9a-f]\{$after_last_slash\}$"
 }
 
@@ -31,6 +31,7 @@ all_notes_have_fanout() {
 }
 
 test_expect_success 'tweak test environment' '
+	test_oid_init &&
 	git checkout -b nondeterminism &&
 	test_commit A &&
 	git checkout --orphan with_notes;
brian m. carlson· Jul 28, 2020, 23:34 UTC · re: brian m. carlson · lore

[PATCH v5 04/39] t3404: prepare 'short SHA-1 collision' tests for SHA-256

From: Johannes Schindelin <johannes.schindelin@gmx.de>

The idea of the magic value "ac4f2ee" in this test is to make the reworded commit `collide2` have the same shortened ID as the commit `collide3`.

To port the same idea to the SHA-256 version of Git, we therefore need another magic value that causes the same collision, but this time with the SHA-256 version of the commit IDs.

In this patch, we add code guarded by `GIT_TEST_FIND_COLLIDER` to do exactly that. Essentially, a large number of integers is appended to the commit message "collide2" to find such a collision. To make it easier to find such a collision, we reduce the number of digits to 4.

As the tests are no longer dependent on SHA-1, we also rename their titles to talk about "commit IDs" instead of "SHA-1s".

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t3404-rebase-interactive.sh | 50 ++++++++++++++++++++++++++++-------
 1 file changed, 41 insertions(+), 9 deletions(-)
Show changes to t/t3404-rebase-interactive.sh +41 −9
diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh
index 4a7d21f898..2fb2c581cb 100755
--- a/t/t3404-rebase-interactive.sh
+++ b/t/t3404-rebase-interactive.sh
@@ -30,6 +30,7 @@ Initial setup:
 . "$TEST_DIRECTORY"/lib-rebase.sh
 
 test_expect_success 'setup' '
+	test_oid_init &&
 	test_commit A file1 &&
 	test_commit B file1 &&
 	test_commit C file2 &&
@@ -1250,7 +1251,7 @@ test_expect_success 'rebase -i error on commits with \ in message' '
 	test_expect_code 1 grep  "	emp" error
 '
 
-test_expect_success SHA1 'short SHA-1 setup' '
+test_expect_success 'short commit ID setup' '
 	test_when_finished "git checkout master" &&
 	git checkout --orphan collide &&
 	git rm -rf . &&
@@ -1262,23 +1263,54 @@ test_expect_success SHA1 'short SHA-1 setup' '
 	)
 '
 
-test_expect_success SHA1 'short SHA-1 collide' '
+if test -n "$GIT_TEST_FIND_COLLIDER"
+then
+	author="$(unset test_tick; test_tick; git var GIT_AUTHOR_IDENT)"
+	committer="$(unset test_tick; test_tick; git var GIT_COMMITTER_IDENT)"
+	blob="$(git rev-parse collide2:collide)"
+	from="$(git rev-parse collide1^0)"
+	repl="commit refs/heads/collider-&\\n"
+	repl="${repl}author $author\\ncommitter $committer\\n"
+	repl="${repl}data <<EOF\\ncollide2 &\\nEOF\\n"
+	repl="${repl}from $from\\nM 100644 $blob collide\\n"
+	test_seq 1 32768 | sed "s|.*|$repl|" >script &&
+	git fast-import <script &&
+	git pack-refs &&
+	git for-each-ref >refs &&
+	grep "^$(test_oid t3404_collision)" <refs >matches &&
+	cat matches &&
+	test_line_count -gt 2 matches || {
+		echo "Could not find a collider" >&2
+		exit 1
+	}
+fi
+
+test_expect_success 'short commit ID collide' '
+	test_oid_cache <<-EOF &&
+	# collision-related constants
+	t3404_collision	sha1:6bcd
+	t3404_collision	sha256:0161
+	t3404_collider	sha1:ac4f2ee
+	t3404_collider	sha256:16697
+	EOF
 	test_when_finished "reset_rebase && git checkout master" &&
 	git checkout collide &&
-	colliding_sha1=6bcda37 &&
-	test $colliding_sha1 = "$(git rev-parse HEAD | cut -c 1-7)" &&
+	colliding_id=$(test_oid t3404_collision) &&
+	hexsz=$(test_oid hexsz) &&
+	test $colliding_id = "$(git rev-parse HEAD | cut -c 1-4)" &&
+	test_config core.abbrev 4 &&
 	(
 		unset test_tick &&
 		test_tick &&
 		set_fake_editor &&
-		FAKE_COMMIT_MESSAGE="collide2 ac4f2ee" \
+		FAKE_COMMIT_MESSAGE="collide2 $(test_oid t3404_collider)" \
 		FAKE_LINES="reword 1 break 2" git rebase -i HEAD~2 &&
-		test $colliding_sha1 = "$(git rev-parse HEAD | cut -c 1-7)" &&
-		grep "^pick $colliding_sha1 " \
+		test $colliding_id = "$(git rev-parse HEAD | cut -c 1-4)" &&
+		grep "^pick $colliding_id " \
 			.git/rebase-merge/git-rebase-todo.tmp &&
-		grep "^pick [0-9a-f]\{40\}" \
+		grep "^pick [0-9a-f]\{$hexsz\}" \
 			.git/rebase-merge/git-rebase-todo &&
-		grep "^pick [0-9a-f]\{40\}" \
+		grep "^pick [0-9a-f]\{$hexsz\}" \
 			.git/rebase-merge/git-rebase-todo.backup &&
 		git rebase --continue
 	) &&
brian m. carlson· Jul 28, 2020, 23:34 UTC · re: brian m. carlson · lore

[PATCH v5 06/39] t6101: make hash size independent

Use $OID_REGEX instead of a hard-coded regular expression.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t6101-rev-parse-parents.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Show changes to t/t6101-rev-parse-parents.sh +1 −1
diff --git a/t/t6101-rev-parse-parents.sh b/t/t6101-rev-parse-parents.sh
index 7683e4a114..7531262a5e 100755
--- a/t/t6101-rev-parse-parents.sh
+++ b/t/t6101-rev-parse-parents.sh
@@ -34,7 +34,7 @@ test_expect_success 'setup' '
 '
 
 test_expect_success 'start is valid' '
-	git rev-parse start | grep "^[0-9a-f]\{40\}$"
+	git rev-parse start | grep "^$OID_REGEX$"
 '
 
 test_expect_success 'start^0' '
brian m. carlson· Jul 28, 2020, 23:34 UTC · re: brian m. carlson · lore

[PATCH v5 10/39] t7003: compute appropriate length constant

Instead of using a specific invalid hard-coded object ID, look one up from the translation table.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t7003-filter-branch.sh | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
Show changes to t/t7003-filter-branch.sh +3 −1
diff --git a/t/t7003-filter-branch.sh b/t/t7003-filter-branch.sh
index e23de7d0b5..beeb2b855a 100755
--- a/t/t7003-filter-branch.sh
+++ b/t/t7003-filter-branch.sh
@@ -5,6 +5,7 @@ test_description='git filter-branch'
 . "$TEST_DIRECTORY/lib-gpg.sh"
 
 test_expect_success 'setup' '
+	test_oid_init &&
 	test_commit A &&
 	GIT_COMMITTER_DATE="@0 +0000" GIT_AUTHOR_DATE="@0 +0000" &&
 	test_commit --notick B &&
@@ -463,10 +464,11 @@ test_expect_success 'rewrite submodule with another content' '
 '
 
 test_expect_success 'replace submodule revision' '
+	invalid=$(test_oid numeric) &&
 	git reset --hard original &&
 	git filter-branch -f --tree-filter \
 	    "if git ls-files --error-unmatch -- submod > /dev/null 2>&1
-	     then git update-index --cacheinfo 160000 0123456789012345678901234567890123456789 submod
+	     then git update-index --cacheinfo 160000 $invalid submod
 	     fi" HEAD &&
 	test $orig_head != $(git show-ref --hash --head HEAD)
 '
brian m. carlson· Jul 28, 2020, 23:34 UTC · re: brian m. carlson · lore

[PATCH v5 12/39] t7201: abstract away SHA-1-specific constants

Adjust the test so that it computes variables for object IDs instead of using hard-coded hashes.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t7201-co.sh | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
Show changes to t/t7201-co.sh +4 −2
diff --git a/t/t7201-co.sh b/t/t7201-co.sh
index b696bae5f5..4d62b9b00f 100755
--- a/t/t7201-co.sh
+++ b/t/t7201-co.sh
@@ -230,9 +230,10 @@ test_expect_success 'switch to another branch while carrying a deletion' '
 test_expect_success 'checkout to detach HEAD (with advice declined)' '
 
 	git config advice.detachedHead false &&
+	rev=$(git rev-parse --short renamer^) &&
 	git checkout -f renamer && git clean -f &&
 	git checkout renamer^ 2>messages &&
-	test_i18ngrep "HEAD is now at 7329388" messages &&
+	test_i18ngrep "HEAD is now at $rev" messages &&
 	test_line_count = 1 messages &&
 	H=$(git rev-parse --verify HEAD) &&
 	M=$(git show-ref -s --verify refs/heads/master) &&
@@ -248,9 +249,10 @@ test_expect_success 'checkout to detach HEAD (with advice declined)' '
 
 test_expect_success 'checkout to detach HEAD' '
 	git config advice.detachedHead true &&
+	rev=$(git rev-parse --short renamer^) &&
 	git checkout -f renamer && git clean -f &&
 	GIT_TEST_GETTEXT_POISON=false git checkout renamer^ 2>messages &&
-	grep "HEAD is now at 7329388" messages &&
+	grep "HEAD is now at $rev" messages &&
 	test_line_count -gt 1 messages &&
 	H=$(git rev-parse --verify HEAD) &&
 	M=$(git show-ref -s --verify refs/heads/master) &&
brian m. carlson· Jul 28, 2020, 23:34 UTC · re: brian m. carlson · lore

[PATCH v5 15/39] t7405: make hash size independent

Use $ZERO_OID instead of hard-coding a fixed size all-zeros object ID.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t7405-submodule-merge.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
Show changes to t/t7405-submodule-merge.sh +2 −2
diff --git a/t/t7405-submodule-merge.sh b/t/t7405-submodule-merge.sh
index aa33978ed2..6a1e5f8232 100755
--- a/t/t7405-submodule-merge.sh
+++ b/t/t7405-submodule-merge.sh
@@ -195,7 +195,7 @@ test_expect_success 'git submodule status should display the merge conflict prop
        url = $TRASH_DIRECTORY/sub
 EOF
        cat >expect <<EOF &&
-U0000000000000000000000000000000000000000 sub
+U$ZERO_OID sub
 EOF
        git submodule status > actual &&
        test_cmp expect actual &&
@@ -214,7 +214,7 @@ test_expect_success 'git submodule status should display the merge conflict prop
        url = $TRASH_DIRECTORY/sub
 EOF
        cat >expect <<EOF &&
-U0000000000000000000000000000000000000000 sub
+U$ZERO_OID sub
 EOF
        git submodule status > actual &&
        test_cmp expect actual &&
brian m. carlson· Jul 28, 2020, 23:34 UTC · re: brian m. carlson · lore

[PATCH v5 13/39] t7102: abstract away SHA-1-specific constants

Adjust the test so that it computes variables for object IDs instead of using hard-coded hashes.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t7102-reset.sh | 93 +++++++++++++++++++++++++++---------------------
 1 file changed, 53 insertions(+), 40 deletions(-)
Show changes to t/t7102-reset.sh +53 −37
diff --git a/t/t7102-reset.sh b/t/t7102-reset.sh
index 97be0d968d..22161b3b2d 100755
--- a/t/t7102-reset.sh
+++ b/t/t7102-reset.sh
@@ -37,17 +37,23 @@ test_expect_success 'creating initial files and commits' '
 
 	echo "2nd line 1st file" >>first &&
 	git commit -a -m "modify 1st file" &&
+	head5p2=$(git rev-parse --verify HEAD) &&
+	head5p2f=$(git rev-parse --short HEAD:first) &&
 
 	git rm first &&
 	git mv second secondfile &&
 	git commit -a -m "remove 1st and rename 2nd" &&
+	head5p1=$(git rev-parse --verify HEAD) &&
+	head5p1s=$(git rev-parse --short HEAD:secondfile) &&
 
 	echo "1st line 2nd file" >secondfile &&
 	echo "2nd line 2nd file" >>secondfile &&
 	# "git commit -m" would break MinGW, as Windows refuse to pass
 	# $test_encoding encoded parameter to git.
 	commit_msg $test_encoding | git -c "i18n.commitEncoding=$test_encoding" commit -a -F - &&
-	head5=$(git rev-parse --verify HEAD)
+	head5=$(git rev-parse --verify HEAD) &&
+	head5s=$(git rev-parse --short HEAD:secondfile) &&
+	head5sl=$(git rev-parse HEAD:secondfile)
 '
 # git log --pretty=oneline # to see those SHA1 involved
 
@@ -94,7 +100,7 @@ test_expect_success 'giving a non existing revision should fail' '
 
 test_expect_success 'reset --soft with unmerged index should fail' '
 	touch .git/MERGE_HEAD &&
-	echo "100644 44c5b5884550c17758737edcced463447b91d42b 1	un" |
+	echo "100644 $head5sl 1	un" |
 		git update-index --index-info &&
 	test_must_fail git reset --soft HEAD &&
 	rm .git/MERGE_HEAD &&
@@ -192,7 +198,7 @@ test_expect_success \
 >.diff_expect
 cat >.cached_expect <<EOF
 diff --git a/secondfile b/secondfile
-index 1bbba79..44c5b58 100644
+index $head5p1s..$head5s 100644
 --- a/secondfile
 +++ b/secondfile
 @@ -1 +1,2 @@
@@ -207,7 +213,7 @@ secondfile:
 EOF
 test_expect_success '--soft reset only should show changes in diff --cached' '
 	git reset --soft HEAD^ &&
-	check_changes d1a4bc3abce4829628ae2dcb0d60ef3d1a78b1c4 &&
+	check_changes $head5p1 &&
 	test "$(git rev-parse ORIG_HEAD)" = \
 			$head5
 '
@@ -242,7 +248,7 @@ EOF
 test_expect_success \
 	'--hard reset should change the files and undo commits permanently' '
 	git reset --hard HEAD~2 &&
-	check_changes ddaefe00f1da16864591c61fdc7adb5d7cd6b74e &&
+	check_changes $head5p2 &&
 	test "$(git rev-parse ORIG_HEAD)" = \
 			$head4
 '
@@ -251,7 +257,7 @@ test_expect_success \
 cat >.cached_expect <<EOF
 diff --git a/first b/first
 deleted file mode 100644
-index 8206c22..0000000
+index $head5p2f..0000000
 --- a/first
 +++ /dev/null
 @@ -1,2 +0,0 @@
@@ -259,14 +265,14 @@ index 8206c22..0000000
 -2nd line 1st file
 diff --git a/second b/second
 deleted file mode 100644
-index 1bbba79..0000000
+index $head5p1s..0000000
 --- a/second
 +++ /dev/null
 @@ -1 +0,0 @@
 -2nd file
 diff --git a/secondfile b/secondfile
 new file mode 100644
-index 0000000..44c5b58
+index 0000000..$head5s
 --- /dev/null
 +++ b/secondfile
 @@ -0,0 +1,2 @@
@@ -286,13 +292,13 @@ test_expect_success \
 	echo "1st line 2nd file" >secondfile &&
 	echo "2nd line 2nd file" >>secondfile &&
 	git add secondfile &&
-	check_changes ddaefe00f1da16864591c61fdc7adb5d7cd6b74e
+	check_changes $head5p2
 '
 
 cat >.diff_expect <<EOF
 diff --git a/first b/first
 deleted file mode 100644
-index 8206c22..0000000
+index $head5p2f..0000000
 --- a/first
 +++ /dev/null
 @@ -1,2 +0,0 @@
@@ -300,7 +306,7 @@ index 8206c22..0000000
 -2nd line 1st file
 diff --git a/second b/second
 deleted file mode 100644
-index 1bbba79..0000000
+index $head5p1s..0000000
 --- a/second
 +++ /dev/null
 @@ -1 +0,0 @@
@@ -314,9 +320,8 @@ secondfile:
 EOF
 test_expect_success '--mixed reset to HEAD should unadd the files' '
 	git reset &&
-	check_changes ddaefe00f1da16864591c61fdc7adb5d7cd6b74e &&
-	test "$(git rev-parse ORIG_HEAD)" = \
-			ddaefe00f1da16864591c61fdc7adb5d7cd6b74e
+	check_changes $head5p2 &&
+	test "$(git rev-parse ORIG_HEAD)" = $head5p2
 '
 
 >.diff_expect
@@ -328,7 +333,7 @@ secondfile:
 EOF
 test_expect_success 'redoing the last two commits should succeed' '
 	git add secondfile &&
-	git reset --hard ddaefe00f1da16864591c61fdc7adb5d7cd6b74e &&
+	git reset --hard $head5p2 &&
 
 	git rm first &&
 	git mv second secondfile &&
@@ -389,47 +394,55 @@ test_expect_success \
 	check_changes $head5
 '
 
-cat > expect << EOF
-diff --git a/file1 b/file1
-index d00491f..7ed6ff8 100644
---- a/file1
-+++ b/file1
-@@ -1 +1 @@
--1
-+5
-diff --git a/file2 b/file2
-deleted file mode 100644
-index 0cfbf08..0000000
---- a/file2
-+++ /dev/null
-@@ -1 +0,0 @@
--2
-EOF
-cat > cached_expect << EOF
-diff --git a/file4 b/file4
-new file mode 100644
-index 0000000..b8626c4
---- /dev/null
-+++ b/file4
-@@ -0,0 +1 @@
-+4
-EOF
 test_expect_success 'test --mixed <paths>' '
 	echo 1 > file1 &&
 	echo 2 > file2 &&
 	git add file1 file2 &&
 	test_tick &&
 	git commit -m files &&
+	before1=$(git rev-parse --short HEAD:file1) &&
+	before2=$(git rev-parse --short HEAD:file2) &&
 	git rm file2 &&
 	echo 3 > file3 &&
 	echo 4 > file4 &&
 	echo 5 > file1 &&
+	after1=$(git rev-parse --short $(git hash-object file1)) &&
+	after4=$(git rev-parse --short $(git hash-object file4)) &&
 	git add file1 file3 file4 &&
 	git reset HEAD -- file1 file2 file3 &&
 	test_must_fail git diff --quiet &&
 	git diff > output &&
+
+	cat > expect <<-EOF &&
+	diff --git a/file1 b/file1
+	index $before1..$after1 100644
+	--- a/file1
+	+++ b/file1
+	@@ -1 +1 @@
+	-1
+	+5
+	diff --git a/file2 b/file2
+	deleted file mode 100644
+	index $before2..0000000
+	--- a/file2
+	+++ /dev/null
+	@@ -1 +0,0 @@
+	-2
+	EOF
+
 	test_cmp expect output &&
 	git diff --cached > output &&
+
+	cat > cached_expect <<-EOF &&
+	diff --git a/file4 b/file4
+	new file mode 100644
+	index 0000000..$after4
+	--- /dev/null
+	+++ b/file4
+	@@ -0,0 +1 @@
+	+4
+	EOF
+
 	test_cmp cached_expect output
 '
 
brian m. carlson· Jul 28, 2020, 23:34 UTC · re: brian m. carlson · lore

[PATCH v5 20/39] t8011: make hash size independent

Allow lines which start with either a 40- or 64-character hex object ID, to allow for both SHA-1 and SHA-256.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t8011-blame-split-file.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Show changes to t/t8011-blame-split-file.sh +1 −1
diff --git a/t/t8011-blame-split-file.sh b/t/t8011-blame-split-file.sh
index 831125047b..bdda0c03fe 100755
--- a/t/t8011-blame-split-file.sh
+++ b/t/t8011-blame-split-file.sh
@@ -54,7 +54,7 @@ test_expect_success 'setup simulated porcelain' '
 	cat >read-porcelain.pl <<-\EOF
 	my $field = shift;
 	while (<>) {
-		if (/^[0-9a-f]{40} /) {
+		if (/^[0-9a-f]{40,} /) {
 			flush();
 			$hash = $&;
 		} elsif (/^$field (.*)/) {
brian m. carlson· Jul 28, 2020, 23:34 UTC · re: brian m. carlson · lore

[PATCH v5 17/39] t7508: use $ZERO_OID instead of hard-coded constant

Use the ZERO_OID variable to abbreviate the all-zeros object ID for maintainability and to avoid depending on a specific size for the hash.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t7508-status.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Show changes to t/t7508-status.sh +1 −1
diff --git a/t/t7508-status.sh b/t/t7508-status.sh
index 8e969f3e36..e81759319f 100755
--- a/t/t7508-status.sh
+++ b/t/t7508-status.sh
@@ -837,7 +837,7 @@ EOF
 '
 
 cat >expect <<EOF
-:100644 100644 $EMPTY_BLOB 0000000000000000000000000000000000000000 M	dir1/modified
+:100644 100644 $EMPTY_BLOB $ZERO_OID M	dir1/modified
 EOF
 test_expect_success 'status refreshes the index' '
 	touch dir2/added &&
brian m. carlson· Jul 28, 2020, 23:34 UTC · re: brian m. carlson · lore

[PATCH v5 27/39] t5308: make test work with SHA-256

This test needs multiple object IDs that have the same first byte. Update the pack test code to generate a suitable packed value for SHA-256. Update the test to use this value when using SHA-256.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/lib-pack.sh                     |  8 ++++++++
 t/t5308-pack-detect-duplicates.sh | 20 ++++++++++++--------
 2 files changed, 20 insertions(+), 8 deletions(-)
Show changes to 2 files +20 −8

t/lib-pack.sh, t/t5308-pack-detect-duplicates.sh

diff --git a/t/lib-pack.sh b/t/lib-pack.sh
index f3463170b3..0c799d53c6 100644
--- a/t/lib-pack.sh
+++ b/t/lib-pack.sh
@@ -93,6 +93,14 @@ pack_obj () {
 			;;
 		esac
 		;;
+	# blob containing "\3\326"
+	471819e8c52bf11513f100b2810a8aa0622d5cd3d1c913758a071dd4b3bad8fe)
+		case "$2" in
+		'')
+			printf '\062\170\234\143\276\006\000\000\336\000\332'
+			return
+			;;
+		esac
 	esac
 
 	# If it's not a delta, we can convince pack-objects to generate a pack
diff --git a/t/t5308-pack-detect-duplicates.sh b/t/t5308-pack-detect-duplicates.sh
index 6845c1f3c3..693b2411c8 100755
--- a/t/t5308-pack-detect-duplicates.sh
+++ b/t/t5308-pack-detect-duplicates.sh
@@ -4,23 +4,27 @@ test_description='handling of duplicate objects in incoming packfiles'
 . ./test-lib.sh
 . "$TEST_DIRECTORY"/lib-pack.sh
 
-if ! test_have_prereq SHA1
-then
-       skip_all='not using SHA-1 for objects'
-       test_done
-fi
+test_expect_success 'setup' '
+	test_oid_cache <<-EOF
+	lo_oid sha1:e68fe8129b546b101aee9510c5328e7f21ca1d18
+	lo_oid sha256:471819e8c52bf11513f100b2810a8aa0622d5cd3d1c913758a071dd4b3bad8fe
+
+	missing_oid sha1:e69d000000000000000000000000000000000000
+	missing_oid sha256:4720000000000000000000000000000000000000000000000000000000000000
+	EOF
+'
 
 # The sha1s we have in our pack. It's important that these have the same
 # starting byte, so that they end up in the same fanout section of the index.
 # That lets us make sure we are exercising the binary search with both sets.
-LO_SHA1=e68fe8129b546b101aee9510c5328e7f21ca1d18
-HI_SHA1=e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
+LO_SHA1=$(test_oid lo_oid)
+HI_SHA1=$EMPTY_BLOB
 
 # And here's a "missing sha1" which will produce failed lookups. It must also
 # be in the same fanout section, and should be between the two (so that during
 # our binary search, we are sure to end up looking at one or the other of the
 # duplicate runs).
-MISSING_SHA1='e69d000000000000000000000000000000000000'
+MISSING_SHA1=$(test_oid missing_oid)
 
 # git will never intentionally create packfiles with
 # duplicate objects, so we have to construct them by hand.
brian m. carlson· Jul 28, 2020, 23:34 UTC · re: brian m. carlson · lore

[PATCH v5 26/39] t9700: make hash size independent

The Perl test script for t9700 was matching on exactly 40 hex characters. With SHA-256, we'll have 64 hex-character object IDs. Create a variable with a regex which matches exactly 40 or 64 hex characters and use that to match the output. Note that both of the uses of this can be anchored, which makes the code simpler, so do that as well.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t9700/test.pl | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
Show changes to t/t9700/test.pl +4 −2
diff --git a/t/t9700/test.pl b/t/t9700/test.pl
index 34cd01366f..071ff6d82a 100755
--- a/t/t9700/test.pl
+++ b/t/t9700/test.pl
@@ -23,6 +23,8 @@ sub adjust_dirsep {
 	return $path;
 }
 
+my $oid_re = qr/^[0-9a-fA-F]{40}(?:[0-9a-fA-F]{24})?$/;
+
 BEGIN { use_ok('Git') }
 
 # set up
@@ -93,7 +95,7 @@ sub adjust_dirsep {
 open TEMPFILE, ">$tmpfile" or die "Can't open $tmpfile: $!";
 print TEMPFILE my $test_text = "test blob, to be inserted\n";
 close TEMPFILE or die "Failed writing to $tmpfile: $!";
-like(our $newhash = $r->hash_and_insert_object($tmpfile), qr/[0-9a-fA-F]{40}/,
+like(our $newhash = $r->hash_and_insert_object($tmpfile), $oid_re,
      "hash_and_insert_object: returns hash");
 open TEMPFILE, "+>$tmpfile" or die "Can't open $tmpfile: $!";
 is($r->cat_blob($newhash, \*TEMPFILE), length $test_text, "cat_blob: roundtrip size");
@@ -119,7 +121,7 @@ sub adjust_dirsep {
 
 # commands in sub directory
 my $last_commit = $r2->command_oneline(qw(rev-parse --verify HEAD));
-like($last_commit, qr/^[0-9a-fA-F]{40}$/, 'rev-parse returned hash');
+like($last_commit, $oid_re, 'rev-parse returned hash');
 my $dir_commit = $r2->command_oneline('log', '-n1', '--pretty=format:%H', '.');
 isnt($last_commit, $dir_commit, 'log . does not show last commit');
 
brian m. carlson· Jul 28, 2020, 23:34 UTC · re: brian m. carlson · lore

[PATCH v5 24/39] t9350: make hash size independent

This test checks for several commit object sizes to verify that objects are encoded as expected. However, the size of a commit object differs between SHA-1 and SHA-256, since each contains a hex representation of the tree's object ID. Since these are root commits, compute the size of each commit by using a constant plus the size of a single hex object ID.

In addition, use $ZERO_OID instead of a hard-coded object ID.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t9350-fast-export.sh | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)
Show changes to t/t9350-fast-export.sh +8 −7
diff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh
index 690c90fb82..ba79db6a17 100755
--- a/t/t9350-fast-export.sh
+++ b/t/t9350-fast-export.sh
@@ -7,6 +7,7 @@ test_description='git fast-export'
 . ./test-lib.sh
 
 test_expect_success 'setup' '
+	test_oid_init &&
 
 	echo break it > file0 &&
 	git add file0 &&
@@ -132,12 +133,12 @@ test_expect_success 'reencoding iso-8859-7' '
 	sed "s/wer/i18n/" iso-8859-7.fi |
 		(cd new &&
 		 git fast-import &&
-		 # The commit object, if not re-encoded, would be 240 bytes.
+		 # The commit object, if not re-encoded, would be 200 bytes plus hash.
 		 # Removing the "encoding iso-8859-7\n" header drops 20 bytes.
 		 # Re-encoding the Pi character from \xF0 (\360) in iso-8859-7
 		 # to \xCF\x80 (\317\200) in UTF-8 adds a byte.  Check for
 		 # the expected size.
-		 test 221 -eq "$(git cat-file -s i18n)" &&
+		 test $(($(test_oid hexsz) + 181)) -eq "$(git cat-file -s i18n)" &&
 		 # ...and for the expected translation of bytes.
 		 git cat-file commit i18n >actual &&
 		 grep $(printf "\317\200") actual &&
@@ -164,12 +165,12 @@ test_expect_success 'preserving iso-8859-7' '
 	sed "s/wer/i18n-no-recoding/" iso-8859-7.fi |
 		(cd new &&
 		 git fast-import &&
-		 # The commit object, if not re-encoded, is 240 bytes.
+		 # The commit object, if not re-encoded, is 200 bytes plus hash.
 		 # Removing the "encoding iso-8859-7\n" header would drops 20
 		 # bytes.  Re-encoding the Pi character from \xF0 (\360) in
 		 # iso-8859-7 to \xCF\x80 (\317\200) in UTF-8 adds a byte.
 		 # Check for the expected size...
-		 test 240 -eq "$(git cat-file -s i18n-no-recoding)" &&
+		 test $(($(test_oid hexsz) + 200)) -eq "$(git cat-file -s i18n-no-recoding)" &&
 		 # ...as well as the expected byte.
 		 git cat-file commit i18n-no-recoding >actual &&
 		 grep $(printf "\360") actual &&
@@ -192,7 +193,7 @@ test_expect_success 'encoding preserved if reencoding fails' '
 		 grep ^encoding actual &&
 		 # Verify that the commit has the expected size; i.e.
 		 # that no bytes were re-encoded to a different encoding.
-		 test 252 -eq "$(git cat-file -s i18n-invalid)" &&
+		 test $(($(test_oid hexsz) + 212)) -eq "$(git cat-file -s i18n-invalid)" &&
 		 # ...and check for the original special bytes
 		 grep $(printf "\360") actual &&
 		 grep $(printf "\377") actual)
@@ -694,7 +695,7 @@ test_expect_success 'delete ref because entire history excluded' '
 	git fast-export to-delete ^to-delete >actual &&
 	cat >expected <<-EOF &&
 	reset refs/heads/to-delete
-	from 0000000000000000000000000000000000000000
+	from $ZERO_OID
 
 	EOF
 	test_cmp expected actual
@@ -704,7 +705,7 @@ test_expect_success 'delete refspec' '
 	git fast-export --refspec :refs/heads/to-delete >actual &&
 	cat >expected <<-EOF &&
 	reset refs/heads/to-delete
-	from 0000000000000000000000000000000000000000
+	from $ZERO_OID
 
 	EOF
 	test_cmp expected actual
brian m. carlson· Jul 28, 2020, 23:34 UTC · re: brian m. carlson · lore

[PATCH v5 22/39] t9300: use $ZERO_OID instead of hard-coded object ID

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t9300-fast-import.sh | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)
Show changes to t/t9300-fast-import.sh +8 −8
diff --git a/t/t9300-fast-import.sh b/t/t9300-fast-import.sh
index f2d2cb06ad..308c1ef42c 100755
--- a/t/t9300-fast-import.sh
+++ b/t/t9300-fast-import.sh
@@ -92,7 +92,7 @@ test_expect_success 'A: create pack from stdin' '
 	EOF
 
 	reset refs/tags/to-be-deleted
-	from 0000000000000000000000000000000000000000
+	from $ZERO_OID
 
 	tag nested
 	mark :6
@@ -102,7 +102,7 @@ test_expect_success 'A: create pack from stdin' '
 	EOF
 
 	reset refs/tags/nested
-	from 0000000000000000000000000000000000000000
+	from $ZERO_OID
 
 	tag nested
 	mark :7
@@ -286,7 +286,7 @@ test_expect_success 'A: verify pack' '
 test_expect_success 'A: verify diff' '
 	copy=$(git rev-parse --verify master:file2) &&
 	cat >expect <<-EOF &&
-	:000000 100755 0000000000000000000000000000000000000000 $copy A	copy-of-file2
+	:000000 100755 $ZERO_OID $copy A	copy-of-file2
 	EOF
 	git diff-tree -M -r master verify--import-marks >actual &&
 	compare_diff_raw expect actual &&
@@ -365,7 +365,7 @@ test_expect_success 'B: fail on invalid blob sha1' '
 	COMMIT
 
 	from refs/heads/master
-	M 755 0000000000000000000000000000000000000001 zero1
+	M 755 $(echo $ZERO_OID | sed -e "s/0$/1/") zero1
 
 	INPUT_END
 
@@ -620,8 +620,8 @@ test_expect_success 'D: validate new files added' '
 	f5id=$(echo "$file5_data" | git hash-object --stdin) &&
 	f6id=$(echo "$file6_data" | git hash-object --stdin) &&
 	cat >expect <<-EOF &&
-	:000000 100755 0000000000000000000000000000000000000000 $f6id A	newdir/exec.sh
-	:000000 100644 0000000000000000000000000000000000000000 $f5id A	newdir/interesting
+	:000000 100755 $ZERO_OID $f6id A	newdir/exec.sh
+	:000000 100644 $ZERO_OID $f5id A	newdir/interesting
 	EOF
 	git diff-tree -M -r branch^ branch >actual &&
 	compare_diff_raw expect actual
@@ -2733,7 +2733,7 @@ test_expect_success 'R: corrupt lines do not mess marks file' '
 	rm -f io.marks &&
 	blob=$(echo hi | git hash-object --stdin) &&
 	cat >expect <<-EOF &&
-	:3 0000000000000000000000000000000000000000
+	:3 $ZERO_OID
 	:1 $blob
 	:2 $blob
 	EOF
@@ -3084,7 +3084,7 @@ test_expect_success 'T: delete branch' '
 	git branch to-delete &&
 	git fast-import <<-EOF &&
 	reset refs/heads/to-delete
-	from 0000000000000000000000000000000000000000
+	from $ZERO_OID
 	EOF
 	test_must_fail git rev-parse --verify refs/heads/to-delete
 '
brian m. carlson· Jul 28, 2020, 23:34 UTC · re: brian m. carlson · lore

[PATCH v5 32/39] setup: add support for reading extensions.objectformat

The transition plan specifies extensions.objectFormat as the indication that we're using a given hash in a certain repo. Read this as one of the extensions we support. If the user has specified an invalid value, fail.

Ensure that we reject the extension if the repository format version is 0.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 setup.c | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)
Show changes to setup.c +15 −1
diff --git a/setup.c b/setup.c
index 3a81307602..94e68bb4f4 100644
--- a/setup.c
+++ b/setup.c
@@ -470,7 +470,16 @@ static int check_repo_format(const char *var, const char *value, void *vdata)
 			data->partial_clone = xstrdup(value);
 		} else if (!strcmp(ext, "worktreeconfig"))
 			data->worktree_config = git_config_bool(var, value);
-		else
+		else if (!strcmp(ext, "objectformat")) {
+			int format;
+
+			if (!value)
+				return config_error_nonbool(var);
+			format = hash_algo_by_name(value);
+			if (format == GIT_HASH_UNKNOWN)
+				return error("invalid value for 'extensions.objectformat'");
+			data->hash_algo = format;
+		} else
 			string_list_append(&data->unknown_extensions, ext);
 	}
 
@@ -613,6 +622,11 @@ int verify_repository_format(const struct repository_format *format,
 		return -1;
 	}
 
+	if (format->version <= 0 && format->hash_algo != GIT_HASH_SHA1) {
+		strbuf_addstr(err, _("extensions.objectFormat is not valid in repo v0"));
+		return -1;
+	}
+
 	return 0;
 }
 
brian m. carlson· Jul 28, 2020, 23:34 UTC · re: brian m. carlson · lore

[PATCH v5 35/39] t: allow testing different hash algorithms via environment

To allow developers to run the testsuite with a different algorithm than the default, provide an environment variable, GIT_TEST_DEFAULT_HASH, to specify the algorithm to use. Compute the fixed constants using test_oid. Move the constant initialization down below the point where test-lib-functions.sh is loaded so the functions are defined.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/test-lib-functions.sh | 4 +---
 t/test-lib.sh           | 3 +++
 2 files changed, 4 insertions(+), 3 deletions(-)
Show changes to 2 files +4 −3

t/test-lib-functions.sh, t/test-lib.sh

diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh
index 2608e80f11..2e01bb2c2b 100644
--- a/t/test-lib-functions.sh
+++ b/t/test-lib-functions.sh
@@ -1417,9 +1417,7 @@ test_set_hash () {
 
 # Detect the hash algorithm in use.
 test_detect_hash () {
-	# Currently we only support SHA-1, but in the future this function will
-	# actually detect the algorithm in use.
-	test_hash_algo='sha1'
+	test_hash_algo="${GIT_TEST_DEFAULT_HASH:-sha1}"
 }
 
 # Load common hash metadata and common placeholder object IDs for use with
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 618a7c8d5b..307bb2207e 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -452,6 +452,9 @@ export GIT_AUTHOR_EMAIL GIT_AUTHOR_NAME
 export GIT_COMMITTER_EMAIL GIT_COMMITTER_NAME
 export EDITOR
 
+GIT_DEFAULT_HASH="${GIT_TEST_DEFAULT_HASH:-sha1}"
+export GIT_DEFAULT_HASH
+
 # Tests using GIT_TRACE typically don't want <timestamp> <file>:<line> output
 GIT_TRACE_BARE=1
 export GIT_TRACE_BARE
brian m. carlson· Jul 28, 2020, 23:34 UTC · re: brian m. carlson · lore

[PATCH v5 38/39] docs: add documentation for extensions.objectFormat

Document the extensions.objectFormat config setting. Warn users not to modify it themselves.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 Documentation/config.txt            | 2 ++
 Documentation/config/extensions.txt | 8 ++++++++
 2 files changed, 10 insertions(+)
 create mode 100644 Documentation/config/extensions.txt
Show changes to 2 files +10 −0

Documentation/config.txt, Documentation/config/extensions.txt

diff --git a/Documentation/config.txt b/Documentation/config.txt
index ef0768b91a..3042d80978 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -348,6 +348,8 @@ include::config/diff.txt[]
 
 include::config/difftool.txt[]
 
+include::config/extensions.txt[]
+
 include::config/fastimport.txt[]
 
 include::config/feature.txt[]
diff --git a/Documentation/config/extensions.txt b/Documentation/config/extensions.txt
new file mode 100644
index 0000000000..4e23d73cdc
--- /dev/null
+++ b/Documentation/config/extensions.txt
@@ -0,0 +1,8 @@
+extensions.objectFormat::
+	Specify the hash algorithm to use.  The acceptable values are `sha1` and
+	`sha256`.  If not specified, `sha1` is assumed.  It is an error to specify
+	this key unless `core.repositoryFormatVersion` is 1.
++
+Note that this setting should only be set by linkgit:git-init[1] or
+linkgit:git-clone[1].  Trying to change it after initialization will not
+work and will produce hard-to-diagnose issues.
brian m. carlson· Jul 28, 2020, 23:34 UTC · re: brian m. carlson · lore

[PATCH v5 36/39] t: make SHA1 prerequisite depend on default hash

Currently, the SHA1 prerequisite depends on the output of git hash-object. However, in order for that to produce sane behavior, we must be in a repository. If we are not, the default will remain SHA-1, and we'll produce wrong results if we're using SHA-256 for the testsuite but the test assertion starts when we're not in a repository.

Check the environment variable we use for this purpose, leaving it to default to SHA-1 if none is specified.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/test-lib.sh | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
Show changes to t/test-lib.sh +5 −1
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 307bb2207e..b278d37d12 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -1689,7 +1689,11 @@ test_lazy_prereq CURL '
 # which will not work with other hash algorithms and tests that work but don't
 # test anything meaningful (e.g. special values which cause short collisions).
 test_lazy_prereq SHA1 '
-	test $(git hash-object /dev/null) = e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
+	case "$GIT_DEFAULT_HASH" in
+	sha1) true ;;
+	"") test $(git hash-object /dev/null) = e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 ;;
+	*) false ::
+	esac
 '
 
 test_lazy_prereq REBASE_P '
Eric Sunshine· Jul 29, 2020, 03:01 UTC · re: brian m. carlson · lore

Re: [PATCH v5 36/39] t: make SHA1 prerequisite depend on default hash

On Tue, Jul 28, 2020 at 7:36 PM brian m. carlson <sandals@crustytoothpaste.net> wrote:

Show 21 quoted lines
> Currently, the SHA1 prerequisite depends on the output of git
> hash-object.  However, in order for that to produce sane behavior, we
> must be in a repository.  If we are not, the default will remain SHA-1,
> and we'll produce wrong results if we're using SHA-256 for the testsuite
> but the test assertion starts when we're not in a repository.
>
> Check the environment variable we use for this purpose, leaving it to
> default to SHA-1 if none is specified.
>
> Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
> ---
> diff --git a/t/test-lib.sh b/t/test-lib.sh
> @@ -1689,7 +1689,11 @@ test_lazy_prereq CURL '
>  test_lazy_prereq SHA1 '
> -       test $(git hash-object /dev/null) = e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> +       case "$GIT_DEFAULT_HASH" in
> +       sha1) true ;;
> +       "") test $(git hash-object /dev/null) = e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 ;;
> +       *) false ::
> +       esac
>  '

This deserves a s/::/;;/ in the final case arm. The "::" does work -- in a manner of speaking -- but only "by accident"; merely because it's being interpreted as an argument to 'false' and it happens to be the final case arm, but it still ought to be fixed.

(I'm pretty sure I typed ";;", not "::", when I gave this code as an example in my previous review... {goes and checks...} indeed, I did type ";;".)

brian m. carlson· Jul 28, 2020, 23:34 UTC · re: brian m. carlson · lore

[PATCH v5 31/39] bundle: add new version for use with SHA-256

Currently we detect the hash algorithm in use by the length of the object ID. This is inelegant and prevents us from using a different hash algorithm that is also 256 bits in length.

Since we cannot extend the v2 format in a backward-compatible way, let's add a v3 format, which is identical, except for the addition of capabilities, which are prefixed by an at sign. We add "object-format" as the only capability and reject unknown capabilities, since we do not have a network connection and therefore cannot negotiate with the other side.

For compatibility, default to the v2 format for SHA-1 and require v3 for SHA-256.

In t5510, always use format v3 so we can be sure we produce consistent results across hash algorithms. Since head -n N lists the top N lines instead of the Nth line, let's run our output through sed to normalize it and compare it against a fixed value, which will make sure we get exactly what we're expecting.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 Documentation/git-bundle.txt              |  9 ++-
 Documentation/technical/bundle-format.txt | 30 +++++++-
 builtin/bundle.c                          |  5 +-
 bundle.c                                  | 83 +++++++++++++++++------
 bundle.h                                  |  4 +-
 t/t5510-fetch.sh                          | 16 +++--
 t/t5607-clone-bundle.sh                   | 31 +++++++++
 7 files changed, 147 insertions(+), 31 deletions(-)
Show changes to 7 files +147 −31

Documentation/git-bundle.txt, Documentation/technical/bundle-format.txt, builtin/bundle.c, bundle.c, bundle.h, t/t5510-fetch.sh, t/t5607-clone-bundle.sh

diff --git a/Documentation/git-bundle.txt b/Documentation/git-bundle.txt
index d34b0964be..53804cad4b 100644
--- a/Documentation/git-bundle.txt
+++ b/Documentation/git-bundle.txt
@@ -9,7 +9,8 @@ git-bundle - Move objects and refs by archive
 SYNOPSIS
 --------
 [verse]
-'git bundle' create [-q | --quiet | --progress | --all-progress] [--all-progress-implied] <file> <git-rev-list-args>
+'git bundle' create [-q | --quiet | --progress | --all-progress] [--all-progress-implied]
+		    [--version=<version>] <file> <git-rev-list-args>
 'git bundle' verify [-q | --quiet] <file>
 'git bundle' list-heads <file> [<refname>...]
 'git bundle' unbundle <file> [<refname>...]
@@ -102,6 +103,12 @@ unbundle <file>::
 	is activated.  Unlike --all-progress this flag doesn't actually
 	force any progress display by itself.
 
+--version=<version>::
+	Specify the bundle version.  Version 2 is the older format and can only be
+	used with SHA-1 repositories; the newer version 3 contains capabilities that
+	permit extensions. The default is the oldest supported format, based on the
+	hash algorithm in use.
+
 -q::
 --quiet::
 	This flag makes the command not to report its progress
diff --git a/Documentation/technical/bundle-format.txt b/Documentation/technical/bundle-format.txt
index 0e828151a5..bac558d049 100644
--- a/Documentation/technical/bundle-format.txt
+++ b/Documentation/technical/bundle-format.txt
@@ -7,6 +7,8 @@ The Git bundle format is a format that represents both refs and Git objects.
 We will use ABNF notation to define the Git bundle format. See
 protocol-common.txt for the details.
 
+A v2 bundle looks like this:
+
 ----
 bundle    = signature *prerequisite *reference LF pack
 signature = "# v2 git bundle" LF
@@ -18,9 +20,28 @@ reference    = obj-id SP refname LF
 pack         = ... ; packfile
 ----
 
+A v3 bundle looks like this:
+
+----
+bundle    = signature *capability *prerequisite *reference LF pack
+signature = "# v3 git bundle" LF
+
+capability   = "@" key ["=" value] LF
+prerequisite = "-" obj-id SP comment LF
+comment      = *CHAR
+reference    = obj-id SP refname LF
+key          = 1*(ALPHA / DIGIT / "-")
+value        = *(%01-09 / %0b-FF)
+
+pack         = ... ; packfile
+----
+
 == Semantics
 
-A Git bundle consists of three parts.
+A Git bundle consists of several parts.
+
+* "Capabilities", which are only in the v3 format, indicate functionality that
+	the bundle requires to be read properly.
 
 * "Prerequisites" lists the objects that are NOT included in the bundle and the
   reader of the bundle MUST already have, in order to use the data in the
@@ -46,3 +67,10 @@ put any string here. The reader of the bundle MUST ignore the comment.
 Note that the prerequisites does not represent a shallow-clone boundary. The
 semantics of the prerequisites and the shallow-clone boundaries are different,
 and the Git bundle v2 format cannot represent a shallow clone repository.
+
+== Capabilities
+
+Because there is no opportunity for negotiation, unknown capabilities cause 'git
+bundle' to abort.  The only known capability is `object-format`, which specifies
+the hash algorithm in use, and can take the same values as the
+`extensions.objectFormat` configuration value.
diff --git a/builtin/bundle.c b/builtin/bundle.c
index f049d27a14..e1a85e7dcc 100644
--- a/builtin/bundle.c
+++ b/builtin/bundle.c
@@ -60,6 +60,7 @@ static int cmd_bundle_create(int argc, const char **argv, const char *prefix) {
 	int all_progress_implied = 0;
 	int progress = isatty(STDERR_FILENO);
 	struct argv_array pack_opts;
+	int version = -1;
 
 	struct option options[] = {
 		OPT_SET_INT('q', "quiet", &progress,
@@ -71,6 +72,8 @@ static int cmd_bundle_create(int argc, const char **argv, const char *prefix) {
 		OPT_BOOL(0, "all-progress-implied",
 			 &all_progress_implied,
 			 N_("similar to --all-progress when progress meter is shown")),
+		OPT_INTEGER(0, "version", &version,
+			    N_("specify bundle format version")),
 		OPT_END()
 	};
 	const char* bundle_file;
@@ -91,7 +94,7 @@ static int cmd_bundle_create(int argc, const char **argv, const char *prefix) {
 
 	if (!startup_info->have_repository)
 		die(_("Need a repository to create a bundle."));
-	return !!create_bundle(the_repository, bundle_file, argc, argv, &pack_opts);
+	return !!create_bundle(the_repository, bundle_file, argc, argv, &pack_opts, version);
 }
 
 static int cmd_bundle_verify(int argc, const char **argv, const char *prefix) {
diff --git a/bundle.c b/bundle.c
index 2a0d744d3f..35585f237c 100644
--- a/bundle.c
+++ b/bundle.c
@@ -12,7 +12,16 @@
 #include "refs.h"
 #include "argv-array.h"
 
-static const char bundle_signature[] = "# v2 git bundle\n";
+
+static const char v2_bundle_signature[] = "# v2 git bundle\n";
+static const char v3_bundle_signature[] = "# v3 git bundle\n";
+static struct {
+	int version;
+	const char *signature;
+} bundle_sigs[] = {
+	{ 2, v2_bundle_signature },
+	{ 3, v3_bundle_signature },
+};
 
 static void add_to_ref_list(const struct object_id *oid, const char *name,
 		struct ref_list *list)
@@ -23,15 +32,30 @@ static void add_to_ref_list(const struct object_id *oid, const char *name,
 	list->nr++;
 }
 
-static const struct git_hash_algo *detect_hash_algo(struct strbuf *buf)
+static int parse_capability(struct bundle_header *header, const char *capability)
 {
-	size_t len = strcspn(buf->buf, " \n");
-	int algo;
+	const char *arg;
+	if (skip_prefix(capability, "object-format=", &arg)) {
+		int algo = hash_algo_by_name(arg);
+		if (algo == GIT_HASH_UNKNOWN)
+			return error(_("unrecognized bundle hash algorithm: %s"), arg);
+		header->hash_algo = &hash_algos[algo];
+		return 0;
+	}
+	return error(_("unknown capability '%s'"), capability);
+}
 
-	algo = hash_algo_by_length(len / 2);
-	if (algo == GIT_HASH_UNKNOWN)
-		return NULL;
-	return &hash_algos[algo];
+static int parse_bundle_signature(struct bundle_header *header, const char *line)
+{
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(bundle_sigs); i++) {
+		if (!strcmp(line, bundle_sigs[i].signature)) {
+			header->version = bundle_sigs[i].version;
+			return 0;
+		}
+	}
+	return -1;
 }
 
 static int parse_bundle_header(int fd, struct bundle_header *header,
@@ -42,14 +66,16 @@ static int parse_bundle_header(int fd, struct bundle_header *header,
 
 	/* The bundle header begins with the signature */
 	if (strbuf_getwholeline_fd(&buf, fd, '\n') ||
-	    strcmp(buf.buf, bundle_signature)) {
+	    parse_bundle_signature(header, buf.buf)) {
 		if (report_path)
-			error(_("'%s' does not look like a v2 bundle file"),
+			error(_("'%s' does not look like a v2 or v3 bundle file"),
 			      report_path);
 		status = -1;
 		goto abort;
 	}
 
+	header->hash_algo = the_hash_algo;
+
 	/* The bundle header ends with an empty line */
 	while (!strbuf_getwholeline_fd(&buf, fd, '\n') &&
 	       buf.len && buf.buf[0] != '\n') {
@@ -57,19 +83,19 @@ static int parse_bundle_header(int fd, struct bundle_header *header,
 		int is_prereq = 0;
 		const char *p;
 
-		if (*buf.buf == '-') {
-			is_prereq = 1;
-			strbuf_remove(&buf, 0, 1);
-		}
 		strbuf_rtrim(&buf);
 
-		if (!header->hash_algo) {
-			header->hash_algo = detect_hash_algo(&buf);
-			if (!header->hash_algo) {
-				error(_("unknown hash algorithm length"));
+		if (header->version == 3 && *buf.buf == '@') {
+			if (parse_capability(header, buf.buf + 1)) {
 				status = -1;
 				break;
 			}
+			continue;
+		}
+
+		if (*buf.buf == '-') {
+			is_prereq = 1;
+			strbuf_remove(&buf, 0, 1);
 		}
 
 		/*
@@ -449,13 +475,14 @@ static int write_bundle_refs(int bundle_fd, struct rev_info *revs)
 }
 
 int create_bundle(struct repository *r, const char *path,
-		  int argc, const char **argv, struct argv_array *pack_options)
+		  int argc, const char **argv, struct argv_array *pack_options, int version)
 {
 	struct lock_file lock = LOCK_INIT;
 	int bundle_fd = -1;
 	int bundle_to_stdout;
 	int ref_count = 0;
 	struct rev_info revs;
+	int min_version = the_hash_algo == &hash_algos[GIT_HASH_SHA1] ? 2 : 3;
 
 	bundle_to_stdout = !strcmp(path, "-");
 	if (bundle_to_stdout)
@@ -464,8 +491,22 @@ int create_bundle(struct repository *r, const char *path,
 		bundle_fd = hold_lock_file_for_update(&lock, path,
 						      LOCK_DIE_ON_ERROR);
 
-	/* write signature */
-	write_or_die(bundle_fd, bundle_signature, strlen(bundle_signature));
+	if (version == -1)
+		version = min_version;
+
+	if (version < 2 || version > 3) {
+		die(_("unsupported bundle version %d"), version);
+	} else if (version < min_version) {
+		die(_("cannot write bundle version %d with algorithm %s"), version, the_hash_algo->name);
+	} else if (version == 2) {
+		write_or_die(bundle_fd, v2_bundle_signature, strlen(v2_bundle_signature));
+	} else {
+		const char *capability = "@object-format=";
+		write_or_die(bundle_fd, v3_bundle_signature, strlen(v3_bundle_signature));
+		write_or_die(bundle_fd, capability, strlen(capability));
+		write_or_die(bundle_fd, the_hash_algo->name, strlen(the_hash_algo->name));
+		write_or_die(bundle_fd, "\n", 1);
+	}
 
 	/* init revs to list objects for pack-objects later */
 	save_commit_buffer = 0;
diff --git a/bundle.h b/bundle.h
index 2dc9442024..70c84cab08 100644
--- a/bundle.h
+++ b/bundle.h
@@ -13,6 +13,7 @@ struct ref_list {
 };
 
 struct bundle_header {
+	unsigned version;
 	struct ref_list prerequisites;
 	struct ref_list references;
 	const struct git_hash_algo *hash_algo;
@@ -21,7 +22,8 @@ struct bundle_header {
 int is_bundle(const char *path, int quiet);
 int read_bundle_header(const char *path, struct bundle_header *header);
 int create_bundle(struct repository *r, const char *path,
-		  int argc, const char **argv, struct argv_array *pack_options);
+		  int argc, const char **argv, struct argv_array *pack_options,
+		  int version);
 int verify_bundle(struct repository *r, struct bundle_header *header, int verbose);
 #define BUNDLE_VERBOSE 1
 int unbundle(struct repository *r, struct bundle_header *header,
diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index a66dbe0bde..9243335ab0 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -281,15 +281,19 @@ test_expect_success 'create bundle 1' '
 	cd "$D" &&
 	echo >file updated again by origin &&
 	git commit -a -m "tip" &&
-	git bundle create bundle1 master^..master
+	git bundle create --version=3 bundle1 master^..master
 '
 
 test_expect_success 'header of bundle looks right' '
-	head -n 4 "$D"/bundle1 &&
-	head -n 1 "$D"/bundle1 | grep "^#" &&
-	head -n 2 "$D"/bundle1 | grep "^-$OID_REGEX " &&
-	head -n 3 "$D"/bundle1 | grep "^$OID_REGEX " &&
-	head -n 4 "$D"/bundle1 | grep "^$"
+	cat >expect <<-EOF &&
+	# v3 git bundle
+	@object-format=$(test_oid algo)
+	-OID updated by origin
+	OID refs/heads/master
+
+	EOF
+	sed -e "s/$OID_REGEX/OID/g" -e "5q" "$D"/bundle1 >actual &&
+	test_cmp expect actual
 '
 
 test_expect_success 'create bundle 2' '
diff --git a/t/t5607-clone-bundle.sh b/t/t5607-clone-bundle.sh
index 6d5a977fcb..26985f4b44 100755
--- a/t/t5607-clone-bundle.sh
+++ b/t/t5607-clone-bundle.sh
@@ -4,6 +4,10 @@ test_description='some bundle related tests'
 . ./test-lib.sh
 
 test_expect_success 'setup' '
+	test_oid_cache <<-EOF &&
+	version sha1:2
+	version sha256:3
+	EOF
 	test_commit initial &&
 	test_tick &&
 	git tag -m tag tag &&
@@ -94,4 +98,31 @@ test_expect_success 'fetch SHA-1 from bundle' '
 	git fetch --no-tags foo/tip.bundle "$(cat hash)"
 '
 
+test_expect_success 'git bundle uses expected default format' '
+	git bundle create bundle HEAD^.. &&
+	head -n1 bundle | grep "^# v$(test_oid version) git bundle$"
+'
+
+test_expect_success 'git bundle v3 has expected contents' '
+	git branch side HEAD &&
+	git bundle create --version=3 bundle HEAD^..side &&
+	head -n2 bundle >actual &&
+	cat >expect <<-EOF &&
+	# v3 git bundle
+	@object-format=$(test_oid algo)
+	EOF
+	test_cmp expect actual &&
+	git bundle verify bundle
+'
+
+test_expect_success 'git bundle v3 rejects unknown capabilities' '
+	cat >new <<-EOF &&
+	# v3 git bundle
+	@object-format=$(test_oid algo)
+	@unknown=silly
+	EOF
+	test_must_fail git bundle verify new 2>output &&
+	test_i18ngrep "unknown capability .unknown=silly." output
+'
+
 test_done
Eric Sunshine· Jul 29, 2020, 02:53 UTC · re: brian m. carlson · lore

Re: [PATCH v5 31/39] bundle: add new version for use with SHA-256

On Tue, Jul 28, 2020 at 7:36 PM brian m. carlson <sandals@crustytoothpaste.net> wrote:

Show 34 quoted lines
> [...]
> Since we cannot extend the v2 format in a backward-compatible way, let's
> add a v3 format, which is identical, except for the addition of
> capabilities, which are prefixed by an at sign.  We add "object-format"
> as the only capability and reject unknown capabilities, since we do not
> have a network connection and therefore cannot negotiate with the other
> side.
> [...]
> Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
> ---
> diff --git a/bundle.c b/bundle.c
> @@ -57,19 +83,19 @@ static int parse_bundle_header(int fd, struct bundle_header *header,
> -               if (*buf.buf == '-') {
> -                       is_prereq = 1;
> -                       strbuf_remove(&buf, 0, 1);
> -               }
>                 strbuf_rtrim(&buf);
>
> -               if (!header->hash_algo) {
> -                       header->hash_algo = detect_hash_algo(&buf);
> -                       if (!header->hash_algo) {
> -                               error(_("unknown hash algorithm length"));
> +               if (header->version == 3 && *buf.buf == '@') {
> +                       if (parse_capability(header, buf.buf + 1)) {
>                                 status = -1;
>                                 break;
>                         }
> +                       continue;
> +               }
> +
> +               if (*buf.buf == '-') {
> +                       is_prereq = 1;
> +                       strbuf_remove(&buf, 0, 1);
>                 }

Moving the strbuf_rtrim() earlier in the loop, as suggested in my previous review, made the diff a lot noisier, uglier, and more difficult to read, however, the code itself ends up being easier to reason about than in the previous round. Good.

By the way (I didn't think of this in my previous review), but wouldn't it be better for this:

    if (header->version == 3 && *buf.buf == '@') {
to instead be written as:
    if (header->version >= 3 && *buf.buf == '@') {

to future-proof it since versions beyond 3 will also almost certainly support "@foo" capabilities?

Show 12 quoted lines
> @@ -449,13 +475,14 @@ static int write_bundle_refs(int bundle_fd, struct rev_info *revs)
> +       int min_version = the_hash_algo == &hash_algos[GIT_HASH_SHA1] ? 2 : 3;
> @@ -464,8 +491,22 @@ int create_bundle(struct repository *r, const char *path,
> +       if (version == -1)
> +               version = min_version;
> +
> +       if (version < 2 || version > 3) {
> +               die(_("unsupported bundle version %d"), version);
> +       } else if (version < min_version) {
> +               die(_("cannot write bundle version %d with algorithm %s"), version, the_hash_algo->name);
> +       } else if (version == 2) {
> +               write_or_die(bundle_fd, v2_bundle_signature, strlen(v2_bundle_signature));

This looks better than the previous attempt; less likely to be fragile as new bundle versions are introduced. Good.

brian m. carlson· Jul 29, 2020, 22:28 UTC · re: Eric Sunshine · lore

Re: [PATCH v5 31/39] bundle: add new version for use with SHA-256

On 2020-07-29 at 02:53:47, Eric Sunshine wrote:
> Moving the strbuf_rtrim() earlier in the loop, as suggested in my
> previous review, made the diff a lot noisier, uglier, and more
> difficult to read, however, the code itself ends up being easier to
> reason about than in the previous round. Good.

Yeah, I noticed that. The code is much nicer, so thank you for that suggestion.

Show 11 quoted lines
> By the way (I didn't think of this in my previous review), but
> wouldn't it be better for this:
> 
>     if (header->version == 3 && *buf.buf == '@') {
> 
> to instead be written as:
> 
>     if (header->version >= 3 && *buf.buf == '@') {
> 
> to future-proof it since versions beyond 3 will also almost certainly
> support "@foo" capabilities?

I think this will need to be dealt with by the author of bundle v4, since currently bundle v4 will error out before we get here. I don't want to make too many assumptions about a future format.

-- 
brian m. carlson: Houston, Texas, US
brian m. carlson· Jul 28, 2020, 23:34 UTC · re: brian m. carlson · lore

[PATCH v5 39/39] t: remove test_oid_init in tests

Now that we call test_oid_init in the setup for all test scripts, there's no point in calling it individually. Remove all of the places where we've done so to help keep tests tidy.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/lib-pack.sh                          | 3 ---
 t/lib-submodule-update.sh              | 1 -
 t/t0000-basic.sh                       | 4 ----
 t/t1006-cat-file.sh                    | 2 --
 t/t1050-large.sh                       | 1 -
 t/t1410-reflog.sh                      | 1 -
 t/t1450-fsck.sh                        | 1 -
 t/t1500-rev-parse.sh                   | 1 -
 t/t3305-notes-fanout.sh                | 1 -
 t/t3308-notes-merge.sh                 | 1 -
 t/t3404-rebase-interactive.sh          | 1 -
 t/t3600-rm.sh                          | 1 -
 t/t3800-mktag.sh                       | 1 -
 t/t4002-diff-basic.sh                  | 2 --
 t/t4027-diff-submodule.sh              | 1 -
 t/t4134-apply-submodule.sh             | 1 -
 t/t4200-rerere.sh                      | 1 -
 t/t4211-line-log.sh                    | 1 -
 t/t5300-pack-object.sh                 | 3 +--
 t/t5302-pack-index.sh                  | 1 -
 t/t5313-pack-bounds-checks.sh          | 1 -
 t/t5318-commit-graph.sh                | 3 +--
 t/t5319-multi-pack-index.sh            | 1 -
 t/t5324-split-commit-graph.sh          | 1 -
 t/t5504-fetch-receive-strict.sh        | 1 -
 t/t5530-upload-pack-error.sh           | 1 -
 t/t5562-http-backend-content-length.sh | 1 -
 t/t5702-protocol-v2.sh                 | 1 -
 t/t5703-upload-pack-ref-in-want.sh     | 1 -
 t/t6006-rev-list-format.sh             | 1 -
 t/t6301-for-each-ref-errors.sh         | 1 -
 t/t6501-freshen-objects.sh             | 4 ----
 t/t7003-filter-branch.sh               | 1 -
 t/t8003-blame-corner-cases.sh          | 2 --
 t/t9301-fast-import-notes.sh           | 1 -
 t/t9350-fast-export.sh                 | 2 --
 36 files changed, 2 insertions(+), 50 deletions(-)
Show changes to 36 files +2 −50

t/lib-pack.sh, t/lib-submodule-update.sh, t/t0000-basic.sh, t/t1006-cat-file.sh, t/t1050-large.sh, t/t1410-reflog.sh, t/t1450-fsck.sh, t/t1500-rev-parse.sh, t/t3305-notes-fanout.sh, t/t3308-notes-merge.sh, t/t3404-rebase-interactive.sh, t/t3600-rm.sh, t/t3800-mktag.sh, t/t4002-diff-basic.sh, t/t4027-diff-submodule.sh, t/t4134-apply-submodule.sh, t/t4200-rerere.sh, t/t4211-line-log.sh, t/t5300-pack-object.sh, t/t5302-pack-index.sh, t/t5313-pack-bounds-checks.sh, t/t5318-commit-graph.sh, t/t5319-multi-pack-index.sh, t/t5324-split-commit-graph.sh, t/t5504-fetch-receive-strict.sh, t/t5530-upload-pack-error.sh, t/t5562-http-backend-content-length.sh, t/t5702-protocol-v2.sh, t/t5703-upload-pack-ref-in-want.sh, t/t6006-rev-list-format.sh, t/t6301-for-each-ref-errors.sh, t/t6501-freshen-objects.sh, t/t7003-filter-branch.sh, t/t8003-blame-corner-cases.sh, t/t9301-fast-import-notes.sh, t/t9350-fast-export.sh

diff --git a/t/lib-pack.sh b/t/lib-pack.sh
index 0c799d53c6..bb8938ccbe 100644
--- a/t/lib-pack.sh
+++ b/t/lib-pack.sh
@@ -35,8 +35,6 @@ pack_header () {
 # have hardcoded some well-known objects. See the case statements below for the
 # complete list.
 pack_obj () {
-	test_oid_init
-
 	case "$1" in
 	# empty blob
 	$EMPTY_BLOB)
@@ -121,7 +119,6 @@ pack_obj () {
 
 # Compute and append pack trailer to "$1"
 pack_trailer () {
-	test_oid_init &&
 	test-tool $(test_oid algo) -b <"$1" >trailer.tmp &&
 	cat trailer.tmp >>"$1" &&
 	rm -f trailer.tmp
diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
index 07c822c8ff..87a759149f 100644
--- a/t/lib-submodule-update.sh
+++ b/t/lib-submodule-update.sh
@@ -196,7 +196,6 @@ test_git_directory_exists () {
 # the submodule repo if it doesn't exist and configures the most problematic
 # settings for diff.ignoreSubmodules.
 prolog () {
-	test_oid_init &&
 	(test -d submodule_update_repo || create_lib_submodule_repo) &&
 	test_config_global diff.ignoreSubmodules all &&
 	test_config diff.ignoreSubmodules all
diff --git a/t/t0000-basic.sh b/t/t0000-basic.sh
index 2599d026a2..362feb43e6 100755
--- a/t/t0000-basic.sh
+++ b/t/t0000-basic.sh
@@ -891,10 +891,6 @@ test_expect_success 'test_atexit is run' "
 	test_path_is_missing also-clean-atexit
 "
 
-test_expect_success 'test_oid setup' '
-	test_oid_init
-'
-
 test_expect_success 'test_oid provides sane info by default' '
 	test_oid zero >actual &&
 	grep "^00*\$" actual &&
diff --git a/t/t1006-cat-file.sh b/t/t1006-cat-file.sh
index 43c4be1e5e..2f501d2dc9 100755
--- a/t/t1006-cat-file.sh
+++ b/t/t1006-cat-file.sh
@@ -140,8 +140,6 @@ test_expect_success '--batch-check without %(rest) considers whole line' '
 	test_cmp expect actual
 '
 
-test_oid_init
-
 tree_sha1=$(git write-tree)
 tree_size=$(($(test_oid rawsz) + 13))
 tree_pretty_content="100644 blob $hello_sha1	hello"
diff --git a/t/t1050-large.sh b/t/t1050-large.sh
index 6a56d1ca24..61e89a8071 100755
--- a/t/t1050-large.sh
+++ b/t/t1050-large.sh
@@ -12,7 +12,6 @@ file_size () {
 }
 
 test_expect_success setup '
-	test_oid_init &&
 	# clone does not allow us to pass core.bigfilethreshold to
 	# new repos, so set core.bigfilethreshold globally
 	git config --global core.bigfilethreshold 200k &&
diff --git a/t/t1410-reflog.sh b/t/t1410-reflog.sh
index 76d9b744a6..730a43d9dd 100755
--- a/t/t1410-reflog.sh
+++ b/t/t1410-reflog.sh
@@ -54,7 +54,6 @@ check_dont_have () {
 }
 
 test_expect_success setup '
-	test_oid_init &&
 	mkdir -p A/B &&
 	echo rat >C &&
 	echo ox >A/D &&
diff --git a/t/t1450-fsck.sh b/t/t1450-fsck.sh
index 344a2aad82..e81f1cbf0e 100755
--- a/t/t1450-fsck.sh
+++ b/t/t1450-fsck.sh
@@ -9,7 +9,6 @@ test_description='git fsck random collection of tests
 . ./test-lib.sh
 
 test_expect_success setup '
-	test_oid_init &&
 	git config gc.auto 0 &&
 	git config i18n.commitencoding ISO-8859-1 &&
 	test_commit A fileA one &&
diff --git a/t/t1500-rev-parse.sh b/t/t1500-rev-parse.sh
index 603019b541..408b97d5af 100755
--- a/t/t1500-rev-parse.sh
+++ b/t/t1500-rev-parse.sh
@@ -59,7 +59,6 @@ test_rev_parse () {
 ROOT=$(pwd)
 
 test_expect_success 'setup' '
-	test_oid_init &&
 	mkdir -p sub/dir work &&
 	cp -R .git repo.git
 '
diff --git a/t/t3305-notes-fanout.sh b/t/t3305-notes-fanout.sh
index d5a94b7174..94c1b02251 100755
--- a/t/t3305-notes-fanout.sh
+++ b/t/t3305-notes-fanout.sh
@@ -31,7 +31,6 @@ all_notes_have_fanout() {
 }
 
 test_expect_success 'tweak test environment' '
-	test_oid_init &&
 	git checkout -b nondeterminism &&
 	test_commit A &&
 	git checkout --orphan with_notes;
diff --git a/t/t3308-notes-merge.sh b/t/t3308-notes-merge.sh
index 790e292966..d69c84c640 100755
--- a/t/t3308-notes-merge.sh
+++ b/t/t3308-notes-merge.sh
@@ -22,7 +22,6 @@ test_expect_success setup '
 	# Copy notes to remote-notes
 	git fetch . refs/notes/*:refs/remote-notes/origin/* &&
 
-	test_oid_init &&
 	test_oid_cache <<-EOF
 	hash4a sha1:5e93d24084d32e1cb61f7070505b9d2530cca987
 	hash3a sha1:8366731eeee53787d2bdf8fc1eff7d94757e8da0
diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh
index 2fb2c581cb..9744e88760 100755
--- a/t/t3404-rebase-interactive.sh
+++ b/t/t3404-rebase-interactive.sh
@@ -30,7 +30,6 @@ Initial setup:
 . "$TEST_DIRECTORY"/lib-rebase.sh
 
 test_expect_success 'setup' '
-	test_oid_init &&
 	test_commit A file1 &&
 	test_commit B file1 &&
 	test_commit C file2 &&
diff --git a/t/t3600-rm.sh b/t/t3600-rm.sh
index f2c0168941..efec8d13b6 100755
--- a/t/t3600-rm.sh
+++ b/t/t3600-rm.sh
@@ -241,7 +241,6 @@ test_expect_success 'refresh index before checking if it is up-to-date' '
 '
 
 test_expect_success 'choking "git rm" should not let it die with cruft' '
-	test_oid_init &&
 	git reset -q --hard &&
 	test_when_finished "rm -f .git/index.lock && git reset -q --hard" &&
 	i=0 &&
diff --git a/t/t3800-mktag.sh b/t/t3800-mktag.sh
index 64dcc5ec28..d696aa4e52 100755
--- a/t/t3800-mktag.sh
+++ b/t/t3800-mktag.sh
@@ -23,7 +23,6 @@ check_verify_failure () {
 # first create a commit, so we have a valid object/type
 # for the tag.
 test_expect_success 'setup' '
-	test_oid_init &&
 	echo Hello >A &&
 	git update-index --add A &&
 	git commit -m "Initial commit" &&
diff --git a/t/t4002-diff-basic.sh b/t/t4002-diff-basic.sh
index cbcdd10464..6a9f010197 100755
--- a/t/t4002-diff-basic.sh
+++ b/t/t4002-diff-basic.sh
@@ -10,8 +10,6 @@ test_description='Test diff raw-output.
 
 . "$TEST_DIRECTORY"/lib-read-tree-m-3way.sh
 
-test_oid_init
-
 test_oid_cache <<\EOF
 aa_1 sha1:ccba72ad3888a3520b39efcf780b9ee64167535d
 aa_1 sha256:9febfbf18197819b2735c45291f138525d2476d59470f98239647544586ba403
diff --git a/t/t4027-diff-submodule.sh b/t/t4027-diff-submodule.sh
index e29deaf4a5..d7145ccca4 100755
--- a/t/t4027-diff-submodule.sh
+++ b/t/t4027-diff-submodule.sh
@@ -6,7 +6,6 @@ test_description='difference in submodules'
 . "$TEST_DIRECTORY"/diff-lib.sh
 
 test_expect_success setup '
-	test_oid_init &&
 	test_tick &&
 	test_create_repo sub &&
 	(
diff --git a/t/t4134-apply-submodule.sh b/t/t4134-apply-submodule.sh
index 99ed4cc546..d1c16ba33c 100755
--- a/t/t4134-apply-submodule.sh
+++ b/t/t4134-apply-submodule.sh
@@ -8,7 +8,6 @@ test_description='git apply submodule tests'
 . ./test-lib.sh
 
 test_expect_success setup '
-	test_oid_init &&
 	cat > create-sm.patch <<EOF &&
 diff --git a/dir/sm b/dir/sm
 new file mode 160000
diff --git a/t/t4200-rerere.sh b/t/t4200-rerere.sh
index 831d424c47..d94d25e4f0 100755
--- a/t/t4200-rerere.sh
+++ b/t/t4200-rerere.sh
@@ -25,7 +25,6 @@ test_description='git rerere
 . ./test-lib.sh
 
 test_expect_success 'setup' '
-	test_oid_init &&
 	cat >a1 <<-\EOF &&
 	Some title
 	==========
diff --git a/t/t4211-line-log.sh b/t/t4211-line-log.sh
index e186c83250..2d1d7b5d19 100755
--- a/t/t4211-line-log.sh
+++ b/t/t4211-line-log.sh
@@ -4,7 +4,6 @@ test_description='test log -L'
 . ./test-lib.sh
 
 test_expect_success 'setup (import history)' '
-	test_oid_init &&
 	git fast-import < "$TEST_DIRECTORY"/t4211/history.export &&
 	git reset --hard
 '
diff --git a/t/t5300-pack-object.sh b/t/t5300-pack-object.sh
index 746cdb626e..cf9f91c941 100755
--- a/t/t5300-pack-object.sh
+++ b/t/t5300-pack-object.sh
@@ -12,8 +12,7 @@ TRASH=$(pwd)
 
 test_expect_success \
     'setup' \
-    'test_oid_init &&
-     rm -f .git/index* &&
+    'rm -f .git/index* &&
      perl -e "print \"a\" x 4096;" > a &&
      perl -e "print \"b\" x 4096;" > b &&
      perl -e "print \"c\" x 4096;" > c &&
diff --git a/t/t5302-pack-index.sh b/t/t5302-pack-index.sh
index 8981c9b90e..119494bd64 100755
--- a/t/t5302-pack-index.sh
+++ b/t/t5302-pack-index.sh
@@ -7,7 +7,6 @@ test_description='pack index with 64-bit offsets and object CRC'
 . ./test-lib.sh
 
 test_expect_success 'setup' '
-	test_oid_init &&
 	rawsz=$(test_oid rawsz) &&
 	rm -rf .git &&
 	git init &&
diff --git a/t/t5313-pack-bounds-checks.sh b/t/t5313-pack-bounds-checks.sh
index 2a4557efc2..535313e4dc 100755
--- a/t/t5313-pack-bounds-checks.sh
+++ b/t/t5313-pack-bounds-checks.sh
@@ -45,7 +45,6 @@ extended_table () {
 }
 
 test_expect_success 'setup' '
-	test_oid_init &&
 	test_oid_cache <<-EOF
 	oid000 sha1:1485
 	oid000 sha256:4222
diff --git a/t/t5318-commit-graph.sh b/t/t5318-commit-graph.sh
index 26f332d6a3..67f2663c7c 100755
--- a/t/t5318-commit-graph.sh
+++ b/t/t5318-commit-graph.sh
@@ -10,8 +10,7 @@ test_expect_success 'setup full repo' '
 	cd "$TRASH_DIRECTORY/full" &&
 	git init &&
 	git config core.commitGraph true &&
-	objdir=".git/objects" &&
-	test_oid_init
+	objdir=".git/objects"
 '
 
 test_expect_success POSIXPERM 'tweak umask for modebit tests' '
diff --git a/t/t5319-multi-pack-index.sh b/t/t5319-multi-pack-index.sh
index 7214cab36c..7dfff0f8f4 100755
--- a/t/t5319-multi-pack-index.sh
+++ b/t/t5319-multi-pack-index.sh
@@ -29,7 +29,6 @@ midx_read_expect () {
 }
 
 test_expect_success 'setup' '
-	test_oid_init &&
 	test_oid_cache <<-EOF
 	idxoff sha1:2999
 	idxoff sha256:3739
diff --git a/t/t5324-split-commit-graph.sh b/t/t5324-split-commit-graph.sh
index 269d0964a3..3a63639fbc 100755
--- a/t/t5324-split-commit-graph.sh
+++ b/t/t5324-split-commit-graph.sh
@@ -12,7 +12,6 @@ test_expect_success 'setup repo' '
 	git config gc.writeCommitGraph false &&
 	infodir=".git/objects/info" &&
 	graphdir="$infodir/commit-graphs" &&
-	test_oid_init &&
 	test_oid_cache <<-EOM
 	shallow sha1:1760
 	shallow sha256:2064
diff --git a/t/t5504-fetch-receive-strict.sh b/t/t5504-fetch-receive-strict.sh
index a32efe2b6c..1a16ac4c0d 100755
--- a/t/t5504-fetch-receive-strict.sh
+++ b/t/t5504-fetch-receive-strict.sh
@@ -4,7 +4,6 @@ test_description='fetch/receive strict mode'
 . ./test-lib.sh
 
 test_expect_success 'setup and inject "corrupt or missing" object' '
-	test_oid_init &&
 	echo hello >greetings &&
 	git add greetings &&
 	git commit -m greetings &&
diff --git a/t/t5530-upload-pack-error.sh b/t/t5530-upload-pack-error.sh
index 4ce9a9f704..205a2631e7 100755
--- a/t/t5530-upload-pack-error.sh
+++ b/t/t5530-upload-pack-error.sh
@@ -14,7 +14,6 @@ corrupt_repo () {
 }
 
 test_expect_success 'setup and corrupt repository' '
-	test_oid_init &&
 	echo file >file &&
 	git add file &&
 	git rev-parse :file &&
diff --git a/t/t5562-http-backend-content-length.sh b/t/t5562-http-backend-content-length.sh
index c6ec625497..e5d3d15ba8 100755
--- a/t/t5562-http-backend-content-length.sh
+++ b/t/t5562-http-backend-content-length.sh
@@ -46,7 +46,6 @@ ssize_b100dots() {
 }
 
 test_expect_success 'setup' '
-	test_oid_init &&
 	HTTP_CONTENT_ENCODING="identity" &&
 	export HTTP_CONTENT_ENCODING &&
 	git config http.receivepack true &&
diff --git a/t/t5702-protocol-v2.sh b/t/t5702-protocol-v2.sh
index 7fc22171e7..5a60fbe3ed 100755
--- a/t/t5702-protocol-v2.sh
+++ b/t/t5702-protocol-v2.sh
@@ -13,7 +13,6 @@ start_git_daemon --export-all --enable=receive-pack
 daemon_parent=$GIT_DAEMON_DOCUMENT_ROOT_PATH/parent
 
 test_expect_success 'create repo to be served by git-daemon' '
-	test_oid_init &&
 	git init "$daemon_parent" &&
 	test_commit -C "$daemon_parent" one
 '
diff --git a/t/t5703-upload-pack-ref-in-want.sh b/t/t5703-upload-pack-ref-in-want.sh
index 748282f058..d9ecf0f4a9 100755
--- a/t/t5703-upload-pack-ref-in-want.sh
+++ b/t/t5703-upload-pack-ref-in-want.sh
@@ -43,7 +43,6 @@ write_command () {
 #             \ | /
 #               a
 test_expect_success 'setup repository' '
-	test_oid_init &&
 	test_commit a &&
 	git checkout -b o/foo &&
 	test_commit b &&
diff --git a/t/t6006-rev-list-format.sh b/t/t6006-rev-list-format.sh
index 7e82e43a63..bc95da8a5f 100755
--- a/t/t6006-rev-list-format.sh
+++ b/t/t6006-rev-list-format.sh
@@ -32,7 +32,6 @@ changed_iso88591=$(echo "$changed" | iconv -f utf-8 -t $test_encoding)
 truncate_count=20
 
 test_expect_success 'setup' '
-	test_oid_init &&
 	: >foo &&
 	git add foo &&
 	git config i18n.commitEncoding $test_encoding &&
diff --git a/t/t6301-for-each-ref-errors.sh b/t/t6301-for-each-ref-errors.sh
index d545a725a5..809854fc0c 100755
--- a/t/t6301-for-each-ref-errors.sh
+++ b/t/t6301-for-each-ref-errors.sh
@@ -7,7 +7,6 @@ test_description='for-each-ref errors for broken refs'
 ZEROS=$ZERO_OID
 
 test_expect_success setup '
-	test_oid_init &&
 	MISSING=$(test_oid deadbeef) &&
 	git commit --allow-empty -m "Initial" &&
 	git tag testtag &&
diff --git a/t/t6501-freshen-objects.sh b/t/t6501-freshen-objects.sh
index 7416ed5ffe..8a3bb4105b 100755
--- a/t/t6501-freshen-objects.sh
+++ b/t/t6501-freshen-objects.sh
@@ -45,10 +45,6 @@ maybe_repack () {
 	fi
 }
 
-test_expect_success 'setup' '
-	test_oid_init
-'
-
 for repack in '' true; do
 	title=${repack:+repack}
 	title=${title:-loose}
diff --git a/t/t7003-filter-branch.sh b/t/t7003-filter-branch.sh
index beeb2b855a..36477cb1f4 100755
--- a/t/t7003-filter-branch.sh
+++ b/t/t7003-filter-branch.sh
@@ -5,7 +5,6 @@ test_description='git filter-branch'
 . "$TEST_DIRECTORY/lib-gpg.sh"
 
 test_expect_success 'setup' '
-	test_oid_init &&
 	test_commit A &&
 	GIT_COMMITTER_DATE="@0 +0000" GIT_AUTHOR_DATE="@0 +0000" &&
 	test_commit --notick B &&
diff --git a/t/t8003-blame-corner-cases.sh b/t/t8003-blame-corner-cases.sh
index ebe79aa4ea..b871dd4f86 100755
--- a/t/t8003-blame-corner-cases.sh
+++ b/t/t8003-blame-corner-cases.sh
@@ -6,8 +6,6 @@ test_description='git blame corner cases'
 pick_fc='s/^[0-9a-f^]* *\([^ ]*\) *(\([^ ]*\) .*/\1-\2/'
 
 test_expect_success setup '
-	test_oid_init &&
-
 	echo A A A A A >one &&
 	echo B B B B B >two &&
 	echo C C C C C >tres &&
diff --git a/t/t9301-fast-import-notes.sh b/t/t9301-fast-import-notes.sh
index 9d1aaac980..ff3338abc0 100755
--- a/t/t9301-fast-import-notes.sh
+++ b/t/t9301-fast-import-notes.sh
@@ -71,7 +71,6 @@ EOF
 INPUT_END
 
 test_expect_success 'set up master branch' '
-	test_oid_init &&
 	git fast-import <input &&
 	git whatchanged master
 '
diff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh
index ba79db6a17..210b9bc4b6 100755
--- a/t/t9350-fast-export.sh
+++ b/t/t9350-fast-export.sh
@@ -7,8 +7,6 @@ test_description='git fast-export'
 . ./test-lib.sh
 
 test_expect_success 'setup' '
-	test_oid_init &&
-
 	echo break it > file0 &&
 	git add file0 &&
 	test_tick &&
brian m. carlson· Jul 28, 2020, 23:34 UTC · re: brian m. carlson · lore

[PATCH v5 34/39] t: add test_oid option to select hash algorithm

In some tests, we have data files which are written with a particular hash algorithm. Instead of keeping two copies of the test files, we can keep one, and translate the value on the fly.

In order to do so, we'll need to read both the source algorithm and the current algorithm, so add an optional flag to the test_oid helper that lets us look up a value for a specified hash algorithm. This should not cause any conflicts with existing tests, since key arguments to test_oid are allowed to contains only shell identifier characters.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t0000-basic.sh        | 11 +++++++++++
 t/test-lib-functions.sh | 12 +++++++++++-
 2 files changed, 22 insertions(+), 1 deletion(-)
Show changes to 2 files +22 −1

t/t0000-basic.sh, t/test-lib-functions.sh

diff --git a/t/t0000-basic.sh b/t/t0000-basic.sh
index 2ff176cd5d..2599d026a2 100755
--- a/t/t0000-basic.sh
+++ b/t/t0000-basic.sh
@@ -928,6 +928,17 @@ test_expect_success 'test_oid can look up data for SHA-256' '
 	test "$hexsz" -eq 64
 '
 
+test_expect_success 'test_oid can look up data for a specified algorithm' '
+	rawsz="$(test_oid --hash=sha1 rawsz)" &&
+	hexsz="$(test_oid --hash=sha1 hexsz)" &&
+	test "$rawsz" -eq 20 &&
+	test "$hexsz" -eq 40 &&
+	rawsz="$(test_oid --hash=sha256 rawsz)" &&
+	hexsz="$(test_oid --hash=sha256 hexsz)" &&
+	test "$rawsz" -eq 32 &&
+	test "$hexsz" -eq 64
+'
+
 test_expect_success 'test_bool_env' '
 	(
 		sane_unset envvar &&
diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh
index 3103be8a32..2608e80f11 100644
--- a/t/test-lib-functions.sh
+++ b/t/test-lib-functions.sh
@@ -1468,7 +1468,17 @@ test_oid_cache () {
 # Look up a per-hash value based on a key ($1).  The value must have been loaded
 # by test_oid_init or test_oid_cache.
 test_oid () {
-	local var="test_oid_${test_hash_algo}_$1" &&
+	local algo="${test_hash_algo}" &&
+
+	case "$1" in
+	--hash=*)
+		algo="${1#--hash=}" &&
+		shift;;
+	*)
+		;;
+	esac &&
+
+	local var="test_oid_${algo}_$1" &&
 
 	# If the variable is unset, we must be missing an entry for this
 	# key-hash pair, so exit with an error.
brian m. carlson· Jul 28, 2020, 23:34 UTC · re: brian m. carlson · lore

[PATCH v5 37/39] ci: run tests with SHA-256

Now that we have Git supporting SHA-256, we'd like to make sure that we don't regress that state. Unfortunately, it's easy to do so, so to help, let's add code to run one of our CI jobs with SHA-256 as the default hash. This will help us detect any problems that may occur.

We pick the linux-clang job because it's relatively fast and the linux-gcc job already runs the testsuite twice. We want our tests to run as fast as possible, so we wouldn't want to add a third run to the linux-gcc job. To make sure we properly exercise the code, let's run the tests in the default mode (SHA-1) first and then run a second time with SHA-256. We explicitly specify SHA-1 for the first run so that if we change the default in the future, we make sure to test both cases.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 ci/run-build-and-tests.sh | 6 ++++++
 1 file changed, 6 insertions(+)
Show changes to ci/run-build-and-tests.sh +6 −0
diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh
index 17e25aade9..6c27b886b8 100755
--- a/ci/run-build-and-tests.sh
+++ b/ci/run-build-and-tests.sh
@@ -24,6 +24,12 @@ linux-gcc)
 	export GIT_TEST_ADD_I_USE_BUILTIN=1
 	make test
 	;;
+linux-clang)
+	export GIT_TEST_DEFAULT_HASH=sha1
+	make test
+	export GIT_TEST_DEFAULT_HASH=sha256
+	make test
+	;;
 linux-gcc-4.8)
 	# Don't run the tests; we only care about whether Git can be
 	# built with GCC 4.8, as it errors out on some undesired (C99)
brian m. carlson· Jul 28, 2020, 23:34 UTC · re: brian m. carlson · lore

[PATCH v5 25/39] t9500: ensure that algorithm info is preserved in config

When we use a hash algorithm other than SHA-1, it's important to preserve the hash-related values in the config file, but this test overwrites the config file with a new one. Ensure we copy these values properly from the old config to the new one so that the repository can be read if it's using SHA-256.

Note that if there is no extensions.objectFormat value set, git config will return unsuccessfully if we try to read it; since this is not an error for us, use test_might_fail.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t9500-gitweb-standalone-no-errors.sh | 22 ++++++++++++++++------
 1 file changed, 16 insertions(+), 6 deletions(-)
Show changes to t/t9500-gitweb-standalone-no-errors.sh +16 −6
diff --git a/t/t9500-gitweb-standalone-no-errors.sh b/t/t9500-gitweb-standalone-no-errors.sh
index 267ddc997d..b484e3e250 100755
--- a/t/t9500-gitweb-standalone-no-errors.sh
+++ b/t/t9500-gitweb-standalone-no-errors.sh
@@ -621,12 +621,22 @@ test_expect_success \
 	 git config gitweb.snapshot "zip,tgz, tbz2" &&
 	 gitweb_run "p=.git;a=tree"'
 
-cat >.git/config <<\EOF
-# testing noval and alternate separator
-[gitweb]
-	blame
-	snapshot = zip tgz
-EOF
+test_expect_success 'setup' '
+	version=$(git config core.repositoryformatversion) &&
+	algo=$(test_might_fail git config extensions.objectformat) &&
+	cat >.git/config <<-\EOF &&
+	# testing noval and alternate separator
+	[gitweb]
+		blame
+		snapshot = zip tgz
+	EOF
+	git config core.repositoryformatversion "$version" &&
+	if test -n "$algo"
+	then
+		git config extensions.objectformat "$algo"
+	fi
+'
+
 test_expect_success \
 	'config override: tree view, features enabled in repo config (2)' \
 	'gitweb_run "p=.git;a=tree"'
brian m. carlson· Jul 28, 2020, 23:34 UTC · re: brian m. carlson · lore

[PATCH v5 01/39] t: make test-bloom initialize repository

The bloom filter code relies on reading object IDs using parse_oid_hex. In order to make that work with an appropriate size, we need to have initialized the repository's hash algorithm. Since the values we're processing depend on the repository in use, let's set up the repository when we run the test helper.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/helper/test-bloom.c | 2 ++
 1 file changed, 2 insertions(+)
Show changes to t/helper/test-bloom.c +2 −0
diff --git a/t/helper/test-bloom.c b/t/helper/test-bloom.c
index f0aa80b98e..5e77d56f59 100644
--- a/t/helper/test-bloom.c
+++ b/t/helper/test-bloom.c
@@ -50,6 +50,8 @@ static const char *bloom_usage = "\n"
 
 int cmd__bloom(int argc, const char **argv)
 {
+	setup_git_directory();
+
 	if (argc < 2)
 		usage(bloom_usage);
 
brian m. carlson· Jul 28, 2020, 23:34 UTC · re: brian m. carlson · lore

[PATCH v5 05/39] t6100: make hash size independent

Instead of hard-coding a constant 40, split the output of rev-list by field.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t6100-rev-list-in-order.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
Show changes to t/t6100-rev-list-in-order.sh +2 −2
diff --git a/t/t6100-rev-list-in-order.sh b/t/t6100-rev-list-in-order.sh
index b2bb0a7f61..e934bc239c 100755
--- a/t/t6100-rev-list-in-order.sh
+++ b/t/t6100-rev-list-in-order.sh
@@ -22,7 +22,7 @@ test_expect_success 'setup a commit history with trees, blobs' '
 
 test_expect_success 'rev-list --in-commit-order' '
 	git rev-list --in-commit-order --objects HEAD >actual.raw &&
-	cut -c 1-40 >actual <actual.raw &&
+	cut -d" " -f1 >actual <actual.raw &&
 
 	git cat-file --batch-check="%(objectname)" >expect.raw <<-\EOF &&
 		HEAD^{commit}
@@ -49,7 +49,7 @@ test_expect_success 'rev-list --in-commit-order' '
 
 test_expect_success 'rev-list lists blobs and trees after commits' '
 	git rev-list --objects HEAD >actual.raw &&
-	cut -c 1-40 >actual <actual.raw &&
+	cut -d" " -f1 >actual <actual.raw &&
 
 	git cat-file --batch-check="%(objectname)" >expect.raw <<-\EOF &&
 		HEAD^{commit}
brian m. carlson· Jul 28, 2020, 23:34 UTC · re: brian m. carlson · lore

[PATCH v5 14/39] t7400: make hash size independent

Instead of using cut with hard-coded hash sizes, use cut with fields, or where that's not possible, sed with $OID_REGEX, so that the tests are independent of hash size.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t7400-submodule-basic.sh | 26 +++++++++++++-------------
 1 file changed, 13 insertions(+), 13 deletions(-)
Show changes to t/t7400-submodule-basic.sh +13 −13
diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
index 956e17abb3..fec7e0299d 100755
--- a/t/t7400-submodule-basic.sh
+++ b/t/t7400-submodule-basic.sh
@@ -1231,7 +1231,7 @@ test_expect_success 'submodule helper list is not confused by common prefixes' '
 	git submodule add /dir1/b dir1/b &&
 	git submodule add /dir2/b dir2/b &&
 	git commit -m "first submodule commit" &&
-	git submodule--helper list dir1/b |cut -c51- >actual &&
+	git submodule--helper list dir1/b | cut -f 2 >actual &&
 	echo "dir1/b" >expect &&
 	test_cmp expect actual
 '
@@ -1260,7 +1260,7 @@ test_expect_success 'submodule update --init with a specification' '
 	pwd=$(pwd) &&
 	git clone file://"$pwd"/multisuper multisuper_clone &&
 	git -C multisuper_clone submodule update --init . ":(exclude)sub0" &&
-	git -C multisuper_clone submodule status |cut -c 1,43- >actual &&
+	git -C multisuper_clone submodule status | sed "s/$OID_REGEX //" >actual &&
 	test_cmp expect actual
 '
 
@@ -1271,7 +1271,7 @@ test_expect_success 'submodule update --init with submodule.active set' '
 	git -C multisuper_clone config submodule.active "." &&
 	git -C multisuper_clone config --add submodule.active ":(exclude)sub0" &&
 	git -C multisuper_clone submodule update --init &&
-	git -C multisuper_clone submodule status |cut -c 1,43- >actual &&
+	git -C multisuper_clone submodule status | sed "s/$OID_REGEX //" >actual &&
 	test_cmp expect actual
 '
 
@@ -1290,7 +1290,7 @@ test_expect_success 'submodule update and setting submodule.<name>.active' '
 	-sub3
 	EOF
 	git -C multisuper_clone submodule update &&
-	git -C multisuper_clone submodule status |cut -c 1,43- >actual &&
+	git -C multisuper_clone submodule status | sed "s/$OID_REGEX //" >actual &&
 	test_cmp expect actual
 '
 
@@ -1307,12 +1307,12 @@ test_expect_success 'clone active submodule without submodule url set' '
 		git submodule update &&
 		git submodule status >actual_raw &&
 
-		cut -c 1,43- actual_raw >actual &&
+		cut -d" " -f3- actual_raw >actual &&
 		cat >expect <<-\EOF &&
-		 sub0 (test2)
-		 sub1 (test2)
-		 sub2 (test2)
-		 sub3 (test2)
+		sub0 (test2)
+		sub1 (test2)
+		sub2 (test2)
+		sub3 (test2)
 		EOF
 		test_cmp expect actual
 	)
@@ -1328,7 +1328,7 @@ test_expect_success 'clone --recurse-submodules with a pathspec works' '
 	EOF
 
 	git clone --recurse-submodules="sub0" multisuper multisuper_clone &&
-	git -C multisuper_clone submodule status |cut -c1,43- >actual &&
+	git -C multisuper_clone submodule status | sed "s/$OID_REGEX //" >actual &&
 	test_cmp expected actual
 '
 
@@ -1345,7 +1345,7 @@ test_expect_success 'clone with multiple --recurse-submodules options' '
 		  --recurse-submodules=":(exclude)sub0" \
 		  --recurse-submodules=":(exclude)sub2" \
 		  multisuper multisuper_clone &&
-	git -C multisuper_clone submodule status |cut -c1,43- >actual &&
+	git -C multisuper_clone submodule status | sed "s/$OID_REGEX //" >actual &&
 	test_cmp expect actual
 '
 
@@ -1373,7 +1373,7 @@ test_expect_success 'clone and subsequent updates correctly auto-initialize subm
 		  --recurse-submodules=":(exclude)sub4" \
 		  multisuper multisuper_clone &&
 
-	git -C multisuper_clone submodule status |cut -c1,43- >actual &&
+	git -C multisuper_clone submodule status | sed "s/$OID_REGEX //" >actual &&
 	test_cmp expect actual &&
 
 	git -C multisuper submodule add ../sub1 sub4 &&
@@ -1382,7 +1382,7 @@ test_expect_success 'clone and subsequent updates correctly auto-initialize subm
 	# obtain the new superproject
 	git -C multisuper_clone pull &&
 	git -C multisuper_clone submodule update --init &&
-	git -C multisuper_clone submodule status |cut -c1,43- >actual &&
+	git -C multisuper_clone submodule status | sed "s/$OID_REGEX //" >actual &&
 	test_cmp expect2 actual
 '
 
brian m. carlson· Jul 28, 2020, 23:34 UTC · re: brian m. carlson · lore

[PATCH v5 11/39] t7063: make hash size independent

Use test_oid instead of hard-coding algorithm-specific constants and all-zero values.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t7063-status-untracked-cache.sh | 150 ++++++++++++++++--------------
 1 file changed, 80 insertions(+), 70 deletions(-)
Show changes to t/t7063-status-untracked-cache.sh +80 −70
diff --git a/t/t7063-status-untracked-cache.sh b/t/t7063-status-untracked-cache.sh
index 428cff9cf3..a682a3d826 100755
--- a/t/t7063-status-untracked-cache.sh
+++ b/t/t7063-status-untracked-cache.sh
@@ -75,14 +75,24 @@ test_expect_success 'setup' '
 	touch one two three done/one dtwo/two dthree/three &&
 	git add one two done/one &&
 	: >.git/info/exclude &&
-	git update-index --untracked-cache
+	git update-index --untracked-cache &&
+	test_oid_cache <<-EOF
+	root sha1:e6fcc8f2ee31bae321d66afd183fcb7237afae6e
+	root sha256:b90c672088c015b9c83876e919da311bad4cd39639fb139f988af6a11493b974
+
+	exclude sha1:13263c0978fb9fad16b2d580fb800b6d811c3ff0
+	exclude sha256:fe4aaa1bbbbce4cb8f73426748a14c5ad6026b26f90505a0bf2494b165a5b76c
+
+	done sha1:1946f0437f90c5005533cbe1736a6451ca301714
+	done sha256:7f079501d79f665b3acc50f5e0e9e94509084d5032ac20113a37dd5029b757cc
+	EOF
 '
 
 test_expect_success 'untracked cache is empty' '
 	test-tool dump-untracked-cache >../actual &&
 	cat >../expect-empty <<EOF &&
-info/exclude 0000000000000000000000000000000000000000
-core.excludesfile 0000000000000000000000000000000000000000
+info/exclude $ZERO_OID
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
 EOF
@@ -100,17 +110,17 @@ EOF
 
 cat >../dump.expect <<EOF &&
 info/exclude $EMPTY_BLOB
-core.excludesfile 0000000000000000000000000000000000000000
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
-/ 0000000000000000000000000000000000000000 recurse valid
+/ $ZERO_OID recurse valid
 dthree/
 dtwo/
 three
-/done/ 0000000000000000000000000000000000000000 recurse valid
-/dthree/ 0000000000000000000000000000000000000000 recurse check_only valid
+/done/ $ZERO_OID recurse valid
+/dthree/ $ZERO_OID recurse check_only valid
 three
-/dtwo/ 0000000000000000000000000000000000000000 recurse check_only valid
+/dtwo/ $ZERO_OID recurse check_only valid
 two
 EOF
 
@@ -190,18 +200,18 @@ test_expect_success 'verify untracked cache dump' '
 	test-tool dump-untracked-cache >../actual &&
 	cat >../expect <<EOF &&
 info/exclude $EMPTY_BLOB
-core.excludesfile 0000000000000000000000000000000000000000
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
-/ 0000000000000000000000000000000000000000 recurse valid
+/ $ZERO_OID recurse valid
 dthree/
 dtwo/
 four
 three
-/done/ 0000000000000000000000000000000000000000 recurse valid
-/dthree/ 0000000000000000000000000000000000000000 recurse check_only valid
+/done/ $ZERO_OID recurse valid
+/dthree/ $ZERO_OID recurse check_only valid
 three
-/dtwo/ 0000000000000000000000000000000000000000 recurse check_only valid
+/dtwo/ $ZERO_OID recurse check_only valid
 two
 EOF
 	test_cmp ../expect ../actual
@@ -239,18 +249,18 @@ test_expect_success 'verify untracked cache dump' '
 	test-tool dump-untracked-cache >../actual &&
 	cat >../expect <<EOF &&
 info/exclude $EMPTY_BLOB
-core.excludesfile 0000000000000000000000000000000000000000
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
-/ e6fcc8f2ee31bae321d66afd183fcb7237afae6e recurse valid
+/ $(test_oid root) recurse valid
 .gitignore
 dthree/
 dtwo/
 three
-/done/ 0000000000000000000000000000000000000000 recurse valid
-/dthree/ 0000000000000000000000000000000000000000 recurse check_only valid
+/done/ $ZERO_OID recurse valid
+/dthree/ $ZERO_OID recurse check_only valid
 three
-/dtwo/ 0000000000000000000000000000000000000000 recurse check_only valid
+/dtwo/ $ZERO_OID recurse check_only valid
 two
 EOF
 	test_cmp ../expect ../actual
@@ -284,16 +294,16 @@ EOF
 test_expect_success 'verify untracked cache dump' '
 	test-tool dump-untracked-cache >../actual &&
 	cat >../expect <<EOF &&
-info/exclude 13263c0978fb9fad16b2d580fb800b6d811c3ff0
-core.excludesfile 0000000000000000000000000000000000000000
+info/exclude $(test_oid exclude)
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
-/ e6fcc8f2ee31bae321d66afd183fcb7237afae6e recurse valid
+/ $(test_oid root) recurse valid
 .gitignore
 dtwo/
-/done/ 0000000000000000000000000000000000000000 recurse valid
-/dthree/ 0000000000000000000000000000000000000000 recurse check_only valid
-/dtwo/ 0000000000000000000000000000000000000000 recurse check_only valid
+/done/ $ZERO_OID recurse valid
+/dthree/ $ZERO_OID recurse check_only valid
+/dtwo/ $ZERO_OID recurse check_only valid
 two
 EOF
 	test_cmp ../expect ../actual
@@ -303,14 +313,14 @@ test_expect_success 'move two from tracked to untracked' '
 	git rm --cached two &&
 	test-tool dump-untracked-cache >../actual &&
 	cat >../expect <<EOF &&
-info/exclude 13263c0978fb9fad16b2d580fb800b6d811c3ff0
-core.excludesfile 0000000000000000000000000000000000000000
+info/exclude $(test_oid exclude)
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
-/ e6fcc8f2ee31bae321d66afd183fcb7237afae6e recurse
-/done/ 0000000000000000000000000000000000000000 recurse valid
-/dthree/ 0000000000000000000000000000000000000000 recurse check_only valid
-/dtwo/ 0000000000000000000000000000000000000000 recurse check_only valid
+/ $(test_oid root) recurse
+/done/ $ZERO_OID recurse valid
+/dthree/ $ZERO_OID recurse check_only valid
+/dtwo/ $ZERO_OID recurse check_only valid
 two
 EOF
 	test_cmp ../expect ../actual
@@ -342,17 +352,17 @@ EOF
 test_expect_success 'verify untracked cache dump' '
 	test-tool dump-untracked-cache >../actual &&
 	cat >../expect <<EOF &&
-info/exclude 13263c0978fb9fad16b2d580fb800b6d811c3ff0
-core.excludesfile 0000000000000000000000000000000000000000
+info/exclude $(test_oid exclude)
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
-/ e6fcc8f2ee31bae321d66afd183fcb7237afae6e recurse valid
+/ $(test_oid root) recurse valid
 .gitignore
 dtwo/
 two
-/done/ 0000000000000000000000000000000000000000 recurse valid
-/dthree/ 0000000000000000000000000000000000000000 recurse check_only valid
-/dtwo/ 0000000000000000000000000000000000000000 recurse check_only valid
+/done/ $ZERO_OID recurse valid
+/dthree/ $ZERO_OID recurse check_only valid
+/dtwo/ $ZERO_OID recurse check_only valid
 two
 EOF
 	test_cmp ../expect ../actual
@@ -362,14 +372,14 @@ test_expect_success 'move two from untracked to tracked' '
 	git add two &&
 	test-tool dump-untracked-cache >../actual &&
 	cat >../expect <<EOF &&
-info/exclude 13263c0978fb9fad16b2d580fb800b6d811c3ff0
-core.excludesfile 0000000000000000000000000000000000000000
+info/exclude $(test_oid exclude)
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
-/ e6fcc8f2ee31bae321d66afd183fcb7237afae6e recurse
-/done/ 0000000000000000000000000000000000000000 recurse valid
-/dthree/ 0000000000000000000000000000000000000000 recurse check_only valid
-/dtwo/ 0000000000000000000000000000000000000000 recurse check_only valid
+/ $(test_oid root) recurse
+/done/ $ZERO_OID recurse valid
+/dthree/ $ZERO_OID recurse check_only valid
+/dtwo/ $ZERO_OID recurse check_only valid
 two
 EOF
 	test_cmp ../expect ../actual
@@ -401,16 +411,16 @@ EOF
 test_expect_success 'verify untracked cache dump' '
 	test-tool dump-untracked-cache >../actual &&
 	cat >../expect <<EOF &&
-info/exclude 13263c0978fb9fad16b2d580fb800b6d811c3ff0
-core.excludesfile 0000000000000000000000000000000000000000
+info/exclude $(test_oid exclude)
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
-/ e6fcc8f2ee31bae321d66afd183fcb7237afae6e recurse valid
+/ $(test_oid root) recurse valid
 .gitignore
 dtwo/
-/done/ 0000000000000000000000000000000000000000 recurse valid
-/dthree/ 0000000000000000000000000000000000000000 recurse check_only valid
-/dtwo/ 0000000000000000000000000000000000000000 recurse check_only valid
+/done/ $ZERO_OID recurse valid
+/dthree/ $ZERO_OID recurse check_only valid
+/dtwo/ $ZERO_OID recurse check_only valid
 two
 EOF
 	test_cmp ../expect ../actual
@@ -447,16 +457,16 @@ EOF
 test_expect_success 'untracked cache correct after commit' '
 	test-tool dump-untracked-cache >../actual &&
 	cat >../expect <<EOF &&
-info/exclude 13263c0978fb9fad16b2d580fb800b6d811c3ff0
-core.excludesfile 0000000000000000000000000000000000000000
+info/exclude $(test_oid exclude)
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
-/ e6fcc8f2ee31bae321d66afd183fcb7237afae6e recurse valid
+/ $(test_oid root) recurse valid
 .gitignore
 dtwo/
-/done/ 0000000000000000000000000000000000000000 recurse valid
-/dthree/ 0000000000000000000000000000000000000000 recurse check_only valid
-/dtwo/ 0000000000000000000000000000000000000000 recurse check_only valid
+/done/ $ZERO_OID recurse valid
+/dthree/ $ZERO_OID recurse check_only valid
+/dtwo/ $ZERO_OID recurse check_only valid
 two
 EOF
 	test_cmp ../expect ../actual
@@ -508,17 +518,17 @@ EOF
 test_expect_success 'untracked cache correct after status' '
 	test-tool dump-untracked-cache >../actual &&
 	cat >../expect <<EOF &&
-info/exclude 13263c0978fb9fad16b2d580fb800b6d811c3ff0
-core.excludesfile 0000000000000000000000000000000000000000
+info/exclude $(test_oid exclude)
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
-/ e6fcc8f2ee31bae321d66afd183fcb7237afae6e recurse valid
+/ $(test_oid root) recurse valid
 .gitignore
 dtwo/
-/done/ 1946f0437f90c5005533cbe1736a6451ca301714 recurse valid
+/done/ $(test_oid done) recurse valid
 five
-/dthree/ 0000000000000000000000000000000000000000 recurse check_only valid
-/dtwo/ 0000000000000000000000000000000000000000 recurse check_only valid
+/dthree/ $ZERO_OID recurse check_only valid
+/dtwo/ $ZERO_OID recurse check_only valid
 two
 EOF
 	test_cmp ../expect ../actual
@@ -580,22 +590,22 @@ EOF
 test_expect_success 'verify untracked cache dump (sparse/subdirs)' '
 	test-tool dump-untracked-cache >../actual &&
 	cat >../expect-from-test-dump <<EOF &&
-info/exclude 13263c0978fb9fad16b2d580fb800b6d811c3ff0
-core.excludesfile 0000000000000000000000000000000000000000
+info/exclude $(test_oid exclude)
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
-/ e6fcc8f2ee31bae321d66afd183fcb7237afae6e recurse valid
+/ $(test_oid root) recurse valid
 .gitignore
 dtwo/
-/done/ 1946f0437f90c5005533cbe1736a6451ca301714 recurse valid
+/done/ $(test_oid done) recurse valid
 five
 sub/
-/done/sub/ 0000000000000000000000000000000000000000 recurse check_only valid
+/done/sub/ $ZERO_OID recurse check_only valid
 sub/
-/done/sub/sub/ 0000000000000000000000000000000000000000 recurse check_only valid
+/done/sub/sub/ $ZERO_OID recurse check_only valid
 file
-/dthree/ 0000000000000000000000000000000000000000 recurse check_only valid
-/dtwo/ 0000000000000000000000000000000000000000 recurse check_only valid
+/dthree/ $ZERO_OID recurse check_only valid
+/dtwo/ $ZERO_OID recurse check_only valid
 two
 EOF
 	test_cmp ../expect-from-test-dump ../actual
@@ -806,8 +816,8 @@ test_expect_success '"status" after file replacement should be clean with UC=tru
 	test-tool dump-untracked-cache >../actual &&
 	grep -F "recurse valid" ../actual >../actual.grep &&
 	cat >../expect.grep <<EOF &&
-/ 0000000000000000000000000000000000000000 recurse valid
-/two/ 0000000000000000000000000000000000000000 recurse valid
+/ $ZERO_OID recurse valid
+/two/ $ZERO_OID recurse valid
 EOF
 	status_is_clean &&
 	test_cmp ../expect.grep ../actual.grep
brian m. carlson· Jul 28, 2020, 23:34 UTC · re: brian m. carlson · lore

[PATCH v5 09/39] t6501: avoid hard-coded objects

This test contains hard-coded invalid object IDs. Make it hash size independent by generating invalid object IDs using the translation tables. Add a setup target to ensure the output of test_oid_init is checked properly.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t6501-freshen-objects.sh | 18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)
Show changes to t/t6501-freshen-objects.sh +11 −7
diff --git a/t/t6501-freshen-objects.sh b/t/t6501-freshen-objects.sh
index f30b4849b6..7416ed5ffe 100755
--- a/t/t6501-freshen-objects.sh
+++ b/t/t6501-freshen-objects.sh
@@ -45,6 +45,10 @@ maybe_repack () {
 	fi
 }
 
+test_expect_success 'setup' '
+	test_oid_init
+'
+
 for repack in '' true; do
 	title=${repack:+repack}
 	title=${title:-loose}
@@ -128,9 +132,9 @@ for repack in '' true; do
 done
 
 test_expect_success 'do not complain about existing broken links (commit)' '
-	cat >broken-commit <<-\EOF &&
-	tree 0000000000000000000000000000000000000001
-	parent 0000000000000000000000000000000000000002
+	cat >broken-commit <<-EOF &&
+	tree $(test_oid 001)
+	parent $(test_oid 002)
 	author whatever <whatever@example.com> 1234 -0000
 	committer whatever <whatever@example.com> 1234 -0000
 
@@ -143,8 +147,8 @@ test_expect_success 'do not complain about existing broken links (commit)' '
 '
 
 test_expect_success 'do not complain about existing broken links (tree)' '
-	cat >broken-tree <<-\EOF &&
-	100644 blob 0000000000000000000000000000000000000003	foo
+	cat >broken-tree <<-EOF &&
+	100644 blob $(test_oid 003)	foo
 	EOF
 	tree=$(git mktree --missing <broken-tree) &&
 	git gc -q 2>stderr &&
@@ -153,8 +157,8 @@ test_expect_success 'do not complain about existing broken links (tree)' '
 '
 
 test_expect_success 'do not complain about existing broken links (tag)' '
-	cat >broken-tag <<-\EOF &&
-	object 0000000000000000000000000000000000000004
+	cat >broken-tag <<-EOF &&
+	object $(test_oid 004)
 	type commit
 	tag broken
 	tagger whatever <whatever@example.com> 1234 -0000
brian m. carlson· Jul 28, 2020, 23:34 UTC · re: brian m. carlson · lore

[PATCH v5 07/39] t6301: make hash size independent

Instead of hard-coding a fixed length example object ID in the test, compute one using the translation tables. Move a variable into the setup block so that we can ensure the exit status of test_oid_init is checked.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t6301-for-each-ref-errors.sh | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
Show changes to t/t6301-for-each-ref-errors.sh +2 −1
diff --git a/t/t6301-for-each-ref-errors.sh b/t/t6301-for-each-ref-errors.sh
index 49cc65bb58..d545a725a5 100755
--- a/t/t6301-for-each-ref-errors.sh
+++ b/t/t6301-for-each-ref-errors.sh
@@ -5,9 +5,10 @@ test_description='for-each-ref errors for broken refs'
 . ./test-lib.sh
 
 ZEROS=$ZERO_OID
-MISSING=abababababababababababababababababababab
 
 test_expect_success setup '
+	test_oid_init &&
+	MISSING=$(test_oid deadbeef) &&
 	git commit --allow-empty -m "Initial" &&
 	git tag testtag &&
 	git for-each-ref >full-list &&
brian m. carlson· Jul 28, 2020, 23:34 UTC · re: brian m. carlson · lore

[PATCH v5 29/39] http-fetch: set up git directory before parsing pack hashes

In dd4b732df7 ("upload-pack: send part of packfile response as uri", 2020-06-10), the git http-fetch code learned how to take ac --packfile option. This option takes an argument, which is the name of a packfile hash, and parses it using parse_oid_hex. It does so before calling setup_git_directory.

However, in a SHA-256 repository this fails to work, since we have not set the hash algorithm in use and parse_oid_hex fails as a consequence. To ensure that we can parse packfile hashes of the right length, let's set up the git directory before we start parsing arguments.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 http-fetch.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
Show changes to http-fetch.c +2 −2
diff --git a/http-fetch.c b/http-fetch.c
index 1df376e745..65050dee2b 100644
--- a/http-fetch.c
+++ b/http-fetch.c
@@ -86,6 +86,8 @@ int cmd_main(int argc, const char **argv)
 	int packfile = 0;
 	struct object_id packfile_hash;
 
+	setup_git_directory_gently(NULL);
+
 	while (arg < argc && argv[arg][0] == '-') {
 		const char *p;
 
@@ -115,8 +117,6 @@ int cmd_main(int argc, const char **argv)
 	if (argc != arg + 2 - (commits_on_stdin || packfile))
 		usage(http_fetch_usage);
 
-	setup_git_directory();
-
 	git_config(git_default_config, NULL);
 
 	if (packfile) {
Eric Sunshine· Jul 29, 2020, 02:36 UTC · re: brian m. carlson · lore

Re: [PATCH v5 29/39] http-fetch: set up git directory before parsing pack hashes

On Tue, Jul 28, 2020 at 7:35 PM brian m. carlson <sandals@crustytoothpaste.net> wrote:

Show 24 quoted lines
> In dd4b732df7 ("upload-pack: send part of packfile response as uri",
> 2020-06-10), the git http-fetch code learned how to take  ac --packfile
> option.  This option takes an argument, which is the name of a packfile
> hash, and parses it using parse_oid_hex.  It does so before calling
> setup_git_directory.
>
> However, in a SHA-256 repository this fails to work, since we have not
> set the hash algorithm in use and parse_oid_hex fails as a consequence.
> To ensure that we can parse packfile hashes of the right length, let's
> set up the git directory before we start parsing arguments.
>
> Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
> ---
> diff --git a/http-fetch.c b/http-fetch.c
> @@ -86,6 +86,8 @@ int cmd_main(int argc, const char **argv)
> +       setup_git_directory_gently(NULL);
> +
>         while (arg < argc && argv[arg][0] == '-') {
>                 const char *p;
> @@ -115,8 +117,6 @@ int cmd_main(int argc, const char **argv)
>         if (argc != arg + 2 - (commits_on_stdin || packfile))
>                 usage(http_fetch_usage);
>
> -       setup_git_directory();

In the previous version of this series the setup_git_directory() call was moved verbatim above the 'while' loop, and my review pointed out that doing so made "git-http-fetch -h" fail when not in a Git repository. It was recommended, therefore, to take advantage of setup_git_directory_gently() to fix this problem.

But note that setup_git_directory() is defined like this:
    const char *setup_git_directory(void)
    {
        return setup_git_directory_gently(NULL);
    }

So, your use of setup_git_directory_gently(NULL) in this version of the patch is no better than the plain setup_git_directory() call in the previous version. It is still impossible to ask git-http-fetch for help when not in a Git repository.

To fix this, you must pass non-NULL for the 'nongit' argument of setup_git_directory_gently().

For robustness, in order to retain the original behavior (prior to this patch), it would be best to check the value of 'nongit' _after_ the 'while' loop and error out if not in a repository. Perhaps something like this (taking a hint from builtin/diff.c):

    setup_git_directory_gently(&nongit);
    while (...) {
        ...
    }
    if (nongit)
        die(_("not a git repository"));
brian m. carlson· Jul 28, 2020, 23:34 UTC · re: brian m. carlson · lore

[PATCH v5 21/39] t9300: abstract away SHA-1-specific constants

Adjust the test so that it computes variables for object IDs instead of using hard-coded hashes. In addition, use cut to filter out the object IDs and verify only the information that we're really interested in.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t9300-fast-import.sh | 108 ++++++++++++++++++++++-------------------
 1 file changed, 59 insertions(+), 49 deletions(-)
Show changes to t/t9300-fast-import.sh +59 −49
diff --git a/t/t9300-fast-import.sh b/t/t9300-fast-import.sh
index e151df81c0..f2d2cb06ad 100755
--- a/t/t9300-fast-import.sh
+++ b/t/t9300-fast-import.sh
@@ -284,8 +284,9 @@ test_expect_success 'A: verify pack' '
 '
 
 test_expect_success 'A: verify diff' '
+	copy=$(git rev-parse --verify master:file2) &&
 	cat >expect <<-EOF &&
-	:000000 100755 0000000000000000000000000000000000000000 7123f7f44e39be127c5eb701e5968176ee9d78b1 A	copy-of-file2
+	:000000 100755 0000000000000000000000000000000000000000 $copy A	copy-of-file2
 	EOF
 	git diff-tree -M -r master verify--import-marks >actual &&
 	compare_diff_raw expect actual &&
@@ -528,6 +529,7 @@ test_expect_success 'B: fail on invalid committer (5)' '
 test_expect_success 'C: incremental import create pack from stdin' '
 	newf=$(echo hi newf | git hash-object -w --stdin) &&
 	oldf=$(git rev-parse --verify master:file2) &&
+	thrf=$(git rev-parse --verify master:file3) &&
 	test_tick &&
 	cat >input <<-INPUT_END &&
 	commit refs/heads/branch
@@ -570,10 +572,11 @@ test_expect_success 'C: verify commit' '
 '
 
 test_expect_success 'C: validate rename result' '
+	zero=$ZERO_OID &&
 	cat >expect <<-EOF &&
-	:000000 100755 0000000000000000000000000000000000000000 f1fb5da718392694d0076d677d6d0e364c79b0bc A	file2/newf
-	:100644 100644 7123f7f44e39be127c5eb701e5968176ee9d78b1 7123f7f44e39be127c5eb701e5968176ee9d78b1 R100	file2	file2/oldf
-	:100644 000000 0d92e9f3374ae2947c23aa477cbc68ce598135f1 0000000000000000000000000000000000000000 D	file3
+	:000000 100755 $zero $newf A	file2/newf
+	:100644 100644 $oldf $oldf R100	file2	file2/oldf
+	:100644 000000 $thrf $zero D	file3
 	EOF
 	git diff-tree -M -r master branch >actual &&
 	compare_diff_raw expect actual
@@ -614,9 +617,11 @@ test_expect_success 'D: verify pack' '
 '
 
 test_expect_success 'D: validate new files added' '
+	f5id=$(echo "$file5_data" | git hash-object --stdin) &&
+	f6id=$(echo "$file6_data" | git hash-object --stdin) &&
 	cat >expect <<-EOF &&
-	:000000 100755 0000000000000000000000000000000000000000 e74b7d465e52746be2b4bae983670711e6e66657 A	newdir/exec.sh
-	:000000 100644 0000000000000000000000000000000000000000 fcf778cda181eaa1cbc9e9ce3a2e15ee9f9fe791 A	newdir/interesting
+	:000000 100755 0000000000000000000000000000000000000000 $f6id A	newdir/exec.sh
+	:000000 100644 0000000000000000000000000000000000000000 $f5id A	newdir/interesting
 	EOF
 	git diff-tree -M -r branch^ branch >actual &&
 	compare_diff_raw expect actual
@@ -779,12 +784,13 @@ test_expect_success 'H: verify pack' '
 '
 
 test_expect_success 'H: validate old files removed, new files added' '
+	f4id=$(git rev-parse HEAD:file4) &&
 	cat >expect <<-EOF &&
-	:100755 000000 f1fb5da718392694d0076d677d6d0e364c79b0bc 0000000000000000000000000000000000000000 D	file2/newf
-	:100644 000000 7123f7f44e39be127c5eb701e5968176ee9d78b1 0000000000000000000000000000000000000000 D	file2/oldf
-	:100755 000000 85df50785d62d3b05ab03d9cbf7e4a0b49449730 0000000000000000000000000000000000000000 D	file4
-	:100644 100644 fcf778cda181eaa1cbc9e9ce3a2e15ee9f9fe791 fcf778cda181eaa1cbc9e9ce3a2e15ee9f9fe791 R100	newdir/interesting	h/e/l/lo
-	:100755 000000 e74b7d465e52746be2b4bae983670711e6e66657 0000000000000000000000000000000000000000 D	newdir/exec.sh
+	:100755 000000 $newf $zero D	file2/newf
+	:100644 000000 $oldf $zero D	file2/oldf
+	:100755 000000 $f4id $zero D	file4
+	:100644 100644 $f5id $f5id R100	newdir/interesting	h/e/l/lo
+	:100755 000000 $f6id $zero D	newdir/exec.sh
 	EOF
 	git diff-tree -M -r H^ H >actual &&
 	compare_diff_raw expect actual
@@ -935,14 +941,15 @@ test_expect_success 'L: verify internal tree sorting' '
 	INPUT_END
 
 	cat >expect <<-EXPECT_END &&
-	:100644 100644 4268632... 55d3a52... M	b.
-	:040000 040000 0ae5cac... 443c768... M	b
-	:100644 100644 4268632... 55d3a52... M	ba
+	:100644 100644 M	b.
+	:040000 040000 M	b
+	:100644 100644 M	ba
 	EXPECT_END
 
 	git fast-import <input &&
 	GIT_PRINT_SHA1_ELLIPSIS="yes" git diff-tree --abbrev --raw L^ L >output &&
-	test_cmp expect output
+	cut -d" " -f1,2,5 output >actual &&
+	test_cmp expect actual
 '
 
 test_expect_success 'L: nested tree copy does not corrupt deltas' '
@@ -1004,7 +1011,7 @@ test_expect_success 'M: rename file in same subdirectory' '
 	INPUT_END
 
 	cat >expect <<-EOF &&
-	:100755 100755 f1fb5da718392694d0076d677d6d0e364c79b0bc f1fb5da718392694d0076d677d6d0e364c79b0bc R100	file2/newf	file2/n.e.w.f
+	:100755 100755 $newf $newf R100	file2/newf	file2/n.e.w.f
 	EOF
 	git fast-import <input &&
 	git diff-tree -M -r M1^ M1 >actual &&
@@ -1025,7 +1032,7 @@ test_expect_success 'M: rename file to new subdirectory' '
 	INPUT_END
 
 	cat >expect <<-EOF &&
-	:100755 100755 f1fb5da718392694d0076d677d6d0e364c79b0bc f1fb5da718392694d0076d677d6d0e364c79b0bc R100	file2/newf	i/am/new/to/you
+	:100755 100755 $newf $newf R100	file2/newf	i/am/new/to/you
 	EOF
 	git fast-import <input &&
 	git diff-tree -M -r M2^ M2 >actual &&
@@ -1046,7 +1053,7 @@ test_expect_success 'M: rename subdirectory to new subdirectory' '
 	INPUT_END
 
 	cat >expect <<-EOF &&
-	:100755 100755 f1fb5da718392694d0076d677d6d0e364c79b0bc f1fb5da718392694d0076d677d6d0e364c79b0bc R100	i/am/new/to/you	other/sub/am/new/to/you
+	:100755 100755 $newf $newf R100	i/am/new/to/you	other/sub/am/new/to/you
 	EOF
 	git fast-import <input &&
 	git diff-tree -M -r M3^ M3 >actual &&
@@ -1067,11 +1074,11 @@ test_expect_success 'M: rename root to subdirectory' '
 	INPUT_END
 
 	cat >expect <<-EOF &&
-	:100644 100644 7123f7f44e39be127c5eb701e5968176ee9d78b1 7123f7f44e39be127c5eb701e5968176ee9d78b1 R100	file2/oldf	sub/file2/oldf
-	:100755 100755 85df50785d62d3b05ab03d9cbf7e4a0b49449730 85df50785d62d3b05ab03d9cbf7e4a0b49449730 R100	file4	sub/file4
-	:100755 100755 f1fb5da718392694d0076d677d6d0e364c79b0bc f1fb5da718392694d0076d677d6d0e364c79b0bc R100	i/am/new/to/you	sub/i/am/new/to/you
-	:100755 100755 e74b7d465e52746be2b4bae983670711e6e66657 e74b7d465e52746be2b4bae983670711e6e66657 R100	newdir/exec.sh	sub/newdir/exec.sh
-	:100644 100644 fcf778cda181eaa1cbc9e9ce3a2e15ee9f9fe791 fcf778cda181eaa1cbc9e9ce3a2e15ee9f9fe791 R100	newdir/interesting	sub/newdir/interesting
+	:100644 100644 $oldf $oldf R100	file2/oldf	sub/file2/oldf
+	:100755 100755 $f4id $f4id R100	file4	sub/file4
+	:100755 100755 $newf $newf R100	i/am/new/to/you	sub/i/am/new/to/you
+	:100755 100755 $f6id $f6id R100	newdir/exec.sh	sub/newdir/exec.sh
+	:100644 100644 $f5id $f5id R100	newdir/interesting	sub/newdir/interesting
 	EOF
 	git fast-import <input &&
 	git diff-tree -M -r M4^ M4 >actual &&
@@ -1097,7 +1104,7 @@ test_expect_success 'N: copy file in same subdirectory' '
 	INPUT_END
 
 	cat >expect <<-EOF &&
-	:100755 100755 f1fb5da718392694d0076d677d6d0e364c79b0bc f1fb5da718392694d0076d677d6d0e364c79b0bc C100	file2/newf	file2/n.e.w.f
+	:100755 100755 $newf $newf C100	file2/newf	file2/n.e.w.f
 	EOF
 	git fast-import <input &&
 	git diff-tree -C --find-copies-harder -r N1^ N1 >actual &&
@@ -1129,9 +1136,9 @@ test_expect_success 'N: copy then modify subdirectory' '
 	INPUT_END
 
 	cat >expect <<-EOF &&
-	:100644 100644 fcf778cda181eaa1cbc9e9ce3a2e15ee9f9fe791 fcf778cda181eaa1cbc9e9ce3a2e15ee9f9fe791 C100	newdir/interesting	file3/file5
-	:100755 100755 f1fb5da718392694d0076d677d6d0e364c79b0bc f1fb5da718392694d0076d677d6d0e364c79b0bc C100	file2/newf	file3/newf
-	:100644 100644 7123f7f44e39be127c5eb701e5968176ee9d78b1 7123f7f44e39be127c5eb701e5968176ee9d78b1 C100	file2/oldf	file3/oldf
+	:100644 100644 $f5id $f5id C100	newdir/interesting	file3/file5
+	:100755 100755 $newf $newf C100	file2/newf	file3/newf
+	:100644 100644 $oldf $oldf C100	file2/oldf	file3/oldf
 	EOF
 	git fast-import <input &&
 	git diff-tree -C --find-copies-harder -r N2^^ N2 >actual &&
@@ -1162,9 +1169,9 @@ test_expect_success 'N: copy dirty subdirectory' '
 '
 
 test_expect_success 'N: copy directory by id' '
-	cat >expect <<-\EOF &&
-	:100755 100755 f1fb5da718392694d0076d677d6d0e364c79b0bc f1fb5da718392694d0076d677d6d0e364c79b0bc C100	file2/newf	file3/newf
-	:100644 100644 7123f7f44e39be127c5eb701e5968176ee9d78b1 7123f7f44e39be127c5eb701e5968176ee9d78b1 C100	file2/oldf	file3/oldf
+	cat >expect <<-EOF &&
+	:100755 100755 $newf $newf C100	file2/newf	file3/newf
+	:100644 100644 $oldf $oldf C100	file2/oldf	file3/oldf
 	EOF
 	subdir=$(git rev-parse refs/heads/branch^0:file2) &&
 	cat >input <<-INPUT_END &&
@@ -1183,9 +1190,9 @@ test_expect_success 'N: copy directory by id' '
 '
 
 test_expect_success PIPE 'N: read and copy directory' '
-	cat >expect <<-\EOF &&
-	:100755 100755 f1fb5da718392694d0076d677d6d0e364c79b0bc f1fb5da718392694d0076d677d6d0e364c79b0bc C100	file2/newf	file3/newf
-	:100644 100644 7123f7f44e39be127c5eb701e5968176ee9d78b1 7123f7f44e39be127c5eb701e5968176ee9d78b1 C100	file2/oldf	file3/oldf
+	cat >expect <<-EOF &&
+	:100755 100755 $newf $newf C100	file2/newf	file3/newf
+	:100644 100644 $oldf $oldf C100	file2/oldf	file3/oldf
 	EOF
 	git update-ref -d refs/heads/N4 &&
 	rm -f backflow &&
@@ -1254,9 +1261,9 @@ test_expect_success PIPE 'N: empty directory reads as missing' '
 '
 
 test_expect_success 'N: copy root directory by tree hash' '
-	cat >expect <<-\EOF &&
-	:100755 000000 f1fb5da718392694d0076d677d6d0e364c79b0bc 0000000000000000000000000000000000000000 D	file3/newf
-	:100644 000000 7123f7f44e39be127c5eb701e5968176ee9d78b1 0000000000000000000000000000000000000000 D	file3/oldf
+	cat >expect <<-EOF &&
+	:100755 000000 $newf $zero D	file3/newf
+	:100644 000000 $oldf $zero D	file3/oldf
 	EOF
 	root=$(git rev-parse refs/heads/branch^0^{tree}) &&
 	cat >input <<-INPUT_END &&
@@ -1275,12 +1282,12 @@ test_expect_success 'N: copy root directory by tree hash' '
 '
 
 test_expect_success 'N: copy root by path' '
-	cat >expect <<-\EOF &&
-	:100755 100755 f1fb5da718392694d0076d677d6d0e364c79b0bc f1fb5da718392694d0076d677d6d0e364c79b0bc C100	file2/newf	oldroot/file2/newf
-	:100644 100644 7123f7f44e39be127c5eb701e5968176ee9d78b1 7123f7f44e39be127c5eb701e5968176ee9d78b1 C100	file2/oldf	oldroot/file2/oldf
-	:100755 100755 85df50785d62d3b05ab03d9cbf7e4a0b49449730 85df50785d62d3b05ab03d9cbf7e4a0b49449730 C100	file4	oldroot/file4
-	:100755 100755 e74b7d465e52746be2b4bae983670711e6e66657 e74b7d465e52746be2b4bae983670711e6e66657 C100	newdir/exec.sh	oldroot/newdir/exec.sh
-	:100644 100644 fcf778cda181eaa1cbc9e9ce3a2e15ee9f9fe791 fcf778cda181eaa1cbc9e9ce3a2e15ee9f9fe791 C100	newdir/interesting	oldroot/newdir/interesting
+	cat >expect <<-EOF &&
+	:100755 100755 $newf $newf C100	file2/newf	oldroot/file2/newf
+	:100644 100644 $oldf $oldf C100	file2/oldf	oldroot/file2/oldf
+	:100755 100755 $f4id $f4id C100	file4	oldroot/file4
+	:100755 100755 $f6id $f6id C100	newdir/exec.sh	oldroot/newdir/exec.sh
+	:100644 100644 $f5id $f5id C100	newdir/interesting	oldroot/newdir/interesting
 	EOF
 	cat >input <<-INPUT_END &&
 	commit refs/heads/N-copy-root-path
@@ -1340,10 +1347,10 @@ test_expect_success 'N: delete directory by copying' '
 '
 
 test_expect_success 'N: modify copied tree' '
-	cat >expect <<-\EOF &&
-	:100644 100644 fcf778cda181eaa1cbc9e9ce3a2e15ee9f9fe791 fcf778cda181eaa1cbc9e9ce3a2e15ee9f9fe791 C100	newdir/interesting	file3/file5
-	:100755 100755 f1fb5da718392694d0076d677d6d0e364c79b0bc f1fb5da718392694d0076d677d6d0e364c79b0bc C100	file2/newf	file3/newf
-	:100644 100644 7123f7f44e39be127c5eb701e5968176ee9d78b1 7123f7f44e39be127c5eb701e5968176ee9d78b1 C100	file2/oldf	file3/oldf
+	cat >expect <<-EOF &&
+	:100644 100644 $f5id $f5id C100	newdir/interesting	file3/file5
+	:100755 100755 $newf $newf C100	file2/newf	file3/newf
+	:100644 100644 $oldf $oldf C100	file2/oldf	file3/oldf
 	EOF
 	subdir=$(git rev-parse refs/heads/branch^0:file2) &&
 	cat >input <<-INPUT_END &&
@@ -3117,6 +3124,9 @@ test_expect_success 'U: initialize for U tests' '
 
 	INPUT_END
 
+	f7id=$(echo "blob 1" | git hash-object --stdin) &&
+	f8id=$(echo "sleep well" | git hash-object --stdin) &&
+	f9id=$(echo "au revoir" | git hash-object --stdin) &&
 	git fast-import <input
 '
 
@@ -3137,7 +3147,7 @@ test_expect_success 'U: filedelete file succeeds' '
 
 test_expect_success 'U: validate file delete result' '
 	cat >expect <<-EOF &&
-	:100644 000000 2907ebb4bf85d91bf0716bb3bd8a68ef48d6da76 0000000000000000000000000000000000000000 D	good/night.txt
+	:100644 000000 $f8id $ZERO_OID D	good/night.txt
 	EOF
 
 	git diff-tree -M -r U^1 U >actual &&
@@ -3162,7 +3172,7 @@ test_expect_success 'U: filedelete directory succeeds' '
 
 test_expect_success 'U: validate directory delete result' '
 	cat >expect <<-EOF &&
-	:100644 000000 69cb75792f55123d8389c156b0b41c2ff00ed507 0000000000000000000000000000000000000000 D	good/bye.txt
+	:100644 000000 $f9id $ZERO_OID D	good/bye.txt
 	EOF
 
 	git diff-tree -M -r U^1 U >actual &&
@@ -3187,7 +3197,7 @@ test_expect_success 'U: filedelete root succeeds' '
 
 test_expect_success 'U: validate root delete result' '
 	cat >expect <<-EOF &&
-	:100644 000000 c18147dc648481eeb65dc5e66628429a64843327 0000000000000000000000000000000000000000 D	hello.c
+	:100644 000000 $f7id $ZERO_OID D	hello.c
 	EOF
 
 	git diff-tree -M -r U^1 U >actual &&
brian m. carlson· Jul 28, 2020, 23:34 UTC · re: brian m. carlson · lore

[PATCH v5 30/39] builtin/verify-pack: implement an --object-format option

A recently added test in t5702 started using git verify-pack outside of a repository. While this poses no problems with SHA-1, with SHA-256 we implicitly rely on the setup of the repository to initialize our hash algorithm settings.

Since we're not in a repository here, we need to provide git verify-pack help to set things up properly. git index-pack already knows an --object-format option, so let's accept one as well and pass it down to our git index-pack invocation. Since we're now dynamically adjusting the elements in argv, let's switch to using struct argv_array to manage them. Finally, let's make t5702 pass the proper argument on down to its git verify-pack caller.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 builtin/verify-pack.c  | 23 +++++++++++++++--------
 t/t5702-protocol-v2.sh |  2 +-
 2 files changed, 16 insertions(+), 9 deletions(-)
Show changes to 2 files +16 −9

builtin/verify-pack.c, t/t5702-protocol-v2.sh

diff --git a/builtin/verify-pack.c b/builtin/verify-pack.c
index c2a1a5c504..3669a90263 100644
--- a/builtin/verify-pack.c
+++ b/builtin/verify-pack.c
@@ -7,21 +7,26 @@
 #define VERIFY_PACK_VERBOSE 01
 #define VERIFY_PACK_STAT_ONLY 02
 
-static int verify_one_pack(const char *path, unsigned int flags)
+static int verify_one_pack(const char *path, unsigned int flags, const char *hash_algo)
 {
 	struct child_process index_pack = CHILD_PROCESS_INIT;
-	const char *argv[] = {"index-pack", NULL, NULL, NULL };
+	struct argv_array *argv = &index_pack.args;
 	struct strbuf arg = STRBUF_INIT;
 	int verbose = flags & VERIFY_PACK_VERBOSE;
 	int stat_only = flags & VERIFY_PACK_STAT_ONLY;
 	int err;
 
+	argv_array_push(argv, "index-pack");
+
 	if (stat_only)
-		argv[1] = "--verify-stat-only";
+		argv_array_push(argv, "--verify-stat-only");
 	else if (verbose)
-		argv[1] = "--verify-stat";
+		argv_array_push(argv, "--verify-stat");
 	else
-		argv[1] = "--verify";
+		argv_array_push(argv, "--verify");
+
+	if (hash_algo)
+		argv_array_pushf(argv, "--object-format=%s", hash_algo);
 
 	/*
 	 * In addition to "foo.pack" we accept "foo.idx" and "foo";
@@ -31,9 +36,8 @@ static int verify_one_pack(const char *path, unsigned int flags)
 	if (strbuf_strip_suffix(&arg, ".idx") ||
 	    !ends_with(arg.buf, ".pack"))
 		strbuf_addstr(&arg, ".pack");
-	argv[2] = arg.buf;
+	argv_array_push(argv, arg.buf);
 
-	index_pack.argv = argv;
 	index_pack.git_cmd = 1;
 
 	err = run_command(&index_pack);
@@ -60,12 +64,15 @@ int cmd_verify_pack(int argc, const char **argv, const char *prefix)
 {
 	int err = 0;
 	unsigned int flags = 0;
+	const char *object_format = NULL;
 	int i;
 	const struct option verify_pack_options[] = {
 		OPT_BIT('v', "verbose", &flags, N_("verbose"),
 			VERIFY_PACK_VERBOSE),
 		OPT_BIT('s', "stat-only", &flags, N_("show statistics only"),
 			VERIFY_PACK_STAT_ONLY),
+		OPT_STRING(0, "object-format", &object_format, N_("hash"),
+			   N_("specify the hash algorithm to use")),
 		OPT_END()
 	};
 
@@ -75,7 +82,7 @@ int cmd_verify_pack(int argc, const char **argv, const char *prefix)
 	if (argc < 1)
 		usage_with_options(verify_pack_usage, verify_pack_options);
 	for (i = 0; i < argc; i++) {
-		if (verify_one_pack(argv[i], flags))
+		if (verify_one_pack(argv[i], flags, object_format))
 			err = 1;
 	}
 
diff --git a/t/t5702-protocol-v2.sh b/t/t5702-protocol-v2.sh
index 1b54c35b01..7fc22171e7 100755
--- a/t/t5702-protocol-v2.sh
+++ b/t/t5702-protocol-v2.sh
@@ -829,7 +829,7 @@ test_expect_success 'part of packfile response provided as URI' '
 	# Ensure that my-blob and other-blob are in separate packfiles.
 	for idx in http_child/.git/objects/pack/*.idx
 	do
-		git verify-pack --verbose $idx >out &&
+		git verify-pack --object-format=$(test_oid algo) --verbose $idx >out &&
 		{
 			grep "^[0-9a-f]\{16,\} " out || :
 		} >out.objectlist &&
brian m. carlson· Jul 28, 2020, 23:34 UTC · re: brian m. carlson · lore

[PATCH v5 33/39] Enable SHA-256 support by default

Now that the we have a complete SHA-256 implementation in Git, let's enable it so people can use it. Remove the ENABLE_SHA256 define constant everywhere it's used. Add tests for initializing a repository with SHA-256.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 builtin/init-db.c |  5 -----
 config.mak.dev    |  2 --
 repository.c      |  4 ----
 t/t0001-init.sh   | 33 +++++++++++++++++++++++++++++++++
 4 files changed, 33 insertions(+), 11 deletions(-)
Show changes to 4 files +33 −11

builtin/init-db.c, config.mak.dev, repository.c, t/t0001-init.sh

diff --git a/builtin/init-db.c b/builtin/init-db.c
index cee64823cb..f70076d38e 100644
--- a/builtin/init-db.c
+++ b/builtin/init-db.c
@@ -183,11 +183,6 @@ void initialize_repository_version(int hash_algo)
 	char repo_version_string[10];
 	int repo_version = GIT_REPO_VERSION;
 
-#ifndef ENABLE_SHA256
-	if (hash_algo != GIT_HASH_SHA1)
-		die(_("The hash algorithm %s is not supported in this build."), hash_algos[hash_algo].name);
-#endif
-
 	if (hash_algo != GIT_HASH_SHA1)
 		repo_version = GIT_REPO_VERSION_READ;
 
diff --git a/config.mak.dev b/config.mak.dev
index cd4a82a9eb..89b218d11a 100644
--- a/config.mak.dev
+++ b/config.mak.dev
@@ -16,8 +16,6 @@ DEVELOPER_CFLAGS += -Wstrict-prototypes
 DEVELOPER_CFLAGS += -Wunused
 DEVELOPER_CFLAGS += -Wvla
 
-DEVELOPER_CFLAGS += -DENABLE_SHA256
-
 ifndef COMPILER_FEATURES
 COMPILER_FEATURES := $(shell ./detect-compiler $(CC))
 endif
diff --git a/repository.c b/repository.c
index 6f7f6f002b..a4174ddb06 100644
--- a/repository.c
+++ b/repository.c
@@ -89,10 +89,6 @@ void repo_set_gitdir(struct repository *repo,
 void repo_set_hash_algo(struct repository *repo, int hash_algo)
 {
 	repo->hash_algo = &hash_algos[hash_algo];
-#ifndef ENABLE_SHA256
-	if (hash_algo != GIT_HASH_SHA1)
-		die(_("The hash algorithm %s is not supported in this build."), repo->hash_algo->name);
-#endif
 }
 
 /*
diff --git a/t/t0001-init.sh b/t/t0001-init.sh
index 6d2467995e..d71d4c7238 100755
--- a/t/t0001-init.sh
+++ b/t/t0001-init.sh
@@ -441,6 +441,39 @@ test_expect_success 're-init from a linked worktree' '
 	)
 '
 
+test_expect_success 'init honors GIT_DEFAULT_HASH' '
+	GIT_DEFAULT_HASH=sha1 git init sha1 &&
+	git -C sha1 rev-parse --show-object-format >actual &&
+	echo sha1 >expected &&
+	test_cmp expected actual &&
+	GIT_DEFAULT_HASH=sha256 git init sha256 &&
+	git -C sha256 rev-parse --show-object-format >actual &&
+	echo sha256 >expected &&
+	test_cmp expected actual
+'
+
+test_expect_success 'init honors --object-format' '
+	git init --object-format=sha1 explicit-sha1 &&
+	git -C explicit-sha1 rev-parse --show-object-format >actual &&
+	echo sha1 >expected &&
+	test_cmp expected actual &&
+	git init --object-format=sha256 explicit-sha256 &&
+	git -C explicit-sha256 rev-parse --show-object-format >actual &&
+	echo sha256 >expected &&
+	test_cmp expected actual
+'
+
+test_expect_success 'extensions.objectFormat is not allowed with repo version 0' '
+	git init --object-format=sha256 explicit-v0 &&
+	git -C explicit-v0 config core.repositoryformatversion 0 &&
+	test_must_fail git -C explicit-v0 rev-parse --show-object-format
+'
+
+test_expect_success 'init rejects attempts to initialize with different hash' '
+	test_must_fail git -C sha1 init --object-format=sha256 &&
+	test_must_fail git -C sha256 init --object-format=sha1
+'
+
 test_expect_success MINGW 'core.hidedotfiles = false' '
 	git config --global core.hidedotfiles false &&
 	rm -rf newdir &&
Eric Sunshine· Jul 29, 2020, 02:57 UTC · re: brian m. carlson · lore

Re: [PATCH v5 33/39] Enable SHA-256 support by default

On Tue, Jul 28, 2020 at 7:36 PM brian m. carlson <sandals@crustytoothpaste.net> wrote:

> Subject: Enable SHA-256 support by default
If you happen to re-roll, perhaps: s/Enable/enable/
> Now that the we have a complete SHA-256 implementation in Git, let's
Still wonky grammar here (mentioned previously): "Now that the we have..."
Show 5 quoted lines
> enable it so people can use it.  Remove the ENABLE_SHA256 define
> constant everywhere it's used.  Add tests for initializing a repository
> with SHA-256.
>
> Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
brian m. carlson· Jul 28, 2020, 23:34 UTC · re: brian m. carlson · lore

[PATCH v5 28/39] t0410: mark test with SHA1 prerequisite

These tests try to check that we behave properly if we encounter a repository with version 0 but an extension. This is a laudable goal, but the test cannot work with SHA-256, since SHA-256 repositories always have an existing extension and are never version 0.

Add a SHA1 prerequisite to these tests.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t0410-partial-clone.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
Show changes to t/t0410-partial-clone.sh +2 −2
diff --git a/t/t0410-partial-clone.sh b/t/t0410-partial-clone.sh
index 6aa0f313bd..a5ebdf9ff3 100755
--- a/t/t0410-partial-clone.sh
+++ b/t/t0410-partial-clone.sh
@@ -42,7 +42,7 @@ test_expect_success 'convert shallow clone to partial clone' '
 	test_cmp_config -C client 1 core.repositoryformatversion
 '
 
-test_expect_success 'convert to partial clone with noop extension' '
+test_expect_success SHA1 'convert to partial clone with noop extension' '
 	rm -fr server client &&
 	test_create_repo server &&
 	test_commit -C server my_commit 1 &&
@@ -53,7 +53,7 @@ test_expect_success 'convert to partial clone with noop extension' '
 	git -C client fetch --unshallow --filter="blob:none"
 '
 
-test_expect_success 'converting to partial clone fails with unrecognized extension' '
+test_expect_success SHA1 'converting to partial clone fails with unrecognized extension' '
 	rm -fr server client &&
 	test_create_repo server &&
 	test_commit -C server my_commit 1 &&
brian m. carlson· Jul 28, 2020, 23:34 UTC · re: brian m. carlson · lore

[PATCH v5 08/39] t6500: specify test values for SHA-256

In this test, we want to produce several blobs whose first two hex characters are "17", since we look at this object directory as a proxy for how many loose objects there are before we need to GC. Use test_oid_cache to specify strings that will hash to the right values when turned into blobs.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t6500-gc.sh | 27 ++++++++++++++++++++++-----
 1 file changed, 22 insertions(+), 5 deletions(-)
Show changes to t/t6500-gc.sh +22 −5
diff --git a/t/t6500-gc.sh b/t/t6500-gc.sh
index 0a69a67117..4a3b8f48ac 100755
--- a/t/t6500-gc.sh
+++ b/t/t6500-gc.sh
@@ -10,7 +10,24 @@ test_expect_success 'setup' '
 	# do not let the amount of physical memory affects gc
 	# behavior, make sure we always pack everything to one pack by
 	# default
-	git config gc.bigPackThreshold 2g
+	git config gc.bigPackThreshold 2g &&
+
+	# These are simply values which, when hashed as a blob with a newline,
+	# produce a hash where the first byte is 0x17 in their respective
+	# algorithms.
+	test_oid_cache <<-EOF
+	obj1 sha1:263
+	obj1 sha256:34
+
+	obj2 sha1:410
+	obj2 sha256:174
+
+	obj3 sha1:523
+	obj3 sha256:313
+
+	obj4 sha1:790
+	obj4 sha256:481
+	EOF
 '
 
 test_expect_success 'gc empty repository' '
@@ -85,13 +102,13 @@ test_expect_success 'auto gc with too many loose objects does not attempt to cre
 	# We need to create two object whose sha1s start with 17
 	# since this is what git gc counts.  As it happens, these
 	# two blobs will do so.
-	test_commit 263 &&
-	test_commit 410 &&
+	test_commit "$(test_oid obj1)" &&
+	test_commit "$(test_oid obj2)" &&
 	# Our first gc will create a pack; our second will create a second pack
 	git gc --auto &&
 	ls .git/objects/pack | sort >existing_packs &&
-	test_commit 523 &&
-	test_commit 790 &&
+	test_commit "$(test_oid obj3)" &&
+	test_commit "$(test_oid obj4)" &&
 
 	git gc --auto 2>err &&
 	test_i18ngrep ! "^warning:" err &&
brian m. carlson· Jul 28, 2020, 23:34 UTC · re: brian m. carlson · lore

[PATCH v5 18/39] t8002: make hash size independent

Compute the length of an object ID instead of hard-coding 40-based values.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t8002-blame.sh | 18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)
Show changes to t/t8002-blame.sh +11 −7
diff --git a/t/t8002-blame.sh b/t/t8002-blame.sh
index eea048e52c..c3b70b025e 100755
--- a/t/t8002-blame.sh
+++ b/t/t8002-blame.sh
@@ -6,6 +6,10 @@ test_description='git blame'
 PROG='git blame -c'
 . "$TEST_DIRECTORY"/annotate-tests.sh
 
+test_expect_success 'setup' '
+	hexsz=$(test_oid hexsz)
+'
+
 test_expect_success 'blame untracked file in empty repo' '
 	>untracked &&
 	test_must_fail git blame untracked
@@ -105,17 +109,17 @@ test_expect_success 'blame --abbrev=<n> works' '
 '
 
 test_expect_success 'blame -l aligns regular and boundary commits' '
-	check_abbrev 40 -l HEAD &&
-	check_abbrev 39 -l ^HEAD
+	check_abbrev $hexsz         -l HEAD &&
+	check_abbrev $((hexsz - 1)) -l ^HEAD
 '
 
-test_expect_success 'blame --abbrev=40 behaves like -l' '
-	check_abbrev 40 --abbrev=40 HEAD &&
-	check_abbrev 39 --abbrev=40 ^HEAD
+test_expect_success 'blame --abbrev with full length behaves like -l' '
+	check_abbrev $hexsz         --abbrev=$hexsz HEAD &&
+	check_abbrev $((hexsz - 1)) --abbrev=$hexsz ^HEAD
 '
 
-test_expect_success '--no-abbrev works like --abbrev=40' '
-	check_abbrev 40 --no-abbrev
+test_expect_success '--no-abbrev works like --abbrev with full length' '
+	check_abbrev $hexsz --no-abbrev
 '
 
 test_expect_success '--exclude-promisor-objects does not BUG-crash' '
Eric Sunshine· Jul 29, 2020, 02:24 UTC · re: brian m. carlson · lore

Re: [PATCH v5 18/39] t8002: make hash size independent

On Tue, Jul 28, 2020 at 7:35 PM brian m. carlson <sandals@crustytoothpaste.net> wrote:

Show 10 quoted lines
> Compute the length of an object ID instead of hard-coding 40-based
> values.
>
> Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
> ---
> diff --git a/t/t8002-blame.sh b/t/t8002-blame.sh
> @@ -6,6 +6,10 @@ test_description='git blame'
> +test_expect_success 'setup' '
> +       hexsz=$(test_oid hexsz)
> +'

In the previous version of this series, the "setup" test invoked test_oid_init() as its very first step. This version doesn't. As a reviewer, I was caught off-guard by this unexpected and unexplained difference between versions. The script works fine without test_oid_init() anyhow since test-lib.sh invokes test_oid_init(), so the test_oid_init() call introduced here by the previous version was redundant.

Some of the patches in this series add test_oid_init() calls to their "setup" tests, while others don't, which makes for a somewhat confusing impression as one reads the series. In general, it would be nice for the patches to paint a consistent picture (i.e either uniformly employ test_oid_init() or don't), however, I would not want to see a re-roll just for that. Also, since the final patch of the series ends up removing all those test_oid_init() calls anyhow, it's all straightened out in the end.

brian m. carlson· Jul 29, 2020, 22:31 UTC · re: Eric Sunshine · lore

Re: [PATCH v5 18/39] t8002: make hash size independent

On 2020-07-29 at 02:24:42, Eric Sunshine wrote:
Show 16 quoted lines
> In the previous version of this series, the "setup" test  invoked
> test_oid_init() as its very first step. This version doesn't. As a
> reviewer, I was caught off-guard by this unexpected and unexplained
> difference between versions. The script works fine without
> test_oid_init() anyhow since test-lib.sh invokes test_oid_init(), so
> the test_oid_init() call introduced here by the previous version was
> redundant.
> 
> Some of the patches in this series add test_oid_init() calls to their
> "setup" tests, while others don't, which makes for a somewhat
> confusing impression as one reads the series. In general, it would be
> nice for the patches to paint a consistent picture (i.e either
> uniformly employ test_oid_init() or don't), however, I would not want
> to see a re-roll just for that. Also, since the final patch of the
> series ends up removing all those test_oid_init() calls anyhow, it's
> all straightened out in the end.

Good point. I'll try to remove them from the existing tests which add them in the rest of the series.

-- 
brian m. carlson: Houston, Texas, US
brian m. carlson· Jul 28, 2020, 23:34 UTC · re: brian m. carlson · lore

[PATCH v5 19/39] t8003: make hash size independent

One assertion in this test invokes git with core.abbrev set to "40". Since we're expecting the full hash length, use test_oid to look up the full hash length for the hash in use.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t8003-blame-corner-cases.sh | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
Show changes to t/t8003-blame-corner-cases.sh +2 −1
diff --git a/t/t8003-blame-corner-cases.sh b/t/t8003-blame-corner-cases.sh
index 9130b887d2..ebe79aa4ea 100755
--- a/t/t8003-blame-corner-cases.sh
+++ b/t/t8003-blame-corner-cases.sh
@@ -6,6 +6,7 @@ test_description='git blame corner cases'
 pick_fc='s/^[0-9a-f^]* *\([^ ]*\) *(\([^ ]*\) .*/\1-\2/'
 
 test_expect_success setup '
+	test_oid_init &&
 
 	echo A A A A A >one &&
 	echo B B B B B >two &&
@@ -306,7 +307,7 @@ test_expect_success 'blame coalesce' '
 	$oid 1) ABC
 	$oid 2) DEF
 	EOF
-	git -c core.abbrev=40 blame -s giraffe >actual &&
+	git -c core.abbrev=$(test_oid hexsz) blame -s giraffe >actual &&
 	test_cmp expect actual
 '
 
brian m. carlson· Jul 28, 2020, 23:34 UTC · re: brian m. carlson · lore

[PATCH v5 23/39] t9301: make hash size independent

Instead of using a hard-coded all-zeros object ID, use $ZERO_OID. Compute the length of the object IDs in use and use this instead of hard-coding the constant 40.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t9301-fast-import-notes.sh | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)
Show changes to t/t9301-fast-import-notes.sh +7 −7
diff --git a/t/t9301-fast-import-notes.sh b/t/t9301-fast-import-notes.sh
index ca223dca98..9d1aaac980 100755
--- a/t/t9301-fast-import-notes.sh
+++ b/t/t9301-fast-import-notes.sh
@@ -71,7 +71,7 @@ EOF
 INPUT_END
 
 test_expect_success 'set up master branch' '
-
+	test_oid_init &&
 	git fast-import <input &&
 	git whatchanged master
 '
@@ -470,12 +470,13 @@ test_expect_success 'add lots of commits and notes' '
 '
 
 test_expect_success 'verify that lots of notes trigger a fanout scheme' '
+	hexsz=$(test_oid hexsz) &&
 
 	# None of the entries in the top-level notes tree should be a full SHA1
 	git ls-tree --name-only refs/notes/many_notes |
 	while read path
 	do
-		if test $(expr length "$path") -ge 40
+		if test $(expr length "$path") -ge $hexsz
 		then
 			return 1
 		fi
@@ -518,7 +519,7 @@ test_expect_success 'verify that importing a notes tree respects the fanout sche
 	git ls-tree --name-only refs/notes/other_notes |
 	while read path
 	do
-		if test $(expr length "$path") -ge 40
+		if test $(expr length "$path") -ge $hexsz
 		then
 			return 1
 		fi
@@ -593,7 +594,7 @@ test_expect_success 'verify that changing notes respect existing fanout' '
 	git ls-tree --name-only refs/notes/many_notes |
 	while read path
 	do
-		if test $(expr length "$path") -ge 40
+		if test $(expr length "$path") -ge $hexsz
 		then
 			return 1
 		fi
@@ -616,7 +617,7 @@ i=$(($num_commits - $remaining_notes))
 for sha1 in $(git rev-list -n $i refs/heads/many_commits)
 do
 	cat >>input <<INPUT_END
-N 0000000000000000000000000000000000000000 $sha1
+N $ZERO_OID $sha1
 INPUT_END
 done
 
@@ -646,7 +647,6 @@ test_expect_success 'remove lots of notes' '
 '
 
 test_expect_success 'verify that removing notes trigger fanout consolidation' '
-
 	# All entries in the top-level notes tree should be a full SHA1
 	git ls-tree --name-only -r refs/notes/many_notes |
 	while read path
@@ -656,7 +656,7 @@ test_expect_success 'verify that removing notes trigger fanout consolidation' '
 		test "$path" = "deadbeef" && continue
 		test "$path" = "de/adbeef" && continue
 
-		if test $(expr length "$path") -ne 40
+		if test $(expr length "$path") -ne $hexsz
 		then
 			return 1
 		fi
brian m. carlson· Jul 28, 2020, 23:34 UTC · re: brian m. carlson · lore

[PATCH v5 16/39] t7506: avoid checking for SHA-1-specific constants

Adjust the test to sanitize the diffs and strip out object IDs from them, as it does for other object IDs, since we are not interested in the particular values used.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t7506-status-submodule.sh | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)
Show changes to t/t7506-status-submodule.sh +8 −4
diff --git a/t/t7506-status-submodule.sh b/t/t7506-status-submodule.sh
index 08629a6e70..3fcb44767f 100755
--- a/t/t7506-status-submodule.sh
+++ b/t/t7506-status-submodule.sh
@@ -22,6 +22,10 @@ sanitize_output () {
 	mv output2 output
 }
 
+sanitize_diff () {
+	sed -e "/^index [0-9a-f,]*\.\.[0-9a-f]*/d" "$1"
+}
+
 
 test_expect_success 'setup' '
 	test_create_repo_with_commit sub &&
@@ -269,7 +273,6 @@ short_sha1_merge_sub1=$(cd sub1 && git rev-parse --short HEAD)
 short_sha1_merge_sub2=$(cd sub2 && git rev-parse --short HEAD)
 cat >diff_expect <<\EOF
 diff --cc .gitmodules
-index badaa4c,44f999a..0000000
 --- a/.gitmodules
 +++ b/.gitmodules
 @@@ -1,3 -1,3 +1,9 @@@
@@ -286,7 +289,6 @@ EOF
 
 cat >diff_submodule_expect <<\EOF
 diff --cc .gitmodules
-index badaa4c,44f999a..0000000
 --- a/.gitmodules
 +++ b/.gitmodules
 @@@ -1,3 -1,3 +1,9 @@@
@@ -306,7 +308,8 @@ test_expect_success 'diff with merge conflict in .gitmodules' '
 		cd super &&
 		git diff >../diff_actual 2>&1
 	) &&
-	test_cmp diff_expect diff_actual
+	sanitize_diff diff_actual >diff_sanitized &&
+	test_cmp diff_expect diff_sanitized
 '
 
 test_expect_success 'diff --submodule with merge conflict in .gitmodules' '
@@ -314,7 +317,8 @@ test_expect_success 'diff --submodule with merge conflict in .gitmodules' '
 		cd super &&
 		git diff --submodule >../diff_submodule_actual 2>&1
 	) &&
-	test_cmp diff_submodule_expect diff_submodule_actual
+	sanitize_diff diff_submodule_actual >diff_sanitized &&
+	test_cmp diff_submodule_expect diff_sanitized
 '
 
 # We'll setup different cases for further testing:
brian m. carlson· Jul 29, 2020, 23:13 UTC · re: brian m. carlson · lore

[PATCH v6 00/39] SHA-256, part 3/3

Changes from v5:
* Remove useless test_oid_init invocations throughout the series.
* Fix a commit message for grammar and style.
* Fix a typo in a shell case statement.
* Fix http-fetch to allow parsing options before failing on a missing
  git directory.
Changes from v4:
* Fix several commit message to fix typos and inaccuracies.
* Remove useless touch statement in tests.
* Adjust several tests to introduce less dependence on previous test
  assertions other than the setup ones.
* Loosen the regexp for object IDs in a Perl script in a test.
* Ensure help functionality works for git http-fetch.
* Switch http-fetch.c to use the already existing struct argv_array
  instance.
* Avoid a memory leak, albeit a minor one.
* Improve error message for unknown bundle hash algorithm.
* Hoist a strbuf_rtrim in the bundle parsing code to simplify things.
* Specify a minimum version instead of a default version in the bundle
  code.
* Remove a useless line in tests.
* Avoid invoking git hash-object needlessly for the SHA1 prerequisite.
* Be explicit about which algorithms we're testing in CI.
* Properly capitalize "core.repositoryFormatVersion" in documentation.
Changes from v2:
* Move bundle signature parsing into a function.
* Improve parsing of bundle capabilities.
* Improve t5510 to select only the interesting lines with sed.
* Use cat and a heredoc instead of multiple echos.
* Use test_i18ngrep where appropriate.
* Reject extensions.objectFormat for repo v0.
* Update documentation accordingly.
Changes from v1:
* Add a patch to add a bundle v3 format with capabilities that include
  the hash algorithm.
* Remove bundle detection of hash algorithm using hash algorithm length.
* Fix typo in the extensions.objectFormat documentation.
* Document that extensions.objectFormat is ignored for repo v0.
* Use test instead of brackets.
* Use a command group instead of a subshell for efficiency.
* Drop the bizarre test_might_fail, since it isn't needed (and is
  bizarre and wrong).
* Write "SHA-256" instead of "NewHash" in commit messages.
* Split SHA-256 CI tests to an independent run and avoid running a third
  test run to a single CI job.
Range-diff below.
Johannes Schindelin (1):
  t3404: prepare 'short SHA-1 collision' tests for SHA-256
brian m. carlson (38):
  t: make test-bloom initialize repository
  t1001: use $ZERO_OID
  t3305: make hash agnostic
  t6100: make hash size independent
  t6101: make hash size independent
  t6301: make hash size independent
  t6500: specify test values for SHA-256
  t6501: avoid hard-coded objects
  t7003: compute appropriate length constant
  t7063: make hash size independent
  t7201: abstract away SHA-1-specific constants
  t7102: abstract away SHA-1-specific constants
  t7400: make hash size independent
  t7405: make hash size independent
  t7506: avoid checking for SHA-1-specific constants
  t7508: use $ZERO_OID instead of hard-coded constant
  t8002: make hash size independent
  t8003: make hash size independent
  t8011: make hash size independent
  t9300: abstract away SHA-1-specific constants
  t9300: use $ZERO_OID instead of hard-coded object ID
  t9301: make hash size independent
  t9350: make hash size independent
  t9500: ensure that algorithm info is preserved in config
  t9700: make hash size independent
  t5308: make test work with SHA-256
  t0410: mark test with SHA1 prerequisite
  http-fetch: set up git directory before parsing pack hashes
  builtin/verify-pack: implement an --object-format option
  bundle: add new version for use with SHA-256
  setup: add support for reading extensions.objectformat
  repository: enable SHA-256 support by default
  t: add test_oid option to select hash algorithm
  t: allow testing different hash algorithms via environment
  t: make SHA1 prerequisite depend on default hash
  ci: run tests with SHA-256
  docs: add documentation for extensions.objectFormat
  t: remove test_oid_init in tests
 Documentation/config.txt                  |   2 +
 Documentation/config/extensions.txt       |   8 ++
 Documentation/git-bundle.txt              |   9 +-
 Documentation/technical/bundle-format.txt |  30 ++++-
 builtin/bundle.c                          |   5 +-
 builtin/init-db.c                         |   5 -
 builtin/verify-pack.c                     |  23 ++--
 bundle.c                                  |  83 +++++++++---
 bundle.h                                  |   4 +-
 ci/run-build-and-tests.sh                 |   6 +
 config.mak.dev                            |   2 -
 http-fetch.c                              |   6 +-
 repository.c                              |   4 -
 setup.c                                   |  16 ++-
 t/helper/test-bloom.c                     |   2 +
 t/lib-pack.sh                             |  11 +-
 t/lib-submodule-update.sh                 |   1 -
 t/t0000-basic.sh                          |  15 ++-
 t/t0001-init.sh                           |  33 +++++
 t/t0410-partial-clone.sh                  |   4 +-
 t/t1006-cat-file.sh                       |   2 -
 t/t1050-large.sh                          |   1 -
 t/t1091-sparse-checkout-builtin.sh        |   4 +-
 t/t1410-reflog.sh                         |   1 -
 t/t1450-fsck.sh                           |   1 -
 t/t1500-rev-parse.sh                      |   1 -
 t/t3305-notes-fanout.sh                   |   2 +-
 t/t3308-notes-merge.sh                    |   1 -
 t/t3404-rebase-interactive.sh             |  49 +++++--
 t/t3600-rm.sh                             |   1 -
 t/t3800-mktag.sh                          |   1 -
 t/t4002-diff-basic.sh                     |   2 -
 t/t4027-diff-submodule.sh                 |   1 -
 t/t4134-apply-submodule.sh                |   1 -
 t/t4200-rerere.sh                         |   1 -
 t/t4211-line-log.sh                       |   1 -
 t/t5300-pack-object.sh                    |   3 +-
 t/t5302-pack-index.sh                     |   1 -
 t/t5308-pack-detect-duplicates.sh         |  20 +--
 t/t5313-pack-bounds-checks.sh             |   1 -
 t/t5318-commit-graph.sh                   |   3 +-
 t/t5319-multi-pack-index.sh               |   1 -
 t/t5324-split-commit-graph.sh             |   1 -
 t/t5504-fetch-receive-strict.sh           |   1 -
 t/t5510-fetch.sh                          |  16 ++-
 t/t5530-upload-pack-error.sh              |   1 -
 t/t5562-http-backend-content-length.sh    |   1 -
 t/t5607-clone-bundle.sh                   |  31 +++++
 t/t5702-protocol-v2.sh                    |   3 +-
 t/t5703-upload-pack-ref-in-want.sh        |   1 -
 t/t6006-rev-list-format.sh                |   1 -
 t/t6100-rev-list-in-order.sh              |   4 +-
 t/t6101-rev-parse-parents.sh              |   2 +-
 t/t6301-for-each-ref-errors.sh            |   2 +-
 t/t6500-gc.sh                             |  27 +++-
 t/t6501-freshen-objects.sh                |  14 +-
 t/t7003-filter-branch.sh                  |   3 +-
 t/t7063-status-untracked-cache.sh         | 150 ++++++++++++----------
 t/t7102-reset.sh                          |  93 ++++++++------
 t/t7201-co.sh                             |   6 +-
 t/t7400-submodule-basic.sh                |  26 ++--
 t/t7405-submodule-merge.sh                |   4 +-
 t/t7506-status-submodule.sh               |  12 +-
 t/t7508-status.sh                         |   2 +-
 t/t8002-blame.sh                          |  18 ++-
 t/t8003-blame-corner-cases.sh             |   3 +-
 t/t8011-blame-split-file.sh               |   2 +-
 t/t9300-fast-import.sh                    | 118 +++++++++--------
 t/t9301-fast-import-notes.sh              |  12 +-
 t/t9350-fast-export.sh                    |  14 +-
 t/t9500-gitweb-standalone-no-errors.sh    |  22 +++-
 t/t9700/test.pl                           |   6 +-
 t/test-lib-functions.sh                   |  16 ++-
 t/test-lib.sh                             |   9 +-
 74 files changed, 638 insertions(+), 351 deletions(-)
 create mode 100644 Documentation/config/extensions.txt
Diff-intervalle contre v5 :
 1:  2d720a6beb =  1:  fb53834bd3 t: make test-bloom initialize repository
 2:  530704736a =  2:  8170f55d65 t1001: use $ZERO_OID
 3:  33e7822d0e !  3:  94b585296d t3305: make hash agnostic
    @@ t/t3305-notes-fanout.sh: test_description='Test that adding/removing many notes
      	echo $path | grep -q "^\([0-9a-f]\{2\}/\)\{$fanout\}[0-9a-f]\{$after_last_slash\}$"
      }
      
    -@@ t/t3305-notes-fanout.sh: all_notes_have_fanout() {
    - }
    - 
    - test_expect_success 'tweak test environment' '
    -+	test_oid_init &&
    - 	git checkout -b nondeterminism &&
    - 	test_commit A &&
    - 	git checkout --orphan with_notes;
 4:  99081937cf !  4:  82b373b830 t3404: prepare 'short SHA-1 collision' tests for SHA-256
    @@ Commit message
         Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
     
      ## t/t3404-rebase-interactive.sh ##
    -@@ t/t3404-rebase-interactive.sh: Initial setup:
    - . "$TEST_DIRECTORY"/lib-rebase.sh
    - 
    - test_expect_success 'setup' '
    -+	test_oid_init &&
    - 	test_commit A file1 &&
    - 	test_commit B file1 &&
    - 	test_commit C file2 &&
     @@ t/t3404-rebase-interactive.sh: test_expect_success 'rebase -i error on commits with \ in message' '
      	test_expect_code 1 grep  "	emp" error
      '
 5:  3cfa8bba75 =  5:  adc32c2903 t6100: make hash size independent
 6:  9a7de5790c =  6:  40db5bb392 t6101: make hash size independent
 7:  76942eec3d !  7:  d4055790f3 t6301: make hash size independent
    @@ Commit message
     
         Instead of hard-coding a fixed length example object ID in the test,
         compute one using the translation tables.  Move a variable into the
    -    setup block so that we can ensure the exit status of test_oid_init is
    +    setup block so that we can ensure the exit status of test_oid is
         checked.
     
         Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
    @@ t/t6301-for-each-ref-errors.sh: test_description='for-each-ref errors for broken
     -MISSING=abababababababababababababababababababab
      
      test_expect_success setup '
    -+	test_oid_init &&
     +	MISSING=$(test_oid deadbeef) &&
      	git commit --allow-empty -m "Initial" &&
      	git tag testtag &&
 8:  b30a8a6736 =  8:  a9578382ab t6500: specify test values for SHA-256
 9:  c3d68dcf1b !  9:  b022dda379 t6501: avoid hard-coded objects
    @@ Commit message
         Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
     
      ## t/t6501-freshen-objects.sh ##
    -@@ t/t6501-freshen-objects.sh: maybe_repack () {
    - 	fi
    - }
    - 
    -+test_expect_success 'setup' '
    -+	test_oid_init
    -+'
    -+
    - for repack in '' true; do
    - 	title=${repack:+repack}
    - 	title=${title:-loose}
     @@ t/t6501-freshen-objects.sh: for repack in '' true; do
      done
      
10:  865662f5c5 ! 10:  d9eda002b9 t7003: compute appropriate length constant
    @@ Commit message
         Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
     
      ## t/t7003-filter-branch.sh ##
    -@@ t/t7003-filter-branch.sh: test_description='git filter-branch'
    - . "$TEST_DIRECTORY/lib-gpg.sh"
    - 
    - test_expect_success 'setup' '
    -+	test_oid_init &&
    - 	test_commit A &&
    - 	GIT_COMMITTER_DATE="@0 +0000" GIT_AUTHOR_DATE="@0 +0000" &&
    - 	test_commit --notick B &&
     @@ t/t7003-filter-branch.sh: test_expect_success 'rewrite submodule with another content' '
      '
      
11:  1a7bd71fb4 = 11:  02da9e5630 t7063: make hash size independent
12:  55c545cf1f = 12:  4f567ee443 t7201: abstract away SHA-1-specific constants
13:  9875576b08 = 13:  cec9eca4a3 t7102: abstract away SHA-1-specific constants
14:  d1a2f91516 = 14:  0408e53002 t7400: make hash size independent
15:  9acdbf294c = 15:  3f0bb9751d t7405: make hash size independent
16:  2b506517de = 16:  406c9088af t7506: avoid checking for SHA-1-specific constants
17:  976974106c = 17:  bea582c49f t7508: use $ZERO_OID instead of hard-coded constant
18:  459e6d2455 = 18:  bb044354bc t8002: make hash size independent
19:  8c19b1280d ! 19:  888ffd32d3 t8003: make hash size independent
    @@ t/t8003-blame-corner-cases.sh: test_description='git blame corner cases'
      pick_fc='s/^[0-9a-f^]* *\([^ ]*\) *(\([^ ]*\) .*/\1-\2/'
      
      test_expect_success setup '
    -+	test_oid_init &&
    - 
    +-
      	echo A A A A A >one &&
      	echo B B B B B >two &&
    + 	echo C C C C C >tres &&
     @@ t/t8003-blame-corner-cases.sh: test_expect_success 'blame coalesce' '
      	$oid 1) ABC
      	$oid 2) DEF
20:  42e8d962dc = 20:  a6f7345990 t8011: make hash size independent
21:  076249fdea = 21:  25458092e4 t9300: abstract away SHA-1-specific constants
22:  36fed34e25 = 22:  637673d0f4 t9300: use $ZERO_OID instead of hard-coded object ID
23:  fd2b44e138 ! 23:  77a42dbf51 t9301: make hash size independent
    @@ Commit message
         Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
     
      ## t/t9301-fast-import-notes.sh ##
    -@@ t/t9301-fast-import-notes.sh: EOF
    - INPUT_END
    - 
    - test_expect_success 'set up master branch' '
    --
    -+	test_oid_init &&
    - 	git fast-import <input &&
    - 	git whatchanged master
    - '
     @@ t/t9301-fast-import-notes.sh: test_expect_success 'add lots of commits and notes' '
      '
      
24:  5cf2612131 ! 24:  3c405c54fd t9350: make hash size independent
    @@ Commit message
         Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
     
      ## t/t9350-fast-export.sh ##
    -@@ t/t9350-fast-export.sh: test_description='git fast-export'
    - . ./test-lib.sh
    - 
    - test_expect_success 'setup' '
    -+	test_oid_init &&
    - 
    - 	echo break it > file0 &&
    - 	git add file0 &&
     @@ t/t9350-fast-export.sh: test_expect_success 'reencoding iso-8859-7' '
      	sed "s/wer/i18n/" iso-8859-7.fi |
      		(cd new &&
25:  64b56a029a = 25:  c75d879bca t9500: ensure that algorithm info is preserved in config
26:  46800750af = 26:  82f2a6146f t9700: make hash size independent
27:  9c5a7b9c38 = 27:  60d7922842 t5308: make test work with SHA-256
28:  1a98d1e4b6 = 28:  c668823701 t0410: mark test with SHA1 prerequisite
29:  359543b8de <  -:  ---------- http-fetch: set up git directory before parsing pack hashes
 -:  ---------- > 29:  e1e0e04e83 http-fetch: set up git directory before parsing pack hashes
30:  bc5e6f6199 = 30:  96e770dce2 builtin/verify-pack: implement an --object-format option
31:  10f9420219 = 31:  dd79604c8b bundle: add new version for use with SHA-256
32:  59717a9f42 = 32:  425fdcca3d setup: add support for reading extensions.objectformat
33:  8313384a88 ! 33:  e7256281f4 Enable SHA-256 support by default
    @@ Metadata
     Author: brian m. carlson <sandals@crustytoothpaste.net>
     
      ## Commit message ##
    -    Enable SHA-256 support by default
    +    repository: enable SHA-256 support by default
     
    -    Now that the we have a complete SHA-256 implementation in Git, let's
    -    enable it so people can use it.  Remove the ENABLE_SHA256 define
    -    constant everywhere it's used.  Add tests for initializing a repository
    -    with SHA-256.
    +    Now that we have a complete SHA-256 implementation in Git, let's enable
    +    it so people can use it.  Remove the ENABLE_SHA256 define constant
    +    everywhere it's used.  Add tests for initializing a repository with
    +    SHA-256.
     
         Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
     
34:  e8439aa0d0 = 34:  65d43826f6 t: add test_oid option to select hash algorithm
35:  e5c9553da9 = 35:  d32ff6431b t: allow testing different hash algorithms via environment
36:  2a6f2ccd55 ! 36:  8cc8c77f48 t: make SHA1 prerequisite depend on default hash
    @@ t/test-lib.sh: test_lazy_prereq CURL '
     +	case "$GIT_DEFAULT_HASH" in
     +	sha1) true ;;
     +	"") test $(git hash-object /dev/null) = e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 ;;
    -+	*) false ::
    ++	*) false ;;
     +	esac
      '
      
37:  2eec901cdf = 37:  01d865b03e ci: run tests with SHA-256
38:  6a7fb3e911 = 38:  83f565d1d2 docs: add documentation for extensions.objectFormat
39:  448faa92fe ! 39:  6c3bf65a7d t: remove test_oid_init in tests
    @@ t/t1500-rev-parse.sh: test_rev_parse () {
      	cp -R .git repo.git
      '
     
    - ## t/t3305-notes-fanout.sh ##
    -@@ t/t3305-notes-fanout.sh: all_notes_have_fanout() {
    - }
    - 
    - test_expect_success 'tweak test environment' '
    --	test_oid_init &&
    - 	git checkout -b nondeterminism &&
    - 	test_commit A &&
    - 	git checkout --orphan with_notes;
    -
      ## t/t3308-notes-merge.sh ##
     @@ t/t3308-notes-merge.sh: test_expect_success setup '
      	# Copy notes to remote-notes
    @@ t/t3308-notes-merge.sh: test_expect_success setup '
      	hash4a sha1:5e93d24084d32e1cb61f7070505b9d2530cca987
      	hash3a sha1:8366731eeee53787d2bdf8fc1eff7d94757e8da0
     
    - ## t/t3404-rebase-interactive.sh ##
    -@@ t/t3404-rebase-interactive.sh: Initial setup:
    - . "$TEST_DIRECTORY"/lib-rebase.sh
    - 
    - test_expect_success 'setup' '
    --	test_oid_init &&
    - 	test_commit A file1 &&
    - 	test_commit B file1 &&
    - 	test_commit C file2 &&
    -
      ## t/t3600-rm.sh ##
     @@ t/t3600-rm.sh: test_expect_success 'refresh index before checking if it is up-to-date' '
      '
    @@ t/t6006-rev-list-format.sh: changed_iso88591=$(echo "$changed" | iconv -f utf-8
      	: >foo &&
      	git add foo &&
      	git config i18n.commitEncoding $test_encoding &&
    -
    - ## t/t6301-for-each-ref-errors.sh ##
    -@@ t/t6301-for-each-ref-errors.sh: test_description='for-each-ref errors for broken refs'
    - ZEROS=$ZERO_OID
    - 
    - test_expect_success setup '
    --	test_oid_init &&
    - 	MISSING=$(test_oid deadbeef) &&
    - 	git commit --allow-empty -m "Initial" &&
    - 	git tag testtag &&
    -
    - ## t/t6501-freshen-objects.sh ##
    -@@ t/t6501-freshen-objects.sh: maybe_repack () {
    - 	fi
    - }
    - 
    --test_expect_success 'setup' '
    --	test_oid_init
    --'
    --
    - for repack in '' true; do
    - 	title=${repack:+repack}
    - 	title=${title:-loose}
    -
    - ## t/t7003-filter-branch.sh ##
    -@@ t/t7003-filter-branch.sh: test_description='git filter-branch'
    - . "$TEST_DIRECTORY/lib-gpg.sh"
    - 
    - test_expect_success 'setup' '
    --	test_oid_init &&
    - 	test_commit A &&
    - 	GIT_COMMITTER_DATE="@0 +0000" GIT_AUTHOR_DATE="@0 +0000" &&
    - 	test_commit --notick B &&
    -
    - ## t/t8003-blame-corner-cases.sh ##
    -@@ t/t8003-blame-corner-cases.sh: test_description='git blame corner cases'
    - pick_fc='s/^[0-9a-f^]* *\([^ ]*\) *(\([^ ]*\) .*/\1-\2/'
    - 
    - test_expect_success setup '
    --	test_oid_init &&
    --
    - 	echo A A A A A >one &&
    - 	echo B B B B B >two &&
    - 	echo C C C C C >tres &&
    -
    - ## t/t9301-fast-import-notes.sh ##
    -@@ t/t9301-fast-import-notes.sh: EOF
    - INPUT_END
    - 
    - test_expect_success 'set up master branch' '
    --	test_oid_init &&
    - 	git fast-import <input &&
    - 	git whatchanged master
    - '
    -
    - ## t/t9350-fast-export.sh ##
    -@@ t/t9350-fast-export.sh: test_description='git fast-export'
    - . ./test-lib.sh
    - 
    - test_expect_success 'setup' '
    --	test_oid_init &&
    --
    - 	echo break it > file0 &&
    - 	git add file0 &&
    - 	test_tick &&
brian m. carlson· Jul 29, 2020, 23:13 UTC · re: brian m. carlson · lore

[PATCH v6 01/39] t: make test-bloom initialize repository

The bloom filter code relies on reading object IDs using parse_oid_hex. In order to make that work with an appropriate size, we need to have initialized the repository's hash algorithm. Since the values we're processing depend on the repository in use, let's set up the repository when we run the test helper.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/helper/test-bloom.c | 2 ++
 1 file changed, 2 insertions(+)
Show changes to t/helper/test-bloom.c +2 −0
diff --git a/t/helper/test-bloom.c b/t/helper/test-bloom.c
index f0aa80b98e..5e77d56f59 100644
--- a/t/helper/test-bloom.c
+++ b/t/helper/test-bloom.c
@@ -50,6 +50,8 @@ static const char *bloom_usage = "\n"
 
 int cmd__bloom(int argc, const char **argv)
 {
+	setup_git_directory();
+
 	if (argc < 2)
 		usage(bloom_usage);
 
brian m. carlson· Jul 29, 2020, 23:13 UTC · re: brian m. carlson · lore

[PATCH v6 04/39] t3404: prepare 'short SHA-1 collision' tests for SHA-256

From: Johannes Schindelin <johannes.schindelin@gmx.de>

The idea of the magic value "ac4f2ee" in this test is to make the reworded commit `collide2` have the same shortened ID as the commit `collide3`.

To port the same idea to the SHA-256 version of Git, we therefore need another magic value that causes the same collision, but this time with the SHA-256 version of the commit IDs.

In this patch, we add code guarded by `GIT_TEST_FIND_COLLIDER` to do exactly that. Essentially, a large number of integers is appended to the commit message "collide2" to find such a collision. To make it easier to find such a collision, we reduce the number of digits to 4.

As the tests are no longer dependent on SHA-1, we also rename their titles to talk about "commit IDs" instead of "SHA-1s".

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t3404-rebase-interactive.sh | 49 ++++++++++++++++++++++++++++-------
 1 file changed, 40 insertions(+), 9 deletions(-)
Show changes to t/t3404-rebase-interactive.sh +40 −9
diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh
index 4a7d21f898..9744e88760 100755
--- a/t/t3404-rebase-interactive.sh
+++ b/t/t3404-rebase-interactive.sh
@@ -1250,7 +1250,7 @@ test_expect_success 'rebase -i error on commits with \ in message' '
 	test_expect_code 1 grep  "	emp" error
 '
 
-test_expect_success SHA1 'short SHA-1 setup' '
+test_expect_success 'short commit ID setup' '
 	test_when_finished "git checkout master" &&
 	git checkout --orphan collide &&
 	git rm -rf . &&
@@ -1262,23 +1262,54 @@ test_expect_success SHA1 'short SHA-1 setup' '
 	)
 '
 
-test_expect_success SHA1 'short SHA-1 collide' '
+if test -n "$GIT_TEST_FIND_COLLIDER"
+then
+	author="$(unset test_tick; test_tick; git var GIT_AUTHOR_IDENT)"
+	committer="$(unset test_tick; test_tick; git var GIT_COMMITTER_IDENT)"
+	blob="$(git rev-parse collide2:collide)"
+	from="$(git rev-parse collide1^0)"
+	repl="commit refs/heads/collider-&\\n"
+	repl="${repl}author $author\\ncommitter $committer\\n"
+	repl="${repl}data <<EOF\\ncollide2 &\\nEOF\\n"
+	repl="${repl}from $from\\nM 100644 $blob collide\\n"
+	test_seq 1 32768 | sed "s|.*|$repl|" >script &&
+	git fast-import <script &&
+	git pack-refs &&
+	git for-each-ref >refs &&
+	grep "^$(test_oid t3404_collision)" <refs >matches &&
+	cat matches &&
+	test_line_count -gt 2 matches || {
+		echo "Could not find a collider" >&2
+		exit 1
+	}
+fi
+
+test_expect_success 'short commit ID collide' '
+	test_oid_cache <<-EOF &&
+	# collision-related constants
+	t3404_collision	sha1:6bcd
+	t3404_collision	sha256:0161
+	t3404_collider	sha1:ac4f2ee
+	t3404_collider	sha256:16697
+	EOF
 	test_when_finished "reset_rebase && git checkout master" &&
 	git checkout collide &&
-	colliding_sha1=6bcda37 &&
-	test $colliding_sha1 = "$(git rev-parse HEAD | cut -c 1-7)" &&
+	colliding_id=$(test_oid t3404_collision) &&
+	hexsz=$(test_oid hexsz) &&
+	test $colliding_id = "$(git rev-parse HEAD | cut -c 1-4)" &&
+	test_config core.abbrev 4 &&
 	(
 		unset test_tick &&
 		test_tick &&
 		set_fake_editor &&
-		FAKE_COMMIT_MESSAGE="collide2 ac4f2ee" \
+		FAKE_COMMIT_MESSAGE="collide2 $(test_oid t3404_collider)" \
 		FAKE_LINES="reword 1 break 2" git rebase -i HEAD~2 &&
-		test $colliding_sha1 = "$(git rev-parse HEAD | cut -c 1-7)" &&
-		grep "^pick $colliding_sha1 " \
+		test $colliding_id = "$(git rev-parse HEAD | cut -c 1-4)" &&
+		grep "^pick $colliding_id " \
 			.git/rebase-merge/git-rebase-todo.tmp &&
-		grep "^pick [0-9a-f]\{40\}" \
+		grep "^pick [0-9a-f]\{$hexsz\}" \
 			.git/rebase-merge/git-rebase-todo &&
-		grep "^pick [0-9a-f]\{40\}" \
+		grep "^pick [0-9a-f]\{$hexsz\}" \
 			.git/rebase-merge/git-rebase-todo.backup &&
 		git rebase --continue
 	) &&
brian m. carlson· Jul 29, 2020, 23:13 UTC · re: brian m. carlson · lore

[PATCH v6 07/39] t6301: make hash size independent

Instead of hard-coding a fixed length example object ID in the test, compute one using the translation tables. Move a variable into the setup block so that we can ensure the exit status of test_oid is checked.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t6301-for-each-ref-errors.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Show changes to t/t6301-for-each-ref-errors.sh +1 −1
diff --git a/t/t6301-for-each-ref-errors.sh b/t/t6301-for-each-ref-errors.sh
index 49cc65bb58..809854fc0c 100755
--- a/t/t6301-for-each-ref-errors.sh
+++ b/t/t6301-for-each-ref-errors.sh
@@ -5,9 +5,9 @@ test_description='for-each-ref errors for broken refs'
 . ./test-lib.sh
 
 ZEROS=$ZERO_OID
-MISSING=abababababababababababababababababababab
 
 test_expect_success setup '
+	MISSING=$(test_oid deadbeef) &&
 	git commit --allow-empty -m "Initial" &&
 	git tag testtag &&
 	git for-each-ref >full-list &&
brian m. carlson· Jul 29, 2020, 23:13 UTC · re: brian m. carlson · lore

[PATCH v6 09/39] t6501: avoid hard-coded objects

This test contains hard-coded invalid object IDs. Make it hash size independent by generating invalid object IDs using the translation tables. Add a setup target to ensure the output of test_oid_init is checked properly.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t6501-freshen-objects.sh | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)
Show changes to t/t6501-freshen-objects.sh +7 −7
diff --git a/t/t6501-freshen-objects.sh b/t/t6501-freshen-objects.sh
index f30b4849b6..8a3bb4105b 100755
--- a/t/t6501-freshen-objects.sh
+++ b/t/t6501-freshen-objects.sh
@@ -128,9 +128,9 @@ for repack in '' true; do
 done
 
 test_expect_success 'do not complain about existing broken links (commit)' '
-	cat >broken-commit <<-\EOF &&
-	tree 0000000000000000000000000000000000000001
-	parent 0000000000000000000000000000000000000002
+	cat >broken-commit <<-EOF &&
+	tree $(test_oid 001)
+	parent $(test_oid 002)
 	author whatever <whatever@example.com> 1234 -0000
 	committer whatever <whatever@example.com> 1234 -0000
 
@@ -143,8 +143,8 @@ test_expect_success 'do not complain about existing broken links (commit)' '
 '
 
 test_expect_success 'do not complain about existing broken links (tree)' '
-	cat >broken-tree <<-\EOF &&
-	100644 blob 0000000000000000000000000000000000000003	foo
+	cat >broken-tree <<-EOF &&
+	100644 blob $(test_oid 003)	foo
 	EOF
 	tree=$(git mktree --missing <broken-tree) &&
 	git gc -q 2>stderr &&
@@ -153,8 +153,8 @@ test_expect_success 'do not complain about existing broken links (tree)' '
 '
 
 test_expect_success 'do not complain about existing broken links (tag)' '
-	cat >broken-tag <<-\EOF &&
-	object 0000000000000000000000000000000000000004
+	cat >broken-tag <<-EOF &&
+	object $(test_oid 004)
 	type commit
 	tag broken
 	tagger whatever <whatever@example.com> 1234 -0000
brian m. carlson· Jul 29, 2020, 23:14 UTC · re: brian m. carlson · lore

[PATCH v6 11/39] t7063: make hash size independent

Use test_oid instead of hard-coding algorithm-specific constants and all-zero values.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t7063-status-untracked-cache.sh | 150 ++++++++++++++++--------------
 1 file changed, 80 insertions(+), 70 deletions(-)
Show changes to t/t7063-status-untracked-cache.sh +80 −70
diff --git a/t/t7063-status-untracked-cache.sh b/t/t7063-status-untracked-cache.sh
index 428cff9cf3..a682a3d826 100755
--- a/t/t7063-status-untracked-cache.sh
+++ b/t/t7063-status-untracked-cache.sh
@@ -75,14 +75,24 @@ test_expect_success 'setup' '
 	touch one two three done/one dtwo/two dthree/three &&
 	git add one two done/one &&
 	: >.git/info/exclude &&
-	git update-index --untracked-cache
+	git update-index --untracked-cache &&
+	test_oid_cache <<-EOF
+	root sha1:e6fcc8f2ee31bae321d66afd183fcb7237afae6e
+	root sha256:b90c672088c015b9c83876e919da311bad4cd39639fb139f988af6a11493b974
+
+	exclude sha1:13263c0978fb9fad16b2d580fb800b6d811c3ff0
+	exclude sha256:fe4aaa1bbbbce4cb8f73426748a14c5ad6026b26f90505a0bf2494b165a5b76c
+
+	done sha1:1946f0437f90c5005533cbe1736a6451ca301714
+	done sha256:7f079501d79f665b3acc50f5e0e9e94509084d5032ac20113a37dd5029b757cc
+	EOF
 '
 
 test_expect_success 'untracked cache is empty' '
 	test-tool dump-untracked-cache >../actual &&
 	cat >../expect-empty <<EOF &&
-info/exclude 0000000000000000000000000000000000000000
-core.excludesfile 0000000000000000000000000000000000000000
+info/exclude $ZERO_OID
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
 EOF
@@ -100,17 +110,17 @@ EOF
 
 cat >../dump.expect <<EOF &&
 info/exclude $EMPTY_BLOB
-core.excludesfile 0000000000000000000000000000000000000000
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
-/ 0000000000000000000000000000000000000000 recurse valid
+/ $ZERO_OID recurse valid
 dthree/
 dtwo/
 three
-/done/ 0000000000000000000000000000000000000000 recurse valid
-/dthree/ 0000000000000000000000000000000000000000 recurse check_only valid
+/done/ $ZERO_OID recurse valid
+/dthree/ $ZERO_OID recurse check_only valid
 three
-/dtwo/ 0000000000000000000000000000000000000000 recurse check_only valid
+/dtwo/ $ZERO_OID recurse check_only valid
 two
 EOF
 
@@ -190,18 +200,18 @@ test_expect_success 'verify untracked cache dump' '
 	test-tool dump-untracked-cache >../actual &&
 	cat >../expect <<EOF &&
 info/exclude $EMPTY_BLOB
-core.excludesfile 0000000000000000000000000000000000000000
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
-/ 0000000000000000000000000000000000000000 recurse valid
+/ $ZERO_OID recurse valid
 dthree/
 dtwo/
 four
 three
-/done/ 0000000000000000000000000000000000000000 recurse valid
-/dthree/ 0000000000000000000000000000000000000000 recurse check_only valid
+/done/ $ZERO_OID recurse valid
+/dthree/ $ZERO_OID recurse check_only valid
 three
-/dtwo/ 0000000000000000000000000000000000000000 recurse check_only valid
+/dtwo/ $ZERO_OID recurse check_only valid
 two
 EOF
 	test_cmp ../expect ../actual
@@ -239,18 +249,18 @@ test_expect_success 'verify untracked cache dump' '
 	test-tool dump-untracked-cache >../actual &&
 	cat >../expect <<EOF &&
 info/exclude $EMPTY_BLOB
-core.excludesfile 0000000000000000000000000000000000000000
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
-/ e6fcc8f2ee31bae321d66afd183fcb7237afae6e recurse valid
+/ $(test_oid root) recurse valid
 .gitignore
 dthree/
 dtwo/
 three
-/done/ 0000000000000000000000000000000000000000 recurse valid
-/dthree/ 0000000000000000000000000000000000000000 recurse check_only valid
+/done/ $ZERO_OID recurse valid
+/dthree/ $ZERO_OID recurse check_only valid
 three
-/dtwo/ 0000000000000000000000000000000000000000 recurse check_only valid
+/dtwo/ $ZERO_OID recurse check_only valid
 two
 EOF
 	test_cmp ../expect ../actual
@@ -284,16 +294,16 @@ EOF
 test_expect_success 'verify untracked cache dump' '
 	test-tool dump-untracked-cache >../actual &&
 	cat >../expect <<EOF &&
-info/exclude 13263c0978fb9fad16b2d580fb800b6d811c3ff0
-core.excludesfile 0000000000000000000000000000000000000000
+info/exclude $(test_oid exclude)
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
-/ e6fcc8f2ee31bae321d66afd183fcb7237afae6e recurse valid
+/ $(test_oid root) recurse valid
 .gitignore
 dtwo/
-/done/ 0000000000000000000000000000000000000000 recurse valid
-/dthree/ 0000000000000000000000000000000000000000 recurse check_only valid
-/dtwo/ 0000000000000000000000000000000000000000 recurse check_only valid
+/done/ $ZERO_OID recurse valid
+/dthree/ $ZERO_OID recurse check_only valid
+/dtwo/ $ZERO_OID recurse check_only valid
 two
 EOF
 	test_cmp ../expect ../actual
@@ -303,14 +313,14 @@ test_expect_success 'move two from tracked to untracked' '
 	git rm --cached two &&
 	test-tool dump-untracked-cache >../actual &&
 	cat >../expect <<EOF &&
-info/exclude 13263c0978fb9fad16b2d580fb800b6d811c3ff0
-core.excludesfile 0000000000000000000000000000000000000000
+info/exclude $(test_oid exclude)
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
-/ e6fcc8f2ee31bae321d66afd183fcb7237afae6e recurse
-/done/ 0000000000000000000000000000000000000000 recurse valid
-/dthree/ 0000000000000000000000000000000000000000 recurse check_only valid
-/dtwo/ 0000000000000000000000000000000000000000 recurse check_only valid
+/ $(test_oid root) recurse
+/done/ $ZERO_OID recurse valid
+/dthree/ $ZERO_OID recurse check_only valid
+/dtwo/ $ZERO_OID recurse check_only valid
 two
 EOF
 	test_cmp ../expect ../actual
@@ -342,17 +352,17 @@ EOF
 test_expect_success 'verify untracked cache dump' '
 	test-tool dump-untracked-cache >../actual &&
 	cat >../expect <<EOF &&
-info/exclude 13263c0978fb9fad16b2d580fb800b6d811c3ff0
-core.excludesfile 0000000000000000000000000000000000000000
+info/exclude $(test_oid exclude)
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
-/ e6fcc8f2ee31bae321d66afd183fcb7237afae6e recurse valid
+/ $(test_oid root) recurse valid
 .gitignore
 dtwo/
 two
-/done/ 0000000000000000000000000000000000000000 recurse valid
-/dthree/ 0000000000000000000000000000000000000000 recurse check_only valid
-/dtwo/ 0000000000000000000000000000000000000000 recurse check_only valid
+/done/ $ZERO_OID recurse valid
+/dthree/ $ZERO_OID recurse check_only valid
+/dtwo/ $ZERO_OID recurse check_only valid
 two
 EOF
 	test_cmp ../expect ../actual
@@ -362,14 +372,14 @@ test_expect_success 'move two from untracked to tracked' '
 	git add two &&
 	test-tool dump-untracked-cache >../actual &&
 	cat >../expect <<EOF &&
-info/exclude 13263c0978fb9fad16b2d580fb800b6d811c3ff0
-core.excludesfile 0000000000000000000000000000000000000000
+info/exclude $(test_oid exclude)
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
-/ e6fcc8f2ee31bae321d66afd183fcb7237afae6e recurse
-/done/ 0000000000000000000000000000000000000000 recurse valid
-/dthree/ 0000000000000000000000000000000000000000 recurse check_only valid
-/dtwo/ 0000000000000000000000000000000000000000 recurse check_only valid
+/ $(test_oid root) recurse
+/done/ $ZERO_OID recurse valid
+/dthree/ $ZERO_OID recurse check_only valid
+/dtwo/ $ZERO_OID recurse check_only valid
 two
 EOF
 	test_cmp ../expect ../actual
@@ -401,16 +411,16 @@ EOF
 test_expect_success 'verify untracked cache dump' '
 	test-tool dump-untracked-cache >../actual &&
 	cat >../expect <<EOF &&
-info/exclude 13263c0978fb9fad16b2d580fb800b6d811c3ff0
-core.excludesfile 0000000000000000000000000000000000000000
+info/exclude $(test_oid exclude)
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
-/ e6fcc8f2ee31bae321d66afd183fcb7237afae6e recurse valid
+/ $(test_oid root) recurse valid
 .gitignore
 dtwo/
-/done/ 0000000000000000000000000000000000000000 recurse valid
-/dthree/ 0000000000000000000000000000000000000000 recurse check_only valid
-/dtwo/ 0000000000000000000000000000000000000000 recurse check_only valid
+/done/ $ZERO_OID recurse valid
+/dthree/ $ZERO_OID recurse check_only valid
+/dtwo/ $ZERO_OID recurse check_only valid
 two
 EOF
 	test_cmp ../expect ../actual
@@ -447,16 +457,16 @@ EOF
 test_expect_success 'untracked cache correct after commit' '
 	test-tool dump-untracked-cache >../actual &&
 	cat >../expect <<EOF &&
-info/exclude 13263c0978fb9fad16b2d580fb800b6d811c3ff0
-core.excludesfile 0000000000000000000000000000000000000000
+info/exclude $(test_oid exclude)
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
-/ e6fcc8f2ee31bae321d66afd183fcb7237afae6e recurse valid
+/ $(test_oid root) recurse valid
 .gitignore
 dtwo/
-/done/ 0000000000000000000000000000000000000000 recurse valid
-/dthree/ 0000000000000000000000000000000000000000 recurse check_only valid
-/dtwo/ 0000000000000000000000000000000000000000 recurse check_only valid
+/done/ $ZERO_OID recurse valid
+/dthree/ $ZERO_OID recurse check_only valid
+/dtwo/ $ZERO_OID recurse check_only valid
 two
 EOF
 	test_cmp ../expect ../actual
@@ -508,17 +518,17 @@ EOF
 test_expect_success 'untracked cache correct after status' '
 	test-tool dump-untracked-cache >../actual &&
 	cat >../expect <<EOF &&
-info/exclude 13263c0978fb9fad16b2d580fb800b6d811c3ff0
-core.excludesfile 0000000000000000000000000000000000000000
+info/exclude $(test_oid exclude)
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
-/ e6fcc8f2ee31bae321d66afd183fcb7237afae6e recurse valid
+/ $(test_oid root) recurse valid
 .gitignore
 dtwo/
-/done/ 1946f0437f90c5005533cbe1736a6451ca301714 recurse valid
+/done/ $(test_oid done) recurse valid
 five
-/dthree/ 0000000000000000000000000000000000000000 recurse check_only valid
-/dtwo/ 0000000000000000000000000000000000000000 recurse check_only valid
+/dthree/ $ZERO_OID recurse check_only valid
+/dtwo/ $ZERO_OID recurse check_only valid
 two
 EOF
 	test_cmp ../expect ../actual
@@ -580,22 +590,22 @@ EOF
 test_expect_success 'verify untracked cache dump (sparse/subdirs)' '
 	test-tool dump-untracked-cache >../actual &&
 	cat >../expect-from-test-dump <<EOF &&
-info/exclude 13263c0978fb9fad16b2d580fb800b6d811c3ff0
-core.excludesfile 0000000000000000000000000000000000000000
+info/exclude $(test_oid exclude)
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
-/ e6fcc8f2ee31bae321d66afd183fcb7237afae6e recurse valid
+/ $(test_oid root) recurse valid
 .gitignore
 dtwo/
-/done/ 1946f0437f90c5005533cbe1736a6451ca301714 recurse valid
+/done/ $(test_oid done) recurse valid
 five
 sub/
-/done/sub/ 0000000000000000000000000000000000000000 recurse check_only valid
+/done/sub/ $ZERO_OID recurse check_only valid
 sub/
-/done/sub/sub/ 0000000000000000000000000000000000000000 recurse check_only valid
+/done/sub/sub/ $ZERO_OID recurse check_only valid
 file
-/dthree/ 0000000000000000000000000000000000000000 recurse check_only valid
-/dtwo/ 0000000000000000000000000000000000000000 recurse check_only valid
+/dthree/ $ZERO_OID recurse check_only valid
+/dtwo/ $ZERO_OID recurse check_only valid
 two
 EOF
 	test_cmp ../expect-from-test-dump ../actual
@@ -806,8 +816,8 @@ test_expect_success '"status" after file replacement should be clean with UC=tru
 	test-tool dump-untracked-cache >../actual &&
 	grep -F "recurse valid" ../actual >../actual.grep &&
 	cat >../expect.grep <<EOF &&
-/ 0000000000000000000000000000000000000000 recurse valid
-/two/ 0000000000000000000000000000000000000000 recurse valid
+/ $ZERO_OID recurse valid
+/two/ $ZERO_OID recurse valid
 EOF
 	status_is_clean &&
 	test_cmp ../expect.grep ../actual.grep
brian m. carlson· Jul 29, 2020, 23:14 UTC · re: brian m. carlson · lore

[PATCH v6 13/39] t7102: abstract away SHA-1-specific constants

Adjust the test so that it computes variables for object IDs instead of using hard-coded hashes.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t7102-reset.sh | 93 +++++++++++++++++++++++++++---------------------
 1 file changed, 53 insertions(+), 40 deletions(-)
Show changes to t/t7102-reset.sh +53 −37
diff --git a/t/t7102-reset.sh b/t/t7102-reset.sh
index 97be0d968d..22161b3b2d 100755
--- a/t/t7102-reset.sh
+++ b/t/t7102-reset.sh
@@ -37,17 +37,23 @@ test_expect_success 'creating initial files and commits' '
 
 	echo "2nd line 1st file" >>first &&
 	git commit -a -m "modify 1st file" &&
+	head5p2=$(git rev-parse --verify HEAD) &&
+	head5p2f=$(git rev-parse --short HEAD:first) &&
 
 	git rm first &&
 	git mv second secondfile &&
 	git commit -a -m "remove 1st and rename 2nd" &&
+	head5p1=$(git rev-parse --verify HEAD) &&
+	head5p1s=$(git rev-parse --short HEAD:secondfile) &&
 
 	echo "1st line 2nd file" >secondfile &&
 	echo "2nd line 2nd file" >>secondfile &&
 	# "git commit -m" would break MinGW, as Windows refuse to pass
 	# $test_encoding encoded parameter to git.
 	commit_msg $test_encoding | git -c "i18n.commitEncoding=$test_encoding" commit -a -F - &&
-	head5=$(git rev-parse --verify HEAD)
+	head5=$(git rev-parse --verify HEAD) &&
+	head5s=$(git rev-parse --short HEAD:secondfile) &&
+	head5sl=$(git rev-parse HEAD:secondfile)
 '
 # git log --pretty=oneline # to see those SHA1 involved
 
@@ -94,7 +100,7 @@ test_expect_success 'giving a non existing revision should fail' '
 
 test_expect_success 'reset --soft with unmerged index should fail' '
 	touch .git/MERGE_HEAD &&
-	echo "100644 44c5b5884550c17758737edcced463447b91d42b 1	un" |
+	echo "100644 $head5sl 1	un" |
 		git update-index --index-info &&
 	test_must_fail git reset --soft HEAD &&
 	rm .git/MERGE_HEAD &&
@@ -192,7 +198,7 @@ test_expect_success \
 >.diff_expect
 cat >.cached_expect <<EOF
 diff --git a/secondfile b/secondfile
-index 1bbba79..44c5b58 100644
+index $head5p1s..$head5s 100644
 --- a/secondfile
 +++ b/secondfile
 @@ -1 +1,2 @@
@@ -207,7 +213,7 @@ secondfile:
 EOF
 test_expect_success '--soft reset only should show changes in diff --cached' '
 	git reset --soft HEAD^ &&
-	check_changes d1a4bc3abce4829628ae2dcb0d60ef3d1a78b1c4 &&
+	check_changes $head5p1 &&
 	test "$(git rev-parse ORIG_HEAD)" = \
 			$head5
 '
@@ -242,7 +248,7 @@ EOF
 test_expect_success \
 	'--hard reset should change the files and undo commits permanently' '
 	git reset --hard HEAD~2 &&
-	check_changes ddaefe00f1da16864591c61fdc7adb5d7cd6b74e &&
+	check_changes $head5p2 &&
 	test "$(git rev-parse ORIG_HEAD)" = \
 			$head4
 '
@@ -251,7 +257,7 @@ test_expect_success \
 cat >.cached_expect <<EOF
 diff --git a/first b/first
 deleted file mode 100644
-index 8206c22..0000000
+index $head5p2f..0000000
 --- a/first
 +++ /dev/null
 @@ -1,2 +0,0 @@
@@ -259,14 +265,14 @@ index 8206c22..0000000
 -2nd line 1st file
 diff --git a/second b/second
 deleted file mode 100644
-index 1bbba79..0000000
+index $head5p1s..0000000
 --- a/second
 +++ /dev/null
 @@ -1 +0,0 @@
 -2nd file
 diff --git a/secondfile b/secondfile
 new file mode 100644
-index 0000000..44c5b58
+index 0000000..$head5s
 --- /dev/null
 +++ b/secondfile
 @@ -0,0 +1,2 @@
@@ -286,13 +292,13 @@ test_expect_success \
 	echo "1st line 2nd file" >secondfile &&
 	echo "2nd line 2nd file" >>secondfile &&
 	git add secondfile &&
-	check_changes ddaefe00f1da16864591c61fdc7adb5d7cd6b74e
+	check_changes $head5p2
 '
 
 cat >.diff_expect <<EOF
 diff --git a/first b/first
 deleted file mode 100644
-index 8206c22..0000000
+index $head5p2f..0000000
 --- a/first
 +++ /dev/null
 @@ -1,2 +0,0 @@
@@ -300,7 +306,7 @@ index 8206c22..0000000
 -2nd line 1st file
 diff --git a/second b/second
 deleted file mode 100644
-index 1bbba79..0000000
+index $head5p1s..0000000
 --- a/second
 +++ /dev/null
 @@ -1 +0,0 @@
@@ -314,9 +320,8 @@ secondfile:
 EOF
 test_expect_success '--mixed reset to HEAD should unadd the files' '
 	git reset &&
-	check_changes ddaefe00f1da16864591c61fdc7adb5d7cd6b74e &&
-	test "$(git rev-parse ORIG_HEAD)" = \
-			ddaefe00f1da16864591c61fdc7adb5d7cd6b74e
+	check_changes $head5p2 &&
+	test "$(git rev-parse ORIG_HEAD)" = $head5p2
 '
 
 >.diff_expect
@@ -328,7 +333,7 @@ secondfile:
 EOF
 test_expect_success 'redoing the last two commits should succeed' '
 	git add secondfile &&
-	git reset --hard ddaefe00f1da16864591c61fdc7adb5d7cd6b74e &&
+	git reset --hard $head5p2 &&
 
 	git rm first &&
 	git mv second secondfile &&
@@ -389,47 +394,55 @@ test_expect_success \
 	check_changes $head5
 '
 
-cat > expect << EOF
-diff --git a/file1 b/file1
-index d00491f..7ed6ff8 100644
---- a/file1
-+++ b/file1
-@@ -1 +1 @@
--1
-+5
-diff --git a/file2 b/file2
-deleted file mode 100644
-index 0cfbf08..0000000
---- a/file2
-+++ /dev/null
-@@ -1 +0,0 @@
--2
-EOF
-cat > cached_expect << EOF
-diff --git a/file4 b/file4
-new file mode 100644
-index 0000000..b8626c4
---- /dev/null
-+++ b/file4
-@@ -0,0 +1 @@
-+4
-EOF
 test_expect_success 'test --mixed <paths>' '
 	echo 1 > file1 &&
 	echo 2 > file2 &&
 	git add file1 file2 &&
 	test_tick &&
 	git commit -m files &&
+	before1=$(git rev-parse --short HEAD:file1) &&
+	before2=$(git rev-parse --short HEAD:file2) &&
 	git rm file2 &&
 	echo 3 > file3 &&
 	echo 4 > file4 &&
 	echo 5 > file1 &&
+	after1=$(git rev-parse --short $(git hash-object file1)) &&
+	after4=$(git rev-parse --short $(git hash-object file4)) &&
 	git add file1 file3 file4 &&
 	git reset HEAD -- file1 file2 file3 &&
 	test_must_fail git diff --quiet &&
 	git diff > output &&
+
+	cat > expect <<-EOF &&
+	diff --git a/file1 b/file1
+	index $before1..$after1 100644
+	--- a/file1
+	+++ b/file1
+	@@ -1 +1 @@
+	-1
+	+5
+	diff --git a/file2 b/file2
+	deleted file mode 100644
+	index $before2..0000000
+	--- a/file2
+	+++ /dev/null
+	@@ -1 +0,0 @@
+	-2
+	EOF
+
 	test_cmp expect output &&
 	git diff --cached > output &&
+
+	cat > cached_expect <<-EOF &&
+	diff --git a/file4 b/file4
+	new file mode 100644
+	index 0000000..$after4
+	--- /dev/null
+	+++ b/file4
+	@@ -0,0 +1 @@
+	+4
+	EOF
+
 	test_cmp cached_expect output
 '
 
brian m. carlson· Jul 29, 2020, 23:14 UTC · re: brian m. carlson · lore

[PATCH v6 16/39] t7506: avoid checking for SHA-1-specific constants

Adjust the test to sanitize the diffs and strip out object IDs from them, as it does for other object IDs, since we are not interested in the particular values used.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t7506-status-submodule.sh | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)
Show changes to t/t7506-status-submodule.sh +8 −4
diff --git a/t/t7506-status-submodule.sh b/t/t7506-status-submodule.sh
index 08629a6e70..3fcb44767f 100755
--- a/t/t7506-status-submodule.sh
+++ b/t/t7506-status-submodule.sh
@@ -22,6 +22,10 @@ sanitize_output () {
 	mv output2 output
 }
 
+sanitize_diff () {
+	sed -e "/^index [0-9a-f,]*\.\.[0-9a-f]*/d" "$1"
+}
+
 
 test_expect_success 'setup' '
 	test_create_repo_with_commit sub &&
@@ -269,7 +273,6 @@ short_sha1_merge_sub1=$(cd sub1 && git rev-parse --short HEAD)
 short_sha1_merge_sub2=$(cd sub2 && git rev-parse --short HEAD)
 cat >diff_expect <<\EOF
 diff --cc .gitmodules
-index badaa4c,44f999a..0000000
 --- a/.gitmodules
 +++ b/.gitmodules
 @@@ -1,3 -1,3 +1,9 @@@
@@ -286,7 +289,6 @@ EOF
 
 cat >diff_submodule_expect <<\EOF
 diff --cc .gitmodules
-index badaa4c,44f999a..0000000
 --- a/.gitmodules
 +++ b/.gitmodules
 @@@ -1,3 -1,3 +1,9 @@@
@@ -306,7 +308,8 @@ test_expect_success 'diff with merge conflict in .gitmodules' '
 		cd super &&
 		git diff >../diff_actual 2>&1
 	) &&
-	test_cmp diff_expect diff_actual
+	sanitize_diff diff_actual >diff_sanitized &&
+	test_cmp diff_expect diff_sanitized
 '
 
 test_expect_success 'diff --submodule with merge conflict in .gitmodules' '
@@ -314,7 +317,8 @@ test_expect_success 'diff --submodule with merge conflict in .gitmodules' '
 		cd super &&
 		git diff --submodule >../diff_submodule_actual 2>&1
 	) &&
-	test_cmp diff_submodule_expect diff_submodule_actual
+	sanitize_diff diff_submodule_actual >diff_sanitized &&
+	test_cmp diff_submodule_expect diff_sanitized
 '
 
 # We'll setup different cases for further testing:
brian m. carlson· Jul 29, 2020, 23:14 UTC · re: brian m. carlson · lore

[PATCH v6 17/39] t7508: use $ZERO_OID instead of hard-coded constant

Use the ZERO_OID variable to abbreviate the all-zeros object ID for maintainability and to avoid depending on a specific size for the hash.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t7508-status.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Show changes to t/t7508-status.sh +1 −1
diff --git a/t/t7508-status.sh b/t/t7508-status.sh
index 8e969f3e36..e81759319f 100755
--- a/t/t7508-status.sh
+++ b/t/t7508-status.sh
@@ -837,7 +837,7 @@ EOF
 '
 
 cat >expect <<EOF
-:100644 100644 $EMPTY_BLOB 0000000000000000000000000000000000000000 M	dir1/modified
+:100644 100644 $EMPTY_BLOB $ZERO_OID M	dir1/modified
 EOF
 test_expect_success 'status refreshes the index' '
 	touch dir2/added &&
brian m. carlson· Jul 29, 2020, 23:14 UTC · re: brian m. carlson · lore

[PATCH v6 19/39] t8003: make hash size independent

One assertion in this test invokes git with core.abbrev set to "40". Since we're expecting the full hash length, use test_oid to look up the full hash length for the hash in use.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t8003-blame-corner-cases.sh | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
Show changes to t/t8003-blame-corner-cases.sh +1 −2
diff --git a/t/t8003-blame-corner-cases.sh b/t/t8003-blame-corner-cases.sh
index 9130b887d2..b871dd4f86 100755
--- a/t/t8003-blame-corner-cases.sh
+++ b/t/t8003-blame-corner-cases.sh
@@ -6,7 +6,6 @@ test_description='git blame corner cases'
 pick_fc='s/^[0-9a-f^]* *\([^ ]*\) *(\([^ ]*\) .*/\1-\2/'
 
 test_expect_success setup '
-
 	echo A A A A A >one &&
 	echo B B B B B >two &&
 	echo C C C C C >tres &&
@@ -306,7 +305,7 @@ test_expect_success 'blame coalesce' '
 	$oid 1) ABC
 	$oid 2) DEF
 	EOF
-	git -c core.abbrev=40 blame -s giraffe >actual &&
+	git -c core.abbrev=$(test_oid hexsz) blame -s giraffe >actual &&
 	test_cmp expect actual
 '
 
brian m. carlson· Jul 29, 2020, 23:14 UTC · re: brian m. carlson · lore

[PATCH v6 22/39] t9300: use $ZERO_OID instead of hard-coded object ID

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t9300-fast-import.sh | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)
Show changes to t/t9300-fast-import.sh +8 −8
diff --git a/t/t9300-fast-import.sh b/t/t9300-fast-import.sh
index f2d2cb06ad..308c1ef42c 100755
--- a/t/t9300-fast-import.sh
+++ b/t/t9300-fast-import.sh
@@ -92,7 +92,7 @@ test_expect_success 'A: create pack from stdin' '
 	EOF
 
 	reset refs/tags/to-be-deleted
-	from 0000000000000000000000000000000000000000
+	from $ZERO_OID
 
 	tag nested
 	mark :6
@@ -102,7 +102,7 @@ test_expect_success 'A: create pack from stdin' '
 	EOF
 
 	reset refs/tags/nested
-	from 0000000000000000000000000000000000000000
+	from $ZERO_OID
 
 	tag nested
 	mark :7
@@ -286,7 +286,7 @@ test_expect_success 'A: verify pack' '
 test_expect_success 'A: verify diff' '
 	copy=$(git rev-parse --verify master:file2) &&
 	cat >expect <<-EOF &&
-	:000000 100755 0000000000000000000000000000000000000000 $copy A	copy-of-file2
+	:000000 100755 $ZERO_OID $copy A	copy-of-file2
 	EOF
 	git diff-tree -M -r master verify--import-marks >actual &&
 	compare_diff_raw expect actual &&
@@ -365,7 +365,7 @@ test_expect_success 'B: fail on invalid blob sha1' '
 	COMMIT
 
 	from refs/heads/master
-	M 755 0000000000000000000000000000000000000001 zero1
+	M 755 $(echo $ZERO_OID | sed -e "s/0$/1/") zero1
 
 	INPUT_END
 
@@ -620,8 +620,8 @@ test_expect_success 'D: validate new files added' '
 	f5id=$(echo "$file5_data" | git hash-object --stdin) &&
 	f6id=$(echo "$file6_data" | git hash-object --stdin) &&
 	cat >expect <<-EOF &&
-	:000000 100755 0000000000000000000000000000000000000000 $f6id A	newdir/exec.sh
-	:000000 100644 0000000000000000000000000000000000000000 $f5id A	newdir/interesting
+	:000000 100755 $ZERO_OID $f6id A	newdir/exec.sh
+	:000000 100644 $ZERO_OID $f5id A	newdir/interesting
 	EOF
 	git diff-tree -M -r branch^ branch >actual &&
 	compare_diff_raw expect actual
@@ -2733,7 +2733,7 @@ test_expect_success 'R: corrupt lines do not mess marks file' '
 	rm -f io.marks &&
 	blob=$(echo hi | git hash-object --stdin) &&
 	cat >expect <<-EOF &&
-	:3 0000000000000000000000000000000000000000
+	:3 $ZERO_OID
 	:1 $blob
 	:2 $blob
 	EOF
@@ -3084,7 +3084,7 @@ test_expect_success 'T: delete branch' '
 	git branch to-delete &&
 	git fast-import <<-EOF &&
 	reset refs/heads/to-delete
-	from 0000000000000000000000000000000000000000
+	from $ZERO_OID
 	EOF
 	test_must_fail git rev-parse --verify refs/heads/to-delete
 '
brian m. carlson· Jul 29, 2020, 23:14 UTC · re: brian m. carlson · lore

[PATCH v6 25/39] t9500: ensure that algorithm info is preserved in config

When we use a hash algorithm other than SHA-1, it's important to preserve the hash-related values in the config file, but this test overwrites the config file with a new one. Ensure we copy these values properly from the old config to the new one so that the repository can be read if it's using SHA-256.

Note that if there is no extensions.objectFormat value set, git config will return unsuccessfully if we try to read it; since this is not an error for us, use test_might_fail.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t9500-gitweb-standalone-no-errors.sh | 22 ++++++++++++++++------
 1 file changed, 16 insertions(+), 6 deletions(-)
Show changes to t/t9500-gitweb-standalone-no-errors.sh +16 −6
diff --git a/t/t9500-gitweb-standalone-no-errors.sh b/t/t9500-gitweb-standalone-no-errors.sh
index 267ddc997d..b484e3e250 100755
--- a/t/t9500-gitweb-standalone-no-errors.sh
+++ b/t/t9500-gitweb-standalone-no-errors.sh
@@ -621,12 +621,22 @@ test_expect_success \
 	 git config gitweb.snapshot "zip,tgz, tbz2" &&
 	 gitweb_run "p=.git;a=tree"'
 
-cat >.git/config <<\EOF
-# testing noval and alternate separator
-[gitweb]
-	blame
-	snapshot = zip tgz
-EOF
+test_expect_success 'setup' '
+	version=$(git config core.repositoryformatversion) &&
+	algo=$(test_might_fail git config extensions.objectformat) &&
+	cat >.git/config <<-\EOF &&
+	# testing noval and alternate separator
+	[gitweb]
+		blame
+		snapshot = zip tgz
+	EOF
+	git config core.repositoryformatversion "$version" &&
+	if test -n "$algo"
+	then
+		git config extensions.objectformat "$algo"
+	fi
+'
+
 test_expect_success \
 	'config override: tree view, features enabled in repo config (2)' \
 	'gitweb_run "p=.git;a=tree"'
brian m. carlson· Jul 29, 2020, 23:14 UTC · re: brian m. carlson · lore

[PATCH v6 27/39] t5308: make test work with SHA-256

This test needs multiple object IDs that have the same first byte. Update the pack test code to generate a suitable packed value for SHA-256. Update the test to use this value when using SHA-256.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/lib-pack.sh                     |  8 ++++++++
 t/t5308-pack-detect-duplicates.sh | 20 ++++++++++++--------
 2 files changed, 20 insertions(+), 8 deletions(-)
Show changes to 2 files +20 −8

t/lib-pack.sh, t/t5308-pack-detect-duplicates.sh

diff --git a/t/lib-pack.sh b/t/lib-pack.sh
index f3463170b3..0c799d53c6 100644
--- a/t/lib-pack.sh
+++ b/t/lib-pack.sh
@@ -93,6 +93,14 @@ pack_obj () {
 			;;
 		esac
 		;;
+	# blob containing "\3\326"
+	471819e8c52bf11513f100b2810a8aa0622d5cd3d1c913758a071dd4b3bad8fe)
+		case "$2" in
+		'')
+			printf '\062\170\234\143\276\006\000\000\336\000\332'
+			return
+			;;
+		esac
 	esac
 
 	# If it's not a delta, we can convince pack-objects to generate a pack
diff --git a/t/t5308-pack-detect-duplicates.sh b/t/t5308-pack-detect-duplicates.sh
index 6845c1f3c3..693b2411c8 100755
--- a/t/t5308-pack-detect-duplicates.sh
+++ b/t/t5308-pack-detect-duplicates.sh
@@ -4,23 +4,27 @@ test_description='handling of duplicate objects in incoming packfiles'
 . ./test-lib.sh
 . "$TEST_DIRECTORY"/lib-pack.sh
 
-if ! test_have_prereq SHA1
-then
-       skip_all='not using SHA-1 for objects'
-       test_done
-fi
+test_expect_success 'setup' '
+	test_oid_cache <<-EOF
+	lo_oid sha1:e68fe8129b546b101aee9510c5328e7f21ca1d18
+	lo_oid sha256:471819e8c52bf11513f100b2810a8aa0622d5cd3d1c913758a071dd4b3bad8fe
+
+	missing_oid sha1:e69d000000000000000000000000000000000000
+	missing_oid sha256:4720000000000000000000000000000000000000000000000000000000000000
+	EOF
+'
 
 # The sha1s we have in our pack. It's important that these have the same
 # starting byte, so that they end up in the same fanout section of the index.
 # That lets us make sure we are exercising the binary search with both sets.
-LO_SHA1=e68fe8129b546b101aee9510c5328e7f21ca1d18
-HI_SHA1=e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
+LO_SHA1=$(test_oid lo_oid)
+HI_SHA1=$EMPTY_BLOB
 
 # And here's a "missing sha1" which will produce failed lookups. It must also
 # be in the same fanout section, and should be between the two (so that during
 # our binary search, we are sure to end up looking at one or the other of the
 # duplicate runs).
-MISSING_SHA1='e69d000000000000000000000000000000000000'
+MISSING_SHA1=$(test_oid missing_oid)
 
 # git will never intentionally create packfiles with
 # duplicate objects, so we have to construct them by hand.
brian m. carlson· Jul 29, 2020, 23:14 UTC · re: brian m. carlson · lore

[PATCH v6 26/39] t9700: make hash size independent

The Perl test script for t9700 was matching on exactly 40 hex characters. With SHA-256, we'll have 64 hex-character object IDs. Create a variable with a regex which matches exactly 40 or 64 hex characters and use that to match the output. Note that both of the uses of this can be anchored, which makes the code simpler, so do that as well.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t9700/test.pl | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
Show changes to t/t9700/test.pl +4 −2
diff --git a/t/t9700/test.pl b/t/t9700/test.pl
index 34cd01366f..071ff6d82a 100755
--- a/t/t9700/test.pl
+++ b/t/t9700/test.pl
@@ -23,6 +23,8 @@ sub adjust_dirsep {
 	return $path;
 }
 
+my $oid_re = qr/^[0-9a-fA-F]{40}(?:[0-9a-fA-F]{24})?$/;
+
 BEGIN { use_ok('Git') }
 
 # set up
@@ -93,7 +95,7 @@ sub adjust_dirsep {
 open TEMPFILE, ">$tmpfile" or die "Can't open $tmpfile: $!";
 print TEMPFILE my $test_text = "test blob, to be inserted\n";
 close TEMPFILE or die "Failed writing to $tmpfile: $!";
-like(our $newhash = $r->hash_and_insert_object($tmpfile), qr/[0-9a-fA-F]{40}/,
+like(our $newhash = $r->hash_and_insert_object($tmpfile), $oid_re,
      "hash_and_insert_object: returns hash");
 open TEMPFILE, "+>$tmpfile" or die "Can't open $tmpfile: $!";
 is($r->cat_blob($newhash, \*TEMPFILE), length $test_text, "cat_blob: roundtrip size");
@@ -119,7 +121,7 @@ sub adjust_dirsep {
 
 # commands in sub directory
 my $last_commit = $r2->command_oneline(qw(rev-parse --verify HEAD));
-like($last_commit, qr/^[0-9a-fA-F]{40}$/, 'rev-parse returned hash');
+like($last_commit, $oid_re, 'rev-parse returned hash');
 my $dir_commit = $r2->command_oneline('log', '-n1', '--pretty=format:%H', '.');
 isnt($last_commit, $dir_commit, 'log . does not show last commit');
 
brian m. carlson· Jul 29, 2020, 23:14 UTC · re: brian m. carlson · lore

[PATCH v6 31/39] bundle: add new version for use with SHA-256

Currently we detect the hash algorithm in use by the length of the object ID. This is inelegant and prevents us from using a different hash algorithm that is also 256 bits in length.

Since we cannot extend the v2 format in a backward-compatible way, let's add a v3 format, which is identical, except for the addition of capabilities, which are prefixed by an at sign. We add "object-format" as the only capability and reject unknown capabilities, since we do not have a network connection and therefore cannot negotiate with the other side.

For compatibility, default to the v2 format for SHA-1 and require v3 for SHA-256.

In t5510, always use format v3 so we can be sure we produce consistent results across hash algorithms. Since head -n N lists the top N lines instead of the Nth line, let's run our output through sed to normalize it and compare it against a fixed value, which will make sure we get exactly what we're expecting.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 Documentation/git-bundle.txt              |  9 ++-
 Documentation/technical/bundle-format.txt | 30 +++++++-
 builtin/bundle.c                          |  5 +-
 bundle.c                                  | 83 +++++++++++++++++------
 bundle.h                                  |  4 +-
 t/t5510-fetch.sh                          | 16 +++--
 t/t5607-clone-bundle.sh                   | 31 +++++++++
 7 files changed, 147 insertions(+), 31 deletions(-)
Show changes to 7 files +147 −31

Documentation/git-bundle.txt, Documentation/technical/bundle-format.txt, builtin/bundle.c, bundle.c, bundle.h, t/t5510-fetch.sh, t/t5607-clone-bundle.sh

diff --git a/Documentation/git-bundle.txt b/Documentation/git-bundle.txt
index d34b0964be..53804cad4b 100644
--- a/Documentation/git-bundle.txt
+++ b/Documentation/git-bundle.txt
@@ -9,7 +9,8 @@ git-bundle - Move objects and refs by archive
 SYNOPSIS
 --------
 [verse]
-'git bundle' create [-q | --quiet | --progress | --all-progress] [--all-progress-implied] <file> <git-rev-list-args>
+'git bundle' create [-q | --quiet | --progress | --all-progress] [--all-progress-implied]
+		    [--version=<version>] <file> <git-rev-list-args>
 'git bundle' verify [-q | --quiet] <file>
 'git bundle' list-heads <file> [<refname>...]
 'git bundle' unbundle <file> [<refname>...]
@@ -102,6 +103,12 @@ unbundle <file>::
 	is activated.  Unlike --all-progress this flag doesn't actually
 	force any progress display by itself.
 
+--version=<version>::
+	Specify the bundle version.  Version 2 is the older format and can only be
+	used with SHA-1 repositories; the newer version 3 contains capabilities that
+	permit extensions. The default is the oldest supported format, based on the
+	hash algorithm in use.
+
 -q::
 --quiet::
 	This flag makes the command not to report its progress
diff --git a/Documentation/technical/bundle-format.txt b/Documentation/technical/bundle-format.txt
index 0e828151a5..bac558d049 100644
--- a/Documentation/technical/bundle-format.txt
+++ b/Documentation/technical/bundle-format.txt
@@ -7,6 +7,8 @@ The Git bundle format is a format that represents both refs and Git objects.
 We will use ABNF notation to define the Git bundle format. See
 protocol-common.txt for the details.
 
+A v2 bundle looks like this:
+
 ----
 bundle    = signature *prerequisite *reference LF pack
 signature = "# v2 git bundle" LF
@@ -18,9 +20,28 @@ reference    = obj-id SP refname LF
 pack         = ... ; packfile
 ----
 
+A v3 bundle looks like this:
+
+----
+bundle    = signature *capability *prerequisite *reference LF pack
+signature = "# v3 git bundle" LF
+
+capability   = "@" key ["=" value] LF
+prerequisite = "-" obj-id SP comment LF
+comment      = *CHAR
+reference    = obj-id SP refname LF
+key          = 1*(ALPHA / DIGIT / "-")
+value        = *(%01-09 / %0b-FF)
+
+pack         = ... ; packfile
+----
+
 == Semantics
 
-A Git bundle consists of three parts.
+A Git bundle consists of several parts.
+
+* "Capabilities", which are only in the v3 format, indicate functionality that
+	the bundle requires to be read properly.
 
 * "Prerequisites" lists the objects that are NOT included in the bundle and the
   reader of the bundle MUST already have, in order to use the data in the
@@ -46,3 +67,10 @@ put any string here. The reader of the bundle MUST ignore the comment.
 Note that the prerequisites does not represent a shallow-clone boundary. The
 semantics of the prerequisites and the shallow-clone boundaries are different,
 and the Git bundle v2 format cannot represent a shallow clone repository.
+
+== Capabilities
+
+Because there is no opportunity for negotiation, unknown capabilities cause 'git
+bundle' to abort.  The only known capability is `object-format`, which specifies
+the hash algorithm in use, and can take the same values as the
+`extensions.objectFormat` configuration value.
diff --git a/builtin/bundle.c b/builtin/bundle.c
index f049d27a14..e1a85e7dcc 100644
--- a/builtin/bundle.c
+++ b/builtin/bundle.c
@@ -60,6 +60,7 @@ static int cmd_bundle_create(int argc, const char **argv, const char *prefix) {
 	int all_progress_implied = 0;
 	int progress = isatty(STDERR_FILENO);
 	struct argv_array pack_opts;
+	int version = -1;
 
 	struct option options[] = {
 		OPT_SET_INT('q', "quiet", &progress,
@@ -71,6 +72,8 @@ static int cmd_bundle_create(int argc, const char **argv, const char *prefix) {
 		OPT_BOOL(0, "all-progress-implied",
 			 &all_progress_implied,
 			 N_("similar to --all-progress when progress meter is shown")),
+		OPT_INTEGER(0, "version", &version,
+			    N_("specify bundle format version")),
 		OPT_END()
 	};
 	const char* bundle_file;
@@ -91,7 +94,7 @@ static int cmd_bundle_create(int argc, const char **argv, const char *prefix) {
 
 	if (!startup_info->have_repository)
 		die(_("Need a repository to create a bundle."));
-	return !!create_bundle(the_repository, bundle_file, argc, argv, &pack_opts);
+	return !!create_bundle(the_repository, bundle_file, argc, argv, &pack_opts, version);
 }
 
 static int cmd_bundle_verify(int argc, const char **argv, const char *prefix) {
diff --git a/bundle.c b/bundle.c
index 2a0d744d3f..35585f237c 100644
--- a/bundle.c
+++ b/bundle.c
@@ -12,7 +12,16 @@
 #include "refs.h"
 #include "argv-array.h"
 
-static const char bundle_signature[] = "# v2 git bundle\n";
+
+static const char v2_bundle_signature[] = "# v2 git bundle\n";
+static const char v3_bundle_signature[] = "# v3 git bundle\n";
+static struct {
+	int version;
+	const char *signature;
+} bundle_sigs[] = {
+	{ 2, v2_bundle_signature },
+	{ 3, v3_bundle_signature },
+};
 
 static void add_to_ref_list(const struct object_id *oid, const char *name,
 		struct ref_list *list)
@@ -23,15 +32,30 @@ static void add_to_ref_list(const struct object_id *oid, const char *name,
 	list->nr++;
 }
 
-static const struct git_hash_algo *detect_hash_algo(struct strbuf *buf)
+static int parse_capability(struct bundle_header *header, const char *capability)
 {
-	size_t len = strcspn(buf->buf, " \n");
-	int algo;
+	const char *arg;
+	if (skip_prefix(capability, "object-format=", &arg)) {
+		int algo = hash_algo_by_name(arg);
+		if (algo == GIT_HASH_UNKNOWN)
+			return error(_("unrecognized bundle hash algorithm: %s"), arg);
+		header->hash_algo = &hash_algos[algo];
+		return 0;
+	}
+	return error(_("unknown capability '%s'"), capability);
+}
 
-	algo = hash_algo_by_length(len / 2);
-	if (algo == GIT_HASH_UNKNOWN)
-		return NULL;
-	return &hash_algos[algo];
+static int parse_bundle_signature(struct bundle_header *header, const char *line)
+{
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(bundle_sigs); i++) {
+		if (!strcmp(line, bundle_sigs[i].signature)) {
+			header->version = bundle_sigs[i].version;
+			return 0;
+		}
+	}
+	return -1;
 }
 
 static int parse_bundle_header(int fd, struct bundle_header *header,
@@ -42,14 +66,16 @@ static int parse_bundle_header(int fd, struct bundle_header *header,
 
 	/* The bundle header begins with the signature */
 	if (strbuf_getwholeline_fd(&buf, fd, '\n') ||
-	    strcmp(buf.buf, bundle_signature)) {
+	    parse_bundle_signature(header, buf.buf)) {
 		if (report_path)
-			error(_("'%s' does not look like a v2 bundle file"),
+			error(_("'%s' does not look like a v2 or v3 bundle file"),
 			      report_path);
 		status = -1;
 		goto abort;
 	}
 
+	header->hash_algo = the_hash_algo;
+
 	/* The bundle header ends with an empty line */
 	while (!strbuf_getwholeline_fd(&buf, fd, '\n') &&
 	       buf.len && buf.buf[0] != '\n') {
@@ -57,19 +83,19 @@ static int parse_bundle_header(int fd, struct bundle_header *header,
 		int is_prereq = 0;
 		const char *p;
 
-		if (*buf.buf == '-') {
-			is_prereq = 1;
-			strbuf_remove(&buf, 0, 1);
-		}
 		strbuf_rtrim(&buf);
 
-		if (!header->hash_algo) {
-			header->hash_algo = detect_hash_algo(&buf);
-			if (!header->hash_algo) {
-				error(_("unknown hash algorithm length"));
+		if (header->version == 3 && *buf.buf == '@') {
+			if (parse_capability(header, buf.buf + 1)) {
 				status = -1;
 				break;
 			}
+			continue;
+		}
+
+		if (*buf.buf == '-') {
+			is_prereq = 1;
+			strbuf_remove(&buf, 0, 1);
 		}
 
 		/*
@@ -449,13 +475,14 @@ static int write_bundle_refs(int bundle_fd, struct rev_info *revs)
 }
 
 int create_bundle(struct repository *r, const char *path,
-		  int argc, const char **argv, struct argv_array *pack_options)
+		  int argc, const char **argv, struct argv_array *pack_options, int version)
 {
 	struct lock_file lock = LOCK_INIT;
 	int bundle_fd = -1;
 	int bundle_to_stdout;
 	int ref_count = 0;
 	struct rev_info revs;
+	int min_version = the_hash_algo == &hash_algos[GIT_HASH_SHA1] ? 2 : 3;
 
 	bundle_to_stdout = !strcmp(path, "-");
 	if (bundle_to_stdout)
@@ -464,8 +491,22 @@ int create_bundle(struct repository *r, const char *path,
 		bundle_fd = hold_lock_file_for_update(&lock, path,
 						      LOCK_DIE_ON_ERROR);
 
-	/* write signature */
-	write_or_die(bundle_fd, bundle_signature, strlen(bundle_signature));
+	if (version == -1)
+		version = min_version;
+
+	if (version < 2 || version > 3) {
+		die(_("unsupported bundle version %d"), version);
+	} else if (version < min_version) {
+		die(_("cannot write bundle version %d with algorithm %s"), version, the_hash_algo->name);
+	} else if (version == 2) {
+		write_or_die(bundle_fd, v2_bundle_signature, strlen(v2_bundle_signature));
+	} else {
+		const char *capability = "@object-format=";
+		write_or_die(bundle_fd, v3_bundle_signature, strlen(v3_bundle_signature));
+		write_or_die(bundle_fd, capability, strlen(capability));
+		write_or_die(bundle_fd, the_hash_algo->name, strlen(the_hash_algo->name));
+		write_or_die(bundle_fd, "\n", 1);
+	}
 
 	/* init revs to list objects for pack-objects later */
 	save_commit_buffer = 0;
diff --git a/bundle.h b/bundle.h
index 2dc9442024..70c84cab08 100644
--- a/bundle.h
+++ b/bundle.h
@@ -13,6 +13,7 @@ struct ref_list {
 };
 
 struct bundle_header {
+	unsigned version;
 	struct ref_list prerequisites;
 	struct ref_list references;
 	const struct git_hash_algo *hash_algo;
@@ -21,7 +22,8 @@ struct bundle_header {
 int is_bundle(const char *path, int quiet);
 int read_bundle_header(const char *path, struct bundle_header *header);
 int create_bundle(struct repository *r, const char *path,
-		  int argc, const char **argv, struct argv_array *pack_options);
+		  int argc, const char **argv, struct argv_array *pack_options,
+		  int version);
 int verify_bundle(struct repository *r, struct bundle_header *header, int verbose);
 #define BUNDLE_VERBOSE 1
 int unbundle(struct repository *r, struct bundle_header *header,
diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index a66dbe0bde..9243335ab0 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -281,15 +281,19 @@ test_expect_success 'create bundle 1' '
 	cd "$D" &&
 	echo >file updated again by origin &&
 	git commit -a -m "tip" &&
-	git bundle create bundle1 master^..master
+	git bundle create --version=3 bundle1 master^..master
 '
 
 test_expect_success 'header of bundle looks right' '
-	head -n 4 "$D"/bundle1 &&
-	head -n 1 "$D"/bundle1 | grep "^#" &&
-	head -n 2 "$D"/bundle1 | grep "^-$OID_REGEX " &&
-	head -n 3 "$D"/bundle1 | grep "^$OID_REGEX " &&
-	head -n 4 "$D"/bundle1 | grep "^$"
+	cat >expect <<-EOF &&
+	# v3 git bundle
+	@object-format=$(test_oid algo)
+	-OID updated by origin
+	OID refs/heads/master
+
+	EOF
+	sed -e "s/$OID_REGEX/OID/g" -e "5q" "$D"/bundle1 >actual &&
+	test_cmp expect actual
 '
 
 test_expect_success 'create bundle 2' '
diff --git a/t/t5607-clone-bundle.sh b/t/t5607-clone-bundle.sh
index 6d5a977fcb..26985f4b44 100755
--- a/t/t5607-clone-bundle.sh
+++ b/t/t5607-clone-bundle.sh
@@ -4,6 +4,10 @@ test_description='some bundle related tests'
 . ./test-lib.sh
 
 test_expect_success 'setup' '
+	test_oid_cache <<-EOF &&
+	version sha1:2
+	version sha256:3
+	EOF
 	test_commit initial &&
 	test_tick &&
 	git tag -m tag tag &&
@@ -94,4 +98,31 @@ test_expect_success 'fetch SHA-1 from bundle' '
 	git fetch --no-tags foo/tip.bundle "$(cat hash)"
 '
 
+test_expect_success 'git bundle uses expected default format' '
+	git bundle create bundle HEAD^.. &&
+	head -n1 bundle | grep "^# v$(test_oid version) git bundle$"
+'
+
+test_expect_success 'git bundle v3 has expected contents' '
+	git branch side HEAD &&
+	git bundle create --version=3 bundle HEAD^..side &&
+	head -n2 bundle >actual &&
+	cat >expect <<-EOF &&
+	# v3 git bundle
+	@object-format=$(test_oid algo)
+	EOF
+	test_cmp expect actual &&
+	git bundle verify bundle
+'
+
+test_expect_success 'git bundle v3 rejects unknown capabilities' '
+	cat >new <<-EOF &&
+	# v3 git bundle
+	@object-format=$(test_oid algo)
+	@unknown=silly
+	EOF
+	test_must_fail git bundle verify new 2>output &&
+	test_i18ngrep "unknown capability .unknown=silly." output
+'
+
 test_done
brian m. carlson· Jul 29, 2020, 23:14 UTC · re: brian m. carlson · lore

[PATCH v6 38/39] docs: add documentation for extensions.objectFormat

Document the extensions.objectFormat config setting. Warn users not to modify it themselves.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 Documentation/config.txt            | 2 ++
 Documentation/config/extensions.txt | 8 ++++++++
 2 files changed, 10 insertions(+)
 create mode 100644 Documentation/config/extensions.txt
Show changes to 2 files +10 −0

Documentation/config.txt, Documentation/config/extensions.txt

diff --git a/Documentation/config.txt b/Documentation/config.txt
index ef0768b91a..3042d80978 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -348,6 +348,8 @@ include::config/diff.txt[]
 
 include::config/difftool.txt[]
 
+include::config/extensions.txt[]
+
 include::config/fastimport.txt[]
 
 include::config/feature.txt[]
diff --git a/Documentation/config/extensions.txt b/Documentation/config/extensions.txt
new file mode 100644
index 0000000000..4e23d73cdc
--- /dev/null
+++ b/Documentation/config/extensions.txt
@@ -0,0 +1,8 @@
+extensions.objectFormat::
+	Specify the hash algorithm to use.  The acceptable values are `sha1` and
+	`sha256`.  If not specified, `sha1` is assumed.  It is an error to specify
+	this key unless `core.repositoryFormatVersion` is 1.
++
+Note that this setting should only be set by linkgit:git-init[1] or
+linkgit:git-clone[1].  Trying to change it after initialization will not
+work and will produce hard-to-diagnose issues.
brian m. carlson· Jul 29, 2020, 23:14 UTC · re: brian m. carlson · lore

[PATCH v6 33/39] repository: enable SHA-256 support by default

Now that we have a complete SHA-256 implementation in Git, let's enable it so people can use it. Remove the ENABLE_SHA256 define constant everywhere it's used. Add tests for initializing a repository with SHA-256.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 builtin/init-db.c |  5 -----
 config.mak.dev    |  2 --
 repository.c      |  4 ----
 t/t0001-init.sh   | 33 +++++++++++++++++++++++++++++++++
 4 files changed, 33 insertions(+), 11 deletions(-)
Show changes to 4 files +33 −11

builtin/init-db.c, config.mak.dev, repository.c, t/t0001-init.sh

diff --git a/builtin/init-db.c b/builtin/init-db.c
index cee64823cb..f70076d38e 100644
--- a/builtin/init-db.c
+++ b/builtin/init-db.c
@@ -183,11 +183,6 @@ void initialize_repository_version(int hash_algo)
 	char repo_version_string[10];
 	int repo_version = GIT_REPO_VERSION;
 
-#ifndef ENABLE_SHA256
-	if (hash_algo != GIT_HASH_SHA1)
-		die(_("The hash algorithm %s is not supported in this build."), hash_algos[hash_algo].name);
-#endif
-
 	if (hash_algo != GIT_HASH_SHA1)
 		repo_version = GIT_REPO_VERSION_READ;
 
diff --git a/config.mak.dev b/config.mak.dev
index cd4a82a9eb..89b218d11a 100644
--- a/config.mak.dev
+++ b/config.mak.dev
@@ -16,8 +16,6 @@ DEVELOPER_CFLAGS += -Wstrict-prototypes
 DEVELOPER_CFLAGS += -Wunused
 DEVELOPER_CFLAGS += -Wvla
 
-DEVELOPER_CFLAGS += -DENABLE_SHA256
-
 ifndef COMPILER_FEATURES
 COMPILER_FEATURES := $(shell ./detect-compiler $(CC))
 endif
diff --git a/repository.c b/repository.c
index 6f7f6f002b..a4174ddb06 100644
--- a/repository.c
+++ b/repository.c
@@ -89,10 +89,6 @@ void repo_set_gitdir(struct repository *repo,
 void repo_set_hash_algo(struct repository *repo, int hash_algo)
 {
 	repo->hash_algo = &hash_algos[hash_algo];
-#ifndef ENABLE_SHA256
-	if (hash_algo != GIT_HASH_SHA1)
-		die(_("The hash algorithm %s is not supported in this build."), repo->hash_algo->name);
-#endif
 }
 
 /*
diff --git a/t/t0001-init.sh b/t/t0001-init.sh
index 6d2467995e..d71d4c7238 100755
--- a/t/t0001-init.sh
+++ b/t/t0001-init.sh
@@ -441,6 +441,39 @@ test_expect_success 're-init from a linked worktree' '
 	)
 '
 
+test_expect_success 'init honors GIT_DEFAULT_HASH' '
+	GIT_DEFAULT_HASH=sha1 git init sha1 &&
+	git -C sha1 rev-parse --show-object-format >actual &&
+	echo sha1 >expected &&
+	test_cmp expected actual &&
+	GIT_DEFAULT_HASH=sha256 git init sha256 &&
+	git -C sha256 rev-parse --show-object-format >actual &&
+	echo sha256 >expected &&
+	test_cmp expected actual
+'
+
+test_expect_success 'init honors --object-format' '
+	git init --object-format=sha1 explicit-sha1 &&
+	git -C explicit-sha1 rev-parse --show-object-format >actual &&
+	echo sha1 >expected &&
+	test_cmp expected actual &&
+	git init --object-format=sha256 explicit-sha256 &&
+	git -C explicit-sha256 rev-parse --show-object-format >actual &&
+	echo sha256 >expected &&
+	test_cmp expected actual
+'
+
+test_expect_success 'extensions.objectFormat is not allowed with repo version 0' '
+	git init --object-format=sha256 explicit-v0 &&
+	git -C explicit-v0 config core.repositoryformatversion 0 &&
+	test_must_fail git -C explicit-v0 rev-parse --show-object-format
+'
+
+test_expect_success 'init rejects attempts to initialize with different hash' '
+	test_must_fail git -C sha1 init --object-format=sha256 &&
+	test_must_fail git -C sha256 init --object-format=sha1
+'
+
 test_expect_success MINGW 'core.hidedotfiles = false' '
 	git config --global core.hidedotfiles false &&
 	rm -rf newdir &&
brian m. carlson· Jul 29, 2020, 23:14 UTC · re: brian m. carlson · lore

[PATCH v6 37/39] ci: run tests with SHA-256

Now that we have Git supporting SHA-256, we'd like to make sure that we don't regress that state. Unfortunately, it's easy to do so, so to help, let's add code to run one of our CI jobs with SHA-256 as the default hash. This will help us detect any problems that may occur.

We pick the linux-clang job because it's relatively fast and the linux-gcc job already runs the testsuite twice. We want our tests to run as fast as possible, so we wouldn't want to add a third run to the linux-gcc job. To make sure we properly exercise the code, let's run the tests in the default mode (SHA-1) first and then run a second time with SHA-256. We explicitly specify SHA-1 for the first run so that if we change the default in the future, we make sure to test both cases.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 ci/run-build-and-tests.sh | 6 ++++++
 1 file changed, 6 insertions(+)
Show changes to ci/run-build-and-tests.sh +6 −0
diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh
index 17e25aade9..6c27b886b8 100755
--- a/ci/run-build-and-tests.sh
+++ b/ci/run-build-and-tests.sh
@@ -24,6 +24,12 @@ linux-gcc)
 	export GIT_TEST_ADD_I_USE_BUILTIN=1
 	make test
 	;;
+linux-clang)
+	export GIT_TEST_DEFAULT_HASH=sha1
+	make test
+	export GIT_TEST_DEFAULT_HASH=sha256
+	make test
+	;;
 linux-gcc-4.8)
 	# Don't run the tests; we only care about whether Git can be
 	# built with GCC 4.8, as it errors out on some undesired (C99)
brian m. carlson· Jul 29, 2020, 23:14 UTC · re: brian m. carlson · lore

[PATCH v6 30/39] builtin/verify-pack: implement an --object-format option

A recently added test in t5702 started using git verify-pack outside of a repository. While this poses no problems with SHA-1, with SHA-256 we implicitly rely on the setup of the repository to initialize our hash algorithm settings.

Since we're not in a repository here, we need to provide git verify-pack help to set things up properly. git index-pack already knows an --object-format option, so let's accept one as well and pass it down to our git index-pack invocation. Since we're now dynamically adjusting the elements in argv, let's switch to using struct argv_array to manage them. Finally, let's make t5702 pass the proper argument on down to its git verify-pack caller.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 builtin/verify-pack.c  | 23 +++++++++++++++--------
 t/t5702-protocol-v2.sh |  2 +-
 2 files changed, 16 insertions(+), 9 deletions(-)
Show changes to 2 files +16 −9

builtin/verify-pack.c, t/t5702-protocol-v2.sh

diff --git a/builtin/verify-pack.c b/builtin/verify-pack.c
index c2a1a5c504..3669a90263 100644
--- a/builtin/verify-pack.c
+++ b/builtin/verify-pack.c
@@ -7,21 +7,26 @@
 #define VERIFY_PACK_VERBOSE 01
 #define VERIFY_PACK_STAT_ONLY 02
 
-static int verify_one_pack(const char *path, unsigned int flags)
+static int verify_one_pack(const char *path, unsigned int flags, const char *hash_algo)
 {
 	struct child_process index_pack = CHILD_PROCESS_INIT;
-	const char *argv[] = {"index-pack", NULL, NULL, NULL };
+	struct argv_array *argv = &index_pack.args;
 	struct strbuf arg = STRBUF_INIT;
 	int verbose = flags & VERIFY_PACK_VERBOSE;
 	int stat_only = flags & VERIFY_PACK_STAT_ONLY;
 	int err;
 
+	argv_array_push(argv, "index-pack");
+
 	if (stat_only)
-		argv[1] = "--verify-stat-only";
+		argv_array_push(argv, "--verify-stat-only");
 	else if (verbose)
-		argv[1] = "--verify-stat";
+		argv_array_push(argv, "--verify-stat");
 	else
-		argv[1] = "--verify";
+		argv_array_push(argv, "--verify");
+
+	if (hash_algo)
+		argv_array_pushf(argv, "--object-format=%s", hash_algo);
 
 	/*
 	 * In addition to "foo.pack" we accept "foo.idx" and "foo";
@@ -31,9 +36,8 @@ static int verify_one_pack(const char *path, unsigned int flags)
 	if (strbuf_strip_suffix(&arg, ".idx") ||
 	    !ends_with(arg.buf, ".pack"))
 		strbuf_addstr(&arg, ".pack");
-	argv[2] = arg.buf;
+	argv_array_push(argv, arg.buf);
 
-	index_pack.argv = argv;
 	index_pack.git_cmd = 1;
 
 	err = run_command(&index_pack);
@@ -60,12 +64,15 @@ int cmd_verify_pack(int argc, const char **argv, const char *prefix)
 {
 	int err = 0;
 	unsigned int flags = 0;
+	const char *object_format = NULL;
 	int i;
 	const struct option verify_pack_options[] = {
 		OPT_BIT('v', "verbose", &flags, N_("verbose"),
 			VERIFY_PACK_VERBOSE),
 		OPT_BIT('s', "stat-only", &flags, N_("show statistics only"),
 			VERIFY_PACK_STAT_ONLY),
+		OPT_STRING(0, "object-format", &object_format, N_("hash"),
+			   N_("specify the hash algorithm to use")),
 		OPT_END()
 	};
 
@@ -75,7 +82,7 @@ int cmd_verify_pack(int argc, const char **argv, const char *prefix)
 	if (argc < 1)
 		usage_with_options(verify_pack_usage, verify_pack_options);
 	for (i = 0; i < argc; i++) {
-		if (verify_one_pack(argv[i], flags))
+		if (verify_one_pack(argv[i], flags, object_format))
 			err = 1;
 	}
 
diff --git a/t/t5702-protocol-v2.sh b/t/t5702-protocol-v2.sh
index 1b54c35b01..7fc22171e7 100755
--- a/t/t5702-protocol-v2.sh
+++ b/t/t5702-protocol-v2.sh
@@ -829,7 +829,7 @@ test_expect_success 'part of packfile response provided as URI' '
 	# Ensure that my-blob and other-blob are in separate packfiles.
 	for idx in http_child/.git/objects/pack/*.idx
 	do
-		git verify-pack --verbose $idx >out &&
+		git verify-pack --object-format=$(test_oid algo) --verbose $idx >out &&
 		{
 			grep "^[0-9a-f]\{16,\} " out || :
 		} >out.objectlist &&
brian m. carlson· Jul 29, 2020, 23:14 UTC · re: brian m. carlson · lore

[PATCH v6 35/39] t: allow testing different hash algorithms via environment

To allow developers to run the testsuite with a different algorithm than the default, provide an environment variable, GIT_TEST_DEFAULT_HASH, to specify the algorithm to use. Compute the fixed constants using test_oid. Move the constant initialization down below the point where test-lib-functions.sh is loaded so the functions are defined.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/test-lib-functions.sh | 4 +---
 t/test-lib.sh           | 3 +++
 2 files changed, 4 insertions(+), 3 deletions(-)
Show changes to 2 files +4 −3

t/test-lib-functions.sh, t/test-lib.sh

diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh
index 2608e80f11..2e01bb2c2b 100644
--- a/t/test-lib-functions.sh
+++ b/t/test-lib-functions.sh
@@ -1417,9 +1417,7 @@ test_set_hash () {
 
 # Detect the hash algorithm in use.
 test_detect_hash () {
-	# Currently we only support SHA-1, but in the future this function will
-	# actually detect the algorithm in use.
-	test_hash_algo='sha1'
+	test_hash_algo="${GIT_TEST_DEFAULT_HASH:-sha1}"
 }
 
 # Load common hash metadata and common placeholder object IDs for use with
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 618a7c8d5b..307bb2207e 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -452,6 +452,9 @@ export GIT_AUTHOR_EMAIL GIT_AUTHOR_NAME
 export GIT_COMMITTER_EMAIL GIT_COMMITTER_NAME
 export EDITOR
 
+GIT_DEFAULT_HASH="${GIT_TEST_DEFAULT_HASH:-sha1}"
+export GIT_DEFAULT_HASH
+
 # Tests using GIT_TRACE typically don't want <timestamp> <file>:<line> output
 GIT_TRACE_BARE=1
 export GIT_TRACE_BARE
brian m. carlson· Jul 29, 2020, 23:14 UTC · re: brian m. carlson · lore

[PATCH v6 32/39] setup: add support for reading extensions.objectformat

The transition plan specifies extensions.objectFormat as the indication that we're using a given hash in a certain repo. Read this as one of the extensions we support. If the user has specified an invalid value, fail.

Ensure that we reject the extension if the repository format version is 0.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 setup.c | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)
Show changes to setup.c +15 −1
diff --git a/setup.c b/setup.c
index 3a81307602..94e68bb4f4 100644
--- a/setup.c
+++ b/setup.c
@@ -470,7 +470,16 @@ static int check_repo_format(const char *var, const char *value, void *vdata)
 			data->partial_clone = xstrdup(value);
 		} else if (!strcmp(ext, "worktreeconfig"))
 			data->worktree_config = git_config_bool(var, value);
-		else
+		else if (!strcmp(ext, "objectformat")) {
+			int format;
+
+			if (!value)
+				return config_error_nonbool(var);
+			format = hash_algo_by_name(value);
+			if (format == GIT_HASH_UNKNOWN)
+				return error("invalid value for 'extensions.objectformat'");
+			data->hash_algo = format;
+		} else
 			string_list_append(&data->unknown_extensions, ext);
 	}
 
@@ -613,6 +622,11 @@ int verify_repository_format(const struct repository_format *format,
 		return -1;
 	}
 
+	if (format->version <= 0 && format->hash_algo != GIT_HASH_SHA1) {
+		strbuf_addstr(err, _("extensions.objectFormat is not valid in repo v0"));
+		return -1;
+	}
+
 	return 0;
 }
 
brian m. carlson· Jul 29, 2020, 23:14 UTC · re: brian m. carlson · lore

[PATCH v6 39/39] t: remove test_oid_init in tests

Now that we call test_oid_init in the setup for all test scripts, there's no point in calling it individually. Remove all of the places where we've done so to help keep tests tidy.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/lib-pack.sh                          | 3 ---
 t/lib-submodule-update.sh              | 1 -
 t/t0000-basic.sh                       | 4 ----
 t/t1006-cat-file.sh                    | 2 --
 t/t1050-large.sh                       | 1 -
 t/t1410-reflog.sh                      | 1 -
 t/t1450-fsck.sh                        | 1 -
 t/t1500-rev-parse.sh                   | 1 -
 t/t3308-notes-merge.sh                 | 1 -
 t/t3600-rm.sh                          | 1 -
 t/t3800-mktag.sh                       | 1 -
 t/t4002-diff-basic.sh                  | 2 --
 t/t4027-diff-submodule.sh              | 1 -
 t/t4134-apply-submodule.sh             | 1 -
 t/t4200-rerere.sh                      | 1 -
 t/t4211-line-log.sh                    | 1 -
 t/t5300-pack-object.sh                 | 3 +--
 t/t5302-pack-index.sh                  | 1 -
 t/t5313-pack-bounds-checks.sh          | 1 -
 t/t5318-commit-graph.sh                | 3 +--
 t/t5319-multi-pack-index.sh            | 1 -
 t/t5324-split-commit-graph.sh          | 1 -
 t/t5504-fetch-receive-strict.sh        | 1 -
 t/t5530-upload-pack-error.sh           | 1 -
 t/t5562-http-backend-content-length.sh | 1 -
 t/t5702-protocol-v2.sh                 | 1 -
 t/t5703-upload-pack-ref-in-want.sh     | 1 -
 t/t6006-rev-list-format.sh             | 1 -
 28 files changed, 2 insertions(+), 37 deletions(-)
Show changes to 28 files +2 −37

t/lib-pack.sh, t/lib-submodule-update.sh, t/t0000-basic.sh, t/t1006-cat-file.sh, t/t1050-large.sh, t/t1410-reflog.sh, t/t1450-fsck.sh, t/t1500-rev-parse.sh, t/t3308-notes-merge.sh, t/t3600-rm.sh, t/t3800-mktag.sh, t/t4002-diff-basic.sh, t/t4027-diff-submodule.sh, t/t4134-apply-submodule.sh, t/t4200-rerere.sh, t/t4211-line-log.sh, t/t5300-pack-object.sh, t/t5302-pack-index.sh, t/t5313-pack-bounds-checks.sh, t/t5318-commit-graph.sh, t/t5319-multi-pack-index.sh, t/t5324-split-commit-graph.sh, t/t5504-fetch-receive-strict.sh, t/t5530-upload-pack-error.sh, t/t5562-http-backend-content-length.sh, t/t5702-protocol-v2.sh, t/t5703-upload-pack-ref-in-want.sh, t/t6006-rev-list-format.sh

diff --git a/t/lib-pack.sh b/t/lib-pack.sh
index 0c799d53c6..bb8938ccbe 100644
--- a/t/lib-pack.sh
+++ b/t/lib-pack.sh
@@ -35,8 +35,6 @@ pack_header () {
 # have hardcoded some well-known objects. See the case statements below for the
 # complete list.
 pack_obj () {
-	test_oid_init
-
 	case "$1" in
 	# empty blob
 	$EMPTY_BLOB)
@@ -121,7 +119,6 @@ pack_obj () {
 
 # Compute and append pack trailer to "$1"
 pack_trailer () {
-	test_oid_init &&
 	test-tool $(test_oid algo) -b <"$1" >trailer.tmp &&
 	cat trailer.tmp >>"$1" &&
 	rm -f trailer.tmp
diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
index 07c822c8ff..87a759149f 100644
--- a/t/lib-submodule-update.sh
+++ b/t/lib-submodule-update.sh
@@ -196,7 +196,6 @@ test_git_directory_exists () {
 # the submodule repo if it doesn't exist and configures the most problematic
 # settings for diff.ignoreSubmodules.
 prolog () {
-	test_oid_init &&
 	(test -d submodule_update_repo || create_lib_submodule_repo) &&
 	test_config_global diff.ignoreSubmodules all &&
 	test_config diff.ignoreSubmodules all
diff --git a/t/t0000-basic.sh b/t/t0000-basic.sh
index 2599d026a2..362feb43e6 100755
--- a/t/t0000-basic.sh
+++ b/t/t0000-basic.sh
@@ -891,10 +891,6 @@ test_expect_success 'test_atexit is run' "
 	test_path_is_missing also-clean-atexit
 "
 
-test_expect_success 'test_oid setup' '
-	test_oid_init
-'
-
 test_expect_success 'test_oid provides sane info by default' '
 	test_oid zero >actual &&
 	grep "^00*\$" actual &&
diff --git a/t/t1006-cat-file.sh b/t/t1006-cat-file.sh
index 43c4be1e5e..2f501d2dc9 100755
--- a/t/t1006-cat-file.sh
+++ b/t/t1006-cat-file.sh
@@ -140,8 +140,6 @@ test_expect_success '--batch-check without %(rest) considers whole line' '
 	test_cmp expect actual
 '
 
-test_oid_init
-
 tree_sha1=$(git write-tree)
 tree_size=$(($(test_oid rawsz) + 13))
 tree_pretty_content="100644 blob $hello_sha1	hello"
diff --git a/t/t1050-large.sh b/t/t1050-large.sh
index 6a56d1ca24..61e89a8071 100755
--- a/t/t1050-large.sh
+++ b/t/t1050-large.sh
@@ -12,7 +12,6 @@ file_size () {
 }
 
 test_expect_success setup '
-	test_oid_init &&
 	# clone does not allow us to pass core.bigfilethreshold to
 	# new repos, so set core.bigfilethreshold globally
 	git config --global core.bigfilethreshold 200k &&
diff --git a/t/t1410-reflog.sh b/t/t1410-reflog.sh
index 76d9b744a6..730a43d9dd 100755
--- a/t/t1410-reflog.sh
+++ b/t/t1410-reflog.sh
@@ -54,7 +54,6 @@ check_dont_have () {
 }
 
 test_expect_success setup '
-	test_oid_init &&
 	mkdir -p A/B &&
 	echo rat >C &&
 	echo ox >A/D &&
diff --git a/t/t1450-fsck.sh b/t/t1450-fsck.sh
index 344a2aad82..e81f1cbf0e 100755
--- a/t/t1450-fsck.sh
+++ b/t/t1450-fsck.sh
@@ -9,7 +9,6 @@ test_description='git fsck random collection of tests
 . ./test-lib.sh
 
 test_expect_success setup '
-	test_oid_init &&
 	git config gc.auto 0 &&
 	git config i18n.commitencoding ISO-8859-1 &&
 	test_commit A fileA one &&
diff --git a/t/t1500-rev-parse.sh b/t/t1500-rev-parse.sh
index 603019b541..408b97d5af 100755
--- a/t/t1500-rev-parse.sh
+++ b/t/t1500-rev-parse.sh
@@ -59,7 +59,6 @@ test_rev_parse () {
 ROOT=$(pwd)
 
 test_expect_success 'setup' '
-	test_oid_init &&
 	mkdir -p sub/dir work &&
 	cp -R .git repo.git
 '
diff --git a/t/t3308-notes-merge.sh b/t/t3308-notes-merge.sh
index 790e292966..d69c84c640 100755
--- a/t/t3308-notes-merge.sh
+++ b/t/t3308-notes-merge.sh
@@ -22,7 +22,6 @@ test_expect_success setup '
 	# Copy notes to remote-notes
 	git fetch . refs/notes/*:refs/remote-notes/origin/* &&
 
-	test_oid_init &&
 	test_oid_cache <<-EOF
 	hash4a sha1:5e93d24084d32e1cb61f7070505b9d2530cca987
 	hash3a sha1:8366731eeee53787d2bdf8fc1eff7d94757e8da0
diff --git a/t/t3600-rm.sh b/t/t3600-rm.sh
index f2c0168941..efec8d13b6 100755
--- a/t/t3600-rm.sh
+++ b/t/t3600-rm.sh
@@ -241,7 +241,6 @@ test_expect_success 'refresh index before checking if it is up-to-date' '
 '
 
 test_expect_success 'choking "git rm" should not let it die with cruft' '
-	test_oid_init &&
 	git reset -q --hard &&
 	test_when_finished "rm -f .git/index.lock && git reset -q --hard" &&
 	i=0 &&
diff --git a/t/t3800-mktag.sh b/t/t3800-mktag.sh
index 64dcc5ec28..d696aa4e52 100755
--- a/t/t3800-mktag.sh
+++ b/t/t3800-mktag.sh
@@ -23,7 +23,6 @@ check_verify_failure () {
 # first create a commit, so we have a valid object/type
 # for the tag.
 test_expect_success 'setup' '
-	test_oid_init &&
 	echo Hello >A &&
 	git update-index --add A &&
 	git commit -m "Initial commit" &&
diff --git a/t/t4002-diff-basic.sh b/t/t4002-diff-basic.sh
index cbcdd10464..6a9f010197 100755
--- a/t/t4002-diff-basic.sh
+++ b/t/t4002-diff-basic.sh
@@ -10,8 +10,6 @@ test_description='Test diff raw-output.
 
 . "$TEST_DIRECTORY"/lib-read-tree-m-3way.sh
 
-test_oid_init
-
 test_oid_cache <<\EOF
 aa_1 sha1:ccba72ad3888a3520b39efcf780b9ee64167535d
 aa_1 sha256:9febfbf18197819b2735c45291f138525d2476d59470f98239647544586ba403
diff --git a/t/t4027-diff-submodule.sh b/t/t4027-diff-submodule.sh
index e29deaf4a5..d7145ccca4 100755
--- a/t/t4027-diff-submodule.sh
+++ b/t/t4027-diff-submodule.sh
@@ -6,7 +6,6 @@ test_description='difference in submodules'
 . "$TEST_DIRECTORY"/diff-lib.sh
 
 test_expect_success setup '
-	test_oid_init &&
 	test_tick &&
 	test_create_repo sub &&
 	(
diff --git a/t/t4134-apply-submodule.sh b/t/t4134-apply-submodule.sh
index 99ed4cc546..d1c16ba33c 100755
--- a/t/t4134-apply-submodule.sh
+++ b/t/t4134-apply-submodule.sh
@@ -8,7 +8,6 @@ test_description='git apply submodule tests'
 . ./test-lib.sh
 
 test_expect_success setup '
-	test_oid_init &&
 	cat > create-sm.patch <<EOF &&
 diff --git a/dir/sm b/dir/sm
 new file mode 160000
diff --git a/t/t4200-rerere.sh b/t/t4200-rerere.sh
index 831d424c47..d94d25e4f0 100755
--- a/t/t4200-rerere.sh
+++ b/t/t4200-rerere.sh
@@ -25,7 +25,6 @@ test_description='git rerere
 . ./test-lib.sh
 
 test_expect_success 'setup' '
-	test_oid_init &&
 	cat >a1 <<-\EOF &&
 	Some title
 	==========
diff --git a/t/t4211-line-log.sh b/t/t4211-line-log.sh
index e186c83250..2d1d7b5d19 100755
--- a/t/t4211-line-log.sh
+++ b/t/t4211-line-log.sh
@@ -4,7 +4,6 @@ test_description='test log -L'
 . ./test-lib.sh
 
 test_expect_success 'setup (import history)' '
-	test_oid_init &&
 	git fast-import < "$TEST_DIRECTORY"/t4211/history.export &&
 	git reset --hard
 '
diff --git a/t/t5300-pack-object.sh b/t/t5300-pack-object.sh
index 746cdb626e..cf9f91c941 100755
--- a/t/t5300-pack-object.sh
+++ b/t/t5300-pack-object.sh
@@ -12,8 +12,7 @@ TRASH=$(pwd)
 
 test_expect_success \
     'setup' \
-    'test_oid_init &&
-     rm -f .git/index* &&
+    'rm -f .git/index* &&
      perl -e "print \"a\" x 4096;" > a &&
      perl -e "print \"b\" x 4096;" > b &&
      perl -e "print \"c\" x 4096;" > c &&
diff --git a/t/t5302-pack-index.sh b/t/t5302-pack-index.sh
index 8981c9b90e..119494bd64 100755
--- a/t/t5302-pack-index.sh
+++ b/t/t5302-pack-index.sh
@@ -7,7 +7,6 @@ test_description='pack index with 64-bit offsets and object CRC'
 . ./test-lib.sh
 
 test_expect_success 'setup' '
-	test_oid_init &&
 	rawsz=$(test_oid rawsz) &&
 	rm -rf .git &&
 	git init &&
diff --git a/t/t5313-pack-bounds-checks.sh b/t/t5313-pack-bounds-checks.sh
index 2a4557efc2..535313e4dc 100755
--- a/t/t5313-pack-bounds-checks.sh
+++ b/t/t5313-pack-bounds-checks.sh
@@ -45,7 +45,6 @@ extended_table () {
 }
 
 test_expect_success 'setup' '
-	test_oid_init &&
 	test_oid_cache <<-EOF
 	oid000 sha1:1485
 	oid000 sha256:4222
diff --git a/t/t5318-commit-graph.sh b/t/t5318-commit-graph.sh
index 26f332d6a3..67f2663c7c 100755
--- a/t/t5318-commit-graph.sh
+++ b/t/t5318-commit-graph.sh
@@ -10,8 +10,7 @@ test_expect_success 'setup full repo' '
 	cd "$TRASH_DIRECTORY/full" &&
 	git init &&
 	git config core.commitGraph true &&
-	objdir=".git/objects" &&
-	test_oid_init
+	objdir=".git/objects"
 '
 
 test_expect_success POSIXPERM 'tweak umask for modebit tests' '
diff --git a/t/t5319-multi-pack-index.sh b/t/t5319-multi-pack-index.sh
index 7214cab36c..7dfff0f8f4 100755
--- a/t/t5319-multi-pack-index.sh
+++ b/t/t5319-multi-pack-index.sh
@@ -29,7 +29,6 @@ midx_read_expect () {
 }
 
 test_expect_success 'setup' '
-	test_oid_init &&
 	test_oid_cache <<-EOF
 	idxoff sha1:2999
 	idxoff sha256:3739
diff --git a/t/t5324-split-commit-graph.sh b/t/t5324-split-commit-graph.sh
index 269d0964a3..3a63639fbc 100755
--- a/t/t5324-split-commit-graph.sh
+++ b/t/t5324-split-commit-graph.sh
@@ -12,7 +12,6 @@ test_expect_success 'setup repo' '
 	git config gc.writeCommitGraph false &&
 	infodir=".git/objects/info" &&
 	graphdir="$infodir/commit-graphs" &&
-	test_oid_init &&
 	test_oid_cache <<-EOM
 	shallow sha1:1760
 	shallow sha256:2064
diff --git a/t/t5504-fetch-receive-strict.sh b/t/t5504-fetch-receive-strict.sh
index a32efe2b6c..1a16ac4c0d 100755
--- a/t/t5504-fetch-receive-strict.sh
+++ b/t/t5504-fetch-receive-strict.sh
@@ -4,7 +4,6 @@ test_description='fetch/receive strict mode'
 . ./test-lib.sh
 
 test_expect_success 'setup and inject "corrupt or missing" object' '
-	test_oid_init &&
 	echo hello >greetings &&
 	git add greetings &&
 	git commit -m greetings &&
diff --git a/t/t5530-upload-pack-error.sh b/t/t5530-upload-pack-error.sh
index 4ce9a9f704..205a2631e7 100755
--- a/t/t5530-upload-pack-error.sh
+++ b/t/t5530-upload-pack-error.sh
@@ -14,7 +14,6 @@ corrupt_repo () {
 }
 
 test_expect_success 'setup and corrupt repository' '
-	test_oid_init &&
 	echo file >file &&
 	git add file &&
 	git rev-parse :file &&
diff --git a/t/t5562-http-backend-content-length.sh b/t/t5562-http-backend-content-length.sh
index c6ec625497..e5d3d15ba8 100755
--- a/t/t5562-http-backend-content-length.sh
+++ b/t/t5562-http-backend-content-length.sh
@@ -46,7 +46,6 @@ ssize_b100dots() {
 }
 
 test_expect_success 'setup' '
-	test_oid_init &&
 	HTTP_CONTENT_ENCODING="identity" &&
 	export HTTP_CONTENT_ENCODING &&
 	git config http.receivepack true &&
diff --git a/t/t5702-protocol-v2.sh b/t/t5702-protocol-v2.sh
index 7fc22171e7..5a60fbe3ed 100755
--- a/t/t5702-protocol-v2.sh
+++ b/t/t5702-protocol-v2.sh
@@ -13,7 +13,6 @@ start_git_daemon --export-all --enable=receive-pack
 daemon_parent=$GIT_DAEMON_DOCUMENT_ROOT_PATH/parent
 
 test_expect_success 'create repo to be served by git-daemon' '
-	test_oid_init &&
 	git init "$daemon_parent" &&
 	test_commit -C "$daemon_parent" one
 '
diff --git a/t/t5703-upload-pack-ref-in-want.sh b/t/t5703-upload-pack-ref-in-want.sh
index 748282f058..d9ecf0f4a9 100755
--- a/t/t5703-upload-pack-ref-in-want.sh
+++ b/t/t5703-upload-pack-ref-in-want.sh
@@ -43,7 +43,6 @@ write_command () {
 #             \ | /
 #               a
 test_expect_success 'setup repository' '
-	test_oid_init &&
 	test_commit a &&
 	git checkout -b o/foo &&
 	test_commit b &&
diff --git a/t/t6006-rev-list-format.sh b/t/t6006-rev-list-format.sh
index 7e82e43a63..bc95da8a5f 100755
--- a/t/t6006-rev-list-format.sh
+++ b/t/t6006-rev-list-format.sh
@@ -32,7 +32,6 @@ changed_iso88591=$(echo "$changed" | iconv -f utf-8 -t $test_encoding)
 truncate_count=20
 
 test_expect_success 'setup' '
-	test_oid_init &&
 	: >foo &&
 	git add foo &&
 	git config i18n.commitEncoding $test_encoding &&
brian m. carlson· Jul 29, 2020, 23:13 UTC · re: brian m. carlson · lore

[PATCH v6 03/39] t3305: make hash agnostic

When computing the fanout length, let's use test_oid to look up the hexadecimal size of the hash in question instead of hard-coding a value.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t3305-notes-fanout.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Show changes to t/t3305-notes-fanout.sh +1 −1
diff --git a/t/t3305-notes-fanout.sh b/t/t3305-notes-fanout.sh
index 3b4753e1b4..94c1b02251 100755
--- a/t/t3305-notes-fanout.sh
+++ b/t/t3305-notes-fanout.sh
@@ -7,7 +7,7 @@ test_description='Test that adding/removing many notes triggers automatic fanout
 path_has_fanout() {
 	path=$1 &&
 	fanout=$2 &&
-	after_last_slash=$((40 - $fanout * 2)) &&
+	after_last_slash=$(($(test_oid hexsz) - $fanout * 2)) &&
 	echo $path | grep -q "^\([0-9a-f]\{2\}/\)\{$fanout\}[0-9a-f]\{$after_last_slash\}$"
 }
 
brian m. carlson· Jul 29, 2020, 23:13 UTC · re: brian m. carlson · lore

[PATCH v6 05/39] t6100: make hash size independent

Instead of hard-coding a constant 40, split the output of rev-list by field.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t6100-rev-list-in-order.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
Show changes to t/t6100-rev-list-in-order.sh +2 −2
diff --git a/t/t6100-rev-list-in-order.sh b/t/t6100-rev-list-in-order.sh
index b2bb0a7f61..e934bc239c 100755
--- a/t/t6100-rev-list-in-order.sh
+++ b/t/t6100-rev-list-in-order.sh
@@ -22,7 +22,7 @@ test_expect_success 'setup a commit history with trees, blobs' '
 
 test_expect_success 'rev-list --in-commit-order' '
 	git rev-list --in-commit-order --objects HEAD >actual.raw &&
-	cut -c 1-40 >actual <actual.raw &&
+	cut -d" " -f1 >actual <actual.raw &&
 
 	git cat-file --batch-check="%(objectname)" >expect.raw <<-\EOF &&
 		HEAD^{commit}
@@ -49,7 +49,7 @@ test_expect_success 'rev-list --in-commit-order' '
 
 test_expect_success 'rev-list lists blobs and trees after commits' '
 	git rev-list --objects HEAD >actual.raw &&
-	cut -c 1-40 >actual <actual.raw &&
+	cut -d" " -f1 >actual <actual.raw &&
 
 	git cat-file --batch-check="%(objectname)" >expect.raw <<-\EOF &&
 		HEAD^{commit}
brian m. carlson· Jul 29, 2020, 23:13 UTC · re: brian m. carlson · lore

[PATCH v6 08/39] t6500: specify test values for SHA-256

In this test, we want to produce several blobs whose first two hex characters are "17", since we look at this object directory as a proxy for how many loose objects there are before we need to GC. Use test_oid_cache to specify strings that will hash to the right values when turned into blobs.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t6500-gc.sh | 27 ++++++++++++++++++++++-----
 1 file changed, 22 insertions(+), 5 deletions(-)
Show changes to t/t6500-gc.sh +22 −5
diff --git a/t/t6500-gc.sh b/t/t6500-gc.sh
index 0a69a67117..4a3b8f48ac 100755
--- a/t/t6500-gc.sh
+++ b/t/t6500-gc.sh
@@ -10,7 +10,24 @@ test_expect_success 'setup' '
 	# do not let the amount of physical memory affects gc
 	# behavior, make sure we always pack everything to one pack by
 	# default
-	git config gc.bigPackThreshold 2g
+	git config gc.bigPackThreshold 2g &&
+
+	# These are simply values which, when hashed as a blob with a newline,
+	# produce a hash where the first byte is 0x17 in their respective
+	# algorithms.
+	test_oid_cache <<-EOF
+	obj1 sha1:263
+	obj1 sha256:34
+
+	obj2 sha1:410
+	obj2 sha256:174
+
+	obj3 sha1:523
+	obj3 sha256:313
+
+	obj4 sha1:790
+	obj4 sha256:481
+	EOF
 '
 
 test_expect_success 'gc empty repository' '
@@ -85,13 +102,13 @@ test_expect_success 'auto gc with too many loose objects does not attempt to cre
 	# We need to create two object whose sha1s start with 17
 	# since this is what git gc counts.  As it happens, these
 	# two blobs will do so.
-	test_commit 263 &&
-	test_commit 410 &&
+	test_commit "$(test_oid obj1)" &&
+	test_commit "$(test_oid obj2)" &&
 	# Our first gc will create a pack; our second will create a second pack
 	git gc --auto &&
 	ls .git/objects/pack | sort >existing_packs &&
-	test_commit 523 &&
-	test_commit 790 &&
+	test_commit "$(test_oid obj3)" &&
+	test_commit "$(test_oid obj4)" &&
 
 	git gc --auto 2>err &&
 	test_i18ngrep ! "^warning:" err &&
brian m. carlson· Jul 29, 2020, 23:14 UTC · re: brian m. carlson · lore

[PATCH v6 12/39] t7201: abstract away SHA-1-specific constants

Adjust the test so that it computes variables for object IDs instead of using hard-coded hashes.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t7201-co.sh | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
Show changes to t/t7201-co.sh +4 −2
diff --git a/t/t7201-co.sh b/t/t7201-co.sh
index b696bae5f5..4d62b9b00f 100755
--- a/t/t7201-co.sh
+++ b/t/t7201-co.sh
@@ -230,9 +230,10 @@ test_expect_success 'switch to another branch while carrying a deletion' '
 test_expect_success 'checkout to detach HEAD (with advice declined)' '
 
 	git config advice.detachedHead false &&
+	rev=$(git rev-parse --short renamer^) &&
 	git checkout -f renamer && git clean -f &&
 	git checkout renamer^ 2>messages &&
-	test_i18ngrep "HEAD is now at 7329388" messages &&
+	test_i18ngrep "HEAD is now at $rev" messages &&
 	test_line_count = 1 messages &&
 	H=$(git rev-parse --verify HEAD) &&
 	M=$(git show-ref -s --verify refs/heads/master) &&
@@ -248,9 +249,10 @@ test_expect_success 'checkout to detach HEAD (with advice declined)' '
 
 test_expect_success 'checkout to detach HEAD' '
 	git config advice.detachedHead true &&
+	rev=$(git rev-parse --short renamer^) &&
 	git checkout -f renamer && git clean -f &&
 	GIT_TEST_GETTEXT_POISON=false git checkout renamer^ 2>messages &&
-	grep "HEAD is now at 7329388" messages &&
+	grep "HEAD is now at $rev" messages &&
 	test_line_count -gt 1 messages &&
 	H=$(git rev-parse --verify HEAD) &&
 	M=$(git show-ref -s --verify refs/heads/master) &&
brian m. carlson· Jul 29, 2020, 23:13 UTC · re: brian m. carlson · lore

[PATCH v6 06/39] t6101: make hash size independent

Use $OID_REGEX instead of a hard-coded regular expression.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t6101-rev-parse-parents.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Show changes to t/t6101-rev-parse-parents.sh +1 −1
diff --git a/t/t6101-rev-parse-parents.sh b/t/t6101-rev-parse-parents.sh
index 7683e4a114..7531262a5e 100755
--- a/t/t6101-rev-parse-parents.sh
+++ b/t/t6101-rev-parse-parents.sh
@@ -34,7 +34,7 @@ test_expect_success 'setup' '
 '
 
 test_expect_success 'start is valid' '
-	git rev-parse start | grep "^[0-9a-f]\{40\}$"
+	git rev-parse start | grep "^$OID_REGEX$"
 '
 
 test_expect_success 'start^0' '
brian m. carlson· Jul 29, 2020, 23:14 UTC · re: brian m. carlson · lore

[PATCH v6 18/39] t8002: make hash size independent

Compute the length of an object ID instead of hard-coding 40-based values.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t8002-blame.sh | 18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)
Show changes to t/t8002-blame.sh +11 −7
diff --git a/t/t8002-blame.sh b/t/t8002-blame.sh
index eea048e52c..c3b70b025e 100755
--- a/t/t8002-blame.sh
+++ b/t/t8002-blame.sh
@@ -6,6 +6,10 @@ test_description='git blame'
 PROG='git blame -c'
 . "$TEST_DIRECTORY"/annotate-tests.sh
 
+test_expect_success 'setup' '
+	hexsz=$(test_oid hexsz)
+'
+
 test_expect_success 'blame untracked file in empty repo' '
 	>untracked &&
 	test_must_fail git blame untracked
@@ -105,17 +109,17 @@ test_expect_success 'blame --abbrev=<n> works' '
 '
 
 test_expect_success 'blame -l aligns regular and boundary commits' '
-	check_abbrev 40 -l HEAD &&
-	check_abbrev 39 -l ^HEAD
+	check_abbrev $hexsz         -l HEAD &&
+	check_abbrev $((hexsz - 1)) -l ^HEAD
 '
 
-test_expect_success 'blame --abbrev=40 behaves like -l' '
-	check_abbrev 40 --abbrev=40 HEAD &&
-	check_abbrev 39 --abbrev=40 ^HEAD
+test_expect_success 'blame --abbrev with full length behaves like -l' '
+	check_abbrev $hexsz         --abbrev=$hexsz HEAD &&
+	check_abbrev $((hexsz - 1)) --abbrev=$hexsz ^HEAD
 '
 
-test_expect_success '--no-abbrev works like --abbrev=40' '
-	check_abbrev 40 --no-abbrev
+test_expect_success '--no-abbrev works like --abbrev with full length' '
+	check_abbrev $hexsz --no-abbrev
 '
 
 test_expect_success '--exclude-promisor-objects does not BUG-crash' '
brian m. carlson· Jul 29, 2020, 23:13 UTC · re: brian m. carlson · lore

[PATCH v6 10/39] t7003: compute appropriate length constant

Instead of using a specific invalid hard-coded object ID, look one up from the translation table.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t7003-filter-branch.sh | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
Show changes to t/t7003-filter-branch.sh +2 −1
diff --git a/t/t7003-filter-branch.sh b/t/t7003-filter-branch.sh
index e23de7d0b5..36477cb1f4 100755
--- a/t/t7003-filter-branch.sh
+++ b/t/t7003-filter-branch.sh
@@ -463,10 +463,11 @@ test_expect_success 'rewrite submodule with another content' '
 '
 
 test_expect_success 'replace submodule revision' '
+	invalid=$(test_oid numeric) &&
 	git reset --hard original &&
 	git filter-branch -f --tree-filter \
 	    "if git ls-files --error-unmatch -- submod > /dev/null 2>&1
-	     then git update-index --cacheinfo 160000 0123456789012345678901234567890123456789 submod
+	     then git update-index --cacheinfo 160000 $invalid submod
 	     fi" HEAD &&
 	test $orig_head != $(git show-ref --hash --head HEAD)
 '
brian m. carlson· Jul 29, 2020, 23:14 UTC · re: brian m. carlson · lore

[PATCH v6 14/39] t7400: make hash size independent

Instead of using cut with hard-coded hash sizes, use cut with fields, or where that's not possible, sed with $OID_REGEX, so that the tests are independent of hash size.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t7400-submodule-basic.sh | 26 +++++++++++++-------------
 1 file changed, 13 insertions(+), 13 deletions(-)
Show changes to t/t7400-submodule-basic.sh +13 −13
diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
index 956e17abb3..fec7e0299d 100755
--- a/t/t7400-submodule-basic.sh
+++ b/t/t7400-submodule-basic.sh
@@ -1231,7 +1231,7 @@ test_expect_success 'submodule helper list is not confused by common prefixes' '
 	git submodule add /dir1/b dir1/b &&
 	git submodule add /dir2/b dir2/b &&
 	git commit -m "first submodule commit" &&
-	git submodule--helper list dir1/b |cut -c51- >actual &&
+	git submodule--helper list dir1/b | cut -f 2 >actual &&
 	echo "dir1/b" >expect &&
 	test_cmp expect actual
 '
@@ -1260,7 +1260,7 @@ test_expect_success 'submodule update --init with a specification' '
 	pwd=$(pwd) &&
 	git clone file://"$pwd"/multisuper multisuper_clone &&
 	git -C multisuper_clone submodule update --init . ":(exclude)sub0" &&
-	git -C multisuper_clone submodule status |cut -c 1,43- >actual &&
+	git -C multisuper_clone submodule status | sed "s/$OID_REGEX //" >actual &&
 	test_cmp expect actual
 '
 
@@ -1271,7 +1271,7 @@ test_expect_success 'submodule update --init with submodule.active set' '
 	git -C multisuper_clone config submodule.active "." &&
 	git -C multisuper_clone config --add submodule.active ":(exclude)sub0" &&
 	git -C multisuper_clone submodule update --init &&
-	git -C multisuper_clone submodule status |cut -c 1,43- >actual &&
+	git -C multisuper_clone submodule status | sed "s/$OID_REGEX //" >actual &&
 	test_cmp expect actual
 '
 
@@ -1290,7 +1290,7 @@ test_expect_success 'submodule update and setting submodule.<name>.active' '
 	-sub3
 	EOF
 	git -C multisuper_clone submodule update &&
-	git -C multisuper_clone submodule status |cut -c 1,43- >actual &&
+	git -C multisuper_clone submodule status | sed "s/$OID_REGEX //" >actual &&
 	test_cmp expect actual
 '
 
@@ -1307,12 +1307,12 @@ test_expect_success 'clone active submodule without submodule url set' '
 		git submodule update &&
 		git submodule status >actual_raw &&
 
-		cut -c 1,43- actual_raw >actual &&
+		cut -d" " -f3- actual_raw >actual &&
 		cat >expect <<-\EOF &&
-		 sub0 (test2)
-		 sub1 (test2)
-		 sub2 (test2)
-		 sub3 (test2)
+		sub0 (test2)
+		sub1 (test2)
+		sub2 (test2)
+		sub3 (test2)
 		EOF
 		test_cmp expect actual
 	)
@@ -1328,7 +1328,7 @@ test_expect_success 'clone --recurse-submodules with a pathspec works' '
 	EOF
 
 	git clone --recurse-submodules="sub0" multisuper multisuper_clone &&
-	git -C multisuper_clone submodule status |cut -c1,43- >actual &&
+	git -C multisuper_clone submodule status | sed "s/$OID_REGEX //" >actual &&
 	test_cmp expected actual
 '
 
@@ -1345,7 +1345,7 @@ test_expect_success 'clone with multiple --recurse-submodules options' '
 		  --recurse-submodules=":(exclude)sub0" \
 		  --recurse-submodules=":(exclude)sub2" \
 		  multisuper multisuper_clone &&
-	git -C multisuper_clone submodule status |cut -c1,43- >actual &&
+	git -C multisuper_clone submodule status | sed "s/$OID_REGEX //" >actual &&
 	test_cmp expect actual
 '
 
@@ -1373,7 +1373,7 @@ test_expect_success 'clone and subsequent updates correctly auto-initialize subm
 		  --recurse-submodules=":(exclude)sub4" \
 		  multisuper multisuper_clone &&
 
-	git -C multisuper_clone submodule status |cut -c1,43- >actual &&
+	git -C multisuper_clone submodule status | sed "s/$OID_REGEX //" >actual &&
 	test_cmp expect actual &&
 
 	git -C multisuper submodule add ../sub1 sub4 &&
@@ -1382,7 +1382,7 @@ test_expect_success 'clone and subsequent updates correctly auto-initialize subm
 	# obtain the new superproject
 	git -C multisuper_clone pull &&
 	git -C multisuper_clone submodule update --init &&
-	git -C multisuper_clone submodule status |cut -c1,43- >actual &&
+	git -C multisuper_clone submodule status | sed "s/$OID_REGEX //" >actual &&
 	test_cmp expect2 actual
 '
 
brian m. carlson· Jul 29, 2020, 23:14 UTC · re: brian m. carlson · lore

[PATCH v6 15/39] t7405: make hash size independent

Use $ZERO_OID instead of hard-coding a fixed size all-zeros object ID.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t7405-submodule-merge.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
Show changes to t/t7405-submodule-merge.sh +2 −2
diff --git a/t/t7405-submodule-merge.sh b/t/t7405-submodule-merge.sh
index aa33978ed2..6a1e5f8232 100755
--- a/t/t7405-submodule-merge.sh
+++ b/t/t7405-submodule-merge.sh
@@ -195,7 +195,7 @@ test_expect_success 'git submodule status should display the merge conflict prop
        url = $TRASH_DIRECTORY/sub
 EOF
        cat >expect <<EOF &&
-U0000000000000000000000000000000000000000 sub
+U$ZERO_OID sub
 EOF
        git submodule status > actual &&
        test_cmp expect actual &&
@@ -214,7 +214,7 @@ test_expect_success 'git submodule status should display the merge conflict prop
        url = $TRASH_DIRECTORY/sub
 EOF
        cat >expect <<EOF &&
-U0000000000000000000000000000000000000000 sub
+U$ZERO_OID sub
 EOF
        git submodule status > actual &&
        test_cmp expect actual &&
brian m. carlson· Jul 29, 2020, 23:13 UTC · re: brian m. carlson · lore

[PATCH v6 02/39] t1001: use $ZERO_OID

Use $ZERO_OID to make the test hash independent.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t1091-sparse-checkout-builtin.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
Show changes to t/t1091-sparse-checkout-builtin.sh +2 −2
diff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh
index 7cd45fc139..84acfc48b6 100755
--- a/t/t1091-sparse-checkout-builtin.sh
+++ b/t/t1091-sparse-checkout-builtin.sh
@@ -369,7 +369,7 @@ test_expect_success 'sparse-checkout (init|set|disable) warns with unmerged stat
 	git clone repo unmerged &&
 
 	cat >input <<-EOF &&
-	0 0000000000000000000000000000000000000000	folder1/a
+	0 $ZERO_OID	folder1/a
 	100644 $(git -C unmerged rev-parse HEAD:folder1/a) 1	folder1/a
 	EOF
 	git -C unmerged update-index --index-info <input &&
@@ -396,7 +396,7 @@ test_expect_success 'sparse-checkout reapply' '
 	echo dirty >tweak/deep/deeper2/a &&
 
 	cat >input <<-EOF &&
-	0 0000000000000000000000000000000000000000	folder1/a
+	0 $ZERO_OID	folder1/a
 	100644 $(git -C tweak rev-parse HEAD:folder1/a) 1	folder1/a
 	EOF
 	git -C tweak update-index --index-info <input &&
brian m. carlson· Jul 29, 2020, 23:14 UTC · re: brian m. carlson · lore

[PATCH v6 36/39] t: make SHA1 prerequisite depend on default hash

Currently, the SHA1 prerequisite depends on the output of git hash-object. However, in order for that to produce sane behavior, we must be in a repository. If we are not, the default will remain SHA-1, and we'll produce wrong results if we're using SHA-256 for the testsuite but the test assertion starts when we're not in a repository.

Check the environment variable we use for this purpose, leaving it to default to SHA-1 if none is specified.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/test-lib.sh | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
Show changes to t/test-lib.sh +5 −1
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 307bb2207e..0483ed51e1 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -1689,7 +1689,11 @@ test_lazy_prereq CURL '
 # which will not work with other hash algorithms and tests that work but don't
 # test anything meaningful (e.g. special values which cause short collisions).
 test_lazy_prereq SHA1 '
-	test $(git hash-object /dev/null) = e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
+	case "$GIT_DEFAULT_HASH" in
+	sha1) true ;;
+	"") test $(git hash-object /dev/null) = e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 ;;
+	*) false ;;
+	esac
 '
 
 test_lazy_prereq REBASE_P '
brian m. carlson· Jul 29, 2020, 23:14 UTC · re: brian m. carlson · lore

[PATCH v6 28/39] t0410: mark test with SHA1 prerequisite

These tests try to check that we behave properly if we encounter a repository with version 0 but an extension. This is a laudable goal, but the test cannot work with SHA-256, since SHA-256 repositories always have an existing extension and are never version 0.

Add a SHA1 prerequisite to these tests.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t0410-partial-clone.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
Show changes to t/t0410-partial-clone.sh +2 −2
diff --git a/t/t0410-partial-clone.sh b/t/t0410-partial-clone.sh
index 6aa0f313bd..a5ebdf9ff3 100755
--- a/t/t0410-partial-clone.sh
+++ b/t/t0410-partial-clone.sh
@@ -42,7 +42,7 @@ test_expect_success 'convert shallow clone to partial clone' '
 	test_cmp_config -C client 1 core.repositoryformatversion
 '
 
-test_expect_success 'convert to partial clone with noop extension' '
+test_expect_success SHA1 'convert to partial clone with noop extension' '
 	rm -fr server client &&
 	test_create_repo server &&
 	test_commit -C server my_commit 1 &&
@@ -53,7 +53,7 @@ test_expect_success 'convert to partial clone with noop extension' '
 	git -C client fetch --unshallow --filter="blob:none"
 '
 
-test_expect_success 'converting to partial clone fails with unrecognized extension' '
+test_expect_success SHA1 'converting to partial clone fails with unrecognized extension' '
 	rm -fr server client &&
 	test_create_repo server &&
 	test_commit -C server my_commit 1 &&
brian m. carlson· Jul 29, 2020, 23:14 UTC · re: brian m. carlson · lore

[PATCH v6 21/39] t9300: abstract away SHA-1-specific constants

Adjust the test so that it computes variables for object IDs instead of using hard-coded hashes. In addition, use cut to filter out the object IDs and verify only the information that we're really interested in.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t9300-fast-import.sh | 108 ++++++++++++++++++++++-------------------
 1 file changed, 59 insertions(+), 49 deletions(-)
Show changes to t/t9300-fast-import.sh +59 −49
diff --git a/t/t9300-fast-import.sh b/t/t9300-fast-import.sh
index e151df81c0..f2d2cb06ad 100755
--- a/t/t9300-fast-import.sh
+++ b/t/t9300-fast-import.sh
@@ -284,8 +284,9 @@ test_expect_success 'A: verify pack' '
 '
 
 test_expect_success 'A: verify diff' '
+	copy=$(git rev-parse --verify master:file2) &&
 	cat >expect <<-EOF &&
-	:000000 100755 0000000000000000000000000000000000000000 7123f7f44e39be127c5eb701e5968176ee9d78b1 A	copy-of-file2
+	:000000 100755 0000000000000000000000000000000000000000 $copy A	copy-of-file2
 	EOF
 	git diff-tree -M -r master verify--import-marks >actual &&
 	compare_diff_raw expect actual &&
@@ -528,6 +529,7 @@ test_expect_success 'B: fail on invalid committer (5)' '
 test_expect_success 'C: incremental import create pack from stdin' '
 	newf=$(echo hi newf | git hash-object -w --stdin) &&
 	oldf=$(git rev-parse --verify master:file2) &&
+	thrf=$(git rev-parse --verify master:file3) &&
 	test_tick &&
 	cat >input <<-INPUT_END &&
 	commit refs/heads/branch
@@ -570,10 +572,11 @@ test_expect_success 'C: verify commit' '
 '
 
 test_expect_success 'C: validate rename result' '
+	zero=$ZERO_OID &&
 	cat >expect <<-EOF &&
-	:000000 100755 0000000000000000000000000000000000000000 f1fb5da718392694d0076d677d6d0e364c79b0bc A	file2/newf
-	:100644 100644 7123f7f44e39be127c5eb701e5968176ee9d78b1 7123f7f44e39be127c5eb701e5968176ee9d78b1 R100	file2	file2/oldf
-	:100644 000000 0d92e9f3374ae2947c23aa477cbc68ce598135f1 0000000000000000000000000000000000000000 D	file3
+	:000000 100755 $zero $newf A	file2/newf
+	:100644 100644 $oldf $oldf R100	file2	file2/oldf
+	:100644 000000 $thrf $zero D	file3
 	EOF
 	git diff-tree -M -r master branch >actual &&
 	compare_diff_raw expect actual
@@ -614,9 +617,11 @@ test_expect_success 'D: verify pack' '
 '
 
 test_expect_success 'D: validate new files added' '
+	f5id=$(echo "$file5_data" | git hash-object --stdin) &&
+	f6id=$(echo "$file6_data" | git hash-object --stdin) &&
 	cat >expect <<-EOF &&
-	:000000 100755 0000000000000000000000000000000000000000 e74b7d465e52746be2b4bae983670711e6e66657 A	newdir/exec.sh
-	:000000 100644 0000000000000000000000000000000000000000 fcf778cda181eaa1cbc9e9ce3a2e15ee9f9fe791 A	newdir/interesting
+	:000000 100755 0000000000000000000000000000000000000000 $f6id A	newdir/exec.sh
+	:000000 100644 0000000000000000000000000000000000000000 $f5id A	newdir/interesting
 	EOF
 	git diff-tree -M -r branch^ branch >actual &&
 	compare_diff_raw expect actual
@@ -779,12 +784,13 @@ test_expect_success 'H: verify pack' '
 '
 
 test_expect_success 'H: validate old files removed, new files added' '
+	f4id=$(git rev-parse HEAD:file4) &&
 	cat >expect <<-EOF &&
-	:100755 000000 f1fb5da718392694d0076d677d6d0e364c79b0bc 0000000000000000000000000000000000000000 D	file2/newf
-	:100644 000000 7123f7f44e39be127c5eb701e5968176ee9d78b1 0000000000000000000000000000000000000000 D	file2/oldf
-	:100755 000000 85df50785d62d3b05ab03d9cbf7e4a0b49449730 0000000000000000000000000000000000000000 D	file4
-	:100644 100644 fcf778cda181eaa1cbc9e9ce3a2e15ee9f9fe791 fcf778cda181eaa1cbc9e9ce3a2e15ee9f9fe791 R100	newdir/interesting	h/e/l/lo
-	:100755 000000 e74b7d465e52746be2b4bae983670711e6e66657 0000000000000000000000000000000000000000 D	newdir/exec.sh
+	:100755 000000 $newf $zero D	file2/newf
+	:100644 000000 $oldf $zero D	file2/oldf
+	:100755 000000 $f4id $zero D	file4
+	:100644 100644 $f5id $f5id R100	newdir/interesting	h/e/l/lo
+	:100755 000000 $f6id $zero D	newdir/exec.sh
 	EOF
 	git diff-tree -M -r H^ H >actual &&
 	compare_diff_raw expect actual
@@ -935,14 +941,15 @@ test_expect_success 'L: verify internal tree sorting' '
 	INPUT_END
 
 	cat >expect <<-EXPECT_END &&
-	:100644 100644 4268632... 55d3a52... M	b.
-	:040000 040000 0ae5cac... 443c768... M	b
-	:100644 100644 4268632... 55d3a52... M	ba
+	:100644 100644 M	b.
+	:040000 040000 M	b
+	:100644 100644 M	ba
 	EXPECT_END
 
 	git fast-import <input &&
 	GIT_PRINT_SHA1_ELLIPSIS="yes" git diff-tree --abbrev --raw L^ L >output &&
-	test_cmp expect output
+	cut -d" " -f1,2,5 output >actual &&
+	test_cmp expect actual
 '
 
 test_expect_success 'L: nested tree copy does not corrupt deltas' '
@@ -1004,7 +1011,7 @@ test_expect_success 'M: rename file in same subdirectory' '
 	INPUT_END
 
 	cat >expect <<-EOF &&
-	:100755 100755 f1fb5da718392694d0076d677d6d0e364c79b0bc f1fb5da718392694d0076d677d6d0e364c79b0bc R100	file2/newf	file2/n.e.w.f
+	:100755 100755 $newf $newf R100	file2/newf	file2/n.e.w.f
 	EOF
 	git fast-import <input &&
 	git diff-tree -M -r M1^ M1 >actual &&
@@ -1025,7 +1032,7 @@ test_expect_success 'M: rename file to new subdirectory' '
 	INPUT_END
 
 	cat >expect <<-EOF &&
-	:100755 100755 f1fb5da718392694d0076d677d6d0e364c79b0bc f1fb5da718392694d0076d677d6d0e364c79b0bc R100	file2/newf	i/am/new/to/you
+	:100755 100755 $newf $newf R100	file2/newf	i/am/new/to/you
 	EOF
 	git fast-import <input &&
 	git diff-tree -M -r M2^ M2 >actual &&
@@ -1046,7 +1053,7 @@ test_expect_success 'M: rename subdirectory to new subdirectory' '
 	INPUT_END
 
 	cat >expect <<-EOF &&
-	:100755 100755 f1fb5da718392694d0076d677d6d0e364c79b0bc f1fb5da718392694d0076d677d6d0e364c79b0bc R100	i/am/new/to/you	other/sub/am/new/to/you
+	:100755 100755 $newf $newf R100	i/am/new/to/you	other/sub/am/new/to/you
 	EOF
 	git fast-import <input &&
 	git diff-tree -M -r M3^ M3 >actual &&
@@ -1067,11 +1074,11 @@ test_expect_success 'M: rename root to subdirectory' '
 	INPUT_END
 
 	cat >expect <<-EOF &&
-	:100644 100644 7123f7f44e39be127c5eb701e5968176ee9d78b1 7123f7f44e39be127c5eb701e5968176ee9d78b1 R100	file2/oldf	sub/file2/oldf
-	:100755 100755 85df50785d62d3b05ab03d9cbf7e4a0b49449730 85df50785d62d3b05ab03d9cbf7e4a0b49449730 R100	file4	sub/file4
-	:100755 100755 f1fb5da718392694d0076d677d6d0e364c79b0bc f1fb5da718392694d0076d677d6d0e364c79b0bc R100	i/am/new/to/you	sub/i/am/new/to/you
-	:100755 100755 e74b7d465e52746be2b4bae983670711e6e66657 e74b7d465e52746be2b4bae983670711e6e66657 R100	newdir/exec.sh	sub/newdir/exec.sh
-	:100644 100644 fcf778cda181eaa1cbc9e9ce3a2e15ee9f9fe791 fcf778cda181eaa1cbc9e9ce3a2e15ee9f9fe791 R100	newdir/interesting	sub/newdir/interesting
+	:100644 100644 $oldf $oldf R100	file2/oldf	sub/file2/oldf
+	:100755 100755 $f4id $f4id R100	file4	sub/file4
+	:100755 100755 $newf $newf R100	i/am/new/to/you	sub/i/am/new/to/you
+	:100755 100755 $f6id $f6id R100	newdir/exec.sh	sub/newdir/exec.sh
+	:100644 100644 $f5id $f5id R100	newdir/interesting	sub/newdir/interesting
 	EOF
 	git fast-import <input &&
 	git diff-tree -M -r M4^ M4 >actual &&
@@ -1097,7 +1104,7 @@ test_expect_success 'N: copy file in same subdirectory' '
 	INPUT_END
 
 	cat >expect <<-EOF &&
-	:100755 100755 f1fb5da718392694d0076d677d6d0e364c79b0bc f1fb5da718392694d0076d677d6d0e364c79b0bc C100	file2/newf	file2/n.e.w.f
+	:100755 100755 $newf $newf C100	file2/newf	file2/n.e.w.f
 	EOF
 	git fast-import <input &&
 	git diff-tree -C --find-copies-harder -r N1^ N1 >actual &&
@@ -1129,9 +1136,9 @@ test_expect_success 'N: copy then modify subdirectory' '
 	INPUT_END
 
 	cat >expect <<-EOF &&
-	:100644 100644 fcf778cda181eaa1cbc9e9ce3a2e15ee9f9fe791 fcf778cda181eaa1cbc9e9ce3a2e15ee9f9fe791 C100	newdir/interesting	file3/file5
-	:100755 100755 f1fb5da718392694d0076d677d6d0e364c79b0bc f1fb5da718392694d0076d677d6d0e364c79b0bc C100	file2/newf	file3/newf
-	:100644 100644 7123f7f44e39be127c5eb701e5968176ee9d78b1 7123f7f44e39be127c5eb701e5968176ee9d78b1 C100	file2/oldf	file3/oldf
+	:100644 100644 $f5id $f5id C100	newdir/interesting	file3/file5
+	:100755 100755 $newf $newf C100	file2/newf	file3/newf
+	:100644 100644 $oldf $oldf C100	file2/oldf	file3/oldf
 	EOF
 	git fast-import <input &&
 	git diff-tree -C --find-copies-harder -r N2^^ N2 >actual &&
@@ -1162,9 +1169,9 @@ test_expect_success 'N: copy dirty subdirectory' '
 '
 
 test_expect_success 'N: copy directory by id' '
-	cat >expect <<-\EOF &&
-	:100755 100755 f1fb5da718392694d0076d677d6d0e364c79b0bc f1fb5da718392694d0076d677d6d0e364c79b0bc C100	file2/newf	file3/newf
-	:100644 100644 7123f7f44e39be127c5eb701e5968176ee9d78b1 7123f7f44e39be127c5eb701e5968176ee9d78b1 C100	file2/oldf	file3/oldf
+	cat >expect <<-EOF &&
+	:100755 100755 $newf $newf C100	file2/newf	file3/newf
+	:100644 100644 $oldf $oldf C100	file2/oldf	file3/oldf
 	EOF
 	subdir=$(git rev-parse refs/heads/branch^0:file2) &&
 	cat >input <<-INPUT_END &&
@@ -1183,9 +1190,9 @@ test_expect_success 'N: copy directory by id' '
 '
 
 test_expect_success PIPE 'N: read and copy directory' '
-	cat >expect <<-\EOF &&
-	:100755 100755 f1fb5da718392694d0076d677d6d0e364c79b0bc f1fb5da718392694d0076d677d6d0e364c79b0bc C100	file2/newf	file3/newf
-	:100644 100644 7123f7f44e39be127c5eb701e5968176ee9d78b1 7123f7f44e39be127c5eb701e5968176ee9d78b1 C100	file2/oldf	file3/oldf
+	cat >expect <<-EOF &&
+	:100755 100755 $newf $newf C100	file2/newf	file3/newf
+	:100644 100644 $oldf $oldf C100	file2/oldf	file3/oldf
 	EOF
 	git update-ref -d refs/heads/N4 &&
 	rm -f backflow &&
@@ -1254,9 +1261,9 @@ test_expect_success PIPE 'N: empty directory reads as missing' '
 '
 
 test_expect_success 'N: copy root directory by tree hash' '
-	cat >expect <<-\EOF &&
-	:100755 000000 f1fb5da718392694d0076d677d6d0e364c79b0bc 0000000000000000000000000000000000000000 D	file3/newf
-	:100644 000000 7123f7f44e39be127c5eb701e5968176ee9d78b1 0000000000000000000000000000000000000000 D	file3/oldf
+	cat >expect <<-EOF &&
+	:100755 000000 $newf $zero D	file3/newf
+	:100644 000000 $oldf $zero D	file3/oldf
 	EOF
 	root=$(git rev-parse refs/heads/branch^0^{tree}) &&
 	cat >input <<-INPUT_END &&
@@ -1275,12 +1282,12 @@ test_expect_success 'N: copy root directory by tree hash' '
 '
 
 test_expect_success 'N: copy root by path' '
-	cat >expect <<-\EOF &&
-	:100755 100755 f1fb5da718392694d0076d677d6d0e364c79b0bc f1fb5da718392694d0076d677d6d0e364c79b0bc C100	file2/newf	oldroot/file2/newf
-	:100644 100644 7123f7f44e39be127c5eb701e5968176ee9d78b1 7123f7f44e39be127c5eb701e5968176ee9d78b1 C100	file2/oldf	oldroot/file2/oldf
-	:100755 100755 85df50785d62d3b05ab03d9cbf7e4a0b49449730 85df50785d62d3b05ab03d9cbf7e4a0b49449730 C100	file4	oldroot/file4
-	:100755 100755 e74b7d465e52746be2b4bae983670711e6e66657 e74b7d465e52746be2b4bae983670711e6e66657 C100	newdir/exec.sh	oldroot/newdir/exec.sh
-	:100644 100644 fcf778cda181eaa1cbc9e9ce3a2e15ee9f9fe791 fcf778cda181eaa1cbc9e9ce3a2e15ee9f9fe791 C100	newdir/interesting	oldroot/newdir/interesting
+	cat >expect <<-EOF &&
+	:100755 100755 $newf $newf C100	file2/newf	oldroot/file2/newf
+	:100644 100644 $oldf $oldf C100	file2/oldf	oldroot/file2/oldf
+	:100755 100755 $f4id $f4id C100	file4	oldroot/file4
+	:100755 100755 $f6id $f6id C100	newdir/exec.sh	oldroot/newdir/exec.sh
+	:100644 100644 $f5id $f5id C100	newdir/interesting	oldroot/newdir/interesting
 	EOF
 	cat >input <<-INPUT_END &&
 	commit refs/heads/N-copy-root-path
@@ -1340,10 +1347,10 @@ test_expect_success 'N: delete directory by copying' '
 '
 
 test_expect_success 'N: modify copied tree' '
-	cat >expect <<-\EOF &&
-	:100644 100644 fcf778cda181eaa1cbc9e9ce3a2e15ee9f9fe791 fcf778cda181eaa1cbc9e9ce3a2e15ee9f9fe791 C100	newdir/interesting	file3/file5
-	:100755 100755 f1fb5da718392694d0076d677d6d0e364c79b0bc f1fb5da718392694d0076d677d6d0e364c79b0bc C100	file2/newf	file3/newf
-	:100644 100644 7123f7f44e39be127c5eb701e5968176ee9d78b1 7123f7f44e39be127c5eb701e5968176ee9d78b1 C100	file2/oldf	file3/oldf
+	cat >expect <<-EOF &&
+	:100644 100644 $f5id $f5id C100	newdir/interesting	file3/file5
+	:100755 100755 $newf $newf C100	file2/newf	file3/newf
+	:100644 100644 $oldf $oldf C100	file2/oldf	file3/oldf
 	EOF
 	subdir=$(git rev-parse refs/heads/branch^0:file2) &&
 	cat >input <<-INPUT_END &&
@@ -3117,6 +3124,9 @@ test_expect_success 'U: initialize for U tests' '
 
 	INPUT_END
 
+	f7id=$(echo "blob 1" | git hash-object --stdin) &&
+	f8id=$(echo "sleep well" | git hash-object --stdin) &&
+	f9id=$(echo "au revoir" | git hash-object --stdin) &&
 	git fast-import <input
 '
 
@@ -3137,7 +3147,7 @@ test_expect_success 'U: filedelete file succeeds' '
 
 test_expect_success 'U: validate file delete result' '
 	cat >expect <<-EOF &&
-	:100644 000000 2907ebb4bf85d91bf0716bb3bd8a68ef48d6da76 0000000000000000000000000000000000000000 D	good/night.txt
+	:100644 000000 $f8id $ZERO_OID D	good/night.txt
 	EOF
 
 	git diff-tree -M -r U^1 U >actual &&
@@ -3162,7 +3172,7 @@ test_expect_success 'U: filedelete directory succeeds' '
 
 test_expect_success 'U: validate directory delete result' '
 	cat >expect <<-EOF &&
-	:100644 000000 69cb75792f55123d8389c156b0b41c2ff00ed507 0000000000000000000000000000000000000000 D	good/bye.txt
+	:100644 000000 $f9id $ZERO_OID D	good/bye.txt
 	EOF
 
 	git diff-tree -M -r U^1 U >actual &&
@@ -3187,7 +3197,7 @@ test_expect_success 'U: filedelete root succeeds' '
 
 test_expect_success 'U: validate root delete result' '
 	cat >expect <<-EOF &&
-	:100644 000000 c18147dc648481eeb65dc5e66628429a64843327 0000000000000000000000000000000000000000 D	hello.c
+	:100644 000000 $f7id $ZERO_OID D	hello.c
 	EOF
 
 	git diff-tree -M -r U^1 U >actual &&
brian m. carlson· Jul 29, 2020, 23:14 UTC · re: brian m. carlson · lore

[PATCH v6 24/39] t9350: make hash size independent

This test checks for several commit object sizes to verify that objects are encoded as expected. However, the size of a commit object differs between SHA-1 and SHA-256, since each contains a hex representation of the tree's object ID. Since these are root commits, compute the size of each commit by using a constant plus the size of a single hex object ID.

In addition, use $ZERO_OID instead of a hard-coded object ID.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t9350-fast-export.sh | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)
Show changes to t/t9350-fast-export.sh +7 −7
diff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh
index 690c90fb82..1372842559 100755
--- a/t/t9350-fast-export.sh
+++ b/t/t9350-fast-export.sh
@@ -132,12 +132,12 @@ test_expect_success 'reencoding iso-8859-7' '
 	sed "s/wer/i18n/" iso-8859-7.fi |
 		(cd new &&
 		 git fast-import &&
-		 # The commit object, if not re-encoded, would be 240 bytes.
+		 # The commit object, if not re-encoded, would be 200 bytes plus hash.
 		 # Removing the "encoding iso-8859-7\n" header drops 20 bytes.
 		 # Re-encoding the Pi character from \xF0 (\360) in iso-8859-7
 		 # to \xCF\x80 (\317\200) in UTF-8 adds a byte.  Check for
 		 # the expected size.
-		 test 221 -eq "$(git cat-file -s i18n)" &&
+		 test $(($(test_oid hexsz) + 181)) -eq "$(git cat-file -s i18n)" &&
 		 # ...and for the expected translation of bytes.
 		 git cat-file commit i18n >actual &&
 		 grep $(printf "\317\200") actual &&
@@ -164,12 +164,12 @@ test_expect_success 'preserving iso-8859-7' '
 	sed "s/wer/i18n-no-recoding/" iso-8859-7.fi |
 		(cd new &&
 		 git fast-import &&
-		 # The commit object, if not re-encoded, is 240 bytes.
+		 # The commit object, if not re-encoded, is 200 bytes plus hash.
 		 # Removing the "encoding iso-8859-7\n" header would drops 20
 		 # bytes.  Re-encoding the Pi character from \xF0 (\360) in
 		 # iso-8859-7 to \xCF\x80 (\317\200) in UTF-8 adds a byte.
 		 # Check for the expected size...
-		 test 240 -eq "$(git cat-file -s i18n-no-recoding)" &&
+		 test $(($(test_oid hexsz) + 200)) -eq "$(git cat-file -s i18n-no-recoding)" &&
 		 # ...as well as the expected byte.
 		 git cat-file commit i18n-no-recoding >actual &&
 		 grep $(printf "\360") actual &&
@@ -192,7 +192,7 @@ test_expect_success 'encoding preserved if reencoding fails' '
 		 grep ^encoding actual &&
 		 # Verify that the commit has the expected size; i.e.
 		 # that no bytes were re-encoded to a different encoding.
-		 test 252 -eq "$(git cat-file -s i18n-invalid)" &&
+		 test $(($(test_oid hexsz) + 212)) -eq "$(git cat-file -s i18n-invalid)" &&
 		 # ...and check for the original special bytes
 		 grep $(printf "\360") actual &&
 		 grep $(printf "\377") actual)
@@ -694,7 +694,7 @@ test_expect_success 'delete ref because entire history excluded' '
 	git fast-export to-delete ^to-delete >actual &&
 	cat >expected <<-EOF &&
 	reset refs/heads/to-delete
-	from 0000000000000000000000000000000000000000
+	from $ZERO_OID
 
 	EOF
 	test_cmp expected actual
@@ -704,7 +704,7 @@ test_expect_success 'delete refspec' '
 	git fast-export --refspec :refs/heads/to-delete >actual &&
 	cat >expected <<-EOF &&
 	reset refs/heads/to-delete
-	from 0000000000000000000000000000000000000000
+	from $ZERO_OID
 
 	EOF
 	test_cmp expected actual
brian m. carlson· Jul 29, 2020, 23:14 UTC · re: brian m. carlson · lore

[PATCH v6 23/39] t9301: make hash size independent

Instead of using a hard-coded all-zeros object ID, use $ZERO_OID. Compute the length of the object IDs in use and use this instead of hard-coding the constant 40.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t9301-fast-import-notes.sh | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)
Show changes to t/t9301-fast-import-notes.sh +6 −6
diff --git a/t/t9301-fast-import-notes.sh b/t/t9301-fast-import-notes.sh
index ca223dca98..14c1baa739 100755
--- a/t/t9301-fast-import-notes.sh
+++ b/t/t9301-fast-import-notes.sh
@@ -470,12 +470,13 @@ test_expect_success 'add lots of commits and notes' '
 '
 
 test_expect_success 'verify that lots of notes trigger a fanout scheme' '
+	hexsz=$(test_oid hexsz) &&
 
 	# None of the entries in the top-level notes tree should be a full SHA1
 	git ls-tree --name-only refs/notes/many_notes |
 	while read path
 	do
-		if test $(expr length "$path") -ge 40
+		if test $(expr length "$path") -ge $hexsz
 		then
 			return 1
 		fi
@@ -518,7 +519,7 @@ test_expect_success 'verify that importing a notes tree respects the fanout sche
 	git ls-tree --name-only refs/notes/other_notes |
 	while read path
 	do
-		if test $(expr length "$path") -ge 40
+		if test $(expr length "$path") -ge $hexsz
 		then
 			return 1
 		fi
@@ -593,7 +594,7 @@ test_expect_success 'verify that changing notes respect existing fanout' '
 	git ls-tree --name-only refs/notes/many_notes |
 	while read path
 	do
-		if test $(expr length "$path") -ge 40
+		if test $(expr length "$path") -ge $hexsz
 		then
 			return 1
 		fi
@@ -616,7 +617,7 @@ i=$(($num_commits - $remaining_notes))
 for sha1 in $(git rev-list -n $i refs/heads/many_commits)
 do
 	cat >>input <<INPUT_END
-N 0000000000000000000000000000000000000000 $sha1
+N $ZERO_OID $sha1
 INPUT_END
 done
 
@@ -646,7 +647,6 @@ test_expect_success 'remove lots of notes' '
 '
 
 test_expect_success 'verify that removing notes trigger fanout consolidation' '
-
 	# All entries in the top-level notes tree should be a full SHA1
 	git ls-tree --name-only -r refs/notes/many_notes |
 	while read path
@@ -656,7 +656,7 @@ test_expect_success 'verify that removing notes trigger fanout consolidation' '
 		test "$path" = "deadbeef" && continue
 		test "$path" = "de/adbeef" && continue
 
-		if test $(expr length "$path") -ne 40
+		if test $(expr length "$path") -ne $hexsz
 		then
 			return 1
 		fi
brian m. carlson· Jul 29, 2020, 23:14 UTC · re: brian m. carlson · lore

[PATCH v6 20/39] t8011: make hash size independent

Allow lines which start with either a 40- or 64-character hex object ID, to allow for both SHA-1 and SHA-256.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t8011-blame-split-file.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Show changes to t/t8011-blame-split-file.sh +1 −1
diff --git a/t/t8011-blame-split-file.sh b/t/t8011-blame-split-file.sh
index 831125047b..bdda0c03fe 100755
--- a/t/t8011-blame-split-file.sh
+++ b/t/t8011-blame-split-file.sh
@@ -54,7 +54,7 @@ test_expect_success 'setup simulated porcelain' '
 	cat >read-porcelain.pl <<-\EOF
 	my $field = shift;
 	while (<>) {
-		if (/^[0-9a-f]{40} /) {
+		if (/^[0-9a-f]{40,} /) {
 			flush();
 			$hash = $&;
 		} elsif (/^$field (.*)/) {
brian m. carlson· Jul 29, 2020, 23:14 UTC · re: brian m. carlson · lore

[PATCH v6 29/39] http-fetch: set up git directory before parsing pack hashes

In dd4b732df7 ("upload-pack: send part of packfile response as uri", 2020-06-10), the git http-fetch code learned how to take ac --packfile option. This option takes an argument, which is the name of a packfile hash, and parses it using parse_oid_hex. It does so before calling setup_git_directory.

However, in a SHA-256 repository this fails to work, since we have not set the hash algorithm in use and parse_oid_hex fails as a consequence. To ensure that we can parse packfile hashes of the right length, let's set up the git directory before we start parsing arguments.

Since we still want to allow the invocation of -h to print the help when we're not in a repository, gracefully handle us being outside of one and produce an error after argument parsing has finished.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 http-fetch.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
Show changes to http-fetch.c +5 −1
diff --git a/http-fetch.c b/http-fetch.c
index 1df376e745..c4ccc5fea9 100644
--- a/http-fetch.c
+++ b/http-fetch.c
@@ -84,8 +84,11 @@ int cmd_main(int argc, const char **argv)
 	int get_verbosely = 0;
 	int get_recover = 0;
 	int packfile = 0;
+	int nongit;
 	struct object_id packfile_hash;
 
+	setup_git_directory_gently(&nongit);
+
 	while (arg < argc && argv[arg][0] == '-') {
 		const char *p;
 
@@ -115,7 +118,8 @@ int cmd_main(int argc, const char **argv)
 	if (argc != arg + 2 - (commits_on_stdin || packfile))
 		usage(http_fetch_usage);
 
-	setup_git_directory();
+	if (nongit)
+		die(_("not a git repository"));
 
 	git_config(git_default_config, NULL);
 
brian m. carlson· Jul 29, 2020, 23:14 UTC · re: brian m. carlson · lore

[PATCH v6 34/39] t: add test_oid option to select hash algorithm

In some tests, we have data files which are written with a particular hash algorithm. Instead of keeping two copies of the test files, we can keep one, and translate the value on the fly.

In order to do so, we'll need to read both the source algorithm and the current algorithm, so add an optional flag to the test_oid helper that lets us look up a value for a specified hash algorithm. This should not cause any conflicts with existing tests, since key arguments to test_oid are allowed to contains only shell identifier characters.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t0000-basic.sh        | 11 +++++++++++
 t/test-lib-functions.sh | 12 +++++++++++-
 2 files changed, 22 insertions(+), 1 deletion(-)
Show changes to 2 files +22 −1

t/t0000-basic.sh, t/test-lib-functions.sh

diff --git a/t/t0000-basic.sh b/t/t0000-basic.sh
index 2ff176cd5d..2599d026a2 100755
--- a/t/t0000-basic.sh
+++ b/t/t0000-basic.sh
@@ -928,6 +928,17 @@ test_expect_success 'test_oid can look up data for SHA-256' '
 	test "$hexsz" -eq 64
 '
 
+test_expect_success 'test_oid can look up data for a specified algorithm' '
+	rawsz="$(test_oid --hash=sha1 rawsz)" &&
+	hexsz="$(test_oid --hash=sha1 hexsz)" &&
+	test "$rawsz" -eq 20 &&
+	test "$hexsz" -eq 40 &&
+	rawsz="$(test_oid --hash=sha256 rawsz)" &&
+	hexsz="$(test_oid --hash=sha256 hexsz)" &&
+	test "$rawsz" -eq 32 &&
+	test "$hexsz" -eq 64
+'
+
 test_expect_success 'test_bool_env' '
 	(
 		sane_unset envvar &&
diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh
index 3103be8a32..2608e80f11 100644
--- a/t/test-lib-functions.sh
+++ b/t/test-lib-functions.sh
@@ -1468,7 +1468,17 @@ test_oid_cache () {
 # Look up a per-hash value based on a key ($1).  The value must have been loaded
 # by test_oid_init or test_oid_cache.
 test_oid () {
-	local var="test_oid_${test_hash_algo}_$1" &&
+	local algo="${test_hash_algo}" &&
+
+	case "$1" in
+	--hash=*)
+		algo="${1#--hash=}" &&
+		shift;;
+	*)
+		;;
+	esac &&
+
+	local var="test_oid_${algo}_$1" &&
 
 	# If the variable is unset, we must be missing an entry for this
 	# key-hash pair, so exit with an error.
Eric Sunshine· Jul 30, 2020, 02:48 UTC · re: brian m. carlson · lore

Re: [PATCH v6 00/39] SHA-256, part 3/3

On Wed, Jul 29, 2020 at 7:15 PM brian m. carlson <sandals@crustytoothpaste.net> wrote:

Show 6 quoted lines
> Changes from v5:
> * Remove useless test_oid_init invocations throughout the series.
> * Fix a commit message for grammar and style.
> * Fix a typo in a shell case statement.
> * Fix http-fetch to allow parsing options before failing on a missing
>   git directory.

Thanks, I think this version addresses all the comments from my reviews of the last several versions of this patch series.

I've read through the entire series a couple times and paid close attention to the range-diffs, so FWIW, I'm reasonably comfortable giving v6 my:

Reviewed-by: Eric Sunshine <sunshine@sunshineco.com>
Junio C Hamano· Jul 30, 2020, 16:18 UTC · re: Eric Sunshine · lore

Re: [PATCH v6 00/39] SHA-256, part 3/3

Eric Sunshine <sunshine@sunshineco.com> writes:
Show 17 quoted lines
> On Wed, Jul 29, 2020 at 7:15 PM brian m. carlson
> <sandals@crustytoothpaste.net> wrote:
>> Changes from v5:
>> * Remove useless test_oid_init invocations throughout the series.
>> * Fix a commit message for grammar and style.
>> * Fix a typo in a shell case statement.
>> * Fix http-fetch to allow parsing options before failing on a missing
>>   git directory.
>
> Thanks, I think this version addresses all the comments from my
> reviews of the last several versions of this patch series.
>
> I've read through the entire series a couple times and paid close
> attention to the range-diffs, so FWIW, I'm reasonably comfortable
> giving v6 my:
>
> Reviewed-by: Eric Sunshine <sunshine@sunshineco.com>
Thanks, both.
Ævar Arnfjörð Bjarmason· Mar 10, 2021, 12:04 UTC · re: brian m. carlson · lore

Re: [PATCH v6 00/39] SHA-256, part 3/3

[I couldn't find a more relevant thing to reply to]
On Thu, Jul 30 2020, brian m. carlson wrote:
> [...]
B.t.w. thanks again for all your work on SHA-1 -> SHA-256.
I found a missing spot that wasn't trivial to fix, so sending an E-Mail:
In git-filter-branch.sh we have:

_x40='[0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f]' _x40="$_x40$_x40$_x40$_x40$_x40$_x40$_x40$_x40"

Then later we have a case condition based on matching a SHA-1:
        $_x40)
                echo "Ref '$ref' was rewritten"
                if ! git update-ref -m "filter-branch: rewrite" \

Just deleting that case arm has filter-branch tests passing, so whatever it's meant to do it has zero coverage, which explains why it hasn't broken with our tests.

I didn't have time to dig, so sending off this E-Mail instead.
Elijah Newren· Mar 10, 2021, 16:36 UTC · re: Ævar Arnfjörð Bjarmason · lore

Re: [PATCH v6 00/39] SHA-256, part 3/3

On Wed, Mar 10, 2021 at 4:04 AM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:

Show 27 quoted lines
>
> [I couldn't find a more relevant thing to reply to]
>
> On Thu, Jul 30 2020, brian m. carlson wrote:
>
> > [...]
>
> B.t.w. thanks again for all your work on SHA-1 -> SHA-256.
>
> I found a missing spot that wasn't trivial to fix, so sending an E-Mail:
>
> In git-filter-branch.sh we have:
>
> _x40='[0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f]'
> _x40="$_x40$_x40$_x40$_x40$_x40$_x40$_x40$_x40"
>
> Then later we have a case condition based on matching a SHA-1:
>
>         $_x40)
>                 echo "Ref '$ref' was rewritten"
>                 if ! git update-ref -m "filter-branch: rewrite" \
>
> Just deleting that case arm has filter-branch tests passing, so whatever
> it's meant to do it has zero coverage, which explains why it hasn't
> broken with our tests.
>
> I didn't have time to dig, so sending off this E-Mail instead.

Hmm, perhaps we should revive the "separate non-core tools out of git.git" thread -- https://lore.kernel.org/git/a784a61e-1320-be1e-9dfb-d533a01827ec@gmail.com/.

Honestly not sure it's worth bothering fixing filter-branch at this point, given the strong WARNINGs we added to it. If anything, I'd just add another warning to the pile for this issue. If folks really still want/need filter-branch once SHA-256 is the only option, I'm sure they'll be motivated to make the necessary fixes (or just use/fix filter-repo's filter-branch-ish).

Jeff King· Mar 10, 2021, 17:08 UTC · re: Elijah Newren · lore

Re: [PATCH v6 00/39] SHA-256, part 3/3

On Wed, Mar 10, 2021 at 08:36:15AM -0800, Elijah Newren wrote:
Show 6 quoted lines
> Honestly not sure it's worth bothering fixing filter-branch at this
> point, given the strong WARNINGs we added to it.  If anything, I'd
> just add another warning to the pile for this issue.  If folks really
> still want/need filter-branch once SHA-256 is the only option, I'm
> sure they'll be motivated to make the necessary fixes (or just use/fix
> filter-repo's filter-branch-ish).

You're probably right that it's not a good use of time, but I just sent a series fixing it. It was one of those cases where I was like "oh, it's probably just X". And then "no, wait...", and before I knew it I had wasted an hour fixing filter-branch. ;)

-Peff
Jeff King· Mar 10, 2021, 17:06 UTC · re: Ævar Arnfjörð Bjarmason · lore

[PATCH 0/3] sha256 fixes for filter-branch

On Wed, Mar 10, 2021 at 01:04:37PM +0100, Ævar Arnfjörð Bjarmason wrote:
Show 16 quoted lines
> I found a missing spot that wasn't trivial to fix, so sending an E-Mail:
> 
> In git-filter-branch.sh we have:
> 
> _x40='[0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f]'
> _x40="$_x40$_x40$_x40$_x40$_x40$_x40$_x40$_x40"
> 
> Then later we have a case condition based on matching a SHA-1:
> 
>         $_x40)
>                 echo "Ref '$ref' was rewritten"
>                 if ! git update-ref -m "filter-branch: rewrite" \
> 
> Just deleting that case arm has filter-branch tests passing, so whatever
> it's meant to do it has zero coverage, which explains why it hasn't
> broken with our tests.

It actually does get covered. Dropping that case-arm means we'll fall through to the one below, which does _roughly_ the same thing with a bunch of extra warnings. But none of the tests actually check the error messages, so they don't notice.

Here's a series which fixes it, plus extra tests to notice the distinction.

  [1/3]: t7003: test ref rewriting explicitly
  [2/3]: filter-branch: drop multiple-ancestor warning
  [3/3]: filter-branch: drop $_x40 glob
 git-filter-branch.sh     | 16 ++--------------
 t/t7003-filter-branch.sh | 31 +++++++++++++++++++++++++++++++
 2 files changed, 33 insertions(+), 14 deletions(-)
-Peff
Jeff King· Mar 10, 2021, 17:07 UTC · re: Jeff King · lore

[PATCH 3/3] filter-branch: drop $_x40 glob

When checking whether a commit was rewritten to a single object id, we use a glob that insists on a 40-hex result. This works for sha1, but fails t7003 when run with GIT_TEST_DEFAULT_HASH=sha256.

Since the previous commit simplified the case statement here, we only have two arms: an empty string or a single object id. We can just loosen our glob to match anything, and still distinguish those cases (we lose the ability to notice bogus input, but that's not a problem; we are the one who wrote the map in the first place, and anyway update-ref will complain loudly if the input isn't a valid hash).

Signed-off-by: Jeff King <peff@peff.net>
---
 git-filter-branch.sh | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
Show changes to git-filter-branch.sh +1 −6
diff --git a/git-filter-branch.sh b/git-filter-branch.sh
index a1e80bd552..cb89372813 100755
--- a/git-filter-branch.sh
+++ b/git-filter-branch.sh
@@ -498,8 +498,6 @@ fi
 
 # Finally update the refs
 
-_x40='[0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f]'
-_x40="$_x40$_x40$_x40$_x40$_x40$_x40$_x40$_x40"
 echo
 while read ref
 do
@@ -519,7 +517,7 @@ do
 		git update-ref -m "filter-branch: delete" -d "$ref" $sha1 ||
 			die "Could not delete $ref"
 	;;
-	$_x40)
+	*)
 		echo "Ref '$ref' was rewritten"
 		if ! git update-ref -m "filter-branch: rewrite" \
 					"$ref" $rewritten $sha1 2>/dev/null; then
@@ -533,9 +531,6 @@ do
 			fi
 		fi
 	;;
-	*)
-		die "BUG: multiple ancestors in map file?"
-	;;
 	esac
 	git update-ref -m "filter-branch: backup" "$orig_namespace$ref" $sha1 ||
 		 exit
-- 
2.31.0.rc2.525.gc2268d2248
Junio C Hamano· Mar 10, 2021, 22:23 UTC · re: Jeff King · lore

Re: [PATCH 3/3] filter-branch: drop $_x40 glob

Jeff King <peff@peff.net> writes:
Show 10 quoted lines
> When checking whether a commit was rewritten to a single object id, we
> use a glob that insists on a 40-hex result. This works for sha1, but
> fails t7003 when run with GIT_TEST_DEFAULT_HASH=sha256.
>
> Since the previous commit simplified the case statement here, we only
> have two arms: an empty string or a single object id. We can just loosen
> our glob to match anything, and still distinguish those cases (we lose
> the ability to notice bogus input, but that's not a problem; we are the
> one who wrote the map in the first place, and anyway update-ref will
> complain loudly if the input isn't a valid hash).

If the input is any valid extended sha-1 expression, it probably would not barf, so we technically are losing a bit of internal safety valve to notice bogus rewrite (if the map recorded output of "git describe --always", this part would not notice and keep working just fine, but other parts of the code may not be happy).

But if this were a new code that said "It is either empty of non-empty, and non-empty ones are object names we write, so we do not have to insist it is hex object names", it is very likely that we would accept it without any extra checks, so I think this is perfectly OK.

Thanks.
Show 37 quoted lines
> Signed-off-by: Jeff King <peff@peff.net>
> ---
>  git-filter-branch.sh | 7 +------
>  1 file changed, 1 insertion(+), 6 deletions(-)
>
> diff --git a/git-filter-branch.sh b/git-filter-branch.sh
> index a1e80bd552..cb89372813 100755
> --- a/git-filter-branch.sh
> +++ b/git-filter-branch.sh
> @@ -498,8 +498,6 @@ fi
>  
>  # Finally update the refs
>  
> -_x40='[0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f]'
> -_x40="$_x40$_x40$_x40$_x40$_x40$_x40$_x40$_x40"
>  echo
>  while read ref
>  do
> @@ -519,7 +517,7 @@ do
>  		git update-ref -m "filter-branch: delete" -d "$ref" $sha1 ||
>  			die "Could not delete $ref"
>  	;;
> -	$_x40)
> +	*)
>  		echo "Ref '$ref' was rewritten"
>  		if ! git update-ref -m "filter-branch: rewrite" \
>  					"$ref" $rewritten $sha1 2>/dev/null; then
> @@ -533,9 +531,6 @@ do
>  			fi
>  		fi
>  	;;
> -	*)
> -		die "BUG: multiple ancestors in map file?"
> -	;;
>  	esac
>  	git update-ref -m "filter-branch: backup" "$orig_namespace$ref" $sha1 ||
>  		 exit
Ævar Arnfjörð Bjarmason· Mar 11, 2021, 00:14 UTC · re: Jeff King · lore

[PATCH 1/4] git-bisect: remove unused SHA-1 $x40 shell variable

This variable was last used in code removed in 06f5608c14 (bisect--helper: `bisect_start` shell function partially in C, 2019-01-02).

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 git-bisect.sh | 2 --
 1 file changed, 2 deletions(-)
Show changes to git-bisect.sh +0 −2
diff --git a/git-bisect.sh b/git-bisect.sh
index 6a7afaea8d..b59f3aaad4 100755
--- a/git-bisect.sh
+++ b/git-bisect.sh
@@ -34,8 +34,6 @@ Please use "git help bisect" to get the full man page.'
 OPTIONS_SPEC=
 . git-sh-setup
 
-_x40='[0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f]'
-_x40="$_x40$_x40$_x40$_x40$_x40$_x40$_x40$_x40"
 TERM_BAD=bad
 TERM_GOOD=good
 
-- 
2.31.0.rc1.213.gbfdb770ff55
Ævar Arnfjörð Bjarmason· Mar 11, 2021, 00:14 UTC · re: Jeff King · lore

[PATCH 0/4] bisect + tests: remove old $_x05, $_x35 and $_x40 variables

A cleanup in the spirit of Jeff King's just-sent series: https://lore.kernel.org/git/YEj82fOf+F4xJC8S@coredump.intra.peff.net/

Applies independently of that series though.
Ævar Arnfjörð Bjarmason (4):
  git-bisect: remove unused SHA-1 $x40 shell variable
  test-lib: remove unused $_x40 and $_z40 variables
  shortlog tests: rewrite to get rid of --abbrev=35 hardcoding
  tests: get rid of $_x05 from the test suite
 git-bisect.sh                       |  2 -
 t/t3101-ls-tree-dirname.sh          |  2 +-
 t/t3508-cherry-pick-many-commits.sh |  4 +-
 t/t4201-shortlog.sh                 | 92 ++++++++++++++++-------------
 t/t6006-rev-list-format.sh          | 12 ++--
 t/t7600-merge.sh                    |  4 +-
 t/test-lib.sh                       |  9 +--
 7 files changed, 65 insertions(+), 60 deletions(-)
-- 
2.31.0.rc1.213.gbfdb770ff55
Ævar Arnfjörð Bjarmason· Mar 11, 2021, 00:14 UTC · re: Jeff King · lore

[PATCH 4/4] tests: get rid of $_x05 from the test suite

Remove the last users of the $_x05 variable from the tests. It turns out that all of these tests can be rewritten unambiguously to simply use [0-9a-f]* instead.

In the case of the tree matching we're relying on there being a <TAB> after the SHA (but a space between the modes and type), then in some of the other tests here that an abbreviated SHA is at the start of the line, etc.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t3101-ls-tree-dirname.sh          |  2 +-
 t/t3508-cherry-pick-many-commits.sh |  4 ++--
 t/t6006-rev-list-format.sh          | 12 +++++++-----
 t/t7600-merge.sh                    |  4 ++--
 t/test-lib.sh                       |  6 +-----
 5 files changed, 13 insertions(+), 15 deletions(-)
Show changes to 5 files +13 −15

t/t3101-ls-tree-dirname.sh, t/t3508-cherry-pick-many-commits.sh, t/t6006-rev-list-format.sh, t/t7600-merge.sh, t/test-lib.sh

diff --git a/t/t3101-ls-tree-dirname.sh b/t/t3101-ls-tree-dirname.sh
index 12bf31022a..110ebf7d52 100755
--- a/t/t3101-ls-tree-dirname.sh
+++ b/t/t3101-ls-tree-dirname.sh
@@ -187,7 +187,7 @@ EOF
 
 test_expect_success 'ls-tree --abbrev=5' '
 	git ls-tree --abbrev=5 $tree >current &&
-	sed -e "s/ $_x05[0-9a-f]*	/ X	/" <current >check &&
+	sed -e "s/ [0-9a-f]*	/ X	/" <current >check &&
 	cat >expected <<\EOF &&
 100644 blob X	1.txt
 100644 blob X	2.txt
diff --git a/t/t3508-cherry-pick-many-commits.sh b/t/t3508-cherry-pick-many-commits.sh
index e8375d1c97..1bf867d139 100755
--- a/t/t3508-cherry-pick-many-commits.sh
+++ b/t/t3508-cherry-pick-many-commits.sh
@@ -84,7 +84,7 @@ test_expect_success 'output to keep user entertained during multi-pick' '
 	git reset --hard first &&
 	test_tick &&
 	git cherry-pick first..fourth >actual &&
-	sed -e "s/$_x05[0-9a-f][0-9a-f]/OBJID/" <actual >actual.fuzzy &&
+	sed -e "s/ [0-9a-f]*\\]/ OBJID]/" <actual >actual.fuzzy &&
 	test_line_count -ge 3 actual.fuzzy &&
 	test_cmp expected actual.fuzzy
 '
@@ -122,7 +122,7 @@ test_expect_success 'output during multi-pick indicates merge strategy' '
 	git reset --hard first &&
 	test_tick &&
 	git cherry-pick --strategy resolve first..fourth >actual &&
-	sed -e "s/$_x05[0-9a-f][0-9a-f]/OBJID/" <actual >actual.fuzzy &&
+	sed -e "s/ [0-9a-f]*\\]/ OBJID]/" <actual >actual.fuzzy &&
 	test_cmp expected actual.fuzzy
 '
 
diff --git a/t/t6006-rev-list-format.sh b/t/t6006-rev-list-format.sh
index 35a2f62392..ec65081e17 100755
--- a/t/t6006-rev-list-format.sh
+++ b/t/t6006-rev-list-format.sh
@@ -453,16 +453,18 @@ test_expect_success 'add SP before non-empty (2)' '
 '
 
 test_expect_success '--abbrev' '
-	echo SHORT SHORT SHORT >expect2 &&
 	echo LONG LONG LONG >expect3 &&
 	git log -1 --format="%h %h %h" HEAD >actual1 &&
 	git log -1 --abbrev=5 --format="%h %h %h" HEAD >actual2 &&
 	git log -1 --abbrev=5 --format="%H %H %H" HEAD >actual3 &&
-	sed -e "s/$OID_REGEX/LONG/g" -e "s/$_x05/SHORT/g" <actual2 >fuzzy2 &&
-	sed -e "s/$OID_REGEX/LONG/g" -e "s/$_x05/SHORT/g" <actual3 >fuzzy3 &&
-	test_cmp expect2 fuzzy2 &&
+	sed -e "s/$OID_REGEX/LONG/g" <actual3 >fuzzy3 &&
+	test_file_size actual2 >expect &&
+	# 3*5 SHAs + 3 separating spaces
+	echo 18 >actual &&
+	test_cmp expect actual &&
 	test_cmp expect3 fuzzy3 &&
-	! test_cmp actual1 actual2
+	! test_cmp actual1 actual2 &&
+	! test_cmp actual2 actual3
 '
 
 test_expect_success '%H is not affected by --abbrev-commit' '
diff --git a/t/t7600-merge.sh b/t/t7600-merge.sh
index 1cbc9715a8..fec5a7b2dd 100755
--- a/t/t7600-merge.sh
+++ b/t/t7600-merge.sh
@@ -189,7 +189,7 @@ test_expect_success 'merge c0 with c1' '
 	verify_head "$c1" &&
 
 	git reflog -1 >reflog.actual &&
-	sed "s/$_x05[0-9a-f]*/OBJID/g" reflog.actual >reflog.fuzzy &&
+	sed "s/^[0-9a-f]*/OBJID/" <reflog.actual >reflog.fuzzy &&
 	test_cmp reflog.expected reflog.fuzzy
 '
 
@@ -220,7 +220,7 @@ test_expect_success 'merge from unborn branch' '
 	verify_head "$c1" &&
 
 	git reflog -1 >reflog.actual &&
-	sed "s/$_x05[0-9a-f][0-9a-f]/OBJID/g" reflog.actual >reflog.fuzzy &&
+	sed "s/^[0-9a-f]*/OBJID/g" reflog.actual >reflog.fuzzy &&
 	test_cmp reflog.expected reflog.fuzzy
 '
 
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 4d5ba558d3..aeb4b2da1c 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -511,7 +511,7 @@ SQ=\'
 # when case-folding filenames
 u200c=$(printf '\342\200\214')
 
-export _x05 LF u200c EMPTY_TREE EMPTY_BLOB ZERO_OID OID_REGEX
+export LF u200c EMPTY_TREE EMPTY_BLOB ZERO_OID OID_REGEX
 
 # Each test should start with something like this, after copyright notices:
 #
@@ -1380,10 +1380,6 @@ then
 	fi
 fi
 
-# Convenience
-# A regexp to match 5 hexdigits
-_x05='[0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f]'
-
 test_oid_init
 
 ZERO_OID=$(test_oid zero)
-- 
2.31.0.rc1.213.gbfdb770ff55
Junio C Hamano· Mar 11, 2021, 01:23 UTC · re: Ævar Arnfjörð Bjarmason · lore

Re: [PATCH 4/4] tests: get rid of $_x05 from the test suite

Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:
> Remove the last users of the $_x05 variable from the tests. It turns
> out that all of these tests can be rewritten unambiguously to simply
> use [0-9a-f]* instead.
I am unsure about the "unambiguously" part, e.g.
> -	sed -e "s/ $_x05[0-9a-f]*	/ X	/" <current >check &&
> +	sed -e "s/ [0-9a-f]*	/ X	/" <current >check &&

does't the preimage say "we expect at least 5 hexdigits to be shown here"? The postimage lets even an empty string to pass.

> In the case of the tree matching we're relying on there being a <TAB>
> after the SHA (but a space between the modes and type), then in some
> of the other tests here that an abbreviated SHA is at the start of the
> line, etc.

Sure, but these being tests, I am not sure we should be assuming the correct input to these transformations.

Show 7 quoted lines
>  test_expect_success 'ls-tree --abbrev=5' '
>  	git ls-tree --abbrev=5 $tree >current &&
> -	sed -e "s/ $_x05[0-9a-f]*	/ X	/" <current >check &&
> +	sed -e "s/ [0-9a-f]*	/ X	/" <current >check &&
>  	cat >expected <<\EOF &&
>  100644 blob X	1.txt
>  100644 blob X	2.txt

This one is particularly iffy. The --abbrev=5 test is designed to ensure that the resulting abbreviated object names are at least 5 hexdigits long, even when the repository is so small that only 4 hexdigits are sufficient to avoid ambiguity, while allowing the output to be longer than specified 5 (when 5 turns out to be insufficient for disambiguation).

So, I dunno.
Ævar Arnfjörð Bjarmason· Mar 11, 2021, 10:29 UTC · re: Junio C Hamano · lore

Re: [PATCH 4/4] tests: get rid of $_x05 from the test suite

On Thu, Mar 11 2021, Junio C Hamano wrote:
Show 38 quoted lines
> Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:
>
>> Remove the last users of the $_x05 variable from the tests. It turns
>> out that all of these tests can be rewritten unambiguously to simply
>> use [0-9a-f]* instead.
>
> I am unsure about the "unambiguously" part, e.g.
>
>> -	sed -e "s/ $_x05[0-9a-f]*	/ X	/" <current >check &&
>> +	sed -e "s/ [0-9a-f]*	/ X	/" <current >check &&
>
> does't the preimage say "we expect at least 5 hexdigits to be shown
> here"?  The postimage lets even an empty string to pass. 
>
>> In the case of the tree matching we're relying on there being a <TAB>
>> after the SHA (but a space between the modes and type), then in some
>> of the other tests here that an abbreviated SHA is at the start of the
>> line, etc.
>
> Sure, but these being tests, I am not sure we should be assuming the
> correct input to these transformations.
>
>>  test_expect_success 'ls-tree --abbrev=5' '
>>  	git ls-tree --abbrev=5 $tree >current &&
>> -	sed -e "s/ $_x05[0-9a-f]*	/ X	/" <current >check &&
>> +	sed -e "s/ [0-9a-f]*	/ X	/" <current >check &&
>>  	cat >expected <<\EOF &&
>>  100644 blob X	1.txt
>>  100644 blob X	2.txt
>
> This one is particularly iffy.  The --abbrev=5 test is designed to
> ensure that the resulting abbreviated object names are at least 5
> hexdigits long, even when the repository is so small that only 4
> hexdigits are sufficient to avoid ambiguity, while allowing the
> output to be longer than specified 5 (when 5 turns out to be
> insufficient for disambiguation).
>
> So, I dunno.

Yes, I think this patch should be dropped. Do you mind just dropping the 4/4 and having it be a 3-patch series? I can also re-submit a v2 like that if it's easier...

I saw the feedback on 3/4, yeah I also think that "cut" is a bit nasty, but probably not worth spending more time on it...

My assumption in writing this patch was that it was fine because we test the details of abbrev behavior somewhere else, surely...

But is it turns out we don't, I was misrecalling that some version of my testing for abbreviations in [1] had landed.

But alas it didn't. From rebasing it now (it's far from submission quality yet) I see that the lack of testing on abbreviations in the interim has silently caused some regressions on master.

E.g. we seemingly unintentionally started accepting "-c core.abbrev=" (empty string) in a9ecaa06a7 (core.abbrev=no disables abbreviations, 2020-09-01) is a side-effect of starting to accept boolean values. Also "git branch --abbrev=-12345" became a non-error in the interim (haven't dug, but some of the refactoring to parse_options() I assume..).

So yeah, I think this patch is a bad idea now, our test coverage for abbreviations is really hanging on by a thread.

1. https://lore.kernel.org/git/20180608224136.20220-1-avarab@gmail.com/
Junio C Hamano· Mar 11, 2021, 17:40 UTC · re: Ævar Arnfjörð Bjarmason · lore

Re: [PATCH 4/4] tests: get rid of $_x05 from the test suite

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
Show 20 quoted lines
>>>  test_expect_success 'ls-tree --abbrev=5' '
>>>  	git ls-tree --abbrev=5 $tree >current &&
>>> -	sed -e "s/ $_x05[0-9a-f]*	/ X	/" <current >check &&
>>> +	sed -e "s/ [0-9a-f]*	/ X	/" <current >check &&
>>>  	cat >expected <<\EOF &&
>>>  100644 blob X	1.txt
>>>  100644 blob X	2.txt
>>
>> This one is particularly iffy.  The --abbrev=5 test is designed to
>> ensure that the resulting abbreviated object names are at least 5
>> hexdigits long, even when the repository is so small that only 4
>> hexdigits are sufficient to avoid ambiguity, while allowing the
>> output to be longer than specified 5 (when 5 turns out to be
>> insufficient for disambiguation).
>>
>> So, I dunno.
>
> Yes, I think this patch should be dropped. Do you mind just dropping the
> 4/4 and having it be a 3-patch series? I can also re-submit a v2 like
> that if it's easier...

As long as the earlier three are good to go, I can just cut the tip of the branch, or just drop it all now and send a three-patch series after the release. Either is fine.

> My assumption in writing this patch was that it was fine because we test
> the details of abbrev behavior somewhere else, surely...

I expected that the test whose title is "ls-tree --abbrev=5" is targetted towards testing the details of abbrev behaviour. Isn't that the case?

Is the same assumption brought silent breakages to the other two patches to the tests, by chance? I only gave cursory look on them and don't think anybody else looked at them carefully.

Ævar Arnfjörð Bjarmason· Mar 11, 2021, 00:14 UTC · re: Jeff King · lore

[PATCH 3/4] shortlog tests: rewrite to get rid of --abbrev=35 hardcoding

Rewrite much of the test logic introduced in 600372497c (shortlog: Document and test --format option, 2010-05-03), this allows us to get rid of the now-unused $_x35 variable in test-lib.sh

There was a minimal migration of this test to SHA-256 in 2ece6ad281 (t: switch $_x40 to $OID_REGEX, 2018-05-13), but actually we can just get rid of all the assumptions about hashing here, and make this easier to understand and maintain while we're at it.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t4201-shortlog.sh | 92 +++++++++++++++++++++++++--------------------
 t/test-lib.sh       |  5 +--
 2 files changed, 54 insertions(+), 43 deletions(-)
Show changes to 2 files +54 −43

t/t4201-shortlog.sh, t/test-lib.sh

diff --git a/t/t4201-shortlog.sh b/t/t4201-shortlog.sh
index 3095b1b2ff..3ef17c06e4 100755
--- a/t/t4201-shortlog.sh
+++ b/t/t4201-shortlog.sh
@@ -19,82 +19,94 @@ test_expect_success 'setup' '
 	commit=$(printf "%s\n" "Test" "" | git commit-tree "$tree") &&
 	git update-ref HEAD "$commit" &&
 
+	echo "This is a very, very long first line for the commit message to see if it is wrapped correctly" >message &&
+
+	sed "s/i/1234/g" <message >tmp &&
+	tr 1234 "\360\235\204\236" <tmp >message.2 &&
+
+	sed "s/i/1234/g" <message >tmp &&
+	tr 1234 "\370\235\204\236" <tmp >message.3 &&
+
+	echo "a								12	34	56	78" >message.4 &&
+	echo "Commit by someone else" >message.5 &&
+
 	echo 2 >a1 &&
-	git commit --quiet -m "This is a very, very long first line for the commit message to see if it is wrapped correctly" a1 &&
+	git commit --quiet -F message a1 &&
 
 	# test if the wrapping is still valid
 	# when replacing all is by treble clefs.
 	echo 3 >a1 &&
-	git commit --quiet -m "$(
-		echo "This is a very, very long first line for the commit message to see if it is wrapped correctly" |
-		sed "s/i/1234/g" |
-		tr 1234 "\360\235\204\236")" a1 &&
+	git commit --quiet -F message.2 a1 &&
 
 	# now fsck up the utf8
 	git config i18n.commitencoding non-utf-8 &&
 	echo 4 >a1 &&
-	git commit --quiet -m "$(
-		echo "This is a very, very long first line for the commit message to see if it is wrapped correctly" |
-		sed "s/i/1234/g" |
-		tr 1234 "\370\235\204\236")" a1 &&
+	git commit --quiet -F message.3 a1 &&
 
 	echo 5 >a1 &&
-	git commit --quiet -m "a								12	34	56	78" a1 &&
+	git commit --quiet -F message.4 a1 &&
 
 	echo 6 >a1 &&
-	git commit --quiet -m "Commit by someone else" \
+	git commit --quiet -F message.5  \
 		--author="Someone else <not!me>" a1 &&
 
-	cat >expect.template <<-\EOF
+	cat >expect.default <<-EOF
 	A U Thor (5):
-	      SUBJECT
-	      SUBJECT
-	      SUBJECT
-	      SUBJECT
-	      SUBJECT
+	      Test
+	      $(cat message)
+	      $(cat message.2)
+	      $(cat message.3)
+	      $(cat message.4)
 
 	Someone else (1):
-	      SUBJECT
+	      $(cat message.5)
 
 	EOF
 '
 
-fuzz() {
-	file=$1 &&
-	sed "
-			s/$OID_REGEX/OBJECT_NAME/g
-			s/$_x35/OBJID/g
-			s/^ \{6\}[CTa].*/      SUBJECT/g
-			s/^ \{8\}[^ ].*/        CONTINUATION/g
-		" <"$file" >"$file.fuzzy" &&
-	sed "/CONTINUATION/ d" <"$file.fuzzy"
-}
-
 test_expect_success 'default output format' '
 	git shortlog HEAD >log &&
-	fuzz log >log.predictable &&
-	test_cmp expect.template log.predictable
+	test_cmp expect.default log
 '
 
 test_expect_success 'pretty format' '
-	sed s/SUBJECT/OBJECT_NAME/ expect.template >expect &&
+	cat >expect <<-EOF &&
+	A U Thor (5):
+	      $(git rev-parse HEAD~5)
+	      $(git rev-parse HEAD~4)
+	      $(git rev-parse HEAD~3)
+	      $(git rev-parse HEAD~2)
+	      $(git rev-parse HEAD~1)
+
+	Someone else (1):
+	      $(git rev-parse HEAD~0)
+
+	EOF
 	git shortlog --format="%H" HEAD >log &&
-	fuzz log >log.predictable &&
-	test_cmp expect log.predictable
+	test_cmp expect log
 '
 
 test_expect_success '--abbrev' '
-	sed s/SUBJECT/OBJID/ expect.template >expect &&
+	cut -c 1-41 <expect >expect.abbrev &&
 	git shortlog --format="%h" --abbrev=35 HEAD >log &&
-	fuzz log >log.predictable &&
-	test_cmp expect log.predictable
+	test_cmp expect.abbrev log
 '
 
 test_expect_success 'output from user-defined format is re-wrapped' '
-	sed "s/SUBJECT/two lines/" expect.template >expect &&
+	cat >expect <<-EOF &&
+	A U Thor (5):
+	      two lines
+	      two lines
+	      two lines
+	      two lines
+	      two lines
+
+	Someone else (1):
+	      two lines
+
+	EOF
 	git shortlog --format="two%nlines" HEAD >log &&
-	fuzz log >log.predictable &&
-	test_cmp expect log.predictable
+	test_cmp expect log
 '
 
 test_expect_success !MINGW 'shortlog wrapping' '
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 5f2ad2fd81..4d5ba558d3 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -511,7 +511,7 @@ SQ=\'
 # when case-folding filenames
 u200c=$(printf '\342\200\214')
 
-export _x05 _x35 LF u200c EMPTY_TREE EMPTY_BLOB ZERO_OID OID_REGEX
+export _x05 LF u200c EMPTY_TREE EMPTY_BLOB ZERO_OID OID_REGEX
 
 # Each test should start with something like this, after copyright notices:
 #
@@ -1381,9 +1381,8 @@ then
 fi
 
 # Convenience
-# A regexp to match 5 and 35 hexdigits
+# A regexp to match 5 hexdigits
 _x05='[0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f]'
-_x35="$_x05$_x05$_x05$_x05$_x05$_x05$_x05"
 
 test_oid_init
 
-- 
2.31.0.rc1.213.gbfdb770ff55
Junio C Hamano· Mar 11, 2021, 01:18 UTC · re: Ævar Arnfjörð Bjarmason · lore

Re: [PATCH 3/4] shortlog tests: rewrite to get rid of --abbrev=35 hardcoding

Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:
Show 17 quoted lines
> +	cat >expect <<-EOF &&
> +	A U Thor (5):
> +	      $(git rev-parse HEAD~5)
> +	      $(git rev-parse HEAD~4)
> +	      $(git rev-parse HEAD~3)
> +	      $(git rev-parse HEAD~2)
> +	      $(git rev-parse HEAD~1)
> +
> +	Someone else (1):
> +	      $(git rev-parse HEAD~0)
> +
> +	EOF
> ...
>  test_expect_success '--abbrev' '
> -	sed s/SUBJECT/OBJID/ expect.template >expect &&
> +	cut -c 1-41 <expect >expect.abbrev &&
>  	git shortlog --format="%h" --abbrev=35 HEAD >log &&
The hardcoded 35 is a bit curious, and 1-41 even more so.

Is the idea that the "expect" prepared earlier has full hexdigits and indented by 6 columns, the full hexdigits for an object name is longer than 35 chars in any hash algorithm we use, any sample name plus (number) plus colon won't be 35 chars long, and 35+6 happens to be 41 so cutting at 41 column we'd get the first 35 hexdigits of the object names without losing anything on the summary line?

That's a bit too subtle to my taste, but 
	test_expect_success '--abbrev' '
		abbrev=35 &&
		cut -c 1-$(( abbrev + 6 )) expect >expect.abbrev &&
		git shortlog --format="%h" --abbrev=$abbrev HEAD >log &&
		...

may not make it much easier to follow (without explaining what that 6 is anyway), so I'll let it pass.

Otherwise the steps so far make quite a lot of sense.
Thanks.
Jeff King· Mar 11, 2021, 19:15 UTC · re: Ævar Arnfjörð Bjarmason · lore

Re: [PATCH 3/4] shortlog tests: rewrite to get rid of --abbrev=35 hardcoding

On Thu, Mar 11, 2021 at 01:14:46AM +0100, Ævar Arnfjörð Bjarmason wrote:
Show 8 quoted lines
> Rewrite much of the test logic introduced in 600372497c (shortlog:
> Document and test --format option, 2010-05-03), this allows us to get
> rid of the now-unused $_x35 variable in test-lib.sh
> 
> There was a minimal migration of this test to SHA-256 in
> 2ece6ad281 (t: switch $_x40 to $OID_REGEX, 2018-05-13), but actually
> we can just get rid of all the assumptions about hashing here, and
> make this easier to understand and maintain while we're at it.

Sounds like a good goal. I had to spend a few minutes deciphering what was going on in the diff, so I'll follow along loudly...

Show 19 quoted lines
> diff --git a/t/t4201-shortlog.sh b/t/t4201-shortlog.sh
> index 3095b1b2ff..3ef17c06e4 100755
> --- a/t/t4201-shortlog.sh
> +++ b/t/t4201-shortlog.sh
> @@ -19,82 +19,94 @@ test_expect_success 'setup' '
>  	commit=$(printf "%s\n" "Test" "" | git commit-tree "$tree") &&
>  	git update-ref HEAD "$commit" &&
>  
> +	echo "This is a very, very long first line for the commit message to see if it is wrapped correctly" >message &&
> +
> +	sed "s/i/1234/g" <message >tmp &&
> +	tr 1234 "\360\235\204\236" <tmp >message.2 &&
> +
> +	sed "s/i/1234/g" <message >tmp &&
> +	tr 1234 "\370\235\204\236" <tmp >message.3 &&
> +
> +	echo "a								12	34	56	78" >message.4 &&
> +	echo "Commit by someone else" >message.5 &&
> +

This is all moving the messages into files so that you can pull them into the expected output later. OK (feels weird to use sed and tr together like this, but maybe it's a portability thing for sed which doesn't understand octal; anyway, it's definitely not new in your patch).

That might have been easier to understand if each message was generated next to its comment. E.g...

Show 8 quoted lines
>  	# test if the wrapping is still valid
>  	# when replacing all is by treble clefs.
>  	echo 3 >a1 &&
> -	git commit --quiet -m "$(
> -		echo "This is a very, very long first line for the commit message to see if it is wrapped correctly" |
> -		sed "s/i/1234/g" |
> -		tr 1234 "\360\235\204\236")" a1 &&
> +	git commit --quiet -F message.2 a1 &&

...if we made message.2 here, then the comment would be explaining what the opaque bytes are doing (and likewise why they differ in message.3).

Also, the mismatch between the "3" going into the file and "message.2" is confusing. It's not important, but I wonder if using "commit --allow-empty" and just skipping a1, etc, would make it easier to read.

Similarly, --quiet seems pointless here. If the test isn't run with --verbose, the user doesn't see it either way. If it is, then they probably want to see the output.

Show 19 quoted lines
> -	cat >expect.template <<-\EOF
> +	cat >expect.default <<-EOF
>  	A U Thor (5):
> -	      SUBJECT
> -	      SUBJECT
> -	      SUBJECT
> -	      SUBJECT
> -	      SUBJECT
> +	      Test
> +	      $(cat message)
> +	      $(cat message.2)
> +	      $(cat message.3)
> +	      $(cat message.4)
>  
>  	Someone else (1):
> -	      SUBJECT
> +	      $(cat message.5)
>  
>  	EOF

OK, and here's where we make the actual expected output, rather than a template. I think this is an improvement in understanding what's going on (and also is more robust).

I was slightly confused that we are not looking for the messages to have been wrapped (just based on the surrounding code), but that is not new to your patch.

Show 20 quoted lines
> -fuzz() {
> -	file=$1 &&
> -	sed "
> -			s/$OID_REGEX/OBJECT_NAME/g
> -			s/$_x35/OBJID/g
> -			s/^ \{6\}[CTa].*/      SUBJECT/g
> -			s/^ \{8\}[^ ].*/        CONTINUATION/g
> -		" <"$file" >"$file.fuzzy" &&
> -	sed "/CONTINUATION/ d" <"$file.fuzzy"
> -}
> [...]
>  test_expect_success 'pretty format' '
> -	sed s/SUBJECT/OBJECT_NAME/ expect.template >expect &&
> +	cat >expect <<-EOF &&
> +	A U Thor (5):
> +	      $(git rev-parse HEAD~5)
> +	      $(git rev-parse HEAD~4)
> +	      $(git rev-parse HEAD~3)
> +	      $(git rev-parse HEAD~2)
> +	      $(git rev-parse HEAD~1)

And here we expect the real hashes rather than the fuzz() magic, which makes sense.

Aside: here and in the others, I cringe a little at the cost of all of
the $() calls inside the here-doc. But I don't think there's a better
way to do it. I'd sometimes use environment variables for this, but the
processing we're doing probably makes it better to keep them in real
files. Or maybe not. TBH, just expanding out the messages in the source
like:
  m1=$(printf "Th\370\235\204\236s \370\235\204\236s a very, very long f\370\235\204\236rst l\370\235\204\236ne for the comm\370\235\204\236t message to see \370\235\204\236f \370\235\204\236t \370\235\204\236s wrapped correctly")

is equally unreadable to the original to me. ;) Possibly it would be even more readable if some more meaningful pattern of replacements was used. But I am firmly in bikeshedding territory there, so feel free to ignore.

Show 8 quoted lines
>  test_expect_success '--abbrev' '
> -	sed s/SUBJECT/OBJID/ expect.template >expect &&
> +	cut -c 1-41 <expect >expect.abbrev &&
>  	git shortlog --format="%h" --abbrev=35 HEAD >log &&
> -	fuzz log >log.predictable &&
> -	test_cmp expect log.predictable
> +	test_cmp expect.abbrev log
>  '
And this is the same thing as %H, but abbreviated.

I agree with Junio that the "1-41" seems magical. His suggestion to show the math helps with that (or even just a comment). You could also just build the "expect" file with:

  $(git rev-parse --abbrev=35 HEAD~5)

etc (I also have to wonder if there is much value in this beyond checking the first commit, but that certainly predates your patch).

Show 19 quoted lines
>  test_expect_success 'output from user-defined format is re-wrapped' '
> -	sed "s/SUBJECT/two lines/" expect.template >expect &&
> +	cat >expect <<-EOF &&
> +	A U Thor (5):
> +	      two lines
> +	      two lines
> +	      two lines
> +	      two lines
> +	      two lines
> +
> +	Someone else (1):
> +	      two lines
> +
> +	EOF
>  	git shortlog --format="two%nlines" HEAD >log &&
> -	fuzz log >log.predictable &&
> -	test_cmp expect log.predictable
> +	test_cmp expect log
>  '
And this one is IMHO much improved in readability.
Show 11 quoted lines
>  test_expect_success !MINGW 'shortlog wrapping' '
> diff --git a/t/test-lib.sh b/t/test-lib.sh
> index 5f2ad2fd81..4d5ba558d3 100644
> --- a/t/test-lib.sh
> +++ b/t/test-lib.sh
> @@ -511,7 +511,7 @@ SQ=\'
>  # when case-folding filenames
>  u200c=$(printf '\342\200\214')
>  
> -export _x05 _x35 LF u200c EMPTY_TREE EMPTY_BLOB ZERO_OID OID_REGEX
> +export _x05 LF u200c EMPTY_TREE EMPTY_BLOB ZERO_OID OID_REGEX
And now we can get rid of _x05. Yay.
-Peff
Ævar Arnfjörð Bjarmason· Mar 11, 2021, 00:14 UTC · re: Jeff King · lore

[PATCH 2/4] test-lib: remove unused $_x40 and $_z40 variables

These two have fallen out of use with the SHA-256 migration.

The last use of $_x40 was removed in fc7e73d7ef (t4013: improve diff-post-processor logic, 2020-08-21) and

The last use of $_z40 was removed in 7a868c51c2 (t5562: use $ZERO_OID, 2019-12-21), but it was then needlessly refactored to be hash-agnostic in 192b517589 (t: use hash-specific lookup tables to define test constants, 2020-02-22). We can just remove it.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/test-lib.sh | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)
Show changes to t/test-lib.sh +2 −4
diff --git a/t/test-lib.sh b/t/test-lib.sh
index d3f6af6a65..5f2ad2fd81 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -511,7 +511,7 @@ SQ=\'
 # when case-folding filenames
 u200c=$(printf '\342\200\214')
 
-export _x05 _x35 _x40 _z40 LF u200c EMPTY_TREE EMPTY_BLOB ZERO_OID OID_REGEX
+export _x05 _x35 LF u200c EMPTY_TREE EMPTY_BLOB ZERO_OID OID_REGEX
 
 # Each test should start with something like this, after copyright notices:
 #
@@ -1381,10 +1381,9 @@ then
 fi
 
 # Convenience
-# A regexp to match 5, 35 and 40 hexdigits
+# A regexp to match 5 and 35 hexdigits
 _x05='[0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f]'
 _x35="$_x05$_x05$_x05$_x05$_x05$_x05$_x05"
-_x40="$_x35$_x05"
 
 test_oid_init
 
@@ -1393,7 +1392,6 @@ OID_REGEX=$(echo $ZERO_OID | sed -e 's/0/[0-9a-f]/g')
 OIDPATH_REGEX=$(test_oid_to_path $ZERO_OID | sed -e 's/0/[0-9a-f]/g')
 EMPTY_TREE=$(test_oid empty_tree)
 EMPTY_BLOB=$(test_oid empty_blob)
-_z40=$ZERO_OID
 
 # Provide an implementation of the 'yes' utility; the upper bound
 # limit is there to help Windows that cannot stop this loop from
-- 
2.31.0.rc1.213.gbfdb770ff55
Junio C Hamano· Mar 11, 2021, 01:02 UTC · re: Ævar Arnfjörð Bjarmason · lore

Re: [PATCH 2/4] test-lib: remove unused $_x40 and $_z40 variables

Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:
Show 14 quoted lines
> These two have fallen out of use with the SHA-256 migration.
>
> The last use of $_x40 was removed in fc7e73d7ef (t4013: improve
> diff-post-processor logic, 2020-08-21) and
>
> The last use of $_z40 was removed in 7a868c51c2 (t5562: use $ZERO_OID,
> 2019-12-21), but it was then needlessly refactored to be hash-agnostic
> in 192b517589 (t: use hash-specific lookup tables to define test
> constants, 2020-02-22). We can just remove it.
>
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> ---
>  t/test-lib.sh | 6 ++----
>  1 file changed, 2 insertions(+), 4 deletions(-)

Nice. Curious why we still want x5 and x35, but we'll learn soon enough in this short series ;-)

Show 33 quoted lines
> diff --git a/t/test-lib.sh b/t/test-lib.sh
> index d3f6af6a65..5f2ad2fd81 100644
> --- a/t/test-lib.sh
> +++ b/t/test-lib.sh
> @@ -511,7 +511,7 @@ SQ=\'
>  # when case-folding filenames
>  u200c=$(printf '\342\200\214')
>  
> -export _x05 _x35 _x40 _z40 LF u200c EMPTY_TREE EMPTY_BLOB ZERO_OID OID_REGEX
> +export _x05 _x35 LF u200c EMPTY_TREE EMPTY_BLOB ZERO_OID OID_REGEX
>  
>  # Each test should start with something like this, after copyright notices:
>  #
> @@ -1381,10 +1381,9 @@ then
>  fi
>  
>  # Convenience
> -# A regexp to match 5, 35 and 40 hexdigits
> +# A regexp to match 5 and 35 hexdigits
>  _x05='[0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f]'
>  _x35="$_x05$_x05$_x05$_x05$_x05$_x05$_x05"
> -_x40="$_x35$_x05"
>  
>  test_oid_init
>  
> @@ -1393,7 +1392,6 @@ OID_REGEX=$(echo $ZERO_OID | sed -e 's/0/[0-9a-f]/g')
>  OIDPATH_REGEX=$(test_oid_to_path $ZERO_OID | sed -e 's/0/[0-9a-f]/g')
>  EMPTY_TREE=$(test_oid empty_tree)
>  EMPTY_BLOB=$(test_oid empty_blob)
> -_z40=$ZERO_OID
>  
>  # Provide an implementation of the 'yes' utility; the upper bound
>  # limit is there to help Windows that cannot stop this loop from
Jeff King· Mar 10, 2021, 17:07 UTC · re: Jeff King · lore

[PATCH 1/3] t7003: test ref rewriting explicitly

After it has rewritten all of the commits, filter-branch will then rewrite each of the input refs based on the resulting map of old/new commits. But we don't have any explicit test coverage of this code. Let's make sure we are covering each of those cases:

  - deleting a ref when all of its commits were pruned
  - rewriting a ref based on the mapping (this happens throughout the
    script, but let's make sure we generate the correct messages)
  - rewriting a ref whose tip was excluded, in which case we rewrite to
    the nearest ancestor. Note in this case that we still insist that no
    "warning" line is present (even though it looks like we'd trigger
    the "... was rewritten into multiple commits" one). See the next
    commit for more details.

Note these all pass currently, but the latter two will fail when run with GIT_TEST_DEFAULT_HASH=sha256.

Signed-off-by: Jeff King <peff@peff.net>
---
 t/t7003-filter-branch.sh | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)
Show changes to t/t7003-filter-branch.sh +31 −0
diff --git a/t/t7003-filter-branch.sh b/t/t7003-filter-branch.sh
index 1c55695034..1349e5b232 100755
--- a/t/t7003-filter-branch.sh
+++ b/t/t7003-filter-branch.sh
@@ -506,4 +506,35 @@ test_expect_success 'rewrite repository including refs that point at non-commit
 	! fgrep fatal filter-output
 '
 
+test_expect_success 'filter-branch handles ref deletion' '
+	git switch --orphan empty-commit &&
+	git commit --allow-empty -m "empty commit" &&
+	git tag empty &&
+	git branch to-delete &&
+	git filter-branch -f --prune-empty to-delete >out 2>&1 &&
+	grep "to-delete.*was deleted" out &&
+	test_must_fail git rev-parse --verify to-delete
+'
+
+test_expect_success 'filter-branch handles ref rewrite' '
+	git checkout empty &&
+	test_commit to-drop &&
+	git branch rewrite &&
+	git filter-branch -f \
+		--index-filter "git rm --ignore-unmatch --cached to-drop.t" \
+		 rewrite >out 2>&1 &&
+	grep "rewrite.*was rewritten" out &&
+	! grep -i warning out &&
+	git diff-tree empty rewrite
+'
+
+test_expect_success 'filter-branch handles ancestor rewrite' '
+	test_commit to-exclude &&
+	git branch ancestor &&
+	git filter-branch -f ancestor -- :^to-exclude.t >out 2>&1 &&
+	grep "ancestor.*was rewritten" out &&
+	! grep -i warning out &&
+	git diff-tree HEAD^ ancestor
+'
+
 test_done
-- 
2.31.0.rc2.525.gc2268d2248
Ævar Arnfjörð Bjarmason· Mar 11, 2021, 00:10 UTC · re: Jeff King · lore

Re: [PATCH 1/3] t7003: test ref rewriting explicitly

On Wed, Mar 10 2021, Jeff King wrote:
I've read it all over, looks good to me.
> Note these all pass currently, but the latter two will fail when run
> with GIT_TEST_DEFAULT_HASH=sha256.
I don't know if anyone relies on that. Probably fine to have this as-is.

But wouldn't fixing that just be a matter of adding these tests as test_expect_failure, and then flipping them to test_expect_success in 3/3?

brian m. carlson· Mar 11, 2021, 01:55 UTC · re: Ævar Arnfjörð Bjarmason · lore

Re: [PATCH 1/3] t7003: test ref rewriting explicitly

On 2021-03-11 at 00:10:29, Ævar Arnfjörð Bjarmason wrote:
Show 10 quoted lines
> 
> On Wed, Mar 10 2021, Jeff King wrote:
> 
> 
> I've read it all over, looks good to me.
> 
> > Note these all pass currently, but the latter two will fail when run
> > with GIT_TEST_DEFAULT_HASH=sha256.
> 
> I don't know if anyone relies on that. Probably fine to have this as-is.

Our CI runs that way, and I always use that mode. Are you saying that these will always fail in that mode, or that they will pass at the end of the series?

-- 
brian m. carlson (he/him or they/them)
Houston, Texas, US
Jeff King· Mar 11, 2021, 02:24 UTC · re: brian m. carlson · lore

Re: [PATCH 1/3] t7003: test ref rewriting explicitly

On Thu, Mar 11, 2021 at 01:55:04AM +0000, brian m. carlson wrote:
Show 15 quoted lines
> On 2021-03-11 at 00:10:29, Ævar Arnfjörð Bjarmason wrote:
> > 
> > On Wed, Mar 10 2021, Jeff King wrote:
> > 
> > 
> > I've read it all over, looks good to me.
> > 
> > > Note these all pass currently, but the latter two will fail when run
> > > with GIT_TEST_DEFAULT_HASH=sha256.
> > 
> > I don't know if anyone relies on that. Probably fine to have this as-is.
> 
> Our CI runs that way, and I always use that mode.  Are you saying that
> these will always fail in that mode, or that they will pass at the end
> of the series?

The latter; everything passes at the end. The only thing that is hurt is if we care about bisecting the tests with GIT_TEST_DEFAULT_HASH set

-Peff
brian m. carlson· Mar 11, 2021, 02:49 UTC · re: Jeff King · lore

Re: [PATCH 1/3] t7003: test ref rewriting explicitly

On 2021-03-11 at 02:24:29, Jeff King wrote:
Show 17 quoted lines
> On Thu, Mar 11, 2021 at 01:55:04AM +0000, brian m. carlson wrote:
> 
> > On 2021-03-11 at 00:10:29, Ævar Arnfjörð Bjarmason wrote:
> > > 
> > > On Wed, Mar 10 2021, Jeff King wrote:
> > > 
> > > > Note these all pass currently, but the latter two will fail when run
> > > > with GIT_TEST_DEFAULT_HASH=sha256.
> > > 
> > > I don't know if anyone relies on that. Probably fine to have this as-is.
> > 
> > Our CI runs that way, and I always use that mode.  Are you saying that
> > these will always fail in that mode, or that they will pass at the end
> > of the series?
> 
> The latter; everything passes at the end. The only thing that is hurt is
> if we care about bisecting the tests with GIT_TEST_DEFAULT_HASH set

I don't have a strong opinion on this. I'm unlikely to bisect with the full testsuite in that way; I'd use a git bisect run with a smaller testcase.

-- 
brian m. carlson (he/him or they/them)
Houston, Texas, US
Jeff King· Mar 10, 2021, 17:07 UTC · re: Jeff King · lore

[PATCH 2/3] filter-branch: drop multiple-ancestor warning

When a ref maps to a commit that is neither rewritten nor kept by filter-branch (e.g., because it was eliminated by rev-list's pathspec selection), we rewrite it to its nearest ancestor.

Since the initial commit in 6f6826c52b (Add git-filter-branch, 2007-06-03), we have warned when there are multiple such ancestors in the map file. However, the warning code is impossible to trigger these days. Since a0e46390d3 (filter-branch: fix ref rewriting with --subdirectory-filter, 2008-08-12), we find the ancestor using "rev-list -1", so it can only ever have a single value.

This code is made doubly confusing by the fact that we append to the map file when mapping ancestors. However, this can never yield multiple values because:

  - we explicitly check whether the map already exists, and if so, do
    nothing (so our "append" will always be to a file that does not
    exist)
  - even if we were to try mapping twice, the process to do so is
    deterministic. I.e., we'd always end up with the same ancestor for a
    given sha1. So warning about it would be pointless; there is no
    ambiguity.

So swap out the warning code for a BUG (which we'll simplify further in the next commit). And let's stop using the append operator to make the ancestor-mapping code less confusing.

Signed-off-by: Jeff King <peff@peff.net>
---
 git-filter-branch.sh | 11 ++---------
 1 file changed, 2 insertions(+), 9 deletions(-)
Show changes to git-filter-branch.sh +2 −9
diff --git a/git-filter-branch.sh b/git-filter-branch.sh
index fea7964617..a1e80bd552 100755
--- a/git-filter-branch.sh
+++ b/git-filter-branch.sh
@@ -492,7 +492,7 @@ then
 		sha1=$(git rev-parse "$ref"^0)
 		test -f "$workdir"/../map/$sha1 && continue
 		ancestor=$(git rev-list --simplify-merges -1 "$ref" "$@")
-		test "$ancestor" && echo $(map $ancestor) >> "$workdir"/../map/$sha1
+		test "$ancestor" && echo $(map $ancestor) >"$workdir"/../map/$sha1
 	done < "$tempdir"/heads
 fi
 
@@ -534,14 +534,7 @@ do
 		fi
 	;;
 	*)
-		# NEEDSWORK: possibly add -Werror, making this an error
-		warn "WARNING: '$ref' was rewritten into multiple commits:"
-		warn "$rewritten"
-		warn "WARNING: Ref '$ref' points to the first one now."
-		rewritten=$(echo "$rewritten" | head -n 1)
-		git update-ref -m "filter-branch: rewrite to first" \
-				"$ref" $rewritten $sha1 ||
-			die "Could not rewrite $ref"
+		die "BUG: multiple ancestors in map file?"
 	;;
 	esac
 	git update-ref -m "filter-branch: backup" "$orig_namespace$ref" $sha1 ||
-- 
2.31.0.rc2.525.gc2268d2248
Junio C Hamano· Mar 10, 2021, 22:16 UTC · re: Jeff King · lore

Re: [PATCH 2/3] filter-branch: drop multiple-ancestor warning

Jeff King <peff@peff.net> writes:
Show 10 quoted lines
> When a ref maps to a commit that is neither rewritten nor kept by
> filter-branch (e.g., because it was eliminated by rev-list's pathspec
> selection), we rewrite it to its nearest ancestor.
>
> Since the initial commit in 6f6826c52b (Add git-filter-branch,
> 2007-06-03), we have warned when there are multiple such ancestors in
> the map file. However, the warning code is impossible to trigger these
> days. Since a0e46390d3 (filter-branch: fix ref rewriting with
> --subdirectory-filter, 2008-08-12), we find the ancestor using "rev-list
> -1", so it can only ever have a single value.
;-)
Elijah Newren· Mar 10, 2021, 17:54 UTC · re: Jeff King · lore

Re: [PATCH 0/3] sha256 fixes for filter-branch

On Wed, Mar 10, 2021 at 9:06 AM Jeff King <peff@peff.net> wrote:
Show 37 quoted lines
>
> On Wed, Mar 10, 2021 at 01:04:37PM +0100, Ævar Arnfjörð Bjarmason wrote:
>
> > I found a missing spot that wasn't trivial to fix, so sending an E-Mail:
> >
> > In git-filter-branch.sh we have:
> >
> > _x40='[0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f]'
> > _x40="$_x40$_x40$_x40$_x40$_x40$_x40$_x40$_x40"
> >
> > Then later we have a case condition based on matching a SHA-1:
> >
> >         $_x40)
> >                 echo "Ref '$ref' was rewritten"
> >                 if ! git update-ref -m "filter-branch: rewrite" \
> >
> > Just deleting that case arm has filter-branch tests passing, so whatever
> > it's meant to do it has zero coverage, which explains why it hasn't
> > broken with our tests.
>
> It actually does get covered. Dropping that case-arm means we'll fall
> through to the one below, which does _roughly_ the same thing with a
> bunch of extra warnings. But none of the tests actually check the error
> messages, so they don't notice.
>
> Here's a series which fixes it, plus extra tests to notice the
> distinction.
>
>   [1/3]: t7003: test ref rewriting explicitly
>   [2/3]: filter-branch: drop multiple-ancestor warning
>   [3/3]: filter-branch: drop $_x40 glob
>
>  git-filter-branch.sh     | 16 ++--------------
>  t/t7003-filter-branch.sh | 31 +++++++++++++++++++++++++++++++
>  2 files changed, 33 insertions(+), 14 deletions(-)
>
> -Peff
Ævar successfully nerd sniped you.

...and sucked me in too, since I went through and read your patches. :-) They look good to me.

Junio C Hamano· Mar 10, 2021, 22:24 UTC · re: Elijah Newren · lore

Re: [PATCH 0/3] sha256 fixes for filter-branch

Elijah Newren <newren@gmail.com> writes:
> Ævar successfully nerd sniped you.
>
> ...and sucked me in too, since I went through and read your patches.
> :-)  They look good to me.
Yup, thanks, both.
brian m. carlson· Mar 10, 2021, 23:32 UTC · re: Jeff King · lore

Re: [PATCH 0/3] sha256 fixes for filter-branch

On 2021-03-10 at 17:06:33, Jeff King wrote:
Show 11 quoted lines
> It actually does get covered. Dropping that case-arm means we'll fall
> through to the one below, which does _roughly_ the same thing with a
> bunch of extra warnings. But none of the tests actually check the error
> messages, so they don't notice.
> 
> Here's a series which fixes it, plus extra tests to notice the
> distinction.
> 
>   [1/3]: t7003: test ref rewriting explicitly
>   [2/3]: filter-branch: drop multiple-ancestor warning
>   [3/3]: filter-branch: drop $_x40 glob
This series looked good to me.  Thanks for sending it.
-- 
brian m. carlson (he/him or they/them)
Houston, Texas, US

← back to recent threads