[PATCH v2 0/3] repo: add degree-of-concern reporting to structure
- From
- Mark C. Chu-Carroll <markchucarroll@fastmail.com>
- Date
- Oct 26, 2026, 17:43 UTC
- Message-ID
- <20261026-b4-repo-structure-concern-v2-0-e0df34228dc8@fastmail.com>
- In-Reply-To
- <20261021-b4-repo-structure-concern-v1-0-e3010552292c@fastmail.com>
"git repo structure" computes a set of statistics that are useful for diagnosing performance issues in a git repository. But for a typical user who hasn't encountered any issues yet, it can be difficult to interpret the data - when should a value be considered concerning?
This patchset introduces a feature to "git repo structure" that reports on how concerning certain measures of storage structure are for a repository. In table mode, concern is displayed as a column containing a number of stars indicating the degree of concern; in line or nul mode, concerns are displayed as additionaL lines with a numeric value.
To select defaults for the thresholds where a value becomes concern, a (hopefully) representative selection of repositories were analyzed. Based on empirical exploration of the data, the measurement that seemed closest to accurately assessing when a repository's structure becomes concerning was based on the base 2 logarithm of the value. Across all of the sample repositories, for all of the measurements, the point at which things become problematic was when a base 2 logarithm of a value was more than 1 standard deviation from the mean of the log_2 values in the sample set. Based on this, the concern threshold was set for each of the measures.
While this seems like a reasonable approach for setting default thresholds, it's obviously still very ad-hoc. It's likely that over time, the default values will evolve; and in the short term, projects may want to set their own thresholds. So this patchset includes support for configuring the thresholds on a per-reposiry basis.
Overview of the change:
* The first commit adds the command-line option "--concern", and creates the basic implementation of concern reporting. * The next commit adds configurability for the concern thresholds. * The final commit update the documentation for the command and configuration options.
Signed-off-by: Mark C. Chu-Carroll <markchucarroll@fastmail.com> --- Changes in v2: - Fix memory leak in concern string generation. - Fix CI/CD test issue involving zlib versions. - Updating the naming of the config value prefix. - Link to v1: https://patch.msgid.link/20261021-b4-repo-structure-concern-v1-0-e3010552292c@fastmail.com
---
Mark C. Chu-Carroll (3):
repo: add level of concern reporting
repo: make concern thresholds configurable
repo: add documentation for level-of-concern.Documentation/config/repo.adoc | 59 +++++++ Documentation/git-repo.adoc | 27 +++- builtin/repo.c | 343 ++++++++++++++++++++++++++++++++++++++--- t/t1901-repo-structure.sh | 137 ++++++++++++++++ 4 files changed, 543 insertions(+), 23 deletions(-)
Range-diff versus v1:
1: c2b2a79967 = 1: e8971a4050 repo: add level of concern reporting
2: 67d77cfcd9 ! 2: 7a78eee517 repo: make concern thresholds configurable
@@ builtin/repo.c
#include "environment.h"
#include "hash.h"
#include "hex.h"
+@@
+ #define REPO_STRUCTURE_USAGE \
+ "git repo structure [--format=(table|lines|nul) | -z] [--[no-]concern]"
+
++#define REPO_STRUCTURE_CONFIG_PREFIX "repo.concern."
++
+ static const char *const repo_usage[] = {
+ REPO_INFO_USAGE,
+ REPO_STRUCTURE_USAGE,
@@ builtin/repo.c: enum concern_stat {
CONCERN_STAT_TREE_INFLATED_SIZE,
CONCERN_STAT_BLOB_INFLATED_SIZE,
@@ builtin/repo.c: enum concern_stat {
- { .name = "tag_inflated_size", .threshold = 25}
+struct concern_stat_threshold concern_stat_thresholds[] = {
+ { .name = "commit-count", .threshold = 19 },
-+ {.name = "tree-count", .threshold = 21 },
++ { .name = "tree-count", .threshold = 21 },
+ { .name = "blob-count", .threshold = 22 },
+ { .name = "tag-count", .threshold = 15 },
+ { .name = "commit-size", .threshold = 28 },
@@ builtin/repo.c: enum concern_stat {
};
/*
+ * Concern thresholds aren't universal. We try to set reasonable default
+ * values, but projects may want to set these thresholds differently. To
+ * support that, they can set the thresholds for their project in the
+- * project's git config, in the section "repo.structure.concern".
++ * project's git config, in the section "repo.concern".
+ */
+ static int config_concern_thresholds(const char *prefixed_var,
+ const char *str_val,
+ const struct config_context *ctx,
+ void *cb UNUSED) {
+- const char *config_prefix = "repo.structure.concern.";
+- if (!strncmp(config_prefix, prefixed_var, strlen(config_prefix))) {
+- const char *concern_name = &prefixed_var[strlen(config_prefix)];
++ if (!strncmp(REPO_STRUCTURE_CONFIG_PREFIX, prefixed_var,
++ strlen(REPO_STRUCTURE_CONFIG_PREFIX))) {
++ const char *concern_name =
++ &prefixed_var[strlen(REPO_STRUCTURE_CONFIG_PREFIX)];
+ for (int i = 0; concern_stat_thresholds[i].name != 0; i++) {
+ if (!strcmp(concern_stat_thresholds[i].name, concern_name)) {
+ int value = git_config_int(prefixed_var, str_val, ctx->kvi);
+@@ builtin/repo.c: static int config_concern_thresholds(const char *prefixed_var,
+ return 0;
+ }
+
+-static size_t approx_log2(long value) {
++static size_t approx_log2(size_t value) {
+ int position = 0;
+ if (value <= 0) {
+ return 0;
@@ builtin/repo.c: static size_t approx_log2(long value) {
static size_t repo_structure_concern_level(enum concern_stat stat_id,
size_t value) {
@@ builtin/repo.c: static size_t approx_log2(long value) {
BUG("unexpected repository stat type");
log_value = approx_log2(value);
if (log_value > concern_stat_thresholds[stat_id].threshold)
+@@ builtin/repo.c: static void stats_table_print_structure(const struct stats_table *table,
+ strbuf_utf8_align(&buf, ALIGN_LEFT, concern_col_width, concern_str.buf);
+ strbuf_addstr(&buf, " |");
+ }
++ strbuf_release(&concern_str);
+ printf("%s\n", buf.buf);
+ }
+
@@ builtin/repo.c: static void structure_keyvalue_print(struct repo_structure *stats,
print_keyvalue("objects.blobs.disk_size", key_delim,
stats->objects.disk_sizes.blobs, value_delim);
@@ builtin/repo.c: static void structure_keyvalue_print(struct repo_structure *stat
repo_structure_concern_level(CONCERN_STAT_BLOB_SIZE,
stats->objects.disk_sizes.blobs),
value_delim);
+@@ builtin/repo.c: static void structure_keyvalue_print(struct repo_structure *stats,
+ stats->objects.largest.commit_size.value),
+ value_delim);
+ print_object_data("objects.trees.max_size", key_delim,
+- &stats->objects.largest.tree_size, value_delim);
++ &stats->objects.largest.tree_size, value_delim);
+ if (show_concern)
+ print_keyvalue("objects.trees.max_size.concern_level", key_delim,
+ repo_structure_concern_level(CONCERN_STAT_TREE_LARGEST,
+@@ builtin/repo.c: static void structure_keyvalue_print(struct repo_structure *stats,
+ stats->objects.largest.blob_size.value),
+ value_delim);
+ print_object_data("objects.tags.max_size", key_delim,
+- &stats->objects.largest.tag_size, value_delim);
++ &stats->objects.largest.tag_size, value_delim);
+ if (show_concern)
+ print_keyvalue("objects.tags.max_size.concern_level", key_delim,
+ repo_structure_concern_level(CONCERN_STAT_TAG_LARGEST,
@@ builtin/repo.c: static int cmd_repo_structure(int argc, const char **argv, const char *prefix,
argc = parse_options(argc, argv, prefix, options, repo_structure_usage, 0);
if (argc)
@@ t/t1901-repo-structure.sh: test_expect_success 'empty repository' '
+ git init repo &&
+ (
+ cd repo &&
++ # differences between zlib and zlib-ng can cause this
++ # test to fail; so we just turn it off.
++ git config core.compression 0 &&
+ # configure thresholds to get concern indications
-+ git config repo.structure.concern.commit-count 7 &&
-+ git config repo.structure.concern.commit-size 14 &&
-+ git config repo.structure.concern.tree-size 10 &&
-+ git config repo.structure.concern.blob-size 8 &&
-+ git config repo.structure.concern.blob-inflated-size 10 &&
-+ git config repo.structure.concern.tag-size 2 &&
-+ git config repo.structure.concern.commit-largest 4 &&
-+ git config repo.structure.concern.blob-largest 2 &&
-+ touch configured &&
++ git config repo.concern.commit-count 7 &&
++ git config repo.concern.commit-size 14 &&
++ git config repo.concern.tree-size 10 &&
++ git config repo.concern.blob-size 8 &&
++ git config repo.concern.blob-inflated-size 10 &&
++ git config repo.concern.tag-size 2 &&
++ git config repo.concern.commit-largest 4 &&
++ git config repo.concern.blob-largest 2 &&
+ test_commit_bulk 1005 &&
+ git tag -a foo -m bar &&
+ oid="$(git rev-parse HEAD)" &&
@@ t/t1901-repo-structure.sh: test_expect_success 'empty repository' '
+ | * Commits | $(object_type_disk_usage commit true) | *** |
+ | * Trees | $(object_type_disk_usage tree true) | ******** |
+ | * Blobs | $(object_type_disk_usage blob true) | ****** |
-+ | * Tags | $(object_type_disk_usage tag) B | **** |
++ | * Tags | $(object_type_disk_usage tag) B | ***** |
+ | | | |
+ | * Largest objects | | |
+ | * Commits | | |
@@ t/t1901-repo-structure.sh: test_expect_success 'empty repository' '
test_expect_success SHA1 'lines and nul format' '
test_when_finished "rm -rf repo" &&
git init repo &&
+@@ t/t1901-repo-structure.sh: test_expect_success SHA1 'lines format with concerns' '
+ git init repo &&
+ (
+ cd repo &&
++ # differences between zlib and zlib-ng can cause this
++ # test to fail; so we just turn it off.
++ git config core.compression 0 &&
+ # configure thresholds to get concern indications
+- git config repo.structure.concern.commit-count 7 &&
+- git config repo.structure.concern.commit-size 14 &&
+- git config repo.structure.concern.tree-size 10 &&
+- git config repo.structure.concern.blob-size 8 &&
+- git config repo.structure.concern.blob-inflated-size 10 &&
+- git config repo.structure.concern.tag-size 2 &&
+- git config repo.structure.concern.commit-largest 4 &&
+- git config repo.structure.concern.blob-largest 2 &&
+- touch configured &&
++ git config repo.concern.commit-count 7 &&
++ git config repo.concern.commit-size 14 &&
++ git config repo.concern.tree-size 10 &&
++ git config repo.concern.blob-size 8 &&
++ git config repo.concern.blob-inflated-size 10 &&
++ git config repo.concern.tag-size 2 &&
++ git config repo.concern.commit-largest 4 &&
++ git config repo.concern.blob-largest 2 &&
+ test_commit_bulk 1005 &&
+ git tag -a foo -m bar &&
+ oid="$(git rev-parse HEAD)" &&
+@@ t/t1901-repo-structure.sh: test_expect_success SHA1 'lines format with concerns' '
+ # Also creates a commit, tree, and blob.
+ git notes add -m foo &&
+
+-
+ # The tags disk size is handled specially due to the
+ # git-rev-list(1) --disk-usage=human option printing the full
+ # "byte/bytes" unit string instead of just "B".
@@ t/t1901-repo-structure.sh: test_expect_success SHA1 'lines format with concerns' '
objects.tags.inflated_size.concern_level=0
objects.commits.disk_size.concern_level=3
objects.trees.disk_size.concern_level=8
- objects.blobs.blob_size.concern_level=6
+- objects.tags.disk_size.concern_level=4
+ objects.blobs.disk_size.concern_level=6
- objects.tags.disk_size.concern_level=4
++ objects.tags.disk_size.concern_level=5
objects.commits.max_size.concern_level=3
objects.trees.max_size.concern_level=0
+ objects.blobs.max_size.concern_level=1
3: ac0adb7c9f ! 3: cc78eaeca0 repo: add documentation for level-of-concern.
@@ Commit message
## Documentation/config/repo.adoc (new) ##
@@
-+`repo.structure.concern.commit-count`::
++`repo.concern.commit-count`::
+ Sets the value of the concern threshold for commit count.
+
-+`repo.structure.concern.tree-count`::
++`repo.concern.tree-count`::
+ Sets the value of the concern threshold for tree count.
+
-+`repo.structure.concern.blob-count`::
++`repo.concern.blob-count`::
+ Sets the value of the concern threshold for blob count.
+
-+`repo.structure.concern.tag-count`::
++`repo.concern.tag-count`::
+ Sets the value of the concern threshold for tag count.
+
-+`repo.structure.concern.commit-size`::
++`repo.concern.commit-size`::
+ Sets the value of the concern threshold for total commit size.
+
-+`repo.structure.concern.tree-size`::
++`repo.concern.tree-size`::
+ Sets the value of the concern threshold for total tree size.
+
-+`repo.structure.concern.blob-size`::
++`repo.concern.blob-size`::
+ Sets the value of the concern threshold for total blob size.
+
-+`repo.structure.concern.tag-size`::
++`repo.concern.tag-size`::
+ Sets the value of the concern threshold for total tag size.
+
-+`repo.structure.concern.commit-inflated-size`::
++`repo.concern.commit-inflated-size`::
+ Sets the value of the concern threshold for total size
+ of all inflated commits.
+
-+`repo.structure.concern.tree-inflated-size`::
++`repo.concern.tree-inflated-size`::
+ Sets the value of the concern threshold for total size of
+ all inflated trees.
+
-+`repo.structure.concern.blob-inflated-size`::
++`repo.concern.blob-inflated-size`::
+ Sets the value of the concern threshold for total size of
+ all inflated blobs.
+
-+`repo.structure.concern.tag-inflated-size`::
++`repo.concern.tag-inflated-size`::
+ Sets the value of the concern threshold for the total size
+ of all inflated tags.
+
-+`repo.structure.concern.commit-largest`::
++`repo.concern.commit-largest`::
+ Sets the value of the concern threshold the size of the largest
+ commit.
+
-+`repo.structure.concern.tree-largest`::
++`repo.concern.tree-largest`::
+ Sets the value of the concern threshold for the size of the
+ largest tree.
+
-+`repo.structure.concern.blob-largest`::
++`repo.concern.blob-largest`::
+ Sets the value of the concern threshold for the size of the
+ largest blob.
+
-+`repo.structure.concern.tag-largest`::
++`repo.concern.tag-largest`::
+ Sets the value of the concern threshold for the size of the
+ largest tag.
-++
-+All of the concern thresholds are integer values for the
++
++All of the concern thresholds are integer values for the
+floor of the base-2 log of the threshold value. The default values
+of each threshold are set to heuristically determined values.
-+
## Documentation/git-repo.adoc ##
@@ Documentation/git-repo.adoc: SYNOPSIS
@@ Documentation/git-repo.adoc: supported:
+users should be concerned about, adding '--concern' will provide a
+level of concern to be added to the output. In 'table' mode, this will
+be a table column indicating the level of concern as a number of stars;
-+in 'lines` or `nul' mode, a `concern_level` line will be added for each
++in `lines` or `nul` mode, a `concern_level` line will be added for each
+statistic with a numeric value.
++
-+The level of corcern for each statistic is computed using the base_2 log of the
++The level of concern for each statistic is computed using the log_2 of the
+statistic value. The value is considered concerning if it exceeds a heuristically
+chosen threshold value; the degree of concern is the difference between the
-+base_2 log of the value, and the base_2 log of the corresponding threshold.
++log_2 of the value, and the log_2 of the corresponding threshold.
++
+The threshold values are configurable; see the configuration section below.
--- base-commit: 3bc0341126508f78f5869cbfc0005e987efdf0c7 change-id: 20261019-b4-repo-structure-concern-f1d6f6326095