{"thread":{"id":"58737","subject":"[RFC/PoC 2/5] Documentation Bundles: ignore artifact files from creating doc bundles","startedAt":"2022-11-02T22:48:57Z","lastAt":"2022-11-03T20:47:01Z","messageCount":15,"participants":["Philip Oakley","Ævar Arnfjörð Bjarmason"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"466349","messageId":"20221102224843.2104-3-philipoakley@iee.email","threadId":"58737","inReplyTo":"20221102224843.2104-1-philipoakley@iee.email","subject":"[RFC/PoC 2/5] Documentation Bundles: ignore artifact files from creating doc bundles","fromName":"Philip Oakley","fromEmail":"philipoakley@iee.email","sentAt":"2022-11-02T22:48:40Z","receivedAt":"2022-11-02T22:48:57Z","isPatch":false,"sender":{"key":"philipoakley@iee.email","avatar":"https://avatars.githubusercontent.com/u/914343?v=4"},"body":"The Git documentation contains many examples that can be prepared as\nrepository bundles for users to clone and explore without any language\nor translation confusions between the written word and intended code.\n\nThere are also a large number of example repositories in the t/ test\nsystem can can also be exposed to users to highlight the critical\ndistinctions tested therein.\n\nThese documentation bundles will be prepared ar part of the test system.\nStart by ignoring those parts which will be considered build artifacts,\nthe GIT-BUNDLE-DIR, and the `.bndl` bundle files themselves.\n\nThe artifacts themselves will later be manipulated by the `make` elements\nof the documentation system.\n\nSigned-off-by: Philip Oakley <philipoakley@iee.email>\n---\n .gitignore | 2 ++\n 1 file changed, 2 insertions(+)\n\ndiff --git a/.gitignore b/.gitignore\nindex 6ded10067a..b96a4ce5df 100644\n--- a/.gitignore\n+++ b/.gitignore\n@@ -3,6 +3,7 @@\n /fuzz-pack-headers\n /fuzz-pack-idx\n /GIT-BUILD-OPTIONS\n+/GIT-BUNDLE-DIR\n /GIT-CFLAGS\n /GIT-LDFLAGS\n /GIT-PREFIX\n@@ -250,3 +251,4 @@ Release/\n *.dSYM\n /contrib/buildsystems/out\n CMakeSettings.json\n+*.bndl\n-- \n2.38.1.281.g83ef3ded8d\n\n"},{"id":"466350","messageId":"20221102224843.2104-1-philipoakley@iee.email","threadId":"58737","inReplyTo":null,"subject":"[RFC/PoC 0/5] Provide example docbundles","fromName":"Philip Oakley","fromEmail":"philipoakley@iee.email","sentAt":"2022-11-02T22:48:38Z","receivedAt":"2022-11-02T22:48:58Z","isPatch":false,"sender":{"key":"philipoakley@iee.email","avatar":"https://avatars.githubusercontent.com/u/914343?v=4"},"body":"The man pages provide user facing examples which explain important\nconcepts, such as history-simplification. However users can have\ndifficulty in reproducing these examples to allow the exploration\nof thr concepts. Let's provide these simple example, pre-packaged\nas part of the documentation.\n\nThis Request for Comment (RFC) / Proof of Concept (PoC) attempts\nto set out some design aims, and an outline implementation based\non using bundles for each example, and the test system for their\npreparation.\n\nAre there any obvious show-stoppers? \nWhere best to hold the docbundle dir on various platforms?\nHow many examples to provide, and how to decide?\nUsage of existing test repo examples?\netc.\n--\nPhilip\n\nPhilip Oakley (5):\n  doc: provide DocBundles design document\n  Documentation Bundles: ignore artifact files from creating doc bundles\n  test-lib: add GIT_BUNDLE_DIR setup\n  Doc Bundles: add t6102 rev-list simple and pulls examples\n  docbundle: provide the git-log History Simplication example\n\n .gitignore                            |   2 +\n Documentation/technical/docbundle.txt |  97 ++++++++++\n t/t6012-rev-list-simplify.sh          |   8 +\n t/t9990-history-simplification.sh     | 261 ++++++++++++++++++++++++++\n t/test-lib.sh                         |  14 ++\n 5 files changed, 382 insertions(+)\n create mode 100644 Documentation/technical/docbundle.txt\n create mode 100644 t/t9990-history-simplification.sh\n\n-- \n2.38.1.281.g83ef3ded8d\n\n"},{"id":"466351","messageId":"20221102224843.2104-2-philipoakley@iee.email","threadId":"58737","inReplyTo":"20221102224843.2104-1-philipoakley@iee.email","subject":"[RFC/PoC 1/5] doc: provide DocBundles design document","fromName":"Philip Oakley","fromEmail":"philipoakley@iee.email","sentAt":"2022-11-02T22:48:39Z","receivedAt":"2022-11-02T22:49:00Z","isPatch":false,"sender":{"key":"philipoakley@iee.email","avatar":"https://avatars.githubusercontent.com/u/914343?v=4"},"body":"DocBundles are exemplar repositories, provided as bundles, which\ncontain either 'interesting' test setups for exploration, or\nthe described examples from within the man pages.\n\nSigned-off-by: Philip Oakley <philipoakley@iee.email>\n---\n Documentation/technical/docbundle.txt | 97 +++++++++++++++++++++++++++\n 1 file changed, 97 insertions(+)\n create mode 100644 Documentation/technical/docbundle.txt\n\ndiff --git a/Documentation/technical/docbundle.txt b/Documentation/technical/docbundle.txt\nnew file mode 100644\nindex 0000000000..8c17a7847f\n--- /dev/null\n+++ b/Documentation/technical/docbundle.txt\n@@ -0,0 +1,97 @@\n+docbundle Design Document\n+=========================\n+\n+The Git man pages contain many example setups to demonstrate aspects\n+of the related Git commands, such as history simplification in the `log`\n+and `show commands. However, while illustrative, these are not\n+accessible, as example repositories, to regular, potentially confused,\n+users who may need to understand the set up of the examples.\n+\n+The docbundle concept is that for each of these examples there is a\n+matching `bundle` file, provided with the Documentation, which can be\n+simply cloned to provide an example repository containing the example.\n+\n+There are also many example repositories in the Git `/t` test system\n+that could be exposed to users which already demonstrate some important\n+testable aspect of Git within a carefully constructed repository.\n+\n+\n+This proposal is to provide DocBundles to users via the Git project.\n+\n+\n+Firstly, it should be reasonably self-evident that the man page examples\n+should, in themselves, be tested for accuracy, especially if provided\n+to users.\n+\n+Secondly, the existing test system is the proper place to setup and\n+test any such docbundle repositories.\n+\n+Third, that the bundle is an obvious pre-existing mechanism for\n+transporting repositories (see also the Bundle-URI proposals).\n+\n+Fourthly, the preparation and distribution of the docbundle should\n+require minimal maintainer effort, preferably being integrated into\n+the existing automation for Documentation preparation and distribution.\n+\n+Fifthly, the docbudles themselves should not to be part of the git.git\n+repo. They are prepared via the test suite and should be stored (locally)\n+in a designated Git_DocBundle_Dir, and be .gitignored. The DocBundles\n+are transferred to the maintainer's Doc repo via the automation.\n+\n+Sixthly, the docbundles to be named based on their `tnnnn` test number\n+and a (short) descriptive name.\n+\n+Seven, (dev) start with an existing test, rather than add a test (most\n+doc/man examples aren't tested!). Add tests for missing examples later.\n+\n+Eight, (option) add a `git docbundle` command to assist in locating and\n+listing the docbundle examples. It also provides place to explain their\n+purpose as \"Tutorial Examples\" (Note, many existing examples are to\n+illustrate complex/difficult scenarios where the wording was insufficient;\n+the docbudles are minimal examples).\n+\n+\n+Status\n+======\n+\n+This is an RFC/PoC (Request for Comment / Proof of Concept).\n+\n+The attached commits establish:\n+\n+Set out this design in the Documentation/technical directory.\n+\n+A default location for the docbundles and an env variable to allow\n+flexibility by extending test-lib.\n+\n+The docbundles are ignored, as is the default directory.\n+\n+Two docbundles are generated from a 'random' test that I happened to\n+have open in my editor (t6102 rev-list-simplify).\n+\n+The docbundle creation is protected by `test_expect_success` wrappers.\n+\n+\n+\n+ToDo\n+====\n+\n+DocBundles - Is this the best name, or just a convenient way of\n+indicating the delivery mechanism?\n+\n+Add Prerequisite to control if docbundles are generated (don't waste\n+maintainers PC time when not needed).\n+\n+CI integration (i.e. whether to test the bundling aspect - should be\n+reliable).\n+\n+How to update the man pages, perhaps by inclusion of the test comment\n+(exported?), and distinguish between man page descriptions (being\n+packaged by the test suite) and tests which offer users insight into\n+Git operations, usually with 'interesting' commit-graph set-ups.\n+\n+Claim t999x as the man doc bundle test sequence ('999' is the UK\n+emergency (\"Help!\") number; 911, 101 and 102 are taken test areas.\n+\n+Other stuff implicit in the eight point list\n+\n+\n-- \n2.38.1.281.g83ef3ded8d\n\n"},{"id":"466352","messageId":"20221102224843.2104-4-philipoakley@iee.email","threadId":"58737","inReplyTo":"20221102224843.2104-1-philipoakley@iee.email","subject":"[RFC/PoC 3/5] test-lib: add GIT_BUNDLE_DIR setup","fromName":"Philip Oakley","fromEmail":"philipoakley@iee.email","sentAt":"2022-11-02T22:48:41Z","receivedAt":"2022-11-02T22:49:02Z","isPatch":false,"sender":{"key":"philipoakley@iee.email","avatar":"https://avatars.githubusercontent.com/u/914343?v=4"},"body":"The test system may output a number of documentation bundles containing\ntest repositories of interest to users, including those explicitly\ndocumented.\n\nProvide a default location for holding those bundles.\nAs build artefact's this location, along with `.bndl` files, was\nignored in an earlier commit.\n\nThis may need further refinement.\n\nSigned-off-by: Philip Oakley <philipoakley@iee.email>\n---\n t/test-lib.sh | 14 ++++++++++++++\n 1 file changed, 14 insertions(+)\n\ndiff --git a/t/test-lib.sh b/t/test-lib.sh\nindex 4e45bc7c4f..b18cd8a34e 100644\n--- a/t/test-lib.sh\n+++ b/t/test-lib.sh\n@@ -56,6 +56,20 @@ then\n \techo \"PANIC: Running in a $TEST_DIRECTORY that doesn't end in '/t'?\" >&2\n \texit 1\n fi\n+if test -z \"$GIT_BUNDLE_DIR\"\n+then\n+\t# Similarly, override this to store the documentation bundles subdir\n+\t# elsewhere\n+\tGIT_BUNDLE_DIR=$TEST_DIRECTORY/GIT_BUNDLE_DIR\n+\tif test ! -d \"$GIT_BUNDLE_DIR\"\n+\tthen\n+\t\tmkdir -p \"$GIT_BUNDLE_DIR\"\n+\tfi\n+elif test ! -d \"$GIT_BUNDLE_DIR\"\n+then\n+\techo \"PANIC: Your \"$GIT_BUNDLE_DIR\" doesn't exist!\" >&2\n+\texit 1\n+fi\n \n # Prepend a string to a VAR using an arbitrary \":\" delimiter, not\n # adding the delimiter if VAR or VALUE is empty. I.e. a generalized:\n-- \n2.38.1.281.g83ef3ded8d\n\n"},{"id":"466353","messageId":"20221102224843.2104-5-philipoakley@iee.email","threadId":"58737","inReplyTo":"20221102224843.2104-1-philipoakley@iee.email","subject":"[RFC/PoC 4/5] Doc Bundles: add t6102 rev-list simple and pulls examples","fromName":"Philip Oakley","fromEmail":"philipoakley@iee.email","sentAt":"2022-11-02T22:48:42Z","receivedAt":"2022-11-02T22:49:04Z","isPatch":false,"sender":{"key":"philipoakley@iee.email","avatar":"https://avatars.githubusercontent.com/u/914343?v=4"},"body":"The choice of the t6012-rev-list-simplify examples as the initial\nexamples is somewhat arbitrary, and based on having the example\navailable in my editor at the time.\n\nThe environment variable GIT_BUILD_DIR is expected to contain\nthe absolute path to that directory. Later commits will set that up\nin the test-lib-setup.\n\nSigned-off-by: Philip Oakley <philipoakley@iee.email>\n---\n t/t6012-rev-list-simplify.sh | 8 ++++++++\n 1 file changed, 8 insertions(+)\n\ndiff --git a/t/t6012-rev-list-simplify.sh b/t/t6012-rev-list-simplify.sh\nindex de1e87f162..8931102392 100755\n--- a/t/t6012-rev-list-simplify.sh\n+++ b/t/t6012-rev-list-simplify.sh\n@@ -101,6 +101,10 @@ test_expect_success setup '\n \tnote L\n '\n \n+test_expect_success 'bundle simple repo' '\n+\tgit bundle create ${GIT_BUNDLE_DIR}/t6102-rev-list-simple.bndl --all\n+'\n+\n FMT='tformat:%P \t%H | %s'\n \n check_outcome () {\n@@ -240,6 +244,10 @@ test_expect_success 'setup rebuild repo' '\n \tnote P\n '\n \n+test_expect_success 'bundle pulls repo' '\n+\tgit bundle create ${GIT_BUNDLE_DIR}/t6102-rev-list-pulls.bndl --all\n+'\n+\n check_result 'X I' -- file\n check_result 'N R X I' --show-pulls -- file\n \n-- \n2.38.1.281.g83ef3ded8d\n\n"},{"id":"466354","messageId":"20221102224843.2104-6-philipoakley@iee.email","threadId":"58737","inReplyTo":"20221102224843.2104-1-philipoakley@iee.email","subject":"[RFC/PoC 5/5] docbundle: provide the git-log History Simplification example","fromName":"Philip Oakley","fromEmail":"philipoakley@iee.email","sentAt":"2022-11-02T22:48:43Z","receivedAt":"2022-11-02T22:49:07Z","isPatch":false,"sender":{"key":"philipoakley@iee.email","avatar":"https://avatars.githubusercontent.com/u/914343?v=4"},"body":"Create the History Simplification example and bundle it.\n\nToDo: add the tests that the man page attempts to demonstrate.\n\nadd the effects of `--graph` to the tests and man page.\n\n/* NEEDSWORK */\n\nSigned-off-by: Philip Oakley <philipoakley@iee.email>\n---\n t/t9990-history-simplification.sh | 261 ++++++++++++++++++++++++++++++\n 1 file changed, 261 insertions(+)\n create mode 100644 t/t9990-history-simplification.sh\n\ndiff --git a/t/t9990-history-simplification.sh b/t/t9990-history-simplification.sh\nnew file mode 100644\nindex 0000000000..ada06698d5\n--- /dev/null\n+++ b/t/t9990-history-simplification.sh\n@@ -0,0 +1,261 @@\n+#!/bin/sh\n+\n+test_description='History simplification man page'\n+\n+GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main\n+export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\n+\n+. ./test-lib.sh\n+\n+note () {\n+\tgit tag \"$1\"\n+}\n+\n+unnote () {\n+\ttest_when_finished \"rm -f tmp\" &&\n+\tgit name-rev --tags --annotate-stdin >tmp &&\n+\tsed -e \"s|$OID_REGEX (tags/\\([^)]*\\)) |\\1 |g\" <tmp\n+}\n+\n+#\n+# Create a test repo with the man(3) History Simplification commit graph:\n+#\n+# We assume, for the TREESAME check,\n+# that you are filtering for a file `foo` in this commit graph:\n+#-----------------------------------------------------------------------\n+#\t  .-A---M---N---O---P---Q\n+#\t /     /   /   /   /   /\n+#\tI     B   C   D   E   Y\n+#\t \\   /   /   /   /   /\n+#\t  `-------------'   X\n+#-----------------------------------------------------------------------\n+#The horizontal line of history A---Q is taken to be the first parent of\n+#each merge.\n+#\n+# See Documentation/rev-list-options.txt\n+# see https://groups.google.com/d/msgid/git-users/11286f4a-c961-4d12-91a0-f49e34d8691bn%40googlegroups.com. for git-user discussion.\n+# https://github.com/ChuckTest/git-history-test\n+# C:\\git-sdk-64\\usr\\src\\git-history-test\n+\n+test_expect_success setup '\n+\t# get the structure set out, then the file contents\n+\techo \"asdf\" >foo &&\n+\techo \"quux\" >quux &&\n+\tgit add foo quux &&\n+\ttest_tick && git commit -m \"I: Initial commit\" &&\n+\tnote I &&\n+\n+\tgit branch B &&\n+\tgit branch C &&\n+\tgit branch D &&\n+\tgit branch E &&\n+\n+\techo \"foo\" >foo &&\n+\tgit add foo &&\n+\ttest_tick && git commit -m \"A\" &&\n+\tnote A &&\n+\n+\tgit checkout B &&\n+\techo \"foo\" >foo &&\n+\tgit add foo &&\n+\ttest_tick && git commit -m \"B\" &&\n+\tnote B &&\n+\n+\tgit checkout main &&\n+\ttest_tick && git merge B -m \"M: trivial merge\" &&\n+\tnote M &&\n+\n+\tgit checkout C &&\n+\t# is this needed?\n+\t#echo \"other\" >other &&\n+\t#git add other &&\n+\ttest_tick && git commit --allow-empty -m \"C: identical to I\" &&\n+\tnote C &&\n+\n+\tgit checkout main &&\n+\ttest_tick && git merge C --no-commit -m \"N: !TREESAME any parent\" &&\n+\techo \"foobar\" >foo &&\n+\tgit add foo &&\n+\tgit commit -m \"N: !TREESAME any parent\" &&\n+\tnote N &&\n+\n+\tgit checkout D &&\n+\techo \"baz\" >foo &&\n+\tgit add foo &&\n+\ttest_tick && git commit -m \"D\" &&\n+\tnote D &&\n+\n+\tgit checkout main &&\n+\ttest_tick &&\n+\ttest_must_fail git merge D --no-commit &&\n+\techo \"foobarbaz\" >foo &&\n+\tgit add foo &&\n+\tgit commit -m \"O: !TREESAME any parent\" &&\n+\tnote O &&\n+\n+\tgit checkout E &&\n+\techo \"xyzzy\" >quux &&\n+\tgit add quux &&\n+\ttest_tick && git commit -m \"E\" &&\n+\tnote E &&\n+\n+\tgit checkout main &&\n+\ttest_tick &&\n+\tgit merge E --no-commit &&\n+\techo \"quux xyzzy\" >quux &&\n+\tgit add quux &&\n+\tgit commit -m \"P: TREESAME O; !E\" &&\n+\tnote S &&\n+\n+\tgit checkout --orphan Side &&\n+\tgit rm -f foo quux &&\n+\techo \"initial\" >side &&\n+\tgit add side &&\n+\tgit commit -m \"X\" &&\n+\tnote X &&\n+\n+\techo \"side\" >side &&\n+\tgit add side &&\n+\tgit commit -m \"Y\" &&\n+\tnote Y &&\n+\n+\tgit checkout main &&\n+\ttest_tick && \n+\tgit merge Side --allow-unrelated-histories -m \"Q: TREESAME P; !Y\" &&\n+\tnote Q\n+'\n+\n+test_expect_success 'bundle history-simplification repo' '\n+\tgit bundle create ${GIT_BUNDLE_DIR}/t9990-history-simplification.bndl --all\n+'\n+\n+# Test it!\n+# `rev-list` walks backwards through history, including or excluding\n+# commits based on whether `--full-history` and/or parent rewriting\n+# (via `--parents` or `--children`) are used. The following settings\n+# are available.\n+# \n+# Default mode::\n+# \tCommits are included if they are not TREESAME to any parent\n+# \t(though this can be changed, see `--sparse` below).  If the\n+# \tcommit was a merge, and it was TREESAME to one parent, follow\n+# \tonly that parent.  (Even if there are several TREESAME\n+# \tparents, follow only one of them.)  Otherwise, follow all\n+# \tparents.\n+# +\n+# This results in:\n+# +\n+# -----------------------------------------------------------------------\n+# \t  .-A---N---O\n+# \t /     /   /\n+# \tI---------D\n+# -----------------------------------------------------------------------\n+# +\n+# Note how the rule to only follow the TREESAME parent, if one is\n+# available, removed `B` from consideration entirely.  `C` was\n+# considered via `N`, but is TREESAME.  Root commits are compared to an\n+# empty tree, so `I` is !TREESAME.\n+# +\n+# Parent/child relations are only visible with `--parents`, but that does\n+# not affect the commits selected in default mode, so we have shown the\n+# parent lines.\n+# \n+# --full-history without parent rewriting::\n+# \tThis mode differs from the default in one point: always follow\n+# \tall parents of a merge, even if it is TREESAME to one of them.\n+# \tEven if more than one side of the merge has commits that are\n+# \tincluded, this does not imply that the merge itself is!  In\n+# \tthe example, we get\n+# +\n+# -----------------------------------------------------------------------\n+# \tI  A  B  N  D  O  P  Q\n+# -----------------------------------------------------------------------\n+# +\n+# `M` was excluded because it is TREESAME to both parents.  `E`,\n+# `C` and `B` were all walked, but only `B` was !TREESAME, so the others\n+# do not appear.\n+# +\n+# Note that without parent rewriting, it is not really possible to talk\n+# about the parent/child relationships between the commits, so we show\n+# them disconnected.\n+# \n+# --full-history with parent rewriting::\n+# \tOrdinary commits are only included if they are !TREESAME\n+# \t(though this can be changed, see `--sparse` below).\n+# +\n+# Merges are always included.  However, their parent list is rewritten:\n+# Along each parent, prune away commits that are not included\n+# themselves.  This results in\n+# +\n+# -----------------------------------------------------------------------\n+# \t  .-A---M---N---O---P---Q\n+# \t /     /   /   /   /\n+# \tI     B   /   D   /\n+# \t \\   /   /   /   /\n+# \t  `-------------'\n+# -----------------------------------------------------------------------\n+# +\n+# Compare to `--full-history` without rewriting above.  Note that `E`\n+# was pruned away because it is TREESAME, but the parent list of P was\n+# rewritten to contain `E`'s parent `I`.  The same happened for `C` and\n+# `N`, and `X`, `Y` and `Q`.\n+# \n+# In addition to the above settings, you can change whether TREESAME\n+# affects inclusion:\n+# \n+# --dense::\n+# \tCommits that are walked are included if they are not TREESAME\n+# \tto any parent.\n+# \n+# --sparse::\n+# \tAll commits that are walked are included.\n+# +\n+# Note that without `--full-history`, this still simplifies merges: if\n+# one of the parents is TREESAME, we follow only that one, so the other\n+# sides of the merge are never walked.\n+# \n+# --simplify-merges::\n+# \tFirst, build a history graph in the same way that\n+# \t`--full-history` with parent rewriting does (see above).\n+# +\n+# Then simplify each commit `C` to its replacement `C'` in the final\n+# history according to the following rules:\n+# +\n+# --\n+# * Set `C'` to `C`.\n+# +\n+# * Replace each parent `P` of `C'` with its simplification `P'`.  In\n+#   the process, drop parents that are ancestors of other parents or that are\n+#   root commits TREESAME to an empty tree, and remove duplicates, but take care\n+#   to never drop all parents that we are TREESAME to.\n+# +\n+# * If after this parent rewriting, `C'` is a root or merge commit (has\n+#   zero or >1 parents), a boundary commit, or !TREESAME, it remains.\n+#   Otherwise, it is replaced with its only parent.\n+# --\n+# +\n+# The effect of this is best shown by way of comparing to\n+# `--full-history` with parent rewriting.  The example turns into:\n+# +\n+# -----------------------------------------------------------------------\n+# \t  .-A---M---N---O\n+# \t /     /       /\n+# \tI     B       D\n+# \t \\   /       /\n+# \t  `---------'\n+# -----------------------------------------------------------------------\n+# +\n+# Note the major differences in `N`, `P`, and `Q` over `--full-history`:\n+# +\n+# --\n+# * `N`'s parent list had `I` removed, because it is an ancestor of the\n+#   other parent `M`.  Still, `N` remained because it is !TREESAME.\n+# +\n+# * `P`'s parent list similarly had `I` removed.  `P` was then\n+#   removed completely, because it had one parent and is TREESAME.\n+# +\n+# * `Q`'s parent list had `Y` simplified to `X`. `X` was then removed, because it\n+#   was a TREESAME root. `Q` was then removed completely, because it had one\n+#   parent and is TREESAME.\n+\n+test_done\n-- \n2.38.1.281.g83ef3ded8d\n\n"},{"id":"466382","messageId":"221103.86zgd81jtm.gmgdl@evledraar.gmail.com","threadId":"58737","inReplyTo":"20221102224843.2104-3-philipoakley@iee.email","subject":"Re: [RFC/PoC 2/5] Documentation Bundles: ignore artifact files from creating doc bundles","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-11-03T09:12:17Z","receivedAt":"2022-11-03T09:14:59Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Wed, Nov 02 2022, Philip Oakley wrote:\n\n> The Git documentation contains many examples that can be prepared as\n> repository bundles for users to clone and explore without any language\n> or translation confusions between the written word and intended code.\n\nI just find this commit entirely confusing, we're adding .gitignore\nentries, but nothing generates these yet (but I'll read on). All to\ndescribe some seeming future feature...\n\n> There are also a large number of example repositories in the t/ test\n> system can can also be exposed to users to highlight the critical\n> distinctions tested therein.\n>\n> These documentation bundles will be prepared ar part of the test system.\n\ns/ar/as/\n\n> Start by ignoring those parts which will be considered build artifacts,\n> the GIT-BUNDLE-DIR, and the `.bndl` bundle files themselves.\n>\n> The artifacts themselves will later be manipulated by the `make` elements\n> of the documentation system.\n>\n> Signed-off-by: Philip Oakley <philipoakley@iee.email>\n> ---\n>  .gitignore | 2 ++\n>  1 file changed, 2 insertions(+)\n>\n> diff --git a/.gitignore b/.gitignore\n> index 6ded10067a..b96a4ce5df 100644\n> --- a/.gitignore\n> +++ b/.gitignore\n> @@ -3,6 +3,7 @@\n>  /fuzz-pack-headers\n>  /fuzz-pack-idx\n>  /GIT-BUILD-OPTIONS\n> +/GIT-BUNDLE-DIR\n>  /GIT-CFLAGS\n>  /GIT-LDFLAGS\n>  /GIT-PREFIX\n> @@ -250,3 +251,4 @@ Release/\n>  *.dSYM\n>  /contrib/buildsystems/out\n>  CMakeSettings.json\n> +*.bndl\n\nWe've been putting quite a few things in .build/ recently, so whatever\nthis is, if you put it in .build/test-bundles/ or whatever you don't\nneed to write \"clean\" rules for it, or add this ignore entry.\n"},{"id":"466383","messageId":"221103.86v8nw1jpi.gmgdl@evledraar.gmail.com","threadId":"58737","inReplyTo":"20221102224843.2104-4-philipoakley@iee.email","subject":"Re: [RFC/PoC 3/5] test-lib: add GIT_BUNDLE_DIR setup","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-11-03T09:14:36Z","receivedAt":"2022-11-03T09:18:00Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Wed, Nov 02 2022, Philip Oakley wrote:\n\n> The test system may output a number of documentation bundles containing\n> test repositories of interest to users, including those explicitly\n> documented.\n>\n> Provide a default location for holding those bundles.\n> As build artefact's this location, along with `.bndl` files, was\n> ignored in an earlier commit.\n>\n> This may need further refinement.\n>\n> Signed-off-by: Philip Oakley <philipoakley@iee.email>\n> ---\n>  t/test-lib.sh | 14 ++++++++++++++\n>  1 file changed, 14 insertions(+)\n>\n> diff --git a/t/test-lib.sh b/t/test-lib.sh\n> index 4e45bc7c4f..b18cd8a34e 100644\n> --- a/t/test-lib.sh\n> +++ b/t/test-lib.sh\n> @@ -56,6 +56,20 @@ then\n>  \techo \"PANIC: Running in a $TEST_DIRECTORY that doesn't end in '/t'?\" >&2\n>  \texit 1\n>  fi\n> +if test -z \"$GIT_BUNDLE_DIR\"\n> +then\n> +\t# Similarly, override this to store the documentation bundles subdir\n> +\t# elsewhere\n> +\tGIT_BUNDLE_DIR=$TEST_DIRECTORY/GIT_BUNDLE_DIR\n> +\tif test ! -d \"$GIT_BUNDLE_DIR\"\n> +\tthen\n> +\t\tmkdir -p \"$GIT_BUNDLE_DIR\"\n\nOkey, so \"mkdir -p\", not \"mkdir\" once one thinks about it for a second:\nBecause we might be testing in parallel, so this will be racy.\n\n> +\tfi\n> +elif test ! -d \"$GIT_BUNDLE_DIR\"\n> +then\n> +\techo \"PANIC: Your \"$GIT_BUNDLE_DIR\" doesn't exist!\" >&2\n> +\texit 1\n> +fi\n\nI think it's better just to put these in the \"test-results\" dir.\n\nFor leak testing I have local patches to e.g. create test-results/<test\nnumber>.leak, and then after a run I slurp that up and process it.\n\nSo, same idea, you could just add stuff there, and afterwards grab it\nthere. No need for specifying the dir etc., you just need a test-lib.sh\nflag to say \"yes, write these out\".\n"},{"id":"466384","messageId":"221103.86r0yk1jld.gmgdl@evledraar.gmail.com","threadId":"58737","inReplyTo":"20221102224843.2104-6-philipoakley@iee.email","subject":"Re: [RFC/PoC 5/5] docbundle: provide the git-log History Simplification example","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-11-03T09:17:58Z","receivedAt":"2022-11-03T09:20:19Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Wed, Nov 02 2022, Philip Oakley wrote:\n\n> +note () {\n> +\tgit tag \"$1\"\n> +}\n\nAren't you just...\n\n> +test_expect_success setup '\n> +\t# get the structure set out, then the file contents\n> +\techo \"asdf\" >foo &&\n> +\techo \"quux\" >quux &&\n> +\tgit add foo quux &&\n> +\ttest_tick && git commit -m \"I: Initial commit\" &&\n> +\tnote I &&\n\n...re-inventing \"test_commit\" here (including the \"echo\")?\n"},{"id":"466392","messageId":"221103.86mt981i9h.gmgdl@evledraar.gmail.com","threadId":"58737","inReplyTo":"20221102224843.2104-2-philipoakley@iee.email","subject":"Re: [RFC/PoC 1/5] doc: provide DocBundles design document","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-11-03T09:19:47Z","receivedAt":"2022-11-03T09:48:04Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Wed, Nov 02 2022, Philip Oakley wrote:\n\n> [...]\n> diff --git a/Documentation/technical/docbundle.txt b/Documentation/technical/docbundle.txt\n> new file mode 100644\n> index 0000000000..8c17a7847f\n> --- /dev/null\n> +++ b/Documentation/technical/docbundle.txt\n> @@ -0,0 +1,97 @@\n> +docbundle Design Document\n> +=========================\n> +\n> +The Git man pages contain many example setups to demonstrate aspects\n> +of the related Git commands, such as history simplification in the `log`\n> +and `show commands. However, while illustrative, these are not\n> +accessible, as example repositories, to regular, potentially confused,\n> +users who may need to understand the set up of the examples.\n> +\n> +The docbundle concept is that for each of these examples there is a\n> +matching `bundle` file, provided with the Documentation, which can be\n> +simply cloned to provide an example repository containing the example.\n> +\n> +There are also many example repositories in the Git `/t` test system\n> +that could be exposed to users which already demonstrate some important\n> +testable aspect of Git within a carefully constructed repository.\n> +\n> +\n> +This proposal is to provide DocBundles to users via the Git project.\n> +\n> +\n> +Firstly, it should be reasonably self-evident that the man page examples\n> +should, in themselves, be tested for accuracy, especially if provided\n> +to users.\n> +\n> +Secondly, the existing test system is the proper place to setup and\n> +test any such docbundle repositories.\n> +\n> +Third, that the bundle is an obvious pre-existing mechanism for\n> +transporting repositories (see also the Bundle-URI proposals).\n> +\n> +Fourthly, the preparation and distribution of the docbundle should\n> +require minimal maintainer effort, preferably being integrated into\n> +the existing automation for Documentation preparation and distribution.\n> +\n> +Fifthly, the docbudles themselves should not to be part of the git.git\n> +repo. They are prepared via the test suite and should be stored (locally)\n> +in a designated Git_DocBundle_Dir, and be .gitignored. The DocBundles\n> +are transferred to the maintainer's Doc repo via the automation.\n> +\n> +Sixthly, the docbundles to be named based on their `tnnnn` test number\n> +and a (short) descriptive name.\n> +\n> +Seven, (dev) start with an existing test, rather than add a test (most\n> +doc/man examples aren't tested!). Add tests for missing examples later.\n> +\n> +Eight, (option) add a `git docbundle` command to assist in locating and\n> +listing the docbundle examples. It also provides place to explain their\n> +purpose as \"Tutorial Examples\" (Note, many existing examples are to\n> +illustrate complex/difficult scenarios where the wording was insufficient;\n> +the docbudles are minimal examples).\n> +\n> +\n> +Status\n> +======\n> +\n> +This is an RFC/PoC (Request for Comment / Proof of Concept).\n> +\n> +The attached commits establish:\n> +\n> +Set out this design in the Documentation/technical directory.\n> +\n> +A default location for the docbundles and an env variable to allow\n> +flexibility by extending test-lib.\n> +\n> +The docbundles are ignored, as is the default directory.\n> +\n> +Two docbundles are generated from a 'random' test that I happened to\n> +have open in my editor (t6102 rev-list-simplify).\n> +\n> +The docbundle creation is protected by `test_expect_success` wrappers.\n> +\n> +ToDo\n> +====\n> +\n> +DocBundles - Is this the best name, or just a convenient way of\n> +indicating the delivery mechanism?\n> +\n> +Add Prerequisite to control if docbundles are generated (don't waste\n> +maintainers PC time when not needed).\n> +\n> +CI integration (i.e. whether to test the bundling aspect - should be\n> +reliable).\n> +\n> +How to update the man pages, perhaps by inclusion of the test comment\n> +(exported?), and distinguish between man page descriptions (being\n> +packaged by the test suite) and tests which offer users insight into\n> +Git operations, usually with 'interesting' commit-graph set-ups.\n> +\n> +Claim t999x as the man doc bundle test sequence ('999' is the UK\n> +emergency (\"Help!\") number; 911, 101 and 102 are taken test areas.\n> +\n> +Other stuff implicit in the eight point list\n\nI've skimmed this, and provided some rough comments inline.\n\nI think the core idea sounds nice, if I'm getting it right. Let me see\nif I can rephrase it, and you can point out where I'm wrong/didn't get\nit:\n\nWe have stuff in our docs like e.g. this, at the top of git-rebase(1);\n\n       Assume the following history exists and the current branch is \"topic\":\n\n                     A---B---C topic\n                    /\n               D---E---F---G master\n\nIt would be nice if for that and other things we could add a blurb like:\n\n\tTo play around with this history, do:\n\n\t\tgit clone git-doc://git-rebase/description-1.bndl && cd description-1\n\nOr whatever, this POC doesn't fleshen that out, but the idea is that\nthrough <some mechanism> we would bridge the gap such that the user\ncould be looking at and playing with the history we're describing in our\ndocs.\n\nThe above assumes we'd ship those with installed git, but these could\nalso be hosted online, e.g.:\n\n\tgit clone https://git-scm.com/test-bundles/git-rebase/description-1.bdl\n\nSo as an end-goal you could both run these locally, and we could imagine\nthat e.g. git-scm.com could eventually have UX similar to Redis's\nwebsite: https://redis.io/commands/hgetall/\n\nI.e. where it drops you into a shell to play with the command.\n\nIn reply to 3/5 I said I had some \"local patches\", but I forgot the\nrelevant part was on master already (the whole subsequent leak analysis\nis local): 366bd129dc3 (test-lib: add a SANITIZE=leak logging mode,\n2022-07-28)\n\nSo I think you're making the whole \"have the test suite make these\" part\nof this too complex, i.e. with a minor variation of that you can do this\nin test-lib.sh:\n\n\tTEST_RESULTS_BDL_DIR=\"$TEST_RESULTS_DIR/$TEST_NAME.test-bdl\"\n\nThen have a helper like:\n\n\tdo_my_test_bundle_stuff () {\n\t\tlocal name=\"$1\" &&\n\t\tshift &&\n\t\tgit bundle create \"$TEST_RESULTS_BDL_DIR/$name.bdl\" \"$@\"\n\t}\n\nThen all you need to make these is:\n\n\tmake test GIT_TEST_OPTS=--debug\n\nAnd then:\n\n\tls -l t/test-results/*.bdl/*\n\nWill give you all the bundles you want.\n\nWe're then missing the \"connect it with the docs\" part, but that part\nwould be e.g.:\n\n- Have a validation target for it that depends on such a \"make test\" run\n- Sanity check that we did create the ones the docs expect\n\nAnyway.\n\nI like the end-goals, but I can't help think that bundles are the wrong\ndirection to go in, as opposed to:\n\n1. We could have \"make install\" ship test-lib{,-functions}.sh along\n   with git itself\n2. Have e.g. t/doc-setup/*.sh with the snippets to set up these test\n   demos, which we'd also ship.\n\nAnd you'd then do e.g.:\n\n\tgit test-case rebase/description-1\n\nWhich would just be a thin wrapper around (pseudocode):\n\n\tdir=$(git test--helper get-tempdir) &&\n\t(\n\t\tcd \"$dir\" &&\n                # And whatever else we need to set up to say \"use this local installed git\"\n\t\tGIT_TEST_INSTALLED=[...] &&\n\t\t. \"$(git --test-dir)\"/test-lib.sh\n\t\t. \"$(git --test-dir)\"/doc-setup/rebase-description-1.sh\"\n\n\t\t# This would run test_commit, and any other arbitrary setup code\n\t\tsetup_rebase_description_1()\n\t) &&\n\techo \"Go play with '$dir', and have fun\"\n\nThe big advantage of that is:\n\n 1. It ships with git, so it works offline, and we won't get into the\n    inevitable confusion of the user reading v2.40 docs locally, but the\n    online url tracking a v2.50 example or whatever.\n 2. We can represent whatever arbitrary repo state, e.g. drop the user\n    into a merge conflict that's already happening.\n 3. We can also represent \"repo-extra\" state, e.g. imagine the\n    \"git-gc/git-repack\" docs discussing loose objects, and wanting to\n    get you into some arbitrary state with regards to your loose objects\n    etc.\n"},{"id":"466449","messageId":"ab275fa0-2977-f880-cc33-836f667287bc@iee.email","threadId":"58737","inReplyTo":"221103.86mt981i9h.gmgdl@evledraar.gmail.com","subject":"Re: [RFC/PoC 1/5] doc: provide DocBundles design document","fromName":"Philip Oakley","fromEmail":"philipoakley@iee.email","sentAt":"2022-11-03T19:50:02Z","receivedAt":"2022-11-03T19:50:13Z","isPatch":false,"sender":{"key":"philipoakley@iee.email","avatar":"https://avatars.githubusercontent.com/u/914343?v=4"},"body":"Hi Ævar,\nThanks for the review,\n\nOn 03/11/2022 09:19, Ævar Arnfjörð Bjarmason wrote:\n> On Wed, Nov 02 2022, Philip Oakley wrote:\n>\n>> [...]\n>> diff --git a/Documentation/technical/docbundle.txt b/Documentation/technical/docbundle.txt\n>> new file mode 100644\n>> index 0000000000..8c17a7847f\n>> --- /dev/null\n>> +++ b/Documentation/technical/docbundle.txt\n>> @@ -0,0 +1,97 @@\n>> +docbundle Design Document\n>> +=========================\n>> +\n>> +The Git man pages contain many example setups to demonstrate aspects\n>> +of the related Git commands, such as history simplification in the `log`\n>> +and `show commands. However, while illustrative, these are not\n>> +accessible, as example repositories, to regular, potentially confused,\n>> +users who may need to understand the set up of the examples.\n>> +\n>> +The docbundle concept is that for each of these examples there is a\n>> +matching `bundle` file, provided with the Documentation, which can be\n>> +simply cloned to provide an example repository containing the example.\n>> +\n>> +There are also many example repositories in the Git `/t` test system\n>> +that could be exposed to users which already demonstrate some important\n>> +testable aspect of Git within a carefully constructed repository.\n>> +\n>> +\n>> +This proposal is to provide DocBundles to users via the Git project.\n>> +\n>> +\n>> +Firstly, it should be reasonably self-evident that the man page examples\n>> +should, in themselves, be tested for accuracy, especially if provided\n>> +to users.\n>> +\n>> +Secondly, the existing test system is the proper place to setup and\n>> +test any such docbundle repositories.\n>> +\n>> +Third, that the bundle is an obvious pre-existing mechanism for\n>> +transporting repositories (see also the Bundle-URI proposals).\n>> +\n>> +Fourthly, the preparation and distribution of the docbundle should\n>> +require minimal maintainer effort, preferably being integrated into\n>> +the existing automation for Documentation preparation and distribution.\n>> +\n>> +Fifthly, the docbudles themselves should not to be part of the git.git\n>> +repo. They are prepared via the test suite and should be stored (locally)\n>> +in a designated Git_DocBundle_Dir, and be .gitignored. The DocBundles\n>> +are transferred to the maintainer's Doc repo via the automation.\n>> +\n>> +Sixthly, the docbundles to be named based on their `tnnnn` test number\n>> +and a (short) descriptive name.\n>> +\n>> +Seven, (dev) start with an existing test, rather than add a test (most\n>> +doc/man examples aren't tested!). Add tests for missing examples later.\n>> +\n>> +Eight, (option) add a `git docbundle` command to assist in locating and\n>> +listing the docbundle examples. It also provides place to explain their\n>> +purpose as \"Tutorial Examples\" (Note, many existing examples are to\n>> +illustrate complex/difficult scenarios where the wording was insufficient;\n>> +the docbudles are minimal examples).\n>> +\n>> +\n>> +Status\n>> +======\n>> +\n>> +This is an RFC/PoC (Request for Comment / Proof of Concept).\n>> +\n>> +The attached commits establish:\n>> +\n>> +Set out this design in the Documentation/technical directory.\n>> +\n>> +A default location for the docbundles and an env variable to allow\n>> +flexibility by extending test-lib.\n>> +\n>> +The docbundles are ignored, as is the default directory.\n>> +\n>> +Two docbundles are generated from a 'random' test that I happened to\n>> +have open in my editor (t6102 rev-list-simplify).\n>> +\n>> +The docbundle creation is protected by `test_expect_success` wrappers.\n>> +\n>> +ToDo\n>> +====\n>> +\n>> +DocBundles - Is this the best name, or just a convenient way of\n>> +indicating the delivery mechanism?\n>> +\n>> +Add Prerequisite to control if docbundles are generated (don't waste\n>> +maintainers PC time when not needed).\n>> +\n>> +CI integration (i.e. whether to test the bundling aspect - should be\n>> +reliable).\n>> +\n>> +How to update the man pages, perhaps by inclusion of the test comment\n>> +(exported?), and distinguish between man page descriptions (being\n>> +packaged by the test suite) and tests which offer users insight into\n>> +Git operations, usually with 'interesting' commit-graph set-ups.\n>> +\n>> +Claim t999x as the man doc bundle test sequence ('999' is the UK\n>> +emergency (\"Help!\") number; 911, 101 and 102 are taken test areas.\n>> +\n>> +Other stuff implicit in the eight point list\n> I've skimmed this, and provided some rough comments inline.\n>\n> I think the core idea sounds nice, if I'm getting it right. Let me see\n> if I can rephrase it, and you can point out where I'm wrong/didn't get\n> it:\n>\n> We have stuff in our docs like e.g. this, at the top of git-rebase(1);\n>\n>        Assume the following history exists and the current branch is \"topic\":\n>\n>                      A---B---C topic\n>                     /\n>                D---E---F---G master\n>\n> It would be nice if for that and other things we could add a blurb like:\n>\n> \tTo play around with this history, do:\n>\n> \t\tgit clone git-doc://git-rebase/description-1.bndl && cd description-1\n\nThat is generically correct. The motivating example was History\nSimplification (see 5/5), where the example needed to cover a range of\nsetups which then confused new users in a catch 22 manner (link in the\n'PoC patch). I had been aware of the issue previously, when looking at\nthe support I'd seen in some other tools which have example setups.\n>\n> Or whatever, this POC doesn't fleshen that out, but the idea is that\n> through <some mechanism> we would bridge the gap such that the user\n> could be looking at and playing with the history we're describing in our\n> docs.\n\nIn terms of fleshing the PoC out, I didn't want to try stepping on too\nmany toes too soon, rather wanted to pick out a few pinch points with\nsuggestions.\n\n>\n> The above assumes we'd ship those with installed git, but these could\n> also be hosted online, e.g.:\n>\n> \tgit clone https://git-scm.com/test-bundles/git-rebase/description-1.bdl\n>\n> So as an end-goal you could both run these locally, and we could imagine\n> that e.g. git-scm.com \n\nI was assuming that the bundles would be static documentation artefacts,\navailable locally and potentially on-line. They could even be hosted on\nthe forges..\n\n> could eventually have UX similar to Redis's\n> website: https://redis.io/commands/hgetall/\n>\n> I.e. where it drops you into a shell to play with the command.\nI wasn't expecting to have a virtualised on-line git experience from\nthis, but never say never ;-)\n> In reply to 3/5 I said I had some \"local patches\", but I forgot the\n> relevant part was on master already (the whole subsequent leak analysis\n> is local): 366bd129dc3 (test-lib: add a SANITIZE=leak logging mode,\n> 2022-07-28)\n>\n> So I think you're making the whole \"have the test suite make these\" part\n> of this too complex,\n\nMy main point was that the examples need to be covered by the test\nsuite, and that bundles were a simple way of capturing the tested setup.\nThe GIT_BUNDLE_DIR was simply a way of collecting them in one place for\nease of distribution with the documentation (as you mentioned that's a\ngap in the automation step)\n>  i.e. with a minor variation of that you can do this\n> in test-lib.sh:\n>\n> \tTEST_RESULTS_BDL_DIR=\"$TEST_RESULTS_DIR/$TEST_NAME.test-bdl\"\n>\n> Then have a helper like:\n>\n> \tdo_my_test_bundle_stuff () {\n> \t\tlocal name=\"$1\" &&\n> \t\tshift &&\n> \t\tgit bundle create \"$TEST_RESULTS_BDL_DIR/$name.bdl\" \"$@\"\n> \t}\nThat would be a simplification\n>\n> Then all you need to make these is:\n>\n> \tmake test GIT_TEST_OPTS=--debug\n\nI didn't quite get the link in that step between the `--debug` and the\nbndl prep.\n>\n> And then:\n>\n> \tls -l t/test-results/*.bdl/*\n>\n> Will give you all the bundles you want.\n>\n> We're then missing the \"connect it with the docs\" part, but that part\n> would be e.g.:\n>\n> - Have a validation target for it that depends on such a \"make test\" run\n> - Sanity check that we did create the ones the docs expect\n>\n> Anyway.\n>\n> I like the end-goals, but I can't help think that bundles are the wrong\n> direction to go in, as opposed to:\n>\n> 1. We could have \"make install\" ship test-lib{,-functions}.sh along\n>    with git itself\n> 2. Have e.g. t/doc-setup/*.sh with the snippets to set up these test\n>    demos, which we'd also ship.\n\nI'm not into the idea of shipping a load of extra `test` infrastructure\nwith Git itself. We don't ship any in the Git for Windows install.\n \nI'd viewed the bundles as an adjunct to the also independent manuals and\ndocumentation.\n>\n> And you'd then do e.g.:\n>\n> \tgit test-case rebase/description-1\n>\n> Which would just be a thin wrapper around (pseudocode):\n>\n> \tdir=$(git test--helper get-tempdir) &&\n> \t(\n> \t\tcd \"$dir\" &&\n>                 # And whatever else we need to set up to say \"use this local installed git\"\n> \t\tGIT_TEST_INSTALLED=[...] &&\n> \t\t. \"$(git --test-dir)\"/test-lib.sh\n> \t\t. \"$(git --test-dir)\"/doc-setup/rebase-description-1.sh\"\n>\n> \t\t# This would run test_commit, and any other arbitrary setup code\n> \t\tsetup_rebase_description_1()\n> \t) &&\n> \techo \"Go play with '$dir', and have fun\"\n>\n> The big advantage of that is:\n>\n>  1. It ships with git, so it works offline, and we won't get into the\n>     inevitable confusion of the user reading v2.40 docs locally, but the\n>     online url tracking a v2.50 example or whatever.\n\nIsn't this always a problem (lack of synchronisation between multiple\nsources). If the user has local documentation, it should have matching\nbundles locally, No?\n>  2. We can represent whatever arbitrary repo state, e.g. drop the user\n>     into a merge conflict that's already happening.\nIsn't the manual's example already meant to describe that? And the user\nthen (hopefully) follows along in their throw-away copy of the repo bundle.\n>  3. We can also represent \"repo-extra\" state, e.g. imagine the\n>     \"git-gc/git-repack\" docs discussing loose objects, and wanting to\n>     get you into some arbitrary state with regards to your loose objects\n>     etc.\nThat's a future option, but way past the initial concept for user\non-boarding.\n\nThanks\n\nPhilip\n\n"},{"id":"466450","messageId":"7baed8ab-075f-52d4-52d2-3e555ea59c5d@iee.email","threadId":"58737","inReplyTo":"221103.86zgd81jtm.gmgdl@evledraar.gmail.com","subject":"Re: [RFC/PoC 2/5] Documentation Bundles: ignore artifact files from creating doc bundles","fromName":"Philip Oakley","fromEmail":"philipoakley@iee.email","sentAt":"2022-11-03T19:50:26Z","receivedAt":"2022-11-03T19:50:31Z","isPatch":false,"sender":{"key":"philipoakley@iee.email","avatar":"https://avatars.githubusercontent.com/u/914343?v=4"},"body":"On 03/11/2022 09:12, Ævar Arnfjörð Bjarmason wrote:\n> On Wed, Nov 02 2022, Philip Oakley wrote:\n>\n>> The Git documentation contains many examples that can be prepared as\n>> repository bundles for users to clone and explore without any language\n>> or translation confusions between the written word and intended code.\n> I just find this commit entirely confusing, we're adding .gitignore\n> entries, but nothing generates these yet (but I'll read on). All to\n> describe some seeming future feature...\n\nProbably could be squashed in with the others once the PoC has matured a\nbit.\n>\n>> There are also a large number of example repositories in the t/ test\n>> system can can also be exposed to users to highlight the critical\n>> distinctions tested therein.\n>>\n>> These documentation bundles will be prepared ar part of the test system.\n> s/ar/as/\n\nOk\n>\n>> Start by ignoring those parts which will be considered build artifacts,\n>> the GIT-BUNDLE-DIR, and the `.bndl` bundle files themselves.\n>>\n>> The artifacts themselves will later be manipulated by the `make` elements\n>> of the documentation system.\n>>\n>> Signed-off-by: Philip Oakley <philipoakley@iee.email>\n>> ---\n>>  .gitignore | 2 ++\n>>  1 file changed, 2 insertions(+)\n>>\n>> diff --git a/.gitignore b/.gitignore\n>> index 6ded10067a..b96a4ce5df 100644\n>> --- a/.gitignore\n>> +++ b/.gitignore\n>> @@ -3,6 +3,7 @@\n>>  /fuzz-pack-headers\n>>  /fuzz-pack-idx\n>>  /GIT-BUILD-OPTIONS\n>> +/GIT-BUNDLE-DIR\n>>  /GIT-CFLAGS\n>>  /GIT-LDFLAGS\n>>  /GIT-PREFIX\n>> @@ -250,3 +251,4 @@ Release/\n>>  *.dSYM\n>>  /contrib/buildsystems/out\n>>  CMakeSettings.json\n>> +*.bndl\n> We've been putting quite a few things in .build/ recently, so whatever\n> this is, if you put it in .build/test-bundles/ or whatever you don't\n> need to write \"clean\" rules for it, or add this ignore entry.\nOk. I'd also hadn't considered the 'clean' rules for the make file.\n\nPhilip\n"},{"id":"466451","messageId":"d8072985-95cb-724b-c21f-cde3b2f6945a@iee.email","threadId":"58737","inReplyTo":"221103.86v8nw1jpi.gmgdl@evledraar.gmail.com","subject":"Re: [RFC/PoC 3/5] test-lib: add GIT_BUNDLE_DIR setup","fromName":"Philip Oakley","fromEmail":"philipoakley@iee.email","sentAt":"2022-11-03T19:50:49Z","receivedAt":"2022-11-03T19:51:00Z","isPatch":false,"sender":{"key":"philipoakley@iee.email","avatar":"https://avatars.githubusercontent.com/u/914343?v=4"},"body":"On 03/11/2022 09:14, Ævar Arnfjörð Bjarmason wrote:\n> On Wed, Nov 02 2022, Philip Oakley wrote:\n>\n>> The test system may output a number of documentation bundles containing\n>> test repositories of interest to users, including those explicitly\n>> documented.\n>>\n>> Provide a default location for holding those bundles.\n>> As build artefact's this location, along with `.bndl` files, was\n>> ignored in an earlier commit.\n>>\n>> This may need further refinement.\n>>\n>> Signed-off-by: Philip Oakley <philipoakley@iee.email>\n>> ---\n>>  t/test-lib.sh | 14 ++++++++++++++\n>>  1 file changed, 14 insertions(+)\n>>\n>> diff --git a/t/test-lib.sh b/t/test-lib.sh\n>> index 4e45bc7c4f..b18cd8a34e 100644\n>> --- a/t/test-lib.sh\n>> +++ b/t/test-lib.sh\n>> @@ -56,6 +56,20 @@ then\n>>  \techo \"PANIC: Running in a $TEST_DIRECTORY that doesn't end in '/t'?\" >&2\n>>  \texit 1\n>>  fi\n>> +if test -z \"$GIT_BUNDLE_DIR\"\n>> +then\n>> +\t# Similarly, override this to store the documentation bundles subdir\n>> +\t# elsewhere\n>> +\tGIT_BUNDLE_DIR=$TEST_DIRECTORY/GIT_BUNDLE_DIR\n>> +\tif test ! -d \"$GIT_BUNDLE_DIR\"\n>> +\tthen\n>> +\t\tmkdir -p \"$GIT_BUNDLE_DIR\"\n> Okey, so \"mkdir -p\", not \"mkdir\" once one thinks about it for a second:\n> Because we might be testing in parallel, so this will be racy.\n>\n>> +\tfi\n>> +elif test ! -d \"$GIT_BUNDLE_DIR\"\n>> +then\n>> +\techo \"PANIC: Your \"$GIT_BUNDLE_DIR\" doesn't exist!\" >&2\n>> +\texit 1\n>> +fi\n> I think it's better just to put these in the \"test-results\" dir.\n>\n> For leak testing I have local patches to e.g. create test-results/<test\n> number>.leak, and then after a run I slurp that up and process it.\n>\n> So, same idea, you could just add stuff there, and afterwards grab it\n> there. No need for specifying the dir etc., you just need a test-lib.sh\n> flag to say \"yes, write these out\".\nOk, I suppose it depends on the choice between pre-slurp and\npost-slurping these delivery artefact bundles.\n\nTaylor and Junio maybe able to suggest ideas about approaches that would\nfit the documentation delivery pipeline.\n\nPhilip\n"},{"id":"466452","messageId":"7d1a9294-2925-5bfc-31b2-9bfbdaac3e35@iee.email","threadId":"58737","inReplyTo":"221103.86r0yk1jld.gmgdl@evledraar.gmail.com","subject":"Re: [RFC/PoC 5/5] docbundle: provide the git-log History Simplification example","fromName":"Philip Oakley","fromEmail":"philipoakley@iee.email","sentAt":"2022-11-03T19:50:55Z","receivedAt":"2022-11-03T19:51:08Z","isPatch":false,"sender":{"key":"philipoakley@iee.email","avatar":"https://avatars.githubusercontent.com/u/914343?v=4"},"body":"On 03/11/2022 09:17, Ævar Arnfjörð Bjarmason wrote:\n> On Wed, Nov 02 2022, Philip Oakley wrote:\n>\n>> +note () {\n>> +\tgit tag \"$1\"\n>> +}\n> Aren't you just...\n>\n>> +test_expect_success setup '\n>> +\t# get the structure set out, then the file contents\n>> +\techo \"asdf\" >foo &&\n>> +\techo \"quux\" >quux &&\n>> +\tgit add foo quux &&\n>> +\ttest_tick && git commit -m \"I: Initial commit\" &&\n>> +\tnote I &&\n> ...re-inventing \"test_commit\" here (including the \"echo\")?\n\nProbably.\n\nIt was a bit of 'monkey coding' of an earlier test without any real\nstudy.. As you can probably tell it is just getting the spirit of the\nhistory simplification setup in place..\n\n"},{"id":"466453","messageId":"221103.861qqj22bl.gmgdl@evledraar.gmail.com","threadId":"58737","inReplyTo":"ab275fa0-2977-f880-cc33-836f667287bc@iee.email","subject":"Re: [RFC/PoC 1/5] doc: provide DocBundles design document","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-11-03T20:36:10Z","receivedAt":"2022-11-03T20:47:01Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Thu, Nov 03 2022, Philip Oakley wrote:\n\n> Hi Ævar,\n> Thanks for the review,\n>\n> On 03/11/2022 09:19, Ævar Arnfjörð Bjarmason wrote:\n\n> My main point was that the examples need to be covered by the test\n> suite, and that bundles were a simple way of capturing the tested setup.\n> The GIT_BUNDLE_DIR was simply a way of collecting them in one place for\n> ease of distribution with the documentation (as you mentioned that's a\n> gap in the automation step)\n>>  i.e. with a minor variation of that you can do this\n>> in test-lib.sh:\n>>\n>> \tTEST_RESULTS_BDL_DIR=\"$TEST_RESULTS_DIR/$TEST_NAME.test-bdl\"\n>>\n>> Then have a helper like:\n>>\n>> \tdo_my_test_bundle_stuff () {\n>> \t\tlocal name=\"$1\" &&\n>> \t\tshift &&\n>> \t\tgit bundle create \"$TEST_RESULTS_BDL_DIR/$name.bdl\" \"$@\"\n>> \t}\n> That would be a simplification\n\nI forgot that there needs to be a \"mkdir\" of the \"$TEST_RESULTS_BDL_DIR\"\nin there somewhere (see the *.leak dir setup), but other than that\nthat's pretty much all there's to it.\n\n>> Then all you need to make these is:\n>>\n>> \tmake test GIT_TEST_OPTS=--debug\n>\n> I didn't quite get the link in that step between the `--debug` and the\n> bndl prep.\n\nSorry, I was misrecalling that we wiped away \"test-results\" except\nwithout \"--debug\", which when I spend more than 2 seconds thinking about\nit makes no sense, that's just for the trash directories. Nevermind.\n\n>> And then:\n>>\n>> \tls -l t/test-results/*.bdl/*\n>>\n>> Will give you all the bundles you want.\n>>\n>> We're then missing the \"connect it with the docs\" part, but that part\n>> would be e.g.:\n>>\n>> - Have a validation target for it that depends on such a \"make test\" run\n>> - Sanity check that we did create the ones the docs expect\n>>\n>> Anyway.\n>>\n>> I like the end-goals, but I can't help think that bundles are the wrong\n>> direction to go in, as opposed to:\n>>\n>> 1. We could have \"make install\" ship test-lib{,-functions}.sh along\n>>    with git itself\n>> 2. Have e.g. t/doc-setup/*.sh with the snippets to set up these test\n>>    demos, which we'd also ship.\n>\n> I'm not into the idea of shipping a load of extra `test` infrastructure\n> with Git itself. We don't ship any in the Git for Windows install.\n\nWe don't ship in anywhere now, but presumably GfW has the required parts\nto use it, since it has git-submodule.sh and other *.sh now?\n\nAnyway....\n\n> I'd viewed the bundles as an adjunct to the also independent manuals and\n> documentation.\n\n...yeah me too, I'm just wondering if we won't back ourselves into a\ncorner with some examples, where we'll want to hand the user more\ncomplex state.\n\nBut maybe simpler is better, and it can just be a bundle plus \"run these\ncommands manually\". The bundle can also contain a script in-tree that\nperforms the finishing touches, if required (unless there's no 'sh' at\nruntime in GfW?).\n\nAnyway, *if* that's a good idea (and I've got no idea, just throwing it\nout there) it doesn't need to be hindered by lack of 'sh', we could just\nship compiled C code that does the required run_command() sequences to\nset up the desired state.\n\nThen in the test suite we'd call that as a test helper, or a new\nbuilt-in helper,.\n\n>>\n>> And you'd then do e.g.:\n>>\n>> \tgit test-case rebase/description-1\n>>\n>> Which would just be a thin wrapper around (pseudocode):\n>>\n>> \tdir=$(git test--helper get-tempdir) &&\n>> \t(\n>> \t\tcd \"$dir\" &&\n>>                 # And whatever else we need to set up to say \"use this local installed git\"\n>> \t\tGIT_TEST_INSTALLED=[...] &&\n>> \t\t. \"$(git --test-dir)\"/test-lib.sh\n>> \t\t. \"$(git --test-dir)\"/doc-setup/rebase-description-1.sh\"\n>>\n>> \t\t# This would run test_commit, and any other arbitrary setup code\n>> \t\tsetup_rebase_description_1()\n>> \t) &&\n>> \techo \"Go play with '$dir', and have fun\"\n>>\n>> The big advantage of that is:\n>>\n>>  1. It ships with git, so it works offline, and we won't get into the\n>>     inevitable confusion of the user reading v2.40 docs locally, but the\n>>     online url tracking a v2.50 example or whatever.\n>\n> Isn't this always a problem (lack of synchronisation between multiple\n> sources). If the user has local documentation, it should have matching\n> bundles locally, No?\n\nYes, I just didn't know if that was your plan, so if that's how it works\nthen no problem.\n\nIt's just a depressingly common pattern these days that manual pages are\n\"just read about it at this link\", which inevitably links to some\nout-of-date documentation, even some of GCC's docs are doing that these\ndays.\n\nIt's good that that's not the plan here.\n\n>>  2. We can represent whatever arbitrary repo state, e.g. drop the user\n>>     into a merge conflict that's already happening.\n> Isn't the manual's example already meant to describe that? And the user\n> then (hopefully) follows along in their throw-away copy of the repo bundle.\n\nYeah, maybe that's fine.\n\n>>  3. We can also represent \"repo-extra\" state, e.g. imagine the\n>>     \"git-gc/git-repack\" docs discussing loose objects, and wanting to\n>>     get you into some arbitrary state with regards to your loose objects\n>>     etc.\n> That's a future option, but way past the initial concept for user\n> on-boarding.\n"}]}