{"thread":{"id":"59425","subject":"[ANNOUNCE] Sharness v1.2.0","startedAt":"2023-03-21T00:15:37Z","lastAt":"2023-12-13T14:01:46Z","messageCount":4,"participants":["Felipe Contreras","Jiang Xin","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"473820","messageId":"CAMP44s3qa_CoM_4--UmwYQTgO-5dHh6=jogH-rxF7OXEWr53Lw@mail.gmail.com","threadId":"59425","inReplyTo":null,"subject":"[ANNOUNCE] Sharness v1.2.0","fromName":"Felipe Contreras","fromEmail":"felipe.contreras@gmail.com","sentAt":"2023-03-21T00:15:17Z","receivedAt":"2023-03-21T00:15:37Z","isPatch":false,"sender":{"key":"felipe.contreras@gmail.com","avatar":"https://avatars.githubusercontent.com/u/8358?v=4"},"body":"Hello,\n\nVersion 1.2.0 of Sharness [1] -- the test harness library derived from\nGit's test lib -- is released.\n\nThis release contains a few features from upstream and many\nimprovements.\n\n * Moved functions into lib-sharness/functions.sh\n * Add simple example\n * Add support for zsh\n * Clarify the purpose of `aggregate-results.sh`\n * Add `SHARNESS_TEST_OUTDIR`\n * Add `SHARNESS_TEST_NB`\n * Add `-x` option for tracing commands\n * Add vim syntax file\n * Minor bugfixes\n\nMany thanks to all that contributed to this release:\n\n 1 Albert Chu\n 1 Chris Dunlap\n 39 Christian Couder\n 2 Eric Curtin\n 72 Felipe Contreras\n 1 Marcel Schilling\n 1 Rafael Ascensão\n 1 Scott Bronson\n 1 Tristan Le Guern\n\nThe previous release was v1.1.0 in September 2018.\n\nTo find more information about the release, including download\nlocations, check the GitHub Release [3].\n\nTo find more information about Sharness, you can check the\nSharness site [4].\n\nChristian Couder has decided to transfer the ownership of the\nproject [2] to me, Felipe Contreras.\n\nThank you Christian for keeping the project afloat these years and your\nmany contributions.\n\nEnjoy.\n\n[1] https://github.com/felipec/sharness\n[2] https://github.com/felipec/sharness/commit/00d30002218928d11d27d7c308909baf6da6da8b\n[3] https://github.com/felipec/sharness/releases/tag/v1.2.0\n[4] https://felipec.github.io/sharness/\n\n-- \nFelipe Contreras\n"},{"id":"485503","messageId":"802ca62b9d9672e9553ab064452d46e0d72dfc76.1702116416.git.zhiyou.jx@alibaba-inc.com","threadId":"59425","inReplyTo":"CAMP44s3qa_CoM_4--UmwYQTgO-5dHh6=jogH-rxF7OXEWr53Lw@mail.gmail.com","subject":"New attempt to export test-lib from Git, maybe Sharness2?","fromName":"Jiang Xin","fromEmail":"worldhello.net@gmail.com","sentAt":"2023-12-09T13:35:39Z","receivedAt":"2023-12-09T13:35:47Z","isPatch":false,"sender":{"key":"worldhello.net@gmail.com","avatar":"https://avatars.githubusercontent.com/u/183860?v=4"},"body":"From: Jiang Xin \n\nOn Mon, 20 Mar 2023 18:15:17 -0600, Felipe Contreras wrote:\n> Hello,\n> \n> Version 1.2.0 of Sharness [1] -- the test harness library derived from\n> Git's test lib -- is released. \n\nI like Sharness which make it possible to reuse the test framework of\nGit. I try to use sharness in my projects and I'm having fun writing\ntest cases using familiar shell scripts.\n\nBut the test framework used by Sharpness is from Git v1.7.9 and is\nquite old. The absence of test helper API and command options such as\n\"--run=...\" is quite inconvenient for me. Furthermore, the lack of\nlint tools results in many errors in my test cases (e.g., missing\n\"&&\" between statements).\n\nIt's not easy to upgrade sharness to the latest test framework of Git.\nSo I decide to start a new project. The new project is named test-lib,\nsee:\n\n * https://github.com/jiangxin/test-lib\n\nSome of my projects have upgraded the test framework from sharkness to\ntest-lib:\n\n * git-po-helper: https://github.com/git-l10n/git-po-helper/tree/main/test\n * git-repo-go: https://github.com/alibaba/git-repo-go/tree/master/test\n\nI wonder if we can start Sharness2 based on this solution. See the\nREADME of the test-lib project for details:\n\n\ndiff --git a/README.md b/README.md\nnew file mode 100644\nindex 0000000000..4cc529ab73\n--- /dev/null\n+++ b/README.md\n@@ -0,0 +1,159 @@\n+Test Lib From Git Core\n+======================\n+\n+Test-lib is a test framework developed by Junio and is specifically\n+designed for the Git project. It allows us to write a test suite\n+using shell script, which contains a collection of test cases. The\n+output of each test suite is presented in TAP ([Test Anything\n+Protocol]) format. We can use test-lib or any other TAP harness\n+programs (e.g., prove) to run and analyze the output of the test\n+suites.\n+\n+In order to reuse the git test framework into other projects,\n+the [sharness project] made a successful attempt. However, it is\n+based on an outdated version of git (v1.7.9), which results in\n+bugs and missing new features. For example:\n+\n+ * Commit d88785e424 (test-lib: set `BASH_XTRACEFD` automatically,\n+ 2016-05-11) and commit a5bf824f3b4d (t: prevent '-x' tracing\n+ from interfering with test helpers' stderr, 2018-02-25) of the Git\n+ project addressed bugs when we run test suites with \"-x\" option.\n+\n+ * Commit 0445e6f0a1 (test-lib: '--run' to run only specific tests,\n+ 2014-04-30) provided better control of the set of tests to run.\n+\n+ * Commit 92b269f5c5 (test-lib: turn on `GIT_TEST_CHAIN_LINT`\n+ by default, 2015-04-22) turned on chain-lint by default to\n+ prevent the accidental omission of \"&&\" between statements in\n+ test cases. Additionally, Git offers more linter tools such as\n+ \"chainlint.pl\" to help write correct test cases.\n+\n+ * The latest version of the Git project includes numerous test\n+ helpers that are not present in sharness. These helpers\n+ provide a more comprehensive and efficient testing developing\n+ tools. E.g.: `test_bool_env`, `test_cmp_bin`, `test_commit`,\n+ `test_config`, `test_env`, `test_file_not_empty`, `test_file_size`,\n+ `test_line_count`, `test_oid`, `test_path_exists`,\n+ `test_path_is_executable`, `test_path_is_missing`, `test_tick`,\n+ `write_script`, etc.\n+\n+In order to reuse the latest test framework of the Git project and\n+easy to maintain, use the following strategies:\n+\n+1. Use [git-filter-repo] to export test-lib related files and their\n+ commit histories from the Git project. The generated tailored\n+ commits are saved in the branch named \"git-test-lib\".\n+\n+2. The test-lib test framework relies on a helper program named\n+ \"test-tool\", which is written in C. To use test-lib without the\n+ need for C compilation, re-implemented part of the \"test-tool\"\n+ subcommands in Python.\n+\n+3. Apply a series of patches to test-lib so that it can be easily\n+ reused in other projects. Some patches are borrowed from Sharness.\n+\n+4. This project is a by-product of the Git project. For each Git\n+ release, we will re-run step 1 to export the latest test\n+ framework from Git projects which may include new files we\n+ missed in older versions. The generated commit history will\n+ overwrite the \"git-test-lib\" branch, and the master branch will\n+ rebase on it.\n+\n+\n+Install test-lib\n+----------------\n+\n+To create test suites in shell scripts powered by test-lib, you can\n+follow these steps:\n+\n+1. Set up a directory (such as \"test\") to save test suites and\n+ files of test-lib.\n+\n+ $ mkdir test\n+\n+2. Clone or copy the test-lib repository inside the test directory.\n+\n+ $ cd test\n+ $ git clone https://github.com/jiangixn/test-lib lib\n+\n+4. Copy files from the example test directory.\n+\n+ $ cp lib/test-example/.gitignore .\n+ $ cp lib/test-example/.gitattributes .\n+ $ cp lib/test-example/Makefile .\n+\n+4. Start writing a test suite powered by test-lib, make sure it\n+ sources the \"test-lib.sh\" file from test-lib. Refer to the example\n+ test suite (e.g. \"test/t0001-test-tool-chmtime.sh\") to write\n+ your own test suite:\n+\n+ test_description='My first test suite'\n+\n+ . lib/test-lib.sh\n+\n+\n+Usage of test-lib\n+-----------------\n+\n+As for how to use test-lib to write, run and manage your test suites,\n+please see the documentation [README.git] for reference.\n+\n+\n+Filtering test-lib from the Git project\n+---------------------------------------\n+\n+\"test-lib\" is part of the Git project and is stored in the \"t/\"\n+subdirectory. We use \"git-filter-repo\" to export test-lib related\n+files to the root directory of this project. We repeat this\n+periodically to save historical commits of test-lib in the\n+\"git-test-lib\" branch of this project. The master branch of this\n+project will merge with or rebase onto the \"git-test-lib\" branch.\n+As how to filter and export test-lib from the Git project, see the\n+steps below.\n+\n+1. Make a fresh clone of the Git project before filtering.\n+\n+ $ git clone --single-branch --no-tags \\\n+ https://github.com/git/git.git \\\n+ git-test-lib\n+\n+ $ cd git-test-lib\n+\n+2. Use git-filter-repo to filter and export test-lib.\n+\n+ $ git filter-repo \\\n+ --preserve-commit-encoding \\\n+ --prune-degenerate always \\\n+ --path COPYING \\\n+ --path shared.mak \\\n+ --path t/.gitattributes \\\n+ --path t/.gitignore \\\n+ --path t/Makefile \\\n+ --path t/README \\\n+ --path t/aggregate-results.sh \\\n+ --path t/chainlint/ \\\n+ --path t/chainlint.pl \\\n+ --path t/check-non-portable-shell.pl \\\n+ --path t/oid-info/ \\\n+ --path t/perf/.gitignore \\\n+ --path t/perf/Makefile \\\n+ --path t/perf/README \\\n+ --path t/perf/aggregate.perl \\\n+ --path t/perf/config \\\n+ --path t/perf/min_time.perl \\\n+ --path t/perf/perf-lib.sh \\\n+ --path t/perf/p0000-perf-lib-sanity.sh \\\n+ --path t/perf/run \\\n+ --path t/test-lib-functions.sh \\\n+ --path t/test-lib-github-workflow-markup.sh \\\n+ --path t/test-lib-junit.sh \\\n+ --path t/test-lib.sh \\\n+ --path t/lib-subtest.sh \\\n+ --path t/t0000-basic.sh \\\n+ --path-rename t/:\n+\n+\n+[Test Anything Protocol]: http://testanything.org/\n+[sharness project]: https://github.com/felipec/sharness\n+[git-filter-repo]: https://github.com/newren/git-filter-repo\n+[README.git]: ./README.git\n-- \n2.42.0.rc2.21.gd1f87c2148\n\n"},{"id":"485508","messageId":"xmqqcyvfmccr.fsf@gitster.g","threadId":"59425","inReplyTo":"802ca62b9d9672e9553ab064452d46e0d72dfc76.1702116416.git.zhiyou.jx@alibaba-inc.com","subject":"Re: New attempt to export test-lib from Git, maybe Sharness2?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2023-12-09T18:59:00Z","receivedAt":"2023-12-09T18:59:06Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jiang Xin writes:\n\n> It's not easy to upgrade sharness to the latest test framework of Git.\n\nSo?\n\n> So I decide to start a new project. The new project is named test-lib,\n> see:\n>\n> * https://github.com/jiangxin/test-lib\n>\n> Some of my projects have upgraded the test framework from sharkness to\n> test-lib:\n>\n> * git-po-helper: https://github.com/git-l10n/git-po-helper/tree/main/test\n> * git-repo-go: https://github.com/alibaba/git-repo-go/tree/master/test\n>\n> I wonder if we can start Sharness2 based on this solution. See the\n> README of the test-lib project for details:\n\nIs it a viable option to stick to the name \"test-lib\" (or possibly,\n\"git-test-lib\" to make it more prominent to say where it came from)?\n\nIf you do not plan to coordinate with those who work on (the remnant\nof) the original sharness based on an ancient version of our test\nframework, and do not plan to actively transition its users to your\nversion, it is less confusing if you named yours differently, as it\navoids hinting that your version is a successor of theirs.\n\nI am not sure if reusing the history of our project verbatim using\nfilter-repo is really a good way to help those who are interested in\nthe test framework, by the way. We make changes for our own purpose\nand as a part of such an effort we may touch the test framework to\nmake it easier to test the changes we made, e.g.,\n\nhttps://github.com/jiangxin/test-lib/commit/0d5db66ef2b9d8ed5bcee9a0167672dc88b1b026\n\nand unedited filter-repo result will describe such a commit\nprimarily to explain why the changes in the commit was made on Git\nside. Most of the changes described in the resulting commit message\nare discarded by filter-repo and the resulting history becomes hard\nto follow.\n"},{"id":"485614","messageId":"CANYiYbGUo6TsOBHLGNF7onRgM1fsE3CU3=8=vwJsiuXnsnuLog@mail.gmail.com","threadId":"59425","inReplyTo":"xmqqcyvfmccr.fsf@gitster.g","subject":"Re: New attempt to export test-lib from Git, maybe Sharness2?","fromName":"Jiang Xin","fromEmail":"worldhello.net@gmail.com","sentAt":"2023-12-13T14:01:33Z","receivedAt":"2023-12-13T14:01:46Z","isPatch":false,"sender":{"key":"worldhello.net@gmail.com","avatar":"https://avatars.githubusercontent.com/u/183860?v=4"},"body":"On Sun, Dec 10, 2023 at 2:59 AM Junio C Hamano wrote:\n> Is it a viable option to stick to the name \"test-lib\" (or possibly,\n> \"git-test-lib\" to make it more prominent to say where it came from)?\n\nWill rename the project to \"git-test-lib\".\n\n> If you do not plan to coordinate with those who work on (the remnant\n> of) the original sharness based on an ancient version of our test\n> framework, and do not plan to actively transition its users to your\n> version, it is less confusing if you named yours differently, as it\n> avoids hinting that your version is a successor of theirs.\n>\n> I am not sure if reusing the history of our project verbatim using\n> filter-repo is really a good way to help those who are interested in\n> the test framework, by the way.\n\nIf all historical commits were squashed, it would be difficult to\ntrack future changes of test-lib in the Git project. Futhermore, who\nwill be the author of the squashed commit?\n\n> and unedited filter-repo result will describe such a commit\n> primarily to explain why the changes in the commit was made on Git\n> side. Most of the changes described in the resulting commit message\n> are discarded by filter-repo and the resulting history becomes hard\n> to follow.\n\nOne solution is to add notes in the README file and users can refer to\nthe commit history of the git project.\n"}]}