Re: [RFC][Outreachy] Seeking Git Community Feedback on My Application

>
> On Wed, Oct 25, 2023 at 2:46 PM Isoken Ibizugbe <isokenjune@gmail.com> wrote:
>
> > Thank you for the review. I have made changes to the project plan and
> > it emphasizes the critical tasks of identifying, selecting, and
> > porting tests, making it more concise and aligned with the project's
> > scope.
>
> Good.
>
> > - Community Bonding (Oct 2 - Nov 20): Microproject contribution, Git
> > project application, get familiar with the Git codebase and testing
> > ecosystem.
> > -Identify and Select Tests: Identify and prioritize tests worth
> > porting, and document the selected tests. (I would classify tests that
> > are worth porting according to the following for now;
> >
> > Relevance: Prioritize tests that are relevant to the current Git codebase.
> > Coverage: Focus on tests that cover core functionality or critical code paths.
> > Usage Frequency: Port tests that are frequently used or run in Git's
> > development process.
> > Isolation: Choose tests that can be easily ported and run independently.
>
> I think the main issue with identification is that now in t/helper/ we
> have both:
>
>   1) code that implements helpers that are used by the end-to-end
> tests scripts written in shell and named "t/tXXXX-*.sh" where XXXX is
> a number, and
>   2) code that implements unit tests for some C code in the code base.
>
> So I think only 2) should be ported to the unit test framework, and 1)
> should not be ported and stay in t/helper/.
>
> > - Write Unit Tests: Write unit tests for the identified test cases
> > using the Git custom test framework.
> > - Port Existing Tests: Port selected test cases from the t/helper
> > directory to the unit testing framework, by modifying them to work
> > within the custom TAP framework.
> > - Test Execution and Debugging: Execute the newly written unit tests
> > and the ported tests using the test framework.
> > - Seek Feedback: Share the progress with mentors and the Git
> > community, and address any concerns or suggestions provided by the
> > community.
> > - Documentation and Reporting: Document the entire process of
> > migrating Git's tests to the unit testing framework, and prepare a
> > final project report summarizing the work done, challenges faced, and
> > lessons learned.
> >
> > What is the custom TAP framework?
> >
> > According to this patch
> > (https://lore.kernel.org/git/ca284c575ece0aee7149641d5fb1977ccd7e7873.1692229626.git.steadmon@google.com/)
> > by Phillip Wood, which contains an example implementation for writing
> > unit tests with TAP output. The custom TAP framework is a Test
> > Anything Protocol (TAP) framework that allows for clear reporting of
> > test results, aiding in debugging and troubleshooting.
>
> Ok. Our end-to-end tests scripts written in shell already use TAP,
> that's why it's nice to have unit tests also using TAP.
>
> > The framework contains the following features:
> >
> > - Test Structure: Unit tests are defined as functions containing
> > multiple checks. The tests are run using the TEST() macro. If any
> > checks within a test fail, the entire test is marked as failed.
> > - Output Format: The output of the test program follows the TAP
> > format. It includes a series of messages describing the test's status.
> > For passed tests, it reports "ok," and for failed tests, it reports
> > "not ok." Each test is numbered, e.g., "ok 1 - static initialization
> > works," to indicate success or failure.
> > - Check Functions: Several check functions are available, including
> > check() for boolean conditions, check_int(), check_uint(), and
> > check_char() for comparing values using various operators. check_str()
> > is used to compare strings.
> > - Skipping Tests: Tests can be skipped using test_skip() and can
> > include a reason for skipping, which is printed as part of the report.
> > - Diagnostic Messages: Tests can generate diagnostic messages using
> > test_msg() to provide additional context or explanations for test
> > failures.
> > - Planned Failing Tests: Tests that are known to fail can be marked
> > with TEST_TODO(). These tests will still run, and the failures will be
> > reported, but they will not cause the entire suite to fail.
> > - Building and Running: The unit tests can be built with "make
> > unit-tests" (with some additional Makefile changes), and they can be
> > executed manually or using a tool like prove.
>
> Ok.
>
> > Using the formerly given criteria, test-ctype.c is suitable for
> > porting because it tests character type checks used extensively in
> > Git. These tests cover various character types and their expected
> > behaviour, ensuring the correctness and reliability of Git's
> > operations, and test-ctype.c isolation makes it suitable for porting
> > without relying on multiple libraries.
>
> Ok.
>
> > Here is a sample of the implementation of how I would write the unit
> > test following the custom TAP framework taking t/helper/test-ctype.c
> >
> > - Create and rename the new .c file;
> > I would rename it according to the convention done in the t/unit-test
> > directory, by starting the name with a “t-” prefix e.g t-ctype.c
> >
> > - Document the tests and include the necessary headers:
> > /**
> >  *Tests the behavior of ctype
> >  *functions
> > */
> > #include "test-lib.h"
> > #include "ctype.h"
> >
> > - Define test functions:
> > #define DIGIT "0123456789"
> >
> > static void t_digit_type(void)
> > {
> >     int i;
> >     const char *digits = DIGIT;
> >     for (i = 0; digits[i]; i++)
> >    {
> >          check_int(isdigit(digits[i]), ==, 0);
> >    }
>
> This tests that isdigit() returns 0 for each of the characters in
> "0123456789", but first I think isdigit() should return 1, not 0 for
> those characters.

>
> And second, I think the test should check the value returned by
> isdigit() for each of the 256 possible values of a char, not just for
> the characters in "0123456789".
>
> test-ctype.c is doing the right thing regarding those 2 issues.
>
> > - Include main function which will call the test functions using the TEST macro;
> > int main(void)
> > {
> >     TEST(t_digit_type(), "Character is a digit");
> >     return test_done();
> > }
> >
> > - Run the tests:
> > ‘make && make’ unit-tests can be used build and run the unit tests
> > Or run the test binaries directly:
> > ./t/unit-tests/t-ctype.c
> >
> > The Makefile will be modified to add the file;
> > UNIT_TEST_PROGRAMS += t-ctype
> > The test output will be in the TAP format and will indicate which
> > tests passed(ok) and which failed(not ok), along with diagnostic
> > messages in case of failures.
> >
> > ok 1 - Character is a digit
> >
> > 1..1
>
> Yeah, this looks right.
>
> Thanks,
> Christian.