Re: [Outreachy][proposal]: Convert unit tests to use the Clar testing framwork

>
> On Fri, Oct 25, 2024 at 11:04 AM Patrick Steinhardt <ps@pks.im> wrote:
> >
> > On Wed, Oct 23, 2024 at 10:45:52PM +0100, Samuel Abraham wrote:
> >
> > Thanks for your application!
> >
> > > ## Project Overview - Convert unit test to use the Clar testing framework:
> > >
> > > I understand that the Git project is in the process of converting its
> > > unit tests to the Clar testing framework to improve test readability,
> > > consistency and maintainability, a move away from homegrown unit test
> > > frameworks such as t/helper/test-tool.h and t/unit-test/test-lib.h.
> >
> > Nit: we've basically moved away from test-tool for unit tests already,
> > to the best of my knowledge. So this is more about the second part,
> > moving away from our own test framework.
>
> Thank you very much for the clarity.
> >
> > > Clar provides a structured and efficient way to write and execute
> > > tests and is well-suited for a project like Git where robust testing
> > > is essential to maintain quality and stability.
> >
> > It would be nice to provide some pointers _why_ we think that clar is
> > better suited ;) Hint: you may have a look at the patch series that
> > introduced the clar for some ideas there.
>
> Okay I will do that.
>
> >
> > [snip]
> > > ## Project Plan:
> >
> > Formatting of this section is a bit funky, which makes it harder than
> > necessary to figure out which parts belong together. I'd propose to
> > first provide a high-level list of the steps you want to do with a
> > bulleted list and then maybe put more detailed explanations into
> > separate "### subsections". You may also want to convert links to use
> > [Markdown syntax](https://www.markdownguide.org/basic-syntax/#links),
> > which provides some structure aronud them.
>
> Okay thank you for this.
> >
> > > The first steps to migrating existing tests to Clar would be studying
> > > existing tests in the t/helper and t/unit-tests directory which will
> > > enable me to determine those appropriate for conversion.
> > >
> > > Liaise with mentors and community members to determine the viability
> > > of converting the selected test for conversion.
> > >
> > > Convert the selected test to Clar while in constant effective
> > > communication with the mentors and Git community, implementing
> > > feedback and review suggestions.
> > >
> > > Properly test converted scripts to ensure validity and correctness.
> > >
> > > Steps to converting the existing unit tests to the Clar Testing Framework
> > >
> > >  From the project description in the Outreachy Project Description page,
> > >
> > >
> > > https://www.outreachy.org/outreachy-dec-2024-internship-cohort/communities/git/#convert-unit-tests-to-use-the-clar-testing-framewo,
> > > the goal is to convert all Git’s existing unit tests to use the Clar
> > > testing framework exclusively.
> >
> > It might also make sense to plan some time to add missing features to
> > the clar if we hit anything during the conversion.
>
> Okay noted.
> >
> > > The existing unit tests which would need to be converted to Clar are;
> > >
> > > .c test files present in t/unit-tests: Tests in this directory use the
> > > t/unit-tests/test-lib testing framework and are “.c” scripts,
> > > converted from the shell-based testing which used the tests in
> > > t/helper/ and corresponding t<number>-<name>.sh test files. Examples
> > > of this conversion can be seen in the link below which references Achu
> > > Luma’s work
> > >
> > > https://lore.kernel.org/git/20240226143350.3596-1-ach.lumap@gmail.com/.
> > >
> > >
> > > The link shows the work done in converting the test from
> > > t/helper/test-sha1.c and test/helper/test-sha256.c to use
> > > t/unit-tests/t-hash.c which uses the t/unit-tests/test-lib.h framework
> > >
> > >
> > > The steps to convert files located in this directory include;
> > >
> > > - Identify the test files to be converted.
> > >
> > > Examples of such files are the t/unit-tests/t-hash.c, t/unit-tests/t-strbuf.c
> > >
> > > - Rename the test file from t-<name>.c to <name>.c. This pattern
> > > follows the style used by Patrick Steinhardt in his conversion of the
> > > t-strvec.c and t-ctype.c files to use the clar framework.
> > >
> > > https://lore.kernel.org/git/604303e31aad3a9e74f7bdddc84f11d4d137c864.1725459142.git.ps@pks.im/
> > > and
> > >
> > > https://lore.kernel.org/git/ba05b9f1eef8136e087846ee54a076558097a240.1725459142.git.ps@pks.im/
> > >
> > > As a sample demonstration, we will convert the t/unit-tests/t-hash.c
> > > to use Clar.
> >
> > I think the previous explanations are sufficient -- going into the
> > technical details like you do below is appreciated, but ultimately not
> > necessary in my opinion. What I'm after here is that you have a rough
> > understanding of what needs to be done for each of the tests, and that
> > is sufficiently demonstrated by a high-level explanation.
>
> Okay noted.
> >
> > > The test scripts in the t/helper directory:
> > >
> > > The tests in this directory are invoked by the corresponding shell
> > > scripts which spawn processes to test the different unit tests in the
> > > t/helper directory.
> > >
> > > The process involved in converting these shell scripts to Clar
> > > framework will typically follow the same steps as illustrated above
> > > bar the following differences.
> > >
> > > Identifying the tests to be converted to Clar
> > >
> > > Create a new .c  file name which should be named appropriately to
> > > illustrate what the file is testing.
> > >
> > > However, the shell-based script which tests the functions in t/helper
> > > will be studied for in-depth understanding and then refactored into a
> > > .c file which follows the steps above in converting to use Clar.
> >
> > As mentioned above, I thought we didn't have any such tests anymore. If
> > we do, it might make sense to provide an example of such a thing that
> > needs to be converted.
>
> Thank you Patrick
> Okay from my understanding, such tests have a .c file in t/helper and
> a corresponding t/t-<number>-<name>.sh file.
> From my master branch which I study, I can see for example
>    - test-env-helper.c in t/helper and t0017-env-helper.sh
>    - test-find-pack.c in t/helper and t0081-find-pack.sh.
> I concluded since the shell script used to compare the output against
> the expected still exists
> and no such file names exist in t/unit-tests, I assumed they have not
> been ported which is why I listed files
> in t/helper will be converted to clear.
> Please I will be happy if I can get more clarity, thanks.
>
> >
> > > Project Timeline:
> > >
> > > Community Bonding (Present - November 26):
> > >
> > > Continue making contributions to the Git codebase working on different
> > > things within my capacity and getting more familiar with the codebase,
> > > participating in patches review.
> > >
> > > Conversion of tests begins (December 9 - December 23): Familiarize me
> > > with the conversion process to further enhance my understanding,
> > > identify files for conversion and start the conversion process, and
> > > set up a blog for weekly updates on my conversion process.
> >
> > Makes sense.
> >
> > > Implementation of community and mentor reviews (December 27 - January 31):
> > >
> > > Continue with conversions while testing converted tests, communicating
> > > with reviewers and implementing reviews.
> > >
> > > Testing (February 1 - March 1): Continuously determine the correctness
> > > of the converted tests by continuously testing and also liaising with
> > > mentors constantly.
> >
> > For this one here'd I'd recommend to start more iteratively. What you
> > have here sounds a bit like a waterfall model, where you first convert
> > all tests and then eventually test and send things over to the mailing
> > list.
> >
> > In the Git community you will likely have more success if you work in
> > smaller patches. E.g. pick a small set of tests to convert, convert
> > them, polish the series and then send it to the mailing list. That cycle
> > would then repeat several times until you have converted all of the
> > tests.
>
> Yes, my contributions have made me understand this process.
> I will make appropriate adjustments. Thank you
> >
> > > Availability:
> > >
> > >  I am not currently enrolled in any academic program or have any jobs,
> > > and will be available to work on the project for a minimum of 45 hours
> > > per week.
> >
> > You really shouldn't be working more than 40 hours per week ;) The
> > [Outreachy Internship Guide](https://www.outreachy.org/docs/internship/)
> > recommends 30 hours per week, and I don't expect any more of you,
> > either.
>
> >
> > Other than that this document looks good to me, thanks!
> >
> > Patrick
>
> Thank you very much Patrick, I appreciate your time. Please I will
> also like to know if i should send the corrected proposal
> as a mail to the list as I did with this first version.
>
> Thanks again for your time.
> Abraham Samuel