{"thread":{"id":"61251","subject":"[GSoC 2024][v2 Proposal]Move existing tests to a unit testing framework","startedAt":"2024-04-02T08:09:10Z","lastAt":"2024-04-02T14:22:42Z","messageCount":3,"participants":["Aishwarya Narayanan","Karthik Nayak"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"492031","messageId":"CAHCXyj2y131B6C+EA9fn2zPhx+LBsL9csHgJSy4GmK9xjdMQSg@mail.gmail.com","threadId":"61251","inReplyTo":null,"subject":"[GSoC 2024][v2 Proposal]Move existing tests to a unit testing framework","fromName":"Aishwarya Narayanan","fromEmail":"aishnana.03@gmail.com","sentAt":"2024-04-02T08:09:02Z","receivedAt":"2024-04-02T08:09:10Z","isPatch":false,"sender":{"key":"aishnana.03@gmail.com","avatar":null},"body":"Dear Git Organization,\nI'm writing to follow up on my Google Summer of Code proposal, \"Move\nExisting Tests to a Unit Testing Framework.\"\nAfter careful consideration and based on our discussions, I've refined\nthe project summary and deliverables to focus specifically on Move\nexisting tests to a unit testing framework.This email aims to gather\nfeedback and initiate a discussion on the\nfeasibility and implementation of this idea.\n---\nGoogle Summer of Code\n\n2024\n\nName: Aishwarya N\n\nUniversity: SASTRA University\n\nProgram: B.Tech Electrical and Electronics Engineering\n\nOrganization: Git\n\nProject Idea: Move existing tests to a unit testing framework\n\nGitHub: https://github.com/Aishwarya-Narayanan03\n\nEmail: aishnana.03@gmail.com\n\nTimezone: IST (GMT+5:30)\n\n\n\nAbout Me\n\nMy name is Aishwarya N., and I am soon to graduate from SASTRA Deemed\nUniversity in Thanjavur, India, with a B.Tech. in Electrical and\nElectronics Engineering. With graduation only two months away, I'm\nready to plunge into the exciting world of open source and offer my\nexpertise to a project that enables developers. The power of building\nsomething from nothing through programming is a truly special and\nempowering feeling. It's no wonder it resonates with me! The Git\nproject particularly appeals to me. Contributing to the tool that\nsupports innumerable software developments provides an opportunity to\nhave a genuine, global impact.\n\nWhy Me?\n\nAdaptable and Open to Challenges: I thrive in lively surroundings and\nsee new challenges as opportunities to learn and progress. I\nacknowledge that mistakes are unavoidable on the route to mastery, and\nI am not afraid to learn from them.\n\nAn avid learner: My thirst for knowledge is boundless. I'm always\nlooking for new methods to improve myself and push the boundaries of\ntechnology.\nCollaborative Spirit: While I'm quite satisfied working independently,\nI believe that collaborating is crucial for accomplishing ambitious\ngoals. I am an adept communicator who is eager to work with the\nbrilliant Git community.\n\nWhy Git?\nThe fast-paced world of software piques my curiosity, and Git, a basic\ntool for collaboration and version control, is a necessary skill set\nfor success in this area of expertise. But, beyond its functionality,\nGit is fascinating.I'm excited to contribute to the project that\nunderpins numerous developer workflows.\n\nProject Summary\nThe Git project has recently implemented a new unit testing framework\nto improve code quality and maintainability. Currently, numerous\nlegacy test cases are stored in separate test helper files\n(t/helper/test-*.c) written in C, as well as test scripts (t/*.sh)\ncreated in the shell to call these helpers. This project focuses on\nmigrating these existing tests to a unified format using the new\nframework within dedicated C source files (t/unit-tests/t-*.c).This\nconversion will offer various advantages:\nBy unifying tests in the new framework, the codebase structure becomes\nmore consistent.\nImproved test readability and maintainability: The new framework makes\nit easier to write clear and maintainable tests.\nBy utilising the new framework, we can expand our testing capabilities\nand features beyond the current approach.\n\nBefore GSoC\n\nMicroproject:Avoiding suppressing git's exit code in test scripts\n\nLearning Git Codebase\n\n1. Understanding Test Scripts: Exploring the test scripts in the Git\nproject's t- directory provided personal experience with real-world\nShell scripting used for integration testing. This helpd me understand\nhow Git's features are tested and confirmed.\n2.Reading Commits: The reference to commit c6f44e1da5 demonstrated how\nto handle Git commands in test scripts. Analysing this commit allowed\nme to examine the adjustments done to address the issue of suppressing\nGit's exit code, as well as get insight into the codebase's structure\nand convention.\n3.Navigating the Git Repository: Searching for instances of\nproblematic code and finding solutions within the Git repository\nenhanced my ability to successfully navigate and search a big\ncodebase.\n\nBest Practices\n\n1. Avoiding Pitfalls:Avoiding typical mistakes in test scripts, such\nas suppressing exit codes, can lead to more dependable testing\ninfrastructure.\n2.Understanding Pipe Usage: Knowing when and how to use pipes in Shell\nscripting is critical. This microproject describes circumstances in\nwhich pipes may mistakenly suppress exit codes and provides\nsuggestions for proper use.\nConnecting With Mentors:\n1.Engaging with Git contributors: This microproject allowed me to\nconnect with mentors and Git project contributors. Seeking clarity on\nspecific code practices or debating changes to test scripts encourages\ninvolvement in the open-source community.\n2.Receiving comments: Submitting updates or queries about the\nmicroproject helped in receving useful comments from experienced\ndevelopers.This feedback helped me learn the Git codebase and improve\nmy scripting skills.\n3.Community Involvement: Participating in discussions about the\nmicroproject on platforms such as Git email lists or forums allowed me\nto communicate with a diverse community of developers while sharing\ninformation and experiences.\n\nRelated Works\n\nhttps://lore.kernel.org/git/CAHCXyj3U69qyhYewOLY9hN2rvi_5ZuSxQEBJbDxrKefm9MzVWg@mail.gmail.com/\n- Microproject Approach\n\nhttps://lore.kernel.org/git/ZgQffea0krKmZUEt@tanuki/ - [RFC PATCH]\n             Description: While printf might be unlikely to fail\nhere,I felt the change aligns with the project's goal of ensuring Git\ncommand exit codes are captured. This approach is more robust and\navoids potential issues in the future.\n               Status: Declined\n\nhttps://public-inbox.org/git/xmqqttkqwfwe.fsf@gitster.g/ - [RFC PATCH]\nFix Git command exit code suppression in test script\nt2104-update-index-skip-worktree.sh\nDescription: This patch increases the robustness of Git's testing\nframework by guaranteeing that Git commands' exit codes are\nappropriately examined during test execution. Previously, suppressed\nexit codes allowed tests to pass despite Git command failures. This\nfix stores the output of Git commands in variables and examines their\nexit codes to detect errors.The discussion focuses on improvements to\nthe patch submission process, such as adhering to coding principles\nand referencing appropriate documentation for proper formatting and\ntest script adjustments.\nStatus: Declined\n\n https://public-inbox.org/git/CAHCXyj1hUVNNuCOgsNv4GJUi79_o9iWZDvV8Ocz3DodreYoL7g@mail.gmail.com/\n  - GSoC 2024 [PATCH v2] Fix Git command exit code suppression in test\nscript t2104-update-index-skip-worktree.sh\n                                 Description: This patch resolves an\nissue in Git test scripts in which the exit code of git ls-files -t\nmay be disregarded, resulting in tests passing despite Git command\nfailures. The new version guarantees that Git commands used in\npipelines produce captured output and perform adequate exit code\nchecks. It also enhances code style by adhering to established rules.\nThis adds to more robust Git testing by ensuring the identification\nand reporting of Git command errors within test scripts.\nStatus: Under Review\n\nPorting Unit Tests:\n\n[1]https://lore.kernel.org/git/cover.1692297001.git.steadmon@google.com/\n\nThis patch series provides unit test functionality for the Git project.\nThe series includes the following patches:\nA project plan document that outlines the goals for introducing unit\ntests, as well as a review of potential frameworks and the features\nused to assess them.\nAn implementation of the TAP unit test framework with a sample unit\ntest and Makefile integration.\nChanges were made to the Git build system so that the unit tests could\nbe run in CI.\nThe TAP framework was selected because of the following advantages:\nSimple to use and comprehend.\nWidely used and integrates seamlessly with existing Git tools.\n\nThe project plan document mentions a number of outstanding TODOs,\nincluding how to ensure tests run on various OS platforms and how to\ngather unit test statistics.\nThis is a well-written patch series with a straightforward and logical\napproach. The documentation is comprehensive and instructive, and the\ncode is well-organized and simple to grasp.\nHere are some extra factors to consider:\nThe patch series discusses collecting unit test statistics. We could\nclarify on the type of statistics we want to collect?\nHow do we intend to deal with flaky tests?\n\n[2]https://lore.kernel.org/git/20240112102743.1440-1-ach.lumap@gmail.com/\n\nThis is an email thread about a patch series submitted by Achu Luma to\nport the unit tests for C character classification functions from the\nlegacy helper tool to the new unit testing framework\n([Outreachy][PATCH v5] Port helper/test-ctype.c to\nunit-tests/t-ctype.c).\n\nHere are the major changes implemented in the patch series:\nThe unit tests for C character classification routines (isdigit(),\nisspace(), etc.) are moved from the historical approach, which used\nthe test-tool command test-tool ctype in t/helper/test-ctype.c, to the\nnew unit testing framework (t/unit-tests/test-lib.h).\nRefactors the tests to use the framework's testing macros (TEST() and\ncheck_*()).\nAdded tests for handling EOF (end-of-file).\nOverall, the patch series was well received and thought to be a\npositive development. The reviewers made various suggestions for\nimprovement, such as expanding on the types of statistics the author\nis interested in gathering for the unit tests and how to deal with\nflaky tests.\n\n[3] https://lore.kernel.org/git/20240112102122.1422-1-ach.lumap@gmail.com/\n\nThis patch migrates unit tests for the advise_if_enabled functionality\nfrom the legacy approach to a new unit testing framework. Here's a\nbreakdown of the changes:\n\nLegacy approach:\n\nUsed the test-tool advise command in t/helper/test-advise.c.\n\nRelied on helper functions from test-tool.h.\n\nTests were written in shell script (t/t0018-advice.sh).\n\nNew approach:\n\nUses the new unit testing framework with TEST() macros\n(t/unit-tests/t-advise.c).\n\nIncludes necessary headers for advise.h, config.h, and setup.h.\n\nTests are written in C code.\n\nThe patch removes the following files from the legacy approach:\n\nt/helper/test-advise.c\n\nt/t0018-advice.sh\n\nThe new unit test file t/unit-tests/t-advise.c includes three tests\ncovering different scenarios:\n\nAdvice is printed when the configuration variable is unset.\n\nAdvice is printed when the configuration variable is set to true.\n\nAdvice is not printed when the configuration variable is set to false.\n\nOverall, this patch improves the codebase by using a more standardized\nand maintainable approach for unit testing.\n\n[4]https://lore.kernel.org/git/20240205162506.1835-1-ach.lumap@gmail.com/\n\nRefactoring 64-bit Pre-requisite Checks in date.h\n\nThis is a patch for the Linux kernel which refactors pre-requisite\nchecks for 64-bit systems into reusable functions in the date.h header\nfile.\n\nContext:\n\nThe codebase recently added a unit testing framework.\n\nSome shell tests use a helper tool that checks for 64-bit\nprerequisites (TIME_IS_64BIT and TIME_T_IS_64BIT).\n\nThese checks are also needed in the new unit tests.\n\nTo avoid duplication, the patch moves these checks to date.h as inline\nfunctions.\n\nChanges:\n\nTwo new inline functions are added to date.h:\n\ncheck_prereq_TIME_IS_64BIT: Checks if the size of timestamp_t is 8\nbytes (indicating a 64-bit system).\n\ncheck_prereq_TIME_T_IS_64BIT: Checks if the size of time_t is 8 bytes\n(indicating a 64-bit system).\n\nThe shell test functions in t/helper/test-date.c are modified to use\nthe new functions instead of directly checking the sizes.\n\nBenefits:\n\nReduces code duplication.\n\nImproves code maintainability.\n\nSimplifies porting shell test code to the new unit test framework.\n\nNext Steps:\n\nA following commit will port the relevant code from\nt/helper/test-date.c to the new unit test file t/unit-tests/t-date.c,\nusing the newly introduced functions for pre-requisite checks.\n\n[5]Unit Testing in Git:\n\nhttps://github.com/git/git/blob/master/Documentation/technical/unit-tests.txt\ndiscusses unit testing for the Git project. This gives an idea of the\nbackground and importance of unit testing in the project applying to.\n\n\n\nDuring GSoc\nThe main goals of this project are:\n1. Understand the existing \"reftable\" unit tests: Examine the present\nimplementation of the \"reftable\" unit tests in\n't0032-reftable-unittest.sh' to ensure that you understand their\nfunctionality and purpose.\n2. Learn the new unit testing framework: Get a thorough grasp of Git's\nnew unit testing framework, including setup, usage, and recommended\npractices.\n3.Ensuring test coverage and reliability: Check that the converted\ntests have sufficient code coverage and accurately simulate the\nexpected behaviour of the \"reftable\" capability. Test and validate the\nnew tests thoroughly to ensure their reliability.\n4. Documentation and code comments: Document the conversion process,\nincluding any issues encountered and the reasoning behind design\ndecisions. Update the code comments and documentation to reflect the\nchanges to the tests.\nDeliverables\n\nü All legacy test cases previously residing in t/helper/test-*.c and\nt/*.sh files will be converted to the new unit testing framework\nwithin dedicated C source files in t/unit-tests/t-*.c.\n\nü The migrated tests will be thoroughly executed and validated to\nensure complete coverage of all existing test functionalities.\n\nü Documentation explaining the migration procedure, as well as\nrevisions to existing documentation.\n\nü The final report summarises the project's accomplishments, problems,\nand future recommendations.\n\nExpected timeline\nCommunity bonding period (May 1 - May 26)\n\nFamiliarise myself with the Git project's code and contribution standards.\nDiscuss the project plan and milestones with mentors.\nEstablish the development environment and tools required for the project.\n\nPhase 1: Analysis and Planning (May 27 - June 15)\nExamine the current reftable unit tests and the unit testing framework\ndocumentation.\nPlan the migration process by identifying the test cases and\ncapabilities that will be covered.\nBreak down tasks and establish a precise project timeline.\n\nPhase 2: Transition to C-based Unit Tests (June 16–July 15)\nBegin rebuilding the existing shell scripts as C-based unit tests.\nEnsure that the migration process is accurate and comprehensive, while\nretaining test coverage.\nCommunicate with mentors on a regular basis to receive feedback and guidance.\n\nPhase 3: Testing and Validation (July 16–August 5)\nEvaluate the migrated unit tests against different scenarios and edge\nsituations.\nDebug and resolve any errors or anomalies that arise during testing.\nBefore finalising, ensure that the unit tests are proper and reliable.\n\nPhase 4: Documentation and Reporting (August 6–August 26)\nDocument the migration process, including the obstacles encountered\nand solutions implemented.\nUpdate the relevant documentation to reflect the modifications to the\nreftable unit tests.\nPrepare a detailed report summarising the project's results and contributions.\n\nAvailability\nMy university holidays begin on June 30. However, considering this is\nmy final year with only three classes and major project, I believe it\nwill not be a problem. I intent to work 6 hours a day, 5 days a week\nthis summer.I have no scheduled trips except for GIM-SRIP from May 6th\nto 31st. I will inform Mentors advance notice of any changes in\navailability.\n\n After GSoC\nI realise that it would be much more beneficial if our GSoC\nparticipants could stay after the event. And that is exactly what I\nwant to do. I think doing open-source, especially with a community\nthat fuels a popular development tool, is simply fascinating. And, in\naddition to being refreshing, such a community allows me to learn and\ngrow, with the possibility of making more significant contributions if\nI continue to participate.While this is my first foray into\nopen-source development and GSoC, I am excited by the prospect of\ncontributing to the ongoing success of Git. I have been actively\nlearning and gaining a strong foundation in Git commands, programming\nlanguages, and version control concepts.\n\nClosing Remarks\nFinally, I'd like to express my gratitude to the community and, in\nparticular, my mentors, Patrick Steinhardt,Christian Couder,Kaartic\nSivaraam,Karthik Nayak,Junio C Hamano. They did and continue to do an\nexceptional task of maintaining and empowering the Git open-source\ncommunity, as well as offering much-needed and kind assistance to a\nnew contributor like me.\n---\nSincerely,\nAishwarya Narayanan\n"},{"id":"492041","messageId":"CAOLa=ZS_pZEU+BHXoTpQyJy6Sn7j+ap5J+mZWOgnL39HKhK1Ww@mail.gmail.com","threadId":"61251","inReplyTo":"CAHCXyj2y131B6C+EA9fn2zPhx+LBsL9csHgJSy4GmK9xjdMQSg@mail.gmail.com","subject":"Re: [GSoC 2024][v2 Proposal]Move existing tests to a unit testing framework","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2024-04-02T10:39:18Z","receivedAt":"2024-04-02T10:39:20Z","isPatch":false,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Hello,\n\nAishwarya Narayanan <aishnana.03@gmail.com> writes:\n> Dear Git Organization,\n> I'm writing to follow up on my Google Summer of Code proposal, \"Move\n> Existing Tests to a Unit Testing Framework.\"\n> After careful consideration and based on our discussions, I've refined\n> the project summary and deliverables to focus specifically on Move\n> existing tests to a unit testing framework.This email aims to gather\n> feedback and initiate a discussion on the\n> feasibility and implementation of this idea.\n\nIt's always better to send updates as responses to your previous emails.\nThis allows for reviewers to contain context in a single thread.\n\n> Related Works\n>\n> https://lore.kernel.org/git/CAHCXyj3U69qyhYewOLY9hN2rvi_5ZuSxQEBJbDxrKefm9MzVWg@mail.gmail.com/\n> - Microproject Approach\n>\n> https://lore.kernel.org/git/ZgQffea0krKmZUEt@tanuki/ - [RFC PATCH]\n>              Description: While printf might be unlikely to fail\n> here,I felt the change aligns with the project's goal of ensuring Git\n> command exit codes are captured. This approach is more robust and\n> avoids potential issues in the future.\n>                Status: Declined\n>\n\nIt would be nice if you could also add a summary on the approach and\nwhat went wrong and why.\n\n> https://public-inbox.org/git/xmqqttkqwfwe.fsf@gitster.g/ - [RFC PATCH]\n> Fix Git command exit code suppression in test script\n> t2104-update-index-skip-worktree.sh\n> Description: This patch increases the robustness of Git's testing\n> framework by guaranteeing that Git commands' exit codes are\n> appropriately examined during test execution. Previously, suppressed\n> exit codes allowed tests to pass despite Git command failures. This\n> fix stores the output of Git commands in variables and examines their\n> exit codes to detect errors.The discussion focuses on improvements to\n> the patch submission process, such as adhering to coding principles\n> and referencing appropriate documentation for proper formatting and\n> test script adjustments.\n> Status: Declined\n>\n\nThis patch actually was reviewed by Junio [1] and there was no follow up\nfrom your side. I'm not sure how this was considered Declined, so it\nwould be nice to have some reasoning about it.\n\n>\n>  https://public-inbox.org/git/CAHCXyj1hUVNNuCOgsNv4GJUi79_o9iWZDvV8Ocz3DodreYoL7g@mail.gmail.com/\n>   - GSoC 2024 [PATCH v2] Fix Git command exit code suppression in test\n> script t2104-update-index-skip-worktree.sh\n>                                  Description: This patch resolves an\n> issue in Git test scripts in which the exit code of git ls-files -t\n> may be disregarded, resulting in tests passing despite Git command\n> failures. The new version guarantees that Git commands used in\n> pipelines produce captured output and perform adequate exit code\n> checks. It also enhances code style by adhering to established rules.\n> This adds to more robust Git testing by ensuring the identification\n> and reporting of Git command errors within test scripts.\n> Status: Under Review\n\nThis seems to be the same patch as the previous. A new thread was\ncreated here instead of replying to the previous thread.\n\n> Porting Unit Tests:\n>\n> [1]https://lore.kernel.org/git/cover.1692297001.git.steadmon@google.com/\n>\n> This patch series provides unit test functionality for the Git project.\n> The series includes the following patches:\n> A project plan document that outlines the goals for introducing unit\n> tests, as well as a review of potential frameworks and the features\n> used to assess them.\n> An implementation of the TAP unit test framework with a sample unit\n> test and Makefile integration.\n> Changes were made to the Git build system so that the unit tests could\n> be run in CI.\n> The TAP framework was selected because of the following advantages:\n> Simple to use and comprehend.\n> Widely used and integrates seamlessly with existing Git tools.\n>\n> The project plan document mentions a number of outstanding TODOs,\n\nMaybe I missed it, but where are the TODOs stated?\n\n> Next Steps:\n>\n> A following commit will port the relevant code from\n> t/helper/test-date.c to the new unit test file t/unit-tests/t-date.c,\n> using the newly introduced functions for pre-requisite checks.\n>\n> [5]Unit Testing in Git:\n>\n> https://github.com/git/git/blob/master/Documentation/technical/unit-tests.txt\n> discusses unit testing for the Git project. This gives an idea of the\n> background and importance of unit testing in the project applying to.\n>\n>\n>\n> During GSoc\n> The main goals of this project are:\n> 1. Understand the existing \"reftable\" unit tests: Examine the present\n> implementation of the \"reftable\" unit tests in\n> 't0032-reftable-unittest.sh' to ensure that you understand their\n> functionality and purpose.\n> 2. Learn the new unit testing framework: Get a thorough grasp of Git's\n> new unit testing framework, including setup, usage, and recommended\n> practices.\n> 3.Ensuring test coverage and reliability: Check that the converted\n> tests have sufficient code coverage and accurately simulate the\n> expected behaviour of the \"reftable\" capability. Test and validate the\n> new tests thoroughly to ensure their reliability.\n> 4. Documentation and code comments: Document the conversion process,\n> including any issues encountered and the reasoning behind design\n> decisions. Update the code comments and documentation to reflect the\n> changes to the tests.\n> Deliverables\n\nKaartic's review in the previous version mentions [2]:\n\n    Your project summary and the deliverables below seem to mention that\n    you're going to work on migration of the reftable unit tests but the\n    project title seems to be \"Move existing tests to a unit testing\n    framework'.\n\nIt seems this version is the same...\n\n> Closing Remarks\n> Finally, I'd like to express my gratitude to the community and, in\n> particular, my mentors, Patrick Steinhardt,Christian Couder,Kaartic\n> Sivaraam,Karthik Nayak,Junio C Hamano. They did and continue to do an\n> exceptional task of maintaining and empowering the Git open-source\n> community, as well as offering much-needed and kind assistance to a\n> new contributor like me.\n> ---\n> Sincerely,\n> Aishwarya Narayanan\n>\n\nIt would also be nice if you mentioned what exactly is different in this\nversion compared to the previous.\n\n[1]: https://public-inbox.org/git/xmqqttkqwfwe.fsf@gitster.g/\n[2]: https://lore.kernel.org/git/CA+ARAtpqD0um9bVrjRKG0DmrxVR-46uSKfDKnO+H1rUp0i+4Ww@mail.gmail.com/#t\n"},{"id":"492058","messageId":"CAHCXyj0UjP5hxh4sk-bqMmh70SUM-FTg-Tv1FWp9+0c9aVGh2g@mail.gmail.com","threadId":"61251","inReplyTo":"CAOLa=ZS_pZEU+BHXoTpQyJy6Sn7j+ap5J+mZWOgnL39HKhK1Ww@mail.gmail.com","subject":"Re: [GSoC 2024][v2 Proposal]Move existing tests to a unit testing framework","fromName":"Aishwarya Narayanan","fromEmail":"aishnana.03@gmail.com","sentAt":"2024-04-02T14:22:34Z","receivedAt":"2024-04-02T14:22:42Z","isPatch":false,"sender":{"key":"aishnana.03@gmail.com","avatar":null},"body":"Dear Git Maintainers,\n>It's always better to send updates as responses to your previous emails.\n>This allows for reviewers to contain context in a single thread.\nThanks for the feedback.\n\n>This patch actually was reviewed by Junio [1] and there was no follow up\n>from your side. I'm not sure how this was considered Declined, so it\n>would be nice to have some reasoning about it.\nI apologize for the confusion and inconvenience caused by creating a\nnew thread for this patch. I'm still under development in my Git\ncontribution workflow and should have replied to the original thread\nreviewed by Junio [1]. Additionally, Junio mentioned that exiting from\nthe test_expect_success block is not recommended. I wasn't aware of\nthis best practice.\n\n>It would also be nice if you mentioned what exactly is different in this\n>version compared to the previous.\nIn essence, the changed version project summary provides a more\ncomprehensive overview of the legacy test migration project, while the\nprevious project summary offers a more targeted perspective on the\n\"reftable\" test case as an example.\nThe project summary in the above version of the proposal addresses an\napproach encompassing all legacy tests, including shell scripts\n(t/.sh) and helper C files (t/helper/test-.c). \"Reftable\" tests might\nnot be part of this group; they're not explicitly mentioned. This\nproposal solely focuses on the process of migrating all legacy tests\nand the advantages of a unified testing approach. It highlights\ncodebase consistency and the potential for expanded testing\ncapabilities.\n\nThanks again for your time and patience.\n\nSincerely,\nAishwarya Narayanan\n\n.\n\nOn Tue, 2 Apr 2024 at 16:09, Karthik Nayak <karthik.188@gmail.com> wrote:\n>\n> Hello,\n>\n> Aishwarya Narayanan <aishnana.03@gmail.com> writes:\n> > Dear Git Organization,\n> > I'm writing to follow up on my Google Summer of Code proposal, \"Move\n> > Existing Tests to a Unit Testing Framework.\"\n> > After careful consideration and based on our discussions, I've refined\n> > the project summary and deliverables to focus specifically on Move\n> > existing tests to a unit testing framework.This email aims to gather\n> > feedback and initiate a discussion on the\n> > feasibility and implementation of this idea.\n>\n> It's always better to send updates as responses to your previous emails.\n> This allows for reviewers to contain context in a single thread.\n>\n> > Related Works\n> >\n> > https://lore.kernel.org/git/CAHCXyj3U69qyhYewOLY9hN2rvi_5ZuSxQEBJbDxrKefm9MzVWg@mail.gmail.com/\n> > - Microproject Approach\n> >\n> > https://lore.kernel.org/git/ZgQffea0krKmZUEt@tanuki/ - [RFC PATCH]\n> >              Description: While printf might be unlikely to fail\n> > here,I felt the change aligns with the project's goal of ensuring Git\n> > command exit codes are captured. This approach is more robust and\n> > avoids potential issues in the future.\n> >                Status: Declined\n> >\n>\n> It would be nice if you could also add a summary on the approach and\n> what went wrong and why.\n>\n> > https://public-inbox.org/git/xmqqttkqwfwe.fsf@gitster.g/ - [RFC PATCH]\n> > Fix Git command exit code suppression in test script\n> > t2104-update-index-skip-worktree.sh\n> > Description: This patch increases the robustness of Git's testing\n> > framework by guaranteeing that Git commands' exit codes are\n> > appropriately examined during test execution. Previously, suppressed\n> > exit codes allowed tests to pass despite Git command failures. This\n> > fix stores the output of Git commands in variables and examines their\n> > exit codes to detect errors.The discussion focuses on improvements to\n> > the patch submission process, such as adhering to coding principles\n> > and referencing appropriate documentation for proper formatting and\n> > test script adjustments.\n> > Status: Declined\n> >\n>\n> This patch actually was reviewed by Junio [1] and there was no follow up\n> from your side. I'm not sure how this was considered Declined, so it\n> would be nice to have some reasoning about it.\n>\n> >\n> >  https://public-inbox.org/git/CAHCXyj1hUVNNuCOgsNv4GJUi79_o9iWZDvV8Ocz3DodreYoL7g@mail.gmail.com/\n> >   - GSoC 2024 [PATCH v2] Fix Git command exit code suppression in test\n> > script t2104-update-index-skip-worktree.sh\n> >                                  Description: This patch resolves an\n> > issue in Git test scripts in which the exit code of git ls-files -t\n> > may be disregarded, resulting in tests passing despite Git command\n> > failures. The new version guarantees that Git commands used in\n> > pipelines produce captured output and perform adequate exit code\n> > checks. It also enhances code style by adhering to established rules.\n> > This adds to more robust Git testing by ensuring the identification\n> > and reporting of Git command errors within test scripts.\n> > Status: Under Review\n>\n> This seems to be the same patch as the previous. A new thread was\n> created here instead of replying to the previous thread.\n>\n> > Porting Unit Tests:\n> >\n> > [1]https://lore.kernel.org/git/cover.1692297001.git.steadmon@google.com/\n> >\n> > This patch series provides unit test functionality for the Git project.\n> > The series includes the following patches:\n> > A project plan document that outlines the goals for introducing unit\n> > tests, as well as a review of potential frameworks and the features\n> > used to assess them.\n> > An implementation of the TAP unit test framework with a sample unit\n> > test and Makefile integration.\n> > Changes were made to the Git build system so that the unit tests could\n> > be run in CI.\n> > The TAP framework was selected because of the following advantages:\n> > Simple to use and comprehend.\n> > Widely used and integrates seamlessly with existing Git tools.\n> >\n> > The project plan document mentions a number of outstanding TODOs,\n>\n> Maybe I missed it, but where are the TODOs stated?\n>\n> > Next Steps:\n> >\n> > A following commit will port the relevant code from\n> > t/helper/test-date.c to the new unit test file t/unit-tests/t-date.c,\n> > using the newly introduced functions for pre-requisite checks.\n> >\n> > [5]Unit Testing in Git:\n> >\n> > https://github.com/git/git/blob/master/Documentation/technical/unit-tests.txt\n> > discusses unit testing for the Git project. This gives an idea of the\n> > background and importance of unit testing in the project applying to.\n> >\n> >\n> >\n> > During GSoc\n> > The main goals of this project are:\n> > 1. Understand the existing \"reftable\" unit tests: Examine the present\n> > implementation of the \"reftable\" unit tests in\n> > 't0032-reftable-unittest.sh' to ensure that you understand their\n> > functionality and purpose.\n> > 2. Learn the new unit testing framework: Get a thorough grasp of Git's\n> > new unit testing framework, including setup, usage, and recommended\n> > practices.\n> > 3.Ensuring test coverage and reliability: Check that the converted\n> > tests have sufficient code coverage and accurately simulate the\n> > expected behaviour of the \"reftable\" capability. Test and validate the\n> > new tests thoroughly to ensure their reliability.\n> > 4. Documentation and code comments: Document the conversion process,\n> > including any issues encountered and the reasoning behind design\n> > decisions. Update the code comments and documentation to reflect the\n> > changes to the tests.\n> > Deliverables\n>\n> Kaartic's review in the previous version mentions [2]:\n>\n>     Your project summary and the deliverables below seem to mention that\n>     you're going to work on migration of the reftable unit tests but the\n>     project title seems to be \"Move existing tests to a unit testing\n>     framework'.\n>\n> It seems this version is the same...\n>\n> > Closing Remarks\n> > Finally, I'd like to express my gratitude to the community and, in\n> > particular, my mentors, Patrick Steinhardt,Christian Couder,Kaartic\n> > Sivaraam,Karthik Nayak,Junio C Hamano. They did and continue to do an\n> > exceptional task of maintaining and empowering the Git open-source\n> > community, as well as offering much-needed and kind assistance to a\n> > new contributor like me.\n> > ---\n> > Sincerely,\n> > Aishwarya Narayanan\n> >\n>\n> It would also be nice if you mentioned what exactly is different in this\n> version compared to the previous.\n>\n> [1]: https://public-inbox.org/git/xmqqttkqwfwe.fsf@gitster.g/\n> [2]: https://lore.kernel.org/git/CA+ARAtpqD0um9bVrjRKG0DmrxVR-46uSKfDKnO+H1rUp0i+4Ww@mail.gmail.com/#t\n"}]}