{"thread":{"id":"59315","subject":"[RFC][PATCH v2] GSoC 2023 proposal: more sparse index integration","startedAt":"2023-02-27T00:35:31Z","lastAt":"2023-02-27T05:29:58Z","messageCount":2,"participants":["Vivan Garg","Ashutosh Pandey"],"isPatch":true,"patchVersion":2,"patchTotal":null},"messages":[{"id":"472768","messageId":"20230227003517.88254-1-gvivan6@gmail.com","threadId":"59315","inReplyTo":null,"subject":"[RFC][PATCH v2] GSoC 2023 proposal: more sparse index integration","fromName":"Vivan Garg","fromEmail":"gvivan6@gmail.com","sentAt":"2023-02-27T00:35:17Z","receivedAt":"2023-02-27T00:35:31Z","isPatch":true,"sender":{"key":"gvivan6@gmail.com","avatar":null},"body":"Signed-off-by: Vivan Garg <gvivan6@gmail.com>\n---\n .../More-Sparse-Index-Integrations.txt        | 312 ++++++++++++++++++\n 1 file changed, 312 insertions(+)\n create mode 100644 Documentation/More-Sparse-Index-Integrations.txt\n\ndiff --git a/Documentation/More-Sparse-Index-Integrations.txt b/Documentation/More-Sparse-Index-Integrations.txt\nnew file mode 100644\nindex 0000000000..2ab6b07f18\n--- /dev/null\n+++ b/Documentation/More-Sparse-Index-Integrations.txt\n@@ -0,0 +1,312 @@\n+# More Sparse Index Integrations\n+\n+# Personal Information\n+\n+Full name: Vivan Garg\n+\n+E-mail: gvivan6@gmail.com \n+Alternate E-mail: v.garg.work@gmail.com\n+Tel: (+1)437-987-2678\n+\n+Education: University of Waterloo (Canada)\n+Major: Computer Science and Financial Management (Double-Major)\n+Year: Rising Junior\n+\n+LinkedIn: https://www.linkedin.com/in/gvivan/\n+GitHub: https://github.com/gvivan\n+Website: https://gvivan.me/\n+\n+# Before GSoC\n+\n+## Synopsis\n+\n+I've chosen the \"More Sparse Index Integrations\" project idea from the\n+SoC 2023 Ideas page. The goal of this project is to integrate the \n+experimental \"sparse-index\" feature and \"sparse-checkout\" command with \n+existing Git commands. \n+\n+Git 2.25.0 introduced a new experimental `git sparse-checkout` command, \n+which simplified the existing feature and improved performance for \n+large repositories. It allows users to restrict their working directory \n+to only the files they care about, allowing them to ensure the developer \n+workflow is as fast as possible while maintaining all the benefits of a \n+monorepo. \n+(Bring your monorepo down to size with sparse-checkout [1], Stolee).\n+\n+The pattern matching process in Git's sparse-checkout feature becomes \n+expensive as the sparse-checkout file and repository size increase, \n+growing quadratically. This can result in billions of pattern checks \n+for large repositories. However, Git's new mechanism for matching based \n+on folder prefix matches drops the quadratic growth, matching M patterns \n+across N files in O(M+N*d) time, where d is the maximum folder depth of a file. \n+To further optimize the matching process, Git inspects files in a sorted \n+order instead of an arbitrary order. When Git evaluates a file path, it \n+checks whether the start of the folder path matches a recursive pattern exactly. \n+If so, it marks everything in that folder as \"included\" without doing any further \n+hashset lookups. Similarly, when Git detects the start of a folder that's outside \n+of the specified cone, it marks everything in that folder as \"excluded\" without \n+doing any further hashset lookups. This reduces the time to be closer to O(M+N) \n+(Bring your monorepo down to size with sparse-checkout [1], Stolee).\n+\n+[1]: https://github.blog/2020-01-17-bring-your-monorepo-down-to-size-with-sparse-checkout/\n+\n+The Git Fundamentals team at GitHub has contributed a new feature to Git called \n+the sparse index, which allows the index to focus on the files within the \n+sparse-checkout cone in a monorepo. The sparse index stores only the information \n+about the files within the sparse-checkout definition, instead of storing information \n+for every file at HEAD, which can make the index much larger in a monorepo. When \n+enabled with other performance features, the sparse index can have a significant \n+impact on performance (Make your monorepo feel small with Git’s sparse index [2], Stolee).\n+\n+[2]: https://github.blog/2021-11-10-make-your-monorepo-feel-small-with-gits-sparse-index/\n+\n+The sparse index differs from a normal \"full\" index in that it can store directory \n+paths with the object ID for its tree object. It can be used to determine if an \n+entire directory is out of the sparse-checkout cone and replace all of its contained \n+file paths with a single directory path. The use of sparse index can significantly \n+reduce the size of the index, resulting in faster operations \n+(Make your monorepo feel small with Git’s sparse index [3], Stolee).\n+\n+[3]: https://github.blog/2021-11-10-make-your-monorepo-feel-small-with-gits-sparse-index/\n+\n+Because \"sparse-checkout\" and \"sparse-index\" may potentially influence the logics of \n+other Git commands and the internal data structure of Git, some work is required to \n+optimize compatibility and user experience. That is exactly what my chosen idea proposed.\n+\n+## Benefits to Community\n+\n+By joining the community and working on this idea, I can collaborate with my mentor \n+and fellow community members to improve the user experience for people who are working \n+with large monorepos. Furthermore, I am committed to continuing my involvement beyond \n+the GSoC program, not only by contributing to the community but also by sharing my \n+experiences and mentoring future potential newcomers.\n+\n+\n+## Microproject\n+\n+t4121: modernize test style [4]\n+Status: WIP\n+Description: Test scripts in file t4121-apply-diffs.sh are written in old style, \n+where the test_expect_success command and test title are written on\n+separate lines. Therefore update the tests to adhere to the new style.\n+\n+## Other Contributions\n+\n+### Reviewing\n+\n+t9700: modernize test script [5]\n+Status: WIP\n+Description: I reviewed this patch and pointed the contributor in the right direction \n+by providing examples, links and mentioning the best practices.\n+\n+### Patches\n+\n+MyFirstContribution: add note about SMTP server config [6]\n+Status: WIP\n+Description: The documentation on using git-send-email previously mentioned the need \n+to configure git for your operating system and email provider, but did not provide \n+specific details on the relevant configuration settings. This commit adds a note \n+specifying that the relevant settings can be found under the 'sendemail' section of \n+Git's configuration file, with a link to the relevant documentation. The aim is to \n+provide users with a more complete understanding of the configuration process and \n+help them avoid potential roadblocks in setting up git-send-email.\n+\n+[4]: https://lore.kernel.org/git/CACzddJrZ8YdJ72ng3UpMGN9CJx0qW1+fZfyi3q01z2487V8fxw@mail.gmail.com/T/#md53157af31a3f347dd899679fafdea7fcaf7ecfc\n+[5]: https://lore.kernel.org/git/CADupsJPpZnjA=Pu_RZZZXy7Titj3UD7ppww48KvcHHHbrGx=rw@mail.gmail.com/T/#m122db9bdca463c12f0b9ccb259fd1d3229d75945\n+[6]: https://lore.kernel.org/git/20230222011317.97943-1-gvivan6@gmail.com/\n+\n+\n+### Related Work\n+\n+Prior works on the idea have been completed by my mentors and other community members, \n+and these works provide a good approximation of the approach I intend to take. Here \n+are some previous examples of commits:\n+\n+Integration with “mv” [7]\n+Integration with “reset” [8]\n+Integration with “sparse-checkout” [9]\n+Integration with “clean” [10]\n+Integration with “blame” [11]\n+\n+[7]: https://lore.kernel.org/git/20220331091755.385961-1-shaoxuan.yuan02@gmail.com/\n+[8]: https://lore.kernel.org/git/pull.1048.v6.git.1638201164.gitgitgadget@gmail.com/\n+[9]: https://lore.kernel.org/git/pull.1208.v3.git.1653313726.gitgitgadget@gmail.com/\n+[10]: https://github.com/git/git/commit/1e9e10e04891a13e5ccd52b36cfadc55dfaa5066\n+[11]: https://github.com/git/git/commit/add4c864b60766174ad4f74ba7be17e66d61ef16\n+\n+# In GSoC\n+\n+## Plan\n+\n+The proposed idea of increasing \"sparse-index\" integrations may seem \n+straightforward at first glance. However, upon reviewing previous \n+implementations, I discovered that this idea can introduce unforeseen \n+difficulties for some functions. For example, to enable \"sparse-index,\" \n+we must ensure that \"sparse-checkout\" is compatible with the target Git \n+command. Achieving this compatibility requires modifying the original \n+command logic, which can lead to other unanticipated issues. Therefore, \n+I have incorporated some additional steps in the plan outlined below to \n+proactively address potential complications. It's worth noting that \n+points 3-7 are part of the SoC 2023 Ideas proposed by the community \n+and mentors.\n+\n+1. Conduct an investigation to determine if a Git command functions \n+properly with sparse-checkout.\n+\n+2. Modify the logic of the Git command, if necessary, to ensure it \n+functions properly with sparse-checkout. Develop corresponding tests \n+to validate the modifications. \n+\n+3. Add tests to t1092-sparse-checkout-compatibility.sh for the \n+builtin, with a focus on what happens for paths outside of the \n+sparse-checkout cone.\n+\n+4. Disable the command_requires_full_index setting in the builtin \n+and ensure the tests pass.\n+\n+5. If the tests do not pass, then alter the logic to work with the \n+sparse index.\n+\n+6. Add tests to check that a sparse index stays sparse.\n+\n+7. Add performance tests to demonstrate speedup.\n+\n+8. If any changes are made that affect the behavior of the Git \n+command, update the documentation accordingly. Note that such \n+changes should be rare.\n+\n+## Timeline\n+\n+During my discussion with Victoria, she informed me that given my \n+commitment of 175 hours, it is expected that I will be able to fully \n+integrate two commands with sparse index during the GSOC program. My \n+plan is to evenly distribute the work for each command over the course \n+of the program. I am confident that I can start the project early as I \n+have already established communication with my mentors and familiarized \n+myself with the related documentation, although my understanding may \n+not be comprehensive.\n+\n+Based on my prior experience with the idea, I believe I will be able \n+to quickly get up to speed and begin working on the project. The exact \n+timeline for each integration is difficult to determine, but I estimate \n+that I should be able to complete one integration every two months. I \n+have already planned out my next term, and there are only three weeks \n+during which I would prefer to focus on other things: June 23-30 and \n+August 1-15. However, even without an extension, I should be able to \n+manage this timeline. With the flexibility to extend the program, it \n+should be even easier to accommodate any potential scheduling conflicts.\t\n+\n+\t\n+## Availability\n+\n+I will respond to all communication daily and will be available throughout \n+the duration of the program. Although I will be taking some summer courses \n+at my university, I will not be enrolled in a typical full course load. As \n+part of GSOC, I plan to commit to 175 hours. I have experience managing my \n+time effectively while taking courses and working full-time internships in \n+the past. My semester ends on August 15th, and I have no commitments for the \n+following month, which allows me to continue working beyond the end of the \n+semester. With the flexibility to extend the timeline of GSOC, I am confident \n+that I will have ample time to complete the project. I have already discussed \n+this with Victoria, the mentor for the project, and she has agreed to extend \n+the deadline until October 2nd, if necessary. After August 15th, I will be \n+able to work at least 8 hours per day, totaling ~360 hours of work until the \n+October 2nd deadline. This exceeds the required commitment of 175 hours, \n+ensuring that I will complete the project on time. Additionally, I am hoping \n+to continue working on the project even after GSOC ends. \n+\n+# After GSoC\n+\n+I recognize the value of having our GSoC participants continue to engage with \n+our community beyond the event. This is why I am committed to doing so myself. \n+Participating in open-source projects, especially with a community that supports \n+a widely-used development tool, is not only cool but also offers an opportunity \n+to learn and grow. By continuing to participate in this community, I believe \n+that I can make important contributions and continue to develop my skills.\n+\n+I am planning to establish an open source club at my university in the near \n+future. The University of Waterloo is known for its strong emphasis on \n+computer science and engineering, earning it the nickname \"MIT of the North.\" \n+Given this, I believe that there will be a great deal of interest in the club \n+for a variety of reasons. Currently, there is another club called Blueprint \n+that provides a valuable opportunity for real-world development experience \n+through developing software products for charities. However, the entry process \n+for this club is extremely competitive. By contrast, I think that an open source \n+club would offer a similar experience but with a lower barrier to entry, thus \n+making it accessible to more motivated students. Additionally, given the \n+widespread use and vibrant community of Git, I plan to direct students to this \n+community and am confident that many will be interested in contributing to its \n+open source projects.\n+\n+# Some Credits to Myself\n+\n+I’ve previously completed three software developer internships and worked \n+with small startups to large sized companies. I am currently interning \n+with Morgan Stanley and am on the architecture team, working on a large \n+scale equity management software. \n+\n+I'm interested in open source development as a way to give back to the \n+community while also growing as a developer. My background in C programming \n+language has made me particularly interested in contributing to Git, which \n+is primarily written in C. I am also comfortable with concepts like memory \n+allocation, thanks to my experience with C programming. Furthermore, I have \n+studied shell scripting as part of my coursework, which makes me well-equipped\n+to handle the project's language requirements. Another personal motivation \n+for contributing to this project is that I have worked with monorepos before, \n+and given that it is used by many of the larger tech companies, I want to \n+learn more about it and help improve the user experience with it.\n+\n+Victoria mentioned that I was the first person to express interest in the \n+project this year, either directly or via the mailing list. In my spare time, \n+I've been contributing and reading documents while also working a full-time \n+job (internship) and taking one course at my university. I expect to have a \n+lot more time next term, so you can expect even more from me ;). Nonetheless, \n+I became familiar and comfortable with the contribution process by writing, \n+responding to, and auditing various types of patches in the community.\n+\n+With the patches I have submitted so far, I have been able to develop a deeper \n+understanding of Git internals, project structures, commonly used APIs, test \n+suites, required tech stacks, and coding guidelines. To further enhance my \n+comprehension of Git, I have either read or skimmed through several relevant \n+documents, including 'Submitting patches', 'Coding guidelines', \n+'Myfirstcontribution.txt', 'Git tutorial', 'Git everyday', 'readme', \n+'Hacking Git', drawing upon my prior knowledge where applicable. Additionally, \n+I have been referring to the book 'Pro Git' on an as-needed basis. Furthermore, \n+I have thoroughly read and referenced blogs such as 'Make your monorepo feel \n+small with Git's sparse index [12]', 'Bring your monorepo down to size with \n+sparse-checkout [13]', and 'Commits are snapshots, not diffs [14]'. The \n+advantage of having prior knowledge and experience with my proposed project \n+idea is that I am well-prepared to tackle any upcoming challenges.\n+\n+[12]: https://github.blog/2021-11-10-make-your-monorepo-feel-small-with-gits-sparse-index/\n+[13]: https://github.blog/2020-01-17-bring-your-monorepo-down-to-size-with-sparse-checkout/\n+[14]: https://github.blog/2020-12-17-commits-are-snapshots-not-diffs/\n+\n+# Closing remarks\n+\n+I am very motivated for this project because I have previously worked with \n+monorepos and will most likely have to work with them again in my future \n+internships. As a result, I intend to continue working on remaining c\n+ommands after GSOC whenever I have free time. \n+\n+I'd like to state that I'm a genuinely enthusiastic open-source newcomer \n+who is very much looking forward to this opportunity. I am grateful for \n+the opportunity to contribute to Git's development, and I am committed to \n+working diligently to strengthen the open-source ecosystem. My ultimate goal \n+is to use this opportunity to bring new energy and ideas to the table, and to \n+make meaningful contributions that benefit the entire community.\n+\n+I am grateful for the community's support, especially Victoria's guidance \n+and feedback. She promptly replied to my inquiries and provided me with \n+several resources that were instrumental in helping me get started on the \n+project. I am truly humbled by the dedication and hard work that the \n+community puts in to nurture and enhance this ecosystem, and I feel \n+fortunate to have received such warm and welcoming support as a new \n+contributor. It is an honor to be a part of this community and to \n+work towards advancing its mission.\n+\n+Thank you so much for reading through my proposal!\n+\n+Kind Regards,\n+Vivan Garg\n+\n-- \n2.37.0 (Apple Git-136)\n\n"},{"id":"472775","messageId":"CACmM78TrE8wG88aRy+H4qzDB08fuH3XGk1vmC47yXPcKkL8u8w@mail.gmail.com","threadId":"59315","inReplyTo":"20230227003517.88254-1-gvivan6@gmail.com","subject":"Re: [RFC][PATCH v2] GSoC 2023 proposal: more sparse index integration","fromName":"Ashutosh Pandey","fromEmail":"ashutosh.pandeyhlr007@gmail.com","sentAt":"2023-02-27T05:29:41Z","receivedAt":"2023-02-27T05:29:58Z","isPatch":true,"sender":{"key":"ashutosh.pandeyhlr007@gmail.com","avatar":"https://avatars.githubusercontent.com/u/57610394?v=4"},"body":"On Mon, Feb 27, 2023 at 6:23 AM Vivan Garg <gvivan6@gmail.com> wrote:\n>\n> Signed-off-by: Vivan Garg <gvivan6@gmail.com>\n> ---\n>  .../More-Sparse-Index-Integrations.txt        | 312 ++++++++++++++++++\n>  1 file changed, 312 insertions(+)\n>  create mode 100644 Documentation/More-Sparse-Index-Integrations.txt\n>\n> diff --git a/Documentation/More-Sparse-Index-Integrations.txt b/Documentation/More-Sparse-Index-Integrations.txt\n> new file mode 100644\n> index 0000000000..2ab6b07f18\n> --- /dev/null\n> +++ b/Documentation/More-Sparse-Index-Integrations.txt\n> @@ -0,0 +1,312 @@\n> +# More Sparse Index Integrations\n> +\n> +# Personal Information\n> +\n> +Full name: Vivan Garg\n> +\n> +E-mail: gvivan6@gmail.com\n> +Alternate E-mail: v.garg.work@gmail.com\n> +Tel: (+1)437-987-2678\n> +\n> +Education: University of Waterloo (Canada)\n> +Major: Computer Science and Financial Management (Double-Major)\n> +Year: Rising Junior\n> +\n> +LinkedIn: https://www.linkedin.com/in/gvivan/\n> +GitHub: https://github.com/gvivan\n> +Website: https://gvivan.me/\n> +\n> +# Before GSoC\n> +\n> +## Synopsis\n> +\n> +I've chosen the \"More Sparse Index Integrations\" project idea from the\n> +SoC 2023 Ideas page. The goal of this project is to integrate the\n> +experimental \"sparse-index\" feature and \"sparse-checkout\" command with\n> +existing Git commands.\n\nShaoxuan Yuan(GSoC'22) also worked on the same project you should also\nCc him.\n\n> +\n> +Git 2.25.0 introduced a new experimental `git sparse-checkout` command,\n> +which simplified the existing feature and improved performance for\n> +large repositories. It allows users to restrict their working directory\n> +to only the files they care about, allowing them to ensure the developer\n> +workflow is as fast as possible while maintaining all the benefits of a\n> +monorepo.\n> +(Bring your monorepo down to size with sparse-checkout [1], Stolee).\n> +\n> +The pattern matching process in Git's sparse-checkout feature becomes\n> +expensive as the sparse-checkout file and repository size increase,\n> +growing quadratically. This can result in billions of pattern checks\n> +for large repositories. However, Git's new mechanism for matching based\n> +on folder prefix matches drops the quadratic growth, matching M patterns\n> +across N files in O(M+N*d) time, where d is the maximum folder depth of a file.\n> +To further optimize the matching process, Git inspects files in a sorted\n> +order instead of an arbitrary order. When Git evaluates a file path, it\n> +checks whether the start of the folder path matches a recursive pattern exactly.\n> +If so, it marks everything in that folder as \"included\" without doing any further\n> +hashset lookups. Similarly, when Git detects the start of a folder that's outside\n> +of the specified cone, it marks everything in that folder as \"excluded\" without\n> +doing any further hashset lookups. This reduces the time to be closer to O(M+N)\n> +(Bring your monorepo down to size with sparse-checkout [1], Stolee).\n> +\n> +[1]: https://github.blog/2020-01-17-bring-your-monorepo-down-to-size-with-sparse-checkout/\n\nall your references should be listed at the bottom in the order in\nwhich they are referenced\nabove not after the end of a paragraph.\n\n> +\n> +The Git Fundamentals team at GitHub has contributed a new feature to Git called\n> +the sparse index, which allows the index to focus on the files within the\n> +sparse-checkout cone in a monorepo. The sparse index stores only the information\n> +about the files within the sparse-checkout definition, instead of storing information\n> +for every file at HEAD, which can make the index much larger in a monorepo. When\n> +enabled with other performance features, the sparse index can have a significant\n> +impact on performance (Make your monorepo feel small with Git’s sparse index [2], Stolee).\n> +\n> +[2]: https://github.blog/2021-11-10-make-your-monorepo-feel-small-with-gits-sparse-index/\n> +\n> +The sparse index differs from a normal \"full\" index in that it can store directory\n> +paths with the object ID for its tree object. It can be used to determine if an\n> +entire directory is out of the sparse-checkout cone and replace all of its contained\n> +file paths with a single directory path. The use of sparse index can significantly\n> +reduce the size of the index, resulting in faster operations\n> +(Make your monorepo feel small with Git’s sparse index [3], Stolee).\n> +\n> +[3]: https://github.blog/2021-11-10-make-your-monorepo-feel-small-with-gits-sparse-index/\n> +\n> +Because \"sparse-checkout\" and \"sparse-index\" may potentially influence the logics of\n> +other Git commands and the internal data structure of Git, some work is required to\n> +optimize compatibility and user experience. That is exactly what my chosen idea proposed.\n> +\n> +## Benefits to Community\n> +\n> +By joining the community and working on this idea, I can collaborate with my mentor\n> +and fellow community members to improve the user experience for people who are working\n> +with large monorepos. Furthermore, I am committed to continuing my involvement beyond\n> +the GSoC program, not only by contributing to the community but also by sharing my\n> +experiences and mentoring future potential newcomers.\n> +\n> +\n> +## Microproject\n> +\n> +t4121: modernize test style [4]\n> +Status: WIP\n> +Description: Test scripts in file t4121-apply-diffs.sh are written in old style,\n> +where the test_expect_success command and test title are written on\n> +separate lines. Therefore update the tests to adhere to the new style.\n> +\n> +## Other Contributions\n> +\n> +### Reviewing\n> +\n> +t9700: modernize test script [5]\n> +Status: WIP\n> +Description: I reviewed this patch and pointed the contributor in the right direction\n> +by providing examples, links and mentioning the best practices.\n> +\n> +### Patches\n> +\n> +MyFirstContribution: add note about SMTP server config [6]\n> +Status: WIP\n> +Description: The documentation on using git-send-email previously mentioned the need\n> +to configure git for your operating system and email provider, but did not provide\n> +specific details on the relevant configuration settings. This commit adds a note\n> +specifying that the relevant settings can be found under the 'sendemail' section of\n> +Git's configuration file, with a link to the relevant documentation. The aim is to\n> +provide users with a more complete understanding of the configuration process and\n> +help them avoid potential roadblocks in setting up git-send-email.\n> +\n> +[4]: https://lore.kernel.org/git/CACzddJrZ8YdJ72ng3UpMGN9CJx0qW1+fZfyi3q01z2487V8fxw@mail.gmail.com/T/#md53157af31a3f347dd899679fafdea7fcaf7ecfc\n> +[5]: https://lore.kernel.org/git/CADupsJPpZnjA=Pu_RZZZXy7Titj3UD7ppww48KvcHHHbrGx=rw@mail.gmail.com/T/#m122db9bdca463c12f0b9ccb259fd1d3229d75945\n> +[6]: https://lore.kernel.org/git/20230222011317.97943-1-gvivan6@gmail.com/\n> +\n> +\n> +### Related Work\n> +\n> +Prior works on the idea have been completed by my mentors and other community members,\n> +and these works provide a good approximation of the approach I intend to take. Here\n> +are some previous examples of commits:\n> +\n> +Integration with “mv” [7]\n> +Integration with “reset” [8]\n> +Integration with “sparse-checkout” [9]\n> +Integration with “clean” [10]\n> +Integration with “blame” [11]\n> +\n> +[7]: https://lore.kernel.org/git/20220331091755.385961-1-shaoxuan.yuan02@gmail.com/\n> +[8]: https://lore.kernel.org/git/pull.1048.v6.git.1638201164.gitgitgadget@gmail.com/\n> +[9]: https://lore.kernel.org/git/pull.1208.v3.git.1653313726.gitgitgadget@gmail.com/\n> +[10]: https://github.com/git/git/commit/1e9e10e04891a13e5ccd52b36cfadc55dfaa5066\n> +[11]: https://github.com/git/git/commit/add4c864b60766174ad4f74ba7be17e66d61ef16\n> +\n> +# In GSoC\n> +\n> +## Plan\n> +\n> +The proposed idea of increasing \"sparse-index\" integrations may seem\n> +straightforward at first glance. However, upon reviewing previous\n> +implementations, I discovered that this idea can introduce unforeseen\n> +difficulties for some functions. For example, to enable \"sparse-index,\"\n> +we must ensure that \"sparse-checkout\" is compatible with the target Git\n> +command. Achieving this compatibility requires modifying the original\n> +command logic, which can lead to other unanticipated issues. Therefore,\n> +I have incorporated some additional steps in the plan outlined below to\n> +proactively address potential complications. It's worth noting that\n> +points 3-7 are part of the SoC 2023 Ideas proposed by the community\n> +and mentors.\n> +\n> +1. Conduct an investigation to determine if a Git command functions\n> +properly with sparse-checkout.\n> +\n> +2. Modify the logic of the Git command, if necessary, to ensure it\n> +functions properly with sparse-checkout. Develop corresponding tests\n> +to validate the modifications.\n> +\n> +3. Add tests to t1092-sparse-checkout-compatibility.sh for the\n> +builtin, with a focus on what happens for paths outside of the\n> +sparse-checkout cone.\n> +\n> +4. Disable the command_requires_full_index setting in the builtin\n> +and ensure the tests pass.\n> +\n> +5. If the tests do not pass, then alter the logic to work with the\n> +sparse index.\n> +\n> +6. Add tests to check that a sparse index stays sparse.\n> +\n> +7. Add performance tests to demonstrate speedup.\n> +\n> +8. If any changes are made that affect the behavior of the Git\n> +command, update the documentation accordingly. Note that such\n> +changes should be rare.\n\nas was pointed out by Victoria high-level understanding won't suffice for a\nthe good proposal you should read the documentation about sparse and\nalso, read Shaoxuan Yuan's blogs to get some technical understanding and\ntake a look at some of his code  and commit messages where he explains the\nchanges.\n\n> +\n> +## Timeline\n> +\n> +During my discussion with Victoria, she informed me that given my\n> +commitment of 175 hours, it is expected that I will be able to fully\n> +integrate two commands with sparse index during the GSOC program. My\n> +plan is to evenly distribute the work for each command over the course\n> +of the program. I am confident that I can start the project early as I\n> +have already established communication with my mentors and familiarized\n> +myself with the related documentation, although my understanding may\n> +not be comprehensive.\n> +\n> +Based on my prior experience with the idea, I believe I will be able\n> +to quickly get up to speed and begin working on the project. The exact\n> +timeline for each integration is difficult to determine, but I estimate\n> +that I should be able to complete one integration every two months. I\n> +have already planned out my next term, and there are only three weeks\n> +during which I would prefer to focus on other things: June 23-30 and\n> +August 1-15. However, even without an extension, I should be able to\n> +manage this timeline. With the flexibility to extend the program, it\n> +should be even easier to accommodate any potential scheduling conflicts.\n> +\n> +\n> +## Availability\n> +\n> +I will respond to all communication daily and will be available throughout\n> +the duration of the program. Although I will be taking some summer courses\n> +at my university, I will not be enrolled in a typical full course load. As\n> +part of GSOC, I plan to commit to 175 hours. I have experience managing my\n> +time effectively while taking courses and working full-time internships in\n> +the past. My semester ends on August 15th, and I have no commitments for the\n> +following month, which allows me to continue working beyond the end of the\n> +semester. With the flexibility to extend the timeline of GSOC, I am confident\n> +that I will have ample time to complete the project. I have already discussed\n> +this with Victoria, the mentor for the project, and she has agreed to extend\n> +the deadline until October 2nd, if necessary. After August 15th, I will be\n> +able to work at least 8 hours per day, totaling ~360 hours of work until the\n> +October 2nd deadline. This exceeds the required commitment of 175 hours,\n> +ensuring that I will complete the project on time. Additionally, I am hoping\n> +to continue working on the project even after GSOC ends.\n> +\n> +# After GSoC\n> +\n> +I recognize the value of having our GSoC participants continue to engage with\n> +our community beyond the event. This is why I am committed to doing so myself.\n> +Participating in open-source projects, especially with a community that supports\n> +a widely-used development tool, is not only cool but also offers an opportunity\n> +to learn and grow. By continuing to participate in this community, I believe\n> +that I can make important contributions and continue to develop my skills.\n> +\n> +I am planning to establish an open source club at my university in the near\n> +future. The University of Waterloo is known for its strong emphasis on\n> +computer science and engineering, earning it the nickname \"MIT of the North.\"\n> +Given this, I believe that there will be a great deal of interest in the club\n> +for a variety of reasons. Currently, there is another club called Blueprint\n> +that provides a valuable opportunity for real-world development experience\n> +through developing software products for charities. However, the entry process\n> +for this club is extremely competitive. By contrast, I think that an open source\n> +club would offer a similar experience but with a lower barrier to entry, thus\n> +making it accessible to more motivated students. Additionally, given the\n> +widespread use and vibrant community of Git, I plan to direct students to this\n> +community and am confident that many will be interested in contributing to its\n> +open source projects.\n> +\n> +# Some Credits to Myself\n> +\n> +I’ve previously completed three software developer internships and worked\n> +with small startups to large sized companies. I am currently interning\n> +with Morgan Stanley and am on the architecture team, working on a large\n> +scale equity management software.\n> +\n> +I'm interested in open source development as a way to give back to the\n> +community while also growing as a developer. My background in C programming\n> +language has made me particularly interested in contributing to Git, which\n> +is primarily written in C. I am also comfortable with concepts like memory\n> +allocation, thanks to my experience with C programming. Furthermore, I have\n> +studied shell scripting as part of my coursework, which makes me well-equipped\n> +to handle the project's language requirements. Another personal motivation\n> +for contributing to this project is that I have worked with monorepos before,\n> +and given that it is used by many of the larger tech companies, I want to\n> +learn more about it and help improve the user experience with it.\n> +\n> +Victoria mentioned that I was the first person to express interest in the\n> +project this year, either directly or via the mailing list. In my spare time,\n> +I've been contributing and reading documents while also working a full-time\n> +job (internship) and taking one course at my university. I expect to have a\n> +lot more time next term, so you can expect even more from me ;). Nonetheless,\n> +I became familiar and comfortable with the contribution process by writing,\n> +responding to, and auditing various types of patches in the community.\n> +\n> +With the patches I have submitted so far, I have been able to develop a deeper\n> +understanding of Git internals, project structures, commonly used APIs, test\n> +suites, required tech stacks, and coding guidelines. To further enhance my\n> +comprehension of Git, I have either read or skimmed through several relevant\n> +documents, including 'Submitting patches', 'Coding guidelines',\n> +'Myfirstcontribution.txt', 'Git tutorial', 'Git everyday', 'readme',\n> +'Hacking Git', drawing upon my prior knowledge where applicable. Additionally,\n> +I have been referring to the book 'Pro Git' on an as-needed basis. Furthermore,\n> +I have thoroughly read and referenced blogs such as 'Make your monorepo feel\n> +small with Git's sparse index [12]', 'Bring your monorepo down to size with\n> +sparse-checkout [13]', and 'Commits are snapshots, not diffs [14]'. The\n> +advantage of having prior knowledge and experience with my proposed project\n> +idea is that I am well-prepared to tackle any upcoming challenges.\n> +\n> +[12]: https://github.blog/2021-11-10-make-your-monorepo-feel-small-with-gits-sparse-index/\n> +[13]: https://github.blog/2020-01-17-bring-your-monorepo-down-to-size-with-sparse-checkout/\n> +[14]: https://github.blog/2020-12-17-commits-are-snapshots-not-diffs/\n> +\n> +# Closing remarks\n> +\n> +I am very motivated for this project because I have previously worked with\n> +monorepos and will most likely have to work with them again in my future\n> +internships. As a result, I intend to continue working on remaining c\n> +ommands after GSOC whenever I have free time.\n> +\n> +I'd like to state that I'm a genuinely enthusiastic open-source newcomer\n> +who is very much looking forward to this opportunity. I am grateful for\n> +the opportunity to contribute to Git's development, and I am committed to\n> +working diligently to strengthen the open-source ecosystem. My ultimate goal\n> +is to use this opportunity to bring new energy and ideas to the table, and to\n> +make meaningful contributions that benefit the entire community.\n> +\n> +I am grateful for the community's support, especially Victoria's guidance\n> +and feedback. She promptly replied to my inquiries and provided me with\n> +several resources that were instrumental in helping me get started on the\n> +project. I am truly humbled by the dedication and hard work that the\n> +community puts in to nurture and enhance this ecosystem, and I feel\n> +fortunate to have received such warm and welcoming support as a new\n> +contributor. It is an honor to be a part of this community and to\n> +work towards advancing its mission.\n> +\n> +Thank you so much for reading through my proposal!\n> +\n> +Kind Regards,\n> +Vivan Garg\n> +\n> --\n> 2.37.0 (Apple Git-136)\n>\n"}]}