{"thread":{"id":"57410","subject":"[GSoC] Interest in \"More Sparse Index Integrations\" project idea","startedAt":"2022-02-14T12:44:25Z","lastAt":"2022-02-16T03:01:41Z","messageCount":6,"participants":["Shaoxuan Yuan","Derrick Stolee","Philippe Blain"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"448354","messageId":"CAJyCBOTuNn24_WvHv-xQsCdtr9-4_fdrMrOC7ZApg1xFp4RNWQ@mail.gmail.com","threadId":"57410","inReplyTo":null,"subject":"[GSoC] Interest in \"More Sparse Index Integrations\" project idea","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-02-14T12:44:06Z","receivedAt":"2022-02-14T12:44:25Z","isPatch":false,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Hi Stolee, Victoria, and Community,\n\nThe self-intro was done during my first practice patch [1].\n\nI've been paying attention to the updates to the SoC 2022 ideas\npage. And find your sparse index integrations project quite interesting.\n\nIt took me around an hour to read through this blog [2] written by\nStolee, and I felt\nthat although it's a relatively new and quite cool idea (seems a bit daunting),\nI think now I have some basic grasp of it (just a distant big idea).\n\nThe most amazing part is the drastic improvement of speed when\nsparse index is used on huge monorepo (not sure but sometimes \"git\nstatus\" on git's\nrepo can make me feel slight lag). I can see this efficiency boost\nis beneficial for almost everyone, especially those who operate on\nhuge monorepo.\n\nAnd I think what makes me feel excited is that it actually connects to\nmany topics that I want to learn as a newbie to git community,\ne.g. index, the new ORT merge strategy by Elijah (I didn't know the\ndefault merge strategy is replaced if not for reading the blog),\nand possibly many other commands' underlying magic (guess because\nI can have the chance to integrate them with sparse index and learn more\nat the same time).\n\nGenerally speaking, I feel that working on and researching about\nthe project opens up the gate to dive deeper into git and meet tons of fun\n(I just finished skimming through the Pro Git book and can barely understand\nwhat the blog is talking about, can't wait to see them in practice). Plus\nit is also a good idea for a beginner to walk on a \"decent paved-path\", quoted\nfrom Stolee.\n\nAnd the main purpose of this message is to first, show my interest\nabout this project idea and second, use it as a thread so I can follow up\nsome questions during the research process.\n\nQuestion: It is a good idea to start looking into the commands listed\nin the idea doc [3] and make some practice patches about sparse index\nintegrations?\n\n[1] https://lore.kernel.org/git/20220123060318.471414-1-shaoxuan.yuan02@gmail.com/\n[2] https://github.blog/2021-11-10-make-your-monorepo-feel-small-with-gits-sparse-index/\n[3] https://git.github.io/SoC-2022-Ideas/#more-sparse-index-integrations\n\n-- \nThanks & Regards,\nShaoxuan\n"},{"id":"448356","messageId":"d6286f84-642d-7448-0d2e-dd4c20bdb289@gmail.com","threadId":"57410","inReplyTo":"CAJyCBOTuNn24_WvHv-xQsCdtr9-4_fdrMrOC7ZApg1xFp4RNWQ@mail.gmail.com","subject":"Re: [GSoC] Interest in \"More Sparse Index Integrations\" project idea","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2022-02-14T14:38:28Z","receivedAt":"2022-02-14T14:38:37Z","isPatch":false,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 2/14/2022 7:44 AM, Shaoxuan Yuan wrote:\n> Hi Stolee, Victoria, and Community,\n> \n> The self-intro was done during my first practice patch [1].\n\nWelcome!\n \n> I've been paying attention to the updates to the SoC 2022 ideas\n> page. And find your sparse index integrations project quite interesting.\n\n...\n\n> And the main purpose of this message is to first, show my interest\n> about this project idea and second, use it as a thread so I can follow up\n> some questions during the research process.\n> \n> Question: It is a good idea to start looking into the commands listed\n> in the idea doc [3] and make some practice patches about sparse index\n> integrations?\n> \n> [1] https://lore.kernel.org/git/20220123060318.471414-1-shaoxuan.yuan02@gmail.com/\n> [2] https://github.blog/2021-11-10-make-your-monorepo-feel-small-with-gits-sparse-index/\n> [3] https://git.github.io/SoC-2022-Ideas/#more-sparse-index-integrations\n \nOne thing you can do is look at the history of t1092, which is used by\nevery patch series that performs these integrations.\n\n$ git log -20 --oneline --graph --show-pulls origin/master -- t/t1092-sparse-checkout-compatibility.sh\n* 3c0e4178277 Merge branch 'ds/fetch-pull-with-sparse-index'\n* 3a9a6ac5d9d t1092: replace 'read-cache --table' with 'ls-files --sparse'\n* 78087097b8b ls-files: add --sparse option\n* 5a4e0547e23 fetch/pull: use the sparse index\n*   8d2c37320bd Merge branch 'ld/sparse-diff-blame'\n|\\  \n| * add4c864b60 blame: enable and test the sparse index\n| * 51ba65b5c35 diff: enable and test the sparse index\n| * 338e2a9acc8 diff: replace --staged with --cached in t1092 tests\n| *   ea6ae410bed Merge branch 'vd/sparse-reset' into ld/sparse-diff-blame\n| |\\  \n* | \\   e1d92882676 Merge branch 'ds/sparse-deep-pattern-checkout-fix'\n|\\ \\ \\  \n| * | | 8c5de0d2657 unpack-trees: use traverse_path instead of name\n| * | | 1b38efc7a03 t1092: add deeper changes during a checkout\n* | | |   62329d336f5 Merge branch 'ds/trace2-regions-in-tests'\n|\\ \\ \\ \\  \n| * | | | 8c4cbad6a3a t/t*: remove custom GIT_TRACE2_EVENT_NESTING\n| |/ / /  \n* | | |   f0850875fd4 Merge branch 'vd/sparse-reset'\n|\\ \\ \\ \\  \n| | |_|/  \n| |/| |   \n| * | | 4d1cfc1351f reset: make --mixed sparse-aware\n| * | | 20ec2d034cd reset: make sparse-aware (except --mixed)\n| * | | 291d77eb3e2 reset: expand test coverage for sparse checkouts\n| * | | 86609db9dae sparse-index: update command for expand/collapse test\n| * | | 71471b2a7c7 reset: preserve skip-worktree bit in mixed reset\n\nBut of course, this doesn't show the full history of the work in\neach of these branches, so you can do a history walk from each\nmerge to get a picture of the full series:\n\n$ git log --oneline --graph 3c0e4178277 -6\n*   3c0e4178277 Merge branch 'ds/fetch-pull-with-sparse-index'\n|\\  \n| * 408c51f0b4b test-read-cache: remove --table, --expand options\n| * c2a29405105 t1091/t3705: remove 'test-tool read-cache --table'\n| * 3a9a6ac5d9d t1092: replace 'read-cache --table' with 'ls-files --sparse'\n| * 78087097b8b ls-files: add --sparse option\n| * 5a4e0547e23 fetch/pull: use the sparse index\n\n$ git log --oneline --graph 8d2c37320bd -8\n*   8d2c37320bd Merge branch 'ld/sparse-diff-blame'\n|\\  \n| * add4c864b60 blame: enable and test the sparse index\n| * 51ba65b5c35 diff: enable and test the sparse index\n| * 338e2a9acc8 diff: replace --staged with --cached in t1092 tests\n| * 44c7e62e51e repo-settings: prepare_repo_settings only in git repos\n| * 27a443b8208 test-read-cache: set up repo after git directory\n| * 0803f9c7cdf commit-graph: return if there is no git directory\n| * e5b17bda8b6 git: ensure correct git directory setup with -h\n\nThese are just a couple examples of merged work in this area. Be\nsure to look at the full commit messages and diffs to get a feel\nfor the area. Then, playing around with the list of commands in\nthe project idea document is a good idea.\n\nThere is also a series that is currently waiting to merge [4], for\neven more examples.\n\n[4] https://lore.kernel.org/git/pull.1109.v2.git.1641924306.gitgitgadget@gmail.com/\n\nThanks,\n-Stolee\n"},{"id":"448383","messageId":"e81f044d-8c3a-90df-ff9e-719b613807d6@gmail.com","threadId":"57410","inReplyTo":"d6286f84-642d-7448-0d2e-dd4c20bdb289@gmail.com","subject":"Re: [GSoC] Interest in \"More Sparse Index Integrations\" project idea","fromName":"Philippe Blain","fromEmail":"levraiphilippeblain@gmail.com","sentAt":"2022-02-14T18:45:16Z","receivedAt":"2022-02-14T18:47:18Z","isPatch":false,"sender":{"key":"levraiphilippeblain@gmail.com","avatar":"https://avatars.githubusercontent.com/u/44212482?v=4"},"body":"\n> But of course, this doesn't show the full history of the work in\n> each of these branches, so you can do a history walk from each\n> merge to get a picture of the full series:\n> \n> $ git log --oneline --graph 3c0e4178277 -6\n> *   3c0e4178277 Merge branch 'ds/fetch-pull-with-sparse-index'\n> |\\  \n\nOr even better (you do not have to guess the number of commits in \neach series):\n\ngit log --oneline --graph 3c0e4178277^-\n\n(see [1]). Why that syntax seems to completely ignore '--graph',\nthough is beyond me!\n\nPhilippe.\n\n[1] https://git-scm.com/docs/gitrevisions#Documentation/gitrevisions.txt-emltrevgt-ltngtemegemHEAD-HEAD-2em\n"},{"id":"448399","messageId":"CAJyCBOSnJNFUV3CWBaBXLw521TLCHifZcrpCro3fPmw41YheoA@mail.gmail.com","threadId":"57410","inReplyTo":"d6286f84-642d-7448-0d2e-dd4c20bdb289@gmail.com","subject":"Re: [GSoC] Interest in \"More Sparse Index Integrations\" project idea","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-02-15T02:29:09Z","receivedAt":"2022-02-15T02:29:24Z","isPatch":false,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"On Mon, Feb 14, 2022 at 10:38 PM Derrick Stolee <stolee@gmail.com> wrote:\n>\n> On 2/14/2022 7:44 AM, Shaoxuan Yuan wrote:\n> > Hi Stolee, Victoria, and Community,\n> >\n> > The self-intro was done during my first practice patch [1].\n>\n> Welcome!\n>\n> > I've been paying attention to the updates to the SoC 2022 ideas\n> > page. And find your sparse index integrations project quite interesting.\n>\n> ...\n>\n> > And the main purpose of this message is to first, show my interest\n> > about this project idea and second, use it as a thread so I can follow up\n> > some questions during the research process.\n> >\n> > Question: It is a good idea to start looking into the commands listed\n> > in the idea doc [3] and make some practice patches about sparse index\n> > integrations?\n> >\n> > [1] https://lore.kernel.org/git/20220123060318.471414-1-shaoxuan.yuan02@gmail.com/\n> > [2] https://github.blog/2021-11-10-make-your-monorepo-feel-small-with-gits-sparse-index/\n> > [3] https://git.github.io/SoC-2022-Ideas/#more-sparse-index-integrations\n>\n> One thing you can do is look at the history of t1092, which is used by\n> every patch series that performs these integrations.\n>\n> $ git log -20 --oneline --graph --show-pulls origin/master -- t/t1092-sparse-checkout-compatibility.sh\n> * 3c0e4178277 Merge branch 'ds/fetch-pull-with-sparse-index'\n> * 3a9a6ac5d9d t1092: replace 'read-cache --table' with 'ls-files --sparse'\n> * 78087097b8b ls-files: add --sparse option\n> * 5a4e0547e23 fetch/pull: use the sparse index\n> *   8d2c37320bd Merge branch 'ld/sparse-diff-blame'\n> |\\\n> | * add4c864b60 blame: enable and test the sparse index\n> | * 51ba65b5c35 diff: enable and test the sparse index\n> | * 338e2a9acc8 diff: replace --staged with --cached in t1092 tests\n> | *   ea6ae410bed Merge branch 'vd/sparse-reset' into ld/sparse-diff-blame\n> | |\\\n> * | \\   e1d92882676 Merge branch 'ds/sparse-deep-pattern-checkout-fix'\n> |\\ \\ \\\n> | * | | 8c5de0d2657 unpack-trees: use traverse_path instead of name\n> | * | | 1b38efc7a03 t1092: add deeper changes during a checkout\n> * | | |   62329d336f5 Merge branch 'ds/trace2-regions-in-tests'\n> |\\ \\ \\ \\\n> | * | | | 8c4cbad6a3a t/t*: remove custom GIT_TRACE2_EVENT_NESTING\n> | |/ / /\n> * | | |   f0850875fd4 Merge branch 'vd/sparse-reset'\n> |\\ \\ \\ \\\n> | | |_|/\n> | |/| |\n> | * | | 4d1cfc1351f reset: make --mixed sparse-aware\n> | * | | 20ec2d034cd reset: make sparse-aware (except --mixed)\n> | * | | 291d77eb3e2 reset: expand test coverage for sparse checkouts\n> | * | | 86609db9dae sparse-index: update command for expand/collapse test\n> | * | | 71471b2a7c7 reset: preserve skip-worktree bit in mixed reset\n>\n> But of course, this doesn't show the full history of the work in\n> each of these branches, so you can do a history walk from each\n> merge to get a picture of the full series:\n>\n> $ git log --oneline --graph 3c0e4178277 -6\n> *   3c0e4178277 Merge branch 'ds/fetch-pull-with-sparse-index'\n> |\\\n> | * 408c51f0b4b test-read-cache: remove --table, --expand options\n> | * c2a29405105 t1091/t3705: remove 'test-tool read-cache --table'\n> | * 3a9a6ac5d9d t1092: replace 'read-cache --table' with 'ls-files --sparse'\n> | * 78087097b8b ls-files: add --sparse option\n> | * 5a4e0547e23 fetch/pull: use the sparse index\n>\n> $ git log --oneline --graph 8d2c37320bd -8\n> *   8d2c37320bd Merge branch 'ld/sparse-diff-blame'\n> |\\\n> | * add4c864b60 blame: enable and test the sparse index\n> | * 51ba65b5c35 diff: enable and test the sparse index\n> | * 338e2a9acc8 diff: replace --staged with --cached in t1092 tests\n> | * 44c7e62e51e repo-settings: prepare_repo_settings only in git repos\n> | * 27a443b8208 test-read-cache: set up repo after git directory\n> | * 0803f9c7cdf commit-graph: return if there is no git directory\n> | * e5b17bda8b6 git: ensure correct git directory setup with -h\n>\n> These are just a couple examples of merged work in this area. Be\n> sure to look at the full commit messages and diffs to get a feel\n> for the area. Then, playing around with the list of commands in\n> the project idea document is a good idea.\n\nThanks for the examples and suggestions! Working on them :)\n\n>\n> There is also a series that is currently waiting to merge [4], for\n> even more examples.\n>\n> [4] https://lore.kernel.org/git/pull.1109.v2.git.1641924306.gitgitgadget@gmail.com/\n>\n> Thanks,\n> -Stolee\n\n-- \nThanks & Regards,\nShaoxuan\n"},{"id":"448400","messageId":"CAJyCBOQbreyTY7NxvkU3e06KLh+HXe=tHtDcoN=iFmSJ9H4Tnw@mail.gmail.com","threadId":"57410","inReplyTo":"e81f044d-8c3a-90df-ff9e-719b613807d6@gmail.com","subject":"Re: [GSoC] Interest in \"More Sparse Index Integrations\" project idea","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-02-15T02:29:54Z","receivedAt":"2022-02-15T02:30:12Z","isPatch":false,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"On Tue, Feb 15, 2022 at 2:45 AM Philippe Blain\n<levraiphilippeblain@gmail.com> wrote:\n>\n>\n> > But of course, this doesn't show the full history of the work in\n> > each of these branches, so you can do a history walk from each\n> > merge to get a picture of the full series:\n> >\n> > $ git log --oneline --graph 3c0e4178277 -6\n> > *   3c0e4178277 Merge branch 'ds/fetch-pull-with-sparse-index'\n> > |\\\n>\n> Or even better (you do not have to guess the number of commits in\n> each series):\n>\n> git log --oneline --graph 3c0e4178277^-\n>\n> (see [1]). Why that syntax seems to completely ignore '--graph',\n> though is beyond me!\n>\n> Philippe.\n>\n> [1] https://git-scm.com/docs/gitrevisions#Documentation/gitrevisions.txt-emltrevgt-ltngtemegemHEAD-HEAD-2em\n\nThanks ;-)\n\n-- \nThanks & Regards,\nShaoxuan\n"},{"id":"448532","messageId":"CAJyCBOSPqP4Rh5ZQhj-DexXXtkwCJTT3tL7DToREfsRR6Hozxg@mail.gmail.com","threadId":"57410","inReplyTo":"e81f044d-8c3a-90df-ff9e-719b613807d6@gmail.com","subject":"Re: [GSoC] Interest in \"More Sparse Index Integrations\" project idea","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-02-16T03:01:24Z","receivedAt":"2022-02-16T03:01:41Z","isPatch":false,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"On Tue, Feb 15, 2022 at 2:45 AM Philippe Blain\n<levraiphilippeblain@gmail.com> wrote:\n> Or even better (you do not have to guess the number of commits in\n> each series):\n>\n> git log --oneline --graph 3c0e4178277^-\n>\n> (see [1]). Why that syntax seems to completely ignore '--graph',\n> though is beyond me!\n\nI read the [1] and this \"<rev>^-\" syntax seems to exclude the first\nparent of <rev>.\nIn another word, it only gets the part of history within the checked-out\nnew branch, i.e. \"This is typically useful for merge commits where you\ncan just pass <commit>^- to get all the commits in the branch that was\nmerged in merge commit <commit> (including <commit> itself).\"\n\nIt seems that the checked-out branch has a linear clean history,\nand that's why it looks like \"completely ignore '--graph'\". Actually, it does\nnot ignore \"--graph\", e.g.\n\nwith \"--graph\":\n\n* 3c0e417827 Merge branch 'ds/fetch-pull-with-sparse-index'\n* 408c51f0b4 test-read-cache: remove --table, --expand options\n* c2a2940510 t1091/t3705: remove 'test-tool read-cache --table'\n* 3a9a6ac5d9 t1092: replace 'read-cache --table' with 'ls-files --sparse'\n* 78087097b8 ls-files: add --sparse option\n* 5a4e0547e2 fetch/pull: use the sparse index\n\nw/out \"--graph\":\n\n3c0e417827 Merge branch 'ds/fetch-pull-with-sparse-index'\n408c51f0b4 test-read-cache: remove --table, --expand options\nc2a2940510 t1091/t3705: remove 'test-tool read-cache --table'\n3a9a6ac5d9 t1092: replace 'read-cache --table' with 'ls-files --sparse'\n78087097b8 ls-files: add --sparse option\n5a4e0547e2 fetch/pull: use the sparse index\n\nLook at the asterisks prefixing every commit in the with \"--graph\" part.\nIf you init a new repo and add a few commits to form a linear history,\nyou will get a similar result by \"git log --oneline --graph\".\n\n> [1] https://git-scm.com/docs/gitrevisions#Documentation/gitrevisions.txt-emltrevgt-ltngtemegemHEAD-HEAD-2em\n\n-- \nThanks & Regards,\nShaoxuan\n"}]}