{"thread":{"id":"45383","subject":"GSoC Project | Submodules related work","startedAt":"2017-03-15T09:34:02Z","lastAt":"2017-03-15T22:13:12Z","messageCount":4,"participants":["Prathamesh Chavan","Stefan Beller","Prathamesh"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"314127","messageId":"CAME+mvW1x6fnGKt1_auGOp+wFYFR=Y_Qhxfd50E7KFe6t+X4kw@mail.gmail.com","threadId":"45383","inReplyTo":null,"subject":"GSoC Project | Submodules related work","fromName":"Prathamesh Chavan","fromEmail":"pc44800@gmail.com","sentAt":"2017-03-15T09:33:56Z","receivedAt":"2017-03-15T09:34:02Z","isPatch":false,"sender":{"key":"pc44800@gmail.com","avatar":"https://avatars.githubusercontent.com/u/17272661?v=4"},"body":"Hey everyone,\n\nI am Prathamesh. I am studying Computer Science and Engineering at IIT\nKharagpur. I am interested to participate in Google Summer of Code 2017\nunder Git organization. I attempted \"Avoid pipes in git related commands\nfor test suite\" as my microproject[1].\n\nAs a part of GSoC, I would like to work on git submodules. The projects I\nhave looked up are:\n        1. \"git -C sub add .\" might behave just like \"git add sub\"\n        2. Teach \"git -C <submodule-path> status\" in an unpopulated\n           submodule to report the submodule being unpopulated, do not\n           fall back to the superproject.\n        3. Teach \"git log -- <path/into/submodule/and/further>\" to behave\n           like \"git -C <path/into/submodule> log -- <and/further>\"\n\nI went through the series of mail (related to projects 1 and 2)[2] for\ngetting a better picture of the projects. I think as the projects are\nquite interrelated together, these may make a complete GSoC project.\n\nAlso the conclusions which I was able to make from the mails[2] are:\n\n1. We are catching commands typed by the user in an unpopulated or\n   even an uninitialized submodule.\n\n2. We first check if we are present in the superproject's root dir.\n   If .git dir is present we check for the presence of .gitmodules file,\n   from which we can check the path give is inside some submodule.\n   *When .git file containing just a .gitlink is present then, I am not\n   sure but even in this case the root folder contains .gitmodules\n   file in the case of submodules(Please correct me here, if I'm going\n   wrong), then we may still carry the same procedure as above.\n\n3. Once we can detect whether the $cwd is in a submodule, we can output\n   \"The submodule 'sub' is not initialized. To init ...\" for all the\n   commands which doesn't initialize and populate the submodule.\n\n4. As similar detection could be used in the third project listed above,\n   hence I even wished to include it.\n\nWhat are your suggestions about these projects? Also, will it be\nrational to consider it as one complete project for GSoC?\nI think this might interfere with Valery's proposal[3] of shell to C\nconversion of submodule related codes. What do you all think?\nIf it does interfere, then can we both work out on something common?\n\nThanks,\nPrathamesh\n\n[1]: https://public-inbox.org/git/20170313065148.10707-1-pc44800@gmail.com/T/#u\n[2]: https://public-inbox.org/git/CAGZ79kYW1zS3-9AYPaiUfBGTFygyg1ZVd3YyOctp3gihfEpHeg@mail.gmail.com/T/#u\n[3]: https://public-inbox.org/git/20170310211348.18887-1-me@vtolstov.org/T/#u\n"},{"id":"314152","messageId":"CAGZ79kbfNBbHgc4FrGoNNc54p65-xkQUQ6X3Ef9Ew9pioMzz7w@mail.gmail.com","threadId":"45383","inReplyTo":"CAME+mvW1x6fnGKt1_auGOp+wFYFR=Y_Qhxfd50E7KFe6t+X4kw@mail.gmail.com","subject":"Re: GSoC Project | Submodules related work","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-15T18:12:58Z","receivedAt":"2017-03-15T18:13:05Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Wed, Mar 15, 2017 at 2:33 AM, Prathamesh Chavan <pc44800@gmail.com> wrote:\n> Hey everyone,\n>\n> I am Prathamesh. I am studying Computer Science and Engineering at IIT\n> Kharagpur. I am interested to participate in Google Summer of Code 2017\n> under Git organization. I attempted \"Avoid pipes in git related commands\n> for test suite\" as my microproject[1].\n>\n> As a part of GSoC, I would like to work on git submodules. The projects I\n> have looked up are:\n>         1. \"git -C sub add .\" might behave just like \"git add sub\"\n>         2. Teach \"git -C <submodule-path> status\" in an unpopulated\n>            submodule to report the submodule being unpopulated, do not\n>            fall back to the superproject.\n>         3. Teach \"git log -- <path/into/submodule/and/further>\" to behave\n>            like \"git -C <path/into/submodule> log -- <and/further>\"\n>\n> I went through the series of mail (related to projects 1 and 2)[2] for\n> getting a better picture of the projects. I think as the projects are\n> quite interrelated together, these may make a complete GSoC project.\n\nSounds reasonable.\n\n> Also the conclusions which I was able to make from the mails[2] are:\n>\n> 1. We are catching commands typed by the user in an unpopulated or\n>    even an uninitialized submodule.\n\nWhat do you mean by catch here?\n\nWhat happens is that Git is summoned in e.g. path/super/sub/ and when\nGit wakes up, it has to find out what is going on, e.g. where the repo\nis that it should work on.\n\n(0) it checks if it is inside the .git dir by looking for files like\nHEAD, config\n    objects/.)\n1) It looks if there is a \".git\" file or directory in the current directory.\n2) if that is not the case go up one directory and check again.\n    repeat this step until either the repo is found or a filesystem boundary\n    is reached.\n\nAnd as uninitialized submodules do not have a .git, we'll find the\nsuperproject repository. Once the superproject is found, the\nsubcommand itself is invoked. (e.g. cmd_add for \"git add\", in\nbuiltin/add.c; the function signature is just like main() except that\nit has an additional prefix parameter, which indicates the path\nfrom where we ended up at to the original invokation point,\ni.e. when invoked in  path/super/sub/, and the superproject\nwas found at path/super/, the prefix is sub/.\n\n> 2. We first check if we are present in the superproject's root dir.\n\nAfter the repo discovery as described above we're in a root of\n*a* repository, and we have a prefix, which may or may not be\nan uninitialized submodule.\n\n>     we check for the presence of .gitmodules file,\n\nWe have an API for that. :)\nSee submodule-config.h#submodule_from_path that\neither returns a struct submodule or NULL if there is no\nsubmodule.\n\nHowever to detect if there is a submodule, we can to check two\nthings: if there is a .gitmodules entry and if there is a gitlink entry\nrecorded in the tree. Depending on the command we'd want to\ndo one before the other. e.g. git-add most likely doesn't need to\nload .gitmodules, but may have the objects already loaded.\nSo checking if a given path is a submodule is cheap compared\nto loading the .gitmodules file, so we'd probably want to do the\ncheap thing first.\n\n>    from which we can check the path give is inside some submodule.\n>    *When .git file containing just a .gitlink is present then, I am not\n>    sure but even in this case the root folder contains .gitmodules\n>    file in the case of submodules(Please correct me here, if I'm going\n>    wrong), then we may still carry the same procedure as above.\n\nI think even when the .gitmodules file is missing, we want to have\nsome sort of warning here, as it is a confusing state to run git\nfrom an uninitialized gitlink'd repository. The user may assume they\nrun the command in the gitlink'd repo, so it may be better to bail out.\n\n>\n> 3. Once we can detect whether the $cwd is in a submodule, we can output\n>    \"The submodule 'sub' is not initialized. To init ...\" for all the\n>    commands which doesn't initialize and populate the submodule.\n\nIt depends on the command what we want to do; for most commands\nthis seems to be the right choice. For git-log we need to do a different\nthing, as you mention in 4)\n\n> 4. As similar detection could be used in the third project listed above,\n>    hence I even wished to include it.\n>\n> What are your suggestions about these projects? Also, will it be\n> rational to consider it as one complete project for GSoC?\n\nI think it is sensible to consider enhancing multiple commands, as\none command is a very small bite for a GSoC project.\nAnd once you have the first command done, you'd generally know\nthe vibe and the next commands ought to be easier. (though we'd\nstill need to figure out different outcomes, e.g. step 3 or 4 as above).\n\n> I think this might interfere with Valery's proposal[3] of shell to C\n> conversion of submodule related codes. What do you all think?\n\nI do not think there is interference with Valery's proposal, as this\nproposal would mostly work in builtin/{add,log,commit}.c\n(cmd_status is in builtin/commit.c for whatever reason)\nwhereas Valery's proposal would mostly revolve around working\nin git-submodule.sh (deleting lines there) and\nbuiltin/submodule--helper.c (adding the deleted lines back here;\nin another language)\n\n> If it does interfere, then can we both work out on something common?\n\nI really do not see a lot of interference of these 2 proposals.\n\nThanks,\nStefan\n"},{"id":"314177","messageId":"CAME+mvX6EH+R6xRiCCYAg_gaguv4yeFD1T765+q7F8YMppeOhw@mail.gmail.com","threadId":"45383","inReplyTo":"CAGZ79kbfNBbHgc4FrGoNNc54p65-xkQUQ6X3Ef9Ew9pioMzz7w@mail.gmail.com","subject":"Re: GSoC Project | Submodules related work","fromName":"Prathamesh","fromEmail":"pc44800@gmail.com","sentAt":"2017-03-15T21:28:51Z","receivedAt":"2017-03-15T21:30:21Z","isPatch":false,"sender":{"key":"pc44800@gmail.com","avatar":"https://avatars.githubusercontent.com/u/17272661?v=4"},"body":">> Also the conclusions which I was able to make from the mails[2] are:\n>>\n>> 1. We are catching commands typed by the user in an unpopulated or\n>>    even an uninitialized submodule.\n>\n> What do you mean by catch here?\n\nBy catching commands, I meant that we identify that the user has entered\nthe command in an unpopulated or uninitialized submodule and respond\nto the user accordingly.\n\n>\n> What happens is that Git is summoned in e.g. path/super/sub/ and when\n> Git wakes up, it has to find out what is going on, e.g. where the repo\n> is that it should work on.\n>\n> (0) it checks if it is inside the .git dir by looking for files like\n> HEAD, config\n>     objects/.)\n> 1) It looks if there is a \".git\" file or directory in the current directory.\n> 2) if that is not the case go up one directory and check again.\n>     repeat this step until either the repo is found or a filesystem boundary\n>     is reached.\n>\n> And as uninitialized submodules do not have a .git, we'll find the\n> superproject repository. Once the superproject is found, the\n> subcommand itself is invoked. (e.g. cmd_add for \"git add\", in\n> builtin/add.c; the function signature is just like main() except that\n> it has an additional prefix parameter, which indicates the path\n> from where we ended up at to the original invokation point,\n> i.e. when invoked in  path/super/sub/, and the superproject\n> was found at path/super/, the prefix is sub/.\n>\n>> 2. We first check if we are present in the superproject's root dir.\n>\n> After the repo discovery as described above we're in a root of\n> *a* repository, and we have a prefix, which may or may not be\n> an uninitialized submodule.\n>\n>>     we check for the presence of .gitmodules file,\n>\n> We have an API for that. :)\n> See submodule-config.h#submodule_from_path that\n> either returns a struct submodule or NULL if there is no\n> submodule.\n>\n> However to detect if there is a submodule, we can to check two\n> things: if there is a .gitmodules entry and if there is a gitlink entry\n> recorded in the tree. Depending on the command we'd want to\n> do one before the other. e.g. git-add most likely doesn't need to\n> load .gitmodules, but may have the objects already loaded.\n> So checking if a given path is a submodule is cheap compared\n> to loading the .gitmodules file, so we'd probably want to do the\n> cheap thing first.\n\nAdding to this, we may use here this functions is_submodule_populated()\nand is_submodule_initialized() from submodule.c\n\n>\n>>    from which we can check the path give is inside some submodule.\n>>    *When .git file containing just a .gitlink is present then, I am not\n>>    sure but even in this case the root folder contains .gitmodules\n>>    file in the case of submodules(Please correct me here, if I'm going\n>>    wrong), then we may still carry the same procedure as above.\n>\n> I think even when the .gitmodules file is missing, we want to have\n> some sort of warning here, as it is a confusing state to run git\n> from an uninitialized gitlink'd repository. The user may assume they\n> run the command in the gitlink'd repo, so it may be better to bail out.\n\nCan you please give an example of such situation ? I would like to\nreproduce it and think further.\n(As even in the case where the superproject is initialized using gitlink,\n.gitmodules is in the same folder as that of the .git file containing\nGIT_DIR path)\n\n>\n>>\n>> 3. Once we can detect whether the $cwd is in a submodule, we can output\n>>    \"The submodule 'sub' is not initialized. To init ...\" for all the\n>>    commands which doesn't initialize and populate the submodule.\n>\n> It depends on the command what we want to do; for most commands\n> this seems to be the right choice. For git-log we need to do a different\n> thing, as you mention in 4)\n>\n>> 4. As similar detection could be used in the third project listed above,\n>>    hence I even wished to include it.\n>>\n>> What are your suggestions about these projects? Also, will it be\n>> rational to consider it as one complete project for GSoC?\n>\n> I think it is sensible to consider enhancing multiple commands, as\n> one command is a very small bite for a GSoC project.\n> And once you have the first command done, you'd generally know\n> the vibe and the next commands ought to be easier. (though we'd\n> still need to figure out different outcomes, e.g. step 3 or 4 as above).\n>\n\nThank you for your suggestions. I will also look into more such cases\nwhere I may enhance multiple commands so as to expand my project.\n\n>> I think this might interfere with Valery's proposal[3] of shell to C\n>> conversion of submodule related codes. What do you all think?\n>\n> I do not think there is interference with Valery's proposal, as this\n> proposal would mostly work in builtin/{add,log,commit}.c\n> (cmd_status is in builtin/commit.c for whatever reason)\n> whereas Valery's proposal would mostly revolve around working\n> in git-submodule.sh (deleting lines there) and\n> builtin/submodule--helper.c (adding the deleted lines back here;\n> in another language)\n>\n>> If it does interfere, then can we both work out on something common?\n>\n> I really do not see a lot of interference of these 2 proposals.\n\nThank you for confirming that. Now I may carry-on working on my\nproposal for the project :) Also if it possible, I would like to\nwork on a smaller task related to my project first, as it will help me\nunderstand about the project more, and which also will help me write\nmy proposal for the project.\n\nThanks,\nPrathamesh\n"},{"id":"314191","messageId":"CAGZ79kbV9oJp0ewp8=qz1Yts_RTDPsRW9LYcuNBQr5EFzK+VYA@mail.gmail.com","threadId":"45383","inReplyTo":"CAME+mvX6EH+R6xRiCCYAg_gaguv4yeFD1T765+q7F8YMppeOhw@mail.gmail.com","subject":"Re: GSoC Project | Submodules related work","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-15T22:13:00Z","receivedAt":"2017-03-15T22:13:12Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Wed, Mar 15, 2017 at 2:28 PM, Prathamesh <pc44800@gmail.com> wrote:\n>> What do you mean by catch here?\n>\n> By catching commands, I meant that we identify that the user has entered\n> the command in an unpopulated or uninitialized submodule and respond\n> to the user accordingly.\n\nWell in that sense, we do not do that, yet. I see what you mean.\n\n>> However to detect if there is a submodule, we can to check two\n>> things: if there is a .gitmodules entry and if there is a gitlink entry\n>> recorded in the tree. Depending on the command we'd want to\n>> do one before the other. e.g. git-add most likely doesn't need to\n>> load .gitmodules, but may have the objects already loaded.\n>> So checking if a given path is a submodule is cheap compared\n>> to loading the .gitmodules file, so we'd probably want to do the\n>> cheap thing first.\n>\n> Adding to this, we may use here this functions is_submodule_populated()\n> and is_submodule_initialized() from submodule.c\n\nNot quite, IMO.\n\nis_submodule_initialized checks for the existence of\nsubmodule.<name>.URL in .git/config; but it sounds as if we want to\ncheck for the existence of submodule.<any name>.path in .gitmodules\ninstead. So we'd end up using only\n\n    module = submodule_from_path(null_sha1, path);\n\nonly from that function.\n\nis_submodule_populated checks if there is a .git file/directory at the given\npath, which at this point we would know is not the case, already?\n\nWe'd roughly need to\nmodule_list_compute(... prefix = \"\", pathspec = prefix, ...),\ni.e.\n\n    struct cache_entry *ce = lookup_cache_entry_for(prefix);\n    if (ce && S_ISGITLINK(ce->ce_mode))\n        /* this is an uninitialized submodule */\n    else\n        /* this is just a normal prefix */\n\n>> I think even when the .gitmodules file is missing, we want to have\n>> some sort of warning here, as it is a confusing state to run git\n>> from an uninitialized gitlink'd repository. The user may assume they\n>> run the command in the gitlink'd repo, so it may be better to bail out.\n>\n> Can you please give an example of such situation ? I would like to\n> reproduce it and think further.\n\nI think you can create such a situation via\n\n    git init tmp\n    cd tmp\n    git init gitlink\n    git -C gitlink commit --allow-empty -m \"initial commit\"\n->  git add gitlink\n    git commit -m \"add 'gitlink' as a gitlink\"\n    rm -rf gitlink\n    mkdir gitlink\n    git -C gitlink status\n\nNote that we used \"git add\" instead of \"git submodule add\". git-add\ndoesn't care about submodule, i.e. doesn't create a .gitmodules entry\nfor you (unlike \"git submodule add\").\n\nAlso note that the \"rm -rf && mkdir\" is just a placeholder to produce this\nstate. An alternative ending after the commit could have been\n\n    cd ..\n    git clone tmp tmp2\n    cd tmp2\n    git -C gitlink status\n\n\n\n> (As even in the case where the superproject is initialized using gitlink,\n> .gitmodules is in the same folder as that of the .git file containing\n> GIT_DIR path)\n\nI do not understand this?\n\n> Also if it possible, I would like to\n> work on a smaller task related to my project first, as it will help me\n> understand about the project more, and which also will help me write\n> my proposal for the project.\n\nHeh, that is the beauty of open source, you don't have to ask permission. ;)\nBut I guess this is meant as a question, on what this smaller project\ncould be? Well as this proposal is heavy on path computation, I'd\nlook for pathspec related leftovers at\nhttps://git-blame.blogspot.com/p/leftover-bits.html\n\nThanks,\nStefan\n"}]}