{"thread":{"id":"59505","subject":"[GSOC] [PROPOSAL v2] Draft of proposal for \"Unify ref-filter formats with other pretty formats\"","startedAt":"2023-03-31T03:22:56Z","lastAt":"2023-04-02T19:40:34Z","messageCount":4,"participants":["Zhang Yi","Christian Couder","ZhangYI"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"474496","messageId":"100814d1.2603.18735b059bb.Coremail.18994118902@163.com","threadId":"59505","inReplyTo":null,"subject":"[GSOC] [PROPOSAL v2] Draft of proposal for \"Unify ref-filter formats with other pretty formats\"","fromName":"Zhang Yi","fromEmail":"18994118902@163.com","sentAt":"2023-03-31T03:22:42Z","receivedAt":"2023-03-31T03:22:56Z","isPatch":false,"sender":{"key":"18994118902@163.com","avatar":"https://avatars.githubusercontent.com/u/48300302?v=4"},"body":"I have changed my proposal according to the comments by Hariom Verma.\n\nImprovement vs v1:\n1. Put more effort into related work and grasp a lot from them.\n2. More details about timeline.\n3. More details about my plan.\n4. Some tiny changes in other content.\n\nOpen to more guidances. Thanks for suggestions.\n\n\n* Unify ref-filter formats with other pretty formats\n\n* Personal Information\n\nFull name: Zhang Yi\n\nE-mail: 18994118902@163.com\nTel: (+86)18994118902\n\nEducation: Wuhan University of Technology (China)\nMajor: Computer engineering \nYear: First-year postgraduate student\n\nGithub: https://github.com/zhanyi22333\n\n*  Synopsis\n\n** Motivation\n\nGit has different implements to format command output, which makes chaos and\nhinder improvement of code quality.\n\nAim to unify the different implementations to format output for different\ncommands, we want to transform pretty into ref-filter formatting logic. According\nto the present situation, I need to add more ref-filter atoms to replace\npretty.\n\n** Previous Work\n\n  - `git for-each-ref`, `git branch` and `git tag` formats into the\nref-filter formats:\n\ndone by Karthik Nayak (GSoC 2015)\n\n  -  `git cat-file` formats and the ref-filter formats:\n\nstarted by Olga Telezhnaya (Outreachy 2017-2018),\ncontinued by ZheNing Hu (GSoC 2021),\n    There are a lot of patches which are concluded in his final blog [1]\nbut still not finished due to tricky performance issues\n\n  - ref-filter formats and pretty formats:\n\nstarted by Hariom Verma (GSoC 2020)\n    There are also a lot of patches which are concluded in his final blog [2]\ncontinued a bit by Jaydeep Das (GSoC 2022)\n    Patch: gpg-interface: add function for converting trust level to string [3]\nand continued by Nsengiyumva Wilberforce and his  work on the \"signature\" atoms\nshould be mostly over when the GSoC starts. (Outreachy 2022-2023)\n    Patch: ref-filter: add new atom \"signature\" atom [4]\n\nps: There seems no conclusion articles of Karthik Nayak's and Olga Telezhnava's\nworks.\n\n** What is left\n\nSince the work of \"signature\" atoms will be finished by Nsengiyumva Wilberforce,\nThere may be some other atoms left for ref-filter formats and pretty formats.\nBut I still need to check.\n\nIf there is no work left for for ref-filter formats and pretty formats, then\nthere may be another command which has a different format implement with\nref-filter.\n\n** Steps\n\nIn my mind, there are 4 steps logically:\n1. Check and find a pretty atom which has no substitute in ref-filter.\n   This step is to decide the whole direction of the next work.\n   Christian Couder informed me that I can do things like the following:\n   - making sure that all the atoms in the pretty formats have similar\n   atoms implemented in the ref-filter formats\n   - find a way to convert any string containing pretty format atoms to\n   a string containing only ref-filter format atoms\n   - find a way to plug-in the ref-filter code into the pretty code, so\n   that callers of the pretty code would not need to be changed much.\n2. Add reasonable test scripts and maybe documents in advance.\n   In my opinion, making a draft of test scripts and documents in advance can\n   help me have a deep understanding of the behavior that I need to code. I learn\n   this development mode from book. And I have really met problems rising from\n   the misunderstanding of needed behavior which will result in a lot of reworks.\n3. Change code.\n   Inspired by Hariom Verma's proposal, I can  start by first looking at what\n   actually needed to be replaced (for example by studying the PRETTY FORMATS\n   section in 'man git-log', what which verbs you can use in the ref-filter\n   ('man git-for-each-ref') to achieve the same thing. Then I can research how\n   one format is implemented in 'pretty.c', and see how a similar thing using\n   the ref-filter is implemented in 'ref-filter.c'.\n4. Recheck documents and run test scripts.\n   Necessary step to check the behavior of code.\n\n\n* Benefits to Community\n\nI'm willing to stay around after the project. By that time, I will be in my\nsecond year without classes. And my tutor has an open mind about my request to\ninvolve in an open source project by now. Considering the subjective and\nobjective conditions, I think there is a high possibility that I will stay\naround.\n\nParticularly, I wish to be a co-mentor if I have the ability. There may be some\ndifficulties. But what I learn from my finite experience is that you should not\nrefuse something positive just because of the difficulties in the mind. A\nfresh new job may be difficult, but it can show me the possibilities of the\nworld, which means changing my mind.\n\nWhat's more, I tried to persuade a schoolmate who I think is kind of obsessed\nwith technology to take part in an open source community for both self-growth and\ncompanion. And I failed, because he thinks it is hard.  It's always hard to\nchange Others' deep-rooted ideas by word. But I think the actions speak louder\nthan words. Maybe after the project, I can change the minds of people around me\nabout joining an open source community. There may be no visual benefits to the\nGit Community but should be beneficial to the whole open source community.\n\n* Microproject\n\nt9700: modernize test scripts [5]\n\nThe microproject patches have been merged. The merge info is as below:\n\ncommit 8760a2b3c63478e8766b7ff45d798bd1be47f52d\nMerge: a2d2b5229e 509d3f5103\nAuthor: Junio C Hamano <gitster@pobox.com>\nDate:   Tue Feb 28 16:38:47 2023 -0800\n\n    Merge branch 'zy/t9700-style'\n\n    Test style fixes.\n\n    * zy/t9700-style:\n      t9700: modernize test scripts\n\n* Plan\n\n** Timeline and deliverables\n\nThe official GSOC code time start from 05-29 to 08-28, which is 13 weeks.\nThe period from 06-05 to 06~30 is near the end of the semester. There are many\nclasses for me. So I guess I may be not productive during this period.\nI think it is a bit time-limited if I follow the official timeline. It seems\nnecessary to do some work in advance.\n\n1. preparatory work:\n Period:\n  04-01 ~ 05-28\n  about 8 weeks\n Tasks:\n  1. Decide which parts need to work and which has priority.\n  2. Read Hariom's blogs.\n  3. Trying to understand the formatting logic behind pretty and ref-filter.\n  (Maybe try gdb?)\n  4. Try to make some trial change\n\n2. Write draft of documents and test scripts.\n Period:\n  05-29 ~ 06-02\n  week 1\n Tasks:\n  Based on the preparatory work, write drafts of doc and test.\n Deliverables:\n  Drafts of documents and test scripts\n3. Inactive Period\n Period:\n  06-05 ~ 06-30\n  week 2~5\n  4 weeks\n Tasks:\n  1. Build the base of other works like atoms.\n  2. Should pass some special tests.\n Deliverables:\n  A new atoms\n\n4. Active code period 1\n Period:\n  07-03 ~ 07-07\n  week 6\n Tasks:\n  1. Add a new argument and grab functions for the atoms\n  2. Need to pass tests and in same with documents\n Deliverables:\n  A new argument and its grab function\n5. Midterm evaluation\n Period:\n  07-10 ~ 07-14\n  week 7\n Tasks:\n  1. Submitting midterm evaluations\n  2. Maybe need to continue the work left from last week\n Deliverables:\n  midterm evaluation\n\n6. Active code period 2\n Period:\n  07-17 ~ 08-04\n  week 8~10\n  3 weeks\n Tasks:\n  1. Add 2~3 new arguments\n  2. Also need to pass tests and in same with documents.\n  3. Drafts of documents and test scripts should be updated.\n Deliverables:\n  1. New arguments\n  2. Documents\n  3. test scripts\n\n7. Finishing touches\n Period:\n  08-07 ~ 08-26\n  week 11~13\n  3 weeks\n Tasks:\n  1. There should be some bugs to fix or work left.\n  2. This period is also left for unexpected events.\n  3. Submit final work product and final mentor evaluation.\n Deliverables:\n  1. final work product\n  2. final mentor evaluation\n\n\n* Grasp from related work\n** From Hariom Verma's blog\nWalking through the blogs of Hariom Verma, I find many things useful.\n\n*** Debugging\n\nAn extremely informative(step-by-step) debugging guide by Christian. [6]\n\n*** 11 questions for understanding someone's work. [7]\n\n1. What was the goal of each patch?\n2. which approach did she took to achieve the goal?\n3. what were the goals of the patch series?\n4. which approach did she took to achieve the goals?\n5. what was the goal of her previous patch series?\n6. what was the general direction her patch series were going?\n7. why did she took that direction?\n8. are there ways to continue in the same direction?\n9. are there ways to achieve similar goals?\n10. how were her goals similar and different from the goals in my proposal?\n11. is it possible to use the same approach?\n\n*** Else\n\nThere are many details about his work progress. I can refer to them when I am in\nsimilar situations.\n\n** From ZheNing Hu's blog\n\n*** Time analyzing\n\nUse performance testing tools to analyze the time-consuming steps of\n`git cat-file --batch`.\n\n Using Google's `gperftools`:\n1. Add the link parameter `-lprofiler` in `config.mak`: `CFLAGS += -lprofiler`.\n2. `make`.\n3. Use `CPUPROFILE=/tmp/prof.out /<path>/git cat-file --batch-check\n--batch-all-objects`\nto run the git and general `prof.out`, which contains the results of\nperformance analysis.\n4. Use `pprof --text /<path>/git /tmp/prof.out` to display the result\nin the terminal.\n\n*** About Github CI\n\n\"GitHub-Travis CI hints\" in Documentation/SubmittingPatches\n\n*** Else\n\nHe also writes his process of debugging and optimization in detail. It's worth\ndeepening into when I need them.\n\nThis proposal draft benefits from the works of predecessors much. Thanks.\n\n* Biograhical information\n\nIt is always funny to recall that I first learned about Linux in a stimulated\nhacker game in my fresh year in college. After that, I tried to teach myself\nLinux and started to know open source projects. Overcome many difficulties and I\nfinally know something shallow about Linux. As a side effect, I am more\nenthusiastic and better at programming compared with my schoolmates. But the\nperiod of stagnation came, I began to write some meaningless projects for school\ntasks and repeated myself without progress. The best out of the worst, I touched\nexcellent open source software during the time, such as vim, emacs, visual\nstudio code, Qt, VLC and, of course, git. Near the end of my junior year, I read\nan article about learning by contributing to an open source project by a geek\nin the community of emacs. Almost at the same time, I knew the GSOC and preferred\nto take part in git. But it was near the start date of my plan for postgraduate\nqualifying examination. So I just postponed the stuff for GSOC.  Luckily, I\npassed the examination. After I got used to life as a postgraduate student, I\nfelt the motivation to progress again. Then I tried to contribute for git. Now I\njust finished a micro project, which seems trivial. But it really let me have a\ndeeper understanding of open source and free software and more motivation to\ncontribute. I hope I can stay here a long time before being involved with other\ninteresting projects since the quality is more important than the quantity.\nI know it seems a bit stubborn to believe that contributing will lead to\nprogress, which is also influenced by my learning attitude. But without action,\nI can not verify the belief.  Sooat least I will try to contribute for one year.\nAfter that, I hope I can have a better understanding.\n\nSorry, the above text may be messing. In short, I will try to contribute for\ngit for at least one year.\n\n* Closing remarks\n\nIt seems blogs will help much for later work. I think It worth rebuilding my\nblog site on github.\n\nThanks for Christian Couder's and Hariom Verma's help.\n\n\n[1] https://public-inbox.org/git/CAOLTT8SxHuH2EbiSwQX6pyJJs5KyVuKx6ZOPxpzWLH+Tbz5F+A@mail.gmail.com/\n[2] https://harry-hov.github.io/blogs/posts/the-final-report\n[3] https://public-inbox.org/git/pull.1281.git.1657202265048.gitgitgadget@gmail.com/\n[4] https://public-inbox.org/git/pull.1452.git.1672102523902.gitgitgadget@gmail.com/#t\n[5] https://lore.kernel.org/git/20230222040745.1511205-1-18994118902@163.com/\n[6] https://public-inbox.org/git/CAP8UFD3Bd4Af1XZ00VyuHnQs=MFrdUufKeePO1tyedWoReRjwQ@mail.gmail.com/\n[7] https://harry-hov.github.io/blogs/posts/week1-the-ten-questions\n"},{"id":"474600","messageId":"CAP8UFD3Lns7pQQ-yNc5W8d2bfPBmJ0pcD52yCbkFOmymhWKw9Q@mail.gmail.com","threadId":"59505","inReplyTo":"100814d1.2603.18735b059bb.Coremail.18994118902@163.com","subject":"Re: [GSOC] [PROPOSAL v2] Draft of proposal for \"Unify ref-filter formats with other pretty formats\"","fromName":"Christian Couder","fromEmail":"christian.couder@gmail.com","sentAt":"2023-04-01T09:04:51Z","receivedAt":"2023-04-01T09:05:08Z","isPatch":false,"sender":{"key":"christian.couder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/208954?v=4"},"body":"On Fri, Mar 31, 2023 at 5:22 AM Zhang Yi <18994118902@163.com> wrote:\n>\n> I have changed my proposal according to the comments by Hariom Verma.\n>\n> Improvement vs v1:\n> 1. Put more effort into related work and grasp a lot from them.\n> 2. More details about timeline.\n> 3. More details about my plan.\n> 4. Some tiny changes in other content.\n>\n> Open to more guidances. Thanks for suggestions.\n\nThanks for improving your proposal based on our feedback!\n\n[...]\n\n> Aim to unify the different implementations to format output for different\n> commands, we want to transform pretty into ref-filter formatting logic. According\n> to the present situation, I need to add more ref-filter atoms to replace\n> pretty.\n\nCould you explain a bit more what that means and why you need to do\nthat? (You might already do that in a different section below, but it\nstill feels a bit strange to see this last sentence without much\nexplanation.)\n\n> ** Previous Work\n>\n>   - `git for-each-ref`, `git branch` and `git tag` formats into the\n> ref-filter formats:\n>\n> done by Karthik Nayak (GSoC 2015)\n>\n>   -  `git cat-file` formats and the ref-filter formats:\n>\n> started by Olga Telezhnaya (Outreachy 2017-2018),\n> continued by ZheNing Hu (GSoC 2021),\n>     There are a lot of patches which are concluded in his final blog [1]\n> but still not finished due to tricky performance issues\n>\n>   - ref-filter formats and pretty formats:\n>\n> started by Hariom Verma (GSoC 2020)\n>     There are also a lot of patches which are concluded in his final blog [2]\n> continued a bit by Jaydeep Das (GSoC 2022)\n>     Patch: gpg-interface: add function for converting trust level to string [3]\n> and continued by Nsengiyumva Wilberforce and his  work on the \"signature\" atoms\n> should be mostly over when the GSoC starts. (Outreachy 2022-2023)\n\nYeah except Wilberforce has actually been working outside of Outreachy\nas he didn't satisfy the requirements for being accepted, but still\nwanted to work on this.\n\n>     Patch: ref-filter: add new atom \"signature\" atom [4]\n>\n> ps: There seems no conclusion articles of Karthik Nayak's and Olga Telezhnava's\n> works.\n\nKarthik's blog posts might have disappeared for some reason. I have\nCc-ed him and he might tell us.\n\nOlga's blog posts seem to still be available on\nhttps://medium.com/@olyatelezhnaya. Medium seems to require signing in\nthese days though.\n\n> ** What is left\n>\n> Since the work of \"signature\" atoms will be finished by Nsengiyumva Wilberforce,\n> There may be some other atoms left for ref-filter formats and pretty formats.\n> But I still need to check.\n>\n> If there is no work left for for ref-filter formats and pretty formats, then\n\ns/for for/for/\n\n> there may be another command which has a different format implement with\n\nMaybe: s/implement/implemented/\n\n> ref-filter.\n\nI am not sure what your last sentence here means. If a command already\nuses ref-filter formats, then there is no more work to do as that's\nthe end state we would like.\n\n> ** Steps\n>\n> In my mind, there are 4 steps logically:\n> 1. Check and find a pretty atom which has no substitute in ref-filter.\n>    This step is to decide the whole direction of the next work.\n\nSo you might want to take a look at this step soon. It might not be\ndifficult to find out, as the implemented atoms are described in the\ndocs.\n\nThey are called \"field names\" in the git for-each-ref documentation\n(Documentation/git-for-each-ref.txt) and \"placeholders\" in the pretty\nformats documentation (Documentation/pretty-formats.txt).\n\n>    Christian Couder informed me that I can do things like the following:\n>    - making sure that all the atoms in the pretty formats have similar\n>    atoms implemented in the ref-filter formats\n>    - find a way to convert any string containing pretty format atoms to\n>    a string containing only ref-filter format atoms\n>    - find a way to plug-in the ref-filter code into the pretty code, so\n>    that callers of the pretty code would not need to be changed much.\n\nYeah I suggested these as possible steps to split the work, hoping\nthat you would dig a bit more what they meant, and how you could\nperform them.\n\n> 2. Add reasonable test scripts and maybe documents in advance.\n>    In my opinion, making a draft of test scripts and documents in advance can\n>    help me have a deep understanding of the behavior that I need to code. I learn\n>    this development mode from book. And I have really met problems rising from\n>    the misunderstanding of needed behavior which will result in a lot of reworks.\n\nI agree that it's a good approach when developing new features. Here\nthe features and associated high level tests and documentation already\nexist. We \"just\" want to replace the internal implementation of the\npretty formats using the ref-filter formats. So the approach could be\na bit different.\n\n> 3. Change code.\n>    Inspired by Hariom Verma's proposal, I can  start by first looking at what\n>    actually needed to be replaced (for example by studying the PRETTY FORMATS\n>    section in 'man git-log', what which verbs you can use in the ref-filter\n>    ('man git-for-each-ref') to achieve the same thing.\n\nYeah and this shouldn't take a lot of time. I think Hariom already\nwrote a correspondence table between the different \"verbs\" (also\ncalled \"atoms\", \"placeholders\" or \"field names\") in the pretty and\nref-filter formats.\n\n> Then I can research how\n>    one format is implemented in 'pretty.c', and see how a similar thing using\n>    the ref-filter is implemented in 'ref-filter.c'.\n\nWhat will you learn from that and how will it help you for the next steps?\n\nYou call this section \"Change code\" but it looks like it's only about\nresearching things.\n\n> 4. Recheck documents and run test scripts.\n>    Necessary step to check the behavior of code.\n\nWe ask for tests and documentation to be part of the patches that are\nsent, so writing documentation and tests and running tests should be\npart of each coding step.\n\n> * Benefits to Community\n>\n> I'm willing to stay around after the project. By that time, I will be in my\n> second year without classes. And my tutor has an open mind about my request to\n> involve in an open source project by now. Considering the subjective and\n> objective conditions, I think there is a high possibility that I will stay\n> around.\n>\n> Particularly, I wish to be a co-mentor if I have the ability. There may be some\n> difficulties. But what I learn from my finite experience is that you should not\n> refuse something positive just because of the difficulties in the mind. A\n> fresh new job may be difficult, but it can show me the possibilities of the\n> world, which means changing my mind.\n\nGreat!\n\n[...]\n\n> * Microproject\n>\n> t9700: modernize test scripts [5]\n>\n> The microproject patches have been merged. The merge info is as below:\n>\n> commit 8760a2b3c63478e8766b7ff45d798bd1be47f52d\n> Merge: a2d2b5229e 509d3f5103\n> Author: Junio C Hamano <gitster@pobox.com>\n> Date:   Tue Feb 28 16:38:47 2023 -0800\n>\n>     Merge branch 'zy/t9700-style'\n>\n>     Test style fixes.\n>\n>     * zy/t9700-style:\n>       t9700: modernize test scripts\n\nThanks for your work on that!\n\n> * Plan\n\nIt's difficult to understand how this section is different from the\n\"Steps\" section above. Maybe these two sections could be merged.\n\n> ** Timeline and deliverables\n>\n> The official GSOC code time start from 05-29 to 08-28, which is 13 weeks.\n> The period from 06-05 to 06~30 is near the end of the semester. There are many\n> classes for me. So I guess I may be not productive during this period.\n\nThanks for telling us about this in advance!\n\n> I think it is a bit time-limited if I follow the official timeline. It seems\n> necessary to do some work in advance.\n\n[...]\n\n> 2. Write draft of documents and test scripts.\n>  Period:\n>   05-29 ~ 06-02\n>   week 1\n>  Tasks:\n>   Based on the preparatory work, write drafts of doc and test.\n>  Deliverables:\n>   Drafts of documents and test scripts\n\nSee what I said above about the fact that a big part of this project\nmight not be about developing new features.\n\n[...]\n\nBest,\nChristian.\n"},{"id":"474661","messageId":"3cea7bba.1e54.18742678107.Coremail.18994118902@163.com","threadId":"59505","inReplyTo":"CAP8UFD3Lns7pQQ-yNc5W8d2bfPBmJ0pcD52yCbkFOmymhWKw9Q@mail.gmail.com","subject":"Re: [GSOC] [PROPOSAL v2] Draft of proposal for \"Unify ref-filter formats with other pretty formats\"","fromName":"ZhangYI","fromEmail":"18994118902@163.com","sentAt":"2023-04-02T14:38:12Z","receivedAt":"2023-04-02T14:38:26Z","isPatch":false,"sender":{"key":"18994118902@163.com","avatar":"https://avatars.githubusercontent.com/u/48300302?v=4"},"body":"Thanks for Christian Couder's Constructive comments.\nI've looked through Olga Telezhnava's detailed and helpful blogs. \nI also tried to understand more about the works of the project today.\n\nI have one questions here:\nI used gdb to track the function call related to ref-filter of the command\n\"git log -2 --pretty=%h \" by setting breaks on all no-static functions in\nref-filter.c but found no stop.\nShould I use another command?\nOr as I know, Git use different branch for different purpose, like todo, next.\nShould I use another branch?\n"},{"id":"474668","messageId":"CAP8UFD2Tjqf0A+xv69pHDZFUDG2rPU7tj=9KSYh36_zTg0hG6g@mail.gmail.com","threadId":"59505","inReplyTo":"3cea7bba.1e54.18742678107.Coremail.18994118902@163.com","subject":"Re: [GSOC] [PROPOSAL v2] Draft of proposal for \"Unify ref-filter formats with other pretty formats\"","fromName":"Christian Couder","fromEmail":"christian.couder@gmail.com","sentAt":"2023-04-02T19:40:17Z","receivedAt":"2023-04-02T19:40:34Z","isPatch":false,"sender":{"key":"christian.couder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/208954?v=4"},"body":"On Sun, Apr 2, 2023 at 4:38 PM ZhangYI <18994118902@163.com> wrote:\n\n> I have one questions here:\n> I used gdb to track the function call related to ref-filter of the command\n> \"git log -2 --pretty=%h \" by setting breaks on all no-static functions in\n> ref-filter.c but found no stop.\n> Should I use another command?\n\n`git log` uses pretty formats. If you want to see how ref-filter\nformats work you should run for example `git for-each-ref`.\n\n> Or as I know, Git use different branch for different purpose, like todo, next.\n> Should I use another branch?\n\nNo this is not an issue with development branches.\n"}]}