{"thread":{"id":"63180","subject":"[GSoC] Proposal Discussion: git-refs Project","startedAt":"2025-03-23T13:37:04Z","lastAt":"2025-04-06T06:08:23Z","messageCount":17,"participants":["Yuting Zheng","Patrick Steinhardt","shejialuo","Zheng Yuting","Karthik Nayak"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"514875","messageId":"CAMvj1+rbYKFNeWEvvN76MTpzfuWc4TN4ViXRE4nTfWy7ZMspWg@mail.gmail.com","threadId":"63180","inReplyTo":null,"subject":"[GSoC] Proposal Discussion: git-refs Project","fromName":"Yuting Zheng","fromEmail":"05zyt30@gmail.com","sentAt":"2025-03-23T13:36:51Z","receivedAt":"2025-03-23T13:37:04Z","isPatch":false,"sender":{"key":"05zyt30@gmail.com","avatar":"https://avatars.githubusercontent.com/u/87643662?v=4"},"body":"Dear Git Community,\n\nI am very interested in applying for the GSoC 2025 project \"Consolidate\nref-related functionality into git-refs\". I have reviewed the relevant\ncode, documentation, and mailing lists, and as part of the application\nprerequisites, I have submitted a microproject patch\n(https://lore.kernel.org/git/20250323022111.20226-1-05ZYT30@gmail.com/).\n\nMy current idea is to extend the `git-refs` command—by calling into the\nexisting code—to add subcommands. This approach would replace the\nfunctionalities of the mentioned commands while ensuring that I do not\nmodify the code underlying them. This guarantees that the new `git-refs`\nsubcommand meets the new requirements without affecting the usage of the\nexisting commands.\n\nHowever, when searching the mailing lists with keywords\n“nq:consolidate ref” and “s: refs”, I did not find any discussion about\nmerging these commands. If anyone has come across any previous discussions\nor could kindly provide additional insights on this matter, I would greatly\nappreciate your help.\n\nThank you for your guidance.\n\nBest regards,\nZheng Yuting\n"},{"id":"514916","messageId":"Z-FJ3EQdFIkQgtkR@pks.im","threadId":"63180","inReplyTo":"CAMvj1+rbYKFNeWEvvN76MTpzfuWc4TN4ViXRE4nTfWy7ZMspWg@mail.gmail.com","subject":"Re: [GSoC] Proposal Discussion: git-refs Project","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-03-24T12:02:36Z","receivedAt":"2025-03-24T12:02:40Z","isPatch":false,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Hi Yuting,\n\nOn Sun, Mar 23, 2025 at 09:36:51PM +0800, Yuting Zheng wrote:\n> Dear Git Community,\n> \n> I am very interested in applying for the GSoC 2025 project \"Consolidate\n> ref-related functionality into git-refs\". I have reviewed the relevant\n> code, documentation, and mailing lists, and as part of the application\n> prerequisites, I have submitted a microproject patch\n> (https://lore.kernel.org/git/20250323022111.20226-1-05ZYT30@gmail.com/).\n> \n> My current idea is to extend the `git-refs` command—by calling into the\n> existing code—to add subcommands. This approach would replace the\n> functionalities of the mentioned commands while ensuring that I do not\n> modify the code underlying them. This guarantees that the new `git-refs`\n> subcommand meets the new requirements without affecting the usage of the\n> existing commands.\n> \n> However, when searching the mailing lists with keywords\n> “nq:consolidate ref” and “s: refs”, I did not find any discussion about\n> merging these commands. If anyone has come across any previous discussions\n> or could kindly provide additional insights on this matter, I would greatly\n> appreciate your help.\n> \n> Thank you for your guidance.\n\nI have been chatting with Peff about this topic quite a while ago, but\nthat was mostly an in-person chat that hasn't made it onto the mailing\nlist. I may also have mentioned on the mailing list on several occasions\nthat it would make sense to consolidate, but there wasn't ever a bigger\ndiscussion around all of this. There's also [1] as a non-authoritative\nsource for this project that documents my intent to consolidate the\ncommands.\n\nSo ultimately there hasn't been a lot of discussion yet around this\nwhole thing. Driving consensus and designing the new interface would\nthus be one of the biggest challenges in this project from my point of\nview.\n\nI'm happy to provide more feedback once an initial draft has been\ncreated for how the project could look like!\n\nThanks.\n\nPatrick\n\n[1]: https://gitlab.com/gitlab-org/git/-/issues/330\n"},{"id":"515128","messageId":"D8QOYSD6NLCS.OVF4RKHUCX0A@gmail.com","threadId":"63180","inReplyTo":"Z-FJ3EQdFIkQgtkR@pks.im","subject":"Re: [GSoC] Proposal Discussion: git-refs Project","fromName":"Yuting Zheng","fromEmail":"05zyt30@gmail.com","sentAt":"2025-03-27T02:26:49Z","receivedAt":"2025-03-27T02:27:12Z","isPatch":false,"sender":{"key":"05zyt30@gmail.com","avatar":"https://avatars.githubusercontent.com/u/87643662?v=4"},"body":"Thanks for your reply!\n\nI have reviewed the changelog and noted that Git version 2.23\nintroduced similar work through the addition of the git-switch and\ngit-restore commands, which replace some legacy commands and incorporate\nvarious functional modifications.\n\nAfter examining the updates, I have summarized the proposed work as\nfollows and would appreciate confirmation on whether these tasks are to be\nincluded in the current project:\n\n1. Code Modifications for Command Implementation:\n\n- Implementation of new commands.\n- Necessary modifications to existing commands to support these changes.\n\n2. Test Modifications:\n\n- Addition of tests for the new features (including help tests, basic\nfunctionality tests, and extended feature tests).\n- Updating tests for old commands to execute tests on the new commands\n(for example, changing the command in git-checkout tests to git-restore).\n\n3. Documentation Updates:\n\n- Creating documentation for the new commands.\nUpdating and unifying existing documentation (including git.txt,\ngit-cli.txt, and git-commit.txt).\n\nAdditionally, I have a few points that require further discussion:\n\n1. Command Migration:\n\nUpon reviewing the commands slated for replacement (e.g., git-update-ref(1),\ngit-for-each-ref(1), git-show-ref(1), git-pack-refs(1), and\ngit-symbolic-ref), it seems that migrating their functionality into a\nsubcommand of git-refs could be sufficient. Could you please confirm if\nthis approach meets our project requirements without introducing\nadditional functionality?\n\n2. Function Call Integration:\n\nRegarding migration, is it acceptable to directly invoke the legacy command\nfunctions by passing parameters from the new command functions?\n\n3. Test Retention:\n\nLastly, should we retain the original tests for the legacy commands, or\nshould they be fully replaced with tests for the new implementations?\n\nI appreciate your guidance and look forward to your feedback on these points.\n\nBest regards,\nZheng Yuting\n"},{"id":"515233","messageId":"Z-aoALIDd-U0bYnI@ArchLinux","threadId":"63180","inReplyTo":"D8QOYSD6NLCS.OVF4RKHUCX0A@gmail.com","subject":"Re: [GSoC] Proposal Discussion: git-refs Project","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-03-28T13:45:36Z","receivedAt":"2025-03-28T13:45:33Z","isPatch":false,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Thu, Mar 27, 2025 at 10:26:49AM +0800, Yuting Zheng wrote:\n> Thanks for your reply!\n> \n> I have reviewed the changelog and noted that Git version 2.23\n> introduced similar work through the addition of the git-switch and\n> git-restore commands, which replace some legacy commands and incorporate\n> various functional modifications.\n> \n> After examining the updates, I have summarized the proposed work as\n> follows and would appreciate confirmation on whether these tasks are to be\n> included in the current project:\n> \n> 1. Code Modifications for Command Implementation:\n> \n> - Implementation of new commands.\n> - Necessary modifications to existing commands to support these changes.\n\nI think \"modifications to existing commands\" may not be accurate. I\nthink what we need to do is we should try to reuse the original logic as\nmuch as possible which requires:\n\n1. Understand the behavior of the existing commands.\n2. Find good design to expose the common interfaces for the new commands\nand existing commands.\n\n> \n> 2. Test Modifications:\n> \n> - Addition of tests for the new features (including help tests, basic\n> functionality tests, and extended feature tests).\n> - Updating tests for old commands to execute tests on the new commands\n> (for example, changing the command in git-checkout tests to git-restore).\n\nI don't think that we should update tests for old commands. We want to\nkeep the original command not broken, right? So, we should use the\noriginal test to exercise your changed code to make sure that everything\nis OK.\n\n> \n> 3. Documentation Updates:\n> \n> - Creating documentation for the new commands.\n> Updating and unifying existing documentation (including git.txt,\n> git-cli.txt, and git-commit.txt).\n> \n> Additionally, I have a few points that require further discussion:\n> \n> 1. Command Migration:\n> \n> Upon reviewing the commands slated for replacement (e.g., git-update-ref(1),\n> git-for-each-ref(1), git-show-ref(1), git-pack-refs(1), and\n> git-symbolic-ref), it seems that migrating their functionality into a\n> subcommand of git-refs could be sufficient. Could you please confirm if\n> this approach meets our project requirements without introducing\n> additional functionality?\n> \n\nFrom my own understanding, we just want to use \"git-refs(1)\" as an entry\npoint about all operations for refs. So, we don't need to add new\nfunctionality in this project.\n\n> 2. Function Call Integration:\n> \n> Regarding migration, is it acceptable to directly invoke the legacy command\n> functions by passing parameters from the new command functions?\n> \n\nSo, you want to say that could we use a subprocess to just invoke the\nlegacy command? I don't think we should use subprocess. If we could use\nsubprocess, should this project be called as a project?\n\nI somehow think that you may first look at \"git-pack-refs(1)\" or\nsomething like which is not so complicated to think about a solution.\nAnd when writing the proposal, you may need to talk about how many\ncommands you want to migrate and how do you plan to migrate.\n\n> 3. Test Retention:\n> \n> Lastly, should we retain the original tests for the legacy commands, or\n> should they be fully replaced with tests for the new implementations?\n> \n\nThis is a good question. From my view, we should not change the original\ntests. And this would introduce another question, if we add the new test\nfor the new command, we'd introduce repetition. I cannot give your\nanswer here because I don't have experience about this.\n\n> I appreciate your guidance and look forward to your feedback on these points.\n> \n> Best regards,\n> Zheng Yuting\n\nThanks,\nJialuo\n"},{"id":"515274","messageId":"D8SU3YXKL9SD.33GRX731LWZE3@gmail.com","threadId":"63180","inReplyTo":"Z-aoALIDd-U0bYnI@ArchLinux","subject":"Re: [GSoC] Proposal Discussion: git-refs Project","fromName":"Yuting Zheng","fromEmail":"05zyt30@gmail.com","sentAt":"2025-03-29T14:54:00Z","receivedAt":"2025-03-29T14:54:06Z","isPatch":false,"sender":{"key":"05zyt30@gmail.com","avatar":"https://avatars.githubusercontent.com/u/87643662?v=4"},"body":"Thank you for clarifying my misunderstandings—some phrasing issues might\nstem from my non-native English. I’ve revised the proposal draft\naccordingly and will share it directly in this mailing list thread for\nyour review.\n"},{"id":"515275","messageId":"20250329150248.2274482-1-05ZYT30@gmail.com","threadId":"63180","inReplyTo":"CAMvj1+rbYKFNeWEvvN76MTpzfuWc4TN4ViXRE4nTfWy7ZMspWg@mail.gmail.com","subject":"[GSoC] git-refs proposal draft","fromName":"Zheng Yuting","fromEmail":"05zyt30@gmail.com","sentAt":"2025-03-29T15:02:46Z","receivedAt":"2025-03-29T15:03:02Z","isPatch":false,"sender":{"key":"05zyt30@gmail.com","avatar":"https://avatars.githubusercontent.com/u/87643662?v=4"},"body":"## Name and Contact Information\n\n- Full Name: Zheng Yuting\n- Email Address: 05ZYT30@gmail.com\n- Time Zone: UTC +8:00\n\n---\n\n## Abstract\n\nThe current Git reference management functionality is fragmented across\nmultiple independent commands (git-show-ref, git-for-each-ref,\ngit-update-ref, git-pack-refs, git-check-ref-format, and\ngit-symbolic-ref), leading to code redundancy and increased maintenance\ncosts. Based on Patrick Steinhardt’s integration vision[1], this project\naims to introduce 8 new subcommands (list, exists, show, resolve, pack,\nupdate, delete, check-format) under the existing git-refs command to\nachieve the following objectives:\n\n- Feature Integration: Consolidate existing reference management\n  commands under git-refs, while maintaining backward compatibility.\n- Feature Enhancement: Introduce recursion depth control for git-refs\n  resolve.\n- Testing & Documentation: Add test cases ensuring consistency and\n  update relevant documentation.\n\n---\n\n## Implementation Plan\n\n### Command Integration Strategy\n\n#### Design Goals\n\nThe project will unify scattered reference management functionalities\nunder the git-refs subcommand framework, ensuring:\n\n1. Complete Feature Coverage: Each subcommand fully replaces its\n   corresponding legacy command.\n2. Parameter Compatibility: Preserve the semantics and output behavior\n   of legacy command options.\n3. Code Reusability: Minimize redundancy by sharing underlying modules\n   (e.g., refs/files-backend.c).\n\n#### Subcommand Mapping\n\n- git-refs list\n  Replaces git-show-ref and git-for-each-ref, merging reference listing\n  functionalities with support for formatting (--format), filtering\n  (--heads, --tags), and sorting (--sort).\n- git-refs exists\n  Replaces git-show-ref --exists, providing reference existence checks\n  with positive (<ref>) and exclusion-based (--exclude-existing)\n  verification.\n- git-refs show\n  Replaces git-show-ref --verify, validating reference correctness with\n  a strict mode (--strict).\n- git-refs resolve\n  Replaces git-symbolic-ref, resolving symbolic references with added\n  recursion depth control (--max-depth), while retaining deletion (-d)\n  and quiet mode (-q) options.\n- git-refs pack\n  Replaces git-pack-refs, packing loose references with support for\n  filtering (--include, --exclude) and automatic cleanup (--prune).\n- git-refs update\n  Replaces git-update-ref, providing transactional reference updates\n  with batch processing (--stdin) and atomic guarantees.\n- git-refs delete\n  Separates the delete functionality from git-update-ref, ensuring\n  explicit handling of reference removals with safety checks and batch\n  operations (--stdin).\n- git-refs check-format\n  Replaces git-check-ref-format, validating reference format with\n  support for normalized output (--normalize).\n\n#### Implementation Strategy\n\n1. Option Parsing: Each subcommand will reuse the argument parsing\n   logic from legacy commands (e.g., git-pack-refs --prune).\n2. Shared Backend Logic: Calls to common functions in refs/ (e.g.,\n   reference traversal, locking mechanisms).\n3. Error Consistency: Maintain the same error codes and message\n   formats as legacy commands.\n\n---\n\n### Example: Implementing git-refs pack\n\n#### Functional Implementation\n\n1. Modify builtin/refs.c:\n   - Add cmd_refs_pack function implementing git-pack-refs logic.\n   - Update cmd_refs to include pack with\n     OPT_SUBCOMMAND(\"pack\", &fn, cmd_refs_pack).\n   - Define REFS_PACK_USAGE:\n     git refs pack [--all] [--no-prune] [--auto] [--include <pattern>]\n     [--exclude <pattern>].\n2. Register New Subcommand in git.c:\n   - Add { \"refs-pack\", cmd_refs_pack }, to the command array.\n3. Reuse refs/files-backend.c Logic:\n   - Ensure cmd_refs_pack calls pack_refs correctly, adjusting as\n     necessary for new options.\n\n#### Testing Plan\n\n- Test Cases:\n  Add t/txxx-refs-pack.sh, leveraging t/t0601-reffiles-pack-refs.sh\n  scenarios to verify:\n  - --prune removes obsolete references correctly.\n  - --include and --exclude apply filtering as expected.\n  - Packed references match legacy command outputs (diff .git/packed-refs).\n- Performance Benchmarking (if needed):\n  Add performance tests in t/perf to ensure no significant regression\n  in execution time or memory usage.\n\n#### Documentation Updates\n\n- User Manual:\n  Add a pack section to Documentation/git-refs.txt, mapping options to\n  legacy command equivalents.\n- Developer Notes:\n  Comment code to highlight functional parity between git-refs pack\n  and git-pack-refs.\n\n---\n\n### Timeline\n\n- May 8 - May 11 (4 days): Initial Testing & Subcommand Framework Setup\n- May 12 - May 28 (17 days): pack Subcommand Implementation\n- May 29 - June 14 (17 days): check-format Subcommand Development\n- June 15 - July 5 (21 days): update and delete Subcommands Development\n- July 6 - July 26 (21 days): show and exists Subcommands Development\n- July 27 - August 16 (21 days): resolve Subcommand Implementation\n- August 17 - September 6 (21 days): list Subcommand Implementation\n- September 7 - September 16 (10 days): Mid-term Review\n- September 17 - September 23 (7 days): Mentor Review & Final Adjustments\n\n---\n\n## Background & Experience\n\nI graduated in June 2024 from Wenzhou University with a degree in\nNetwork Engineering. My experience includes C programming and\ncommand-line tool development, along with proficiency in Shell\nscripting. I am currently in a transitional phase and expect to finalize\nmy schedule by late April, and then update my weekly schedule for GSoC,\nestimating 25-30 hours per week for this project currently.\n\n### Project Experience\n\n- One Student One Chip Project[2]\n  Extending the open-source NEMU simulator by implementing CPU cycle\n  functionalities in C.\n- Web Development\n  Developed a Django-based campus website, including user chat, news\n  publishing, and teacher management modules.\n- Custom Communication Protocols\n  Built a UDP-based chatroom with peer-to-peer and group messaging.\n- Stock Monitoring Tool\n  Implemented real-time monitoring and historical data analysis, with\n  email alerting and planned AI-driven strategy optimization.\n\nI have also obtained CCNA certification and gained hands-on experience\nas a network engineer. Additionally, I contributed a patch optimizing\nsend-email functionality in Git[3], giving me insights into the Git\ncodebase.\n\n## Appendix\n\n[1] https://gitlab.com/gitlab-org/git/-/issues/330\n[2] https://ysyx.oscc.cc/en/project/intro.html\n[3]https://lore.kernel.org/git/20250312064639.668875-1-05ZYT30@gmail.com/\n"},{"id":"515357","messageId":"Z-pjjQhtCjLvghGl@pks.im","threadId":"63180","inReplyTo":"20250329150248.2274482-1-05ZYT30@gmail.com","subject":"Re: [GSoC] git-refs proposal draft","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-03-31T09:42:37Z","receivedAt":"2025-03-31T09:42:46Z","isPatch":false,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Sat, Mar 29, 2025 at 11:02:46PM +0800, Zheng Yuting wrote:\n> ## Name and Contact Information\n> \n> - Full Name: Zheng Yuting\n> - Email Address: 05ZYT30@gmail.com\n> - Time Zone: UTC +8:00\n> \n> ---\n> \n> ## Abstract\n> \n> The current Git reference management functionality is fragmented across\n> multiple independent commands (git-show-ref, git-for-each-ref,\n> git-update-ref, git-pack-refs, git-check-ref-format, and\n> git-symbolic-ref), leading to code redundancy and increased maintenance\n> costs. Based on Patrick Steinhardt’s integration vision[1], this project\n> aims to introduce 8 new subcommands (list, exists, show, resolve, pack,\n> update, delete, check-format) under the existing git-refs command to\n> achieve the following objectives:\n\nI have a couple of opinions on the exact naming of the subcommands, more\non that below.\n\nIn any case, I don't think the naming and how exactly each of these\ncommands should look and work like needs to be hashed out in this\ndocument. It's nice to scope out _what_ we want to achieve and propose\nhow this could look like, but ultimately I think that most of the design\nshould happen during the project itself.\n\n> - Feature Integration: Consolidate existing reference management\n>   commands under git-refs, while maintaining backward compatibility.\n> - Feature Enhancement: Introduce recursion depth control for git-refs\n>   resolve.\n> - Testing & Documentation: Add test cases ensuring consistency and\n>   update relevant documentation.\n> \n> ---\n> \n> ## Implementation Plan\n> \n> ### Command Integration Strategy\n> \n> #### Design Goals\n> \n> The project will unify scattered reference management functionalities\n> under the git-refs subcommand framework, ensuring:\n> \n> 1. Complete Feature Coverage: Each subcommand fully replaces its\n>    corresponding legacy command.\n> 2. Parameter Compatibility: Preserve the semantics and output behavior\n>    of legacy command options.\n\nThis one is something that is up for debate. While I do expect that most\nof the commands should remain current semantics and options, we could\nalso use this as an opportunity to think whether there are any issues\nwith the current design and improve upon it.\n\n> 3. Code Reusability: Minimize redundancy by sharing underlying modules\n>    (e.g., refs/files-backend.c).\n> \n> #### Subcommand Mapping\n> \n> - git-refs list\n>   Replaces git-show-ref and git-for-each-ref, merging reference listing\n>   functionalities with support for formatting (--format), filtering\n>   (--heads, --tags), and sorting (--sort).\n\nYup. One thing to note is that git-show-ref(1) and git-for-each-ref(1)\nare very similar, but not quite the same. One should find good arguments\nwhich of the two semantics are preferable to us and why that is.\n\nFor example, git-show-ref(1) outperforms git-for-each-ref(1) due to the\ndefault format:\n\n    Benchmark 1: git show-ref\n      Time (mean ± σ):      99.0 ms ±   0.5 ms    [User: 55.6 ms, System: 43.0 ms]\n      Range (min … max):    98.0 ms … 100.8 ms    100 runs\n\n    Benchmark 2: git for-each-ref\n      Time (mean ± σ):     134.0 ms ±   0.6 ms    [User: 82.3 ms, System: 50.8 ms]\n      Range (min … max):   132.7 ms … 135.8 ms    100 runs\n\n    Summary\n      git show-ref ran\n        1.35 ± 0.01 times faster than git for-each-ref\n\n> - git-refs exists\n>   Replaces git-show-ref --exists, providing reference existence checks\n>   with positive (<ref>) and exclusion-based (--exclude-existing)\n>   verification.\n\nI'm not quite clear what exclusion-based existence checks is. How do you\ncheck whether something exists when you exclude it? I don't think that\nthis option is relevant in the context of `git refs exists`.\n\n> - git-refs show\n>   Replaces git-show-ref --verify, validating reference correctness with\n>   a strict mode (--strict).\n\nYup. In contrast to `git refs resolve` this command shouldn't resolve\nthe ref, but directly show what it's pointing to. And this should be\ntrue for both symbolic and normal refs.\n\n> - git-refs resolve\n>   Replaces git-symbolic-ref, resolving symbolic references with added\n>   recursion depth control (--max-depth), while retaining deletion (-d)\n>   and quiet mode (-q) options.\n\nNot quite. The difference to `git refs show` is that this command always\nresolves the ref to an object. So it's rather more similar to `git\nrev-parse --verify`, except that it only ever handles references.\n\n> - git-refs pack\n>   Replaces git-pack-refs, packing loose references with support for\n>   filtering (--include, --exclude) and automatic cleanup (--prune).\n\nI would probably call this `git refs optimize` or something like that.\ngit-pack-refs(1) is mostly called this way because it was introduced to\npack refs into the \"packed-refs\" file. But nowadays with the reftable\nbackend I think that the command name is somewhat inaccurate.\n\n> - git-refs update\n>   Replaces git-update-ref, providing transactional reference updates\n>   with batch processing (--stdin) and atomic guarantees.\n> - git-refs delete\n>   Separates the delete functionality from git-update-ref, ensuring\n>   explicit handling of reference removals with safety checks and batch\n>   operations (--stdin).\n\nIt's up for debate whether we should even have something like `git refs\ndelete`. As you rightfully notice `git refs update` already handles the\nusecase, so it feels like needless duplication.\n\n> - git-refs check-format\n>   Replaces git-check-ref-format, validating reference format with\n>   support for normalized output (--normalize).\n\nAh, nice, this is a command I forgot about.\n\n> #### Implementation Strategy\n> \n> 1. Option Parsing: Each subcommand will reuse the argument parsing\n>    logic from legacy commands (e.g., git-pack-refs --prune).\n\nWe cannot and do not want to do this for every case. As mentioned above,\nwe may want to iterate on some of the subcommands to address historic\nwarts. But overall I agree, we should of course aim to reduce\nduplication as far as it is sensible to do.\n\n> 2. Shared Backend Logic: Calls to common functions in refs/ (e.g.,\n>    reference traversal, locking mechanisms).\n> 3. Error Consistency: Maintain the same error codes and message\n>    formats as legacy commands.\n\nSame reasoning here, we may want to adapt some of them. The old commands\nwon't go away as they are used everywhere, and that makes it more\nreasonable for us to change behaviour in their newer equivalents.\n\n> ---\n> \n> ### Example: Implementing git-refs pack\n> \n> #### Functional Implementation\n> \n> 1. Modify builtin/refs.c:\n>    - Add cmd_refs_pack function implementing git-pack-refs logic.\n>    - Update cmd_refs to include pack with\n>      OPT_SUBCOMMAND(\"pack\", &fn, cmd_refs_pack).\n>    - Define REFS_PACK_USAGE:\n>      git refs pack [--all] [--no-prune] [--auto] [--include <pattern>]\n>      [--exclude <pattern>].\n> 2. Register New Subcommand in git.c:\n>    - Add { \"refs-pack\", cmd_refs_pack }, to the command array.\n\nYou don't actually have to change \"git.c\" to introduce new subcommands.\nWe don't want `git refs-pack`, but rather `git refs pack`, which is an\nimportant distinction.\n\n> 3. Reuse refs/files-backend.c Logic:\n>    - Ensure cmd_refs_pack calls pack_refs correctly, adjusting as\n>      necessary for new options.\n\nWe shouldn't have to touch any of the backends at all. You should rather\nmake sure to integrate with \"refs.c\", which wraps the backends and\nprovides a backend-agnostic interface to refs.\n\n> #### Testing Plan\n> \n> - Test Cases:\n>   Add t/txxx-refs-pack.sh, leveraging t/t0601-reffiles-pack-refs.sh\n>   scenarios to verify:\n>   - --prune removes obsolete references correctly.\n>   - --include and --exclude apply filtering as expected.\n>   - Packed references match legacy command outputs (diff .git/packed-refs).\n> - Performance Benchmarking (if needed):\n>   Add performance tests in t/perf to ensure no significant regression\n>   in execution time or memory usage.\n> \n> #### Documentation Updates\n> \n> - User Manual:\n>   Add a pack section to Documentation/git-refs.txt, mapping options to\n>   legacy command equivalents.\n> - Developer Notes:\n>   Comment code to highlight functional parity between git-refs pack\n>   and git-pack-refs.\n> \n> ---\n> \n> ### Timeline\n> \n> - May 8 - May 11 (4 days): Initial Testing & Subcommand Framework Setup\n> - May 12 - May 28 (17 days): pack Subcommand Implementation\n> - May 29 - June 14 (17 days): check-format Subcommand Development\n> - June 15 - July 5 (21 days): update and delete Subcommands Development\n> - July 6 - July 26 (21 days): show and exists Subcommands Development\n> - July 27 - August 16 (21 days): resolve Subcommand Implementation\n> - August 17 - September 6 (21 days): list Subcommand Implementation\n> - September 7 - September 16 (10 days): Mid-term Review\n> - September 17 - September 23 (7 days): Mentor Review & Final Adjustments\n\nYou probably underestimate the time to review and land a specific change\nquite significantly. Landing new features in ~2 weeks is thus not quite\nrealistic and you should allocate a lot more time for each of the\nspecific subcommands.\n\nThat of course raises the question of how to squeeze all of the\nsubcommands into a single GSoC. And the answer is that you don't: it's\nperfectly fine to implement only a subset of the new proposed\nsubcommands. I'd rather you spend more time thinking about how to\nimprove upon the status quo for each of the subcommands and thus spend\nmore time on it than trying to do everything in a hurry.\n\nSo: there isn't any expectation that you manage to implement all of\nthem. I'd recommend to pick a subset of commands that you want to\nimplement as a realistic goal. You may define other commands as a\nstretch goal in case you manage to speed through the implementation way\nfaster than I anticipate.\n\nThanks!\n\nPatrick\n"},{"id":"515433","messageId":"CAMvj1+qdBb-6nDVzw1y60-C5+wknJVr=JM+4ZiAftob3Ynbs5Q@mail.gmail.com","threadId":"63180","inReplyTo":"Z-pjjQhtCjLvghGl@pks.im","subject":"Re: [GSoC] git-refs proposal draft","fromName":"Yuting Zheng","fromEmail":"05zyt30@gmail.com","sentAt":"2025-04-01T13:37:50Z","receivedAt":"2025-04-01T13:38:03Z","isPatch":false,"sender":{"key":"05zyt30@gmail.com","avatar":"https://avatars.githubusercontent.com/u/87643662?v=4"},"body":"Hi Patrick,\n\nThanks for your feedback! Here are some adjustments based on your\nsuggestions:\n\n> In any case, I don't think the naming and how exactly each of these\n> commands should look and work like needs to be hashed out in this\n> document. It's nice to scope out _what_ we want to achieve and propose\n> how this could look like, but ultimately I think that most of the design\n> should happen during the project itself.\n\nOK! I may have misunderstood it. I will remove it.\n\n> This one is something that is up for debate. While I do expect that most\n> of the commands should remain current semantics and options, we could\n> also use this as an opportunity to think whether there are any issues\n> with the current design and improve upon it.\n\nSo, discussing the specific implementation of the command should also\nbe included in the proposal, right?\n\n>> - git-refs exists\n>>   Replaces git-show-ref --exists, providing reference existence checks\n>>   with positive (<ref>) and exclusion-based (--exclude-existing)\n>>   verification.\n>\n> I'm not quite clear what exclusion-based existence checks is. How do you\n> check whether something exists when you exclude it? I don't think that\n> this option is relevant in the context of `git refs exists`.\n\nSorry, I made a mistake. I meant to convey that the `--exclude-existing`\noption should be included in `git-refs list` (replacing\n`git-show-ref --exclude-existing`), which then lists refs within a certain\nscope.\n\n>> - git-refs resolve\n>>   Replaces git-symbolic-ref, resolving symbolic references with added\n>>   recursion depth control (--max-depth), while retaining deletion (-d)\n>>   and quiet mode (-q) options.\n>\n> Not quite. The difference to `git refs show` is that this command always\n> resolves the ref to an object. So it's rather more similar to `git\n> rev-parse --verify`, except that it only ever handles references.\n\nThanks for pointing that out. I will correct it afterward.\n\n>> - git-refs pack\n>>   Replaces git-pack-refs, packing loose references with support for\n>>   filtering (--include, --exclude) and automatic cleanup (--prune).\n>\n> I would probably call this `git refs optimize` or something like that.\n> git-pack-refs(1) is mostly called this way because it was introduced to\n> pack refs into the \"packed-refs\" file. But nowadays with the reftable\n> backend I think that the command name is somewhat inaccurate.\n\nAgree with it.\n\n>> - git-refs update\n>>   Replaces git-update-ref, providing transactional reference updates\n>>   with batch processing (--stdin) and atomic guarantees.\n>> - git-refs delete\n>>   Separates the delete functionality from git-update-ref, ensuring\n>>   explicit handling of reference removals with safety checks and batch\n>>   operations (--stdin).\n>\n> It's up for debate whether we should even have something like `git refs\n> delete`. As you rightfully notice `git refs update` already handles the\n> usecase, so it feels like needless duplication.\n>\n\nI think maybe separate `update` and `delete` can be more direct. Separating\nthese commands can enhance clarity in their usage, although I'm open to\nfurther discussion if the community prefers a unified command.\n\n>> 1. Option Parsing: Each subcommand will reuse the argument parsing\n>>    logic from legacy commands (e.g., git-pack-refs --prune).\n>\n> We cannot and do not want to do this for every case. As mentioned above,\n> we may want to iterate on some of the subcommands to address historic\n> warts. But overall I agree, we should of course aim to reduce\n> duplication as far as it is sensible to do.\n\n>\n>> 2. Shared Backend Logic: Calls to common functions in refs/ (e.g.,\n>>    reference traversal, locking mechanisms).\n>> 3. Error Consistency: Maintain the same error codes and message\n>>    formats as legacy commands.\n>\n> Same reasoning here, we may want to adapt some of them. The old commands\n> won't go away as they are used everywhere, and that makes it more\n> reasonable for us to change behaviour in their newer equivalents.\n>\n\nGot it. I will list my thoughts below.\n\n> You don't actually have to change \"git.c\" to introduce new subcommands.\n> We don't want `git refs-pack`, but rather `git refs pack`, which is an\n> important distinction.\n\nSorry for my oversight. I will be more careful from now on.\n\n>> 3. Reuse refs/files-backend.c Logic:\n>>    - Ensure cmd_refs_pack calls pack_refs correctly, adjusting as\n>>      necessary for new options.\n>\n> We shouldn't have to touch any of the backends at all. You should rather\n> make sure to integrate with \"refs.c\", which wraps the backends and\n> provides a backend-agnostic interface to refs.\n\nGot it.\n\n> You probably underestimate the time to review and land a specific change\n> quite significantly. Landing new features in ~2 weeks is thus not quite\n> realistic and you should allocate a lot more time for each of the\n> specific subcommands.\n>\n> That of course raises the question of how to squeeze all of the\n> subcommands into a single GSoC. And the answer is that you don't: it's\n> perfectly fine to implement only a subset of the new proposed\n> subcommands. I'd rather you spend more time thinking about how to\n> improve upon the status quo for each of the subcommands and thus spend\n> more time on it than trying to do everything in a hurry.\n>\n\nThanks for your reminder! I plan to focus on implementing `git-refs list` and\n`git-refs update` first. These will form the foundation of the new design, and\nonce stable, I will consider addressing `git-refs resolve` and additional\ncommands if time permits.\n\nSo, I need to update my proposal to reduce the number of subcommands so\nthat I can complete this project with high quality. I also need to\nfurther discuss\nthe implications of these commands. By reducing the number of subcommands,\nI can dedicate more time to refining each one and ensuring they integrate well\nwith the existing system. I will also detail the implications of each command in\nmy updated proposal.\n\nThanks!\nZheng Yuting\n"},{"id":"515488","messageId":"Z-zvGzD-mdwmaYrX@pks.im","threadId":"63180","inReplyTo":"CAMvj1+qdBb-6nDVzw1y60-C5+wknJVr=JM+4ZiAftob3Ynbs5Q@mail.gmail.com","subject":"Re: [GSoC] git-refs proposal draft","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-02T08:02:35Z","receivedAt":"2025-04-02T08:02:39Z","isPatch":false,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Tue, Apr 01, 2025 at 09:37:50PM +0800, Yuting Zheng wrote:\n> Hi Patrick,\n> \n> Thanks for your feedback! Here are some adjustments based on your\n> suggestions:\n> \n> > In any case, I don't think the naming and how exactly each of these\n> > commands should look and work like needs to be hashed out in this\n> > document. It's nice to scope out _what_ we want to achieve and propose\n> > how this could look like, but ultimately I think that most of the design\n> > should happen during the project itself.\n> \n> OK! I may have misunderstood it. I will remove it.\n> \n> > This one is something that is up for debate. While I do expect that most\n> > of the commands should remain current semantics and options, we could\n> > also use this as an opportunity to think whether there are any issues\n> > with the current design and improve upon it.\n> \n> So, discussing the specific implementation of the command should also\n> be included in the proposal, right?\n\nAt least the general direction should become clear, yes. The intent is\nthat we want to double check that the candidate has indeed invested a\nbit of time to understand the problem space and what is being asked of\nthem. So you don't have to provide all the nitty-gritty details of how\nexactly you plan on doing the conversion, but provide a bit of an\noverview of what the project would entail.\n\n> >> - git-refs exists\n> >>   Replaces git-show-ref --exists, providing reference existence checks\n> >>   with positive (<ref>) and exclusion-based (--exclude-existing)\n> >>   verification.\n> >\n> > I'm not quite clear what exclusion-based existence checks is. How do you\n> > check whether something exists when you exclude it? I don't think that\n> > this option is relevant in the context of `git refs exists`.\n> \n> Sorry, I made a mistake. I meant to convey that the `--exclude-existing`\n> option should be included in `git-refs list` (replacing\n> `git-show-ref --exclude-existing`), which then lists refs within a certain\n> scope.\n\nNo need to be sorry, we all do mistakes.\n\n[snip]\n> >> - git-refs update\n> >>   Replaces git-update-ref, providing transactional reference updates\n> >>   with batch processing (--stdin) and atomic guarantees.\n> >> - git-refs delete\n> >>   Separates the delete functionality from git-update-ref, ensuring\n> >>   explicit handling of reference removals with safety checks and batch\n> >>   operations (--stdin).\n> >\n> > It's up for debate whether we should even have something like `git refs\n> > delete`. As you rightfully notice `git refs update` already handles the\n> > usecase, so it feels like needless duplication.\n> >\n> \n> I think maybe separate `update` and `delete` can be more direct. Separating\n> these commands can enhance clarity in their usage, although I'm open to\n> further discussion if the community prefers a unified command.\n\n`update` will have to support deletions regardless as you won't be able\nto do atomic updates of many refs at once if that update would include a\ndeletion. So let's start with that, and then we can still figure out\nwhether `delete` would be desirable.\n\n> > You probably underestimate the time to review and land a specific change\n> > quite significantly. Landing new features in ~2 weeks is thus not quite\n> > realistic and you should allocate a lot more time for each of the\n> > specific subcommands.\n> >\n> > That of course raises the question of how to squeeze all of the\n> > subcommands into a single GSoC. And the answer is that you don't: it's\n> > perfectly fine to implement only a subset of the new proposed\n> > subcommands. I'd rather you spend more time thinking about how to\n> > improve upon the status quo for each of the subcommands and thus spend\n> > more time on it than trying to do everything in a hurry.\n> >\n> \n> Thanks for your reminder! I plan to focus on implementing `git-refs list` and\n> `git-refs update` first. These will form the foundation of the new design, and\n> once stable, I will consider addressing `git-refs resolve` and additional\n> commands if time permits.\n> \n> So, I need to update my proposal to reduce the number of subcommands so\n> that I can complete this project with high quality. I also need to\n> further discuss\n> the implications of these commands. By reducing the number of subcommands,\n> I can dedicate more time to refining each one and ensuring they integrate well\n> with the existing system. I will also detail the implications of each command in\n> my updated proposal.\n\nGreat, thanks!\n\nPatrick\n"},{"id":"515593","messageId":"20250403154404.3459805-1-05ZYT30@gmail.com","threadId":"63180","inReplyTo":"20250329150248.2274482-1-05ZYT30@gmail.com","subject":"Discussion on git-refs list Implementation and Possible Approaches","fromName":"Zheng Yuting","fromEmail":"05zyt30@gmail.com","sentAt":"2025-04-03T15:44:04Z","receivedAt":"2025-04-03T15:44:40Z","isPatch":false,"sender":{"key":"05zyt30@gmail.com","avatar":"https://avatars.githubusercontent.com/u/87643662?v=4"},"body":"After an initial review of the code and documentation for `git-show-ref`\nand `git-for-each-ref`, I believe the functionality of the `git-refs list`\nsubcommand can be categorized into two major types:\n\n1. **Filtering options**\n   - In `git-for-each-ref`:\n     - `--count`\n     - `--sort=<key>`\n     - `--points-at=<object>`\n     - `--merged[=<object>]`\n     - `--no-merged[=<object>]`\n     - `--contains[=<object>]`\n     - `--no-contains[=<object>]`\n     - `--omit-empty`\n     - `--exclude=<pattern>`\n     - `--include-root-refs`\n   - In `git-show-ref`:\n     - `--head`\n     - `--branches`\n     - `--tags`\n     - `--exclude-existing[=<pattern>]`\n\n2. **Formatting options**\n   - In `git-for-each-ref`:\n     - `--format=<format>`\n     - `--color[=<when>]`\n     - `--tcl`\n     - `--shell`\n     - `--perl`\n   - In `git-show-ref`:\n     - `--dereference`\n     - `--hash`\n\nAdditionally, for filtering functionality, the `--ignore-case` option\nfrom `git-for-each-ref` should be supported across the board.\n\n**Note**: The `--verify`, `--quiet` and `--exist` options in\n`git-show-ref` are intended to be implemented as separate\n`git-refs` subcommands and are not within the scope of this\ndiscussion.\n\n## Implementation Considerations\n\nAt this point, I haven't come up with a perfect implementation\nplan, as each approach has some issues:\n\n### Approach 1:\n`git-refs list` would support both filtering and formatting options,\nmeaning it could provide:\n- Filtered output\n- Formatted output\n- Combined filter + format output\n\nHowever, I see two potential problems with this approach:\n1. Would it make the `list` subcommand too complex?\n2. The performance could be worse than `git-for-each-ref`.\n\n### Approach 2:\nSplit the functionality into two separate subcommands:\n- `git-refs filter`: Handles filtering and filter + format output\n- `git-refs show`: Supports formatting options\n\nFor implementation, my initial thought is that `git-refs filter` could\nreuse the formatting options from `git-refs show`. Perhaps this could\nwork similarly to how `git-add --patch` and `git-restore --patch`\nshare logic, though I haven’t thoroughly reviewed that part of the\ncode yet. Would this be a reasonable approach?\n\n## Overall Plan\n\nIf Approach 2 is preferable, I could start with `git-refs show` since it\nonly deals with basic ref listing and formatting. I would then make\nthe formatting code more reusable to support `git-refs filter`, which\nwould focus solely on filtering.\n\nIf Approach 1 is chosen, the implementation plan would remain the\nsame, but everything would be handled within a single `git-refs list`\ncommand.\n\nI would appreciate any feedback or alternative suggestions on the\nbest way to structure this functionality.\n\nThanks!\n"},{"id":"515657","messageId":"CAOLa=ZTTPuNyaE5Z-bfkQougmKQSrRZZwLaxJUL7mdmj8uHoFw@mail.gmail.com","threadId":"63180","inReplyTo":"20250403154404.3459805-1-05ZYT30@gmail.com","subject":"Re: Discussion on git-refs list Implementation and Possible Approaches","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-04-04T11:08:26Z","receivedAt":"2025-04-04T11:08:28Z","isPatch":false,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Zheng Yuting <05zyt30@gmail.com> writes:\n\n> After an initial review of the code and documentation for `git-show-ref`\n> and `git-for-each-ref`, I believe the functionality of the `git-refs list`\n> subcommand can be categorized into two major types:\n>\n> 1. **Filtering options**\n>    - In `git-for-each-ref`:\n>      - `--count`\n>      - `--sort=<key>`\n\nI would categorize '--sort' into a third subcategory. Filtering refers\nto possible change in the size of the sample set. While sorting is more\nof a presentation utility.\n\n>      - `--points-at=<object>`\n>      - `--merged[=<object>]`\n>      - `--no-merged[=<object>]`\n>      - `--contains[=<object>]`\n>      - `--no-contains[=<object>]`\n>      - `--omit-empty`\n>      - `--exclude=<pattern>`\n>      - `--include-root-refs`\n>    - In `git-show-ref`:\n>      - `--head`\n>      - `--branches`\n>      - `--tags`\n>      - `--exclude-existing[=<pattern>]`\n>\n> 2. **Formatting options**\n>    - In `git-for-each-ref`:\n>      - `--format=<format>`\n>      - `--color[=<when>]`\n>      - `--tcl`\n>      - `--shell`\n>      - `--perl`\n>    - In `git-show-ref`:\n>      - `--dereference`\n>      - `--hash`\n>\n> Additionally, for filtering functionality, the `--ignore-case` option\n> from `git-for-each-ref` should be supported across the board.\n>\n\nThis is indeed a special case which applies to both sorting and\nfiltering.\n\n> **Note**: The `--verify`, `--quiet` and `--exist` options in\n> `git-show-ref` are intended to be implemented as separate\n> `git-refs` subcommands and are not within the scope of this\n> discussion.\n>\n>\n> ## Implementation Considerations\n>\n> At this point, I haven't come up with a perfect implementation\n> plan, as each approach has some issues:\n>\n> ### Approach 1:\n> `git-refs list` would support both filtering and formatting options,\n> meaning it could provide:\n> - Filtered output\n> - Formatted output\n> - Combined filter + format output\n>\n> However, I see two potential problems with this approach:\n> 1. Would it make the `list` subcommand too complex?\n\nYou mean complex from the user perspective of having too many options or\nfrom the implementation perspective.\n\nI think from the UX perspective, it is a good time to rethink usage and\nneed for the options you mentioned above. , for e.g. with '--format', do\nwe need to have '--tcl', `--shell` and `--perl`?\n\n> 2. The performance could be worse than `git-for-each-ref`.\n>\n\nWhy would it be worse? The performance difference between\n`git-for-each-ref(1)` and `git-show-ref(1)` stem from the formats they\nuse by default.\n\n$ hyperfine --shell=none --warmup=3 \"git for-each-ref\" \"git show-ref\"\nBenchmark 1: git for-each-ref\n  Time (mean ± σ):       4.0 ms ±   0.6 ms    [User: 1.9 ms, System: 1.9 ms]\n  Range (min … max):     3.0 ms …   5.7 ms    680 runs\n\nBenchmark 2: git show-ref\n  Time (mean ± σ):       2.9 ms ±   0.4 ms    [User: 1.2 ms, System: 1.5 ms]\n  Range (min … max):     2.0 ms …   4.3 ms    909 runs\n\nSummary\n  git show-ref ran\n    1.38 ± 0.28 times faster than git for-each-ref\n\nWhat I found interesting was that changing the format for\n'git-for-each-ref(1)' gives it a boost:\n\n$ hyperfine --shell=none --warmup=3 'git for-each-ref\n--format=\"%(objectname) %(refname)\"' \"git show-ref\"\nBenchmark 1: git for-each-ref --format=\"%(objectname) %(refname)\"\n  Time (mean ± σ):       2.4 ms ±   0.3 ms    [User: 1.1 ms, System: 1.1 ms]\n  Range (min … max):     1.7 ms …   3.6 ms    1070 runs\n\nBenchmark 2: git show-ref\n  Time (mean ± σ):       2.9 ms ±   0.4 ms    [User: 1.2 ms, System: 1.5 ms]\n  Range (min … max):     2.0 ms …   4.5 ms    833 runs\n\nSummary\n  git for-each-ref --format=\"%(objectname) %(refname)\" ran\n    1.20 ± 0.23 times faster than git show-ref\n\n> ### Approach 2:\n> Split the functionality into two separate subcommands:\n> - `git-refs filter`: Handles filtering and filter + format output\n> - `git-refs show`: Supports formatting options\n>\n> For implementation, my initial thought is that `git-refs filter` could\n> reuse the formatting options from `git-refs show`. Perhaps this could\n> work similarly to how `git-add --patch` and `git-restore --patch`\n> share logic, though I haven’t thoroughly reviewed that part of the\n> code yet. Would this be a reasonable approach?\n>\n\nAnd what is the expectation that when you want to do both filtering and\nformatting, would the user be expected to do `git refs filter | git refs\nshow`? Generally users want to combine both of these options.\n\nAlso wasn't the idea to already implement `git-refs show` as a\nstandalone which simply shows what value a reference holds (without\nderefence)?\n\n> ## Overall Plan\n>\n> If Approach 2 is preferable, I could start with `git-refs show` since it\n> only deals with basic ref listing and formatting. I would then make\n> the formatting code more reusable to support `git-refs filter`, which\n> would focus solely on filtering.\n>\n> If Approach 1 is chosen, the implementation plan would remain the\n> same, but everything would be handled within a single `git-refs list`\n> command.\n\nWhile I would think Approach 1 is the better option here, I'm also\nseeing how it is complex, perhaps a good option to get started would be\nto implement a simpler subcommand as a first case? Perhaps the\noriginally discussed `git refs show`?\n\n>\n> I would appreciate any feedback or alternative suggestions on the\n> best way to structure this functionality.\n>\n> Thanks!\n\nThanks for the proposal!\nKarthik\n"},{"id":"515658","messageId":"Z--_TvQ9MXgjxqOV@pks.im","threadId":"63180","inReplyTo":"20250403154404.3459805-1-05ZYT30@gmail.com","subject":"Re: Discussion on git-refs list Implementation and Possible Approaches","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-04T11:15:26Z","receivedAt":"2025-04-04T11:15:34Z","isPatch":false,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Thu, Apr 03, 2025 at 11:44:04PM +0800, Zheng Yuting wrote:\n> After an initial review of the code and documentation for `git-show-ref`\n> and `git-for-each-ref`, I believe the functionality of the `git-refs list`\n> subcommand can be categorized into two major types:\n> \n> 1. **Filtering options**\n>    - In `git-for-each-ref`:\n>      - `--count`\n>      - `--sort=<key>`\n>      - `--points-at=<object>`\n>      - `--merged[=<object>]`\n>      - `--no-merged[=<object>]`\n>      - `--contains[=<object>]`\n>      - `--no-contains[=<object>]`\n>      - `--omit-empty`\n>      - `--exclude=<pattern>`\n>      - `--include-root-refs`\n>    - In `git-show-ref`:\n>      - `--head`\n>      - `--branches`\n>      - `--tags`\n>      - `--exclude-existing[=<pattern>]`\n> \n> 2. **Formatting options**\n>    - In `git-for-each-ref`:\n>      - `--format=<format>`\n>      - `--color[=<when>]`\n>      - `--tcl`\n>      - `--shell`\n>      - `--perl`\n>    - In `git-show-ref`:\n>      - `--dereference`\n>      - `--hash`\n> \n> Additionally, for filtering functionality, the `--ignore-case` option\n> from `git-for-each-ref` should be supported across the board.\n> \n> **Note**: The `--verify`, `--quiet` and `--exist` options in\n> `git-show-ref` are intended to be implemented as separate\n> `git-refs` subcommands and are not within the scope of this\n> discussion.\n\nYup, makes sense.\n\nAnother factor is the default format that these two commands use which\ndiffers. I would heavily lean towards using the format exposed by `git\nshow-ref` because it doesn't require us to hit the ODB, and thus it is\nway more efficient. This has bitten me quite often already.\n\n> ## Implementation Considerations\n> \n> At this point, I haven't come up with a perfect implementation\n> plan, as each approach has some issues:\n> \n> ### Approach 1:\n> `git-refs list` would support both filtering and formatting options,\n> meaning it could provide:\n> - Filtered output\n> - Formatted output\n> - Combined filter + format output\n> \n> However, I see two potential problems with this approach:\n> 1. Would it make the `list` subcommand too complex?\n\nI don't think it would, both are orthogonal to one another. I don't\nthink people _only_ want to format or _only_ want to filter. Quite\noften, they'll want to do both at the same time.\n\n> 2. The performance could be worse than `git-for-each-ref`.\n\nWhy is that? git-for-each-ref(1) already knows to filter and format, so\nI'd expect the performance to be roughly the same. In fact, I think we\nwould be able to improve performance if we changed the default format as\nmentioned above.\n\n> ### Approach 2:\n> Split the functionality into two separate subcommands:\n> - `git-refs filter`: Handles filtering and filter + format output\n> - `git-refs show`: Supports formatting options\n> \n> For implementation, my initial thought is that `git-refs filter` could\n> reuse the formatting options from `git-refs show`. Perhaps this could\n> work similarly to how `git-add --patch` and `git-restore --patch`\n> share logic, though I haven’t thoroughly reviewed that part of the\n> code yet. Would this be a reasonable approach?\n\nI don't think this plan would make sense as it would mean that current\nusers of git-for-each-ref(1) wouldn't be able to migrate.\n\nPatrick\n"},{"id":"515674","messageId":"CAMvj1+qx8DgNp7kp==YNT6eTmmdA-zyNuYRuEovk5L+eqGw8xQ@mail.gmail.com","threadId":"63180","inReplyTo":"20250403154404.3459805-1-05ZYT30@gmail.com","subject":"Re: Discussion on git-refs list Implementation and Possible Approaches","fromName":"Yuting Zheng","fromEmail":"05zyt30@gmail.com","sentAt":"2025-04-04T15:16:42Z","receivedAt":"2025-04-04T15:16:57Z","isPatch":false,"sender":{"key":"05zyt30@gmail.com","avatar":"https://avatars.githubusercontent.com/u/87643662?v=4"},"body":"Hi everyone,\n\nFollowing the initial discussion, I’ve updated the design for the\n`git-refs list` subcommand. Below are the key changes and a\ndiscussion about subcommand options.\n\n### `git-refs list implement plan`\n\n1. Output Format:\n\nThe default output format now follows the `git-show-ref` style:\n`<oid> SP <ref> LF`. This avoids dependency on ODB and aligns with\nlightweight ref listing.\n\n2. Option Categorization:\n\nThe functionality is now divided into three distinct types of options\n(filter, sort, format) that can be combined:\n\n2.1. **Filtering options**\n   - In `git-for-each-ref`:\n     - `--count`\n     - `--points-at=<object>`\n     - `--merged[=<object>]`\n     - `--no-merged[=<object>]`\n     - `--contains[=<object>]`\n     - `--no-contains[=<object>]`\n     - `--omit-empty`\n     - `--exclude=<pattern>`\n     - `--include-root-refs`\n   - In `git-show-ref`:\n     - `--head`\n     - `--branches`\n     - `--tags`\n     - `--exclude-existing[=<pattern>]`\n\n2.2. **Sorting options**\n   - In `git-for-each-ref`:\n     - `--sort=<key>`\n\n2.3. **Formatting options**\n   - In `git-for-each-ref`:\n     - `--format=<format>`\n     - `--color[=<when>]`\n     - `--tcl`\n     - `--shell`\n     - `--perl`\n   - In `git-show-ref`:\n     - `--dereference`\n     - `--hash`\n\nAdditionally, for filtering and sorting functionality, the\n`--ignore-case` option from `git-for-each-ref` should be\nsupported across the board.\n\n**Note**: The `--verify`, `--quiet` and `--exist` options in\n`git-show-ref` are intended to be implemented as separate\n`git-refs` subcommands and are not within the scope of this\ndiscussion.\n\n3. Implementation Approach:\n\n> ### Approach 1:\n> `git-refs list` would support both filtering and formatting options,\n> meaning it could provide:\n> - Filtered output\n> - Formatted output\n> - Combined filter + format output\n>\n\nI will proceed with Approach 1 by implementing `git-refs list` as a\nsingle subcommand that combines filtering, sorting, and formatting\ncapabilities. To establish a foundation for this, I will first develop\n`git-refs show` as a standalone subcommand to replace\n`git-show-ref --verify`. The `git-refs list` functionality will then be built\non top of the `git-refs show` codebase.\"\n\n## Discussion About Options\n\n1. Legacy Formatting Options:\n\nShould `--tcl`, `--shell`, `--perl` be retained?\n\n2. New Options:\n\nHave you used these legacy options or needed modern alternatives?\nAny pain points?\n\nI would appreciate any feedback or alternative suggestions on the\nbest way to structure this functionality.\n\nThanks!\nZheng Yuting\n"},{"id":"515675","messageId":"CAMvj1+qpX8Q2nV62zLAut14-2w399y2V-eJmGc7J+amtJ7d1VA@mail.gmail.com","threadId":"63180","inReplyTo":"CAMvj1+rMY2YR8_GGFeDoJ6HCiVDusZZk9fAguKh=kbctHO=2Qg@mail.gmail.com","subject":"Fwd: Discussion on git-refs list Implementation and Possible Approaches","fromName":"Yuting Zheng","fromEmail":"05zyt30@gmail.com","sentAt":"2025-04-04T15:20:10Z","receivedAt":"2025-04-04T15:20:23Z","isPatch":false,"sender":{"key":"05zyt30@gmail.com","avatar":"https://avatars.githubusercontent.com/u/87643662?v=4"},"body":"On Fri, Apr 4, 2025 at 10:48 PM Yuting Zheng <05zyt30@gmail.com> wrote:\n>\n> Thanks for your review!\n>\n> > Another factor is the default format that these two commands use which\n> > differs. I would heavily lean towards using the format exposed by `git\n> > show-ref` because it doesn't require us to hit the ODB, and thus it is\n> > way more efficient. This has bitten me quite often already.\n>\n> Thanks for your reminder! I will explain this output format in my next\n> proposal, and I agree that we should adopt the `git show-ref` format for\n> its superior efficiency.\n>\n> > I don't think it would, both are orthogonal to one another. I don't\n> > think people _only_ want to format or _only_ want to filter. Quite\n> > often, they'll want to do both at the same time.\n> >\n>\n> On the topic of filtering and formatting, I plan to implement these as\n> basic functions that work together seamlessly. In other words, the filter\n> and format functionalities will be integrated (without being exposed as\n> separate options) so that users can combine them as needed. I will\n> submit another email for further discussion about options.\n>\n> > > 2. The performance could be worse than `git-for-each-ref`.\n> >\n> > Why is that? git-for-each-ref(1) already knows to filter and format, so\n> > I'd expect the performance to be roughly the same. In fact, I think we\n> > would be able to improve performance if we changed the default format as\n> > mentioned above.\n> >\n>\n> I am concerned that iterating over all available options might introduce\n> additional overhead.\n>\n> >\n> > I don't think this plan would make sense as it would mean that current\n> > users of git-for-each-ref(1) wouldn't be able to migrate.\n> >\n>\n> Finally, in light of your feedback and Karthik’s, I have decided that\n> Approach 1 will be my final plan.\n>\n> Thanks !\n> Zheng Yuting\n"},{"id":"515676","messageId":"CAMvj1+paWq5LV1imUz0HcQh1eoGvdfkYi0B5FPV33Xt-OUe1Dg@mail.gmail.com","threadId":"63180","inReplyTo":"CAOLa=ZTTPuNyaE5Z-bfkQougmKQSrRZZwLaxJUL7mdmj8uHoFw@mail.gmail.com","subject":"Re: Discussion on git-refs list Implementation and Possible Approaches","fromName":"Yuting Zheng","fromEmail":"05zyt30@gmail.com","sentAt":"2025-04-04T15:25:45Z","receivedAt":"2025-04-04T15:25:58Z","isPatch":false,"sender":{"key":"05zyt30@gmail.com","avatar":"https://avatars.githubusercontent.com/u/87643662?v=4"},"body":"Thanks for your reply!\n\n> I would categorize '--sort' into a third subcategory. Filtering refers\n> to possible change in the size of the sample set. While sorting is more\n> of a presentation utility.\n>\n\nThat’s a good idea, it makes my plan more clear. I will separate\nthe “--filter” options into “--filter” and “--sort” so that users can clearly\ndistinguish them.\n\n> This is indeed a special case which applies to both sorting and\n> filtering.\n\nUnderstood!\n\n>\n> You mean complex from the user perspective of having too many options or\n> from the implementation perspective.\n>\n> I think from the UX perspective, it is a good time to rethink usage and\n> need for the options you mentioned above. , for e.g. with '--format', do\n> we need to have '--tcl', `--shell` and `--perl`?\n>\n\nI think it’s important to discuss all available options, and I will\nsubmit another\nemail for further discussion.\n\n\n> > 2. The performance could be worse than `git-for-each-ref`.\n> >\n>\n> Why would it be worse? The performance difference between\n> `git-for-each-ref(1)` and `git-show-ref(1)` stem from the formats they\n> use by default.\n>\n> $ hyperfine --shell=none --warmup=3 \"git for-each-ref\" \"git show-ref\"\n> Benchmark 1: git for-each-ref\n>   Time (mean ± σ):       4.0 ms ±   0.6 ms    [User: 1.9 ms, System: 1.9 ms]\n>   Range (min … max):     3.0 ms …   5.7 ms    680 runs\n>\n> Benchmark 2: git show-ref\n>   Time (mean ± σ):       2.9 ms ±   0.4 ms    [User: 1.2 ms, System: 1.5 ms]\n>   Range (min … max):     2.0 ms …   4.3 ms    909 runs\n>\n> Summary\n>   git show-ref ran\n>     1.38 ± 0.28 times faster than git for-each-ref\n>\n> What I found interesting was that changing the format for\n> 'git-for-each-ref(1)' gives it a boost:\n>\n> $ hyperfine --shell=none --warmup=3 'git for-each-ref\n> --format=\"%(objectname) %(refname)\"' \"git show-ref\"\n> Benchmark 1: git for-each-ref --format=\"%(objectname) %(refname)\"\n>   Time (mean ± σ):       2.4 ms ±   0.3 ms    [User: 1.1 ms, System: 1.1 ms]\n>   Range (min … max):     1.7 ms …   3.6 ms    1070 runs\n>\n> Benchmark 2: git show-ref\n>   Time (mean ± σ):       2.9 ms ±   0.4 ms    [User: 1.2 ms, System: 1.5 ms]\n>   Range (min … max):     2.0 ms …   4.5 ms    833 runs\n>\n> Summary\n>   git for-each-ref --format=\"%(objectname) %(refname)\" ran\n>     1.20 ± 0.23 times faster than git show-ref\n>\n\nThank you for the reminder. Once each option is implemented, I will test its\nperformance to ensure that it maintains—or improves upon—the efficiency\nof the previous version.\n\n>\n> And what is the expectation that when you want to do both filtering and\n> formatting, would the user be expected to do `git refs filter | git refs\n> show`? Generally users want to combine both of these options.\n>\n> Also wasn't the idea to already implement `git-refs show` as a\n> standalone which simply shows what value a reference holds (without\n> derefence)?\n>\n> While I would think Approach 1 is the better option here, I'm also\n> seeing how it is complex, perhaps a good option to get started would be\n> to implement a simpler subcommand as a first case? Perhaps the\n> originally discussed `git refs show`?\n\nI agree that implementing `git-refs show` first would provide a solid foundation\nfor other options. I will add these improvements in the next version\nof the proposal.\n\nThanks!\nZheng Yuting\n"},{"id":"515677","messageId":"CAMvj1+qer9--SteiYM+ZLJ2evJos-MGC_RPssHDhB-FwYaWPyw@mail.gmail.com","threadId":"63180","inReplyTo":"Z--_TvQ9MXgjxqOV@pks.im","subject":"Re: Discussion on git-refs list Implementation and Possible Approaches","fromName":"Yuting Zheng","fromEmail":"05zyt30@gmail.com","sentAt":"2025-04-04T15:26:45Z","receivedAt":"2025-04-04T15:26:58Z","isPatch":false,"sender":{"key":"05zyt30@gmail.com","avatar":"https://avatars.githubusercontent.com/u/87643662?v=4"},"body":"Thanks for your review!\n\n> Another factor is the default format that these two commands use which\n> differs. I would heavily lean towards using the format exposed by `git\n> show-ref` because it doesn't require us to hit the ODB, and thus it is\n> way more efficient. This has bitten me quite often already.\n\nThanks for your reminder! I will explain this output format in my next\nproposal, and I agree that we should adopt the `git show-ref` format for\nits superior efficiency.\n\n> I don't think it would, both are orthogonal to one another. I don't\n> think people _only_ want to format or _only_ want to filter. Quite\n> often, they'll want to do both at the same time.\n>\n\nOn the topic of filtering and formatting, I plan to implement these as\nbasic functions that work together seamlessly. In other words, the filter\nand format functionalities will be integrated (without being exposed as\nseparate options) so that users can combine them as needed. I will\nsubmit another email for further discussion about options.\n\n> > 2. The performance could be worse than `git-for-each-ref`.\n>\n> Why is that? git-for-each-ref(1) already knows to filter and format, so\n> I'd expect the performance to be roughly the same. In fact, I think we\n> would be able to improve performance if we changed the default format as\n> mentioned above.\n>\n\nI am concerned that iterating over all available options might introduce\nadditional overhead.\n\n>\n> I don't think this plan would make sense as it would mean that current\n> users of git-for-each-ref(1) wouldn't be able to migrate.\n>\n\nFinally, in light of your feedback and Karthik’s, I have decided that\nApproach 1 will be my final plan.\n\nThanks !\nZheng Yuting\n"},{"id":"515715","messageId":"CAMvj1+rUYPpOzo78RJurj4Lcoop=hPQ0G6_e2eK6TD55tdGPkg@mail.gmail.com","threadId":"63180","inReplyTo":"20250329150248.2274482-1-05ZYT30@gmail.com","subject":"[GSoC] git-refs proposal v2","fromName":"Yuting Zheng","fromEmail":"05zyt30@gmail.com","sentAt":"2025-04-06T06:08:10Z","receivedAt":"2025-04-06T06:08:23Z","isPatch":false,"sender":{"key":"05zyt30@gmail.com","avatar":"https://avatars.githubusercontent.com/u/87643662?v=4"},"body":"## Name and Contact Information\n\n- Full Name: Zheng Yuting\n- Email Address: 05ZYT30@gmail.com\n- Time Zone: UTC +8:00\n\n---\n\n## Abstract\n\nThe current Git reference management functionality is fragmented across\nmultiple independent commands (`git-show-ref`, `git-for-each-ref`,\n`git-update-ref`,\n`git-pack-refs`, `git-check-ref-format`, and `git-symbolic-ref`),\nleading to code\nredundancy and increased maintenance costs.\nBased on Patrick Steinhardt’s integration vision[1], this project aims to\nconsolidate functionality under the unified `git-refs` command by initially\nimplementing three core subcommands: **show**, **list**, and **update**.\nThese subcommands will cover the most essential reference management\noperations while ensuring backward compatibility and laying the foundation\nfor further refinement.\n\nIf time permits, additional subcommands (such as `exists`, `resolve`, `pack`,\nand `check-format`) will be gradually integrated to extend and enhance the\nexisting functionality. Comprehensive testing and updated documentation\nwill support this phased approach, ensuring a robust transition from the\nlegacy tools.\n\n---\n\n## Implementation Plan\n\n### Command Integration Strategy\n\n#### Implementation Sequence\n\nThe development will proceed in the following order:\n\n1. `git-refs show`\n   - **Purpose:** Replace `git-show-ref --verify` with strict\nreference validation.\n\n2. `git-refs list`\n   - **Purpose:** Merge `git-show-ref` and `git-for-each-ref` for\nlisting references.\n   - **Output Format:** `<oid> SP <ref> LF` (git-show-ref style).\n   - **Options:**\n     - **Filtering:**\n       - From `git-for-each-ref`:\n         - `--count`,\n         - `--points-at=<object>`,\n         - `--merged[=<object>]`,\n         - `--no-merged[=<object>]`,\n         - `--contains[=<object>]`,\n         - `--no-contains[=<object>]`,\n         - `--omit-empty`,\n         - `--exclude=<pattern>`,\n         - `--include-root-refs`.\n       - From `git-show-ref`:\n         - `--head`,\n         - `--branches`,\n         - `--tags`,\n         - `--exclude-existing`.\n     - **Sorting:**\n       - From `git-for-each-ref`: `--sort=<key>`.\n     - **Formatting:**\n       - From `git-for-each-ref`:\n         - `--format=<format>`,\n         - `--color[=<when>]`,\n         - `--tcl` (under discussion),\n         - `--shell`(under discussion),\n         - `--perl`(under discussion).\n       - From `git-show-ref`:\n         - `--dereference`,\n         - `--hash`.\n     - **Global:** `--ignore-case` (applies to all filtering/sorting).\n\n3. `git-refs update`\n   - **Purpose:** Replace `git-update-ref` with transactional updates and\nbatch processing.\n   - **Options (all from `git-update-ref`):**\n     - `<ref>`: Target reference.\n     - `<newvalue>`: New object identifier.\n     - `[<oldvalue>]`: Expected old value (atomic check).\n     - `--stdin`: Read batch updates from stdin.\n     - `-d, --delete`: Delete the reference.\n     - `-m <message>, --message <message>`: Custom reflog message.\n     - `--no-reflog`: Skip reflog updates.\n     - `--no-deref`: Update symbolic refs directly.\n\n---\n\n#### Testing & Documentation Updates:\n\n- **Unified Testing:**\n  - Develop comprehensive test cases for each subcommand to ensure\nthat the new commands produce outputs consistent with the legacy ones.\n  - Leverage existing test scenarios (e.g., those used for `git-show-ref`\nand `git-update-ref`) and add new tests specific to the new option\ncategories and output formats.\n\n- **Documentation:**\n  - Update the user manual (e.g., Documentation/git-refs.txt) to include\ndetailed sections for each subcommand, mapping the new options to\ntheir legacy equivalents.\n  - Provide developer notes to explain changes, highlight areas of\nfunctional parity, and outline the phased implementation approach.\n\n---\n\n### Timeline\n\n- **May 8 – May 17 (10 days):** Design Finalization & Alignment (publish\nproposals, resolve conflicts).\n\n- **May 18 – June 7 (21 days):** Implement `git-refs show` (includes\ntesting/docs).\n\n- **June 8 – July 3 (26 days):** Implement `git-refs list` (includes\ntesting/docs).\n\n- **July 4 – August 4 (32 days):** Implement `git-refs update` (includes\ntesting/docs).\n\n- **August 5 – August 25 (21 days):** Cross-command validation &\nedge-case fixes.\n\n- **August 26 – September 1 (7 days):** Final Review & Adjustments.\n\n---\n\n## Background & Experience\n\nI graduated in June 2024 from Wenzhou University with a degree in\nNetwork Engineering. My experience includes C programming and\ncommand-line tool development, along with proficiency in Shell\nscripting. I am currently in a transitional phase and expect to finalize\nmy schedule by late April, and then update my weekly schedule for GSoC,\nestimating 25-30 hours per week for this project currently.\n\n### Project Experience\n\n- **One Student One Chip Project[2]**\n  Extending the open-source NEMU simulator by implementing CPU cycle\n  functionalities in C.\n- **Web Development**\n  Developed a Django-based campus website, including user chat, news\n  publishing, and teacher management modules.\n- **Custom Communication Protocols**\n  Built a UDP-based chatroom with peer-to-peer and group messaging.\n- **Stock Monitoring Tool**\n  Implemented real-time monitoring and historical data analysis, with\n  email alerting and planned AI-driven strategy optimization.\n\nI have also obtained CCNA certification and gained hands-on experience\nas a network engineer. Additionally, I contributed a patch (currently pending\nmerge) optimizing send-email functionality in Git [3], which has given me\nvaluable insights into the Git codebase. For reference, my draft proposal\ndiscussions can be reviewed on the mailing list [4], and `git-refs list`\ndiscussion on the mailing list [5].\n\n---\n\n## Appendix\n\n[1] https://gitlab.com/gitlab-org/git/-/issues/330\n[2] https://ysyx.oscc.cc/en/project/intro.html\n[3] https://lore.kernel.org/git/20250312064639.668875-1-05ZYT30@gmail.com/\n[4] https://lore.kernel.org/git/CAMvj1+rbYKFNeWEvvN76MTpzfuWc4TN4ViXRE4nTfWy7ZMspWg@mail.gmail.com/\n[5] https://lore.kernel.org/git/20250403154404.3459805-1-05ZYT30@gmail.com/\n"}]}