{"thread":{"id":"56232","subject":"[GSOC][QUESTION] How to parse the properties of the object at once","startedAt":"2021-08-07T06:32:05Z","lastAt":"2021-08-07T07:15:28Z","messageCount":2,"participants":["ZheNing Hu","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"432230","messageId":"CAOLTT8Q8BEKCVwPDypW1w66P9_xP7QC0T-CnLqamqAL4haGzwA@mail.gmail.com","threadId":"56232","inReplyTo":null,"subject":"[GSOC][QUESTION] How to parse the properties of the object at once","fromName":"ZheNing Hu","fromEmail":"adlternative@gmail.com","sentAt":"2021-08-07T06:32:51Z","receivedAt":"2021-08-07T06:32:05Z","isPatch":false,"sender":{"key":"adlternative@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58138461?v=4"},"body":"Hi guys,\n\nparse_object_buffer() which will call parse_tag_buffer() and\nparse_commit_buffer()\nto parse object data and store in `struct tag` and `struct commit`, which can\ndirectly obtain the parsed data later by something like grab_tag_values() and\ngrab_commit_values().\n\nBut parse_object_buffer() will only parse part of the object data, so\nthat we need\nsome additional parsing like grab_person() and grab_sub_body_contents() in\nref-filter. It is a repetitive parsing and will affect performance.\n\nSo I am thinking if we can add some members in `struct commit` or `struct tag`,\nso that we can get more different types of data in the parsing process.\n\nAt the same time, these parsing are optional, which means that we can set\nseveral hook pointers to decide whether we need this type data, like\noid_object_info_extended() does, in this way we will not bring a lot of\nperformance loss when we don't need them.\n\nBut I find in commit.h, there is such a comment:\n\n/*\n * The size of this struct matters in full repo walk operations like\n * 'git clone' or 'git gc'. Consider using commit-slab to attach data\n * to a commit instead of adding new fields here.\n */\n\nThis means that I shouldn't touch the content of struct commit. So I see the\ncode of `commit-slab`, it seems that it is doing additional parsing.\nBut what I hope\nis that let parse_commit_buffer() can parse commit data only once.\n\nIn addition, I am thinking about whether to build a huge \"struct object_view\"\nto store the parsed objects' properties states and results.\n\nAny good ideas?\n--\nZheNing Hu\n"},{"id":"432231","messageId":"xmqqzgttiv3r.fsf@gitster.g","threadId":"56232","inReplyTo":"CAOLTT8Q8BEKCVwPDypW1w66P9_xP7QC0T-CnLqamqAL4haGzwA@mail.gmail.com","subject":"Re: [GSOC][QUESTION] How to parse the properties of the object at once","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-08-07T07:15:20Z","receivedAt":"2021-08-07T07:15:28Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"ZheNing Hu <adlternative@gmail.com> writes:\n\n> This means that I shouldn't touch the content of struct commit. So I see the\n> code of `commit-slab`, it seems that it is doing additional parsing.\n\nWe should keep what is in \"struct commit\" and parsing overhead to\nthe minimum, as it matters to performance (especially when auxiliary\ndata structures like commit-graph are not available for the part of\nhistory).  If some pieces of data (like \"from this byte to the end\nis %(body)\") do not matter in commit traversal, they are optional,\nand (1) we should not always parse them out, instead we should do so\nonly on demand, and (2) we should not add members for them in the\ncommit object, but use commit slabs to store them.\n\nAs to the slab, it is not like you have to have a slab per these\noptional fields you may want to parse.  If for example you need the\nauthorship ident and timestamp, even if you do not need committer\nident and timestamp, it is plausible to have a type of slab that\nholds these four data items together (and only fill parts of them\nthat are actually requested by the callers).  Also, things that are\nstrings may want to be stored as a relative offset into the commit\nbuffer, instead of duplicated copies of string values.\n\n\n"}]}