{"thread":{"id":"59477","subject":"[GSoC][Project Idea] Refactor lazy-fetching in a partial clone","startedAt":"2023-03-26T17:08:13Z","lastAt":"2023-03-26T17:08:13Z","messageCount":1,"participants":["Kousik Sanagavarapu"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"474195","messageId":"20230326170801.7955-1-five231003@gmail.com","threadId":"59477","inReplyTo":null,"subject":"[GSoC][Project Idea] Refactor lazy-fetching in a partial clone","fromName":"Kousik Sanagavarapu","fromEmail":"five231003@gmail.com","sentAt":"2023-03-26T17:08:01Z","receivedAt":"2023-03-26T17:08:13Z","isPatch":false,"sender":{"key":"five231003@gmail.com","avatar":"https://avatars.githubusercontent.com/u/75560439?v=4"},"body":"The term \"object\" below always means a blob or a tree since git doesn't\nyet allow filters on commits or tags.\n\nWhenever an object is missing in a partial clone and we check for it\nor read it, we trigger a lazy-fetch depending on whether fetch_if_missing\nis set to 1 or 0. Currently, this global is set to 1 everywhere and some\ncommands which do not want to lazy-fetch have it set to 0 internally (for\nexample index-pack, fetch-pack, rev-list).\n\nThe goal of this project is to look into all the commands where fetch_if_missing\nis set to 1 (in which case, whenever an object is missing, a connection\nis made, the object that is missing is fetched and we disconnect. This\nis bad because we are fetching for each object individually and this\nleads to huge performance loss) and make changes so that fetching of\nobjects is done in a batch. In this way, when all of commands know how\nto nicely fetch objects, we can change fetch_if_missing to default to 0.\n\nI think this can be implemented by looking for all the places in the code\nwhere has_object_file*(), read_object_file() or really any other function\nwhich uses oid_object_info_extended(), is used and where fetch_if_missing is\nset to 1 and make the necessary and appropriate changes to either fetch\nefficiently or to not fetch at all and also write the tests necessary\naccording to the family [1] they belong to.\n\nThe above idea is the result of the discussion on the patch\n\n  https://lore.kernel.org/git/20230225052439.27096-1-five231003@gmail.com/\n\nPlease let me know if it is doable as a project.\n\nThanks\n\n[1] https://lore.kernel.org/git/20230311025906.4170554-1-jonathantanmy@google.com/\n"}]}