{"thread":{"id":"66811","subject":"[Outreachy] Proposal draft v1: Implement promisor remote fetch ordering","startedAt":"2026-11-09T11:39:09Z","lastAt":"2026-11-09T11:39:09Z","messageCount":1,"participants":["Hussain Sitamouwala"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"557926","messageId":"20261109113909.199326-1-husainsitamuwala21@gmail.com","threadId":"66811","inReplyTo":null,"subject":"[Outreachy] Proposal draft v1: Implement promisor remote fetch ordering","fromName":"Hussain Sitamouwala","fromEmail":"husainsitamuwala21@gmail.com","sentAt":"2026-11-09T11:39:09Z","receivedAt":"2026-11-09T11:39:09Z","isPatch":false,"sender":{"key":"husainsitamuwala21@gmail.com","avatar":null},"body":"Hi team,\n\nThis is the first draft of my proposal for \"Implement promisor\nremote fetch ordering\" for the December 2026 Outreachy round. I would\nappreciate feedback, especially on the config shape (6.1), the\nprecedence rule (6.4), the protocol field (7) and the timeline (10).\n\nIn short\n--------\nWhen a partial clone has several promisor remotes, Git chooses which\none to ask for a missing object by a fixed rule that neither the user\nnor the server can change. I will (1) lock today's order in tests,\n(2) add a local way to set it, and (3) let a server advertise a\npreferred order through the existing \"promisor-remote\" capability,\nwith a stated precedence rule, defensive parsing of untrusted input\nand proof that nothing changes unless someone opts in. The design\ngoes to the list in week 2 because it needs the longest review.\n\nSummary of deliverables\n-----------------------\n D1. A way to observe the fetch order in tests, and tests that lock\n     in today's behaviour.\n D2. A local setting that orders promisor remotes.\n D3. A server-advertised priority with documented precedence.\n D4. Documentation updated; the \"order cannot be set\" limitation\n     removed.\n D5. A final report listing what remains.\n\n1. About me and logistics\n-------------------------\nName:     Hussain Sitamouwala\nEmail:    husainsitamuwala21@gmail.com\nGitHub:   https://github.com/CodeWithHusain\nBlog:     https://husain-git-blog.vercel.app/\nLocation: Mumbai, India (IST, UTC+5:30)\n\nI have read the Outreachy eligibility rules and I meet them.\n\nI learned C through CS50 and use it with C++ on Linux (WSL). I build\nGit from source, and I send and revise patches with git send-email,\nthreading versions with --in-reply-to and answering every review\ncomment before sending the next version. I also wrote a Python\nnetwork vulnerability scanner on top of Nmap, where I learned to\ntreat anything that arrives over a network as hostile until checked.\nThat habit matters here, because a server-advertised fetch order is\ndata the client does not control.\n\n2. My Git work\n--------------\nPatch 1: doc: describe promisor remote fetch order (v1 to v3)\n  Thread: https://lore.kernel.org/git/20261108195518.943-1-husainsitamuwala21@gmail.com/t/#u\n  What it does: documents that remotes accepted from a server's\n  advertisement are tried before other promisor remotes. This is the\n  exact behaviour the project will make configurable, so I had to read\n  the fetch path before writing it.\n  Review: Junio reviewed v3 (\"This iteration looks good. Will\n  queue.\") and asked Christian for an Ack.\n  Status: accepted\n\nPatch 2: check-ref-format: use parse-options API (v1, v2)\n  Thread:https://lore.kernel.org/git/20261109074352.163019-1-husainsitamuwala21@gmail.com/T/#me7e8cc9786a5a023d6c028b93185b3c6aa0d1d50\n  https://github.com/git/git/pull/2470\n  What it does: replaces a hand-written option loop with\n  parse_options(). The commit message states the behaviour changes\n\nExperiments I ran:\n  - A partial clone with three local promisor remotes, reading the\n    order of the \"git fetch <remote>\" subprocesses from\n    GIT_TRACE2_EVENT\n  - Read t5710 and t0410 and ran them: 40 out of 40 test pass\n\nHow this relates to the project: patch 1 documents the behaviour I\nwill change, and patch 2 shows I can take a change that alters\nbehaviour through review: state the change openly, keep it small,\nrework after comments.\n\n3. How it works today (checked against master at 6200d284e9, 2026-11-08))\n---------------------------------------------------------------\n - promisor_remote_get_direct() calls try_promisor_remotes() twice:\n   first with accepted_only=true, then with accepted_only=false.\n - Within each pass, remotes come in config-file order, and the\n   extensions.partialClone remote comes last.\n - Protocol: each advertised remote is a list of fields. \"name\" and\n   \"url\" come first and are mandatory; optional fields follow in any\n   order (today partialCloneFilter and token). Unknown fields are\n   ignored. The server sends what promisor.sendFields lists, the\n   client validates with promisor.checkFields and can save with\n   promisor.storeFields into remote.<name>.<field>.\n - The partial-clone technical documentation lists as a limitation\n   that the order cannot be set except by position in the config\n   file.\n\nWorked example of today's behaviour (config order A, B, C; C is\nextensions.partialClone; the server advertised B):\n\n   pass 1 (accepted only):   B\n   pass 2 (the rest):        A, then C\n\nI will confirm this with the trace experiment in section 2 before\nbuilding on it.\n\n4. Problem\n----------\nA user cannot say \"try the cheap mirror first and the expensive\norigin last\", and a server operator who knows which remote is\ncheapest or nearest cannot tell clients. The existing rules are only\npartly documented and not fully covered by tests, so any change risks\naltering them silently.\n\n5. Goals and acceptance criteria\n--------------------------------\n G1 (D1) Done when: a test can observe the order in which promisor\n    remotes are tried, and tests for accepted-first, partialClone-last\n    and fallback-on-failure pass on unmodified master.\n G2 (D2) Done when: with remote.<name>.priority set, remotes are\n    tried in the documented order; with it unset, every G1 test\n    passes unchanged.\n G3 (D3) Done when: a server can advertise a priority, a client that\n    opts in uses it under the documented precedence, and every\n    malformed or hostile case in section 8 is tested.\n G4 (D4) Done when: the limitation is removed, the new settings and\n    protocol field are documented, and the docs build.\n\n6. Design\n---------\n6.1 Local setting: three candidate shapes\n\n Shape A: remote.<name>.priority (integer, lower first)\n   + fits promisor.storeFields: a server value lands in the same key\n     a user would write by hand\n   + each remote is overridable per config file scope\n   + no dangling names\n   - order is spread over several keys; harder to read at a glance\n Shape B: promisor.fetchOrder (one list of names)\n   + reads well and is easy to explain\n   - the last config file that sets it replaces it whole\n   - needs a rule for names that do not exist\n   - a server value cannot be stored into one key per remote\n Shape C: remote.<name>.fetchPosition or a named group\n   + explicit\n   - adds a new concept for little gain\n\nMy proposal is A, with stable sorting so that equal or unset\npriorities keep today's relative order. Another contributor has a\npublic patch for shape B. I do not want to duplicate\nit or compete with it. In week 2 I will send a short trade-off note\nand implement whichever shape the maintainers prefer. Section 9\nexplains how the plan adapts.\n\n6.2 Semantics, with proposed defaults to settle on the list\n - Accepted remotes stay in the first pass, as documented. Priority\n   orders remotes within each pass. I will ask whether an explicit\n   local priority should be able to override accepted-first.\n - Remotes without a priority go after prioritized ones, in today's\n   order. The extensions.partialClone remote stays last among them.\n - A bad local value is a clear config error. A bad advertised value\n   is ignored.\n\nWorked example with the proposal (priorities: A=20, B unset, C unset,\nD=10; the server advertised B and D):\n\n   pass 1 (accepted only):   D(10), B(unset)\n   pass 2 (the rest):        A(20), C(partialClone, unset)\n\n6.3 Algorithm sketch (the refactor patch)\n 1. Collect the candidate remotes for the pass.\n 2. Sort with a stable comparison: prioritized remotes by ascending\n    value, then unprioritized remotes in today's order; the\n    partialClone remote last among the unprioritized.\n 3. Try them in that order, as today, falling back on failure.\n With no priorities set, step 2 must produce exactly today's order.\n That equivalence is what the G1 tests check.\n\n6.4 Precedence between local and server values\n Recommendation: local config wins outright; the server's value\n orders only remotes with no local setting. Operators who pay for\n bandwidth may prefer the opposite. I will put both arguments on the\n list. Early check: storeFields must never overwrite a priority the\n user already set.\n\n Draft user-facing text (for the config documentation):\n   remote.<name>.priority\n     An integer. Promisor remotes with a lower value are tried first\n     when Git fetches a missing object. Remotes without a value are\n     tried after those that have one. Remotes with the same value\n     keep their configuration order.\n Draft error text:\n   error: invalid value '<value>' for remote.<name>.priority:\n   expected an integer\n\n7. Server advertisement (protocol)\n----------------------------------\nAn optional field \"priority=<n>\". I prefer it to \"advertisement\norder is preference order\" because it survives any reordering of the\nlist and, since unknown fields are already ignored, old clients are\nunaffected. I will write the protocol wording first, as an RFC.\n\n Server: sends the field only if promisor.sendFields includes\n   priority, taking the value from remote.<name>.priority. An earlier\n   review noted that one global sendFields is inflexible for\n   per-remote values; I will check whether per-remote gating is\n   needed.\n Client: parses defensively (malformed values ignored; the remote is\n   accepted or rejected on its other fields) and saves the value only\n   if promisor.storeFields includes priority. A client that does\n   nothing never follows a server's order.\n Compatibility: old client with new server, new client with old\n   server, and a value outside the supported range all behave as\n   \"no priority\".\n\n8. Threat model\n---------------\nAn advertised priority comes from a server the client may not trust.\n\n Input                      Attack or failure       Expected result\n -------------------------  ----------------------  -----------------\n priority=abc               non-numeric             ignored\n priority= (empty)          empty value             ignored\n priority=99999999999999    integer overflow        ignored, no crash\n priority twice             duplicate field         \n priority for a remote the  push an unaccepted      remote not tried;\n client rejected            remote                  no effect\n priority for a remote with local value set        local value kept\n server sends many remotes  resource use            existing limits\n                                                    apply \n client without storeFields no persistence          order unchanged\n\nEach row becomes a test. I will also avoid in-place string splitting\non lists that own their strings; I saw such a crash reported on the\ncompeting patch.\n\n9. If the competing patch lands first\n-------------------------------------\nPlan A (not merged by the start): I deliver G1, then G2 in the shape\nthe maintainers choose.\nPlan B (in next or master): I skip G2's implementation, keep the G1\ntests and the docs, and spend the time on G3, building on its config.\nIn both plans G1 and G3 are mine, and the G1 patches do not depend on\nany design decision. I will review that patch and tell its author\nwhat I find, because I will need to understand it well either way.\n\n10. Testing\n-----------\nFollow t5710 and t0410.\n\nObservation mechanism: first choice is a trace2 event naming each\npromisor remote as it is tried (a small patch of its own). If the\nmaintainers prefer no new event, fall back to a test helper that\nwraps the fetch and logs remote names.\n\nTest matrix:\n Area         Cases\n -----------  ---------------------------------------------------\n Baseline     accepted before others; partialClone last; fallback\n              after a failing remote\n Local        priority respected; ties keep config order; unset\n              after set; bad value errors; per-scope override\n Server       value honoured when stored; ignored when not stored;\n              malformed values (section 8); duplicate field\n Precedence   local beats server; storeFields does not overwrite\n Compat       old client with new server; new client with old\n              server\n\nEvery new branch must make a test fail when I break it on purpose,\nand the G1 tests must pass unchanged after the refactor.\n\n11. Series plan \n---------------------------------------------------------\n 1. Trace2 event or test helper to observe the order. No behaviour\n    change.\n 2. Tests that lock today's order. No behaviour change.\n 3. Refactor so the remote list can be sorted. No behaviour change,\n    proven by patch 2.\n 4. remote.<name>.priority, with tests and documentation.\n 5. Protocol wording (RFC), then the server and client changes,\n    with the section 8 tests.\n 6. Docs: remove the limitation, document the field.\n\n12. Timeline\n------------------------------------------------------------------\n Before:  Follow up on the docs patch; review the other patch; run\n          the experiments in section 2.\n Wk 1:    Baseline write-up of today's order with traces.\n Wk 2:    Trade-off note AND protocol RFC to the list, so design\n          review runs while I code.\n Wk 2-3:  Series patches 1 and 2 sent.\n Wk 4-5:  Patch 3 (refactor); patch 4 started.\n Wk 6:    Patch 4 finished; first review round. Milestone: G2 sent.\n Wk 7-9:  Patch 5: server field, defensive parsing, precedence.\n Wk 10:   Decide storeFields persistence; section 8 tests complete.\n Wk 11:   Final reviewable series; second review round.\n          Milestone: G3 sent.\n Wk 12:   Patch 6, release notes, final report (D5).\n Stretch: Remember failed remotes within a process; measure latency\n          or cost.\n Weekly:  Blog post and written update.\n\nRanges overlap because each series waits days for review. If I fall\nbehind I cut, in this order: the stretch, persistence of server\nvalues (7), then the section 8 cases beyond the core ones; local\npriority, the tests and the docs are protected.\n\n13. Risks\n---------\n Risk                         Mitigation\n ---------------------------  ---------------------------------------\n Surrounding code moving      Rebase often; check master before\n                              writing tests; talk to the other\n                              author\n Design disagreement          Raise early; keep patches 1-3 and docs\n                              independent of the design\n Slow review                  Small patches; next piece in parallel\n Silent regressions           G1 lands before any change\n Scope too large              Cut order in section 12\n\n14. Communication\n-----------------\nWeekly blog post, published at https://husain-git-blog.vercel.app/\nand linked on the list, and a weekly written update to the mentors.\nDesign questions go to the list before code. I answer every review\ncomment before sending a new version, as Junio asked me to, and\nreply within 1-2 days.\n\n15. After the internship\n------------------------\nI will keep following review of anything still open and keep\ncontributing to the partial-clone and promisor-remote code, starting\nwith the stretch items.\n\n16. References\n--------------\nYour patches\n[1] doc: describe promisor remote fetch order, v1 to v3:\n    v1: https://lore.kernel.org/git/xmqq5wy89rq6.fsf@gitster.g/\n    v2: https://lore.kernel.org/git/20261108071933.6586-1-husainsitamuwala21@gmail.com/\n    v3: https://lore.kernel.org/git/20261108195518.943-1-husainsitamuwala21@gmail.com/\n[2] Junio's review of v3 (\"Will queue.\") and the request for an Ack:\n    https://lore.kernel.org/git/xmqqse1a1th8.fsf@gitster.g/\n[3] check-ref-format: use parse-options API, v1 and v2:\n    https://github.com/git/git/pull/2470\n    https://lore.kernel.org/git/pull.2470.git.git.1793298807397.gitgitgadget@gmail.com/\n[4] Reviews of other people's patches:\n    https://lore.kernel.org/git/CAP8UFD0uA+zF4x8WBRA6sGxSGX52Nw=GFBznzdFnX3SptHDUcw@mail.gmail.com/\n\nCode and documentation (checked at master 6200d284e9, 2026-11-08)\n[5] promisor-remote.c: try_promisor_remotes() (line 282) and its\n    two calls (lines 351 and 353); the field lists for\n    promisor.sendFields, checkFields and storeFields (lines\n    465-481)\n[6] Documentation/config/promisor.adoc: promisor.sendFields (line\n    12), promisor.checkFields (line 166), promisor.storeFields\n    (line 207)\n[7] Documentation/technical/partial-clone.adoc: the limitation on\n    ordering (lines 229-234) and the future work item (lines\n    265-271)\n[8] Tests: t/t5710-promisor-remote-capability.sh and\n    t/t0410-partial-clone.sh\n\nMy own material\n[9] Blog: https://husain-git-blog.vercel.app/\n\nThanks,\nHussain\n\n"}]}