git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 4/6] receive-pack: quarantine objects until pre-receive accepts

From
Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date
Apr 8, 2017, 14:53 UTC
Message-ID
<CACBZZX7Rt4zLGk=iy_Z9+8M-xUexwYA060R-v0nvseY7g+zA0w@mail.gmail.com>
In-Reply-To
<20161001091245.jkju5rlytbcb2r52@sigill.intra.peff.net>
On Sat, Oct 1, 2016 at 11:12 AM, Jeff King <peff@peff.net> wrote:
Show 43 quoted lines
> On Fri, Sep 30, 2016 at 03:36:30PM -0400, Jeff King wrote:
>
>> @@ -1639,6 +1666,18 @@ static const char *unpack(int err_fd, struct shallow_info *si)
>>               argv_array_push(&child.args, alt_shallow_file);
>>       }
>>
>> +     tmp_objdir = tmp_objdir_create();
>> +     if (!tmp_objdir)
>> +             return "unable to create temporary object directory";
>> +     child.env = tmp_objdir_env(tmp_objdir);
>
> One thing to note here: this new code kicks in all the time. My
> reasoning was that there's basically no time you _wouldn't_ want it to,
> and certainly that was the case for us when I wrote it. But I tried to
> think of user-visible changes. Here's what I came up with:
>
>   - we currently leave the tmp_pack_* for a failed push sitting around
>     (e.g., if the client hangs up halfway through, or index-pack rejects
>     the pack for some reason). But with this series, it would always be
>     cleaned up. That's a very good thing if you're running a git hosting
>     site. It might make things harder if you're debugging.
>
>     I don't think it's a good reason not to enable this by default, but
>     it _could_ be a reason to have a config switch to turn it off
>     temporarily (or just leave the "incoming-*" directory in place).
>
>   - the environment that pre-receive pack runs in has access to objects
>     that the rest of the repository doesn't. So if you were to do
>     something silly in your pre-receive like:
>
>       # reject the push, but log a copy of the objects
>       git update-ref refs/rejected/$(date +%s) $new_sha1
>       exit 1
>
>     Then your ref-update would succeed (you have $new_sha1), but the
>     objects would be deleted immediately afterwards. I find this a
>     somewhat questionable pattern, and I have no idea if anybody else
>     has thought of it. But it _does_ work today, and not with this
>     series.
>
> I don't think it would be too hard to put a config conditional around
> this tmp_objdir_create(). And then all of the tmp_objdir_env() calls
> would just return NULL, and effectively become noops.

Very late reply, but I have a question about this. Is there anything you can do on the plumbing level to figure out which area an object is in (of course that's not mutually exclusive).

The use-case for that is e.g. having a hook that rejects large binaries, but has an exception for a binary that's been added in the past, before your change there's no optimal way to find that out, you'd need to go over the whole history and list all blobs that have ever been added, with your change it would be trivial if something could give me "this object is in the quarantine area", but AFAICT there's no API that'll show me that.

Also, I think this whole thing could really do with some documentation in githooks(5). E.g. what hooks does it apply for? The test is just for pre-receive but the patch changes run_update_hook(), does it also take effect for update? Ditto the caveat about update-ref.

Previous: Jeff KingNext: Jeff King
Message 16 of 32 in “receive-pack: quarantine pushed objects”
  1. 0/6 receive-pack: quarantine pushed objectsJeff King, Sep 30, 2016
  2. 1/6 check_connected: accept an env argumentJeff King, Sep 30, 2016
  3. 2/6 sha1_file: always allow relative paths to alternatesJeff King, Sep 30, 2016
  4. René ScharfeOct 2, 2016
  5. Jeff KingOct 2, 2016
  6. Jeff KingOct 2, 2016
  7. Jeff KingOct 2, 2016
  8. 3/6 tmp-objdir: introduce API for temporary object directoriesJeff King, Sep 30, 2016
  9. Junio C HamanoSep 30, 2016
  10. Jeff KingSep 30, 2016
  11. David TurnerSep 30, 2016
  12. Jeff KingSep 30, 2016
  13. David TurnerSep 30, 2016
  14. 4/6 receive-pack: quarantine objects until pre-receive acceptsJeff King, Sep 30, 2016
  15. Jeff KingOct 1, 2016
  16. Ævar Arnfjörð BjarmasonApr 8, 2017
  17. Jeff KingApr 10, 2017
  18. 5/6 tmp-objdir: put quarantine information in the environmentJeff King, Sep 30, 2016
  19. 6/6 tmp-objdir: do not migrate files starting with '.'Jeff King, Sep 30, 2016
  20. Christian CouderOct 2, 2016
  21. Jeff KingOct 2, 2016
  22. Christian CouderOct 3, 2016
  23. 0/5 receive-pack: quarantine pushed objectsJeff King, Oct 3, 2016
  24. 1/5 check_connected: accept an env argumentJeff King, Oct 3, 2016
  25. Jakub NarębskiOct 5, 2016
  26. Jeff KingOct 5, 2016
  27. 2/5 tmp-objdir: introduce API for temporary object directoriesJeff King, Oct 3, 2016
  28. 3/5 receive-pack: quarantine objects until pre-receive acceptsJeff King, Oct 3, 2016
  29. 5/5 tmp-objdir: do not migrate files starting with '.'Jeff King, Oct 3, 2016
  30. 4/5 tmp-objdir: put quarantine information in the environmentJeff King, Oct 3, 2016
  31. Junio C HamanoOct 3, 2016
  32. Jeff KingOct 3, 2016

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.