Re: Exec upload-pack on remote with what parameters to get direntries.

From: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date: Aug 31, 2021, 14:01 UTC
Message-ID: <87v93lwv7m.fsf@evledraar.gmail.com>
In-Reply-To: <CANXojcxF8V2RR=xMLrwcpwa=R8fvhsn2Wj=pnthXNnvxX7YLxQ@mail.gmail.com>

On Tue, Aug 31 2021, Stef Bon wrote:

Show 13 quoted lines

> Op di 31 aug. 2021 om 09:07 schreef Jeff King <peff@peff.net>:
>>
>> On Tue, Aug 31, 2021 at 08:38:39AM +0200, Stef Bon wrote:
>>
>
>> You might also set GIT_TRACE_PACKET=1 in your environment and try
>> running some Git commands. They will show you what's being said on the
>> wire, up until the packfile is sent (decoding the packfile itself is a
>> whole other story).
>>
>
> Yes that will give me the insight I need.
> I will come back when it comes to decoding the packfile.

Aside from the "here's how you can do it", you haven't said why you'd like to do such "online" browsing of the repository.

I'd think that even for something that e.g. implements a file browser with magic git-remote support (think GNOME VFS-like), what you'd want to do in the background would be to do a "clone", although a clone with some combination of --single-branch, --no-tags, and perhaps --depth and the filters discussed upthread.

It will take the same time to get the pack, but once you do you can use libgit2, git's plumbing etc. to do really fast browsing/wildcarding etc. of the entries locally.

So is there a real performance or other use-case for wanting to do this, or does it just come down a lack of nice a "one-shot" API for "list remote files?".

In any case, on the topic of clever things you can (ab)use to do this, some remotes support running "git archive" for you. Notably GitHub doesn't, but GitLab does. Please don't take this as an endorsement to run this command "in production"

    $ time (git archive --format=tar  --remote=git@gitlab.com:git-vcs/git.git --prefix=t/t4018/ HEAD:t/t4018 | tar -tf- | head -n 3)
    t/t4018/
    t/t4018/README
    t/t4018/bash-arithmetic-function

    real    0m1.545s

I idly wonder if there's a want/need for a file listing API whether doing so via the tar/zip format wouldn't be a more viable & widely supported thing than expecting everyone to come up with their own git packfile decoders. I.e. if we just supported some option to create all-empty dummy files via "git archive" this could be even better as a dummy file listing API. Right now this (ab)use of it requires e.g. sending ~10MB of t/'s content just to list everything in the t/ directory.