From: lilinchao@oschina.cn Date: Tue, 02 Mar 2021 08:44:13 GMT Subject: Re: Re: [PATCH v4] builtin/clone.c: add --reject-shallow option Message-ID: <7a2fa0ec7b3311eb83b80024e87935e7@oschina.cn> In-Reply-To: <20210301220319.3426185-1-jonathantanmy@google.com> -------------- lilinchao >> This may reject to clone from a shallow repository, but at this >> point the bulk of the tranfer from the origin repository has already >> happened, no?  Rejecting after transferring many megabytes feels a >> bit too late.  That is one of the reasons why I kept hinting that >> the transport layer needs to be taught an option to reject talking >> to a shallow counterpart if we want to add this feature [*1*]. > >Extending the transport layer in this way might not be too difficult in >the case of native (SSH, git://) protocols and using protocol v0, since >handshake() in transport.c (called indirectly from >transport_get_remote_refs()) writes shallow information to a data >structure that we could potentially expose for the caller to use (before >it calls transport_fetch_refs(). I couldn't see how remote-using >protocols (e.g. HTTP) communicate shallow information, though >(remote-curl.c seems to just keep it for itself), so that will be a more >difficult task. And of course there's the matter of protocol v2, which I >discuss below. > These discussions were based on PATCH_v4, which is quiet immature then, In the latest PATCH_v5, for the case of native protocols(ssh, git, file://), they will eventually call do_fetch_pack_v2() if it's protocol v2,  and then will call receive_shallow_info() for the case FETCH_GET_PACK, so this is the place I made the change. As for HTTPs protocl, as long as it's still smart protocol, which means do not fallback to dumb protocol, it will also call do_fetch_pack_v2(), and go to  "check shallow info" trigger in receive_shallow_info(). So, base on PATCH_V5, I have tested protocol v2, which goes like this: First, I created a shallow repo on gitlab and gitee respectively. Then tried to clone them in my terminal, (in order not to look too verbose, I ommited the result when GIT_TRACE_PACKET is on). $ ./git-clone --reject-shallow https://gitlab.com/Cactusinhand/rugged.git Cloning into 'rugged'... fatal: source repository is shallow, reject to clone. $ ./git-clone --reject-shallow ssh://gitlab.com/Cactusinhand/rugged.git Cloning into 'rugged'... fatal: source repository is shallow, reject to clone. $ ./git-clone --reject-shallow git@gitlab.com:Cactusinhand/rugged.git Cloning into 'rugged'... fatal: source repository is shallow, reject to clone. $ ./git-clone --reject-shallow https://gitee.com/cactusinhand/rugged.git Cloning into 'rugged'... fatal: source repository is shallow, reject to clone. $ ./git-clone --reject-shallow ssh://gitee.com/cactusinhand/rugged.git Cloning into 'rugged'... fatal: source repository is shallow, reject to clone. $ ./git-clone --reject-shallow git@gitee.com:cactusinhand/rugged.git Cloning into 'rugged'... fatal: source repository is shallow, reject to clone. I haven't tested protocol v1, but I've made the change in do_fetch_pack(), which is for protocol version 0 and version 1. I hope you can review the latest patch, and give me some suggestions. Thanks! >> [Footnote] >> >> *1* Looking at Documentation/technical/pack-protocol.txt, "git >>     fetch" seem to learn if the repository is shallow immediately >>     upon contacting "upload-pack" during the Reference Discovery >>     phase (we'd see 'shallow' packets if they are shallow). I >>     suspect that the right solution would be to teach the codepath >>     on the "git fetch" side that accepts, parses, and acts on this >>     packet to optionally stop communication and error out when the >>     caller asks not to talk with a shallow repository. > >This is true with protocol v0, but protocol v2 bundles all shallow >information (whether coming from the fact that the remote is shallow or >the fact that the fetcher specified --depth etc.) and sends them >together with the packfile. It may be possible to stop packfile download >(saving bandwidth on the client side, at least) once such information is >returned, though. >