Re: SHA256 support not experimental, or?

From: Son Luong Ngoc <sluongng@gmail.com>
Date: Jun 30, 2023, 12:20 UTC
Message-ID: <CAL3xRKfQLj7Zufy5fMMs=ykeexBn7duqFH1jZ84+WRKKOaEpFA@mail.gmail.com>
In-Reply-To: <5880fe56-aa98-64ce-4d91-ca078d3a7354@zombino.com>

Hi,

> On 30 Jun 2023, at 13:25, Adam Majer <adamm@zombino.com> wrote:...
> On 6/30/23 11:31, Patrick Steinhardt wrote:

...

Show 7 quoted lines

> > In any case I'm fully supportive of relaxing the current warning. Except
> > for the recently discussed edge case where cloning empty repositories
> > didn't create a SHA256 repository I have found the SHA256 code to be
> > stable and working as advertised. We should caution people that many
> > services will not work with SHA256 yet though.
>
> That is exactly true. But this is also chicken-egg problem. Services are not adapted for sha256 repositories because there is simply no demand for them. Only when people will start using sha256 repos, will there be some demand generated.

FWIW, in the Bazel ecosystem where SHA256 is very popular, there has been an increasing appetite for FUSE file system to lazily fetch contents of a git repository.

Build tools such as Bazel would often need to hash the content of the source files to build a dependency graph. And in a FUSE setup, it would be ideal if the FUSE server could supply the hash via an xattr, so that FUSE client does not need to fetch the whole file content and only the metadata.

Most tools in this space (Bazel, Buck2) are using SHA256 and are exploring faster hash such as Blake3, Aegis, KangarooTwelve for larger file support. As these matured build tools gains popularity, so will the usage of SHA256 (and newer hash algorithm).

Another point I think might help motivate different forges to move would be switching from the object's hash to digest (hash and file size). The additional file size information would help tremendously in predicting compute resources when serving files of a repository.

So I think Git would simply need a bit more time for these related ecosystems to reach a critical mass and help fuel the transition to a <new-hasher>.

> - Adam

Regards, Son Luong.

References:

- https://buck2.build/docs/rfcs/drafts/digest-kinds/#use-cases
- https://github.com/bazelbuild/bazel/pull/18784