Re: [PATCH 6/6] t5608: add regression test for >4GB object clone

From: Johannes Schindelin <johannes.schindelin@gmx.de>
Date: May 4, 2026, 17:07 UTC
Message-ID: <7ab69861-1508-106e-5d55-5ce5508653f8@gmx.de>
In-Reply-To: <ed226571-a095-456b-9d9e-bcc545d7ddfb@gmail.com>

Hi Stolee & Jeff,

On Sun, 3 May 2026, Derrick Stolee wrote:

Show 20 quoted lines

> On 5/1/2026 2:38 AM, Jeff King wrote:
> > On Wed, Apr 29, 2026 at 09:34:21AM -0400, Derrick Stolee wrote:
> 
> >>> +test_expect_success SIZE_T_IS_64BIT 'set up repo with >4GB object' '
> >>
> >> Your prereq here prevents it from running on 32-bit builds, which is
> >> good. However, I wonder if it would be worth also specifying these
> >> tests as expensive. It's less likely that these layers will be touched
> >> often, so it should be enough to run these on major occasions, such as
> >> testing a release candidate.
> > 
> > I think it is already skipped in most cases, because t5608 requires the
> > GIT_TEST_CLONE_2GB environment variable be set. Arguably it should just
> > be using EXPENSIVE, too, as I do not think there is much value in having
> > individual flags for all of the expensive tests. I think that test just
> > predates the modern prereq system entirely.
> 
> Thanks for the extra details here! That helps avoid the issues that I
> was thinking about, but maybe doubling-down and adding EXPENSIVE is
> still worth it.

Indeed. The `GIT_TEST_CLONE_2GB` flag is set for all CI jobs: https://gitlab.com/git-scm/git/-/blob/v2.54.0/ci/lib.sh#L314

So I've tried to accelerate it.

First, by using the "unsafe" SHA-1 (because we're in no danger here, we generate the data ourselves). That helped some, but unfortunately the most efficient implementation (OpenSSL) is out of bounds for us because we cannot enable its use in the default configuration for licensing reasons: OpenSSL's license and Git's GPLv2 are fundamentally incompatible with each other, and even though Linus' intention with https://gitlab.com/git-scm/git/-/blob/e83c5163316f89bfbde7d9ab23ca2e25604af290/Makefile#L11 was clearly to allow linking to OpenSSL (actually, requiring it), there is one Git contributor I spoke to who flatly stated that they'd block every attempt to add an exception to Git's license retroactively to allow linking to OpenSSL (at least for distribution, which is when the GPLv2 would kick in). So that's that.

Second, I added shortcuts for the 4GiB+1 size exercised in the added test cases. That did help! But only the generation time of those packfiles is helpd by that. The `git clone` that is tested is still awfully slow, and that _cannot_ be worked around in the same ways.

Therefore, I ended up marking the test cases as `EXPENSIVE`.

To allow them to be run regularly anyway, I added a final patch to the series that lets the CI runs on the integration branches (other than `seen`) run all `EXPENSIVE` test cases. One could argue that this patch should be split out, and I'm open to it, but in the interest of keeping the time I am working on this patch series _somewhat_ closer to what could be called reasonable, I'll just keep it in the patch series for now, hedging for the possibility that maybe nobody objects?

Ciao, Johannes

Show 18 quoted lines

> >> I suppose this also is a question for Junio and our process for
> >> validating releases. Do we have a certain cadence where we run the
> >> expensive tests? What has been our threshold for hiding a test case
> >> behind the expensive label?
> > 
> > AFAIK the labeling of expensive things is mostly ad-hoc, and nobody is
> > systematically running them. Likewise for the t/perf tests, which are
> > super expensive but do (very occasionally) turn up interesting
> > regressions.
> I used to be more diligent about running the performance tests myself
> around release windows. The EXPENSIVE tests would also be good to do
> on rc0. I will contemplate how to put this into my routine.
> 
> Thanks,
> -Stolee
> 
> 
>