From: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date: Wed, 26 May 2021 18:49:24 GMT
Subject: Runaway sed memory use in test on older sed+glibc (was "Re: [PATCH v6 1/3] test: add helper functions for git-bundle")
Message-ID: <87bl8xmipo.fsf@evledraar.gmail.com>
In-Reply-To: <20210112022703.1884-2-worldhello.net@gmail.com>


On Mon, Jan 11 2021, Jiang Xin wrote:

> From: Jiang Xin <zhiyou.jx@alibaba-inc.com>
>
> Move git-bundle related functions from t5510 to a library, and this lib
> will be shared with a new testcase t6020 which finds a known breakage of
> "git-bundle".
> [...]
> +
> +# Format the output of git commands to make a user-friendly and stable
> +# text.  We can easily prepare the expect text without having to worry
> +# about future changes of the commit ID and spaces of the output.
> +make_user_friendly_and_stable_output () {
> +	sed \
> +		-e "s/${A%${A#???????}}[0-9a-f]*/<COMMIT-A>/g" \
> +		-e "s/${B%${B#???????}}[0-9a-f]*/<COMMIT-B>/g" \
> +		-e "s/${C%${C#???????}}[0-9a-f]*/<COMMIT-C>/g" \
> +		-e "s/${D%${D#???????}}[0-9a-f]*/<COMMIT-D>/g" \
> +		-e "s/${E%${E#???????}}[0-9a-f]*/<COMMIT-E>/g" \
> +		-e "s/${F%${F#???????}}[0-9a-f]*/<COMMIT-F>/g" \
> +		-e "s/${G%${G#???????}}[0-9a-f]*/<COMMIT-G>/g" \
> +		-e "s/${H%${H#???????}}[0-9a-f]*/<COMMIT-H>/g" \
> +		-e "s/${I%${I#???????}}[0-9a-f]*/<COMMIT-I>/g" \
> +		-e "s/${J%${J#???????}}[0-9a-f]*/<COMMIT-J>/g" \
> +		-e "s/${K%${K#???????}}[0-9a-f]*/<COMMIT-K>/g" \
> +		-e "s/${L%${L#???????}}[0-9a-f]*/<COMMIT-L>/g" \
> +		-e "s/${M%${M#???????}}[0-9a-f]*/<COMMIT-M>/g" \
> +		-e "s/${N%${N#???????}}[0-9a-f]*/<COMMIT-N>/g" \
> +		-e "s/${O%${O#???????}}[0-9a-f]*/<COMMIT-O>/g" \
> +		-e "s/${P%${P#???????}}[0-9a-f]*/<COMMIT-P>/g" \
> +		-e "s/${TAG1%${TAG1#???????}}[0-9a-f]*/<TAG-1>/g" \
> +		-e "s/${TAG2%${TAG2#???????}}[0-9a-f]*/<TAG-2>/g" \
> +		-e "s/${TAG3%${TAG3#???????}}[0-9a-f]*/<TAG-3>/g" \
> +		-e "s/ *\$//"
> +}

On one of the gcc farm boxes, a i386 box (gcc45) this fails because sed
gets killed after >500MB of memory use (I was just eyeballing it in
htop) on the "reate bundle from special rev: main^!" test. This with GNU
sed 4.2.2.

I suspect this regex pattern creates some runaway behavior in sed that's
since been fixed (or maybe it's the glibc regex engine?). The glibc is
2.19-18+deb8u10:
    
    + git bundle list-heads special-rev.bdl
    + make_user_friendly_and_stable_output
    + sed -e s/[0-9a-f]*/<COMMIT-A>/g -e s/[0-9a-f]*/<COMMIT-B>/g -e s/[0-9a-f]*/<COMMIT-C>/g -e s/[0-9a-f]*/<COMMIT-D>/g -e s/[0-9a-f]*/<COMMIT-E>/g -e s/[0-9a-f]*/<COMMIT-F>/g -e s/[0-9a-f]*/<COMMIT-G>/g -e s/[0-9a-f]*/<COMMIT-H>/g -e s/[0-9a-f]*/<COMMIT-I>/g -e s/[0-9a-f]*/<COMMIT-J>/g -e s/[0-9a-f]*/<COMMIT-K>/g -e s/[0-9a-f]*/<COMMIT-L>/g -e s/[0-9a-f]*/<COMMIT-M>/g -e s/[0-9a-f]*/<COMMIT-N>/g -e s/[0-9a-f]*/<COMMIT-O>/g -e s/[0-9a-f]*/<COMMIT-P>/g -e s/[0-9a-f]*/<TAG-1>/g -e s/[0-9a-f]*/<TAG-2>/g -e s/[0-9a-f]*/<TAG-3>/g -e s/ *$//
    sed: couldn't re-allocate memory