git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 2/2] commit-graph tests: fix cryptic unportable "dd" invocation

From
Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date
Feb 22, 2019, 14:34 UTC
Message-ID
<8736of6gej.fsf@evledraar.gmail.com>
In-Reply-To
<20190222105053.GU1622@szeder.dev>
On Fri, Feb 22 2019, SZEDER Gábor wrote:
Show 42 quoted lines
> On Thu, Feb 21, 2019 at 11:26:26PM +0100, Ævar Arnfjörð Bjarmason wrote:
>> >> -	dd of="$objdir/info/commit-graph" bs=1 seek="$zero_pos" count=0 &&
>> >> +	perl -we 'truncate $ARGV[0], $ARGV[1] if -s $ARGV[0] > $ARGV[1]' \
>> >> +		$objdir/info/commit-graph $zero_pos &&
>> >
>> > This will make Dscho unhappy :)
>>
>> Sorry Dscho :)
>>
>> Although this is a one-off in one test, as opposed to a new "perl -e" in
>> test-lib-functions.sh
>>
>> > Is there a problem with:
>> >
>> >   dd if=/dev/null of="$objdir/info/commit-graph" bs=1 seek="$zero_pos"
>> >
>> > ?
>> >
>> > To my understanding of the specs it's well-defined what it should do,
>> > even when $zero_pos is larget than the file size,  it's shorter,
>> > simpler, and doesn't introduce yet another Perl dependency.
>>
>> I tried that as a one-off and it indeed works as a "truncate" on NetBSD
>> & GNU.
>>
>> My reading of POSIX "dd" and "lseek" docs is that we'd need some similar
>> guard if we're going to be paranoid about a $zero_pos value past the end
>> of the file. It doesn't look like that's portable, my assumption from
>> reading the docs is that the seek=* will devolve without a stat() check
>> on some "dd" implementations to an "lseek".
>
> Could you point to the part of the specs where your assumption comes
> from?  The specs are quite clear on what should happen:
>
>   If the size of the seek plus the size of the input file is less than
>   the previous size of the output file, the output file shall be
>   shortened by the copy. If the input file is empty and either the
>   size of the seek is greater than the previous size of the output
>   file or the output file did not previously exist, the size of the
>   output file shall be set to the file offset after the seek.
>
> IOW no such guard is necessary.

It was my reading of the seek=* section ("the implementation shall seek to the specified offset"). I didn't spot that bit covered in of=*. Yeah, I see that's defined & safe after reading that.

Show 5 quoted lines
> I checked the man pages of FreeBSD's, NetBSD's, OpenBSD's and Solaris'
> 'dd', and they are clearly following the specs in this respect.  I
> tried NetBSD 6.0's and 8.0's 'dd', and both behave as advertised.
>
> And using 'dd' doesn't add a condition after statement...
Show 13 quoted lines
>> I'm not going to submit a re-roll of this because it works, and I'd
>> still trust Perl's truncate(...) portability over dd.
>>
>> But more importantly because it takes me *ages* to fully re-test
>> anything on the slow BSD VMs I have access to, and I already tore town
>> my one-off hacking env there after testing these patches...
>>
>> >>  	generate_zero_bytes $(($orig_size - $zero_pos)) >>"$objdir/info/commit-graph" &&
>> >>  	test_must_fail git commit-graph verify 2>test_err &&
>> >>  	grep -v "^+" test_err >err &&
>> >> --
>> >> 2.21.0.rc0.258.g878e2cd30e
>> >>
Previous: SZEDER GáborNext: Junio C Hamano
Message 22 of 25 in “[ANNOUNCE] Git v2.21.0-rc2”
  1. Junio C HamanoFeb 19, 2019
  2. Randall S. BeckerFeb 20, 2019
  3. Junio C HamanoFeb 20, 2019
  4. Johannes SchindelinFeb 20, 2019
  5. Duy NguyenFeb 21, 2019
  6. brian m. carlsonFeb 21, 2019
  7. Duy NguyenFeb 22, 2019
  8. Randall S. BeckerFeb 21, 2019
  9. Junio C HamanoFeb 21, 2019
  10. SZEDER GáborFeb 21, 2019
  11. Randall S. BeckerFeb 21, 2019
  12. Randall S. BeckerFeb 21, 2019
  13. Git for Windows v2.21.0-rc2, was Re: [ANNOUNCE] Git v2.21.0-rc2Johannes Schindelin, Feb 21, 2019
  14. 0/2 BSD portability fixes for 2.21.0-rc2Ævar Arnfjörð Bjarmason, Feb 21, 2019
  15. Josh SteadmonFeb 22, 2019
  16. 1/2 tests: fix unportable "\?" and "\+" regex syntaxÆvar Arnfjörð Bjarmason, Feb 21, 2019
  17. Junio C HamanoFeb 22, 2019
  18. 2/2 commit-graph tests: fix cryptic unportable "dd" invocationÆvar Arnfjörð Bjarmason, Feb 21, 2019
  19. SZEDER GáborFeb 21, 2019
  20. Ævar Arnfjörð BjarmasonFeb 21, 2019
  21. SZEDER GáborFeb 22, 2019
  22. Ævar Arnfjörð BjarmasonFeb 22, 2019
  23. Junio C HamanoFeb 22, 2019
  24. Todd ZullingerFeb 22, 2019
  25. commit-graph tests: fix unportable "dd" invocationJunio C Hamano, Feb 22, 2019

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.