Re: [PATCH v2 1/1] midx.c: fix an integer overflow
On 3/12/2020 1:35 PM, Damien Robert wrote:
Show 11 quoted lines
> When verifying a midx index with 0 objects, the
> m->num_objects - 1
> overflows to 4294967295.
>
> Fix this both by checking that the midx contains at least one oid,
> and also that we don't write any midx when there is no packfiles.
>
> Signed-off-by: Damien Robert <damien.olivier.robert+git@gmail.com>
> ---
> Should I add a test? It is a bit troublesome to generate a zero object midx
> file since this patch prevents it from using 'midx write'...
I'm glad that your patch makes it impossible to generate a zero-object multi-pack-index, and that makes a test hard to implement. I'm not sure what history Git has for storing explicit binary content into the test suite. There really is only one "empty" multi-pack-index, but it is unfortunately still a bit big for a test case to write explicitly due to the 256-word fanout table.
I _think_ the t/tXXXX directories are used for this kind of data storage, so you could generate an empty multi-pack-index from an older version of Git then store it there. Please wait for someone else on-list to say that this is a good idea, though. It may not be worth the pain of a binary file in the patch.
Show 13 quoted lines
> midx.c | 35 +++++++++++++++++++++++------------
> 1 file changed, 23 insertions(+), 12 deletions(-)
>
> diff --git a/midx.c b/midx.c
> index 1527e464a7..2cece7f9ea 100644
> --- a/midx.c
> +++ b/midx.c
> @@ -923,6 +923,12 @@ static int write_midx_internal(const char *object_dir, struct multi_pack_index *
> cur_chunk = 0;
> num_chunks = large_offsets_needed ? 5 : 4;
>
> + if (packs.nr - dropped_packs == 0) {
> + error(_("no pack files to index."));nit: I would use "pack-files" here. Second best is "packfiles".
Show 18 quoted lines
> + result = 1;
> + goto cleanup;
> + }
> +
> written = write_midx_header(f, num_chunks, packs.nr - dropped_packs);
>
> chunk_ids[cur_chunk] = MIDX_CHUNKID_PACKNAMES;
> @@ -1124,22 +1130,27 @@ int verify_midx_file(struct repository *r, const char *object_dir, unsigned flag
> i, oid_fanout1, oid_fanout2, i + 1);
> }
>
> - if (flags & MIDX_PROGRESS)
> - progress = start_sparse_progress(_("Verifying OID order in multi-pack-index"),
> - m->num_objects - 1);
> - for (i = 0; i < m->num_objects - 1; i++) {
> - struct object_id oid1, oid2;
> + if (m->num_objects == 0)
> + midx_report(_("Warning: the midx contains no oid."));Should this "Warning: " be here? The other calls to midx_report() do not have such prefix.
It could be valuable to add "warning: %s\n" to the fprintf inside midx_report(), but that should be done as its own patch.
Also, it may be valuable to return from this block so you do not need to put the block below in a tabbed block, reducing the complexity of this patch.
Show 30 quoted lines
> + else
> + {
> + if (flags & MIDX_PROGRESS)
> + progress = start_sparse_progress(_("Verifying OID order in multi-pack-index"),
> + m->num_objects - 1);
> + for (i = 0; i < m->num_objects - 1; i++) {
> + struct object_id oid1, oid2;
>
> - nth_midxed_object_oid(&oid1, m, i);
> - nth_midxed_object_oid(&oid2, m, i + 1);
> + nth_midxed_object_oid(&oid1, m, i);
> + nth_midxed_object_oid(&oid2, m, i + 1);
>
> - if (oidcmp(&oid1, &oid2) >= 0)
> - midx_report(_("oid lookup out of order: oid[%d] = %s >= %s = oid[%d]"),
> - i, oid_to_hex(&oid1), oid_to_hex(&oid2), i + 1);
> + if (oidcmp(&oid1, &oid2) >= 0)
> + midx_report(_("oid lookup out of order: oid[%d] = %s >= %s = oid[%d]"),
> + i, oid_to_hex(&oid1), oid_to_hex(&oid2), i + 1);
>
> - midx_display_sparse_progress(progress, i + 1);
> + midx_display_sparse_progress(progress, i + 1);
> + }
> + stop_progress(&progress);
> }
> - stop_progress(&progress);
>
> /*
> * Create an array mapping each object to its packfile id. Sort it
> Thanks for digging into this!
-Stolee
