Re: [PATCH 4/5] strmap: add strdup_strings option

From: Elijah Newren <newren@gmail.com>
Date: Aug 21, 2020, 20:41 UTC
Message-ID: <CABPp-BGJK4C7U+-MB-+TUgFAuuFJKE-E7y17FFJDRTyFydVX2A@mail.gmail.com>
In-Reply-To: <20200821200121.GF1165@coredump.intra.peff.net>

On Fri, Aug 21, 2020 at 1:01 PM Jeff King <peff@peff.net> wrote:

Show 20 quoted lines

>
> On Fri, Aug 21, 2020 at 06:52:28PM +0000, Elijah Newren via GitGitGadget wrote:
>
> > From: Elijah Newren <newren@gmail.com>
> >
> > Just as it is sometimes useful for string_list to duplicate and take
> > ownership of memory management of the strings it contains, the same is
> > sometimes true for strmaps as well.  Add the same flag from string_list
> > to strmap.
>
> This is actually one of the ugliest parts of string_list, IMHO, and I'd
> prefer if we can avoid duplicating it. Yes, sometimes we can manage to
> avoid an extra copy of a string. But the resulting ownership and
> lifetime questions are often very error-prone. In other data structures
> we've moved towards just having the structure own its data (e.g.,
> strvec does so, and things like oidmap store their own oids). I've been
> happy with the simplicity of it.
>
> It also works if you use a flex-array for the key storage in the
> strmap_entry. :)

I can see how it's easier, but that worries me about the number of extra copies for my usecase. In order to minimize actual computation, I track an awful lot of auxiliary data in merge-ort so that I know when I can safely perform many different case-specific optimizations. Among other things, this means 15 strmaps. 1 of those stores a mapping from all paths that traverse_trees() walks over (file or directory) to metadata about the content on the three different sides. 9 of the remaining 14 simply share the strings in the main strmap, because I don't need extra copies of the paths in the repository. I could (and maybe should) extend that to 11 of the 14. Only 3 actually do need to store a copy of the paths (because they store data used beyond the end of an inner recursive merge or can be used to accelerate subsequent commits in a rebase or cherry-pick sequence).

So, in most my cases, I don't want to duplicate strings. I actually started my implementation using FLEX_ALLOC_STR(), as you suggested earlier in this thread, but tossed it because of this same desire to not duplicate strings but just share them between the strmaps.

Granted, I made that decision before I had a complete implementation, so I didn't measure the actual costs. It's possible that was a premature optimization.