Re: [PATCH v2] pack-objects: fix performance issues on packing large deltas
- From
Duy Nguyen <pclouds@gmail.com>
- Date
- Jul 23, 2018, 18:49 UTC
- Message-ID
- <CACsJy8Cex0Uby-eBRBqc_ScZd=EnCV6e0ZuaZEmopGkrvhrBzQ@mail.gmail.com>
- In-Reply-To
- <CACsJy8BskYdANe9HnuLj8sqa8hRqzSAQ+q11C8faJ-YBtA3Xeg@mail.gmail.com>
On Mon, Jul 23, 2018 at 8:38 PM Duy Nguyen <pclouds@gmail.com> wrote:
> I will have to study the thread dispatch code more to have a better > answer, unfortunately.
Well.. I thought I would need this weekend for this, but a quick look and ll_find_deltas() suggests that what we're doing is safe. At least you and Jeff are way to familiar with the delta window concept in pack-objects. So in multithread mode, we have a big list of all objects, then the list is cut in N sublists and each is owned entirely by one thread. Each thread then can slide a window over this sublist to search for the best delta.
Because of this partitioning, if trg_entry is being processed now, it will not be accessed by any other thread. It's owned by this thread and will be accessed again as src_entry when the next entry becomes the delta target in the same thread. As long as we don't access objects of another thread (and my v1 violates this) we should be ok.
At the end of ll_find_deltas() though, we have the "stealing" stuff, but it's protected by progress_lock() already, outside try_delta() so we're sure nobody is updating any object_entry when the stealing happens.
I will double check again this weekend just to be sure.
-- Duy