Merging (only) subdirectories

This might take a bit of explaining... and for those familiar with it I'm talking about Moodle themes which will help one or two of you.

We develop themes for Moodle. All themes live in their own self-contained directory. So, when creating a new theme we create a new branch (from some version of Moodle - more in a moment) then a new theme folder, do the dev, and commit/push the results.

We have lots of clients and for one reason or another we have lots of different versions of Moodle and/or different customisations. So each client ends up with their own branch.

The problem comes when Client A wants to purchase Theme B. We can't just merge the theme branch because all the rest of the project contains (potentially) a different version of Moodle. So we have to do lots of messing around to get the theme branch in and/or loose the theme commit history in the process.

What I would like to do is to be able to merge branch B into branch A but ONLY stuff that concerns the theme directory. I don't think this is possible. I'm reluctant to keep themes in a completely separate repo as that just makes testing them more problematic.

Does anybody have any thoughts or advice (assuming this all makes sense)?  Cheers!