how to filter a pull

I'm a git beginner (and an old user of cvs, and more recently of subversion, and occasionally of arch)

I'm very pleased with the decentralized character of git and the ease to manage branches. But I have a usual problem that I don't know how to solve properly in git.

I have projects that draw some parts from two or three other projects. But it is usually some small part, that are included, and patched in my project. I want to follow the development of these fellow projects.

My problem is that I can of course get a branch to host a copy of the project (if the are under git I can clone and pull, if not I use the native scm to import and commit in the branch). But now I cannot merge in my development branch as I include only a small part.

I found no way to register that I copy these part. The only one I can think of, is to have a script to extract a sub branch with only the appropriate part and then pull from it (or push to my project). i.e. I pull from the project (if git) or update in cvs, or ..., then I filter to extract the appropiate part, then push to my development branch.

But I suppose there are a lot of cleaner way to do it. And moreover I'm quite sure that developers have met the same problem, and have solved it.

Sorry to ask such a stupid question but (1) git naming itself stupid content tracker encourage dumb people like me to ask stupid questions . (2) I tried to RTFM, but could not find the appropriate page.

Thank you for any hint.

On Thu, May 03, 2007 at 03:17:05PM +0200, marc.zonzon@gmail.com wrote:

Perhaps the newly added merge strategy subtree [1] could work. Or the just added subproject support, but this is very new and only the lowlevel machinery is implemented right now.

-Peter

[1]: http://git.kernel.org/?p=git/git.git;a=commit;h=68faf68938ee943fc251c702f2027e4dfda354db

Peter Baumann <waste.manager@gmx.de> writes:

I do not think subtree merge would work with this situation, as it is only for cases that one side of the merge represents a tree that is a subset of the tree of the other side of the merge. marc.zonzon's case does not fit the pattern.

If I were doing this 6 months from now, I would probably use subproject to host the whole tree of other projects somewhere, adjust the build procedure of the primary project to borrow the whole of these other projects not just subtree -- or have appropriate symlinks in the superproject that point into relevant subtrees in the subprojects.

If I were doing this today, I would probably use separate repositories, next to the primary project, to host the whole tree of other projects, adjust the build procedure of the primary project to borrow the whole of these other projects not just subtree -- and/or have appropriate symlinks in the primary project that point into relevant subtrees in the neighbouring repositories that host these other projects.

On 5/3/07, Junio C Hamano <junkio@cox.net> wrote:

Thank you for your explanations, I'm not sure to truly understand what you propose, (I told you I'm a beginner and not amonq the clever ones!)

Do you mean that I have to do the patches I do on the fellow project in the mirror directory, as to keep it as a link? It could be possible when the project is managed by git (with the drawback of adding an extra complexity to dereference the links in the repository when archiving a release, and a lot of problem to export to a public repository) but is not a possibility when the other project is managed by an other scm.

I don't understand neither the first solution 'borrow the whole project'. To better explain the problem I give a practical case.

I'm working to package software for openwrt arm-mipsel system. The whole openwrt buildsystem is available under subversion, it is a big tree (not so huge than the kernel you are accustomed to!)

I need in my packages to produce rc scripts and udev conf, hotplug scripts. They depends of the base rc functions of openwrt (int etc), the config system (in lib/config), and the hotplug scripts (/etc/hotplug.d), and I need to patch some of these scripts. I would like to stay in sync with the development of openwrt, the change in the patched files are quite frequent but usualy merge easily. Of course this is only one part of my dependencies, the main tree that is imported in my project is the original package that I modify to tailor it to openwrt.

My problem is how to deal with this neither major nor minor dependency.

Thanks for taking time to answer.

Marc

marc zonzon, Thu, May 03, 2007 21:32:52 +0200:

It looks like you need neither subproject nor partial checkouts (another Git-specific feature which is just about to come into existence). Just manage everything in one repo. Git deals with large, big and even huge repos splendidly. It's the monstrous repos which still make problems, but people working on them.

On 2007-05-03 11:21:04 -0700, Junio C Hamano wrote:

It should be straightforward (and efficient) to make a script that takes an existing branch and makes a parallel branch that contains only one subtree of the first branch. This derived branch can then be used as a subproject or whatever.

Or is there some obvious reason why this wouldn't work, or would be inconvenient?

On 5/4/07, Karl Hasselström <kha@treskal.com> wrote:

This was my initial idea, pull and filter!, but of course it is somewhat a hack, you pull inside git, and filter outside git, I don't know how to keep history, it seems that once filtered the origin of the subtree is lost. But I lack of git proficiency, I have no experience wich git core and mainly have used it at the user 'porcelain' level. I have had some hope that there is some mean to forward the internal state recording of the database state. The solution proposed by alex does not seem to be an option, if you draw from many projects, you cannot host the whole trees because some nodes are duplicates, and if they are outside your field you don't bother of merging them. Moreover I don't know how you mark what belongs to your small project island, among this big ocean.

So if I have no further advice I think the wiser is to try the raw script option, at least for the next 6 months the time to git, and my own proficiency to ripen..

Marc