patchtests: handle NO_PYTHON setting

Without this, test-lib checks that the git_remote_helpers directory has been built. However, if we are building without python, we will not have done anything at all in that directory, and test-lib's sanity check will fail.

Signed-off-by: Jeff King <peff@peff.net>
---
On top of sr/vcs-helper.

This feels a little funny for NO_PYTHON to mean "no remote helpers at all". But that is the way the Makefile is set up, since we seem to have only python helpers.

 Makefile      |    1 +
 t/test-lib.sh |    2 +-
 2 files changed, 2 insertions(+), 1 deletions(-)

Heya,

On Mon, Nov 30, 2009 at 08:52, Jeff King <peff@peff.net> wrote:

I don't understand what you mean? Do you mean NO_PYTHON implies "no remote helpers at all", or "not having any remote helpers" implies NO_PYTHON? Either way, I'm not sure how to set it up differently, not having that much Makefile foo myself, so maybe Johan and Daniel could comment?

On Mon, Nov 30, 2009 at 08:55:51AM +0100, Sverre Rabbelier wrote:

I mean, I would think that the "git_remote_helpers" directory contained remote helpers of all sorts, not just the python ones. Right now we _only_ have python ones. So checking for NO_PYTHON in test-lib.sh before looking at git_remote_helpers makes sense. But I am concerned that assumption will be broken silently in the future if non-python helpers are added to git_remote_helpers.

It is probably not worth caring about too much, though.

-Peff

Heya,

On Mon, Nov 30, 2009 at 08:59, Jeff King <peff@peff.net> wrote:

I don't think that's true, git.git currently does not have such a structure (everything is just dumped in the root directory). The only reason git_remote_helpers exists is to make it easier to create a python egg out of it and install that. At least, that's what I think is going on, Johan and Daniel might have comments to the contrary.

On Mon, Nov 30, 2009 at 09:04:25AM +0100, Sverre Rabbelier wrote:

OK. It is just my confusion, then. Don't worry about it.

-Peff

Sverre Rabbelier <srabbelier@gmail.com> writes:

If that is the case, shouldn't each of the helper written in Python need to have a separate directory, not just a single git_remote_helpers directory shared among them?

Heya,

On Mon, Nov 30, 2009 at 09:28, Junio C Hamano <gitster@pobox.com> wrote:

I don't understand why that would be needed? The reason we added a single git_remote_helpers directory is because we wanted to share common code, having a single python package makes that easy.

Sverre Rabbelier <srabbelier@gmail.com> writes:

Sorry, I don't understand that. With that reasoning, isn't having a single git package, be it python or not, even easier? Why would anybody want a separate egg that includes everything that _happen_ to be written in Python in the first place? It doesn't make sense at all from packaging point of view to me.

A separate egg per remote-helper that you can pick and choose which ones to install and which ones to leave out would make perfect sense, exactly the same way that distros already split git into "git-core", "git-svn", etc., though. Your "git-hg" may consist of a single egg and perhaps some other supporting code, and people who want to convert away from legacy Hg repository may want to install it, but it is entirely up to them if they also want to install "git-cvs" that is implemented as a remote-helper that happens to be written in Python, no?

Heya,

On Mon, Nov 30, 2009 at 09:49, Junio C Hamano <gitster@pobox.com> wrote:

Because that's the recommended way to create a python package, create a new directory, put the files in it, and distribute it.

Yes, fair enough, but we don't do that for any other files in git.git. The packagers do so, sure, but a different concern. The reason we want to distribute the git_remote_helpers package is so that git-remote-hg.py, which lives in git.git/ and is installed as git-remote-hg can say "from git_remote_helpers.hg import export".

The only reason this is needed in the first place is because we can't just add the python files to libgit.a or embed it in git-remote-hg statically, python does not support that. That is the only reason we need to distribute the package, which is why we need a separate directory.

That is, if I've understood Johan's reasoning and intention correctly.

On Monday 30 November 2009, Sverre Rabbelier wrote:

Yes, you have. I'll repeat the history of this to hopefully clarify:

I first created a "git-remote-cvs.py" script (originally under a different name, but that is not important), that was transformed (essentially copied) to "git-remote-cvs" by the build process. This script was large and unwieldy, however, and I found it more maintainable to split it into several python scripts. So I created a "git_remote_cvs", and put the supporting scripts in there.

Now, in order for the smaller "git-remote-cvs" executable to do its job, it needed to import the supporting scripts at runtime, so I decided to make a python egg from the "git_remote_cvs" subdir, that would be installed using Python's infrastructure. This, I deemed, was the safest way to ensure that the "git_remote_cvs" scripts were always available at runtime.

Next, Sverre started his "git-remote-hg.py" script, and found that he could re-use some of the scripts in the "git_remote_cvs" package. He therefore (with my blessing) refactored this subdir into the "git_remote_helpers" subdir/package that is currently under discussion. Hence, this subdir is meant to contain the support scripts for remote helper written in Python, both common support scripts, and support scripts specific to each remote helper. So far, only the common code is there, but we expect the Hg and CVS helpers to add scripts to the "hg" and "cvs" subsubdirs, respectively.

END_OF_HISTORY

Just to clarify, Git remote helpers does not live in the "git_remote_helpers" subdir. The subdir (which is installed as a Python library package) does not contain anything resembling a complete remote helper. Instead, the remote helper scripts live as regular scripts in the git.git root dir, and (like any other git program) are turned into executables by the build process. The difference is that two of the remote helpers currently being developed (Sverre's "git-remote-hg", and my "git- remote-cvs") are written in Python, and (want to) share some common supporting code currently located in the "git_remote_helpers" subdir.

We _could_ split up the "git_remote_helpers" package into a "git-remote-hg"- specific package, and a "git-remote-cvs"-specific package, but that would mean either having two copies of the current support code. Alternately, we could create a _third_ package containing the common support code, that each of our hg/cvs support packages would in turn depend on. I don't think we want to go there, at least not yet.

Also, to prevent this misunderstanding, we could create a "python" subdir in git.git, and move the "git_remote_helpers" into there. However, it would slightly complicate (at least) the git-remote-cvs.py script which currently exploits the "coincidence" that the subdir has the same name as the corresponding python package, and is therefore testable in both its unbuilt and built state.

Hope this helps,

...Johan

Johan Herland <johan@herland.net> writes:

Ahh, Ok, that clarifies it. The packager (not me) can package what is (in|built from) remote_helpers directory into one package, and declare that as a dependency to each of the individual remote helper packages that uses the common stuff found in there, if the distro wants to have "one end-user feature per package" granularity.

Just to make sure you didn't misunderstand me, I wasn't complaining that the directory was not further split for each helper. I wanted to know the reason behind wanting to have _one_ directory that is common across multiple helpers that are supposed to be independent. "A common library" is a good reason to have that organization.

I think that is typically done by the distros; in the longer term it would be nicer to them if we did so in our build structure, but I do not think we are there yet.

If (and this is a big if, as we are not migrating to Python) the use of Python proliferates over time, a single "python" subdir to hold "a common library" to create the third package will not make sense in the longer term, as different services written in Python will have different set of common supporting code, so there will be multiple common libraries and you would need multiple third packages. You would probably need multiple directories there, perhaps as subdirectories of that single toplevel directory you called "python", so that scripts written in Python would say "import pythonGit.remoteHelper" or something like that.

On Monday 30 November 2009, Junio C Hamano wrote:

Agreed.

Yes, I agree that if we get multiple Python support packages, it makes sense to create a "python" subdir, and store each package as a subsubdir of "python". Still, I don't think we need to plan that far ahead now. We can do that restructuring when it becomes necessary. If there are no more objections, I vote for keeping the current structure for now.

Have fun! :)

...Johan

Jeff King wrote:

Shouldn't this section be moved down below the sourcing of ../GIT-BUILD-OPTIONS on line 656 so that the value of NO_PYTHON will be available when running the test scripts directly?

-brandon

ps. There's something eerily familiar about this patch.

On Mon, Nov 30, 2009 at 12:07:40PM -0600, Brandon Casey wrote:

Oops, good catch. I stupidly tested with "make NO_PYTHON=1 test" instead of actually checking that GIT-BUILD-OPTIONS was propagating it correctly.

Hmmm. Yes, I didn't search before writing it, but you probably mean:

  http://article.gmane.org/gmane.comp.version-control.git/127172

But that is missing the NO-PYTHON bit in GIT-BUILD-OPTIONS (did you forget it there, or was it part of some other patch that also didn't get applied?).

Also, I am tempted to move the GIT-BUILD-OPTIONS invocation _up_. It is about reading config and should probably come before we start doing _anything_.

So maybe this instead:

-- >8 --
Subject: [PATCH] tests: handle NO_PYTHON setting

Without this, test-lib checks that the git_remote_helpers directory has been built. However, if we are building without python, we will not have done anything at all in that directory, and test-lib's sanity check will fail.

We bump the inclusion of GIT-BUILD-OPTIONS further up in test-lib; it contains configuration, and as such should be read before we do any checks (and in this particular case, we need its value to do our check properly).

Signed-off-by: Jeff King <peff@peff.net>
---
I moved the BUILD-OPTIONS thing to just above the beginning of the
"have you built anything" checks, but after all of the function
definitions. But perhaps it should simply go at the very top of the
script. After all, in the case of "make NO_PYTHON=1 test", those
variables will already be defined at the very beginning of the script.

 Makefile      |    1 +
 t/test-lib.sh |    6 +++---
 2 files changed, 4 insertions(+), 3 deletions(-)

Jeff King wrote:

:) yeah, that was it, nbd.

It was 1/2 of that series.

<snip the patch>

Looks fine to me.

No strong opinion on whether the BUILD-OPTIONS thing should be at the beginning of the script, or in the place where you placed it.

-brandon

On Monday 30 November 2009, Brandon Casey wrote:

Oops. I got Brandon's patches in my local tree, but I never got around to resend the series until Sverre picked up and refactored it. Sorry for the screwup.

Indeed.

As with Brandon's original patch, this is of course

Acked-by: Johan Herland <johan@herland.net>

Me neither.

...Johan