FR: Provide Out-Of-Tree Building; Provide Cross-Compile Parameters

Let me know if I should be submitting each of these feature-requests separately.

For projects I built, up to the git project (think LFS steps (altered)), they've all provided OOT builds so the source directory remains pristine (albeit those that needed autoconf to be run for `configure` generation).

They've also all provided `--{build,host,target,with-sysroot}=` parameters for ease of cross-compiling.

Looking at what the git project offers, and I'm not seeing either features.

Looking at the Makefile, I see mention of `HOST_CPU` which looks pertinent, but being able to point to the path of the target root for files to link is important and I'm not noticing anything for that.

Grepping through Documentation for `HOST_CPU` comes up with nothing.

On Mon, Jul 8, 2024 at 12:38 PM Nathan Royce <nroycea+kernel@gmail.com> wrote:

Indeed, HOST_CPU was chosen intentionally for compatibility with `autotools` in the event that support for cross-compilation was ever added[*].

A few years ago, I had started adding cross-compilation support to the project but never finished the task. I'm pretty sure I still have the patches sitting around somewhere. I'll look for them, but I'm not sure how much they will help. Aside from the obvious patch adding `config.guess` and `config.sub`, I recall creating a patch to fool `autoconf` into not demanding that the project also carry the bunch of other scripts/tools `autoconf` normally wants (since we don't use those tools in our build process).

[*] https://lore.kernel.org/git/20171209094310.GA60808@flurp.local/

Well goodness me, seems I spoke too soon. I found the that zlib was required (looking for "zlib.h"), so I built that first and that too wasn't all that cross-compile friendly. I saw "CHOST" is used, and was surprised that it didn't seem to need anything to link against from the target sysroot, so that turned out better than I thought it would.

I then used `configure` prefixed with the`CFLAGS="--sysroot=<pathToSysroot>"`, along with `HOST_CPU=<tuple>` for `make`, and it worked out fine. Before moving it to my device, I just nspawned/chrooted into it and `git --help` worked. So looks good and easy steps. Of course, it'll depend on whether or not a git function using zlib also passes (hoping zlib actually built fine without needing any outside linkage).

I'd still suggest and prefer that git (and zlib) follows what others have settled on doing to be cross-compile-friendly.

On Mon, Jul 8, 2024 at 11:52 AM Eric Sunshine <sunshine@sunshineco.com> wrote:

[please avoid top-posting on this mailing list(*)]

On Mon, Jul 8, 2024 at 2:33 PM Nathan Royce <nroycea+kernel@gmail.com> wrote:

The Git build system determines some aspects of the environment dynamically, so if you were cross-compiling for a different architecture, it is possible that this did not enable every feature of Git. For instance, if you look inside `config.mak.uname` and `Makefile`, you will find a number of invocations of $(shell ...) which pluck some host system information at build time rather than at configuration time.

A successful `git --help` is one small victory. Running the full test suite on the cross-compiled project will give a more complete picture of whether or not the effort was successful.

I can't speak for the zlib project, but for this to happen in Git, someone with an interest in seeing such an outcome will need to submit patches.

[*] https://lore.kernel.org/all/YQK0JuI1w1zsEHeC@kroah.com/

Eric Sunshine <sunshine@sunshineco.com> writes:

Sure.  

Somebody unknown to the community suddenly coming here and suggesting a feature alone would not achieve anything. If there were infinite engineering resources and motivated contributors, and if sufficient number of contributors thought something is worth doing, it would already have been done. And "cross compilation" is one of the things that is so obvious "isn't it nice if we had..." items. At least the offer has to be a bit more, like "I'll help in this and that area (e.g., organizing the effort, keeping track of progress, researching dependencies, ...). Any others who want to join forces?"

Thanks.

On 2024-07-08 12:56:41-0700, Junio C Hamano <gitster@pobox.com> wrote:

I thought in Git project, Makefile is the official build system, and the autotools build system is only an after-thought, no?

For cross-compilation, I think various project has been cross-compiling Git from forever. They only need to provide a file named `config.mak' with proper information for that platform, e.g:

	cat <<-EOF
	prefix = /usr
	CC = $CC
	CFLAGS = $CFLAGS
	LDFLAGS = $LDFLAGS
	USE_LIBPCRE2 := $(if true; then echo Yes; fi)
	perllibdir=/usr/share/perl5/vendor_perl
	HOST_CPU = $(config.guess | cut -d- -f1)
	ICONV_OMITS_BOM = Yes
	NO_REGEX = Yes
	EOF

Those last values need to be specified manually because they can't be detected by running a test program anyway. Those keys are already listed in Makefile.

On Tue, Jul 9, 2024 at 10:20 PM Đoàn Trần Công Danh <congdanhqx@gmail.com> wrote:

(noted the "top-most" comment earlier, I didn't realize that was a thing. Even "ticket/issue" emails always say "Don't write *below* this line")

Danh, I was referring to the "--' parameters that is common amongst most of the configure/make-based projects. While I believe I had already had a "config.auto.mak" file generated, it didn't appear that I could do OOT builds with it. If perhaps there was some var I could pass into "make" where I could point to the file, that'd be good (though I'd still question whether that would even be enough to get OOT builds).

Someone on the IRC channel pointed out to me that there IS a CMakeLists.txt file and I found it is included in "contrib/buildsystems". Awesome, right? So I thought and hoped... Before I get into it, it might be nice to have that mentioned in INSTALL/README.

While that gets me the OOT builds, it seems it forces the use of pcre2, even when it isn't installed in sysroot. If it's installed in the build system, it'll find it there and say things are good to go even though the actual "make" will fail because of it. "configure" defaults pcre2 to "no", so it builds fine (apart from not being OOT and other stuff I have to do).

Getting closer (after manually commenting that USE_PCRE2 stuff in the file (which I also don't like since source is no longer pristine), though I just now got: ***** ... [ 76%] Built target scalar make[2]: *** No rule to make target 'git-remote-http', needed by 'git-add'. Stop. ***** so I'll have to peek as to what that's about.

Đoàn Trần Công Danh <congdanhqx@gmail.com> writes:

True. In addition to autotools, there also is cmake "support" (only meant to be used for Windows) in our tree, which us non-Windows folks pretty much treat like how Makefile-only folks treat autotools.

But even if they started as and still are after-thought, with enough interest by other participant, there is nothing forbidding a contributor to organize an effort to improve them. The only constraint from our side is not to butcher the main build system too badly to make Makefile-only build less pleasant to work with and harder to maintain.

Thanks.