{"thread":{"id":"61756","subject":"FR: Provide Out-Of-Tree Building; Provide Cross-Compile Parameters","startedAt":"2024-07-08T16:38:05Z","lastAt":"2024-07-10T15:53:32Z","messageCount":8,"participants":["Nathan Royce","Eric Sunshine","Junio C Hamano","Đoàn Trần Công Danh"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"498285","messageId":"CALaQ_hoDqD6CXEDy0YT8no3SaoJSqV6toMtyRHdJr6h3RZUiLA@mail.gmail.com","threadId":"61756","inReplyTo":null,"subject":"FR: Provide Out-Of-Tree Building; Provide Cross-Compile Parameters","fromName":"Nathan Royce","fromEmail":"nroycea+kernel@gmail.com","sentAt":"2024-07-08T16:37:29Z","receivedAt":"2024-07-08T16:38:05Z","isPatch":false,"sender":{"key":"nroycea+kernel@gmail.com","avatar":null},"body":"Let me know if I should be submitting each of these feature-requests separately.\n\nFor projects I built, up to the git project (think LFS steps\n(altered)), they've all provided OOT builds so the source directory\nremains pristine (albeit those that needed autoconf to be run for\n`configure` generation).\n\nThey've also all provided `--{build,host,target,with-sysroot}=`\nparameters for ease of cross-compiling.\n\nLooking at what the git project offers, and I'm not seeing either features.\n\nLooking at the Makefile, I see mention of `HOST_CPU` which looks\npertinent, but being able to point to the path of the target root for\nfiles to link is important and I'm not noticing anything for that.\n\nGrepping through Documentation for `HOST_CPU` comes up with nothing.\n"},{"id":"498288","messageId":"CAPig+cSB0d7aAwMpToLCa+6Be5JFqLAr+0pvBXQxg_=DEk7p2A@mail.gmail.com","threadId":"61756","inReplyTo":"CALaQ_hoDqD6CXEDy0YT8no3SaoJSqV6toMtyRHdJr6h3RZUiLA@mail.gmail.com","subject":"Re: FR: Provide Out-Of-Tree Building; Provide Cross-Compile Parameters","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2024-07-08T16:52:14Z","receivedAt":"2024-07-08T16:52:27Z","isPatch":false,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Mon, Jul 8, 2024 at 12:38 PM Nathan Royce <nroycea+kernel@gmail.com> wrote:\n> For projects I built, up to the git project (think LFS steps\n> (altered)), they've all provided OOT builds so the source directory\n> remains pristine (albeit those that needed autoconf to be run for\n> `configure` generation).\n>\n> They've also all provided `--{build,host,target,with-sysroot}=`\n> parameters for ease of cross-compiling.\n>\n> Looking at what the git project offers, and I'm not seeing either features.\n>\n> Looking at the Makefile, I see mention of `HOST_CPU` which looks\n> pertinent, but being able to point to the path of the target root for\n> files to link is important and I'm not noticing anything for that.\n>\n> Grepping through Documentation for `HOST_CPU` comes up with nothing.\n\nIndeed, HOST_CPU was chosen intentionally for compatibility with\n`autotools` in the event that support for cross-compilation was ever\nadded[*].\n\nA few years ago, I had started adding cross-compilation support to the\nproject but never finished the task. I'm pretty sure I still have the\npatches sitting around somewhere. I'll look for them, but I'm not sure\nhow much they will help. Aside from the obvious patch adding\n`config.guess` and `config.sub`, I recall creating a patch to fool\n`autoconf` into not demanding that the project also carry the bunch of\nother scripts/tools `autoconf` normally wants (since we don't use\nthose tools in our build process).\n\n[*] https://lore.kernel.org/git/20171209094310.GA60808@flurp.local/\n"},{"id":"498303","messageId":"CALaQ_hr2Hzri6y4KwYOPmGzfvM8EjJpddvLL7CQ=d3H4QLCzJw@mail.gmail.com","threadId":"61756","inReplyTo":"CAPig+cSB0d7aAwMpToLCa+6Be5JFqLAr+0pvBXQxg_=DEk7p2A@mail.gmail.com","subject":"Re: FR: Provide Out-Of-Tree Building; Provide Cross-Compile Parameters","fromName":"Nathan Royce","fromEmail":"nroycea+kernel@gmail.com","sentAt":"2024-07-08T18:32:34Z","receivedAt":"2024-07-08T18:33:11Z","isPatch":false,"sender":{"key":"nroycea+kernel@gmail.com","avatar":null},"body":"Well goodness me, seems I spoke too soon.\nI found the that zlib was required (looking for \"zlib.h\"), so I built\nthat first and that too wasn't all that cross-compile friendly.\nI saw \"CHOST\" is used, and was surprised that it didn't seem to need\nanything to link against from the target sysroot, so that turned out\nbetter than I thought it would.\n\nI then used `configure` prefixed with\nthe`CFLAGS=\"--sysroot=<pathToSysroot>\"`, along with `HOST_CPU=<tuple>`\nfor `make`, and it worked out fine.\nBefore moving it to my device, I just nspawned/chrooted into it and\n`git --help` worked. So looks good and easy steps. Of course, it'll\ndepend on whether or not a git function using zlib also passes (hoping\nzlib actually built fine without needing any outside linkage).\n\nI'd still suggest and prefer that git (and zlib) follows what others\nhave settled on doing to be cross-compile-friendly.\n\nOn Mon, Jul 8, 2024 at 11:52 AM Eric Sunshine <sunshine@sunshineco.com> wrote:\n>\n> A few years ago, I had started adding cross-compilation support to the\n> project but never finished the task. I'm pretty sure I still have the\n> patches sitting around somewhere. I'll look for them, but I'm not sure\n> how much they will help. Aside from the obvious patch adding\n> `config.guess` and `config.sub`, I recall creating a patch to fool\n> `autoconf` into not demanding that the project also carry the bunch of\n> other scripts/tools `autoconf` normally wants (since we don't use\n> those tools in our build process).\n>\n> [*] https://lore.kernel.org/git/20171209094310.GA60808@flurp.local/\n"},{"id":"498306","messageId":"CAPig+cTaH+TiD9Ut5Q_BPinqdAirW51J56R_tUTSnL=XGzxvfg@mail.gmail.com","threadId":"61756","inReplyTo":"CALaQ_hr2Hzri6y4KwYOPmGzfvM8EjJpddvLL7CQ=d3H4QLCzJw@mail.gmail.com","subject":"Re: FR: Provide Out-Of-Tree Building; Provide Cross-Compile Parameters","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2024-07-08T19:08:35Z","receivedAt":"2024-07-08T19:08:47Z","isPatch":false,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"[please avoid top-posting on this mailing list(*)]\n\nOn Mon, Jul 8, 2024 at 2:33 PM Nathan Royce <nroycea+kernel@gmail.com> wrote:\n> Well goodness me, seems I spoke too soon.\n> I found the that zlib was required (looking for \"zlib.h\"), so I built\n> that first and that too wasn't all that cross-compile friendly.\n> I saw \"CHOST\" is used, and was surprised that it didn't seem to need\n> anything to link against from the target sysroot, so that turned out\n> better than I thought it would.\n>\n> I then used `configure` prefixed with\n> the`CFLAGS=\"--sysroot=<pathToSysroot>\"`, along with `HOST_CPU=<tuple>`\n> for `make`, and it worked out fine.\n\nThe Git build system determines some aspects of the environment\ndynamically, so if you were cross-compiling for a different\narchitecture, it is possible that this did not enable every feature of\nGit. For instance, if you look inside `config.mak.uname` and\n`Makefile`, you will find a number of invocations of $(shell ...)\nwhich pluck some host system information at build time rather than at\nconfiguration time.\n\n> Before moving it to my device, I just nspawned/chrooted into it and\n> `git --help` worked. So looks good and easy steps. Of course, it'll\n> depend on whether or not a git function using zlib also passes (hoping\n> zlib actually built fine without needing any outside linkage).\n\nA successful `git --help` is one small victory. Running the full test\nsuite on the cross-compiled project will give a more complete picture\nof whether or not the effort was successful.\n\n> I'd still suggest and prefer that git (and zlib) follows what others\n> have settled on doing to be cross-compile-friendly.\n\nI can't speak for the zlib project, but for this to happen in Git,\nsomeone with an interest in seeing such an outcome will need to submit\npatches.\n\n[*] https://lore.kernel.org/all/YQK0JuI1w1zsEHeC@kroah.com/\n"},{"id":"498312","messageId":"xmqqjzhvejye.fsf@gitster.g","threadId":"61756","inReplyTo":"CAPig+cTaH+TiD9Ut5Q_BPinqdAirW51J56R_tUTSnL=XGzxvfg@mail.gmail.com","subject":"Re: FR: Provide Out-Of-Tree Building; Provide Cross-Compile Parameters","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-07-08T19:56:41Z","receivedAt":"2024-07-08T19:56:46Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Eric Sunshine <sunshine@sunshineco.com> writes:\n\n>> I'd still suggest and prefer that git (and zlib) follows what others\n>> have settled on doing to be cross-compile-friendly.\n>\n> I can't speak for the zlib project, but for this to happen in Git,\n> someone with an interest in seeing such an outcome will need to submit\n> patches.\n\nSure.  \n\nSomebody unknown to the community suddenly coming here and\nsuggesting a feature alone would not achieve anything.  If there\nwere infinite engineering resources and motivated contributors, and\nif sufficient number of contributors thought something is worth\ndoing, it would already have been done.  And \"cross compilation\" is\none of the things that is so obvious \"isn't it nice if we had...\"\nitems.  At least the offer has to be a bit more, like \"I'll help in\nthis and that area (e.g., organizing the effort, keeping track of\nprogress, researching dependencies, ...).  Any others who want to\njoin forces?\"\n\nThanks.\n\n\n\n\n\n"},{"id":"498411","messageId":"Zo3-FVT5EFyKsdGc@danh.dev","threadId":"61756","inReplyTo":"xmqqjzhvejye.fsf@gitster.g","subject":"Re: FR: Provide Out-Of-Tree Building; Provide Cross-Compile Parameters","fromName":"Đoàn Trần Công Danh","fromEmail":"congdanhqx@gmail.com","sentAt":"2024-07-10T03:20:53Z","receivedAt":"2024-07-10T03:20:57Z","isPatch":false,"sender":{"key":"congdanhqx@gmail.com","avatar":"https://avatars.githubusercontent.com/u/42673067?v=4"},"body":"On 2024-07-08 12:56:41-0700, Junio C Hamano <gitster@pobox.com> wrote:\n> Eric Sunshine <sunshine@sunshineco.com> writes:\n> \n> >> I'd still suggest and prefer that git (and zlib) follows what others\n> >> have settled on doing to be cross-compile-friendly.\n> >\n> > I can't speak for the zlib project, but for this to happen in Git,\n> > someone with an interest in seeing such an outcome will need to submit\n> > patches.\n> \n> Sure.  \n> \n> Somebody unknown to the community suddenly coming here and\n> suggesting a feature alone would not achieve anything.  If there\n> were infinite engineering resources and motivated contributors, and\n> if sufficient number of contributors thought something is worth\n> doing, it would already have been done.  And \"cross compilation\" is\n> one of the things that is so obvious \"isn't it nice if we had...\"\n> items.  At least the offer has to be a bit more, like \"I'll help in\n> this and that area (e.g., organizing the effort, keeping track of\n> progress, researching dependencies, ...).  Any others who want to\n> join forces?\"\n\nI thought in Git project, Makefile is the official build system, and\nthe autotools build system is only an after-thought, no?\n\nFor cross-compilation, I think various project has been\ncross-compiling Git from forever.  They only need to provide a file\nnamed `config.mak' with proper information for that platform, e.g:\n\n\tcat <<-EOF\n\tprefix = /usr\n\tCC = $CC\n\tCFLAGS = $CFLAGS\n\tLDFLAGS = $LDFLAGS\n\tUSE_LIBPCRE2 := $(if true; then echo Yes; fi)\n\tperllibdir=/usr/share/perl5/vendor_perl\n\tHOST_CPU = $(config.guess | cut -d- -f1)\n\tICONV_OMITS_BOM = Yes\n\tNO_REGEX = Yes\n\tEOF\n\nThose last values need to be specified manually because they can't be\ndetected by running a test program anyway.  Those keys are already\nlisted in Makefile.\n\n-- \nDanh\n"},{"id":"498417","messageId":"CALaQ_hrhZ7qr2D+2q5ygQYG+M8=feMiYJYyuN7C+7b7gdkCZ=g@mail.gmail.com","threadId":"61756","inReplyTo":"Zo3-FVT5EFyKsdGc@danh.dev","subject":"Re: FR: Provide Out-Of-Tree Building; Provide Cross-Compile Parameters","fromName":"Nathan Royce","fromEmail":"nroycea+kernel@gmail.com","sentAt":"2024-07-10T08:21:20Z","receivedAt":"2024-07-10T08:21:57Z","isPatch":false,"sender":{"key":"nroycea+kernel@gmail.com","avatar":null},"body":"On Tue, Jul 9, 2024 at 10:20 PM Đoàn Trần Công Danh\n<congdanhqx@gmail.com> wrote:\n>\n> I thought in Git project, Makefile is the official build system, and\n> the autotools build system is only an after-thought, no?\n>\n> For cross-compilation, I think various project has been\n> cross-compiling Git from forever.  They only need to provide a file\n> named `config.mak' with proper information for that platform, e.g:\n>\n>         cat <<-EOF\n>         prefix = /usr\n>         CC = $CC\n>         CFLAGS = $CFLAGS\n>         LDFLAGS = $LDFLAGS\n>         USE_LIBPCRE2 := $(if true; then echo Yes; fi)\n>         perllibdir=/usr/share/perl5/vendor_perl\n>         HOST_CPU = $(config.guess | cut -d- -f1)\n>         ICONV_OMITS_BOM = Yes\n>         NO_REGEX = Yes\n>         EOF\n>\n> Those last values need to be specified manually because they can't be\n> detected by running a test program anyway.  Those keys are already\n> listed in Makefile.\n>\n> --\n> Danh\n\n(noted the \"top-most\" comment earlier, I didn't realize that was a\nthing. Even \"ticket/issue\" emails always say \"Don't write *below* this\nline\")\n\nDanh, I was referring to the \"--' parameters that is common amongst\nmost of the configure/make-based projects.\nWhile I believe I had already had a \"config.auto.mak\" file generated,\nit didn't appear that I could do OOT builds with it.\nIf perhaps there was some var I could pass into \"make\" where I could\npoint to the file, that'd be good (though I'd still question whether\nthat would even be enough to get OOT builds).\n\nSomeone on the IRC channel pointed out to me that there IS a\nCMakeLists.txt file and I found it is included in\n\"contrib/buildsystems\".\nAwesome, right? So I thought and hoped...\nBefore I get into it, it might be nice to have that mentioned in INSTALL/README.\n\nWhile that gets me the OOT builds, it seems it forces the use of\npcre2, even when it isn't installed in sysroot.\nIf it's installed in the build system, it'll find it there and say\nthings are good to go even though the actual \"make\" will fail because\nof it.\n\"configure\" defaults pcre2 to \"no\", so it builds fine (apart from not\nbeing OOT and other stuff I have to do).\n\nGetting closer (after manually commenting that USE_PCRE2 stuff in the\nfile (which I also don't like since source is no longer pristine),\nthough I just now got:\n*****\n...\n[ 76%] Built target scalar\nmake[2]: *** No rule to make target 'git-remote-http', needed by\n'git-add'.  Stop.\n*****\nso I'll have to peek as to what that's about.\n"},{"id":"498465","messageId":"xmqqfrshz1ja.fsf@gitster.g","threadId":"61756","inReplyTo":"Zo3-FVT5EFyKsdGc@danh.dev","subject":"Re: FR: Provide Out-Of-Tree Building; Provide Cross-Compile Parameters","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-07-10T15:53:29Z","receivedAt":"2024-07-10T15:53:32Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Đoàn Trần Công Danh <congdanhqx@gmail.com> writes:\n\n>> ...  And \"cross compilation\" is\n>> one of the things that is so obvious \"isn't it nice if we had...\"\n>> items.  At least the offer has to be a bit more, like \"I'll help in\n>> this and that area (e.g., organizing the effort, keeping track of\n>> progress, researching dependencies, ...).  Any others who want to\n>> join forces?\"\n>\n> I thought in Git project, Makefile is the official build system, and\n> the autotools build system is only an after-thought, no?\n\nTrue.  In addition to autotools, there also is cmake \"support\" (only\nmeant to be used for Windows) in our tree, which us non-Windows\nfolks pretty much treat like how Makefile-only folks treat\nautotools.\n\nBut even if they started as and still are after-thought, with enough\ninterest by other participant, there is nothing forbidding a\ncontributor to organize an effort to improve them.  The only\nconstraint from our side is not to butcher the main build system too\nbadly to make Makefile-only build less pleasant to work with and\nharder to maintain.\n\nThanks.\n\n\n"}]}