{"thread":{"id":"56112","subject":"subordinate trees ? we have %G-space, we should add %K-space","startedAt":"2021-07-15T18:03:03Z","lastAt":"2021-07-15T18:03:03Z","messageCount":1,"participants":["jim.cromie@gmail.com"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"430258","messageId":"CAJfuBxwZbrAuzvfHRxLHhh8a9nva7fR4ZiLc7NECspVZ0gx2ow@mail.gmail.com","threadId":"56112","inReplyTo":null,"subject":"subordinate trees ? we have %G-space, we should add %K-space","fromName":"","fromEmail":"jim.cromie@gmail.com","sentAt":"2021-07-15T18:02:33Z","receivedAt":"2021-07-15T18:03:03Z","isPatch":false,"sender":{"key":"jim.cromie@gmail.com","avatar":null},"body":"hi git people,\n\nmanaging .configs, over time, and across work-trees, is an ongoing burden.\nI wonder if/how git could help.\n\nI imagine a subordinate/secondary tree,\nwhich basically has only 1 tracked file ( .config ) in it.\nAuto-commit on .config ( perhaps as 1st step of make ) would mean no\nchanges are missed\n\nA tertiary tree could hold all of ./builds/*\nor really any set of files that are ignored by the primary tree's\n.gitignore file\nits purpose would be to track build products; all *.o etc\nits kind-of a superset of what might be found in your .ccache,\n(it also includes *.cmd.o etc, and all other intermediate outputs)\nbut it is attributable to a TooChainCompilationTransferFn(.config, HEAD)\n\nSince HEAD already has a sha*, we just need one for .config,\nthe secondary tree keeps that for us.\n\nthe tertiary tree doesnt REALLY need to keep *all* the build products,\nit just needs to keep track of them (just the sha*, since they are\nreproducible),\nbut every commit (to either source, or to .config) should result in 1 or more\nbuild products changing.  It might be useful for QA purposes to predict and\nverify what products should change.\n\nAutocommit here means that make *anything* has known inputs,\nso if results are useful/reportable, they have known inputs (no -dirty)\n\nAlso, it might be handy to be able to diff any 2 objdumps/readelfs of foo.o,\nwhich would be easier with a cryptographically known DB\n\n\nHeres how I think it could get leveraged for linux itself\n\nWe have %G space, expressed in the kernel name:\n  5.14.0-rc1-lm1-00003-g4a7356173ef9-dirty\n\nLets add %K space, to extend that:\n  5.14.0-rc1-lm1-00003-g4a7356173ef9-dirty-k6173ef96173\n\nnow posit a kverstr(\"%D-%G-%K\") producing the 2nd name above.\n    %T - tag :  \"5.14.0-rc1\"\n    %L - localversion : \"-lm1\"\n    %C - count : \"00003\", %C{5} default\n    %G - HEAD-sha : \"-g4a7356173ef9-dirty\"\n    %K - sha(.config) : \"-k6c173efc961c\", obviously 'c' is taken.\n    %D - desc : \"%T-%L-%C\"\n\n%L is seductive, but ultimately a nuisance; mine above is a\nsuper-imaginitive \"-localmod-1\" abbreviation.\n\nIn contrast, %K would give:\n\n- updates every time the .config is tweaked.\n\n- each arch make defconfig yields single %Kval\n  or a few (1 per salt-val)\n\n- normalization done by make oldconfig narrows %K-space\n  some %Kvals are \"impossible\" (aka: hand altered or fuzzed .configs)\n\n- many .configs will get \"well-known\" %Kvals\n  bots will report probs with %Gs against specific %Ks\n\n- all well-known configs: defconfig, allmodconfig, etc...\n  get unique %Ks, maybe become well-known\n  esp as %K{5} which is short enough to recognize\n  %Ks could be annotated\n  %Ks change slowly as **/KBuild.* composition changes\n  traceable to commits in %G\n\nWith (imagined) Git extensions:\n\n1. subordinate trees, in same src/work-dir:\n\na configs subtree would have only .config file checked in, and\nprobably should auto-commit on changes.\n\nls ./.gitinclude.*\n\nThese actively define inclusions of files in the named subtree, rather\nthan ignores.  They dont see files that are not explicitly ignored by\nprimary tree.\n\nso ./.gitinclude.1.configs:\n# with\n.config\n\ncould always, actively, track and commit changes to .config,\n`history|grep config | tail -n5`, automatically added to commit-log,\nwould help (a little) explain why.  A string serialization of the\nCONFIG_* changes, would completely characterize the .config diff, and\n\nand ./.gitinclude.3.builds\n# with\n./builds/\n\ncaptures all build product files written under ./builds/* into a\nsingle subordinate tree.  This tree can only contain files ignored by\n.gitignore (ie the primary/source tree)\n\nobviously this will include all of ./**/*.o, which is essentially\na %G+%K versioned (and clean) ccache result set.\n\n./builds/{*}   # distinct build-product (%G+%K dependent) sub-trees\n\n./builds/foo # foo\n./builds/bar\n\nnot really sure how the 2ndary tree (.config tracker) should play with\nworktrees,\nor even with multiple ./builds/ subdirs, each with a .config\nthose subdirs are our %K-space-explored,\nie those configs which we have at least tried\n\nmake randconfig is a rather low-quality way to expand the tested %K-space\nbut with %K handled a bit better, we could at least track how much of\nthe space gets tested this way.\n\nok, now Im meandering.\n"}]}