{"thread":{"id":"9751","subject":"Large-scale configuration backup with GIT?","startedAt":"2007-09-02T20:17:24Z","lastAt":"2007-09-03T00:35:17Z","messageCount":5,"participants":["Jan-Benedict Glaw","David Kastrup","Junio C Hamano","Martin Langhoff"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"52256","messageId":"20070902201724.GB10567@lug-owl.de","threadId":"9751","inReplyTo":null,"subject":"Large-scale configuration backup with GIT?","fromName":"Jan-Benedict Glaw","fromEmail":"jbglaw@lug-owl.de","sentAt":"2007-09-02T20:17:24Z","receivedAt":"2007-09-02T20:17:24Z","isPatch":false,"sender":{"key":"jbglaw@lug-owl.de","avatar":null},"body":"Hi!\n\nI'm just thinking about storing our whole company's configuration into\nGIT, because I'm all too used to it. That is, there are configuration\ndumps of n*10000 routers and switches, as well as \"regular\"\nconfiguration files on server machines (mostly Linux and Solaris.)\nWhile probably all of the server machines could run GIT natively, we\nalready have some scripts to dump all router's/switch's configuration\nto a Solaris system, so we could it import/commit from there. There\nmight be a small number of Windows machines, but I guess these will be\ndone by exporting the interesting stuff to Linux/Solaris machines...\n\nI initially thought about running a git-init-db on each machine's root\ndirectory and adding all interesting files, but that might hurt GIT's\nusage for single software projects on those machines, no?\nAdditionally, a lot of configuration files will be common, or at least\nvery similar. A lot of repos would probably result in worse\ncompression when starting with packs.\n\nAnother idea would be to regularly copy all interesting files into a\nstaging directory (with the same directory structure as the root\nfilesystem) and git-init-db'ing this staging directory, to not have a\nmachine-wide .git/ in the root directory.\n\nIn both cases, I'd be left with a good number of GIT repos, which\nshould probably be bound together with the GIT subproject functions.\nHowever, one really interesting thing would be to be able to get the\ndiff of two machine's configuration files. (Think of machines that\n*should* be all identical!) For this, it probably would be easier to\nnot put each machine into its own GIT repo, but to use a single one\nwith a zillion branches, one for each machine.\n\nDid anybody already try to do something like that and can help me with\nsome real-life experience on that topic?\n\nMfG, JBG\n\n-- \n Jan-Benedict Glaw jbglaw@lug-owl.de +49-172-7608481\nSignature of: http://catb.org/~esr/faqs/smart-questions.html\nthe second :\n"},{"id":"52258","messageId":"85642spzvs.fsf@lola.goethe.zz","threadId":"9751","inReplyTo":"20070902201724.GB10567@lug-owl.de","subject":"Re: Large-scale configuration backup with GIT?","fromName":"David Kastrup","fromEmail":"dak@gnu.org","sentAt":"2007-09-02T20:37:43Z","receivedAt":"2007-09-02T20:37:43Z","isPatch":false,"sender":{"key":"dak@gnu.org","avatar":"https://avatars.githubusercontent.com/u/52141349?v=4"},"body":"Jan-Benedict Glaw writes:\n\n> I'm just thinking about storing our whole company's configuration into\n> GIT, because I'm all too used to it. That is, there are configuration\n> dumps of n*10000 routers and switches, as well as \"regular\"\n> configuration files on server machines (mostly Linux and Solaris.)\n> While probably all of the server machines could run GIT natively, we\n> already have some scripts to dump all router's/switch's configuration\n> to a Solaris system, so we could it import/commit from there. There\n> might be a small number of Windows machines, but I guess these will be\n> done by exporting the interesting stuff to Linux/Solaris machines...\n>\n> I initially thought about running a git-init-db on each machine's root\n> directory and adding all interesting files, but that might hurt GIT's\n> usage for single software projects on those machines, no?\n\nIt could break shell scripts, since\ncd /;echo `pwd`/filename\ndoes not return /filename.\n\nI don't think that the root directory is a good place for starting\ngit.\n\n-- \nDavid Kastrup, Kriemhildstr. 15, 44793 Bochum\n"},{"id":"52259","messageId":"20070902212431.GC10567@lug-owl.de","threadId":"9751","inReplyTo":"85642spzvs.fsf@lola.goethe.zz","subject":"Re: Large-scale configuration backup with GIT?","fromName":"Jan-Benedict Glaw","fromEmail":"jbglaw@lug-owl.de","sentAt":"2007-09-02T21:24:31Z","receivedAt":"2007-09-02T21:24:31Z","isPatch":false,"sender":{"key":"jbglaw@lug-owl.de","avatar":null},"body":"On Sun, 2007-09-02 22:37:43 +0200, David Kastrup wrote:\n> Jan-Benedict Glaw writes:\n> \n> > I'm just thinking about storing our whole company's configuration into\n> > GIT, because I'm all too used to it. That is, there are configuration\n> > dumps of n*10000 routers and switches, as well as \"regular\"\n> > configuration files on server machines (mostly Linux and Solaris.)\n> > While probably all of the server machines could run GIT natively, we\n> > already have some scripts to dump all router's/switch's configuration\n> > to a Solaris system, so we could it import/commit from there. There\n> > might be a small number of Windows machines, but I guess these will be\n> > done by exporting the interesting stuff to Linux/Solaris machines...\n> >\n> > I initially thought about running a git-init-db on each machine's root\n> > directory and adding all interesting files, but that might hurt GIT's\n> > usage for single software projects on those machines, no?\n> \n> It could break shell scripts, since\n> cd /;echo `pwd`/filename\n> does not return /filename.\n\nWell, I don't think that this is much of a problem.\n\n> I don't think that the root directory is a good place for starting\n> git.\n\nMaybe :) But this is why I brought up the discussion, to get other\npeople's oppinion on this topic.\n\nMfG, JBG\n\n-- \n Jan-Benedict Glaw jbglaw@lug-owl.de +49-172-7608481\nSignature of: http://catb.org/~esr/faqs/smart-questions.html\nthe second :\n"},{"id":"52264","messageId":"7vd4x0pwjm.fsf@gitster.siamese.dyndns.org","threadId":"9751","inReplyTo":"20070902201724.GB10567@lug-owl.de","subject":"Re: Large-scale configuration backup with GIT?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2007-09-02T21:49:49Z","receivedAt":"2007-09-02T21:49:49Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jan-Benedict Glaw writes:\n\n> I'm just thinking about storing our whole company's configuration into\n> GIT, because I'm all too used to it. That is, there are configuration\n> ...\n> In both cases, I'd be left with a good number of GIT repos, which\n> should probably be bound together with the GIT subproject functions.\n> However, one really interesting thing would be to be able to get the\n> diff of two machine's configuration files. (Think of machines that\n> *should* be all identical!) For this, it probably would be easier to\n> not put each machine into its own GIT repo, but to use a single one\n> with a zillion branches, one for each machine.\n>\n> Did anybody already try to do something like that and can help me with\n> some real-life experience on that topic?\n\nThis is something similar to what I and others in my group did\nlong time before git was even invented. I'd suggest you go in\nthe opposite direction.\n\nIf you have 5 configurations, each of which have 20 machines\nthat _should_ share that configuration (modulo obvious\ndifferences that come from hostname, IP address assignment,\netc), then\n\n - You keep track of 5 configurations; in git, you would\n probably maintain them as 5 branches.\n\n - You have a build mechanism to create systemic variation among\n 20 machines that shares one configuration; this can be\n different per branch. So if you have 20 solaris machines all\n should share logically the same configuration, you would:\n\n\t$ git checkout solarisconf\n\n ... tweak the config for machine #27, adjusting for\n ... hostname, IP address variation, etc...\n $ make target=solaris27 output=../solaris27.expect\n\n Make that makefile produce the output in named directory;\n\n - You get the config dump from your machines (your \"staging\n area\"), as you planned. Then after running the above, you\n could:\n\n\t$ cd ..\n $ diff -r solaris27.expect solaris27.actual\n\n if your \"staging area\" for machine #27 is \"solaris27.actual\".\n\nThe difference you would see is something done by *hand* on the\nmachine, which you would want to propagate back to the solaris\nconfiguration *source* you keep track in git. For some changes,\nyou may even want to adjust that single manual change done on\nmachine #27 so that you do not have to do that on other 19\nsolaris boxes manually, by adjusting the build procedure in the\nsolarisconf branch.\n\n\n\n \n"},{"id":"52289","messageId":"46a038f90709021735n3d82061eh6d8d35989075022f@mail.gmail.com","threadId":"9751","inReplyTo":"7vd4x0pwjm.fsf@gitster.siamese.dyndns.org","subject":"Re: Large-scale configuration backup with GIT?","fromName":"Martin Langhoff","fromEmail":"martin.langhoff@gmail.com","sentAt":"2007-09-03T00:35:17Z","receivedAt":"2007-09-03T00:35:17Z","isPatch":false,"sender":{"key":"martin.langhoff@gmail.com","avatar":"https://gravatar.com/avatar/1e3f311b6c4c15836501901ca58f8c0b0667246488084ba524d8bc9867e22fd9?d=mp&s=160"},"body":"On 9/3/07, Junio C Hamano wrote:\n> This is something similar to what I and others in my group did\n> long time before git was even invented. I'd suggest you go in\n> the opposite direction.\n\nAgreed. The infrustructures.org crowd has been exploring this space\nquite a bit, and developing tools like isconf that allow you to manage\na huge number of machines across various unixen.\n\nAnd recently someone's integrated Debian APT with git tracking config\nfiles (called IsiSetup http://www.isisetup.ch/ ). I haven't reviewed\nit in detail, but it'd be first on my list.\n\n> If you have 5 configurations, each of which have 20 machines\n> that _should_ share that configuration (modulo obvious\n> differences that come from hostname, IP address assignment,\n> etc), then\n\nIndeed. I ended up liking the makefile-stanzas approach it is\nincredibly simple and flexible. Add debian/rpm packages, some of the\ntools in the cfengine toolchain, git to track your\nscripts/configuration and you are golden.\n\nFor the Windows side of things, see \"Real Men Don't Click\"\n -- a bunch of unixy sysadmins\nwith the \"infrastructures.org\" background took on Windows\nservers/desktops management -- and succeeded. PG-rated, fun for the\nwhole family. ;-)\n\nHTH,\n\n\n\nmartin-who-survived-the-sysadmin-wars\n"}]}