{"thread":{"id":"9013","subject":"sharing between local \"work\" and \"nightly build\" git repos","startedAt":"2007-07-12T23:36:58Z","lastAt":"2007-07-13T18:23:03Z","messageCount":6,"participants":["David Frech","Junio C Hamano","Alex Riesen"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"47214","messageId":"7154c5c60707121636l585b42d4l931b08f1468ddfc@mail.gmail.com","threadId":"9013","inReplyTo":null,"subject":"sharing between local \"work\" and \"nightly build\" git repos","fromName":"David Frech","fromEmail":"nimblemachines@gmail.com","sentAt":"2007-07-12T23:36:58Z","receivedAt":"2007-07-12T23:36:58Z","isPatch":false,"sender":{"key":"nimblemachines@gmail.com","avatar":null},"body":"I'd like to have the following setup: a ~david/git directory, where I\nam free to work on things, and a ~david/nightly-git, where a cron job\nis going build and test a nightly \"next\" branch.\n\nI'd like to share as much as possible between the two repos. My naive\nfirst attempt was to clone the local repo (~david/git) using -l and -s\n(which I admit I do not completely understand). This sort of worked,\nbut one issue is that doing a \"git pull\" in nightly is going to pull\nfrom the *locally*-cloned repo, not from the main git. Another is that\na checkout in nightly failed with the obscure error:\n\n[david@tashtego ~/git-nightly]% git checkout -b nightly-next next\ngit checkout: updating paths is incompatible with switching branches/forcing\nDid you intend to checkout 'next' which can not be resolved as commit?\n\nI assume this is because too much state is being shared the repos, and\nsomething is unfinished in the \"git\" directory.\n\nI'd love some pointers on how to:\n\n* share as many objects as possible\n* share as little state as possible\n* make git pull pull from remote in both repos.\n\nCheers,\n\n- David\n-- \nIf I have not seen farther, it is because I have stood in the\nfootsteps of giants.\n"},{"id":"47219","messageId":"7vmyy1rwza.fsf@assigned-by-dhcp.cox.net","threadId":"9013","inReplyTo":"7154c5c60707121636l585b42d4l931b08f1468ddfc@mail.gmail.com","subject":"Re: sharing between local \"work\" and \"nightly build\" git repos","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2007-07-13T00:03:21Z","receivedAt":"2007-07-13T00:03:21Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"David Frech\" writes:\n\n> I'd like to share as much as possible between the two repos. My naive\n> first attempt was to clone the local repo (~david/git) using -l and -s\n> (which I admit I do not completely understand). This sort of worked,\n> but one issue is that doing a \"git pull\" in nightly is going to pull\n> from the *locally*-cloned repo, not from the main git. Another is that\n> a checkout in nightly failed with the obscure error:\n>\n> [david@tashtego ~/git-nightly]% git checkout -b nightly-next next\n> git checkout: updating paths is incompatible with switching branches/forcing\n> Did you intend to checkout 'next' which can not be resolved as commit?\n>\n> I assume this is because too much state is being shared the repos, and\n> something is unfinished in the \"git\" directory.\n\nYou assumed wrong. \"-l and -s\" does not have to do anything\nwith the above symptoms.\n\ngit-clone helps people by setting up the new repository to\nfollow where you cloned from, but there is no rule that you\ncannot change it. Look at .git/config in the nightly repository\nand find [remote \"origin\"] section; update its URL to whichever\nrepository you would want to track from and you are done.\n\n\"git checkout -b nightly-next next\" is telling git to:\n\n - create a new branch nightly-next starting from 'next'\n - check it out\n\nYou most likely would want to fork off of \"origin/next\", not\n'next' which probably does not exist in your repository.\n\nIf you are willing to redo the nightly repository from scratch,\nI would probably recommend using --reference option when\ncloning, like this:\n\n $ git clone --reference ~david/git git://git.kernel.org/... ~/nightly-git\n $ cd ~/nightly-git\n $ git checkout --track -b next origin/next\n\nThen a nightly update would go like this:\n\n $ cd ~/nightly-git\n $ git pull origin next\n $ make clean\n $ make test || barf\n"},{"id":"47221","messageId":"7154c5c60707121727k36854891u82afc4a8be822861@mail.gmail.com","threadId":"9013","inReplyTo":"7vmyy1rwza.fsf@assigned-by-dhcp.cox.net","subject":"Re: sharing between local \"work\" and \"nightly build\" git repos","fromName":"David Frech","fromEmail":"nimblemachines@gmail.com","sentAt":"2007-07-13T00:27:16Z","receivedAt":"2007-07-13T00:27:16Z","isPatch":false,"sender":{"key":"nimblemachines@gmail.com","avatar":null},"body":"On 7/12/07, Junio C Hamano wrote:\n> \"David Frech\" writes:\n> If you are willing to redo the nightly repository from scratch,\n> I would probably recommend using --reference option when\n> cloning, like this:\n>\n> $ git clone --reference ~david/git git://git.kernel.org/... ~/nightly-git\n> $ cd ~/nightly-git\n> $ git checkout --track -b next origin/next\n>\n> Then a nightly update would go like this:\n>\n> $ cd ~/nightly-git\n> $ git pull origin next\n> $ make clean\n> $ make test || barf\n\nThis makes sense, including the --track, which I neglected to set.\n\nCould you, for my sake and the list's (if others are as confused) be\nclearer about the distinctions among -l, -s, and --reference? Exactly\nwhat they do, and their orthogonality (or lack of) really isn't clear\nfrom reading the man page.\n\nThanks for your help! I'll set this up and hopefully start running it tonight.\n\nIf the build or test barfs, who should get the mail?\n\nBTW, I figured out a sneaky way (using filters in Gmail) to send it\nthru my Gmail account, so I won't have any issues with sending via\nSMTP from a dynamic IP.\n\nCheers,\n\n- David\n-- \nIf I have not seen farther, it is because I have stood in the\nfootsteps of giants.\n"},{"id":"47222","messageId":"7154c5c60707121733r6584a407r8d60d5890b9c89e2@mail.gmail.com","threadId":"9013","inReplyTo":"7154c5c60707121727k36854891u82afc4a8be822861@mail.gmail.com","subject":"Re: sharing between local \"work\" and \"nightly build\" git repos","fromName":"David Frech","fromEmail":"nimblemachines@gmail.com","sentAt":"2007-07-13T00:33:19Z","receivedAt":"2007-07-13T00:33:19Z","isPatch":false,"sender":{"key":"nimblemachines@gmail.com","avatar":null},"body":"On 7/12/07, David Frech wrote:\n> On 7/12/07, Junio C Hamano wrote:\n> > Then a nightly update would go like this:\n> >\n> > $ cd ~/nightly-git\n> > $ git pull origin next\n> > $ make clean\n> > $ make test || barf\n\nOne more thing: would it make sense to do \"make -k test\" so that *all*\nfailures (if >1) show up?\n\nCheers,\n- David\n-- \nIf I have not seen farther, it is because I have stood in the\nfootsteps of giants.\n"},{"id":"47232","messageId":"20070713051140.GB2430@steel.home","threadId":"9013","inReplyTo":"7154c5c60707121733r6584a407r8d60d5890b9c89e2@mail.gmail.com","subject":"Re: sharing between local \"work\" and \"nightly build\" git repos","fromName":"Alex Riesen","fromEmail":"raa.lkml@gmail.com","sentAt":"2007-07-13T05:11:40Z","receivedAt":"2007-07-13T05:11:40Z","isPatch":false,"sender":{"key":"raa.lkml@gmail.com","avatar":"https://avatars.githubusercontent.com/u/324101?v=4"},"body":"David Frech, Fri, Jul 13, 2007 02:33:19 +0200:\n> On 7/12/07, David Frech wrote:\n> >On 7/12/07, Junio C Hamano wrote:\n> >> Then a nightly update would go like this:\n> >>\n> >> $ cd ~/nightly-git\n> >> $ git pull origin next\n> >> $ make clean\n> >> $ make test || barf\n> \n> One more thing: would it make sense to do \"make -k test\" so that *all*\n> failures (if >1) show up?\n\nYes, definitely. You'll find you will see failures which you know are\nalready fixed, just didn't reach the nightly-build yet.\n\nBTW, do _not_ send the errors to this mailing list. It just too easy\ngets out of control. And consider saving the build log somewhere after\ndoing a _completely_ clean build, i.e. like this:\n\n #!/bin/bash\n cd ~/nightly-git || exit\n echo 'Subject: Nightly build git next ' $(date) >/tmp/mail.msg\n echo >>/tmp/mail.msg\n {\n\trm -rf * # it does not remove .git\n\tgit reset --hard\n\tgit pull origin next || exit 1\n\tmake || exit 1\n\tmake -k test|| exit 1\n } &>> /tmp/mail.msg\n\n $ nightly-build || sendmail local-user@localhost < /tmp/mail.msg\n\nIt is simplier to find out what went wrong if you know what state did\nyou have before doing the test. And the state after rm -rf * is very\nsimple to predict.\n"},{"id":"47280","messageId":"7vfy3snoxk.fsf@assigned-by-dhcp.cox.net","threadId":"9013","inReplyTo":"7154c5c60707121727k36854891u82afc4a8be822861@mail.gmail.com","subject":"Re: sharing between local \"work\" and \"nightly build\" git repos","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2007-07-13T18:23:03Z","receivedAt":"2007-07-13T18:23:03Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"David Frech\" writes:\n\n> Could you, for my sake and the list's (if others are as confused) be\n> clearer about the distinctions among -l, -s, and --reference? Exactly\n> what they do, and their orthogonality (or lack of) really isn't clear\n> from reading the man page.\n\nClone \"-l\" makes the new repository share the initial set of\nobjects from the original by hardlinking each and every file\nunder .git/objects/. After \"-l\" clone, two repositories are\ntruly independent, except for the fact that the new one\nobviously treats the old one as its \"origin\".\n\nThere are a few implications:\n\n - New objects created in the new repository is not visible in\n the original repository and vice versa. If you do\n development in the new repository, and push the results back\n to the original, the push will involve object transfer. If\n your original one is updated (perhaps with git-pull from the\n upstream), you would need to update the clone separately.\n\n - You can safely rewind the refs in the original repository and\n run git-prune there without worrying about corrupting the\n cloned repository.\n\nClone \"-s\" does not make hardlinked copy like \"-l\" makes. It\ncreates an entry in .git/objects/info/alternates file in the new\nrepository, that instructs git that any object missing in the\nnew repository should be looked up in the cloned repository.\nNew objects created in the original will be shared with the\ncloned repository because of this, but the sharing does not\nhappen in the other direction. Also if you run \"rebase\" (or\n\"reset --hard\" to rewind the tip of branches) in the original\nrepository and then run git-gc, it would cause some objects to\nbecome unneeded in the original repository and they can get\ndiscarded. If your cloned repository depends in them, you will\nbe in trouble. So you would need to be a bit careful when you\nuse \"-s\" --- the rule of thumb is not to rewind the branches, or\nif you do, make sure the cloned ones are kept in sync\n(i.e. rewind the commits lost in the original the same way in\nthe clone).\n\nThe \"--reference\" option uses the same mechanism as \"-s\" uses to\nshare objects from a local repository. It is useful if you have\na repository of a different project that is related to the\nproject you are cloning from outside. For example, if you have\na copy of LInus's Linux 2.6 kernel repository locally, and if\nyou want to clone 2.6.x.y maintenance \"stable\" repository, you\nknow the two repositories are alike and share a lot of objects,\nso you would:\n\n $ git clone --reference ../linus-2.6/ git://.../linux-2.6.x.y/ stable.git\n\nTo this option, because it uses the same \"alternate object\nstore\" mechanism as \"-s\" option does, the same \"be careful\nrewinding the branches in the original\" caveat applies. Because\nLinus never rewind his branches, the above clone is safe.\n\nFor the purpose of \"keep rebuilding 'next' from git.git\nrepository\", because my 'next' will never rewind by social\nconvention, you do not have to worry about the \"original\nrepository being rewound and losing objects\" problem, either, as\nlong as 'next' is the branch you rebuild in the \"nightly\"\nrepository.\n"}]}