{"thread":{"id":"13012","subject":"Integration branching","startedAt":"2008-04-07T13:02:03Z","lastAt":"2008-04-10T20:04:13Z","messageCount":11,"participants":["Kai Hendry","Junio C Hamano","Avery Pennarun","Mike Hommey"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"73821","messageId":"b24851260804070602o11b93923m7ce7d79d5f59547b@mail.gmail.com","threadId":"13012","inReplyTo":null,"subject":"Integration branching","fromName":"Kai Hendry","fromEmail":"hendry@aplixcorp.com","sentAt":"2008-04-07T13:02:03Z","receivedAt":"2008-04-07T13:02:03Z","isPatch":false,"sender":{"key":"hendry@aplixcorp.com","avatar":null},"body":"I've been asked to setup & host a git repo of an \"integration branch\"\nof WebKit for our developers.\n\nSo I expected it to work like so:\n\ngit clone git://git.webkit.org/WebKit.git\ngit checkout -b test\n\n<Make a few small changes and commit>\n\ngit-config remote.upload.url ssh://git.webvm.net/srv/git/webkit-test\nx61:~/aplix/test/WebKit% git push --thin -v upload test\nPushing to ssh://git.webvm.net/srv/git/webkit-test\nCounting objects: 371612, done.\nCompressing objects: 100% (67851/67851), done.\nWriting objects:  11% (43886/371612), 25.32 MiB | 88 KiB/\n\nHowever when uploading the branch it seems to upload _everything_\nwhich is far too big and impractical. Couldn't it just somehow\nreference the remote branch and just publish the patch of the \"small\nchanges\"?\n\nWhen doing a git-http-push instead of ssh, git resulted in a \"Out of\nmemory\" error:\n\n   1. git clone git://git.webkit.org/WebKit.git # Check it out\n   2. git checkout -b test # create a \"test\" branch and switch to it\n   3. echo blah >> ChangeLog # change something\n   4. git commit -a # commit that test change\n   5. git-config remote.upload.url http://moto@moto.webvm.net/moto.git\n# configure the upload url\n   6. git push upload test # fatal: Out of memory, malloc failed\n\nWas this the fault of the server(nox) or client(x61)?\n\nx61:~% git --version\ngit version 1.5.4.5\nnox:~% git --version\ngit version 1.5.4.4\n\nSo is there some better way of publishing a branch? Without uploading\nthe whole WebKit?\n\n\n\nI did try something else, that was to create a webkit repo on the\nserver nox like so:\n\nnox:/srv/git/webkit% git --bare init --shared\nnox:/srv/git/webkit% git --bare fetch git://git.webkit.org/WebKit.git\nmaster:master\ngit fetch -f git://git.webkit.org/WebKit.git master:master # again\nforcing updates\n\nAnd let developers branch from that. This proved a little cumbersome\nas I need to manually force updates to webkit trunk when developers\nasked me to do that.\nHowever once a developer cloned ssh://git.webvm.net/srv/git/webkit.\nCreating a test branch 'deleteme' and then `git push origin deleteme`\nwas extremely fast. So I guess this is the right way to achieve my\ngoals?\n\n\nAny suggestions and tips would be greatly appreciated!\n"},{"id":"73829","messageId":"7vod8lpnpb.fsf@gitster.siamese.dyndns.org","threadId":"13012","inReplyTo":"b24851260804070602o11b93923m7ce7d79d5f59547b@mail.gmail.com","subject":"Re: Integration branching","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-04-07T17:41:20Z","receivedAt":"2008-04-07T17:41:20Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Kai Hendry\" <hendry@aplixcorp.com> writes:\n\n> So I expected it to work like so:\n>\n> git clone git://git.webkit.org/WebKit.git\n> git checkout -b test\n>\n> <Make a few small changes and commit>\n>\n> git-config remote.upload.url ssh://git.webvm.net/srv/git/webkit-test\n\nA set of big questions is how that git.webvm.net/srv/git/webkit-test\nrepository was prepared, what its history look like and how it relates to\nthe history your repository has.  I presume that that is different from\nthe repository you are pushing from (which is \"x61:aplix/test/WebKit\"\nrepository), but other than that you did not give any details to guess\nfurther.\n\n> x61:~/aplix/test/WebKit% git push --thin -v upload test\n\nBy the way, that --thin is an implementation detail and not meant for the\nend user consumption.  I do not think it would matter in this case,\nthough.\n\n> Pushing to ssh://git.webvm.net/srv/git/webkit-test\n> Counting objects: 371612, done.\n> Compressing objects: 100% (67851/67851), done.\n> Writing objects:  11% (43886/371612), 25.32 MiB | 88 KiB/\n>\n> However when uploading the branch it seems to upload _everything_\n> which is far too big and impractical. Couldn't it just somehow\n> reference the remote branch and just publish the patch of the \"small\n> changes\"?\n\nWhat you described is exactly what is supposed to be doing, but again as\nwe do not know how that git.webvm.net:/srv/gi/webkit-test/ repository was\nprepared, we cannot tell.\n\nBut I have a theory:\n\n    http://thread.gmane.org/gmane.comp.version-control.git/77643\n\nand the latest code is still parked in 'pu' branch.\n"},{"id":"73833","messageId":"b24851260804071118h4eaadef8i8313c5a4cb26495d@mail.gmail.com","threadId":"13012","inReplyTo":"7vod8lpnpb.fsf@gitster.siamese.dyndns.org","subject":"Re: Integration branching","fromName":"Kai Hendry","fromEmail":"hendry@aplixcorp.com","sentAt":"2008-04-07T18:18:45Z","receivedAt":"2008-04-07T18:18:45Z","isPatch":false,"sender":{"key":"hendry@aplixcorp.com","avatar":null},"body":"On Mon, Apr 7, 2008 at 6:41 PM, Junio C Hamano <gitster@pobox.com> wrote:\n>  A set of big questions is how that git.webvm.net/srv/git/webkit-test\n>  repository was prepared, what its history look like and how it relates to\n>  the history your repository has.  I presume that that is different from\n>  the repository you are pushing from (which is \"x61:aplix/test/WebKit\"\n>  repository), but other than that you did not give any details to guess\n>  further.\n\nwebkit-test was just a `git --bare init --shared` and completely empty\nwith I tried pushing (publishing) the changes to WebKit trunk.\n\nWith thanks,\n"},{"id":"73834","messageId":"32541b130804071132q24acff6an853730af4fc7273e@mail.gmail.com","threadId":"13012","inReplyTo":"b24851260804071118h4eaadef8i8313c5a4cb26495d@mail.gmail.com","subject":"Re: Integration branching","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2008-04-07T18:32:45Z","receivedAt":"2008-04-07T18:32:45Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On Mon, Apr 7, 2008 at 2:18 PM, Kai Hendry <hendry@aplixcorp.com> wrote:\n> On Mon, Apr 7, 2008 at 6:41 PM, Junio C Hamano <gitster@pobox.com> wrote:\n>  >  A set of big questions is how that git.webvm.net/srv/git/webkit-test\n>  >  repository was prepared, what its history look like and how it relates to\n>  >  the history your repository has.  I presume that that is different from\n>  >  the repository you are pushing from (which is \"x61:aplix/test/WebKit\"\n>  >  repository), but other than that you did not give any details to guess\n>  >  further.\n>\n>  webkit-test was just a `git --bare init --shared` and completely empty\n>  with I tried pushing (publishing) the changes to WebKit trunk.\n>\n>  With thanks,\n\nBasically every git repository needs to be entirely self-sufficient,\nthat is, it should be possible to check out a copy without referring\nto any external repositories.  So your initial push to an empty\nrepository will always upload all the objects, even if you pulled from\nsome other.\n\nThe good news is that if your friends who checked out of the main\nrepository want to try out your branch, it *won't* accidentally\ndownload all the extra objects over again.  git will notice that they\nalready have the same objects (retrieved from the main repo) and\ndownload only the new ones.\n\nSo yes, some space is \"wasted\" on your extra repo, but disk space is\ncheap anyway (and you now have a backup of the main repo for free :))\n\nHave fun,\n\nAvery\n"},{"id":"73835","messageId":"7vd4p1pks1.fsf@gitster.siamese.dyndns.org","threadId":"13012","inReplyTo":"b24851260804071118h4eaadef8i8313c5a4cb26495d@mail.gmail.com","subject":"Re: Integration branching","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-04-07T18:44:30Z","receivedAt":"2008-04-07T18:44:30Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Kai Hendry\" <hendry@aplixcorp.com> writes:\n\n> On Mon, Apr 7, 2008 at 6:41 PM, Junio C Hamano <gitster@pobox.com> wrote:\n>>  A set of big questions is how that git.webvm.net/srv/git/webkit-test\n>>  repository was prepared, what its history look like and how it relates to\n>>  the history your repository has.  I presume that that is different from\n>>  the repository you are pushing from (which is \"x61:aplix/test/WebKit\"\n>>  repository), but other than that you did not give any details to guess\n>>  further.\n>\n> webkit-test was just a `git --bare init --shared` and completely empty\n> with I tried pushing (publishing) the changes to WebKit trunk.\n\nThat explains it without the \"theory\" in my previous message.  The\ndestination repository must be populated with full history somehow (and\none way would have been to start it by cloning from the upstream), but if\nit was empty that is alright as well, because your push made it\nup-to-date.  IOW, there is nothing to worry about and the system is\nworking as designed.\n\nFurther pushes will update it incrementally.\n"},{"id":"73847","messageId":"b24851260804080656u4c42c31cv3984fa2943d83b8a@mail.gmail.com","threadId":"13012","inReplyTo":"32541b130804071132q24acff6an853730af4fc7273e@mail.gmail.com","subject":"Re: Integration branching","fromName":"Kai Hendry","fromEmail":"hendry@aplixcorp.com","sentAt":"2008-04-08T13:56:53Z","receivedAt":"2008-04-08T13:56:53Z","isPatch":false,"sender":{"key":"hendry@aplixcorp.com","avatar":null},"body":"Ok it makes sense that I need at least one checkout of all the\nobjects. But still it seems that when I branch and publish via http it\nwants to upload zillions of objects.\n\nFor example see the attached webkit-http.txt log. I think there is\nsome bug in the http push transport as if the transport is ssh it's\nmuch *much* faster:\n\nx61:~/aplix/webkit-test/webkit% git-config remote.origin.url\nssh://git.webvm.net/srv/git/webkit\nx61:~/aplix/webkit-test/webkit% git push origin deleteme\nCounting objects: 5, done.\nCompressing objects: 100% (3/3), done.\nWriting objects: 100% (3/3), 289 bytes, done.\nTotal 3 (delta 2), reused 0 (delta 0)\nTo ssh://git.webvm.net/srv/git/webkit\n * [new branch]      deleteme -> deleteme\nx61:~/aplix/webkit-test/webkit%\n\nI need http:// to work as some of the engineers I work with are behind\na \"corporate firewall\" that doesn't allow ssh://. Crazy, I know.\n\n\nx61:~/aplix/webkit-test% git clone git://git.webvm.net/webkit\nInitialized empty Git repository in /home/hendry/aplix/webkit-test/webkit/.git/\nremote: Counting objects: 406041, done.\nremote: Compressing objects: 100% (98138/98138), done.\nremote: Total 406041 (delta 295642), reused 398766 (delta 288407)\nReceiving objects: 100% (406041/406041), 440.93 MiB | 2484 KiB/s, done.\nResolving deltas: 100% (295642/295642), done.\nChecking out files: 100% (44750/44750), done.\nx61:~/aplix/webkit-test% ll\ntotal 12K\ndrwxr-xr-x  3 hendry hendry 4.0K 2008-04-08 13:39 ./\ndrwxr-xr-x 20 hendry hendry 4.0K 2008-04-08 13:22 ../\ndrwxr-xr-x 15 hendry hendry 4.0K 2008-04-08 13:54 webkit/\nx61:~/aplix/webkit-test% cd webkit\nx61:~/aplix/webkit-test/webkit% vim .git/config\nx61:~/aplix/webkit-test/webkit% git-config remote.origin.url http://moto@moto.webvm.net/webkit.git\nx61:~/aplix/webkit-test/webkit% vim .git/config\nx61:~/aplix/webkit-test/webkit% ll\ntotal 220K\ndrwxr-xr-x 15 hendry hendry 4.0K 2008-04-08 13:54 ./\ndrwxr-xr-x  3 hendry hendry 4.0K 2008-04-08 13:39 ../\n-rwxr-xr-x  1 hendry hendry 1.6K 2008-04-08 13:54 autogen.sh*\ndrwxr-xr-x 12 hendry hendry 4.0K 2008-04-08 13:54 BugsSite/\n-rw-r--r--  1 hendry hendry 104K 2008-04-08 13:54 ChangeLog\n-rw-r--r--  1 hendry hendry  19K 2008-04-08 13:54 configure.ac\ndrwxr-xr-x  8 hendry hendry 4.0K 2008-04-08 14:34 .git/\n-rw-r--r--  1 hendry hendry 9.6K 2008-04-08 13:54 GNUmakefile.am\ndrwxr-xr-x 13 hendry hendry 4.0K 2008-04-08 13:54 JavaScriptCore/\ndrwxr-xr-x  7 hendry hendry 4.0K 2008-04-08 13:54 JavaScriptGlue/\ndrwxr-xr-x 18 hendry hendry 4.0K 2008-04-08 13:54 LayoutTests/\n-rw-r--r--  1 hendry hendry 1.1K 2008-04-08 13:54 Makefile\n-rw-r--r--  1 hendry hendry 1.1K 2008-04-08 13:54 Makefile.shared\ndrwxr-xr-x  3 hendry hendry 4.0K 2008-04-08 13:54 PageLoadTests/\ndrwxr-xr-x  5 hendry hendry 4.0K 2008-04-08 13:54 PlanetWebKit/\ndrwxr-xr-x  5 hendry hendry 4.0K 2008-04-08 13:54 SunSpider/\ndrwxr-xr-x 25 hendry hendry 4.0K 2008-04-08 13:54 WebCore/\ndrwxr-xr-x  9 hendry hendry 4.0K 2008-04-08 13:54 WebKit/\ndrwxr-xr-x  4 hendry hendry 4.0K 2008-04-08 13:54 WebKitLibraries/\n-rw-r--r--  1 hendry hendry 3.3K 2008-04-08 13:54 WebKit.pri\n-rw-r--r--  1 hendry hendry  474 2008-04-08 13:54 WebKit.pro\ndrwxr-xr-x 14 hendry hendry 4.0K 2008-04-08 13:54 WebKitSite/\ndrwxr-xr-x 19 hendry hendry 4.0K 2008-04-08 13:54 WebKitTools/\nx61:~/aplix/webkit-test/webkit% git checkout -b deleteme\nSwitched to a new branch \"deleteme\"\nx61:~/aplix/webkit-test/webkit% vim ChangeLog\nx61:~/aplix/webkit-test/webkit% git ci\nCreated commit 4b56425: Test\n 1 files changed, 1 insertions(+), 0 deletions(-)\nx61:~/aplix/webkit-test/webkit% git push origin deleteme\nFetching remote heads...\n  refs/\n  refs/heads/\n  refs/heads/origin/\n  refs/remotes/\n  refs/remotes/webkit/\n  refs/remotes/origin/\n  refs/remotes/debian/\n  refs/tags/\nupdating 'refs/heads/deleteme'\n  from 0000000000000000000000000000000000000000\n  to   4b5642541a73f105cf6e99b0b0c4ee00f0b869c2\n    sending 29257 objects\n"},{"id":"73862","messageId":"7vmyo4mbb6.fsf@gitster.siamese.dyndns.org","threadId":"13012","inReplyTo":"b24851260804080656u4c42c31cv3984fa2943d83b8a@mail.gmail.com","subject":"Re: Integration branching","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-04-08T18:49:33Z","receivedAt":"2008-04-08T18:49:33Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Kai Hendry\" <hendry@aplixcorp.com> writes:\n\n> I need http:// to work as some of the engineers I work with are behind\n> a \"corporate firewall\" that doesn't allow ssh://. Crazy, I know.\n\nIt is not \"crazy\", but simply is \"unfortunate\".  I am not involved in\nhttp-push at all myself, but I recall there was somebody working on\nimproving it recently?\n"},{"id":"73866","messageId":"20080408190127.GA28556@glandium.org","threadId":"13012","inReplyTo":"7vmyo4mbb6.fsf@gitster.siamese.dyndns.org","subject":"Re: Integration branching","fromName":"Mike Hommey","fromEmail":"mh@glandium.org","sentAt":"2008-04-08T19:01:27Z","receivedAt":"2008-04-08T19:01:27Z","isPatch":false,"sender":{"key":"mh@glandium.org","avatar":"https://avatars.githubusercontent.com/u/1038527?v=4"},"body":"On Tue, Apr 08, 2008 at 11:49:33AM -0700, Junio C Hamano wrote:\n> \"Kai Hendry\" <hendry@aplixcorp.com> writes:\n> \n> > I need http:// to work as some of the engineers I work with are behind\n> > a \"corporate firewall\" that doesn't allow ssh://. Crazy, I know.\n> \n> It is not \"crazy\", but simply is \"unfortunate\".  I am not involved in\n> http-push at all myself, but I recall there was somebody working on\n> improving it recently?\n\nThat was me, but it has been stalled by various debian work in progress.\n\nI hope to be able to work on http-push again in a couple weeks.\n\nMike\n"},{"id":"73873","messageId":"b24851260804081332l74beb7e6o2f48b18f65af7b55@mail.gmail.com","threadId":"13012","inReplyTo":"20080408190127.GA28556@glandium.org","subject":"Re: Integration branching","fromName":"Kai Hendry","fromEmail":"hendry@aplixcorp.com","sentAt":"2008-04-08T20:32:31Z","receivedAt":"2008-04-08T20:32:31Z","isPatch":false,"sender":{"key":"hendry@aplixcorp.com","avatar":null},"body":"Hi Mike!\n\nYou're busy. :)\n\nSo do you acknowledge this as a bug or am I doing something wrong?\n\nIf you know where the problem lies, perhaps I can get a C ninja from\nwork to look at it.\n\n\n\nKind regards from the UK,\n"},{"id":"73876","messageId":"20080408204319.GA12323@glandium.org","threadId":"13012","inReplyTo":"b24851260804081332l74beb7e6o2f48b18f65af7b55@mail.gmail.com","subject":"Re: Integration branching","fromName":"Mike Hommey","fromEmail":"mh@glandium.org","sentAt":"2008-04-08T20:43:19Z","receivedAt":"2008-04-08T20:43:19Z","isPatch":false,"sender":{"key":"mh@glandium.org","avatar":"https://avatars.githubusercontent.com/u/1038527?v=4"},"body":"On Tue, Apr 08, 2008 at 09:32:31PM +0100, Kai Hendry wrote:\n> Hi Mike!\n> \n> You're busy. :)\n> \n> So do you acknowledge this as a bug or am I doing something wrong?\n> \n> If you know where the problem lies, perhaps I can get a C ninja from\n> work to look at it.\n\nCould you provide a trace with GIT_CURL_VERBOSE set so that i can get a\nbetter idea ?\n\nMike\n"},{"id":"74094","messageId":"20080410200413.GA16951@glandium.org","threadId":"13012","inReplyTo":"20080408204319.GA12323@glandium.org","subject":"Re: Integration branching","fromName":"Mike Hommey","fromEmail":"mh@glandium.org","sentAt":"2008-04-10T20:04:13Z","receivedAt":"2008-04-10T20:04:13Z","isPatch":false,"sender":{"key":"mh@glandium.org","avatar":"https://avatars.githubusercontent.com/u/1038527?v=4"},"body":"On Tue, Apr 08, 2008 at 10:43:19PM +0200, Mike Hommey wrote:\n> On Tue, Apr 08, 2008 at 09:32:31PM +0100, Kai Hendry wrote:\n> > Hi Mike!\n> > \n> > You're busy. :)\n> > \n> > So do you acknowledge this as a bug or am I doing something wrong?\n> > \n> > If you know where the problem lies, perhaps I can get a C ninja from\n> > work to look at it.\n> \n> Could you provide a trace with GIT_CURL_VERBOSE set so that i can get a\n> better idea ?\n\nFWIW, it was made clear from off-list discussion with Kai that his\nproblem was a lack of running git update-server-info when required.\n\nMike\n"}]}