{"thread":{"id":"720","subject":"[doc][git] playing with git, and netdev/libata-dev trees","startedAt":"2005-05-26T05:26:15Z","lastAt":"2005-06-04T18:16:07Z","messageCount":7,"participants":["Jeff Garzik","Frank Sorenson","James Purser","Dave Airlie"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"3976","messageId":"42955DF7.4000805@pobox.com","threadId":"720","inReplyTo":null,"subject":"[doc][git] playing with git, and netdev/libata-dev trees","fromName":"Jeff Garzik","fromEmail":"jgarzik@pobox.com","sentAt":"2005-05-26T05:26:15Z","receivedAt":"2005-05-26T05:26:15Z","isPatch":false,"sender":{"key":"jgarzik@pobox.com","avatar":null},"body":"\nHopefully, this email can quick-start some people on git.\n\nOne of the things Linus's new 'git' tool allows me to do is make public \nthe 50+ repositories that were previously only available on my local \nworkstation.  This should make it a lot easier for developers to see \nprecisely what I have merged, and makes generating follow-up patches a \nwhole lot easier.\n\nWhen I merge a patch for drivers/net/forcedeth.c, I merge it into a \nbrand new 'forcedeth' repository, a peer to the 40+ other such \nrepository.  Under BitKeeper, I made these repositories available merged \ntogether into one big \"netdev-2.6\" repository because it was too time \nconsuming to make the individual 50+ trees publicly available.  With \ngit, developers have direct access to the individual trees.\n\nI thought I would write up a quick guide describing how to mess around \nwith the netdev and libata-dev trees, and with git in general.\n\n\n1) installing git\n\ngit requires bootstrapping, since you must have git installed in order \nto check out git.git (git repo), and linux-2.6.git (kernel repo).  I \nhave put together a bootstrap tarball of today's git repository.\n\nDownload tarball from:\nhttp://www.kernel.org/pub/linux/kernel/people/jgarzik/git-20050526.tar.bz2\n\ntarball build-deps:  zlib, libcurl\n\ninstall tarball:  unpack && make && sudo make prefix=/usr/local install\n\njgarzik helper scripts, not in official git distribution:\nhttp://www.kernel.org/pub/linux/kernel/people/jgarzik/git-switch-tree\nhttp://www.kernel.org/pub/linux/kernel/people/jgarzik/git-new-branch\nhttp://www.kernel.org/pub/linux/kernel/people/jgarzik/git-changes-script\n\nAfter reading the rest of this document, come back and update your copy \nof git to the latest:\nrsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/git.git\n\n\n2) download a linux kernel tree for the very first time\n\nmkdir -p linux-2.6/.git\ncd linux-2.6\nrsync -a --delete --verbose --stats --progress \\ \nrsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/ \n\\          <- word-wrapped backslash; sigh\n    .git/\n\n\n3) download latest changes to on-disk local tree\n\ncd linux-2.6\ngit-pull-script \\\nrsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git\n\n\n4) check out files from the git repository into the working directory\n\ncd linux-2.6\ngit-read-tree -m HEAD && git-checkout-cache -q -f -u -a\n\n\n5) check in your own modifications (e.g. apply a patch)\n\n# go to repo\ncd linux-2.6\n\n# make some modifications\npatch -sp1 < /tmp/my.patch\ndiffstat -p1 < /tmp/my.patch\n\n# NOTE: add '--add' and/or '--remove' if files were added or removed\ngit-update-cache <list of all files changed>\n\n# commit changes\nGIT_AUTHOR_NAME=\"John Doe\"\t\t\\\n    GIT_AUTHOR_EMAIL=\"jdoe@foo.com\"\t\\\n    GIT_COMMITTER_NAME=\"Jeff Garzik\"\t\\\n    GIT_COMMITTER_EMAIL=\"jgarzik@pobox.com\"\t\\\n    git-commit-tree `git-write-tree`\t\\\n    -p $(cat .git/HEAD )\t\t\t\\\n    < changelog.txt\t\t\t\\\n    > .git/HEAD\n\n\n6) List all changes in working dir, in diff format.\n\ngit-diff-cache -p HEAD\n\n\n7) List all changesets (i.e. show each cset's description text) in local \ntree that are not present in remote tree.\n\ncd my-kernel-tree-2.6\ngit-changes-script -L ../linux-2.6 | less\n\n\n8) List all changesets:\n\ngit-whatchanged\n\n\n9) apply all patches in a Berkeley mbox-format file\n\nFirst, download and add to your PATH Linus's git tools:\nrsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/git-tools.git\n\ncd my-kernel-tree-2.6\ndotest /path/to/mbox  # yes, Linus has no taste in naming useful scripts\n\n\n10) don't forget to download tags from time to time.\n\ngit-pull-script only downloads sha1-indexed object data, and the \nrequested remote head.  This misses updates to the .git/refs/tags/ and \n.git/refs/heads directories.  It is advisable to update your kernel .git \ndirectories periodically with a full rsync command, to make sure you got \neverything:\n\ncd linux-2.6\nrsync -a --delete --verbose --stats --progress \\ \nrsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/ \n\\          <- word-wrapped backslash; sigh\n    .git/\n\n\n11) [jg-specific] list all branches found in netdev-2.6 or libata-dev trees.\n\nDownload\nrsync://rsync.kernel.org/pub/scm/linux/kernel/git/jgarzik/netdev-2.6.git\n\tor\nrsync://rsync.kernel.org/pub/scm/linux/kernel/git/jgarzik/libata-dev.git\n\n\ncd netdev-2.6\nls .git/refs/heads/\n\n{ these are the current netdev-2.6 branches }\n> 8139cp       forcedeth    master     qeth           smc91x         we18\n> 8139too-iomap  for-linus    natsemi      r8169      smc91x-eeprom  wifi\n> airo           hdlc         ns83820      register-netdev  starfire\n> atmel          ieee80211    orinoco      remove-drivers   tlan\n> chelsio        iff-running  orinoco-hch  sis900           veth\n> dm9000         janitor      ppp          skge             viro\n\n\n\n12) [jg-specific] make desired branch current in working directory\n\ngit-switch-tree $branch\n\n\n13) [jg-specific] create a new branch, and make it current\n\ngit-new-branch $branch\n\n\n14) [jg-specific] examine which branch is current\n\nls -l .git/HEAD\n\n\n15) undo all local modifications (same as checkout):\n\ngit-read-tree -m HEAD && git-checkout-cache -q -f -u -a\n\n\n\n"},{"id":"3977","messageId":"4295868A.5070509@tuxrocks.com","threadId":"720","inReplyTo":"42955DF7.4000805@pobox.com","subject":"Re: [doc]playing with git, and netdev/libata-dev trees","fromName":"Frank Sorenson","fromEmail":"frank@tuxrocks.com","sentAt":"2005-05-26T08:19:22Z","receivedAt":"2005-05-26T08:19:22Z","isPatch":false,"sender":{"key":"frank@tuxrocks.com","avatar":null},"body":"-----BEGIN PGP SIGNED MESSAGE-----\nHash: SHA1\n\nJeff Garzik wrote:\n> Hopefully, this email can quick-start some people on git.\n\nI think the quick-start is great.  Definitely needed to get people\nup-to-speed with git.\n\n> 1) installing git\n> \n> git requires bootstrapping, since you must have git installed in order\n> to check out git.git (git repo), and linux-2.6.git (kernel repo).  I\n> have put together a bootstrap tarball of today's git repository.\n> \n> Download tarball from:\n> http://www.kernel.org/pub/linux/kernel/people/jgarzik/git-20050526.tar.bz2\n\nA tarball of git will work, but it's a big bootstrap, and will need\nperiodic updating.\n\nI'm curious whether we couldn't make a git-bootstrap that contained a\nsignificantly stripped-down version that did nothing other than\nbootstrap git.  A single program/file (perl script perhaps?) that...\n\nmkdir git\ncd git\nrsync -rl rsync://rsync.kernel.org/pub/scm/git/git.git/ .git\necho rsync://rsync.kernel.org/pub/scm/git/git.git > .git/branches/origin\nbeginning with .git/HEAD, start extracting files\n\n...and nothing more.  Then, we could tell people to just download\ngit-bootstrap and run it to create an up-to-date git repo.\n\nFrank\n- --\nFrank Sorenson - KD7TZK\nSystems Manager, Computer Science Department\nBrigham Young University\nfrank@tuxrocks.com\n-----BEGIN PGP SIGNATURE-----\nVersion: GnuPG v1.2.6 (GNU/Linux)\nComment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org\n\niD8DBQFClYaKaI0dwg4A47wRAp1MAKCcGK8vTWtw1gnTCjFbMFpbZkSO8QCff5RE\nNC8Z7RVHFP4qcbKRMSJ2rzg=\n=fXsr\n-----END PGP SIGNATURE-----\n"},{"id":"3983","messageId":"1117108478.3102.116.camel@kryten","threadId":"720","inReplyTo":"4295868A.5070509@tuxrocks.com","subject":"Re: [doc]playing with git, and netdev/libata-dev trees","fromName":"James Purser","fromEmail":"purserj@k-sit.com","sentAt":"2005-05-26T11:54:38Z","receivedAt":"2005-05-26T11:54:38Z","isPatch":false,"sender":{"key":"purserj@k-sit.com","avatar":null},"body":"On Thu, 2005-05-26 at 18:19, Frank Sorenson wrote:\n\n> A tarball of git will work, but it's a big bootstrap, and will need\n> periodic updating.\n> \n\nI don't really see the problem with an initial install from a tarball.\nIf the tarball is a direct pull of the git.git tree .git/ and all then\nall anyone has to do to keep it up to date is either grab a copy of\ncogito, or set up their own git based scripts.\n\n\n-- \nJames Purser\nhttp://k-sit.com\n\n"},{"id":"4532","messageId":"21d7e99705060401405cfd5a11@mail.gmail.com","threadId":"720","inReplyTo":"42955DF7.4000805@pobox.com","subject":"Re: [doc]playing with git, and netdev/libata-dev trees","fromName":"Dave Airlie","fromEmail":"airlied@gmail.com","sentAt":"2005-06-04T08:40:40Z","receivedAt":"2005-06-04T08:40:40Z","isPatch":false,"sender":{"key":"airlied@gmail.com","avatar":"https://gravatar.com/avatar/8ed1990dd745eed102534e9b1629e8f7adb5ae47b01640093ca24a9b2174bb02?d=mp&s=160"},"body":"> \n> When I merge a patch for drivers/net/forcedeth.c, I merge it into a\n> brand new 'forcedeth' repository, a peer to the 40+ other such\n> repository.  Under BitKeeper, I made these repositories available merged\n> together into one big \"netdev-2.6\" repository because it was too time\n> consuming to make the individual 50+ trees publicly available.  With\n> git, developers have direct access to the individual trees.\n> \n> I thought I would write up a quick guide describing how to mess around\n> with the netdev and libata-dev trees, and with git in general.\n> \n\nThanks for this, I'm starting to get up to speed on git now...\n\nTwo questions,\n\n1. when you want to publish your tree what do you do? just rsync it\nonto kernel.org?\n2. When you are taking things from your queue for Linus do you create\nanother tree and merge your branches into it or what?\n\nDave.\n"},{"id":"4533","messageId":"42A181C1.3010902@pobox.com","threadId":"720","inReplyTo":"21d7e99705060401405cfd5a11@mail.gmail.com","subject":"Re: [doc]playing with git, and netdev/libata-dev trees","fromName":"Jeff Garzik","fromEmail":"jgarzik@pobox.com","sentAt":"2005-06-04T10:26:09Z","receivedAt":"2005-06-04T10:26:09Z","isPatch":false,"sender":{"key":"jgarzik@pobox.com","avatar":null},"body":"Dave Airlie wrote:\n>>When I merge a patch for drivers/net/forcedeth.c, I merge it into a\n>>brand new 'forcedeth' repository, a peer to the 40+ other such\n>>repository.  Under BitKeeper, I made these repositories available merged\n>>together into one big \"netdev-2.6\" repository because it was too time\n>>consuming to make the individual 50+ trees publicly available.  With\n>>git, developers have direct access to the individual trees.\n>>\n>>I thought I would write up a quick guide describing how to mess around\n>>with the netdev and libata-dev trees, and with git in general.\n>>\n> \n> \n> Thanks for this, I'm starting to get up to speed on git now...\n> \n> Two questions,\n> \n> 1. when you want to publish your tree what do you do? just rsync it\n> onto kernel.org?\n\nBasically.  I copy the attached script into each repo, customize the \nscript for the upload destination.\n\nWhen I publish the tree, I just cd to the toplevel dir on my local \nworkstation, and run \"./push\"\n\n\n> 2. When you are taking things from your queue for Linus do you create\n> another tree and merge your branches into it or what?\n\nNot quite sure what you're asking, but I'll attempt to answer anyway :)\n\nWhen I prepare a submission for Linus, I will pull several branches into \na single 'for-linus' conglomeration branch.  I do this with the \nfollowing incantation, for each branch I am pushing upstream:\n\ngit-resolve-script $(cat .git/HEAD) $(cat .git/refs/heads/$branch) \\\n\t`pwd` $branch\n\n(Note that I modified my git-resolve-script to take an additional $4 \nargument, which causes the string \" branch $4\" to be added to the merge \ncset's commit msg)\n\nAfter I take care of all the merge conflicts[1] from the \ngit-resolve-script run, I give the big pile of mush a final build and \ntest, and then run \"./push\".\n\nFor the \"please pull\" email, I use diffstat, git-diff-tree, \ngit-changes-script and the newly written git-shortlog.\n\n\tJeff\n\n\n\n[1] I'm still scared of conflicts in the merge process.  Simple and \nautomatic merging works just fine, like it did under BitKeeper.  But if \nthere are conflicts that cause git-pull-script/git-resolve-script to \nbail, then I bail as well:  I export a patch, run patch(1), and then \nhandle the merge the Old Fashioned Way(tm) by looking at .rej files.\n\nI really wish somebody would write a merge helper for git that places \nthe conflicts side-by-side in the code [in the working dir].  BitKeeper \nand CVS both presented conflicts to you in this manner.\n\nThe \"I resolved this conflict, now let's continue where we left off\" \nprocess is still quite raw in git.  I suppose this is something that is \nleft for others to script, above the plumbing, but hey...\n\n\n\n#!/bin/sh\n\nrsync -e ssh --verbose --delete --stats --progress -az .git/ master.kernel.org:/pub/scm/linux/kernel/git/jgarzik/misc-2.6.git\n"},{"id":"4534","messageId":"21d7e99705060403312234aa07@mail.gmail.com","threadId":"720","inReplyTo":"42A181C1.3010902@pobox.com","subject":"Re: [doc]playing with git, and netdev/libata-dev trees","fromName":"Dave Airlie","fromEmail":"airlied@gmail.com","sentAt":"2005-06-04T10:31:34Z","receivedAt":"2005-06-04T10:31:34Z","isPatch":false,"sender":{"key":"airlied@gmail.com","avatar":"https://gravatar.com/avatar/8ed1990dd745eed102534e9b1629e8f7adb5ae47b01640093ca24a9b2174bb02?d=mp&s=160"},"body":"> > 1. when you want to publish your tree what do you do? just rsync it\n> > onto kernel.org?\n> \n> Basically.  I copy the attached script into each repo, customize the\n> script for the upload destination.\n> \n> When I publish the tree, I just cd to the toplevel dir on my local\n> workstation, and run \"./push\"\n> \n> \n> > 2. When you are taking things from your queue for Linus do you create\n> > another tree and merge your branches into it or what?\n> \n> Not quite sure what you're asking, but I'll attempt to answer anyway :)\n\nYes that's what I'm asking, mainly the pulling of multiple trees into\none tree for giving to Linus, for Andrew I'm quite happy to have him\npull multiple HEADs from the one tree assuming I don't have many\ninterdependencies between trees...\n\nSay I want something like this one tree with\n\ndrm-2.6 - HEAD <- linus tree\n            - drm-via < a via driver\n            - drm-initmap \n                      - drm-savage <- a savage driver that depends on\nthe drm-initmap tree\n\nHow would I construct such a beast, how does it work out from where to\nbranch, can I branch a branch for something like drm-savage so I can\nsay send Linus the initmap branch and then have -mm pulling the savage\none...\n\nif that makes any sense :-)\n\nDave.\n> \n> \n> [1] I'm still scared of conflicts in the merge process.  Simple and\n> automatic merging works just fine, like it did under BitKeeper.  But if\n> there are conflicts that cause git-pull-script/git-resolve-script to\n> bail, then I bail as well:  I export a patch, run patch(1), and then\n> handle the merge the Old Fashioned Way(tm) by looking at .rej files.\n> \n> I really wish somebody would write a merge helper for git that places\n> the conflicts side-by-side in the code [in the working dir].  BitKeeper\n> and CVS both presented conflicts to you in this manner.\n> \n> The \"I resolved this conflict, now let's continue where we left off\"\n> process is still quite raw in git.  I suppose this is something that is\n> left for others to script, above the plumbing, but hey...\n> \n> \n> \n> #!/bin/sh\n> \n> rsync -e ssh --verbose --delete --stats --progress -az .git/ master.kernel.org:/pub/scm/linux/kernel/git/jgarzik/misc-2.6.git\n> \n> \n>\n"},{"id":"4551","messageId":"42A1EFE7.2080903@pobox.com","threadId":"720","inReplyTo":"21d7e99705060403312234aa07@mail.gmail.com","subject":"Re: [doc]playing with git, and netdev/libata-dev trees","fromName":"Jeff Garzik","fromEmail":"jgarzik@pobox.com","sentAt":"2005-06-04T18:16:07Z","receivedAt":"2005-06-04T18:16:07Z","isPatch":false,"sender":{"key":"jgarzik@pobox.com","avatar":null},"body":"Dave Airlie wrote:\n>>>1. when you want to publish your tree what do you do? just rsync it\n>>>onto kernel.org?\n>>\n>>Basically.  I copy the attached script into each repo, customize the\n>>script for the upload destination.\n>>\n>>When I publish the tree, I just cd to the toplevel dir on my local\n>>workstation, and run \"./push\"\n>>\n>>\n>>\n>>>2. When you are taking things from your queue for Linus do you create\n>>>another tree and merge your branches into it or what?\n>>\n>>Not quite sure what you're asking, but I'll attempt to answer anyway :)\n> \n> \n> Yes that's what I'm asking, mainly the pulling of multiple trees into\n> one tree for giving to Linus, for Andrew I'm quite happy to have him\n> pull multiple HEADs from the one tree assuming I don't have many\n> interdependencies between trees...\n> \n> Say I want something like this one tree with\n> \n> drm-2.6 - HEAD <- linus tree\n>             - drm-via < a via driver\n>             - drm-initmap \n>                       - drm-savage <- a savage driver that depends on\n> the drm-initmap tree\n> \n> How would I construct such a beast, how does it work out from where to\n> branch, can I branch a branch for something like drm-savage so I can\n> say send Linus the initmap branch and then have -mm pulling the savage\n> one...\n\nYou used to use BitKeeper, right?\n\nWell, git works just fine for creating and maintaining separate repos, \nwhere each repo is its own branch.  Dependent branches are handled \nsimply by pulling new changes from the parent (drm-initmap) repo into \nthe child (drm-savage) repo.\n\nIf you just have a few branches, such as your example above, rather than \nthe 50+ branches I have, then I would recommend _ignoring_ the advice in \nmy git howto, and instead create a new branch by cloning:\n\n\tcd /repos\n\tmkdir drm-via\n\tcp -a linux-2.6/.git drm-via\n\trm -rf drm-via/.git/objects\n\tcp -al linux-2.6/.git/objects drm-via/.git\n\nOnce created by the above recipes, or similar, the directory will \nfunction as a repo just like with BK.  When you wish to update it from \nupstream, update your local linux-2.6 repo and then\n\n\tcd /repos/drm-via\n\tgit-pull-script /repos/linux-2.6/.git\n\nor to pull new changes from initmap into savage:\n\n\tcd /repos/drm-initmap\n\t# { check in some changes }\n\tcd /repos/drm-savage\n\tgit-pull-script /repos/drm-initmap/.git\n\nDoes all that make sense?  It's really quite similar to BK, if you \nignore the \"[jg-specific]\" portions of the git howto I wrote.\n\n\tJeff\n\n\n"}]}