{"thread":{"id":"27491","subject":"How would Git chapter look like in \"The Architecture of Open Source Applications\"?","startedAt":"2011-05-28T12:17:38Z","lastAt":"2011-05-30T10:30:10Z","messageCount":3,"participants":["Jakub Narebski","Jeff King"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"168911","messageId":"201105281417.39883.jnareb@gmail.com","threadId":"27491","inReplyTo":null,"subject":"How would Git chapter look like in \"The Architecture of Open Source Applications\"?","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2011-05-28T12:17:38Z","receivedAt":"2011-05-28T12:17:38Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":">From LWN.net (http://lwn.net/Articles/444981/):\n\n  \"The Architecture of Open Source Applications\"[1] is a new book with\n  chapters on the design of a wide variety of programs, including\n  Asterisk, bash, Eclipse, LLVM, *Mercurial*, Sendmail, Telepathy,\n  and many more. It's available for purchase or downloadable under\n  the terms of the CC Attribution 3.0 license; some readers have\n  already taken advantage of that license to make an epub[2] version\n  available. Revenue from sales go to Amnesty International.\n\n[1]: http://www.aosabook.org/en/\n[2]: http://media.dropdo.com.s3.amazonaws.com/2Wo/Architecture%20of%20Open%20Source%20Applications.epub\n\nAmong covered programs is Mercurial (chapter by Dirkjan Ochtman)...\nbut unfortunately no Git (they probably thought that one DVCS is enough).\n\nHow would such chapter on Git look like?  Authors of this book\nencourage (among others) to write new chapters.\n\n-- \nJakub Narebski\nPoland\n"},{"id":"168966","messageId":"20110530034044.GC27691@sigill.intra.peff.net","threadId":"27491","inReplyTo":"201105281417.39883.jnareb@gmail.com","subject":"Re: How would Git chapter look like in \"The Architecture of Open Source Applications\"?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2011-05-30T03:40:45Z","receivedAt":"2011-05-30T03:40:45Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sat, May 28, 2011 at 02:17:38PM +0200, Jakub Narebski wrote:\n\n> Among covered programs is Mercurial (chapter by Dirkjan Ochtman)...\n> but unfortunately no Git (they probably thought that one DVCS is enough).\n> \n> How would such chapter on Git look like?  Authors of this book\n> encourage (among others) to write new chapters.\n\nI just skimmed the Mercurial chapter, but they do cover a fair bit of\ngeneral DVCS architecture. For git, I would guess a good approach would\nbe to describe the data structures (i.e., content-addressable object\ndatabase, DAG of commits, refs storing branches and tags), as everything\nelse falls out from there. Most of the basic commands can be explained\nas \"do some simple operation to the history graph or object db\" and the\nmore complex commands are compositions of the simple ones. So the\narchitecture is really about having a data structure that represents the\nproblem, exposing it to the user, and then building some niceties around\nthe basic data structure operations.\n\nOf course that's just my perspective. Linus might have written something\ntotally different. :)\n\n-Peff\n"},{"id":"168985","messageId":"201105301230.11772.jnareb@gmail.com","threadId":"27491","inReplyTo":"20110530034044.GC27691@sigill.intra.peff.net","subject":"Re: How would Git chapter look like in \"The Architecture of Open Source Applications\"?","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2011-05-30T10:30:10Z","receivedAt":"2011-05-30T10:30:10Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"On Mon, 30 May 2011, Jeff King <peff@peff.org> wrote:\n> On Sat, May 28, 2011 at 02:17:38PM +0200, Jakub Narebski wrote:\n> \n> > Among covered programs is Mercurial (chapter by Dirkjan Ochtman)...\n> > but unfortunately no Git (they probably thought that one DVCS is enough).\n> > \n> > How would such chapter on Git look like?  Authors of this book\n> > encourage (among others) to write new chapters.\n> \n> I just skimmed the Mercurial chapter, but they do cover a fair bit of\n> general DVCS architecture. For git, I would guess a good approach would\n> be to describe the data structures (i.e., content-addressable object\n> database, DAG of commits, refs storing branches and tags), as everything\n> else falls out from there. Most of the basic commands can be explained\n> as \"do some simple operation to the history graph or object db\" and the\n> more complex commands are compositions of the simple ones. So the\n> architecture is really about having a data structure that represents the\n> problem, exposing it to the user, and then building some niceties around\n> the basic data structure operations.\n\nThe repository model that Git uses is quite well described in \"Pro Git\",\nin \"Discussion\" section of git(1) manpage, in \"Git concepts\" section of\nGit User's Manual and in gitcore-tutorial(7).\n\nWhat I am more interested in is design *goals*, i.e. what's behind\nchoosing this and not other architecture.  \n\nThe chapter on Mercurial, in '12.2. Data Structures > 12.2.1. Challenges'\nsubsection says about limiting technology factors (quoting [Mac06]):\n * speed: CPU\n * capacity: disk and memory\n * bandwidth: memory, LAN, disk, and WAN\n * disk seek rate\n\nThis was for Mercurial; from what I remember from KernelTrap articles,\nwhich covered beginnings of Git development quite well, and from other\nsources, the main limiting factor considered was __speed__.  \n\nNot disk space.  At first Git had only 'loose' format -- do you remember\n\"disk space is cheap\" comment by Linus?  Admittedly Git used zlib\ncompression from very beginning (which works well for text).  IIRC at\nfirst when _model_ that Git uses for repository was being drafted\nLAN/WAN bandwidth wasn't consideration; AFAIK first transport that Git\nused was nowadays deprecated rsync:// (UNIX philosophy of prototyping\nand developing using existing ready tools, see [TAOUP], [Ben86]).\n\nI think it was assumed that operating system would be good enough that\nwe don't have to worry about seek rates: Git is optimized for \"hot cache\"\ncase.  Note however that adoption of 'packed' format as on-disk format\nwas driven by speed (disk seek rate) as well as disk capacity i.e. \nreducing repository size.  Well, at least from what I remember.\n\n\nThe Mercurial's '12.2.1. Challenges' subsection continues from:\n\n  The paper [i.e. [Mac06]] goes on to review common scenarios or\n  criteria for evaluating the performance of such a system at\n  the file level:\n\n    * Storage compression: what kind of compression is best suited\n      to save the file history on disk? Effectively, what algorithm\n      makes the most out of the I/O performance while preventing\n      CPU time from becoming a bottleneck?\n    * Retrieving arbitrary file revisions: a number of version control\n      systems will store a given revision in such a way that a large\n      number of older revisions must be read to reconstruct the newer\n      one (using deltas). We want to control this to make sure that\n      retrieving old revisions is still fast.\n    * Adding file revisions: we regularly add new revisions. We don't\n      want to rewrite old revisions every time we add a new one, because\n      that would become too slow when there are many revisions.\n    * Showing file history: we want to be able to review a history of\n      all changesets that touched a certain file. This also allows us\n      to do annotations (which used to be called `blame` in CVS but was\n      renamed to `annotate` in some later systems to remove the negative\n      connotation): reviewing the originating changeset for each line\n      currently in a file.\n\n>From what *I* understand Linus approached the problem of DVCS design\nfrom different direction: he is maintainer rather than ordinary developer,\nand (from what he said) filesystem designer at heart, and not version\ncontrol developer.  Thus the common scenarios or criteria were different:\n\n * Merging and applying patches\n * Showing _subsystem_ history\n * ???\n\nThat is what I am interested in.\n\n\nSome of Git history, and I think of motivations behind design, can be\nfound in \"Git Chronicle\" slides by Junio from GitTogether.\n\n> Of course that's just my perspective. Linus might have written something\n> totally different. :)\n\nWell, only Linus can be definitive source of initial *design goals*\n(behind core design of Git)...\n\n\nReferences:\n~~~~~~~~~~~\n[Mac06]: Matt Mackall: \"Towards a Better SCM: Revlog and Mercurial\".\n         2006 Ottawa Linux Symposium, 2006.\n         http://selenic.com/mercurial/wiki/index.cgi/Presentations?action=AttachFile&do=get&target=ols-mercurial-paper.pdf\n         (see also http://mercurial.selenic.com/wiki/Presentations)\n[TAOUP]: Eric Raymond: \"The Art of Unix Programming\", 2003\n         http://www.faqs.org/docs/artu/\n         http://www.catb.org/~esr/writings/taoup/\n[Ben86]: Jon Bentley: \"Programming Pearls\", chapter about implementing\n         and prototyping UNIX 'spell' program (from Polish translation).\n-- \nJakub Narebski\nPoland\n"}]}