{"thread":{"id":"16110","subject":"libgit2 - a true git library","startedAt":"2008-10-31T17:07:04Z","lastAt":"2008-11-09T21:02:48Z","messageCount":83,"participants":["Shawn O. Pearce","Pieter de Bie","Pierre Habouzit","Junio C Hamano","Nicolas Pitre","Brian Gernhardt","david@lang.hm","Andreas Ericsson","Johannes Schindelin","Jakub Narebski","Bruno Santos","Scott Chacon","David Brown","Steve Frécinaux"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"94424","messageId":"20081031170704.GU14786@spearce.org","threadId":"16110","inReplyTo":null,"subject":"libgit2 - a true git library","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2008-10-31T17:07:04Z","receivedAt":"2008-10-31T17:07:04Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"During the GitTogether we were kicking around the idea of a ground-up\nimplementation of a Git library. This may be easier than trying\nto grind down git.git into a library, as we aren't tied to any\nof the current global state baggage or the current die() based\nerror handling.\n\nI've started an _extremely_ rough draft. The code compiles into a\nlibgit.a but it doesn't even implement what it describes in the API,\nlet alone a working Git implementation. Really what I'm trying to\nincite here is some discussion on what the API looks like.\n\nAPI Docs:\nhttp://www.spearce.org/projects/scm/libgit2/apidocs/html/modules.html\n\nSource Code Clone URL:\nhttp://www.spearce.org/projects/scm/libgit2/libgit2.git\n\n-- \nShawn.\n"},{"id":"94426","messageId":"680C5F4A-C55A-45B1-99E0-A2A74B2DC634@ai.rug.nl","threadId":"16110","inReplyTo":"20081031170704.GU14786@spearce.org","subject":"Re: libgit2 - a true git library","fromName":"Pieter de Bie","fromEmail":"pdebie@ai.rug.nl","sentAt":"2008-10-31T17:28:37Z","receivedAt":"2008-10-31T17:28:37Z","isPatch":false,"sender":{"key":"pdebie@ai.rug.nl","avatar":null},"body":"\nOn 31 okt 2008, at 18:07, Shawn O. Pearce wrote:\n\n> Source Code Clone URL:\n> http://www.spearce.org/projects/scm/libgit2/libgit2.git\n\nThis 404's for me\n\n- Pieter\n"},{"id":"94427","messageId":"93643F5A-22F9-4412-9948-5251362ED3B1@ai.rug.nl","threadId":"16110","inReplyTo":"680C5F4A-C55A-45B1-99E0-A2A74B2DC634@ai.rug.nl","subject":"Re: libgit2 - a true git library","fromName":"Pieter de Bie","fromEmail":"pdebie@ai.rug.nl","sentAt":"2008-10-31T17:29:46Z","receivedAt":"2008-10-31T17:29:46Z","isPatch":false,"sender":{"key":"pdebie@ai.rug.nl","avatar":null},"body":"\nOn 31 okt 2008, at 18:28, Pieter de Bie wrote:\n>> Source Code Clone URL:\n>> http://www.spearce.org/projects/scm/libgit2/libgit2.git\n>\n> This 404's for me\n\nNevermind, it's only a clone url.. I'd expected a gitweb or so ;)\n\nSorry for the noise,\n"},{"id":"94429","messageId":"20081031174745.GA4058@artemis.corp","threadId":"16110","inReplyTo":"20081031170704.GU14786@spearce.org","subject":"Re: libgit2 - a true git library","fromName":"Pierre Habouzit","fromEmail":"madcoder@debian.org","sentAt":"2008-10-31T17:47:45Z","receivedAt":"2008-10-31T17:47:45Z","isPatch":false,"sender":{"key":"madcoder@debian.org","avatar":"https://avatars.githubusercontent.com/u/44708?v=4"},"body":"On Fri, Oct 31, 2008 at 05:07:04PM +0000, Shawn O. Pearce wrote:\n> During the GitTogether we were kicking around the idea of a ground-up\n> implementation of a Git library. This may be easier than trying\n> to grind down git.git into a library, as we aren't tied to any\n> of the current global state baggage or the current die() based\n> error handling.\n> \n> I've started an _extremely_ rough draft. The code compiles into a\n> libgit.a but it doesn't even implement what it describes in the API,\n> let alone a working Git implementation. Really what I'm trying to\n> incite here is some discussion on what the API looks like.\n\nI know this isn't actually helping a lot to define the real APIs, but we\nshould really not repeat current git mistakes and have a really uniform\nAPIs, meaning that first we must decide:\n * proper namespacing (e.g. OBJ_* looks like failure to me, it's a way\n too common prefix);\n\n * proper public \"stuff\" naming (I e.g. realy like types names -- not\n struct or enum tags, that I don't really care -- ending with _t as\n it helps navigating source.\n\n * ...\n\nAnd write that down _first_. It's not a lot of work, but it must be\ndone. Working on a library really asks us to create something coherent\nfor our users.\n\n\nSecond, if we want this to be a successful stuff, we all agree we must\nlet git be able to use it medium term. That means that when git-core is\nexperimenting with new interfaces, it will probably need to hook into\nsome more internal aspects of the library. This is a problem to solve\nelegantly, linking git against a static library won't please a lot of\nvendors and linux distributions, and exporting \"private\" symbols is a\nsure way to see them being abused.\n\n\nLast but not least, I believe parts of git-core are currently easy to\njust take. For example, any code *I* wrote, I hereby give permission to\nrelicense it in any of the following licenses: BSD-like, MIT-like,\nWTFPL.\n\nFor example, on parse-options.c, git blame yields:\n\n git blame -C -C -M parse-options.c|cut -d\\( -f2|cut -d2 -f1|sort|uniq -c\n\t 16 Alex Riesen\n\t 6 Jeff King\n\t 47 Johannes Schindelin\n\t 12 Junio C Hamano\n\t 19 Michele Ballabio\n\t 1 Nanako Shiraishi\n\t 1 Olivier Marin\n\t395 Pierre Habouzit\n\nOkay, arguably parse-options.c in libgit quite doesn't makes sense\n(though it can help bringing some kind of uniformity to other git\necosystem tools built on libgit but that's not the point I'm trying to\nmake), I'm sure this kind of pattern where it's likely to be easy to\nrelicense code happens to some source files. Nicolas already said I\nthink that he was okay with relicensing his work too e.g.\n\nMaybe we could, in parallel to that, contact people who \"own\" code in\nthe core parts of git to ask them where they stand, and see if that can\nfree some bits of the code. Attached is the current owners of the non\nbuiltin-* C, non header, code in git core, got using this on top of\nnext:\n\nfor i in *.c; do\n case $i in\n builtin-*)\n continue;;\n *)\n git blame -C -C -M $i|cut -d\\( -f2|cut -d2 -f1;;\n esac\ndone\n\nand doing sort | uniq -c | sort -n >owners on it is attached.\nInterestingly, it yields around 200 contributors \"only\" but more\ninterestingly, only 41 people \"own\" more than 100 lines of code in\nthere, and 23 more if you add people with more than 50. IOW, it wouldn't\nbe absurd to mail those roughly 65 people ask them what they think of\nrelicensing their work (for those where it's needed because of current\nGPL-ness of the code) and see what result it yields. Worst case we lost\nlike 2 or 3 weeks, best case scenario, we can reuse some bits of git to\nreimplement some of the algorithms verbatim.\n\n\n\n-- \n·O· Pierre Habouzit\n··O madcoder@debian.org\nOOO http://www.madism.org\n\n\n16818\tJunio C Hamano\n8892\tLinus Torvalds\n4978\tJohannes Schindelin\n3664\tShawn O. Pearce\n2938\tNick Hengeveld\n2857\tNicolas Pitre\n2455\tDaniel Barkalow\n1660\tPierre Habouzit\n1294\tAdam Simpkins\n1126\tMike McCormack\n1110\tRené Scharfe\n861\tJeff King\n653\tLukas Sandström\n564\tMiklos Vajna\n531\tJohannes Sixt\n495\tAlex Riesen\n400\tMartin Koegler\n332\tPetr Baudis\n310\tJon Loeliger\n263\tTimo Hirvonen\n236\tBrandon Casey\n231\tLars Hjemli\n221\tSergey Vlasov\n200\tH. Peter Anvin\n181\tDmitry Potapov\n165\tMike Hommey\n163\tMatthias Lederhofer\n158\tRobert Shearman\n155\tAndreas Ericsson\n151\tYOSHIFUJI Hideaki\n147\tFranck Bui-Huu\n137\tChristian Couder\n127\tStephan Beyer\n127\tDavid Reiss\n123\tPaolo Bonzini\n120\tSteffen Prohaska\n115\tScott R Parish\n113\tBradford C. Smith\n109\tKristian Høgsberg\n105\tHeikki Orsila\n100\tAndy Whitcroft\n93\tJim Meyering\n90\tWincent Colaiuta\n90\tJulian Phillips\n87\tStephen R. van den Berg\n85\tMarco Costalba\n82\tDavid Rientjes\n76\tSven Verdoolaege\n75\tEric Wong\n72\tEdgar Toernig\n71\tRamsay Allan Jones\n69\tMartin Waitz\n68\tMark Wooding\n68\tEric W. Biederman\n67\tGeert Bosch\n64\tMichal Ostrowski\n64\tDavid Kastrup\n63\tTheodore Ts'o\n63\tAlexandre Julliard\n61\tBrian Downing\n59\tAlexander Gavrilov\n58\tAndy Parkins\n56\tJon Seymour\n52\tCarlos Rica\n47\tSean Estabrooks\n44\tDana L. How\n43\tJ. Bruce Fields\n41\tPing Yin\n40\tFlorian Forster\n38\tTilman Sauerbeck\n38\tSerge E. Hallyn\n38\tKay Sievers\n38\tJason Riedy\n37\tDustin Sallings\n35\tNguyễn Thái Ngọc Duy\n35\tLuiz Fernando N. Capitulino\n35\tClemens Buchacher\n34\tFredrik Kuivinen\n33\tPavel Roskin\n33\tLars Knoll\n32\tJosef Weidendorfer\n32\tJonas Fonseca\n30\tPeter Eriksen\n28\tJay Soffian\n26\tGrégoire Barbier\n25\tAdam Roben\n24\tMarius Storm-Olsen\n24\tLuben Tuikov\n24\tJürgen Rühle\n24\tBryan Larsen\n24\tAnders Melchiorsen\n23\tPieter de Bie\n23\tNanako Shiraishi\n23\tMichael S. Tsirkin\n23\tBrian Hetro\n22\tPaul Mackerras\n22\tJens Axboe\n20\tThomas Rast\n19\tPaul Collins\n19\tMatthias Kestenholz\n19\tJoachim Berdal Haga\n19\tDavid Woodhouse\n18\tMark Levedahl\n17\tMichele Ballabio\n15\tGerrit Pape\n14\tSam Vilain\n13\tOlivier Marin\n13\tAdam Brewster\n12\tSZEDER Gábor\n12\tKai Ruemmler\n12\tGovind Salinas\n11\tSasha Khapyorsky\n11\tRaphael Zimmerer\n11\tBrian Gernhardt\n10\tSteven Grimm\n10\tHolger Eitzenberger\n10\tDmitry V. Levin\n10\tDennis Stosberg\n10\tAvery Pennarun\n9\tWilly Tarreau\n9\tSanti Béjar\n9\tRobin H. Johnson\n9\tJames Bowes\n9\tBjörn Steinbrink\n8\tMarkus Amsler\n8\tJonathan del Strother\n8\tJohan Herland\n8\tChris Parsons\n8\tBoyd Lynn Gerber\n7\tRobin Rosenberg\n7\tPaul Serice\n7\tMatt Kraai\n7\tJason McMullan\n7\tFrank Lichtenheld\n7\tChristopher Li\n6\tQingning Huo\n6\tHan-Wen Nienhuys\n6\tDavid Soria Parra\n6\tBjörn Engelmann\n6\tAriel Badichi\n5\tSam Ravnborg\n5\tPeter Hagervall\n5\tPaul T Darga\n5\tMichal Vitecek\n5\tJames Bottomley\n5\tDotan Barak\n5\tDavid S. Miller\n5\tAndré Goddard Rosa\n4\tUwe Kleine-König\n4\tTeemu Likonen\n4\tSamuel Tardieu\n4\tPatrick Welche\n4\tMichael Spang\n4\tMatthieu Moy\n4\tJoey Hess\n4\tFinn Arne Gangstad\n4\tDmitry Kakurin\n4\tCarl Worth\n4\tArjen Laarhoven\n3\tSteven Drake\n3\tMatthew Ogilvie\n3\tLi Hong\n3\tJosh Triplett\n3\tJakub Narebski\n3\tEygene Ryabinkin\n3\tBrian Gerst\n2\tTom Prince\n2\tTodd Zullinger\n2\tShawn Bohrer\n2\tMatthias Urlichs\n2\tMartin Sivak\n2\tKrzysztof Kowalczyk\n2\tKevin Ballard\n2\tJan Harkes\n2\tFernando J. Pereda\n2\tEyvind Bernhardsen\n2\tDeskin Miller\n2\tCharles Bailey\n2\tAndrew Ruder\n2\tAmos Waterland\n2\tAlp Toker\n2\tAlexey Nezhdanov\n1\tYann Dirson\n1\tTuncer Ayaz\n1\tTony Luck\n1\tTimo Sirainen\n1\tThomas Harning\n1\tThomas Glanzmann\n1\tSverre Hvammen Johansen\n1\tStephan Feder\n1\tSimon Hausmann\n1\tSalikh Zakirov\n1\tRyan Anderson\n1\tRutger Nijlunsing\n1\tRandal L. Schwartz\n1\tPeter Valdemar Mørch\n1\tPaul Eggert\n1\tPatrick Higgins\n1\tMika Kukkonen\n1\tMatt Draisey\n1\tMarco Roeland\n1\tLars Doelle\n1\tJerald Fitzjerald\n1\tJean-Luc Herren\n1\tJan Andres\n1\tIngo Molnar\n1\tDavid Symonds\n1\tDavid Meybohm\n1\tDarrin Thompson\n1\tBryan Donlan\n1\tBrad Roberts\n1\tBlake Ramsdell\n1\tBenoit Sigoure\n1\tAdeodato Simó\n"},{"id":"94435","messageId":"20081031184154.GV14786@spearce.org","threadId":"16110","inReplyTo":"20081031174745.GA4058@artemis.corp","subject":"Re: libgit2 - a true git library","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2008-10-31T18:41:54Z","receivedAt":"2008-10-31T18:41:54Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Pierre Habouzit wrote:\n> \n> I know this isn't actually helping a lot to define the real APIs, but we\n> should really not repeat current git mistakes and have a really uniform\n> APIs, meaning that first we must decide:\n\nAgreed.\n\n> * proper namespacing (e.g. OBJ_* looks like failure to me, it's a way\n> too common prefix);\n\nFixed. Its now GIT_OBJ_*.\n\n> * proper public \"stuff\" naming (I e.g. realy like types names -- not\n> struct or enum tags, that I don't really care -- ending with _t as\n> it helps navigating source.\n\nFixed, types now end in _t.\n\n> And write that down _first_. It's not a lot of work, but it must be\n> done. Working on a library really asks us to create something coherent\n> for our users.\n\nHow about this?\n\nhttp://www.spearce.org/projects/scm/libgit2/apidocs/CONVENTIONS\n \n> Second, if we want this to be a successful stuff, we all agree we must\n> let git be able to use it medium term. That means that when git-core is\n> experimenting with new interfaces, it will probably need to hook into\n> some more internal aspects of the library. This is a problem to solve\n> elegantly, linking git against a static library won't please a lot of\n> vendors and linux distributions, and exporting \"private\" symbols is a\n> sure way to see them being abused.\n\nPrivate symbols are a problem. On some systems we can use link\nediting to strip them out of the .so, but that isn't always going to\nwork everywhere. I've outlined the idea of using double underscore\nto name private functions, and we can link-edit out '*__*' if the\nplatform's linker supports it (e.g. GNU ld).\n \n> Last but not least, I believe parts of git-core are currently easy to\n> just take. For example, any code *I* wrote, I hereby give permission to\n> relicense it in any of the following licenses: BSD-like, MIT-like,\n> WTFPL.\n\nYea. We could try to do that. I don't know how far it will get us,\nbut if we have to \"steal\" code we can rip a good part from JGit.\nIts BSD-like, but has that \"icky Java smell\" to it. :-)\n\nBefore worrying about where we get implementation bits from I'm\nmore interested in trying to get a consistent view of what our\nnamespace looks like, and what our calling conventions are, so we\nhave some sort of benchmark to measure APIs against as we add them\nto the implementation.\n\n-- \nShawn.\n"},{"id":"94436","messageId":"20081031185435.GC8464@artemis.corp","threadId":"16110","inReplyTo":"20081031184154.GV14786@spearce.org","subject":"Re: libgit2 - a true git library","fromName":"Pierre Habouzit","fromEmail":"madcoder@debian.org","sentAt":"2008-10-31T18:54:35Z","receivedAt":"2008-10-31T18:54:35Z","isPatch":false,"sender":{"key":"madcoder@debian.org","avatar":"https://avatars.githubusercontent.com/u/44708?v=4"},"body":"On Fri, Oct 31, 2008 at 06:41:54PM +0000, Shawn O. Pearce wrote:\n> Pierre Habouzit wrote:\n\n> How about this?\n> \n> http://www.spearce.org/projects/scm/libgit2/apidocs/CONVENTIONS\n\nlooks like a good start.\n\n> > Second, if we want this to be a successful stuff, we all agree we must\n> > let git be able to use it medium term. That means that when git-core is\n> > experimenting with new interfaces, it will probably need to hook into\n> > some more internal aspects of the library. This is a problem to solve\n> > elegantly, linking git against a static library won't please a lot of\n> > vendors and linux distributions, and exporting \"private\" symbols is a\n> > sure way to see them being abused.\n> \n> Private symbols are a problem. On some systems we can use link\n> editing to strip them out of the .so, but that isn't always going to\n> work everywhere. I've outlined the idea of using double underscore\n> to name private functions, and we can link-edit out '*__*' if the\n> platform's linker supports it (e.g. GNU ld).\n\nWell, I propose the following: we set-up on GNU-ld + gcc enabled systems\nall what is needed to use symbol visibility, which isn't that intrusive,\nand also rather easy given your GIT_EXPORT macro definition.\n\nThis way, people who care about portability across all libgit2 supported\nplatforms will have to align on the lowest common denominator, which\nwill not have any kind of private stuff available, so we're safe. And\nGCC/GNU-ld enabled platforms cover most of the popular platforms (namely\nlinux and *BSD, I'm not sure about Macos dynlibs). Even win32 has kind\nof what you need to do visibility I think.\n\nIOW prehistoric systems will be have to cope with that because of Linux\n(yeah, this is kind of deliciously backwards ;p).\n\nNo, my worry was rather wrt git core itself, I really think we _must_\nmake it link against libgit2 if we want libgit2 to stay current, but git\ncore will _very likely_ need the private stuff, and it _will_ be a\nproblem. I mean we cannot seriously so-name a library and show its guts\nat the same time, and I'm unsure how to fix that problem. _that_ was my\nactual question.\n\n> > Last but not least, I believe parts of git-core are currently easy to\n> > just take. For example, any code *I* wrote, I hereby give permission to\n> > relicense it in any of the following licenses: BSD-like, MIT-like,\n> > WTFPL.\n> \n> Yea. We could try to do that. I don't know how far it will get us,\n> but if we have to \"steal\" code we can rip a good part from JGit.\n> Its BSD-like, but has that \"icky Java smell\" to it. :-)\n> \n> Before worrying about where we get implementation bits from I'm\n> more interested in trying to get a consistent view of what our\n> namespace looks like, and what our calling conventions are, so we\n> have some sort of benchmark to measure APIs against as we add them\n> to the implementation.\n\nI'd say we should do both at the same time. Asking people if they would\nagree to relicense code can be done in parallel. We could extract a list\nof source files that we may need (my extraction included stuff that is\nvery unlikely to be useful like test-*.c that aren't useful, and some\nthat are already BSD I think), and see who it yields. It should be\npossible to do a matrix source-file x people and see on a per-file basis\nwhat they think.\n\nIf someone gives me the list of files we should consider (I'm not sure\nabout a good list right now) I could do the matrix at some fixed sha1\nfrom git.git using git blame -C -M -M -w, and ask people see where it\nleads us ?\n\n-- \n·O· Pierre Habouzit\n··O madcoder@debian.org\nOOO http://www.madism.org\n"},{"id":"94446","messageId":"20081031195711.GW14786@spearce.org","threadId":"16110","inReplyTo":"20081031185435.GC8464@artemis.corp","subject":"Re: libgit2 - a true git library","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2008-10-31T19:57:11Z","receivedAt":"2008-10-31T19:57:11Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Yet more updates to the API:\n\nhttp://www.spearce.org/projects/scm/libgit2/apidocs/html/modules.html\n\nIn particular I've started to define what revision machinary might\nlook like, based somewhat on JGit's (public) RevWalk API. The guts\nof how to make it work of course aren't yet defined.\n\nMy goal here is to have the git_revp_t be non-thread safe, but\nalso to contain the entire object pool, so git_revp_free() will\ndispose of any and all git_commit_t's which were created from it.\n\nThis \"fixes\" the \"we leak everything\" behavior and allows\napplications to have multiple pools on different threads if\nit needs to. Thus the core git_revp_t isn't thread-safe and\ndoesn't get weighed down by locking if the application wants\nmultiple threads.\n\n\nPierre Habouzit wrote:\n> \n> Well, I propose the following: we set-up on GNU-ld + gcc enabled systems\n> all what is needed to use symbol visibility, which isn't that intrusive,\n> and also rather easy given your GIT_EXPORT macro definition.\n\nYes, agreed. Only I don't know how to do it myself. I know its\npossible, so if someone wants to contribute a patch for this ... :-)\n \n> No, my worry was rather wrt git core itself, I really think we _must_\n> make it link against libgit2 if we want libgit2 to stay current, but git\n> core will _very likely_ need the private stuff, and it _will_ be a\n> problem. I mean we cannot seriously so-name a library and show its guts\n> at the same time, and I'm unsure how to fix that problem. _that_ was my\n> actual question.\n\nHmmph. I agree git-core needs to link to libgit2.\n\nI disagree it needs private bits. If we do the library right\ngit-core can use the public API. And where it cannot its either\nnot something that is \"right\" for libgit2 (e.g. its parseopts and\nwe aren't committed yet to including option parsing) or its highly\nexperimental and we shouldn't put it into the library (and thus\nalso git-core) until its more frozen.\n\nThat said, I don't think its criminal to have git-core include and\nlink to a static libgit2, especially if git-core's usage of the\nlibrary is ahead of what the library itself is able to expose at\nthe present time.\n \n> I'd say we should do both at the same time. Asking people if they would\n> agree to relicense code can be done in parallel. We could extract a list\n> of source files that we may need (my extraction included stuff that is\n> very unlikely to be useful like test-*.c that aren't useful, and some\n> that are already BSD I think), and see who it yields. It should be\n> possible to do a matrix source-file x people and see on a per-file basis\n> what they think.\n> \n> If someone gives me the list of files we should consider (I'm not sure\n> about a good list right now) I could do the matrix at some fixed sha1\n> from git.git using git blame -C -M -M -w, and ask people see where it\n> leads us ?\n\nOff the top of my head some really important ones:\n\n\tdiff-delta.c\n\tobject.c\n\tpatch-delta.c\n\trefs.c\n\trevision.c\n\tsha1_file.c\n\tsha1_name.c\n\nThey form a pretty large part of the guts of what most people want\nfrom a git library.\n\nSlightly less important, but still fairly core:\n\n\tbuiltin-fetch-pack.c\n\tbuiltin-send-pack.c\n\tconnect.c\n\tremote.c\n\ttransport.c\n\nIs most of the client side of the git:// transport, something people want.\n\n-- \nShawn.\n"},{"id":"94448","messageId":"7vfxmc8r8g.fsf@gitster.siamese.dyndns.org","threadId":"16110","inReplyTo":"20081031184154.GV14786@spearce.org","subject":"Re: libgit2 - a true git library","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-10-31T20:05:51Z","receivedAt":"2008-10-31T20:05:51Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Shawn O. Pearce\" writes:\n\n>> * proper public \"stuff\" naming (I e.g. realy like types names -- not\n>> struct or enum tags, that I don't really care -- ending with _t as\n>> it helps navigating source.\n>\n> Fixed, types now end in _t.\n\nUgh.\n\nYou could talk me into it if you promise never typedef structures (or\npointer to structures) with such symbols, I guess.\n"},{"id":"94450","messageId":"20081031201235.GE8464@artemis.corp","threadId":"16110","inReplyTo":"20081031195711.GW14786@spearce.org","subject":"Re: libgit2 - a true git library","fromName":"Pierre Habouzit","fromEmail":"madcoder@debian.org","sentAt":"2008-10-31T20:12:35Z","receivedAt":"2008-10-31T20:12:35Z","isPatch":false,"sender":{"key":"madcoder@debian.org","avatar":"https://avatars.githubusercontent.com/u/44708?v=4"},"body":"On Fri, Oct 31, 2008 at 07:57:11PM +0000, Shawn O. Pearce wrote:\n> This \"fixes\" the \"we leak everything\" behavior and allows\n> applications to have multiple pools on different threads if\n> it needs to. Thus the core git_revp_t isn't thread-safe and\n> doesn't get weighed down by locking if the application wants\n> multiple threads.\n\nNote that on Linux (and also BSDs I think) many pthread locking\nfunctions have stubs in the glibc and cost 0 until you use the\nlibpthread. So unless we need to use locking inside the tight-loop, it's\nvirtually free to write a thread-safe library (a function call is\nbasically 10 times cheaper than an xchg-based lock).\n\n(Though for git it would suck as we get libpthread in our depends\nthrough some of the other dependencies.\n\nAnother way is to use function pointers for the locking, and have a\nfunction to make the object store thread safe by setting the pointers to\nfunctions actually doing locking, and to let it point to functions doing\nnothing else.\n\nIt looks unrealistic to me to let people deal with the locking,\nespecially if we mean this library to _also_ be used in language\nbindings, hence used in sloppily written scripts.\n\nBut of course, if locking calls are used in the tight loop that would\nrather suck :/\n\n> Pierre Habouzit wrote:\n> > \n> > Well, I propose the following: we set-up on GNU-ld + gcc enabled systems\n> > all what is needed to use symbol visibility, which isn't that intrusive,\n> > and also rather easy given your GIT_EXPORT macro definition.\n> \n> Yes, agreed. Only I don't know how to do it myself. I know its\n> possible, so if someone wants to contribute a patch for this ... :-)\n\nI will, basically, you need to build everything with -fvisibilty=hidden\nin your CFLAGS, and mark the prototypes of symbols you want to export\nwith __attribute__((visibility(\"default\"))) (that you can set into your\nEXPORT_GIT macro when you're building with __GNUC__).\n\nYou don't even need a linker script (unless we're going to do some\nsymbol versioning but I'm unsure whether it's that useful for now).\n\n> > No, my worry was rather wrt git core itself, I really think we _must_\n> > make it link against libgit2 if we want libgit2 to stay current, but git\n> > core will _very likely_ need the private stuff, and it _will_ be a\n> > problem. I mean we cannot seriously so-name a library and show its guts\n> > at the same time, and I'm unsure how to fix that problem. _that_ was my\n> > actual question.\n> \n> Hmmph. I agree git-core needs to link to libgit2.\n> \n> I disagree it needs private bits. If we do the library right\n> git-core can use the public API. And where it cannot its either\n> not something that is \"right\" for libgit2\n\n> (e.g. its parseopts and\n> we aren't committed yet to including option parsing)\n\nSure, I didn't mean we have to put it in libgit2, it's highly UI related\nand other tool may want to use something else, and it makes no sense for\nmany languages that have their library already (python, perl do e.g.).\n \n> or its highly\n> experimental and we shouldn't put it into the library (and thus\n> also git-core) until its more frozen.\n> \n> That said, I don't think its criminal to have git-core include and\n> link to a static libgit2, especially if git-core's usage of the\n> library is ahead of what the library itself is able to expose at\n> the present time.\n\nOkay, we'll see how that turns out to work then :)\n\n> Off the top of my head some really important ones:\n> \n> \tdiff-delta.c\n> \tobject.c\n> \tpatch-delta.c\n> \trefs.c\n> \trevision.c\n> \tsha1_file.c\n> \tsha1_name.c\n> \n> They form a pretty large part of the guts of what most people want\n> from a git library.\n> \n> Slightly less important, but still fairly core:\n> \n> \tbuiltin-fetch-pack.c\n> \tbuiltin-send-pack.c\n> \tconnect.c\n> \tremote.c\n> \ttransport.c\n> \n> Is most of the client side of the git:// transport, something people want.\n\nOkay I'll let people mention what they would like to see too, and I'll\nwork from that then.\n-- \n·O· Pierre Habouzit\n··O madcoder@debian.org\nOOO http://www.madism.org\n"},{"id":"94451","messageId":"alpine.LFD.2.00.0810311558540.13034@xanadu.home","threadId":"16110","inReplyTo":"20081031174745.GA4058@artemis.corp","subject":"Re: libgit2 - a true git library","fromName":"Nicolas Pitre","fromEmail":"nico@cam.org","sentAt":"2008-10-31T20:24:14Z","receivedAt":"2008-10-31T20:24:14Z","isPatch":false,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Fri, 31 Oct 2008, Pierre Habouzit wrote:\n\n> Last but not least, I believe parts of git-core are currently easy to\n> just take. For example, any code *I* wrote, I hereby give permission to\n> relicense it in any of the following licenses: BSD-like, MIT-like,\n> WTFPL.\n\nFirst........... is there really a need to re-license it?\nIf so then the choice of license is IMHO rather important.\n\n> Nicolas already said I think that he was okay with relicensing his \n> work too e.g.\n\nDepends. Sure, I gave permission to copy some of my code for JGIT \nbecause 1) JGIT is Java code in which I have little interest, 2) the \ndifferent license was justified by the nature of the JGIT project, and \n3) although no license convey this I asked for the C version of git to \nremain the authoritative reference and that any improvements done to JGIT \nfirst be usable in the C version under the GPL.\n\nOf course a library might need a different license than the GPL to be \nwidely useful from a linkage stand point, but the code within that \nlibrary does not need to be miles away from the GPL. What I personally \ncare about is for improvements to my code to always be contributed back, \nwhich pretty much discards BSD-like licenses.\n\nMy favorite license for a library is the GPL with the gcc exception, \ni.e. what libraries coming with gcc are using. They're GPL but with an \nexception allowing them to be linked with anything. And because \neverything on a Linux system, including proprietary applications, is \nlikely linked against those gcc libs, then there is nothing that would \nprevent libgit to be linked against anything as well. But the library \ncode itself has GPL protection.\n\nFor reference, here's the exception text:\n\n In addition to the permissions in the GNU General Public License, the\n Free Software Foundation gives you unlimited permission to link the\n compiled version of this file into combinations with other programs,\n and to distribute those combinations without any restriction coming\n from the use of this file. (The General Public License restrictions\n do apply in other respects; for example, they cover modification of\n the file, and distribution when not linked into a combine\n executable.)\n\n\nNicolas\n"},{"id":"94452","messageId":"58DFE9D9-B847-41D5-9ADF-330B79F175D4@silverinsanity.com","threadId":"16110","inReplyTo":"20081031174745.GA4058@artemis.corp","subject":"Re: libgit2 - a true git library","fromName":"Brian Gernhardt","fromEmail":"benji@silverinsanity.com","sentAt":"2008-10-31T20:24:52Z","receivedAt":"2008-10-31T20:24:52Z","isPatch":false,"sender":{"key":"benji@silverinsanity.com","avatar":"https://gravatar.com/avatar/e06c101dbc25c68114d859b4a9ec7cf8a2c52fd2b0270ef0eac0e2e63ff22311?d=mp&s=160"},"body":"\nOn Oct 31, 2008, at 1:47 PM, Pierre Habouzit wrote:\n\n> Last but not least, I believe parts of git-core are currently easy to\n> just take. For example, any code *I* wrote, I hereby give permission \n> to\n> relicense it in any of the following licenses: BSD-like, MIT-like,\n> WTFPL.\n\nFWIW, I give the same permissions.\n\n> 11\tBrian Gernhardt\n\n~~ Brian Gernhardt\n"},{"id":"94454","messageId":"alpine.DEB.1.10.0810311325490.5851@asgard.lang.hm","threadId":"16110","inReplyTo":"alpine.LFD.2.00.0810311558540.13034@xanadu.home","subject":"Re: libgit2 - a true git library","fromName":"","fromEmail":"david@lang.hm","sentAt":"2008-10-31T20:29:40Z","receivedAt":"2008-10-31T20:29:40Z","isPatch":false,"sender":{"key":"david@lang.hm","avatar":null},"body":"On Fri, 31 Oct 2008, Nicolas Pitre wrote:\n\n> On Fri, 31 Oct 2008, Pierre Habouzit wrote:\n>\n>> Last but not least, I believe parts of git-core are currently easy to\n>> just take. For example, any code *I* wrote, I hereby give permission to\n>> relicense it in any of the following licenses: BSD-like, MIT-like,\n>> WTFPL.\n>\n> First........... is there really a need to re-license it?\n> If so then the choice of license is IMHO rather important.\n\nat the very least you should go from GPLv2 to LGPLv2 for the library.\n\nwhile it can be argued that this really shouldn't be nessasary, the water \nis muddy enough that it would be a very good thing to do this.\n\nI don't see any need to switch to a BSD/MIT/etc license for a library, the \nLGPL lets it get linked with those licenses anyway.\n\n> My favorite license for a library is the GPL with the gcc exception,\n> i.e. what libraries coming with gcc are using. They're GPL but with an\n> exception allowing them to be linked with anything. And because\n> everything on a Linux system, including proprietary applications, is\n> likely linked against those gcc libs, then there is nothing that would\n> prevent libgit to be linked against anything as well. But the library\n> code itself has GPL protection.\n>\n> For reference, here's the exception text:\n>\n> In addition to the permissions in the GNU General Public License, the\n> Free Software Foundation gives you unlimited permission to link the\n> compiled version of this file into combinations with other programs,\n> and to distribute those combinations without any restriction coming\n> from the use of this file. (The General Public License restrictions\n> do apply in other respects; for example, they cover modification of\n> the file, and distribution when not linked into a combine\n> executable.)\n\n, I don't see why this is needed with the LGPL, but I'm not a \nlawyer.\n\nDavid Lang\n"},{"id":"94458","messageId":"alpine.LFD.2.00.0810311651451.13034@xanadu.home","threadId":"16110","inReplyTo":"alpine.DEB.1.10.0810311325490.5851@asgard.lang.hm","subject":"Re: libgit2 - a true git library","fromName":"Nicolas Pitre","fromEmail":"nico@cam.org","sentAt":"2008-10-31T20:56:08Z","receivedAt":"2008-10-31T20:56:08Z","isPatch":false,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Fri, 31 Oct 2008, david@lang.hm wrote:\n\n> On Fri, 31 Oct 2008, Nicolas Pitre wrote:\n> \n> > On Fri, 31 Oct 2008, Pierre Habouzit wrote:\n> > \n> > > Last but not least, I believe parts of git-core are currently easy to\n> > > just take. For example, any code *I* wrote, I hereby give permission to\n> > > relicense it in any of the following licenses: BSD-like, MIT-like,\n> > > WTFPL.\n> > \n> > First........... is there really a need to re-license it?\n> > If so then the choice of license is IMHO rather important.\n> \n> at the very least you should go from GPLv2 to LGPLv2 for the library.\n\nSure.\n\n> while it can be argued that this really shouldn't be nessasary, the water is\n> muddy enough that it would be a very good thing to do this.\n> \n> I don't see any need to switch to a BSD/MIT/etc license for a library, the\n> LGPL lets it get linked with those licenses anyway.\n\nRight.\n\n> > My favorite license for a library is the GPL with the gcc exception,\n> > i.e. what libraries coming with gcc are using. They're GPL but with an\n> > exception allowing them to be linked with anything. And because\n> > everything on a Linux system, including proprietary applications, is\n> > likely linked against those gcc libs, then there is nothing that would\n> > prevent libgit to be linked against anything as well. But the library\n> > code itself has GPL protection.\n> > \n> > For reference, here's the exception text:\n> > \n> > In addition to the permissions in the GNU General Public License, the\n> > Free Software Foundation gives you unlimited permission to link the\n> > compiled version of this file into combinations with other programs,\n> > and to distribute those combinations without any restriction coming\n> > from the use of this file. (The General Public License restrictions\n> > do apply in other respects; for example, they cover modification of\n> > the file, and distribution when not linked into a combine\n> > executable.)\n> \n> , I don't see why this is needed with the LGPL, but I'm not a lawyer.\n\nThe LGPL also asks that proprietary applications provides necessary \nobject files so you can link it against an alternative implementation of \nthe LGPL library if you so wish. With dynamic libraries this is rather \nmoot but I think that's the main difference.\n\n\nNicolas\n"},{"id":"94461","messageId":"20081031213114.GA21799@artemis.corp","threadId":"16110","inReplyTo":"alpine.LFD.2.00.0810311558540.13034@xanadu.home","subject":"Re: libgit2 - a true git library","fromName":"Pierre Habouzit","fromEmail":"madcoder@debian.org","sentAt":"2008-10-31T21:31:14Z","receivedAt":"2008-10-31T21:31:14Z","isPatch":false,"sender":{"key":"madcoder@debian.org","avatar":"https://avatars.githubusercontent.com/u/44708?v=4"},"body":"On Fri, Oct 31, 2008 at 08:24:14PM +0000, Nicolas Pitre wrote:\n> On Fri, 31 Oct 2008, Pierre Habouzit wrote:\n> \n> > Last but not least, I believe parts of git-core are currently easy to\n> > just take. For example, any code *I* wrote, I hereby give permission to\n> > relicense it in any of the following licenses: BSD-like, MIT-like,\n> > WTFPL.\n> \n> First........... is there really a need to re-license it?\n> If so then the choice of license is IMHO rather important.\n\nHmm yeah, GPL has the viral link issue, and there is a real use of\nembedding the libgit in many projects. There is a use for it: front-ends\nto git from editors, for closed-source projects of various sorts, ...\nAll of those right now must do a reimplementation of what they need. As\nsoon as they need some kind of performance, exec()ing git-* commands\ndoesn't fly.\n\nGit is currently mostly \"GPLv2 or later\". A BSDish license was\nmentioned, because it's the most permissive one and that nobody cared\nthat much, though a LGPL/GPL-with-GCC-exception would probably fly.\n\nMany of the people needing a library for libgit are probably reading the\nlist, I'll let them comment. The kind of license you propose would\ntotally suite my needs, and I think, most of the one discussed at\nGitTogether'08 (except for the eclipse people disliking GPL'ed stuff,\nbut anyways there was the issue of C code being non pure java anyways,\nso maybe Shawn can comment on that bit, I don't recall the exact\nspecifics I must reckon).\n\n\nOT: FWIW I prefer BSDish licenses (even the MIT actually) for libraries\nbecause I believe that computing is overall better if everyone can use\nthe right tool for the task, and I don't want to prevent people from\nusing good stuff (I hope I write good stuff ;P) because of the license.\nAnd I don't care about people don't giving back to me, those are not the\nkind of people who would have given back if it was GPL'ed anyways.\n\nBut I understand this is a completely personal view, and I'm not even\ntrying to persuade you :)\n-- \n·O· Pierre Habouzit\n··O madcoder@debian.org\nOOO http://www.madism.org\n"},{"id":"94467","messageId":"20081031214356.GX14786@spearce.org","threadId":"16110","inReplyTo":"alpine.LFD.2.00.0810311651451.13034@xanadu.home","subject":"Re: libgit2 - a true git library","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2008-10-31T21:43:56Z","receivedAt":"2008-10-31T21:43:56Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Nicolas Pitre wrote:\n> On Fri, 31 Oct 2008, david@lang.hm wrote:\n> > On Fri, 31 Oct 2008, Nicolas Pitre wrote:\n> > > On Fri, 31 Oct 2008, Pierre Habouzit wrote:\n> > > \n> > > > Last but not least, I believe parts of git-core are currently easy to\n> > > > just take. For example, any code *I* wrote, I hereby give permission to\n> > > > relicense it in any of the following licenses: BSD-like, MIT-like,\n> > > > WTFPL.\n> > > \n> > > First........... is there really a need to re-license it?\n> > > If so then the choice of license is IMHO rather important.\n\nSome people want to be able to link the library into an application\nthat they redistribute binaries of, but not sources to. Those folks\nhave also volunteered to help write the library. If they put their\ncode where their mouth is, then I think they should be able to use\ntheir code the way they want to.\n\nThat said, I think the license choice that makes the most sense\nhere is probably LGPL or GPL+gcc exception, like you note below.\nBSD and MIT are probably not serious contenders.\n\n> > at the very least you should go from GPLv2 to LGPLv2 for the library.\n> \n> Sure.\n\nWell, we cannot do a GPL->LGPL switch on code without author\npermission for that sort of re-licensing.\n\nThat said, I think many authors of git.git code would be more\ncomfortable with a GPL->LGPL change, where they wouldn't be OK with\na GPL->BSD/MIT change. There may be some folks though who still\nwouldn't accept a GPL->LGPL move.\n\n> > > My favorite license for a library is the GPL with the gcc exception,\n...\n> > > \n> > > For reference, here's the exception text:\n> > > \n> > > In addition to the permissions in the GNU General Public License, the\n> > > Free Software Foundation gives you unlimited permission to link the\n> > > compiled version of this file into combinations with other programs,\n> > > and to distribute those combinations without any restriction coming\n> > > from the use of this file. (The General Public License restrictions\n> > > do apply in other respects; for example, they cover modification of\n> > > the file, and distribution when not linked into a combine\n> > > executable.)\n> > \n> > , I don't see why this is needed with the LGPL, but I'm not a lawyer.\n> \n> The LGPL also asks that proprietary applications provides necessary \n> object files so you can link it against an alternative implementation of \n> the LGPL library if you so wish. With dynamic libraries this is rather \n> moot but I think that's the main difference.\n\nI'm happy with either the LGPL or the GPL+exception above. If I\nread these correctly the GPL+exception allows one to distribute\nstatic executables without source or object files, so long as the\nlibrary source wasn't modified. I'd almost prefer just using the\nstandard LGPL then, static linking isn't very common anymore.\n\n-- \nShawn.\n"},{"id":"94468","messageId":"20081031215041.GY14786@spearce.org","threadId":"16110","inReplyTo":"20081031214356.GX14786@spearce.org","subject":"Re: libgit2 - a true git library","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2008-10-31T21:50:41Z","receivedAt":"2008-10-31T21:50:41Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"\"Shawn O. Pearce\" wrote:\n> That said, I think the license choice that makes the most sense\n> here is probably LGPL or GPL+gcc exception, like you note below.\n> BSD and MIT are probably not serious contenders.\n\nI should clarify that I said the above paragraph...\n \n> That said, I think many authors of git.git code would be more\n> comfortable with a GPL->LGPL change, where they wouldn't be OK with\n> a GPL->BSD/MIT change. There may be some folks though who still\n> wouldn't accept a GPL->LGPL move.\n\nbecause of this paragraph.\n\nLike Pierre I also prefer a BSD style license, and JGit is under\nthat, as it offers quite a bit of freedom for the consumer of\nthe code.\n\nI'm also not too worried about not getting changes back. If someone\nforks away from the base project and doesn't contribute back,\nthat's their problem. So long as the base project has sufficient\nmomentum under it making changes and improving things, everyone\nelse will want to pull and either face merge-hell once in a while,\nor send changes back upstream to avoid merge-hell.\n\nBut I doubt Git regulars share our views on this, and I think most\nof the major contributors to git.git have stated multiple times\nthat they prefer a GPL style license on their code. I want those\npeople to contribute to libgit2 (assuming the project moves past the\npie-in-the-sky theory stage), so I want the license to be something\nthey will be comfortable with.\n \n-- \nShawn.\n"},{"id":"94469","messageId":"20081031215155.GC21799@artemis.corp","threadId":"16110","inReplyTo":"20081031214356.GX14786@spearce.org","subject":"Re: libgit2 - a true git library","fromName":"Pierre Habouzit","fromEmail":"madcoder@debian.org","sentAt":"2008-10-31T21:51:55Z","receivedAt":"2008-10-31T21:51:55Z","isPatch":false,"sender":{"key":"madcoder@debian.org","avatar":"https://avatars.githubusercontent.com/u/44708?v=4"},"body":"On Fri, Oct 31, 2008 at 09:43:56PM +0000, Shawn O. Pearce wrote:\n> Nicolas Pitre wrote:\n> > On Fri, 31 Oct 2008, david@lang.hm wrote:\n> > > at the very least you should go from GPLv2 to LGPLv2 for the library.\n> > \n> > Sure.\n> \n> Well, we cannot do a GPL->LGPL switch on code without author\n> permission for that sort of re-licensing.\n> \n> That said, I think many authors of git.git code would be more\n> comfortable with a GPL->LGPL change, where they wouldn't be OK with\n> a GPL->BSD/MIT change. There may be some folks though who still\n> wouldn't accept a GPL->LGPL move.\n\nFWIW, I dislike the LGPL for many reasons, and I prefer 100x times a GPL\nwith GCC kind of exceptions. But if other people hate it, I wont be\nbe a problem and refuse it.\n\n\n-- \n·O· Pierre Habouzit\n··O madcoder@debian.org\nOOO http://www.madism.org\n"},{"id":"94470","messageId":"20081031215857.GZ14786@spearce.org","threadId":"16110","inReplyTo":"7vfxmc8r8g.fsf@gitster.siamese.dyndns.org","subject":"Re: libgit2 - a true git library","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2008-10-31T21:58:57Z","receivedAt":"2008-10-31T21:58:57Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Junio C Hamano wrote:\n> \"Shawn O. Pearce\" writes:\n> \n> >> * proper public \"stuff\" naming (I e.g. realy like types names -- not\n> >> struct or enum tags, that I don't really care -- ending with _t as\n> >> it helps navigating source.\n> >\n> > Fixed, types now end in _t.\n> \n> Ugh.\n> \n> You could talk me into it if you promise never typedef structures (or\n> pointer to structures) with such symbols, I guess.\n \nI should write that one down in CONVENTIONS.\n\nIMHO:\n\n typedef uint32_t uid_t; /* sane */\n typedef enum {...} status_t; /* sane */\n typedef struct foo_t foo_t; /* sane */\n\n typedef struct {...} foo_t; /* borderline insane */\n\n typedef char* str_t; /* totally nuts */\n typedef char**** str_pppp_t; /* totally nuts */\n\nHiding the fact that scalar types like a uid_t are 32 bits on\nthis system is reasonable. Heck, uid_t is already in POSIX,\nwe shouldn't fight that sort of idea. It at least improves\ndocumentation somewhat.\n\nHiding the fact that some scalar type is an enum, so you don't have\nto type \"enum blah\" everywhere is also reasonable. Its slightly\nbetter than #define some magic constants and passing an int\neverywhere. Its a reasonable balance between reducing keystrokes\nand keeping the code semi-self-documenting.\n\nHiding the fact that an opaque struct (or union) you cannot ever\nsee the members of is a struct or union is good API design. You can\nlater change the major class from struct to union or back, or totally\nredefine it, but the caller never needs to know what is going on.\n\nHiding a pointer is wrong. Callers should know they are getting a\npointer, or are being asked to supply a pointer-to-a-pointer. So the\n\"FILE*\" stdio functions are sane, because we don't know what is under\na FILE type but we do know when we are dealing with a pointer to one.\n\n\nMy original proposal didn't stick _t onto the end of everything,\nbecause I didn't think it was really necessary. I'm fine with it\neither way. It may be better to include the _t suffix, it seems\nto be somewhat common in libraries.\n\n-- \nShawn.\n"},{"id":"94471","messageId":"490B7FD3.8060003@op5.se","threadId":"16110","inReplyTo":"20081031174745.GA4058@artemis.corp","subject":"Re: libgit2 - a true git library","fromName":"Andreas Ericsson","fromEmail":"ae@op5.se","sentAt":"2008-10-31T21:59:47Z","receivedAt":"2008-10-31T21:59:47Z","isPatch":false,"sender":{"key":"ae@op5.se","avatar":"https://gravatar.com/avatar/426e89595c75a8f5252dd0c989e5fabe5bcac616e68557427ad9aef6b0ca342a?d=mp&s=160"},"body":"Pierre Habouzit wrote:\n> On Fri, Oct 31, 2008 at 05:07:04PM +0000, Shawn O. Pearce wrote:\n>> During the GitTogether we were kicking around the idea of a ground-up\n>> implementation of a Git library. This may be easier than trying\n>> to grind down git.git into a library, as we aren't tied to any\n>> of the current global state baggage or the current die() based\n>> error handling.\n>>\n>> I've started an _extremely_ rough draft. The code compiles into a\n>> libgit.a but it doesn't even implement what it describes in the API,\n>> let alone a working Git implementation. Really what I'm trying to\n>> incite here is some discussion on what the API looks like.\n> \n> I know this isn't actually helping a lot to define the real APIs, but we\n> should really not repeat current git mistakes and have a really uniform\n> APIs, meaning that first we must decide:\n> * proper namespacing (e.g. OBJ_* looks like failure to me, it's a way\n> too common prefix);\n> \n\nAs it's the git-lib, all public functions should almost certainly be\nprefixed with \"git\" or \"git_\". I favor \"git_\".\n\n> * proper public \"stuff\" naming (I e.g. realy like types names -- not\n> struct or enum tags, that I don't really care -- ending with _t as\n> it helps navigating source.\n> \n\n*_t types are reserved by POSIX for future implementations, so that's\na no-go (although I doubt POSIX will ever make types named git_*_t).\n\n\nApart from that, please consider reading Ulrich Drepper's musings on\nlibrary design at http://people.redhat.com/drepper/goodpractice.pdf\n\nIt's pretty short but brings up nearly all the crucial points one really\ndon't want to forget.\n\n-- \nAndreas Ericsson andreas.ericsson@op5.se\nOP5 AB www.op5.se\nTel: +46 8-230225 Fax: +46 8-230231\n"},{"id":"94472","messageId":"20081031220133.GA14786@spearce.org","threadId":"16110","inReplyTo":"490B7FD3.8060003@op5.se","subject":"Re: libgit2 - a true git library","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2008-10-31T22:01:33Z","receivedAt":"2008-10-31T22:01:33Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Andreas Ericsson wrote:\n>> * proper public \"stuff\" naming (I e.g. realy like types names -- not\n>> struct or enum tags, that I don't really care -- ending with _t as\n>> it helps navigating source.\n>\n> *_t types are reserved by POSIX for future implementations, so that's\n> a no-go (although I doubt POSIX will ever make types named git_*_t).\n\nYikes. Anyone know where a concise list of the reserved names are?\n\n> Apart from that, please consider reading Ulrich Drepper's musings on\n> library design at http://people.redhat.com/drepper/goodpractice.pdf\n\nI think I've read that before, but I'll skim over it again.\nThanks for the link.\n\n-- \nShawn.\n"},{"id":"94475","messageId":"alpine.LFD.2.00.0810311756160.13034@xanadu.home","threadId":"16110","inReplyTo":"20081031213114.GA21799@artemis.corp","subject":"Re: libgit2 - a true git library","fromName":"Nicolas Pitre","fromEmail":"nico@cam.org","sentAt":"2008-10-31T22:10:54Z","receivedAt":"2008-10-31T22:10:54Z","isPatch":false,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Fri, 31 Oct 2008, Pierre Habouzit wrote:\n\n> Git is currently mostly \"GPLv2 or later\". A BSDish license was\n> mentioned, because it's the most permissive one and that nobody cared\n> that much, though a LGPL/GPL-with-GCC-exception would probably fly.\n\nI do care. I think the BSD license is too permissive. There are really \nnifty pieces of code in Git that I would be really sorry to see go \nproprietary.\n\n> Many of the people needing a library for libgit are probably reading the\n> list, I'll let them comment. The kind of license you propose would\n> totally suite my needs, and I think, most of the one discussed at\n> GitTogether'08 (except for the eclipse people disliking GPL'ed stuff,\n> but anyways there was the issue of C code being non pure java anyways,\n> so maybe Shawn can comment on that bit, I don't recall the exact\n> specifics I must reckon).\n\nEclipse is Java and that issue is already solved with JGIT which doesn't \nreuse C code from git.\n\n> OT: FWIW I prefer BSDish licenses (even the MIT actually) for libraries\n> because I believe that computing is overall better if everyone can use\n> the right tool for the task, and I don't want to prevent people from\n> using good stuff (I hope I write good stuff ;P) because of the license.\n\nEverybody can and does link against glibc on Linux which is LGPL. So \nthat doesn't affect \"usage\".\n\n> And I don't care about people don't giving back to me, those are not the\n> kind of people who would have given back if it was GPL'ed anyways.\n> But I understand this is a completely personal view, and I'm not even\n> trying to persuade you :)\n\nSure, and that's where we differ. I let you use my code for free, as \nlong as you give me back your improvements to it. This way everybody \nstays honnest. I think this is Linus' view as well which he often \nresume as \"tit for tat\".\n\n\nNicolas\n"},{"id":"94484","messageId":"7v3aic8jl4.fsf@gitster.siamese.dyndns.org","threadId":"16110","inReplyTo":"20081031220133.GA14786@spearce.org","subject":"Re: libgit2 - a true git library","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-10-31T22:51:03Z","receivedAt":"2008-10-31T22:51:03Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Shawn O. Pearce\" writes:\n\n> Andreas Ericsson wrote:\n>>> * proper public \"stuff\" naming (I e.g. realy like types names -- not\n>>> struct or enum tags, that I don't really care -- ending with _t as\n>>> it helps navigating source.\n>>\n>> *_t types are reserved by POSIX for future implementations, so that's\n>> a no-go (although I doubt POSIX will ever make types named git_*_t).\n>\n> Yikes. Anyone know where a concise list of the reserved names are?\n\nEssentially, anything that ends with \"_t\" ;-)\n\nhttp://www.opengroup.org/onlinepubs/000095399/functions/xsh_chap02_02.html#tag_02_02_02\n\nLook for \"Any Header\" in the table.\n\n>> Apart from that, please consider reading Ulrich Drepper's musings on\n>> library design at http://people.redhat.com/drepper/goodpractice.pdf\n>\n> I think I've read that before, but I'll skim over it again.\n> Thanks for the link.\n>\n> -- \n> Shawn.\n"},{"id":"94487","messageId":"7viqr873x7.fsf@gitster.siamese.dyndns.org","threadId":"16110","inReplyTo":"alpine.LFD.2.00.0810311558540.13034@xanadu.home","subject":"Re: libgit2 - a true git library","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-10-31T23:14:44Z","receivedAt":"2008-10-31T23:14:44Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Nicolas Pitre writes:\n\n> Depends. Sure, I gave permission to copy some of my code for JGIT \n> because 1) JGIT is Java code in which I have little interest, 2) the \n> different license was justified by the nature of the JGIT project, and \n> 3) although no license convey this I asked for the C version of git to \n> remain the authoritative reference and that any improvements done to JGIT \n> first be usable in the C version under the GPL.\n\nThis reminds me that Shawn earlier in an unrelated thread asked me if I\ncan relicense builtin-blame.c for JGIT; your reasoning fully matches that\nof mine regarding that part of the code.\n\n> My favorite license for a library is the GPL with the gcc exception,\n> i.e. what libraries coming with gcc are using. They're GPL but with an \n> exception allowing them to be linked with anything.\n\nAlthough I'd be Ok with either GPL + gcc exception on whatever core-ish\n(i.e. what will be necessary for libgit2; \"blame\" would not count) pieces\nI have in C-git codebase, \"can be linked with anything\" allows a gaping\nhole to the library, which I'm a bit hesitant to swallow without thinking.\n\nE.g. our read_object() may look like this:\n\n void *read_object(const object_name_t sha1,\n enum object_type *type,\n size_t *size)\n {\n ...\n }\n\n\nbut an extension a closed-source person may sell you back may do:\n\n +typedef void *read_object_fn(const object_name_t,\n + enum object_type *,\n + size_t *);\n +read_object_fn read_object_custom = NULL;\n void *read_object(const object_name_t sha1,\n enum object_type *type,\n size_t *size)\n {\n + if (read_object_custom != NULL)\n + return read_object_custom(sha1, type, size);\n ...\n }\n\nI.e. use the supplied custom function to do proprietary magic, such as\nreading the object lazily from elsewhere over the network. And we will\nnever get that magic bit back.\n\nAlthough no license asks this, my wish is that if somebody built on top of\nwhat I wrote to make the world a better place, I'd like the same access to\nthat additional code so that I too can enjoy the improved world. Because\nalmost all of my code in git.git are under GPLv2, in reality I do not have\nany access to your software as long as you do not distribute your\nadditional code that made the world a better place, which is a bit sad.\n"},{"id":"94506","messageId":"490B9262.5070905@gmail.com","threadId":"16110","inReplyTo":"20081031170704.GU14786@spearce.org","subject":"Re: libgit2 - a true git library","fromName":"Bruno Santos","fromEmail":"nayart3@gmail.com","sentAt":"2008-10-31T23:18:58Z","receivedAt":"2008-10-31T23:18:58Z","isPatch":false,"sender":{"key":"nayart3@gmail.com","avatar":null},"body":"Shawn O. Pearce wrote:\n> During the GitTogether we were kicking around the idea of a ground-up\n> implementation of a Git library. This may be easier than trying\n> to grind down git.git into a library, as we aren't tied to any\n> of the current global state baggage or the current die() based\n> error handling.\n> \n> I've started an _extremely_ rough draft. The code compiles into a\n> libgit.a but it doesn't even implement what it describes in the API,\n> let alone a working Git implementation. Really what I'm trying to\n> incite here is some discussion on what the API looks like.\n> \n> API Docs:\n> http://www.spearce.org/projects/scm/libgit2/apidocs/html/modules.html\n> \n> Source Code Clone URL:\n> http://www.spearce.org/projects/scm/libgit2/libgit2.git\n> \n\nWe should take the opportunity a make it more portable. Instead of using\nthe posix api directly we should warp it in \"git_\" APIs. And be carefull\nwith certain APIs like fork or fork+exec and instead provided a more\ngeneric solution: for fork one that would use the best solution in the\ngiven platform, either by forking or threading; and for fork+exec a\ngeneric create_process/run_command.\n\n\nHere's an example, for the 'read' API, on how we can simply do this\nwithout worries for the posix crowd:\n\nssize_t git_read(git_fildes_t fildes, void* buf, size_t bufsize);\n\nWere git_fildes_t would be an int for posix and an HANDLE for win32.\nFor the posix case git_read can be simply inlined and we get zero overhead:\n\nstatic inline ssize_t git_read(git_fildes_t fildes, void *buf,\n\t\t\tsize_t bufsize)\n{\n\treturn read(fildes, buf, bufsize);\n}\n\nAnd for the win32 case it would be much more easier to implement the\nequivalent, something like:\n\nssize_t git_read(git_fildes_t fildes, void *buf, size_t bufsize)\n{\n\tDWORD rd;\n\n\tif (!ReadFile(fildes, buf, bufsize, &rd, NULL)) {\n\t\t//translate win32 error to errno\n\t\treturn -1;\n\t}\n\treturn rd;\n}\n\n\nOf course, there is also the issue of using the c runtime on win32, but\nthat problem can be easily solved outside git, provided that we don't\nuse a 'fileno' like API.\n\n\n\nBruno Santos\n"},{"id":"94486","messageId":"alpine.DEB.1.00.0811010019450.22125@pacific.mpi-cbg.de.mpi-cbg.de","threadId":"16110","inReplyTo":"490B7FD3.8060003@op5.se","subject":"Re: libgit2 - a true git library","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-10-31T23:22:56Z","receivedAt":"2008-10-31T23:22:56Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Fri, 31 Oct 2008, Andreas Ericsson wrote:\n\n> Apart from that, please consider reading Ulrich Drepper's musings on\n> library design at http://people.redhat.com/drepper/goodpractice.pdf\n\nI do not know if I want to trust a person that has shown a certain \neagerness to ignore good library design by breaking the well-established \ndont-change-apis-on-minor-versions idiom, and instead of listening to \nusers that have problems as a consequence rather ignore them.\n\nInstead, let's build on the knowledge of people we have learnt to trust, \non this list.\n\nThank you,\nDscho\n"},{"id":"94491","messageId":"CBF2AF68-BA41-4394-A837-F62864CF8BFB@ai.rug.nl","threadId":"16110","inReplyTo":"20081031213114.GA21799@artemis.corp","subject":"Re: libgit2 - a true git library","fromName":"Pieter de Bie","fromEmail":"pdebie@ai.rug.nl","sentAt":"2008-10-31T23:24:39Z","receivedAt":"2008-10-31T23:24:39Z","isPatch":false,"sender":{"key":"pdebie@ai.rug.nl","avatar":null},"body":"\nOn 31 okt 2008, at 22:31, Pierre Habouzit wrote:\n\n> Git is currently mostly \"GPLv2 or later\". A BSDish license was\n> mentioned, because it's the most permissive one and that nobody cared\n> that much, though a LGPL/GPL-with-GCC-exception would probably fly.\n>\n> Many of the people needing a library for libgit are probably reading \n> the\n> list, I'll let them comment\n\nAs an implementor of a git GUI, I don't really care what license\nlibgit2 will be. GitX is currently GPLv2, though that might change\nto LGPL to allow it to ship/link with non-GPL libraries. GPL and LGPL\nwould both suit me.\n\nAs a more concrete comment, is there anything you would like to hear\nfrom GUI developers during the development of libgit2? I'm not sure I\ncan contribute much in terms of code, but if you need any constructive\ncomments, I can help with that.\n\n- Pieter\n"},{"id":"94493","messageId":"20081031232539.GB14786@spearce.org","threadId":"16110","inReplyTo":"490B9262.5070905@gmail.com","subject":"Re: libgit2 - a true git library","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2008-10-31T23:25:39Z","receivedAt":"2008-10-31T23:25:39Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Bruno Santos wrote:\n> Shawn O. Pearce wrote:\n> > During the GitTogether we were kicking around the idea of a ground-up\n> > implementation of a Git library.\n> \n> We should take the opportunity a make it more portable. Instead of using\n> the posix api directly we should warp it in \"git_\" APIs. And be carefull\n> with certain APIs like fork or fork+exec [...]\n> \n> Here's an example, for the 'read' API, on how we can simply do this\n> without worries for the posix crowd:\n> \n> ssize_t git_read(git_fildes_t fildes, void* buf, size_t bufsize);\n> \n> Were git_fildes_t would be an int for posix and an HANDLE for win32.\n...\n\nYes, already on my mind. _IF_ this carries further I'll be involved\nwith its development, and I'll make certain its reasonably portable\nlike you are asking.\n\nThere are only a handful of things we really need to from the OS\nand I think we can wrap most of them up into little inline stubs\non POSIX (so POSIX folks have no impact) and on Win32 we can have\nsmall stubs (or again inline) so its pretty native on Win32.\n\nYes, I have considered say NPR or APR; I'm not going there.\nBoth are nice packages but there's also downsides to bringing them\ninto libgit2. IMHO its just easier to wrap the handful of things\nwe really need.\n\n-- \nShawn.\n"},{"id":"94494","messageId":"20081031232829.GC14786@spearce.org","threadId":"16110","inReplyTo":"CBF2AF68-BA41-4394-A837-F62864CF8BFB@ai.rug.nl","subject":"Re: libgit2 - a true git library","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2008-10-31T23:28:29Z","receivedAt":"2008-10-31T23:28:29Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Pieter de Bie wrote:\n> On 31 okt 2008, at 22:31, Pierre Habouzit wrote:\n>>\n>> Many of the people needing a library for libgit are probably reading \n>> the\n>> list, I'll let them comment\n>\n> As a more concrete comment, is there anything you would like to hear\n> from GUI developers during the development of libgit2? I'm not sure I\n> can contribute much in terms of code, but if you need any constructive\n> comments, I can help with that.\n\nWe'd like to hear feedback on the API. Look at the operations\nyour application does with git-core.\n\n Can you express that with the libgit2 API?\n If not, why not?\n Is it just that the docs are unclear?\n Is the API missing?\n What would you like to invoke to get the data you need?\n ...\n\nYou are the end-user of the library, so it needs to suit you. Ok,\nyou aren't the only end-user, but you and other developers like\nyou... :-)\n\n-- \nShawn.\n"},{"id":"94495","messageId":"20081031233317.GA29036@artemis.corp","threadId":"16110","inReplyTo":"7viqr873x7.fsf@gitster.siamese.dyndns.org","subject":"Re: libgit2 - a true git library","fromName":"Pierre Habouzit","fromEmail":"madcoder@debian.org","sentAt":"2008-10-31T23:33:18Z","receivedAt":"2008-10-31T23:33:18Z","isPatch":false,"sender":{"key":"madcoder@debian.org","avatar":"https://avatars.githubusercontent.com/u/44708?v=4"},"body":"On Fri, Oct 31, 2008 at 11:14:44PM +0000, Junio C Hamano wrote:\n> Nicolas Pitre writes:\n> \n> > My favorite license for a library is the GPL with the gcc exception,\n> > i.e. what libraries coming with gcc are using. They're GPL but with an \n> > exception allowing them to be linked with anything.\n> \n> Although I'd be Ok with either GPL + gcc exception on whatever core-ish\n> (i.e. what will be necessary for libgit2; \"blame\" would not count) pieces\n> I have in C-git codebase, \"can be linked with anything\" allows a gaping\n> hole to the library, which I'm a bit hesitant to swallow without thinking.\n\nWell I wasn't thinking about anything else than what is needed for the\nlibgit2. I love BSDish for libraries, though like GPL for the actual\n_tools_ I write with it.\n\n> E.g. our read_object() may look like this:\n> \n> void *read_object(const object_name_t sha1,\n> enum object_type *type,\n> size_t *size)\n> {\n> ...\n> }\n> \n> \n> but an extension a closed-source person may sell you back may do:\n> \n> +typedef void *read_object_fn(const object_name_t,\n> + enum object_type *,\n> + size_t *);\n> +read_object_fn read_object_custom = NULL;\n> void *read_object(const object_name_t sha1,\n> enum object_type *type,\n> size_t *size)\n> {\n> + if (read_object_custom != NULL)\n> + return read_object_custom(sha1, type, size);\n> ...\n> }\n> \n> I.e. use the supplied custom function to do proprietary magic, such as\n> reading the object lazily from elsewhere over the network. And we will\n> never get that magic bit back.\n\nWell, for one \"we're\" not supposed to accept any patch that does that,\nand I don't expect that the people who end up maintaining libgit2 will\nbecome rogue. Though if such bits of APIs do exist one day, then well, I\nsee no license except the GPL that can prevent you from that.\n\n\nMy idea of trying to be able to reuse git.git code is not a necessity,\na new implementation from scratch is likely to be possible. Though we\nall know that if the core git contributors don't contribute and\neventually use libgit2 this will not fly. That's why we must think about\nit.\n\nI assume given your answer that if libgit2 is BSD you may not be as\nmotivated to contribute code to it as you are to git.git, and this IMHO\nwould be a big no-go, like shawn said in another mail.\n\n-- \n·O· Pierre Habouzit\n··O madcoder@debian.org\nOOO http://www.madism.org\n"},{"id":"94497","messageId":"20081031234115.GD14786@spearce.org","threadId":"16110","inReplyTo":"7viqr873x7.fsf@gitster.siamese.dyndns.org","subject":"Re: libgit2 - a true git library","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2008-10-31T23:41:15Z","receivedAt":"2008-10-31T23:41:15Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Junio C Hamano wrote:\n> \n> Although I'd be Ok with either GPL + gcc exception on whatever core-ish\n> (i.e. what will be necessary for libgit2; \"blame\" would not count) pieces\n> I have in C-git codebase,\n\nSomeday I'm going to come back to you and ask for \"blame\" in libgit2.\nIts an important function to be able to execute for an end-user.\nLook at \"git gui blame\", its a major feature of the GUI.\n\nIf your blame implementation will never be available except under\nthe GPL then either it should be clean-room rewritten under the\nlibrary's license, or maybe there is a \"libgitblame\" that is GPL\nand can be optionally linked with libgit2 and a GPL'd application\nto get blame support.\n\n>\"can be linked with anything\" allows a gaping\n> hole to the library, which I'm a bit hesitant to swallow without thinking.\n> \n> E.g. our read_object() may look like this:\n> \n> void *read_object(const object_name_t sha1,\n> enum object_type *type,\n> size_t *size)\n> {\n> ...\n> }\n> \n> \n> but an extension a closed-source person may sell you back may do:\n> \n> +typedef void *read_object_fn(const object_name_t,\n> + enum object_type *,\n> + size_t *);\n> +read_object_fn read_object_custom = NULL;\n> void *read_object(const object_name_t sha1,\n> enum object_type *type,\n> size_t *size)\n> {\n> + if (read_object_custom != NULL)\n> + return read_object_custom(sha1, type, size);\n> ...\n> }\n> \n> I.e. use the supplied custom function to do proprietary magic, such as\n> reading the object lazily from elsewhere over the network. And we will\n> never get that magic bit back.\n\nAs a maintainer I'd never accept such a patch. I'd ask for the\ncode under read_object_custom, or toss the patch on the floor.\nBut that doesn't stop them from distributing the patched sources\nlike above, keeping the fun bits in the closed source portion of\nthe executable they distribute.\n\nMaybe I just think too highly of the other guy, but I'd hope that\nanyone patching libgit2 like above would try to avoid it, because\nthey'd face merge issues in the future.\n \n> Although no license asks this, my wish is that if somebody built on top of\n> what I wrote to make the world a better place, I'd like the same access to\n> that additional code so that I too can enjoy the improved world. Because\n> almost all of my code in git.git are under GPLv2, in reality I do not have\n> any access to your software as long as you do not distribute your\n> additional code that made the world a better place, which is a bit sad.\n\nIMHO, its a flaw of the GPL. GitHub anyone? Heck, even Google uses\na lot of GPL'd software internally (yes, we have Linux desktops and\nservers) but not all of the software we distribute internally goes\nexternal, so not all of our patches are published. *sigh*\n\nI've actually stayed awake at night sometimes wondering what the\nworld would be like if the GPL virual clause forced the source code\nfor a website to be opened, or forced you to publish your code\neven if you never distribute binaries beyond \"you\" (where \"you\"\nis some mega corp in many countries with many employees).\n\n-- \nShawn.\n"},{"id":"94498","messageId":"7v63n872bs.fsf@gitster.siamese.dyndns.org","threadId":"16110","inReplyTo":"20081031232829.GC14786@spearce.org","subject":"Re: libgit2 - a true git library","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-10-31T23:49:11Z","receivedAt":"2008-10-31T23:49:11Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Shawn O. Pearce\" writes:\n\n> You are the end-user of the library, so it needs to suit you. Ok,\n> you aren't the only end-user, but you and other developers like\n> you... :-)\n\nI will be the end-user of the library because if we want libgit2 to be\nanywhere close to successful, you should be able to port C-git to it.\n\nI understand that the apidocs/ is a very early work-in-progress, but\nstill, it bothers me that it is unclear to me what lifetime rules are in\neffect on the in-core objects. For example, in C-git, commit objects are\nnot just parsed but are modified in place as history is traversed\n(e.g. their flags smudged and their parents simplified). You have \"flags\"\nfield in commit, which implies to me that the design shares this same\n\"modified by processing in-place\" assumption. It is great for processing\nefficiency as long as you are a \"run once and let exit(3) clean-up\" type\nof program, but is quite problematic otherwise. commit.flags that C-git\nuses for traversal marker purposes, together with \"who are parents and\nchildren of this commit\", should probably be kept inside traversal module,\nif you want to make this truly reusable.\n\nBy the way, I hate git_result_t. That should be \"int\", the most natural\nintegral type on the platform.\n"},{"id":"94500","messageId":"m3tzasfhgb.fsf@localhost.localdomain","threadId":"16110","inReplyTo":"20081031234115.GD14786@spearce.org","subject":"Re: libgit2 - a true git library","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2008-10-31T23:56:51Z","receivedAt":"2008-10-31T23:56:51Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"\"Shawn O. Pearce\" writes:\n> Junio C Hamano wrote:\n\n> > Although no license asks this, my wish is that if somebody built on top of\n> > what I wrote to make the world a better place, I'd like the same access to\n> > that additional code so that I too can enjoy the improved world. Because\n> > almost all of my code in git.git are under GPLv2, in reality I do not have\n> > any access to your software as long as you do not distribute your\n> > additional code that made the world a better place, which is a bit sad.\n> \n> IMHO, its a flaw of the GPL. GitHub anyone? Heck, even Google uses\n> a lot of GPL'd software internally (yes, we have Linux desktops and\n> servers) but not all of the software we distribute internally goes\n> external, so not all of our patches are published. *sigh*\n> \n> I've actually stayed awake at night sometimes wondering what the\n> world would be like if the GPL virual clause forced the source code\n> for a website to be opened, or forced you to publish your code\n> even if you never distribute binaries beyond \"you\" (where \"you\"\n> is some mega corp in many countries with many employees).\n\nThere is such license, and it is called AGPLv3, Affero GPL[1].\n\nAnd of course Google prohibits (or did prohibit) using it for projects\nhosted at Google Code... wonder why... ;-)\n\n[1] http://en.wikipedia.org/wiki/AGPL\n-- \nJakub Narebski\nPoland\nShadeHawk on #git\n"},{"id":"94502","messageId":"20081101000213.GB29036@artemis.corp","threadId":"16110","inReplyTo":"7v63n872bs.fsf@gitster.siamese.dyndns.org","subject":"Re: libgit2 - a true git library","fromName":"Pierre Habouzit","fromEmail":"madcoder@debian.org","sentAt":"2008-11-01T00:02:13Z","receivedAt":"2008-11-01T00:02:13Z","isPatch":false,"sender":{"key":"madcoder@debian.org","avatar":"https://avatars.githubusercontent.com/u/44708?v=4"},"body":"On Fri, Oct 31, 2008 at 11:49:11PM +0000, Junio C Hamano wrote:\n> \"Shawn O. Pearce\" writes:\n> \n> > You are the end-user of the library, so it needs to suit you. Ok,\n> > you aren't the only end-user, but you and other developers like\n> > you... :-)\n> \n> I will be the end-user of the library because if we want libgit2 to be\n> anywhere close to successful, you should be able to port C-git to it.\n> \n> I understand that the apidocs/ is a very early work-in-progress, but\n> still, it bothers me that it is unclear to me what lifetime rules are in\n> effect on the in-core objects. For example, in C-git, commit objects are\n> not just parsed but are modified in place as history is traversed\n> (e.g. their flags smudged and their parents simplified). You have \"flags\"\n> field in commit, which implies to me that the design shares this same\n> \"modified by processing in-place\" assumption. It is great for processing\n> efficiency as long as you are a \"run once and let exit(3) clean-up\" type\n> of program, but is quite problematic otherwise. commit.flags that C-git\n> uses for traversal marker purposes, together with \"who are parents and\n> children of this commit\", should probably be kept inside traversal module,\n> if you want to make this truly reusable.\n\nI don't think it's impossible to have something efficient without this\nkind of hacks. You just need to dissociate the objects from their\nannotations, though use some kind of allocator that allow numbering of\nthe objects, and use that number as a lookup in an array of annotations.\nIt will require pool allocators for the annotations, but that should\nwork fine and efficientely.\n\n> By the way, I hate git_result_t. That should be \"int\", the most natural\n> integral type on the platform.\n\nI concur.\n\n-- \n·O· Pierre Habouzit\n··O madcoder@debian.org\nOOO http://www.madism.org\n"},{"id":"94503","messageId":"20081101001300.GE14786@spearce.org","threadId":"16110","inReplyTo":"7v63n872bs.fsf@gitster.siamese.dyndns.org","subject":"Re: libgit2 - a true git library","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2008-11-01T00:13:00Z","receivedAt":"2008-11-01T00:13:00Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Junio C Hamano wrote:\n> I understand that the apidocs/ is a very early work-in-progress, but\n> still, it bothers me that it is unclear to me what lifetime rules are in\n> effect on the in-core objects.\n\nYes, this needs a lot more documentation.\n\n> For example, in C-git, commit objects are\n> not just parsed but are modified in place as history is traversed\n> (e.g. their flags smudged and their parents simplified). You have \"flags\"\n> field in commit, which implies to me that the design shares this same\n> \"modified by processing in-place\" assumption.\n\nYup. I was assuming the same model, we modify in-place.\n\n> It is great for processing\n> efficiency as long as you are a \"run once and let exit(3) clean-up\" type\n> of program, but is quite problematic otherwise. commit.flags that C-git\n> uses for traversal marker purposes, together with \"who are parents and\n> children of this commit\", should probably be kept inside traversal module,\n> if you want to make this truly reusable.\n\nIts not efficient to keep this data inside of the \"traversal module\"\ninstance (aka what I called git_revp_t). You really want it inside\nof the commit itself (aka git_commit_t).\n\nMy thought here is that git_commit_t's are scoped within a given\ngit_revp_t that was used when they were parsed. That is:\n\n git_revp_t *pool_a = git_revp_alloc(db, NULL);\n git_revp_t *pool_b = git_revp_alloc(db, NULL);\n git_oid_t id;\n git_commit_t *commit_a, *commit_b, *commit_c;\n\n git_oid_mkstr(&id, \"3c223b36af9cace4f802a855fbb588b1dccf0648\");\n commit_a = git_commit_parse(pool_a, &id);\n commit_b = git_commit_parse(pool_b, &id);\n commit_c = git_commit_parse(pool_a, &id);\n\n if (commit_a == commit_b)\n die(\"the world just exploded\");\n else\n printf(\"this was correct behavior\\n\");\n\n if (commit_a == commit_c)\n printf(\"this was correct behavior\\n\");\n else\n die(\"the hash table is broken\");\n\nTo completely different git_revp_t's on the same database yeild\ndifferent commit pointers, but successive calls to parse the same\ncommit in the same pool yield the same pointer.\n\nCertain operations on the pool can cause it to alter its state\nin a way that cannot be reversed (e.g. rewrite parents). In such\ncases the caller should free the pool and alloc a new one in order\nto issue new traversals against the same object database, but with\nthe original (or differently rewritten) parent information.\n\n> By the way, I hate git_result_t. That should be \"int\", the most natural\n> integral type on the platform.\n\nYea, I'm torn on git_result_t myself. Some library APIs use their\nown result type, but as a typedef off int.\n\nI'm tempted to stick with int for the result type, but I don't\nwant readers to confuse our result type of 0 == success, <0 ==\nfailure with some case where we return a signed integral value as\na result of a computation.\n\nI'm also debating the error handling. Do we return the error\ncode as the return value from the function, or do we stick it into\nsome sort of thread-global like classic \"errno\", or do we ask the\napplication to pass in a structure to us?\n\nE.g.:\n\nReturn code:\n\n git_result_t r = git_foo_bar(...);\n if (r < 0)\n \tdie(\"foo_bar failed: %s\", git_strerr(r));\n\nUse an errno:\n\n if (git_foo_bar(...))\n \tdie(\"foo_bar failed: %s\", git_strerr(git_errno));\n\nUse a caller allocated struct:\n\n git_error_t err;\n if (git_foo_bar(..., &err))\n \tdie(\"foo_bar failed: %s\", git_strerr(&err));\n\nI'm slightly leaning towards the result code approach, as it means\nwe don't have to mess around with thread local variables.\n\nWe don't get to pass back anything more complex than an int (possibly\nlosing context about parameter values and/or on-disk state we want\nto report on), but we also don't have to deal with thread-locals\nor some messy \"always pass a thread context parameter\".\n\n-- \nShawn.\n"},{"id":"94507","messageId":"20081101001926.GF14786@spearce.org","threadId":"16110","inReplyTo":"20081101000213.GB29036@artemis.corp","subject":"Re: libgit2 - a true git library","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2008-11-01T00:19:26Z","receivedAt":"2008-11-01T00:19:26Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Pierre Habouzit wrote:\n> On Fri, Oct 31, 2008 at 11:49:11PM +0000, Junio C Hamano wrote:\n> > \n> > I understand that the apidocs/ is a very early work-in-progress, but\n> > still, it bothers me that it is unclear to me what lifetime rules are in\n> > effect on the in-core objects. For example, in C-git, commit objects are\n> > not just parsed but are modified in place as history is traversed\n> > (e.g. their flags smudged and their parents simplified). You have \"flags\"\n> > field in commit, which implies to me that the design shares this same\n> > \"modified by processing in-place\" assumption.\n> \n> I don't think it's impossible to have something efficient without this\n> kind of hacks. You just need to dissociate the objects from their\n> annotations, though use some kind of allocator that allow numbering of\n> the objects, and use that number as a lookup in an array of annotations.\n> It will require pool allocators for the annotations, but that should\n> work fine and efficientely.\n\nInteresting approach. I don't know why I didn't think of that one.\n\nYou'll still need to be able to toss parts of the git graph though.\nIf you just pin everything in memory under a single global object\ntable you'll run server processes out of memory as they chug through\nlarge numbers of repositories.\n\n> > By the way, I hate git_result_t. That should be \"int\", the most natural\n> > integral type on the platform.\n> \n> I concur.\n\nint it is then.\n\n-- \nShawn.\n"},{"id":"94509","messageId":"alpine.DEB.1.10.0810311738100.5851@asgard.lang.hm","threadId":"16110","inReplyTo":"20081031234115.GD14786@spearce.org","subject":"Re: libgit2 - a true git library","fromName":"","fromEmail":"david@lang.hm","sentAt":"2008-11-01T00:41:22Z","receivedAt":"2008-11-01T00:41:22Z","isPatch":false,"sender":{"key":"david@lang.hm","avatar":null},"body":"On Fri, 31 Oct 2008, Shawn O. Pearce wrote:\n\n> Junio C Hamano wrote:\n>>\n>>\n>> I.e. use the supplied custom function to do proprietary magic, such as\n>> reading the object lazily from elsewhere over the network. And we will\n>> never get that magic bit back.\n>\n> As a maintainer I'd never accept such a patch. I'd ask for the\n> code under read_object_custom, or toss the patch on the floor.\n> But that doesn't stop them from distributing the patched sources\n> like above, keeping the fun bits in the closed source portion of\n> the executable they distribute.\n>\n> Maybe I just think too highly of the other guy, but I'd hope that\n> anyone patching libgit2 like above would try to avoid it, because\n> they'd face merge issues in the future.\n\nthe issue that I see is that libgit2 will be (on most systems) a shared \nlibrary.\n\nwhat's to stop someone from taking the libgit2 code, adding the magic \nproprietary piece, and selling a new libgit2 library binary 'just replace \nyour existing shared library with this new one and all your git related \nprograms gain this feature'\n\nthey would only face merge issues if they need to keep up to date with \nyou, and git makes it pretty easy to maintain a fork if you only have to \ndo one-way merging (rere)\n\nDavid Lang\n"},{"id":"94511","messageId":"20081101010011.GG14786@spearce.org","threadId":"16110","inReplyTo":"alpine.DEB.1.10.0810311738100.5851@asgard.lang.hm","subject":"Re: libgit2 - a true git library","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2008-11-01T01:00:11Z","receivedAt":"2008-11-01T01:00:11Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"david@lang.hm wrote:\n> On Fri, 31 Oct 2008, Shawn O. Pearce wrote:\n>> Junio C Hamano wrote:\n>>>\n>>> I.e. use the supplied custom function to do proprietary magic, such as\n>>> reading the object lazily from elsewhere over the network. And we will\n>>> never get that magic bit back.\n>>\n>> Maybe I just think too highly of the other guy, but I'd hope that\n>> anyone patching libgit2 like above would try to avoid it, because\n>> they'd face merge issues in the future.\n>\n> the issue that I see is that libgit2 will be (on most systems) a shared \n> library.\n>\n> what's to stop someone from taking the libgit2 code, adding the magic \n> proprietary piece, and selling a new libgit2 library binary 'just replace \n> your existing shared library with this new one and all your git related \n> programs gain this feature'\n\nTrue. The only thing that prevents that is the normal GPL. The\nLGPL and GPL+\"gcc exception\" allow this sort of mean behavior.\nI doubt there's enough of a market for that; replacing a library\nis something of a pain and if the feature really is interesting or\nuseful someone will write a clean-room re-implementation and submit\npatches to do the same thing.\n\n> they would only face merge issues if they need to keep up to date with \n> you, and git makes it pretty easy to maintain a fork if you only have to \n> do one-way merging (rere)\n\nIn other words, we're too good for our own good. ;-)\n\n-- \nShawn.\n"},{"id":"94510","messageId":"20081101010211.GC29036@artemis.corp","threadId":"16110","inReplyTo":"20081101001926.GF14786@spearce.org","subject":"Re: libgit2 - a true git library","fromName":"Pierre Habouzit","fromEmail":"madcoder@debian.org","sentAt":"2008-11-01T01:02:11Z","receivedAt":"2008-11-01T01:02:11Z","isPatch":false,"sender":{"key":"madcoder@debian.org","avatar":"https://avatars.githubusercontent.com/u/44708?v=4"},"body":"On Sat, Nov 01, 2008 at 12:19:26AM +0000, Shawn O. Pearce wrote:\n> Pierre Habouzit wrote:\n> > On Fri, Oct 31, 2008 at 11:49:11PM +0000, Junio C Hamano wrote:\n> > > \n> > > I understand that the apidocs/ is a very early work-in-progress, but\n> > > still, it bothers me that it is unclear to me what lifetime rules are in\n> > > effect on the in-core objects. For example, in C-git, commit objects are\n> > > not just parsed but are modified in place as history is traversed\n> > > (e.g. their flags smudged and their parents simplified). You have \"flags\"\n> > > field in commit, which implies to me that the design shares this same\n> > > \"modified by processing in-place\" assumption.\n> > \n> > I don't think it's impossible to have something efficient without this\n> > kind of hacks. You just need to dissociate the objects from their\n> > annotations, though use some kind of allocator that allow numbering of\n> > the objects, and use that number as a lookup in an array of annotations.\n> > It will require pool allocators for the annotations, but that should\n> > work fine and efficientely.\n> \n> Interesting approach. I don't know why I didn't think of that one.\n> \n> You'll still need to be able to toss parts of the git graph though.\n> If you just pin everything in memory under a single global object\n> table you'll run server processes out of memory as they chug through\n> large numbers of repositories.\n\nSure, but for that you just need to reinject the numbers into some kind\nof free list (hint a bitmap) to reuse old slots. Of course this is some\nkind of take-once never-release approach _BUT_ one can do better and\n\"defrag\" this at times.\n\nE.g. for a server if we take your idea, there are some times we probably\n*know* nobody has kept a reference to one of the pointers and we can\nreorganize some pointers around to free chunks of data not allocated.\n\nAn escape way is to use mmap + madvise for those pools, the former to\nallocate the memory and the latter to drop large unused ranges when\nneeded (remapping with MAP_FIXED is also supposed to work but fails on\nMacos it seems). Even win32 has what it takes to do so (just skim\nthrough the code of jemalloc, that is used for mozilla, it's quite\nportable on POSIX + Windows).\n\nI was thinking that one should create and register pools of annotations\nas such, define the size of the annotation (IOW the size of cell), and\nlet deal with that. I imagine the stuff as some kind of allocator that\nwould allocate the first time e.g. 4k of objects, and if you need more\n8k, and if need more 16k and so on exponentially. Mapping an integer to\na cell in this kind of ropes is _really_ efficient: you need to know the\nfirst bit set (__builtin_clz is of help with gcc, it can be emulated\nwith proper asm for many non-gcc platforms also) to give you the number\nof the rope component that you intend to address, you clear that bit,\nthe resulting number is the index in that rope component of the cell\nyou're interested in. On most machines such an operation would be a few\ncycles, which should be fairly little wrt the operation you would do\nduring a traversal.\n\nMaintaining the bitmap is important, so that we can give back large\nchunks of physical memory back to the system, and many malloc\nimplementations would likely use such kind of implementations, we would\njust do our (which is arguably heavy) but gives us control on the cell\nnumber, which is just what we need.\n\nOf _course_ when we have a better / more natural place to put\nannotations we need, let's do it ! But the kind of thing I just imagined\nis more generic and can help to do arbitrary complex stuff during a\ntraversal.\n\nWe may want to explore a storage that is aware of the objects type also,\nas I expect the liveness of objects to depend on the objects type a lot,\nand some traversal to not care about some kind of objects at all, which\nwhen combined with lazy allocation in the annotations pools, could end\nup with reduced memory footprint. We could e.g. use two bits of the\n\"object handler\" to store the type and dispatch into 3 (commit, tree,\nblob) ropes instead of a big one.\n\n\nThe object and annotations store _is_ what makes git efficient, and is\nthe very service the library will have to support well. We _will_ have\nto put some clever code in there and we cannot sacrifice any kind of\nperformance, this will be the tight loop every time. I don't think\nthere will be anything else in git that is so central for performance.\n\nOn the other hand, it's probably ok for the first versions of the\nlibrary to have some things in it that don't perform well in long lived\nprocesses (repacking e.g., if we ever want to do that in the library, as\nit's slow and that forking a git-repack away should not be a too\nexpensive cost anyway).\n-- \n·O· Pierre Habouzit\n··O madcoder@debian.org\nOOO http://www.madism.org\n"},{"id":"94512","messageId":"alpine.DEB.1.10.0810311802360.5851@asgard.lang.hm","threadId":"16110","inReplyTo":"20081101010011.GG14786@spearce.org","subject":"Re: libgit2 - a true git library","fromName":"","fromEmail":"david@lang.hm","sentAt":"2008-11-01T01:04:50Z","receivedAt":"2008-11-01T01:04:50Z","isPatch":false,"sender":{"key":"david@lang.hm","avatar":null},"body":"On Fri, 31 Oct 2008, Shawn O. Pearce wrote:\n\n> david@lang.hm wrote:\n>> On Fri, 31 Oct 2008, Shawn O. Pearce wrote:\n>>> Junio C Hamano wrote:\n>>>>\n>>>> I.e. use the supplied custom function to do proprietary magic, such as\n>>>> reading the object lazily from elsewhere over the network. And we will\n>>>> never get that magic bit back.\n>>>\n>>> Maybe I just think too highly of the other guy, but I'd hope that\n>>> anyone patching libgit2 like above would try to avoid it, because\n>>> they'd face merge issues in the future.\n>>\n>> the issue that I see is that libgit2 will be (on most systems) a shared\n>> library.\n>>\n>> what's to stop someone from taking the libgit2 code, adding the magic\n>> proprietary piece, and selling a new libgit2 library binary 'just replace\n>> your existing shared library with this new one and all your git related\n>> programs gain this feature'\n>\n> True. The only thing that prevents that is the normal GPL. The\n> LGPL and GPL+\"gcc exception\" allow this sort of mean behavior.\n> I doubt there's enough of a market for that; replacing a library\n> is something of a pain and if the feature really is interesting or\n> useful someone will write a clean-room re-implementation and submit\n> patches to do the same thing.\n\nhow would the LGPL of GPL+gcc extention allow this? if they modify the \ncode in the library and then distribute the modified library wouldn't they \nbe required to distribute the changes to that library?\n\nthey could use the LGPL or GPL+exception library with their propriatary \nprogram, but I don't see how they could get away with modifying the \nlibrary.\n\nDavid Lang\n"},{"id":"94513","messageId":"20081101010620.GD29036@artemis.corp","threadId":"16110","inReplyTo":"alpine.DEB.1.10.0810311738100.5851@asgard.lang.hm","subject":"Re: libgit2 - a true git library","fromName":"Pierre Habouzit","fromEmail":"madcoder@debian.org","sentAt":"2008-11-01T01:06:20Z","receivedAt":"2008-11-01T01:06:20Z","isPatch":false,"sender":{"key":"madcoder@debian.org","avatar":"https://avatars.githubusercontent.com/u/44708?v=4"},"body":"On Sat, Nov 01, 2008 at 12:41:22AM +0000, david@lang.hm wrote:\n> On Fri, 31 Oct 2008, Shawn O. Pearce wrote:\n> \n> >Junio C Hamano wrote:\n> >>\n> >>\n> >>I.e. use the supplied custom function to do proprietary magic, such as\n> >>reading the object lazily from elsewhere over the network. And we will\n> >>never get that magic bit back.\n> >\n> >As a maintainer I'd never accept such a patch. I'd ask for the\n> >code under read_object_custom, or toss the patch on the floor.\n> >But that doesn't stop them from distributing the patched sources\n> >like above, keeping the fun bits in the closed source portion of\n> >the executable they distribute.\n> >\n> >Maybe I just think too highly of the other guy, but I'd hope that\n> >anyone patching libgit2 like above would try to avoid it, because\n> >they'd face merge issues in the future.\n> \n> the issue that I see is that libgit2 will be (on most systems) a shared \n> library.\n> \n> what's to stop someone from taking the libgit2 code, adding the magic \n> proprietary piece, and selling a new libgit2 library binary 'just replace \n> your existing shared library with this new one and all your git related \n> programs gain this feature'\n\nIts license. GPL even with GCC exception would not allow you to do that.\nThough they could propose a fork of the library patched, with the patch\ndistributed. The downside would be that their code would not be binary\ncompatible with the \"true\" libgit2, so they would probably have to\nchange the name to avoid namespace clashes, or overwrite the \"real\"\nlibrary.\n\nBut yes, it's theoretically feasible. I'm not sure it would be worth the\nhassle, and if they respect the license (if they don't they can already\ndo that with the current git anyway) then the fact that someone would\nwant to do something like that would be known fact, probably not\navoided, but known.\n\n-- \n·O· Pierre Habouzit\n··O madcoder@debian.org\nOOO http://www.madism.org\n"},{"id":"94514","messageId":"20081101010824.GE29036@artemis.corp","threadId":"16110","inReplyTo":"alpine.DEB.1.10.0810311802360.5851@asgard.lang.hm","subject":"Re: libgit2 - a true git library","fromName":"Pierre Habouzit","fromEmail":"madcoder@debian.org","sentAt":"2008-11-01T01:08:24Z","receivedAt":"2008-11-01T01:08:24Z","isPatch":false,"sender":{"key":"madcoder@debian.org","avatar":"https://avatars.githubusercontent.com/u/44708?v=4"},"body":"On Sat, Nov 01, 2008 at 01:04:50AM +0000, david@lang.hm wrote:\n> On Fri, 31 Oct 2008, Shawn O. Pearce wrote:\n> \n> >david@lang.hm wrote:\n> >>On Fri, 31 Oct 2008, Shawn O. Pearce wrote:\n> >>>Junio C Hamano wrote:\n> >>>>\n> >>>>I.e. use the supplied custom function to do proprietary magic, such \n> >>>>as\n> >>>>reading the object lazily from elsewhere over the network. And we \n> >>>>will\n> >>>>never get that magic bit back.\n> >>>\n> >>>Maybe I just think too highly of the other guy, but I'd hope that\n> >>>anyone patching libgit2 like above would try to avoid it, because\n> >>>they'd face merge issues in the future.\n> >>\n> >>the issue that I see is that libgit2 will be (on most systems) a shared\n> >>library.\n> >>\n> >>what's to stop someone from taking the libgit2 code, adding the magic\n> >>proprietary piece, and selling a new libgit2 library binary 'just \n> >>replace\n> >>your existing shared library with this new one and all your git related\n> >>programs gain this feature'\n> >\n> >True. The only thing that prevents that is the normal GPL. The\n> >LGPL and GPL+\"gcc exception\" allow this sort of mean behavior.\n> >I doubt there's enough of a market for that; replacing a library\n> >is something of a pain and if the feature really is interesting or\n> >useful someone will write a clean-room re-implementation and submit\n> >patches to do the same thing.\n> \n> how would the LGPL of GPL+gcc extention allow this? if they modify the \n> code in the library and then distribute the modified library wouldn't \n> they be required to distribute the changes to that library?\n\nSee junio's example. It's rather easy to add hooks into the library to\nimplement a feature outside of it. It's even possible to do it while\npreserving the ABI fully IMHO (by being a strict superset of it).\n\nThe patch would be so trivial, that I see no reason why they wouldn't\nprovide it. Though the real implementation of the feature that would be\ndelegated through it would be in their closed source stuff.\n\n-- \n·O· Pierre Habouzit\n··O madcoder@debian.org\nOOO http://www.madism.org\n"},{"id":"94515","messageId":"alpine.LFD.2.00.0810312106310.13034@xanadu.home","threadId":"16110","inReplyTo":"20081101001300.GE14786@spearce.org","subject":"Re: libgit2 - a true git library","fromName":"Nicolas Pitre","fromEmail":"nico@cam.org","sentAt":"2008-11-01T01:15:10Z","receivedAt":"2008-11-01T01:15:10Z","isPatch":false,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Fri, 31 Oct 2008, Shawn O. Pearce wrote:\n\n> I'm tempted to stick with int for the result type, but I don't\n> want readers to confuse our result type of 0 == success, <0 ==\n> failure with some case where we return a signed integral value as\n> a result of a computation.\n\nDoes this actually happen ?\n\n> I'm also debating the error handling. Do we return the error\n> code as the return value from the function, or do we stick it into\n> some sort of thread-global like classic \"errno\", or do we ask the\n> application to pass in a structure to us?\n\nPassing a structure pointer for errors is adding overhead to the API in \nall cases, please don't do that.\nBoth the negative code and errno style are lightweight in the common \"no \nerror\" case. The errno style is probably more handy for those functions \nreturning a pointer which should be NULL in the error case.\n\n\nNicolas\n"},{"id":"94516","messageId":"20081101011910.GH14786@spearce.org","threadId":"16110","inReplyTo":"alpine.LFD.2.00.0810312106310.13034@xanadu.home","subject":"Re: libgit2 - a true git library","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2008-11-01T01:19:10Z","receivedAt":"2008-11-01T01:19:10Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Nicolas Pitre wrote:\n> On Fri, 31 Oct 2008, Shawn O. Pearce wrote:\n> \n> > I'm tempted to stick with int for the result type, but I don't\n> > want readers to confuse our result type of 0 == success, <0 ==\n> > failure with some case where we return a signed integral value as\n> > a result of a computation.\n> \n> Does this actually happen ?\n\nSometimes, but yea, its not that often. I think we've already\nsettled that it shall be \"int\".\n\nI'm updating some stuff like that (and dropping _t) as I write\nthis email.\n \n> > I'm also debating the error handling. Do we return the error\n> > code as the return value from the function, or do we stick it into\n> > some sort of thread-global like classic \"errno\", or do we ask the\n> > application to pass in a structure to us?\n> \n> Passing a structure pointer for errors is adding overhead to the API in \n> all cases, please don't do that.\n\nAgreed. Not going there.\n\n> Both the negative code and errno style are lightweight in the common \"no \n> error\" case. The errno style is probably more handy for those functions \n> returning a pointer which should be NULL in the error case.\n\nI'm sticking with return a negative code for now, to the extent\nthat some functions which return a pointer but also have many\ncommon failure modes (e.g. git_odb_open) use an output parameter\nas their first arg, so the error code can be returned as the result\nof the function.\n\n-- \nShawn.\n"},{"id":"94517","messageId":"alpine.LFD.2.00.0810312121000.13034@xanadu.home","threadId":"16110","inReplyTo":"20081101010824.GE29036@artemis.corp","subject":"Re: libgit2 - a true git library","fromName":"Nicolas Pitre","fromEmail":"nico@cam.org","sentAt":"2008-11-01T01:33:51Z","receivedAt":"2008-11-01T01:33:51Z","isPatch":false,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Sat, 1 Nov 2008, Pierre Habouzit wrote:\n\n> See junio's example. It's rather easy to add hooks into the library to\n> implement a feature outside of it. It's even possible to do it while\n> preserving the ABI fully IMHO (by being a strict superset of it).\n> \n> The patch would be so trivial, that I see no reason why they wouldn't\n> provide it. Though the real implementation of the feature that would be\n> delegated through it would be in their closed source stuff.\n\nBut at that point it's a matter of public perception. It would clearly \nbe against the spirit of the license even though the license itself \ncouldn't prevent such tortuous practices. Those people doing such \nthings would clearly be identified as bad guys and get bad press, and \nlibgit contributors could even attempt law suits based on the derived \nwork angle. That might be just enough to prevent such things to happen.\n\nOTOH they would certainly come out clean if the license was BSD since \nthe spirit of that license explicitly allows closing up the whole \nlibrary and adding extra features.\n\n\nNicolas\n"},{"id":"94518","messageId":"alpine.DEB.1.10.0810311835500.5851@asgard.lang.hm","threadId":"16110","inReplyTo":"20081101010620.GD29036@artemis.corp","subject":"Re: libgit2 - a true git library","fromName":"","fromEmail":"david@lang.hm","sentAt":"2008-11-01T01:36:32Z","receivedAt":"2008-11-01T01:36:32Z","isPatch":false,"sender":{"key":"david@lang.hm","avatar":null},"body":"On Sat, 1 Nov 2008, Pierre Habouzit wrote:\n\n> On Sat, Nov 01, 2008 at 12:41:22AM +0000, david@lang.hm wrote:\n>> On Fri, 31 Oct 2008, Shawn O. Pearce wrote:\n>>\n>>> Junio C Hamano wrote:\n>>>>\n>>>>\n>>>> I.e. use the supplied custom function to do proprietary magic, such as\n>>>> reading the object lazily from elsewhere over the network. And we will\n>>>> never get that magic bit back.\n>>>\n>>> As a maintainer I'd never accept such a patch. I'd ask for the\n>>> code under read_object_custom, or toss the patch on the floor.\n>>> But that doesn't stop them from distributing the patched sources\n>>> like above, keeping the fun bits in the closed source portion of\n>>> the executable they distribute.\n>>>\n>>> Maybe I just think too highly of the other guy, but I'd hope that\n>>> anyone patching libgit2 like above would try to avoid it, because\n>>> they'd face merge issues in the future.\n>>\n>> the issue that I see is that libgit2 will be (on most systems) a shared\n>> library.\n>>\n>> what's to stop someone from taking the libgit2 code, adding the magic\n>> proprietary piece, and selling a new libgit2 library binary 'just replace\n>> your existing shared library with this new one and all your git related\n>> programs gain this feature'\n>\n> Its license. GPL even with GCC exception would not allow you to do that.\n> Though they could propose a fork of the library patched, with the patch\n> distributed. The downside would be that their code would not be binary\n> compatible with the \"true\" libgit2, so they would probably have to\n> change the name to avoid namespace clashes, or overwrite the \"real\"\n> library.\n\nthe proposal had been for the library to be BSD, not GPL. and I don't see \nany reason why such a thing couldn't be done with a BSE library.\n\nDavid Lang\n\n> But yes, it's theoretically feasible. I'm not sure it would be worth the\n> hassle, and if they respect the license (if they don't they can already\n> do that with the current git anyway) then the fact that someone would\n> want to do something like that would be known fact, probably not\n> avoided, but known.\n>\n>\n"},{"id":"94519","messageId":"20081101013815.GF29036@artemis.corp","threadId":"16110","inReplyTo":"alpine.LFD.2.00.0810312121000.13034@xanadu.home","subject":"Re: libgit2 - a true git library","fromName":"Pierre Habouzit","fromEmail":"madcoder@debian.org","sentAt":"2008-11-01T01:38:15Z","receivedAt":"2008-11-01T01:38:15Z","isPatch":false,"sender":{"key":"madcoder@debian.org","avatar":"https://avatars.githubusercontent.com/u/44708?v=4"},"body":"On Sat, Nov 01, 2008 at 01:33:51AM +0000, Nicolas Pitre wrote:\n> OTOH they would certainly come out clean if the license was BSD since \n> the spirit of that license explicitly allows closing up the whole \n> library and adding extra features.\n\nI think it's pretty clear most contributor from git.git would be driven\naway if the license isn't GPLish. So I was basing my answer on a\nsupposition it would be.\n\n-- \n·O· Pierre Habouzit\n··O madcoder@debian.org\nOOO http://www.madism.org\n"},{"id":"94520","messageId":"20081101014336.GI14786@spearce.org","threadId":"16110","inReplyTo":"alpine.LFD.2.00.0810312121000.13034@xanadu.home","subject":"Re: libgit2 - a true git library","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2008-11-01T01:43:36Z","receivedAt":"2008-11-01T01:43:36Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Nicolas Pitre wrote:\n> On Sat, 1 Nov 2008, Pierre Habouzit wrote:\n> \n> > See junio's example. It's rather easy to add hooks into the library to\n> > implement a feature outside of it. It's even possible to do it while\n> > preserving the ABI fully IMHO (by being a strict superset of it).\n> > \n> > The patch would be so trivial, that I see no reason why they wouldn't\n> > provide it. Though the real implementation of the feature that would be\n> > delegated through it would be in their closed source stuff.\n> \n> But at that point it's a matter of public perception. It would clearly \n> be against the spirit of the license even though the license itself \n> couldn't prevent such tortuous practices. Those people doing such \n> things would clearly be identified as bad guys and get bad press, and \n> libgit contributors could even attempt law suits based on the derived \n> work angle. That might be just enough to prevent such things to happen.\n\nYes.\n \n> OTOH they would certainly come out clean if the license was BSD since \n> the spirit of that license explicitly allows closing up the whole \n> library and adding extra features.\n\nMy take on the consensus for the license part of the discussion is\nthat libgit2 should be under the \"GPL gcc library\" license.\n\nBTW, I can't actually find a copy of that license; the only thing\nI can locate in the GCC SVN tree is a copy of the LGPL.\n\n-- \nShawn.\n"},{"id":"94521","messageId":"alpine.LFD.2.00.0810312135190.13034@xanadu.home","threadId":"16110","inReplyTo":"20081101011910.GH14786@spearce.org","subject":"Re: libgit2 - a true git library","fromName":"Nicolas Pitre","fromEmail":"nico@cam.org","sentAt":"2008-11-01T01:45:39Z","receivedAt":"2008-11-01T01:45:39Z","isPatch":false,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Fri, 31 Oct 2008, Shawn O. Pearce wrote:\n\n> > Both the negative code and errno style are lightweight in the common \"no \n> > error\" case. The errno style is probably more handy for those functions \n> > returning a pointer which should be NULL in the error case.\n> \n> I'm sticking with return a negative code for now, to the extent\n> that some functions which return a pointer but also have many\n> common failure modes (e.g. git_odb_open) use an output parameter\n> as their first arg, so the error code can be returned as the result\n> of the function.\n\nActually, the pointer-returning functions can encode error cases into a \n\"negative\" pointer. See include/linux/err.h for example.\n\n\tvoid *ptr = git_alloc_foo(...);\n\tif (IS_ERR(ptr))\n\t\tdie(\"git_alloc_foo failed: %s\", git_strerr(PTR_ERR(ptr)));\n\n\nNicolas\n"},{"id":"94522","messageId":"alpine.LFD.2.00.0810312146420.13034@xanadu.home","threadId":"16110","inReplyTo":"20081101013815.GF29036@artemis.corp","subject":"Re: libgit2 - a true git library","fromName":"Nicolas Pitre","fromEmail":"nico@cam.org","sentAt":"2008-11-01T01:49:51Z","receivedAt":"2008-11-01T01:49:51Z","isPatch":false,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Sat, 1 Nov 2008, Pierre Habouzit wrote:\n\n> On Sat, Nov 01, 2008 at 01:33:51AM +0000, Nicolas Pitre wrote:\n> > OTOH they would certainly come out clean if the license was BSD since \n> > the spirit of that license explicitly allows closing up the whole \n> > library and adding extra features.\n> \n> I think it's pretty clear most contributor from git.git would be driven\n> away if the license isn't GPLish. So I was basing my answer on a\n> supposition it would be.\n\nSure. My point is the difference in guiltiness. Even if a LGPL license \ndoesn't prevent all abuses, it at least allows for public denunciations \nat the very least.\n\n\nNicolas\n"},{"id":"94523","messageId":"20081101015217.GJ14786@spearce.org","threadId":"16110","inReplyTo":"alpine.LFD.2.00.0810312135190.13034@xanadu.home","subject":"Re: libgit2 - a true git library","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2008-11-01T01:52:17Z","receivedAt":"2008-11-01T01:52:17Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Nicolas Pitre wrote:\n> On Fri, 31 Oct 2008, Shawn O. Pearce wrote:\n> \n> > > Both the negative code and errno style are lightweight in the common \"no \n> > > error\" case. The errno style is probably more handy for those functions \n> > > returning a pointer which should be NULL in the error case.\n> > \n> > I'm sticking with return a negative code for now, to the extent\n> > that some functions which return a pointer but also have many\n> > common failure modes (e.g. git_odb_open) use an output parameter\n> > as their first arg, so the error code can be returned as the result\n> > of the function.\n> \n> Actually, the pointer-returning functions can encode error cases into a \n> \"negative\" pointer. See include/linux/err.h for example.\n> \n> \tvoid *ptr = git_alloc_foo(...);\n> \tif (IS_ERR(ptr))\n> \t\tdie(\"git_alloc_foo failed: %s\", git_strerr(PTR_ERR(ptr)));\n\nOh, good point. We could also stagger the errors so they are\nalways odd, and never return an odd-alignment pointer from a\nsuccessful function. Thus IS_ERR can be written as:\n\n #define IS_ERR(ptr) (((intptr_t)(ptr)) & 1)\n\nwhich is quite cheap, and given the (probably required anyway)\naligned allocation policy means we still have 2^31 possible\nerror codes available.\n\n-- \nShawn.\n"},{"id":"94524","messageId":"alpine.LFD.2.00.0810312150200.13034@xanadu.home","threadId":"16110","inReplyTo":"20081101014336.GI14786@spearce.org","subject":"Re: libgit2 - a true git library","fromName":"Nicolas Pitre","fromEmail":"nico@cam.org","sentAt":"2008-11-01T01:53:53Z","receivedAt":"2008-11-01T01:53:53Z","isPatch":false,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Fri, 31 Oct 2008, Shawn O. Pearce wrote:\n\n> My take on the consensus for the license part of the discussion is\n> that libgit2 should be under the \"GPL gcc library\" license.\n> \n> BTW, I can't actually find a copy of that license; the only thing\n> I can locate in the GCC SVN tree is a copy of the LGPL.\n\nThe exception is usually found at the top of files constituting \nlibgcc.a. One example is gcc/config/arm/ieee754-df.S. ;-)\n\n\nNicolas\n"},{"id":"94528","messageId":"alpine.DEB.1.00.0811010320370.22125@pacific.mpi-cbg.de.mpi-cbg.de","threadId":"16110","inReplyTo":"20081101015217.GJ14786@spearce.org","subject":"Re: libgit2 - a true git library","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-11-01T02:26:45Z","receivedAt":"2008-11-01T02:26:45Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Fri, 31 Oct 2008, Shawn O. Pearce wrote:\n\n> Nicolas Pitre wrote:\n> > On Fri, 31 Oct 2008, Shawn O. Pearce wrote:\n> > \n> > > > Both the negative code and errno style are lightweight in the \n> > > > common \"no error\" case. The errno style is probably more handy \n> > > > for those functions returning a pointer which should be NULL in \n> > > > the error case.\n\nUnfortunately, errno would not be thread-safe, unless you can guarantee \nthat errno is a thread-local variable.\n\n> > > \n> > > I'm sticking with return a negative code for now, to the extent that \n> > > some functions which return a pointer but also have many common \n> > > failure modes (e.g. git_odb_open) use an output parameter as their \n> > > first arg, so the error code can be returned as the result of the \n> > > function.\n> > \n> > Actually, the pointer-returning functions can encode error cases into a \n> > \"negative\" pointer. See include/linux/err.h for example.\n> > \n> > \tvoid *ptr = git_alloc_foo(...);\n> > \tif (IS_ERR(ptr))\n> > \t\tdie(\"git_alloc_foo failed: %s\", git_strerr(PTR_ERR(ptr)));\n> \n> Oh, good point. We could also stagger the errors so they are\n> always odd, and never return an odd-alignment pointer from a\n> successful function. Thus IS_ERR can be written as:\n> \n> #define IS_ERR(ptr) (((intptr_t)(ptr)) & 1)\n> \n> which is quite cheap, and given the (probably required anyway)\n> aligned allocation policy means we still have 2^31 possible\n> error codes available.\n\nOh boy, both solutions are ugly as hell. Although the &1 method does not \nlimit the memory space as much (except if you plan to work in \nspace-contrained environments, where you do not want to be forced to \nword-align structs).\n\nThe only pointer game I would remotely consider clean is if you had\n\n\tconst char *errors[] = {\n\t\t...\n\t};\n\n\tinline int is_error(void *ptr) {\n\t\treturn ptr >= errors && ptr < errors + ARRAY_SIZE(errors);\n\t}\n\nalthough that would be less performant.\n\nCiao,\nDscho\n"},{"id":"94544","messageId":"490C34DD.8070201@op5.se","threadId":"16110","inReplyTo":"alpine.LFD.2.00.0810311756160.13034@xanadu.home","subject":"Re: libgit2 - a true git library","fromName":"Andreas Ericsson","fromEmail":"ae@op5.se","sentAt":"2008-11-01T10:52:13Z","receivedAt":"2008-11-01T10:52:13Z","isPatch":false,"sender":{"key":"ae@op5.se","avatar":"https://gravatar.com/avatar/426e89595c75a8f5252dd0c989e5fabe5bcac616e68557427ad9aef6b0ca342a?d=mp&s=160"},"body":"Nicolas Pitre wrote:\n> On Fri, 31 Oct 2008, Pierre Habouzit wrote:\n> \n>> Git is currently mostly \"GPLv2 or later\". A BSDish license was\n>> mentioned, because it's the most permissive one and that nobody cared\n>> that much, though a LGPL/GPL-with-GCC-exception would probably fly.\n> \n> I do care. I think the BSD license is too permissive. There are really \n> nifty pieces of code in Git that I would be really sorry to see go \n> proprietary.\n> \n\nI agree with Nicolas, for what it's worth. Although I realize my small\ncontributions to the git code can be trivially rewritten, I think that\nwill definitely be trickier with Nicolas' contributions.\n\n> \n>> OT: FWIW I prefer BSDish licenses (even the MIT actually) for libraries\n>> because I believe that computing is overall better if everyone can use\n>> the right tool for the task, and I don't want to prevent people from\n>> using good stuff (I hope I write good stuff ;P) because of the license.\n> \n> Everybody can and does link against glibc on Linux which is LGPL. So \n> that doesn't affect \"usage\".\n> \n\nFor dynamic libraries, yes, as you can replace them in-flight if you need\nto. For static libraries, it's a different matter. I think the gcc exception\nis rather special, as parts of gcc is the dynamic linker initialization code,\nwhich has to be shipped in object form and included in all executables\nproduced with gcc. Perhaps we'd be better off asking one of EFF's lawyers\nwhat each license means, exactly.\n\n-- \nAndreas Ericsson andreas.ericsson@op5.se\nOP5 AB www.op5.se\nTel: +46 8-230225 Fax: +46 8-230231\n"},{"id":"94545","messageId":"20081101110120.GA3819@artemis.corp","threadId":"16110","inReplyTo":"alpine.DEB.1.00.0811010320370.22125@pacific.mpi-cbg.de.mpi-cbg.de","subject":"Re: libgit2 - a true git library","fromName":"Pierre Habouzit","fromEmail":"madcoder@debian.org","sentAt":"2008-11-01T11:01:20Z","receivedAt":"2008-11-01T11:01:20Z","isPatch":false,"sender":{"key":"madcoder@debian.org","avatar":"https://avatars.githubusercontent.com/u/44708?v=4"},"body":"On Sat, Nov 01, 2008 at 02:26:45AM +0000, Johannes Schindelin wrote:\n> Hi,\n> \n> On Fri, 31 Oct 2008, Shawn O. Pearce wrote:\n> \n> > Nicolas Pitre wrote:\n> > > On Fri, 31 Oct 2008, Shawn O. Pearce wrote:\n> > > \n> > > > > Both the negative code and errno style are lightweight in the \n> > > > > common \"no error\" case. The errno style is probably more handy \n> > > > > for those functions returning a pointer which should be NULL in \n> > > > > the error case.\n> \n> Unfortunately, errno would not be thread-safe, unless you can guarantee \n> that errno is a thread-local variable.\n\nWell, TLS afaict is implemented on arches that have GNU ld, or on win32\nwith a recent enough mingw. Though this is quite a requirement.\n\n> > Oh, good point. We could also stagger the errors so they are\n> > always odd, and never return an odd-alignment pointer from a\n> > successful function. Thus IS_ERR can be written as:\n> > \n> > #define IS_ERR(ptr) (((intptr_t)(ptr)) & 1)\n> > \n> > which is quite cheap, and given the (probably required anyway)\n> > aligned allocation policy means we still have 2^31 possible\n> > error codes available.\n> \n> Oh boy, both solutions are ugly as hell. Although the &1 method does not \n> limit the memory space as much (except if you plan to work in \n> space-contrained environments, where you do not want to be forced to \n> word-align structs).\n> \n> The only pointer game I would remotely consider clean is if you had\n> \n> \tconst char *errors[] = {\n> \t\t...\n> \t};\n> \n> \tinline int is_error(void *ptr) {\n> \t\treturn ptr >= errors && ptr < errors + ARRAY_SIZE(errors);\n> \t}\n\nWell, you can't return _sanely_ an error through a pointer. The &1\nmethod is broken as soon as you return a char* (there is an alignment\nrequirement for malloc, not for any pointer out there), hence shall not\nbe used, as it would not be the sole way to test for error.\n\nAnother option, that is _theorically_ not portable, but is ttbomk on all\nthe platforms we intend to support (IOW POSIX-ish and windows), is to\nuse \"small\" values of the pointers for errors. [NULL .. (void *)(PAGE_SIZE - 1)[\ncannot exist, which gives us probably always 512 different errors, and\nthe test is ((uintptr_t)ptr < (PAGE_SIZE)) which is cheap. It's butt\nugly, but encoding errors into pointers is butt ugly in the first place.\n\n\nAnother option that is what I would prefer, would be for the use of\nerrno where it makes sense. E.g. if you want a function that fetches an\nobject, this is somehow what read(3) would look like on our store, more\nor less, and errno's are enough. For the other functions where errno\ncannot be used, I'm pretty sure we will always pass a handle to some\nkind of libgit2 stuff, like the \"repository\" we're working on. The\n_easiest_ way is to put the \"last error\" into that structure and use\nthat. I mean, if we want libgit2 to be useful for _everyone_ we *WILL*\nhave to pass a repository context around. I see almost no way around it.\nAnd there, NULL means error, and if you want to know about the specific\nerror, git_repo_errno(&ctx) / git_repo_strerr(&ctx) is just easy.\n\nNote: What is important is to be able to check for errors _fast_, I\ndon't think printing out the error and knowing which error it was would\nbe in the fast path, so it's less useful to have this information\nimmediately.\n\n\n_My_ taste (but again, like the _t I would use what is there, and won't\nmake a fuss about it at all) is to see function return -1 or NULL for\nerrors, and \"abuse\" errno for system-like functions, or put the last\nerror into the context on which you're working (your \"this\" more or\nless). We don't need a specific error context at all, because we already\nhave a repository context available.\n\nI'm not really a fan of pointer semantics abuse (it's sometimes useful,\nbut as the public interface of a library, this is butt-ugly.\n\n-- \n·O· Pierre Habouzit\n··O madcoder@debian.org\nOOO http://www.madism.org\n"},{"id":"94549","messageId":"490C3AC2.4080002@op5.se","threadId":"16110","inReplyTo":"20081031220133.GA14786@spearce.org","subject":"Re: libgit2 - a true git library","fromName":"Andreas Ericsson","fromEmail":"ae@op5.se","sentAt":"2008-11-01T11:17:22Z","receivedAt":"2008-11-01T11:17:22Z","isPatch":false,"sender":{"key":"ae@op5.se","avatar":"https://gravatar.com/avatar/426e89595c75a8f5252dd0c989e5fabe5bcac616e68557427ad9aef6b0ca342a?d=mp&s=160"},"body":"Shawn O. Pearce wrote:\n> Andreas Ericsson wrote:\n>>> * proper public \"stuff\" naming (I e.g. realy like types names -- not\n>>> struct or enum tags, that I don't really care -- ending with _t as\n>>> it helps navigating source.\n>> *_t types are reserved by POSIX for future implementations, so that's\n>> a no-go (although I doubt POSIX will ever make types named git_*_t).\n> \n> Yikes. Anyone know where a concise list of the reserved names are?\n> \n\nThe ones I know of off the top of my head are:\n* Anything ending with _t (posix)\n pthread_t, size_t, socklen_t, ...\n* str/mem/? function name prefix NOT followed by an underscore (C lib)\n strlen, strchr, memcmp, memcpy, ...\n (strbuf_ stuff doesn't break this, as it contains an underscore in the\n name - the underscore following it doesn't have to be immediate).\n* Double underscore prefix (compiler / C lib)\n __WORDSIZE, __cplusplus, ...\n\n\nIt's also a good idea to stay away from single underscore prefix for\ngeneric-ish names, as some compilers/architectures abuse it extensively.\n\nPrefixing all public functions and macros of the library with 'git_'\n(lower-case for functions and function-like macros, upper-case for\nthe rest) will probably see us through safe. Exceptions can probably\nbe made for already completed API's, such as the arg-parsing stuff\nand the strbuf code.\n\n>> Apart from that, please consider reading Ulrich Drepper's musings on\n>> library design at http://people.redhat.com/drepper/goodpractice.pdf\n> \n> I think I've read that before, but I'll skim over it again.\n> Thanks for the link.\n> \n\nI swear by it at work, where I'm \"the library guy\". I'll make sure to\nreview stuff and can chip in code or thoughts to make the library fly\nwith a minimum amount of problems and maintenance burden.\n\n-- \nAndreas Ericsson andreas.ericsson@op5.se\nOP5 AB www.op5.se\nTel: +46 8-230225 Fax: +46 8-230231\n"},{"id":"94559","messageId":"alpine.LFD.2.00.0811010944000.13034@xanadu.home","threadId":"16110","inReplyTo":"20081101110120.GA3819@artemis.corp","subject":"Re: libgit2 - a true git library","fromName":"Nicolas Pitre","fromEmail":"nico@cam.org","sentAt":"2008-11-01T13:50:45Z","receivedAt":"2008-11-01T13:50:45Z","isPatch":false,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Sat, 1 Nov 2008, Pierre Habouzit wrote:\n\n> Well, you can't return _sanely_ an error through a pointer. The &1\n> method is broken as soon as you return a char* (there is an alignment\n> requirement for malloc, not for any pointer out there), hence shall not\n> be used, as it would not be the sole way to test for error.\n> \n> Another option, that is _theorically_ not portable, but is ttbomk on all\n> the platforms we intend to support (IOW POSIX-ish and windows), is to\n> use \"small\" values of the pointers for errors. [NULL .. (void *)(PAGE_SIZE - 1)[\n> cannot exist, which gives us probably always 512 different errors, and\n\n4095 actually. You don't need to align error codes.\n\n> the test is ((uintptr_t)ptr < (PAGE_SIZE)) which is cheap. It's butt\n> ugly, but encoding errors into pointers is butt ugly in the first place.\n\nOr use \"negative\" pointers. Again, please have a look at \ninclude/linux/err.h. The pointer range from 0xffffffff (or \n0xffffffffffffffff on 64-bit machines) down to the range you want is for \nerrors, and the top of the address range is almost certain to never be \nvalid in user space either.\n\n\nNicolas\n"},{"id":"94563","messageId":"20081101170158.GB26229@artemis.corp","threadId":"16110","inReplyTo":"alpine.LFD.2.00.0811010944000.13034@xanadu.home","subject":"Re: libgit2 - a true git library","fromName":"Pierre Habouzit","fromEmail":"madcoder@debian.org","sentAt":"2008-11-01T17:01:58Z","receivedAt":"2008-11-01T17:01:58Z","isPatch":false,"sender":{"key":"madcoder@debian.org","avatar":"https://avatars.githubusercontent.com/u/44708?v=4"},"body":"On Sat, Nov 01, 2008 at 01:50:45PM +0000, Nicolas Pitre wrote:\n> On Sat, 1 Nov 2008, Pierre Habouzit wrote:\n> \n> > Well, you can't return _sanely_ an error through a pointer. The &1\n> > method is broken as soon as you return a char* (there is an alignment\n> > requirement for malloc, not for any pointer out there), hence shall not\n> > be used, as it would not be the sole way to test for error.\n> > \n> > Another option, that is _theorically_ not portable, but is ttbomk on all\n> > the platforms we intend to support (IOW POSIX-ish and windows), is to\n> > use \"small\" values of the pointers for errors. [NULL .. (void *)(PAGE_SIZE - 1)[\n> > cannot exist, which gives us probably always 512 different errors, and\n> \n> 4095 actually. You don't need to align error codes.\n\nSure, I'm just not sure there isn't an arch where a page would be 512\noctets. And you really have 4096 errors, from 0 to 4095 *included* :)\n\n> > the test is ((uintptr_t)ptr < (PAGE_SIZE)) which is cheap. It's butt\n> > ugly, but encoding errors into pointers is butt ugly in the first place.\n> \n> Or use \"negative\" pointers. Again, please have a look at \n> include/linux/err.h. The pointer range from 0xffffffff (or \n> 0xffffffffffffffff on 64-bit machines) down to the range you want is for \n> errors, and the top of the address range is almost certain to never be \n> valid in user space either.\n\nIndeed.\n\n-- \n·O· Pierre Habouzit\n··O madcoder@debian.org\nOOO http://www.madism.org\n"},{"id":"94566","messageId":"20081101173042.GE26229@artemis.corp","threadId":"16110","inReplyTo":"20081031184154.GV14786@spearce.org","subject":"Re: libgit2 - a true git library","fromName":"Pierre Habouzit","fromEmail":"madcoder@debian.org","sentAt":"2008-11-01T17:30:42Z","receivedAt":"2008-11-01T17:30:42Z","isPatch":false,"sender":{"key":"madcoder@debian.org","avatar":"https://avatars.githubusercontent.com/u/44708?v=4"},"body":"On Fri, Oct 31, 2008 at 06:41:54PM +0000, Shawn O. Pearce wrote:\n> How about this?\n> \n> http://www.spearce.org/projects/scm/libgit2/apidocs/CONVENTIONS\n\nFWIW I've read what you say about types, while this is good design to\nmake things abstract, accessors are slower _and_ disallow many\noptimizations as it's a function call and that it may clobber all your\npointers values.\n\nFor types that _will_ be in the tight loops, we must make the types\nexplicit or it'll bite us hard performance-wise. I'm thinking what is\n\"struct object\" or \"struct commit\" in git.git. It's likely that we will\nloose a *lot* of those types are opaque.\n\nstruct object in git has not changed since 2006.06. struct commit hasn't\nsince 2005.04 if you ignore { unsigned int indegree; void *util; } that\nif I'm correct are annotations, and is a problem we (I think) have to\naddress differently anyways (I gave my proposal on this, I'm eager to\nhear about what other think on the subject). So if in git.git that _is_\na moving target we have had a 2 year old implementation for those types,\nit's that they're pretty well like this.\n\nIt's IMNSHO on the matter that core structures of git _will_ have to be\nmade explicit. I'm thinking objects and their \"subtypes\" (commits,\ntrees, blobs). Maybe a couple of things on the same vein.\n\n-- \n·O· Pierre Habouzit\n··O madcoder@debian.org\nOOO http://www.madism.org\n"},{"id":"94574","messageId":"490CA37C.1070107@op5.se","threadId":"16110","inReplyTo":"20081101173042.GE26229@artemis.corp","subject":"Re: libgit2 - a true git library","fromName":"Andreas Ericsson","fromEmail":"ae@op5.se","sentAt":"2008-11-01T18:44:12Z","receivedAt":"2008-11-01T18:44:12Z","isPatch":false,"sender":{"key":"ae@op5.se","avatar":"https://gravatar.com/avatar/426e89595c75a8f5252dd0c989e5fabe5bcac616e68557427ad9aef6b0ca342a?d=mp&s=160"},"body":"Pierre Habouzit wrote:\n> On Fri, Oct 31, 2008 at 06:41:54PM +0000, Shawn O. Pearce wrote:\n>> How about this?\n>>\n>> http://www.spearce.org/projects/scm/libgit2/apidocs/CONVENTIONS\n> \n> FWIW I've read what you say about types, while this is good design to\n> make things abstract, accessors are slower _and_ disallow many\n> optimizations as it's a function call and that it may clobber all your\n> pointers values.\n> \n\nAccessors are very nifty for one thing though; With a debugging flag,\nyou can use an accessor-function, while without that debugging flag you\ncan use a macro instead of a function. In other words, you use the\ncompiler as a sort of sanity-checker that you're only accessing the\nvariables through the proper macros.\n\nThis method introduces a bit of extra code (50% of which is always\ndead) for each struct it's used on, but it makes debugging large-ish\npieces of software relatively simple, since access to all object types\nis controlled through the use of macros.\n\n> For types that _will_ be in the tight loops, we must make the types\n> explicit or it'll bite us hard performance-wise. I'm thinking what is\n> \"struct object\" or \"struct commit\" in git.git. It's likely that we will\n> loose a *lot* of those types are opaque.\n> \n\nThe last sentence doesn't parse. I assume you mean \"if those types are..\",\nin which case it'll be solved by using accessor-macros and forward-declaring\nthe structs.\n\n> struct object in git has not changed since 2006.06. struct commit hasn't\n> since 2005.04 if you ignore { unsigned int indegree; void *util; } that\n> if I'm correct are annotations, and is a problem we (I think) have to\n> address differently anyways (I gave my proposal on this, I'm eager to\n> hear about what other think on the subject). So if in git.git that _is_\n> a moving target we have had a 2 year old implementation for those types,\n> it's that they're pretty well like this.\n> \n> It's IMNSHO on the matter that core structures of git _will_ have to be\n> made explicit. I'm thinking objects and their \"subtypes\" (commits,\n> trees, blobs). Maybe a couple of things on the same vein.\n> \n\nI agree. \"git_commit\", \"git_tree\", \"git_blob\" and \"git_tag\" can almost\ncertainly be set in stone straight away.\n\n-- \nAndreas Ericsson andreas.ericsson@op5.se\nOP5 AB www.op5.se\nTel: +46 8-230225 Fax: +46 8-230231\n"},{"id":"94576","messageId":"20081101184850.GH26229@artemis.corp","threadId":"16110","inReplyTo":"490CA37C.1070107@op5.se","subject":"Re: libgit2 - a true git library","fromName":"Pierre Habouzit","fromEmail":"madcoder@debian.org","sentAt":"2008-11-01T18:48:50Z","receivedAt":"2008-11-01T18:48:50Z","isPatch":false,"sender":{"key":"madcoder@debian.org","avatar":"https://avatars.githubusercontent.com/u/44708?v=4"},"body":"On Sat, Nov 01, 2008 at 06:44:12PM +0000, Andreas Ericsson wrote:\n> Pierre Habouzit wrote:\n\n> >For types that _will_ be in the tight loops, we must make the types\n> >explicit or it'll bite us hard performance-wise. I'm thinking what is\n> >\"struct object\" or \"struct commit\" in git.git. It's likely that we will\n> >loose a *lot* of those types are opaque.\n> \n> The last sentence doesn't parse. I assume you mean \"if those types \n> are..\",\n\nThis was a typo, indeed s/of/if/\n\n-- \n·O· Pierre Habouzit\n··O madcoder@debian.org\nOOO http://www.madism.org\n"},{"id":"94577","messageId":"490CAB6D.90209@op5.se","threadId":"16110","inReplyTo":"20081031170704.GU14786@spearce.org","subject":"Re: libgit2 - a true git library","fromName":"Andreas Ericsson","fromEmail":"ae@op5.se","sentAt":"2008-11-01T19:18:05Z","receivedAt":"2008-11-01T19:18:05Z","isPatch":false,"sender":{"key":"ae@op5.se","avatar":"https://gravatar.com/avatar/426e89595c75a8f5252dd0c989e5fabe5bcac616e68557427ad9aef6b0ca342a?d=mp&s=160"},"body":"Shawn O. Pearce wrote:\n> During the GitTogether we were kicking around the idea of a ground-up\n> implementation of a Git library. This may be easier than trying\n> to grind down git.git into a library, as we aren't tied to any\n> of the current global state baggage or the current die() based\n> error handling.\n> \n> I've started an _extremely_ rough draft. The code compiles into a\n> libgit.a but it doesn't even implement what it describes in the API,\n> let alone a working Git implementation. Really what I'm trying to\n> incite here is some discussion on what the API looks like.\n> \n> API Docs:\n> http://www.spearce.org/projects/scm/libgit2/apidocs/html/modules.html\n> \n> Source Code Clone URL:\n> http://www.spearce.org/projects/scm/libgit2/libgit2.git\n> \n\nHaving looked briefly at the code, I've got a couple of comments:\n* GIT_EXTERN() does nothing. Ever. It's noise and should be removed.\n Instead it would be better to have GIT_PRIVATE(), which could\n set visibility to \"internal\" or \"hidden\", meaning the symbol it's\n attached to can be used for lookups when creating a shared library\n but won't be usable from programs linking to that shared library\n (visibility-attributes have zero effect on static libraries). At\n least on all archs anyone really cares about.\n* Prefixing the files themselves with git_ is useless and only leads\n to developer frustration. I imagine we'd be installing any header\n files in a git/ directory anyway, so we're gaining absolutely\n nothing with the git_ prefix on source-files.\n\nApart from that, it seems you've been designing a lot rather than\ntrying to use the API to actually do something. It would, imo, be\na lot better to start development with adding functionality shared\nbetween all programs and then expand further on that, such as\nincorporating all functions needed for manipulating tags into the\nlibrary and then modify existing code to use the library to get\ntag-ish things done. That would also mean that the library would\nquickly get used by core git, as once a certain part of it is\ncomplete patches can be fitted to the library rather than to the\ncurrent non-libish dying() functions.\n\n\nI also think it's quite alright to not strive *too* hard to make\nall functions thread-safe, as very few of them will actually need\nthat. It's unlikely that a user program will spawn one thread to\nwrite a lot of tags while another is trying to parse them, for\nexample.\n\nBy adding an init routine that determines the workdir and the\ngitdir, one could start using the library straight away.\n\nint git_init(const char *db, const char *worktree)\n{\n if (git_set_db_dir(db))\n return -1;\n git_set_worktree((worktree))\n return -1;\n\n return 0;\n}\n\nand already you have a some few small helpers that are nifty to\nto have around:\nint git_is_gitdir(const char *path); /* returns 1 on success */\nint git_has_gitdir(const char *path); /* returns 1 on success */\nconst char *git_mkpath(const char *fmt, ...)\n\nThis way one will notice rather quickly what's needed (making it\neasy to keep a more-or-less public TODO available, with small stuff\non it for the most part), and one can then go look for it in the\nexisting git code and, if possible, convert stuff or, best case\nscenario, steal it straight off so that more apps can benefit from\ntried and tested code.\n\n-- \nAndreas Ericsson andreas.ericsson@op5.se\nOP5 AB www.op5.se\nTel: +46 8-230225 Fax: +46 8-230231\n"},{"id":"94586","messageId":"alpine.DEB.1.00.0811012124250.22125@pacific.mpi-cbg.de.mpi-cbg.de","threadId":"16110","inReplyTo":"alpine.LFD.2.00.0811010944000.13034@xanadu.home","subject":"Re: libgit2 - a true git library","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-11-01T20:26:01Z","receivedAt":"2008-11-01T20:26:01Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Sat, 1 Nov 2008, Nicolas Pitre wrote:\n\n> Again, please have a look at include/linux/err.h. The pointer range \n> from 0xffffffff (or 0xffffffffffffffff on 64-bit machines) down to the \n> range you want is for errors, and the top of the address range is almost \n> certain to never be valid in user space either.\n\nHow certain may I be of that assumption? Because an assumption it is.\n\nCiao,\nDscho\n"},{"id":"94589","messageId":"20081101202922.GB15463@spearce.org","threadId":"16110","inReplyTo":"490CA37C.1070107@op5.se","subject":"Re: libgit2 - a true git library","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2008-11-01T20:29:22Z","receivedAt":"2008-11-01T20:29:22Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Andreas Ericsson wrote:\n> Pierre Habouzit wrote:\n>> On Fri, Oct 31, 2008 at 06:41:54PM +0000, Shawn O. Pearce wrote:\n>>> How about this?\n>>>\n>>> http://www.spearce.org/projects/scm/libgit2/apidocs/CONVENTIONS\n>>\n>> FWIW I've read what you say about types, while this is good design to\n>> make things abstract, accessors are slower _and_ disallow many\n>> optimizations as it's a function call and that it may clobber all your\n>> pointers values.\n\nTrue, accessors slow things down. But I'm not sure that the\naccessors at the application level are going to be a huge problem.\n\nWhere the CPU time really matters is inside the tight loops of the\nlibrary, where we can expose the struct to ourselves, because if\nthe layout changes we'd be relinking the library anyway with the\nupdated object code.\n\nI would rather stick with accessors right now. We could in the\nfuture expose the structs and convert the accessors to macros or\ninline functions in a future version of the ABI if performance is\nreally shown to be a problem here from the *application*.\n\nRemember we are mostly talking about applications that are happy to\nfork+exec git right now. A little accessor function call is *still*\nfaster than that fork call was.\n\n>> struct object in git has not changed since 2006.06. struct commit hasn't\n>> since 2005.04 if you ignore { unsigned int indegree; void *util; } that\n>> if I'm correct are annotations, and is a problem we (I think) have to\n>> address differently anyways (I gave my proposal on this, I'm eager to\n>> hear about what other think on the subject). So if in git.git that _is_\n>> a moving target we have had a 2 year old implementation for those types,\n>> it's that they're pretty well like this.\n>>\n>> It's IMNSHO on the matter that core structures of git _will_ have to be\n>> made explicit. I'm thinking objects and their \"subtypes\" (commits,\n>> trees, blobs). Maybe a couple of things on the same vein.\n>\n> I agree. \"git_commit\", \"git_tree\", \"git_blob\" and \"git_tag\" can almost\n> certainly be set in stone straight away.\n\nEh, I disagree here. In git.git today \"struct commit\" exposes its\nbuffer with the canonical commit encoding. Having that visible\nwrecks what Nico and I were thinking about doing with pack v4 and\nencoding commits in a non-canonical format when stored in packs.\nDitto with trees.\n\nBecause git.git code goes against that canonical buffer we cannot\neasily insert pack v4 and test the improvements we want to make.\nThe refactoring required is one of the reasons we haven't done pack\nv4 yet. _IF_ we really are going through this effort of building\na different API and shifting to its use in git.git I want to make\nsure we at least initially leave the door open to make changes\nwithout rewriting everything *again*.\n\nAccessor functions can usually be inlined or macro'd away. But\nthey cannot be magically inserted by the compiler if they aren't\nthere in the first place. This isn't Python... :-)\n\n-- \nShawn.\n"},{"id":"94592","messageId":"20081101204259.GC15463@spearce.org","threadId":"16110","inReplyTo":"490CAB6D.90209@op5.se","subject":"Re: libgit2 - a true git library","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2008-11-01T20:42:59Z","receivedAt":"2008-11-01T20:42:59Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Andreas Ericsson wrote:\n> Shawn O. Pearce wrote:\n>> During the GitTogether we were kicking around the idea of a ground-up\n>> implementation of a Git library.\n>\n> Having looked briefly at the code, I've got a couple of comments:\n> * GIT_EXTERN() does nothing. Ever. It's noise and should be removed.\n\nI feel the same way.\n\nBut I was also under the impression that the brilliant engineers\nwho work for Microsoft decided that on their platform special\nannotations have to be inserted on functions that a DLL wants to\nexport to applications.\n\nHence any cross-platform library that I have seen annotates their\nexported functions this way, with the macro being empty on POSIX\nand expanding to some magic keyword on Microsoft's OS. I think it\ngoes between the return type and the function name too...\n\n> Instead it would be better to have GIT_PRIVATE(),\n\nI can see why you said this; needing GIT_PRIVATE() is a lot more\nrare than needing GIT_EXTERN(). Only a handful of cross-module,\nbut private, functions are likely to exist, so it makes sense to\nmark the smaller subset. But see above. *sigh*\n\n> * Prefixing the files themselves with git_ is useless and only leads\n> to developer frustration. I imagine we'd be installing any header\n> files in a git/ directory anyway, so we're gaining absolutely\n> nothing with the git_ prefix on source-files.\n\nYes, I realized that this morning. I plan on changing that mess\naround so we have \"include/git/oid.h\" and library and application\ncode can use \"#include \". Library modules should just\nbe \"src/oid.c\" then.\n\n> Apart from that, it seems you've been designing a lot rather than\n> trying to use the API to actually do something.\n\nI wanted to get a solid idea of what our API conventions should be,\nbefore we started writing a lot of code around them. Part of the\nproblem with the git.git code is we don't have conventions that are\nreally suited for use in a shared library (assuming we even have\nconventions in there) so we can't use that code as a library today.\n\n> It would, imo, be\n> a lot better to start development with adding functionality shared\n> between all programs and then expand further on that, such as\n> incorporating all functions needed for manipulating tags into the\n> library and then modify existing code to use the library to get\n> tag-ish things done.\n\nTags are mostly pointless. Its a tiny part of the code that isn't\nthat interesting to most people. And it requires object database\naccess anyway if you want to talk about parsing or reading a tag.\nThere's almost no point in a git library that can't read the on\ndisk object database, or write to it.\n\n> I also think it's quite alright to not strive *too* hard to make\n> all functions thread-safe, as very few of them will actually need\n> that. It's unlikely that a user program will spawn one thread to\n> write a lot of tags while another is trying to parse them, for\n> example.\n\nOh really?\n\nMaybe true for tags, just because they are such an unimportant part\nof the git suite compared to everything else.\n\nBut right now I'm running a production system using a threaded server\nprocess that is operating on Git repositories. Fortunately threads\nsuck less on Java than they do on POSIX, and we have a 100% pure\nJava library available for Git.\n\nIt would be nice if a library created in the late part of 2008\nrecognized that threads exist, aren't going to disappear tomorrow,\nand that consumers of libraries actually may need to run the library\nwithin a threaded process.\n\nOr are you one of those developers who think threads only exist\nin the giant monolithic kernel land, and all user space should\nbe isolated process? I often wonder who such people can justify\nthe kernel address space being multi-threaded but userland being\nstuck to single threaded applications. Oh, right, the kernel has\nto go fast...\n\n-- \nShawn.\n"},{"id":"94594","messageId":"490CD101.1030604@op5.se","threadId":"16110","inReplyTo":"20081101202922.GB15463@spearce.org","subject":"Re: libgit2 - a true git library","fromName":"Andreas Ericsson","fromEmail":"ae@op5.se","sentAt":"2008-11-01T21:58:25Z","receivedAt":"2008-11-01T21:58:25Z","isPatch":false,"sender":{"key":"ae@op5.se","avatar":"https://gravatar.com/avatar/426e89595c75a8f5252dd0c989e5fabe5bcac616e68557427ad9aef6b0ca342a?d=mp&s=160"},"body":"Shawn O. Pearce wrote:\n> Andreas Ericsson wrote:\n>> Pierre Habouzit wrote:\n>>> On Fri, Oct 31, 2008 at 06:41:54PM +0000, Shawn O. Pearce wrote:\n>>>> How about this?\n>>>>\n>>>> http://www.spearce.org/projects/scm/libgit2/apidocs/CONVENTIONS\n>>> FWIW I've read what you say about types, while this is good design to\n>>> make things abstract, accessors are slower _and_ disallow many\n>>> optimizations as it's a function call and that it may clobber all your\n>>> pointers values.\n> \n> True, accessors slow things down. But I'm not sure that the\n> accessors at the application level are going to be a huge problem.\n> \n> Where the CPU time really matters is inside the tight loops of the\n> library, where we can expose the struct to ourselves, because if\n> the layout changes we'd be relinking the library anyway with the\n> updated object code.\n> \n> I would rather stick with accessors right now. We could in the\n> future expose the structs and convert the accessors to macros or\n> inline functions in a future version of the ABI if performance is\n> really shown to be a problem here from the *application*.\n> \n> Remember we are mostly talking about applications that are happy to\n> fork+exec git right now. A little accessor function call is *still*\n> faster than that fork call was.\n> \n>>> struct object in git has not changed since 2006.06. struct commit hasn't\n>>> since 2005.04 if you ignore { unsigned int indegree; void *util; } that\n>>> if I'm correct are annotations, and is a problem we (I think) have to\n>>> address differently anyways (I gave my proposal on this, I'm eager to\n>>> hear about what other think on the subject). So if in git.git that _is_\n>>> a moving target we have had a 2 year old implementation for those types,\n>>> it's that they're pretty well like this.\n>>>\n>>> It's IMNSHO on the matter that core structures of git _will_ have to be\n>>> made explicit. I'm thinking objects and their \"subtypes\" (commits,\n>>> trees, blobs). Maybe a couple of things on the same vein.\n>> I agree. \"git_commit\", \"git_tree\", \"git_blob\" and \"git_tag\" can almost\n>> certainly be set in stone straight away.\n> \n> Eh, I disagree here. In git.git today \"struct commit\" exposes its\n> buffer with the canonical commit encoding. Having that visible\n> wrecks what Nico and I were thinking about doing with pack v4 and\n> encoding commits in a non-canonical format when stored in packs.\n> Ditto with trees.\n> \n\nErr... isn't that backwards? Surely you want to store stuff in the\ncanonical format so you're forced to do as few translations as\npossible? Or are you trying to speed up packing by skipping the\ncanonicalization part? If so, that would slow down reading (or\nrather, presenting) the commits, wouldn't it?\n\n> Because git.git code goes against that canonical buffer we cannot\n> easily insert pack v4 and test the improvements we want to make.\n> The refactoring required is one of the reasons we haven't done pack\n> v4 yet. _IF_ we really are going through this effort of building\n> a different API and shifting to its use in git.git I want to make\n> sure we at least initially leave the door open to make changes\n> without rewriting everything *again*.\n> \n\nWell, if macro usage is adhered to one wouldn't have to worry,\nsince the macro can just be rewritten with a function later (if,\nfor example, translation or some such happens to be required).\nOlder code linking to a newer library would work (assuming the\nsize of the commit object doesn't change anyway), but newer code\nlinking to an older library would not. Otoh, they wouldn't even\nbuild unless they used the wrong header files, so there is nothing\nto worry about there.\n\n> Accessor functions can usually be inlined or macro'd away. But\n> they cannot be magically inserted by the compiler if they aren't\n> there in the first place. This isn't Python... :-)\n> \n\nWhat I meant was this (I'm a tad drunk, so read the spirit, not\nthe letter):\n\nin \"foo-api.h\":\n--%<--%<--\n#ifdef BUILDING_FOR_DEPLOYING\n#include \"git_foo_decls.h\"\n# define git_foo_get_buf(git_foo) (git_foo->buf)\n#else\n#include \"git_foo_fwd_decls.h\"\nextern const char *git_foo_get_buf(git_foo *foo);\n#endif\n--%<--%<--\n\nfoo.c\n--%<--%<--\n#include \"foo-api.h\"\n#include \"git_foo_decls.h\"\n#ifndef BUILDING_FOR_DEPLOYING\nconst char git_foo_get_buf(git_foo *foo)\n{\n\treturn foo->buf;\n}\n\n/* other accessors go here */\n#endif\n\n/* rest of git_foo manipulators go here */\n--%<--%<--\n\nIt's almost certainly not worth it for libgit2 though, as git@vger\nprovides a good review system.\n\n-- \nAndreas Ericsson andreas.ericsson@op5.se\nOP5 AB www.op5.se\nTel: +46 8-230225 Fax: +46 8-230231\n"},{"id":"94602","messageId":"20081101225714.GD15463@spearce.org","threadId":"16110","inReplyTo":"alpine.LFD.2.00.0810312150200.13034@xanadu.home","subject":"Re: libgit2 - a true git library","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2008-11-01T22:57:14Z","receivedAt":"2008-11-01T22:57:14Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Nicolas Pitre wrote:\n> On Fri, 31 Oct 2008, Shawn O. Pearce wrote:\n> \n> > My take on the consensus for the license part of the discussion is\n> > that libgit2 should be under the \"GPL gcc library\" license.\n> > \n> > BTW, I can't actually find a copy of that license; the only thing\n> > I can locate in the GCC SVN tree is a copy of the LGPL.\n> \n> The exception is usually found at the top of files constituting \n> libgcc.a. One example is gcc/config/arm/ieee754-df.S. ;-)\n\nHeaders updated. Its now GPL+gcc library exception.\n\nNot that the 5 lines of useful code there really needs copyright,\nbut hey, whatever.\n\n-- \nShawn.\n"},{"id":"94606","messageId":"d411cc4a0811011726h1fb1ad0ct5c37af753940f4a4@mail.gmail.com","threadId":"16110","inReplyTo":"20081101225714.GD15463@spearce.org","subject":"Re: libgit2 - a true git library","fromName":"Scott Chacon","fromEmail":"schacon@gmail.com","sentAt":"2008-11-02T00:26:28Z","receivedAt":"2008-11-02T00:26:28Z","isPatch":false,"sender":{"key":"schacon@gmail.com","avatar":"https://gravatar.com/avatar/9b13a8a078e1dcf8588c4eea9554445d51ebed6c41b51f56f4d96738130b05c6?d=mp&s=160"},"body":"I'm sorry - why is that better than LGPL? Wouldn't it be better to\nuse a license that people have heard of rather than one that can't be\nlooked up or it's implications easily researched? What is this\naffording the library that offsets the headaches of everyone trying to\nfigure out if they can use it or not?\n\nScott\n\nOn Sat, Nov 1, 2008 at 3:57 PM, Shawn O. Pearce wrote:\n> Nicolas Pitre wrote:\n>> On Fri, 31 Oct 2008, Shawn O. Pearce wrote:\n>>\n>> > My take on the consensus for the license part of the discussion is\n>> > that libgit2 should be under the \"GPL gcc library\" license.\n>> >\n>> > BTW, I can't actually find a copy of that license; the only thing\n>> > I can locate in the GCC SVN tree is a copy of the LGPL.\n>>\n>> The exception is usually found at the top of files constituting\n>> libgcc.a. One example is gcc/config/arm/ieee754-df.S. ;-)\n>\n> Headers updated. Its now GPL+gcc library exception.\n>\n> Not that the 5 lines of useful code there really needs copyright,\n> but hey, whatever.\n>\n> --\n> Shawn.\n> --\n> To unsubscribe from this list: send the line \"unsubscribe git\" in\n> the body of a message to majordomo@vger.kernel.org\n> More majordomo info at http://vger.kernel.org/majordomo-info.html\n>\n"},{"id":"94609","messageId":"d411cc4a0811011807g229f8becs9f411d6e19fb6c12@mail.gmail.com","threadId":"16110","inReplyTo":"d411cc4a0811011726h1fb1ad0ct5c37af753940f4a4@mail.gmail.com","subject":"Re: libgit2 - a true git library","fromName":"Scott Chacon","fromEmail":"schacon@gmail.com","sentAt":"2008-11-02T01:07:04Z","receivedAt":"2008-11-02T01:07:04Z","isPatch":false,"sender":{"key":"schacon@gmail.com","avatar":"https://gravatar.com/avatar/9b13a8a078e1dcf8588c4eea9554445d51ebed6c41b51f56f4d96738130b05c6?d=mp&s=160"},"body":"> On Sat, Nov 1, 2008 at 3:57 PM, Shawn O. Pearce wrote:\n>> Nicolas Pitre wrote:\n>>> On Fri, 31 Oct 2008, Shawn O. Pearce wrote:\n>>>\n>>> > My take on the consensus for the license part of the discussion is\n>>> > that libgit2 should be under the \"GPL gcc library\" license.\n>>> >\n>>> > BTW, I can't actually find a copy of that license; the only thing\n>>> > I can locate in the GCC SVN tree is a copy of the LGPL.\n>>>\n>>> The exception is usually found at the top of files constituting\n>>> libgcc.a. One example is gcc/config/arm/ieee754-df.S. ;-)\n>>\n>> Headers updated. Its now GPL+gcc library exception.\n>>\n>> Not that the 5 lines of useful code there really needs copyright,\n>> but hey, whatever.\n\nI guess my main concern is that if a company wanted to direct\nresources at supporting Git in something (say, an editor or GUI or\nwhatnot), and that company is of _any_ size, they are going to have to\nget their legal department to review this strange and almost totally\nunused license - only knowing that it's barely different than GPL and\nthey know GPL will not fly. LGPL will likely be known to them and a\npolicy may already be in place.\n\nThink about trying to incorporate this into something proprietary,\nShawn - how much of a pain is it going to be to get that license\nreviewed in Google? However, LGPL I'm sure there is already a\nreviewed policy. Now, since that may be a pain, time that Shawn could\nhave been spending being paid to work on the library is lost because\nthey can't use it, or it takes weeks/months to review it. That's my\nconcern.\n\nI personally would rather see it BSD or something more permissive so\nthat no human has to waste even a second of their valuable time\nfiguring out if they can work with it or not, but I understand that\nmany people here are much more protective of their code. I simply\nthink that LGPL is a much more widely used and understood compromise\nthat affords nearly the same protectionism.\n\nScott\n\n>>\n>> --\n>> Shawn.\n>> --\n>> To unsubscribe from this list: send the line \"unsubscribe git\" in\n>> the body of a message to majordomo@vger.kernel.org\n>> More majordomo info at http://vger.kernel.org/majordomo-info.html\n>>\n>\n"},{"id":"94613","messageId":"20081102013636.GF15463@spearce.org","threadId":"16110","inReplyTo":"d411cc4a0811011807g229f8becs9f411d6e19fb6c12@mail.gmail.com","subject":"Re: libgit2 - a true git library","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2008-11-02T01:36:36Z","receivedAt":"2008-11-02T01:36:36Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Scott Chacon wrote:\n> > On Sat, Nov 1, 2008 at 3:57 PM, Shawn O. Pearce wrote:\n> >>\n> >> Headers updated. Its now GPL+gcc library exception.\n> \n> I personally would rather see it BSD or something more permissive so\n> that no human has to waste even a second of their valuable time\n> figuring out if they can work with it or not, but I understand that\n> many people here are much more protective of their code. I simply\n> think that LGPL is a much more widely used and understood compromise\n> that affords nearly the same protectionism.\n\nApparently BSD won't fly, as you have already seen on the list.\n\nIf we did put the library under a BSD license we'd lose some core\ncontributors. Or they at least wouldn't improve the library code,\neven if git.git linked to it in the future. I don't want to lose\nthese folks.\n\nIANAL, but from what I can tell the main difference between LGPL\nand GPL+\"gcc library exception\" is that the LGPL requires that\nthe end-user must be able to relink the derived executable with\ntheir own replacement library. The GPL+\"gcc library exception\"\nmakes no such requirement.\n\nIf you read the exception clause it practically makes the library\neven easier to use commerically than the BSD license does, however\nmodifications to the library sources must still be distributed.\n\nIsn't that actually somewhat close to the Mozilla Public License?\n\n-- \nShawn.\n"},{"id":"94614","messageId":"20081102015041.GG15463@spearce.org","threadId":"16110","inReplyTo":"490CD101.1030604@op5.se","subject":"Re: libgit2 - a true git library","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2008-11-02T01:50:41Z","receivedAt":"2008-11-02T01:50:41Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Andreas Ericsson wrote:\n> Shawn O. Pearce wrote:\n>>\n>> Eh, I disagree here. In git.git today \"struct commit\" exposes its\n>> buffer with the canonical commit encoding. Having that visible\n>> wrecks what Nico and I were thinking about doing with pack v4 and\n>> encoding commits in a non-canonical format when stored in packs.\n>> Ditto with trees.\n>\n> Err... isn't that backwards?\n\nNo.\n\n> Surely you want to store stuff in the\n> canonical format so you're forced to do as few translations as\n> possible?\n\nNo. We suspect that canonical format is harder to decompress and\nparse during revision traversal. Other encodings in the pack file\nmay produce much faster runtime performance, and reduce page faults\n(due to smaller pack sizes).\n\nWe hardly ever use the canonical format for actual output; most\noutput rips the canonical format apart and then formats the data\nthe way it was requested. If we have the data *already* parsed in\nthe pack its much faster to output.\n\n> Or are you trying to speed up packing by skipping the\n> canonicalization part?\n\nWrong; we're trying to speed up reading. Packing may go slower,\nespecially during the first conversion of v2->v4 for any given\nrepository, but packing is infrequent so the minor (if any) drop\nin performance here is probably worth the reading performance gains.\n\n> Well, if macro usage is adhered to one wouldn't have to worry,\n> since the macro can just be rewritten with a function later (if,\n> for example, translation or some such happens to be required).\n> Older code linking to a newer library would work (assuming the\n> size of the commit object doesn't change anyway),\n\nYou are assuming too much magic. If the older ABI used a macro\nand the newer one (which supports pack v4) organized struct commit\ndifferently and the user upgrades libgit2.so the older applications\njust broke, horribly.\n\nWe know we want to do pack v4 in the near future. Or at least\nexperiment with it and see if it works. If it does, we don't\nwant to have to cause a major ABI breakage across all those newly\ninstalled libgit2s... yikes.\n\nI'm really in favor of accessor functions for the first version of\nthe library. They can always be converted to macros once someone\nshows that their git visualizer program saves 10 ms on a 8,000 ms\nrender operation by avoiding accessor functions. I'd rather spend\nour brain cycles optimizing the runtime and the in-core data so\nwe spend less time in our tight revision traversal loops.\n\nSeriously. We make at least 10 or 11 function calls *per commit*\nthat comes out of get_revision(). If the formatting application is\nreally suffering from its 4 or 5 accessor function calls in order\nto get that returned data, we probably should also be looking at\nhow we can avoid function cals in the library.\n\nOh, and even with 4 or 5 accessor functions per commit in the\napplication that is *still* better than the 10 or so calls the\napplication probably makes today scraping \"git log --format=raw\"\noff a pipe and segment it into the different fields it needs.\n\nUnless pipes in Linux somehow allow negative time warping with\nCPU counters. Though on dual-core systems they might, since the\ntwo processes can run on different cores. But oh, you didn't want\nto worry about threading support too much in libgit2, so I guess\nyou also don't want to use multi-core systems.\n\n-- \nShawn.\n"},{"id":"94615","messageId":"20081102015611.GH15463@spearce.org","threadId":"16110","inReplyTo":"20081101173042.GE26229@artemis.corp","subject":"Re: libgit2 - a true git library","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2008-11-02T01:56:11Z","receivedAt":"2008-11-02T01:56:11Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Pierre Habouzit wrote:\n> On Fri, Oct 31, 2008 at 06:41:54PM +0000, Shawn O. Pearce wrote:\n> > How about this?\n> > \n> > http://www.spearce.org/projects/scm/libgit2/apidocs/CONVENTIONS\n> \n> FWIW I've read what you say about types, while this is good design to\n> make things abstract, accessors are slower _and_ disallow many\n> optimizations as it's a function call and that it may clobber all your\n> pointers values.\n\nYea, optimizing C is a bitch.\n\nI'm in favor of accessors *IN THE APPLICATION*.\n\nWithin the library's own C code, I think we should expose the struct,\nand use its members where it makes sense to. Especially in the\nreally tight loops where we don't want to introduce more overhead.\n\nMy rationale here is that we can change the struct at any time,\nand yet not change the ABI.\n\n> For types that _will_ be in the tight loops, we must make the types\n> explicit or it'll bite us hard performance-wise. I'm thinking what is\n> \"struct object\" or \"struct commit\" in git.git. It's likely that we will\n> loose a *lot* of those types are opaque.\n\nYes, but I'm arguing they should be opaque to the application, and\nvisible to the library. Today the application is suffering from\nmassive fork+exec overhead. I really don't give a damn if the\napplication's compiler has to deal with a function call to read\nfrom a private member of an opaque type. Its still thousands of\nCPU instructions less per operation.\n\nCome back to me a year after libgit2 has been widely deployed on\nLinux distros and we have multiple applications linking to it.\nLets talk then about the harmful performance problems caused by\nmaking these types opaque to the application. About that time\nwe'll also be talking about how great pack v4 is and why its a good\nthing those types were opaque, as we didn't have to break the ABI\nto introduce it.\n \n> It's IMNSHO on the matter that core structures of git _will_ have to be\n> made explicit. I'm thinking objects and their \"subtypes\" (commits,\n> trees, blobs). Maybe a couple of things on the same vein.\n\nSure, but in the library only.\n\n-- \nShawn.\n"},{"id":"94616","messageId":"alpine.DEB.1.00.0811020328070.22125@pacific.mpi-cbg.de.mpi-cbg.de","threadId":"16110","inReplyTo":"20081101204259.GC15463@spearce.org","subject":"Re: libgit2 - a true git library","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-11-02T02:30:12Z","receivedAt":"2008-11-02T02:30:12Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Sat, 1 Nov 2008, Shawn O. Pearce wrote:\n\n> But I was also under the impression that the brilliant engineers who \n> work for Microsoft decided that on their platform special annotations \n> have to be inserted on functions that a DLL wants to export to \n> applications.\n\nExactly. This is the \"good\" old __declspec(dllexport) for you. It is a \npain in the butt, but that is what you have to go for if libgit2 is \nsupposed to be any more portable than ligit.a.\n\nCiao,\nDscho\n"},{"id":"94630","messageId":"20081102050917.GA26634@linode.davidb.org","threadId":"16110","inReplyTo":"d411cc4a0811011807g229f8becs9f411d6e19fb6c12@mail.gmail.com","subject":"Re: libgit2 - a true git library","fromName":"David Brown","fromEmail":"git@davidb.org","sentAt":"2008-11-02T05:09:18Z","receivedAt":"2008-11-02T05:09:18Z","isPatch":false,"sender":{"key":"git@davidb.org","avatar":"https://gravatar.com/avatar/94c86a2938470a74c2eac5e2b69afc0871f79a660295c02219597aba8cb101c1?d=mp&s=160"},"body":"On Sat, Nov 01, 2008 at 06:07:04PM -0700, Scott Chacon wrote:\n\n>Think about trying to incorporate this into something proprietary,\n>Shawn - how much of a pain is it going to be to get that license\n>reviewed in Google? However, LGPL I'm sure there is already a\n>reviewed policy. Now, since that may be a pain, time that Shawn could\n>have been spending being paid to work on the library is lost because\n>they can't use it, or it takes weeks/months to review it. That's my\n>concern.\n\nThe gcc exception license should have been reviewed by anyone who has\never build anything proprietary out of gcc.\n\nGPL+link exception is a very common license. It's most common use is\nfor runtime libraries for various programming languages.\n\nLawyers I know are significantly less fearful of the GPL+exception\nthan the LGPL. The exception basically says that if you use it in a\ncertain way, then none of the GPL applies.\n\nDavid\n"},{"id":"94640","messageId":"20081102091902.GB4066@artemis","threadId":"16110","inReplyTo":"20081101204259.GC15463@spearce.org","subject":"Re: libgit2 - a true git library","fromName":"Pierre Habouzit","fromEmail":"madcoder@debian.org","sentAt":"2008-11-02T09:19:02Z","receivedAt":"2008-11-02T09:19:02Z","isPatch":false,"sender":{"key":"madcoder@debian.org","avatar":"https://avatars.githubusercontent.com/u/44708?v=4"},"body":"On Sat, Nov 01, 2008 at 08:42:59PM +0000, Shawn O. Pearce wrote:\n> Andreas Ericsson wrote:\n> > Shawn O. Pearce wrote:\n> >> During the GitTogether we were kicking around the idea of a ground-up\n> >> implementation of a Git library.\n> >\n> > Having looked briefly at the code, I've got a couple of comments:\n> > * GIT_EXTERN() does nothing. Ever. It's noise and should be removed.\n> \n> I feel the same way.\n> \n> But I was also under the impression that the brilliant engineers\n> who work for Microsoft decided that on their platform special\n> annotations have to be inserted on functions that a DLL wants to\n> export to applications.\n> \n> Hence any cross-platform library that I have seen annotates their\n> exported functions this way, with the macro being empty on POSIX\n> and expanding to some magic keyword on Microsoft's OS. I think it\n> goes between the return type and the function name too...\n> \n> > Instead it would be better to have GIT_PRIVATE(),\n> \n> I can see why you said this; needing GIT_PRIVATE() is a lot more\n> rare than needing GIT_EXTERN(). Only a handful of cross-module,\n> but private, functions are likely to exist, so it makes sense to\n> mark the smaller subset. But see above. *sigh*\n\nNot only there is the windows thing, but the *best* way to design a\nlibrary is to make _everything_ by default and only show what you mean\nto.\n\nGIT_EXTERN allow us to do just that with\n__attribute__((visibility(\"default\"))) on GNU-ld/GCC.\n\nOf course we can achieve the same using nothing on public symbols and\n__attribute__((visibility(\"hidden\"))) on the private ones, but it's way\nmore cumbersome and there's always a chance to forget one. It's bad\ndesign IMHO.\n\n\nFWIW GIT_EXTERN(...) prototype(arguments); isn't unredable to me,\neveryone does that, it doesn't break tags, it's _good_. And you can ask\ndoxygen to substitute this GIT_EXTERN(x) with x so that it doesn't shows\nup in the documentation at all (it can include these kind of partially\npreprocessed headers in the documentation) for the people who are really\nannoyed with them.\n\nBut it's definitely a necessary evil.\n\n-- \n·O· Pierre Habouzit\n··O madcoder@debian.org\nOOO http://www.madism.org\n"},{"id":"94645","messageId":"20081102092555.GC4066@artemis","threadId":"16110","inReplyTo":"20081102015611.GH15463@spearce.org","subject":"Re: libgit2 - a true git library","fromName":"Pierre Habouzit","fromEmail":"madcoder@debian.org","sentAt":"2008-11-02T09:25:55Z","receivedAt":"2008-11-02T09:25:55Z","isPatch":false,"sender":{"key":"madcoder@debian.org","avatar":"https://avatars.githubusercontent.com/u/44708?v=4"},"body":"On Sun, Nov 02, 2008 at 01:56:11AM +0000, Shawn O. Pearce wrote:\n> Pierre Habouzit wrote:\n> > For types that _will_ be in the tight loops, we must make the types\n> > explicit or it'll bite us hard performance-wise. I'm thinking what is\n> > \"struct object\" or \"struct commit\" in git.git. It's likely that we will\n> > loose a *lot* of those types are opaque.\n> \n> Yes, but I'm arguing they should be opaque to the application, and\n> visible to the library. Today the application is suffering from\n> massive fork+exec overhead. I really don't give a damn if the\n> application's compiler has to deal with a function call to read\n> from a private member of an opaque type. Its still thousands of\n> CPU instructions less per operation.\n\nThe problem is not the function call, it's really cheap on modern CPUs.\nThe problem is that across a function call the compiler cannot make\nsome kind of optimizations and _that_ isn't cheap.\n\nFor example, pointers whose value have been loaded from the store into a\nregister have to be loaded again. A function call trashes all the\nregisters, etc...\n\n\n> Come back to me a year after libgit2 has been widely deployed on\n> Linux distros and we have multiple applications linking to it.\n> Lets talk then about the harmful performance problems caused by\n> making these types opaque to the application. About that time\n> we'll also be talking about how great pack v4 is and why its a good\n> thing those types were opaque, as we didn't have to break the ABI\n> to introduce it.\n\nWell I fear it'll be more than a few percent harm, and I can already see\nLinus argue against the switch to libgit2 for git-core because it lost\n2% performances ;P\n\n-- \n·O· Pierre Habouzit\n··O madcoder@debian.org\nOOO http://www.madism.org\n"},{"id":"94743","messageId":"490ECFC2.1070901@op5.se","threadId":"16110","inReplyTo":"20081102015041.GG15463@spearce.org","subject":"Re: libgit2 - a true git library","fromName":"Andreas Ericsson","fromEmail":"ae@op5.se","sentAt":"2008-11-03T10:17:38Z","receivedAt":"2008-11-03T10:17:38Z","isPatch":false,"sender":{"key":"ae@op5.se","avatar":"https://gravatar.com/avatar/426e89595c75a8f5252dd0c989e5fabe5bcac616e68557427ad9aef6b0ca342a?d=mp&s=160"},"body":"Shawn O. Pearce wrote:\n> Andreas Ericsson wrote:\n>> Shawn O. Pearce wrote:\n>>> Eh, I disagree here. In git.git today \"struct commit\" exposes its\n>>> buffer with the canonical commit encoding. Having that visible\n>>> wrecks what Nico and I were thinking about doing with pack v4 and\n>>> encoding commits in a non-canonical format when stored in packs.\n>>> Ditto with trees.\n>> Err... isn't that backwards?\n> \n> No.\n> \n>> Surely you want to store stuff in the\n>> canonical format so you're forced to do as few translations as\n>> possible?\n> \n> No. We suspect that canonical format is harder to decompress and\n> parse during revision traversal. Other encodings in the pack file\n> may produce much faster runtime performance, and reduce page faults\n> (due to smaller pack sizes).\n> \n> We hardly ever use the canonical format for actual output; most\n> output rips the canonical format apart and then formats the data\n> the way it was requested. If we have the data *already* parsed in\n> the pack its much faster to output.\n> \n\nI'll have to look into the pack v4 stuff, as I can't get what you're\nsaying to make sense to me. Not canonicalizing the data when storing\nit means you'll have to have conversion routines from all the various\nencodings, unless you first canonicalize it and then encode it in the\nway you need it in the pack way. Never using canonical format means\nit has no potential for combinatorial explosion, and every converter\nneeds to know about every other format.\n\n\n>> Or are you trying to speed up packing by skipping the\n>> canonicalization part?\n> \n> Wrong; we're trying to speed up reading. Packing may go slower,\n> especially during the first conversion of v2->v4 for any given\n> repository, but packing is infrequent so the minor (if any) drop\n> in performance here is probably worth the reading performance gains.\n> \n>> Well, if macro usage is adhered to one wouldn't have to worry,\n>> since the macro can just be rewritten with a function later (if,\n>> for example, translation or some such happens to be required).\n>> Older code linking to a newer library would work (assuming the\n>> size of the commit object doesn't change anyway),\n> \n> You are assuming too much magic. If the older ABI used a macro\n> and the newer one (which supports pack v4) organized struct commit\n> differently and the user upgrades libgit2.so the older applications\n> just broke, horribly.\n> \n\nNaturally. Re-sizing objects that weren't previously protected by\naccessor functions will always break the ABI unless the layout of\nthe previously existing items in the object doesn't change, which\nis exactly what I said.\n\n> We know we want to do pack v4 in the near future. Or at least\n> experiment with it and see if it works. If it does, we don't\n> want to have to cause a major ABI breakage across all those newly\n> installed libgit2s... yikes.\n> \n\nThat's not necessary, although it requires a bit more thought when\nchanging the objects.\n\n> I'm really in favor of accessor functions for the first version of\n> the library. They can always be converted to macros once someone\n> shows that their git visualizer program saves 10 ms on a 8,000 ms\n> render operation by avoiding accessor functions. I'd rather spend\n> our brain cycles optimizing the runtime and the in-core data so\n> we spend less time in our tight revision traversal loops.\n> \n\nI agree that for early versions they should definitely be functions,\nbut since we've been talking about \"final design\" earlier in the\nthread, that's what I was referring to here to.\n\n> Seriously. We make at least 10 or 11 function calls *per commit*\n> that comes out of get_revision(). If the formatting application is\n> really suffering from its 4 or 5 accessor function calls in order\n> to get that returned data, we probably should also be looking at\n> how we can avoid function cals in the library.\n> \n> Oh, and even with 4 or 5 accessor functions per commit in the\n> application that is *still* better than the 10 or so calls the\n> application probably makes today scraping \"git log --format=raw\"\n> off a pipe and segment it into the different fields it needs.\n> \n\nRight.\n\n> Unless pipes in Linux somehow allow negative time warping with\n> CPU counters. Though on dual-core systems they might, since the\n> two processes can run on different cores. But oh, you didn't want\n> to worry about threading support too much in libgit2, so I guess\n> you also don't want to use multi-core systems.\n> \n\nAll comps where I use git today are multi-core systems, so I'm very\nmuch in favour of parallell processing. Otoh, I also don't think it\nmakes sense to jump through hoops to make sure one can, fe, read and\nparse all tags in multiple threads simultaneously. In other words, I\ndon't think we need to bother adding thread-safety for stuff that \nreally only should happen if the application is poorly designed\n(unless it's straightforward to do so, ofcourse).\n\n-- \nAndreas Ericsson andreas.ericsson@op5.se\nOP5 AB www.op5.se\nTel: +46 8-230225 Fax: +46 8-230231\n"},{"id":"94754","messageId":"490EF7C0.3000909@op5.se","threadId":"16110","inReplyTo":"20081101204259.GC15463@spearce.org","subject":"Re: libgit2 - a true git library","fromName":"Andreas Ericsson","fromEmail":"ae@op5.se","sentAt":"2008-11-03T13:08:16Z","receivedAt":"2008-11-03T13:08:16Z","isPatch":false,"sender":{"key":"ae@op5.se","avatar":"https://gravatar.com/avatar/426e89595c75a8f5252dd0c989e5fabe5bcac616e68557427ad9aef6b0ca342a?d=mp&s=160"},"body":"Shawn O. Pearce wrote:\n> Andreas Ericsson wrote:\n>> Shawn O. Pearce wrote:\n>>> During the GitTogether we were kicking around the idea of a ground-up\n>>> implementation of a Git library.\n>> Having looked briefly at the code, I've got a couple of comments:\n>> * GIT_EXTERN() does nothing. Ever. It's noise and should be removed.\n> \n> I feel the same way.\n> \n> But I was also under the impression that the brilliant engineers\n> who work for Microsoft decided that on their platform special\n> annotations have to be inserted on functions that a DLL wants to\n> export to applications.\n> \n> Hence any cross-platform library that I have seen annotates their\n> exported functions this way, with the macro being empty on POSIX\n> and expanding to some magic keyword on Microsoft's OS. I think it\n> goes between the return type and the function name too...\n> \n>> Instead it would be better to have GIT_PRIVATE(),\n> \n> I can see why you said this; needing GIT_PRIVATE() is a lot more\n> rare than needing GIT_EXTERN(). Only a handful of cross-module,\n> but private, functions are likely to exist, so it makes sense to\n> mark the smaller subset. But see above. *sigh*\n> \n\nThanks for the detailed explanation.\n\n>> * Prefixing the files themselves with git_ is useless and only leads\n>> to developer frustration. I imagine we'd be installing any header\n>> files in a git/ directory anyway, so we're gaining absolutely\n>> nothing with the git_ prefix on source-files.\n> \n> Yes, I realized that this morning. I plan on changing that mess\n> around so we have \"include/git/oid.h\" and library and application\n> code can use \"#include \". Library modules should just\n> be \"src/oid.c\" then.\n> \n\nI noticed when I fetched the latest head today that it's already\ndone. I fail to understand why headers need to be in a separate\npath so that oid.c can't just '#include \"oid.h\"'.\n\nWith the risk of nitpicking you to death, put public headers in a\nseparate dir (I'd suggest public/%.h in Make-speak, but I have no\nstrong preference) and keep private headers next to %.c. Always\n#include the public header file from the private one (that should\nprobably be in CONVENTIONS).\n\n>> Apart from that, it seems you've been designing a lot rather than\n>> trying to use the API to actually do something.\n> \n> I wanted to get a solid idea of what our API conventions should be,\n> before we started writing a lot of code around them. Part of the\n> problem with the git.git code is we don't have conventions that are\n> really suited for use in a shared library (assuming we even have\n> conventions in there) so we can't use that code as a library today.\n> \n\nRight. I guess I'm too firm a believer in system evolution by constant\nrefactoring (with fluctuating api's, yes) rather than thinking initial\ndesign can ever be done exactly right.\n\n>> It would, imo, be\n>> a lot better to start development with adding functionality shared\n>> between all programs and then expand further on that, such as\n>> incorporating all functions needed for manipulating tags into the\n>> library and then modify existing code to use the library to get\n>> tag-ish things done.\n> \n> Tags are mostly pointless. Its a tiny part of the code that isn't\n> that interesting to most people. And it requires object database\n> access anyway if you want to talk about parsing or reading a tag.\n> There's almost no point in a git library that can't read the on\n> disk object database, or write to it.\n> \n\nTrue, but designing top-down means you'll need to write one more\nAPI to get the first stuff working, so you'll always be using the\nnew code you write immediately and for something real. IMO, that\nmakes it much more fun and productive to write the lib itself.\n\n>> I also think it's quite alright to not strive *too* hard to make\n>> all functions thread-safe, as very few of them will actually need\n>> that. It's unlikely that a user program will spawn one thread to\n>> write a lot of tags while another is trying to parse them, for\n>> example.\n> \n> Oh really?\n> \n> Maybe true for tags, just because they are such an unimportant part\n> of the git suite compared to everything else.\n> \n> But right now I'm running a production system using a threaded server\n> process that is operating on Git repositories. Fortunately threads\n> suck less on Java than they do on POSIX, and we have a 100% pure\n> Java library available for Git.\n> \n> It would be nice if a library created in the late part of 2008\n> recognized that threads exist, aren't going to disappear tomorrow,\n> and that consumers of libraries actually may need to run the library\n> within a threaded process.\n> \n> Or are you one of those developers who think threads only exist\n> in the giant monolithic kernel land, and all user space should\n> be isolated process? I often wonder who such people can justify\n> the kernel address space being multi-threaded but userland being\n> stuck to single threaded applications. Oh, right, the kernel has\n> to go fast...\n> \n\nNo, I'm one of those developers who think that if implementing a\nfunction as thread-safe means it'll take 50 times longer than just\nwriting something that works, the right decision is to go with the\nfaster way to get the job done and then expand on it later when the\nneed arises. Reading my original post, I realize I should have made\nthat more clear. Sorry for making your gall rise unnecessarily.\n\n-- \nAndreas Ericsson andreas.ericsson@op5.se\nOP5 AB www.op5.se\nTel: +46 8-230225 Fax: +46 8-230231\n"},{"id":"94776","messageId":"20081103162017.GL15463@spearce.org","threadId":"16110","inReplyTo":"20081102050917.GA26634@linode.davidb.org","subject":"Re: libgit2 - a true git library","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2008-11-03T16:20:18Z","receivedAt":"2008-11-03T16:20:18Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"David Brown wrote:\n> On Sat, Nov 01, 2008 at 06:07:04PM -0700, Scott Chacon wrote:\n>\n>> Think about trying to incorporate this into something proprietary,\n>> Shawn - how much of a pain is it going to be to get that license\n>> reviewed in Google? However, LGPL I'm sure there is already a\n>> reviewed policy. Now, since that may be a pain, time that Shawn could\n>> have been spending being paid to work on the library is lost because\n>> they can't use it, or it takes weeks/months to review it. That's my\n>> concern.\n>\n> The gcc exception license should have been reviewed by anyone who has\n> ever build anything proprietary out of gcc.\n\nIndeed. And gcc is a *very* popular compiler on Linux distributions.\nA lot of commerical software is built under it, and distributed in\nbinary-only form. Those companies have already done the legal review\nthey felt necessary before shipping (and selling) that software.\n\n-- \nShawn.\n"},{"id":"95214","messageId":"4915939B.1070306@gmail.com","threadId":"16110","inReplyTo":"20081031170704.GU14786@spearce.org","subject":"Re: libgit2 - a true git library","fromName":"Steve Frécinaux","fromEmail":"nudrema@gmail.com","sentAt":"2008-11-08T13:26:51Z","receivedAt":"2008-11-08T13:26:51Z","isPatch":false,"sender":{"key":"nudrema@gmail.com","avatar":null},"body":"Shawn O. Pearce wrote:\n> During the GitTogether we were kicking around the idea of a ground-up\n> implementation of a Git library. This may be easier than trying\n> to grind down git.git into a library, as we aren't tied to any\n> of the current global state baggage or the current die() based\n> error handling.\n> \n> I've started an _extremely_ rough draft. The code compiles into a\n> libgit.a but it doesn't even implement what it describes in the API,\n> let alone a working Git implementation. Really what I'm trying to\n> incite here is some discussion on what the API looks like.\n\nJust a random question: is there a reason why you have put all the .h in \na separate includes/ directory instead of relying on the install target \nto put the include files at the right place ?\n\nTo me it makes it much harder to hack on the files as one is always \nrequired to switch between both directories...\n"},{"id":"95217","messageId":"4915A3CB.5010909@op5.se","threadId":"16110","inReplyTo":"4915939B.1070306@gmail.com","subject":"Re: libgit2 - a true git library","fromName":"Andreas Ericsson","fromEmail":"ae@op5.se","sentAt":"2008-11-08T14:35:55Z","receivedAt":"2008-11-08T14:35:55Z","isPatch":false,"sender":{"key":"ae@op5.se","avatar":"https://gravatar.com/avatar/426e89595c75a8f5252dd0c989e5fabe5bcac616e68557427ad9aef6b0ca342a?d=mp&s=160"},"body":"Steve Frécinaux wrote:\n> Shawn O. Pearce wrote:\n>> During the GitTogether we were kicking around the idea of a ground-up\n>> implementation of a Git library. This may be easier than trying\n>> to grind down git.git into a library, as we aren't tied to any\n>> of the current global state baggage or the current die() based\n>> error handling.\n>>\n>> I've started an _extremely_ rough draft. The code compiles into a\n>> libgit.a but it doesn't even implement what it describes in the API,\n>> let alone a working Git implementation. Really what I'm trying to\n>> incite here is some discussion on what the API looks like.\n> \n> Just a random question: is there a reason why you have put all the .h in \n> a separate includes/ directory instead of relying on the install target \n> to put the include files at the right place ?\n> \n> To me it makes it much harder to hack on the files as one is always \n> required to switch between both directories...\n\nI agree with this, but as I guess Shawn will do roughly 45 times more\nwork on it than me (according to current commit-count in git.git), I'll\nlive with it.\n\n-- \nAndreas Ericsson andreas.ericsson@op5.se\nOP5 AB www.op5.se\nTel: +46 8-230225 Fax: +46 8-230231\n"},{"id":"95226","messageId":"20081108172759.GA31655@artemis.corp","threadId":"16110","inReplyTo":"4915A3CB.5010909@op5.se","subject":"Re: libgit2 - a true git library","fromName":"Pierre Habouzit","fromEmail":"madcoder@debian.org","sentAt":"2008-11-08T17:27:59Z","receivedAt":"2008-11-08T17:27:59Z","isPatch":false,"sender":{"key":"madcoder@debian.org","avatar":"https://avatars.githubusercontent.com/u/44708?v=4"},"body":"On Sat, Nov 08, 2008 at 02:35:55PM +0000, Andreas Ericsson wrote:\n> Steve Frécinaux wrote:\n> > Just a random question: is there a reason why you have put all the\n> > .h in a separate includes/ directory instead of relying on the\n> > install target to put the include files at the right place ?\n> > To me it makes it much harder to hack on the files as one is always\n> > required to switch between both directories...\n> \n> I agree with this, but as I guess Shawn will do roughly 45 times more\n> work on it than me (according to current commit-count in git.git), I'll\n> live with it.\n\nI don't, modifying the public includes may break the ABI and the API.\n\nI believe it to be a good practice to put them in a separate directory\nso that people modifying them will know this particular header is\npublic. Yes you can name your private headers differently, but it's not\nreally the same, it doesn't make editing public headers hard, and it has\nto. People modifying them _have_ to thing \"err why am I modifying this\nspecific header in the first place\" before doing anything in it.\n\n-- \n·O· Pierre Habouzit\n··O madcoder@debian.org\nOOO http://www.madism.org\n"},{"id":"95250","messageId":"4916B8AA.2080602@op5.se","threadId":"16110","inReplyTo":"20081108172759.GA31655@artemis.corp","subject":"Re: libgit2 - a true git library","fromName":"Andreas Ericsson","fromEmail":"ae@op5.se","sentAt":"2008-11-09T10:17:14Z","receivedAt":"2008-11-09T10:17:14Z","isPatch":false,"sender":{"key":"ae@op5.se","avatar":"https://gravatar.com/avatar/426e89595c75a8f5252dd0c989e5fabe5bcac616e68557427ad9aef6b0ca342a?d=mp&s=160"},"body":"Pierre Habouzit wrote:\n> On Sat, Nov 08, 2008 at 02:35:55PM +0000, Andreas Ericsson wrote:\n>> Steve Frécinaux wrote:\n>>> Just a random question: is there a reason why you have put all the\n>>> .h in a separate includes/ directory instead of relying on the\n>>> install target to put the include files at the right place ?\n>>> To me it makes it much harder to hack on the files as one is always\n>>> required to switch between both directories...\n>> I agree with this, but as I guess Shawn will do roughly 45 times more\n>> work on it than me (according to current commit-count in git.git), I'll\n>> live with it.\n> \n> I don't, modifying the public includes may break the ABI and the API.\n> \n> I believe it to be a good practice to put them in a separate directory\n> so that people modifying them will know this particular header is\n> public. Yes you can name your private headers differently, but it's not\n> really the same, it doesn't make editing public headers hard, and it has\n> to. People modifying them _have_ to thing \"err why am I modifying this\n> specific header in the first place\" before doing anything in it.\n> \n\nWell, I suggested putting \"src/public/public_header.h\" quite early on,\nwith private headers next to the source. AFAIU, the private and public\nheaders both are now located in the same directory, and that directory is\nseparate from the .c files.\n\n-- \nAndreas Ericsson andreas.ericsson@op5.se\nOP5 AB www.op5.se\nTel: +46 8-230225 Fax: +46 8-230231\n"},{"id":"95283","messageId":"20081109210248.GF2932@spearce.org","threadId":"16110","inReplyTo":"4916B8AA.2080602@op5.se","subject":"Re: libgit2 - a true git library","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2008-11-09T21:02:48Z","receivedAt":"2008-11-09T21:02:48Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Andreas Ericsson wrote:\n>\n> Well, I suggested putting \"src/public/public_header.h\" quite early on,\n\nI must have missed that suggestion. Its not a bad idea.\n\n> with private headers next to the source. AFAIU, the private and public\n> headers both are now located in the same directory, and that directory is\n> separate from the .c files.\n\nCurrently there are only public headers, and the public headers are\nall under include/git/. Private headers are going to be under src/\nso they are isolated from the public headers.\n\nBut I haven't had a chance to touch libgit2 in over a week. :-\\\n\nI've simply got too many projects going on at once. This is one\nI really want to work on though, so I'm going to try and make time\nfor it next week. But I'm also in the middle of a major overhaul\nof Gerrit, so it can run on non-Google infrastructure and thus is\nusable by pretty much anyone.\n\n-- \nShawn.\n"}]}