{"thread":{"id":"29632","subject":"[PATCH/RFC] Document format of basic Git objects","startedAt":"2012-02-15T13:22:20Z","lastAt":"2012-02-20T18:27:08Z","messageCount":17,"participants":["Nguyễn Thái Ngọc Duy","Jonathan Nieder","Junio C Hamano","Philip Oakley","徐迪","Thomas Rast","Nguyen Thai Ngoc Duy","Jeff King"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"184761","messageId":"1329312140-24089-1-git-send-email-pclouds@gmail.com","threadId":"29632","inReplyTo":null,"subject":"[PATCH/RFC] Document format of basic Git objects","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2012-02-15T13:22:20Z","receivedAt":"2012-02-15T13:22:20Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"Basic objects' format is pretty simple and (I think) well-known.\nHowever it's good that we document them. At least we can keep track of\nthe evolution of an object format. The commit object, for example,\nover the years has learned \"encoding\" and recently GPG signing.\n\nThis is just a draft text with a bunch of fixmes. But I'd like to hear\nfrom the community if this is a worthy effort. If so, then whether\ngit-cat-file is a proper place for it. Or maybe we put relevant text\nin commit-tree, write-tree and mktag, then refer to them in cat-file\nbecause cat-file can show raw objects.\n\nSo comments?\n\nSigned-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n---\n PS. This also makes me wonder if tag object supports \"encoding\".\n Haven't dug down in history yet.\n\n Documentation/git-cat-file.txt |   40 ++++++++++++++++++++++++++++++++++++++++\n 1 files changed, 40 insertions(+), 0 deletions(-)\n\ndiff --git a/Documentation/git-cat-file.txt b/Documentation/git-cat-file.txt\nindex 2fb95bb..e3dd6d9 100644\n--- a/Documentation/git-cat-file.txt\n+++ b/Documentation/git-cat-file.txt\n@@ -100,6 +100,46 @@ for each object specified on stdin that does not exist in the repository:\n <object> SP missing LF\n ------------\n \n+OBJECT FORMAT\n+-------------\n+\n+Tree object consists of a series of tree entries sorted in memcmp()\n+order by entry name. Each entry consists of:\n+\n+- POSIX file mode encoded in octal ascii\n+- One space character\n+- Entry name terminated by one character NUL\n+- 20 byte SHA-1 of the entry\n+\n+Tag object is ascii plain text in a format similar to email format\n+(RFC 822). It consists of a header and a body, separated by a blank\n+line. The header includes exactly four fields in the following order:\n+\n+1. \"object\" field, followed by SHA-1 in ascii of the tagged object\n+2. \"type\" field, followed by the type in ascii of the tagged object\n+   (either \"commit\", \"tag\", \"blob\" or \"tree\" without quotes,\n+   case-sensitive)\n+3. \"tag\" field, followed by the tag name\n+4. \"tagger\" field, followed by the <XXX, to be named>\n+\n+The tag body contains the tag's message and possibly GPG signature.\n+\n+Commit object is in similar format to tag object. The commit body is\n+in plain text of the chosen encoding (by default UTF-8). The commit\n+header has the following fields in listed order\n+\n+1. One \"tree\" field, followed by the commit's tree's SHA-1 in ascii\n+2. Zero, one or more \"parent\" field\n+3. One \"author\" field, in <XXX to be named> format\n+3. One \"committer\" field, in <XXX to be named> format\n+4. Optionally one \"encoding\" field, followed by the encoding used for\n+   commit body\n+5. GPG signature (fixme)\n+\n+More headers after these fields are allowed. Unrecognized header\n+fields must be kept untouched if the commit is rewritten. However, a\n+compliant Git implementation produces the above header fields only.\n+\n GIT\n ---\n Part of the linkgit:git[1] suite\n-- \n1.7.8.36.g69ee2\n"},{"id":"184771","messageId":"20120215173127.GA22004@burratino","threadId":"29632","inReplyTo":"1329312140-24089-1-git-send-email-pclouds@gmail.com","subject":"Re: [PATCH/RFC] Document format of basic Git objects","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2012-02-15T17:31:27Z","receivedAt":"2012-02-15T17:31:27Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Hi,\n\nNguyễn Thái Ngọc Duy wrote:\n\n> Basic objects' format is pretty simple and (I think) well-known.\n> However it's good that we document them. At least we can keep track of\n> the evolution of an object format. The commit object, for example,\n> over the years has learned \"encoding\" and recently GPG signing.\n\nYes, I agree.\n\n> This is just a draft text with a bunch of fixmes. But I'd like to hear\n> from the community if this is a worthy effort. If so, then whether\n> git-cat-file is a proper place for it. Or maybe we put relevant text\n> in commit-tree, write-tree and mktag, then refer to them in cat-file\n> because cat-file can show raw objects.\n\nAbout where to place this text, I am of two minds.\n\n1. On one hand, from the user's perspective it would be most intuitive\nto place it in a separate git-object(5) manual page.  That way,\ngitrepository-layout(5), git-fsck(1), git-hash-object(1), the user\nmanual, and so on would all have one document to link to.\n\n2. On the other hand, from a development perspective I suspect it\nwould be valuable to put it in the git-fsck(1) page, since that would\nhave two consequences:\n\n - when changing the documentation, this would provide a reminder to\n   update fsck.c at the same time\n\n - when changing fsck.c, this would provide a reminder to update the\n   documentation at the same time\n\nOk, (2) was tongue in cheek. :)  I believe this information belongs in\na dedicated page with a name like gitobject(5), and that you are right\nto put it in user-visible documentation instead of hiding it in\nDocumentation/technical, since it is information needed if one is to\nuse \"git hash-object -w\" correctly.\n\nOk, on to the text itself.\n\n[...]\n> --- a/Documentation/git-cat-file.txt\n> +++ b/Documentation/git-cat-file.txt\n> @@ -100,6 +100,46 @@ for each object specified on stdin that does not exist in the repository:\n>  <object> SP missing LF\n>  ------------\n>  \n> +OBJECT FORMAT\n> +-------------\n> +\n> +Tree object consists of a series of tree entries sorted in memcmp()\n> +order by entry name.\n\nMissing article (\"A tree object\", \"The tree object\", or \"Each tree\nobject\").  More importantly, the curious reader might want to know\nwhether a tree object is supposed to contain entries pointing to other\ntree objects for subdirectories or whether the subdirectory's\ninformation is included inline like in the index.\n\nI guess I would expect something like (stealing from the user manual):\n\n\tTREE OBJECTS\n\t------------\n\tA tree object contains a list of entries, each with a mode,\n\tobject type, object name, and filename, sorted by filename.  It\n\trepresents the contents of a single directory tree.\n\n\tThe object type may be a blob, representing the contents of a\n\tfile, another tree, representing the contents of a subdirectory,\n\tor a commit (representing a subproject).  Since trees and blobs,\n\tlike all other objects, are named by a hash of their contents,\n\ttwo trees have the same object name if and only if their\n\tcontents (including, recursively, the contents of all\n\tsubdirectories) are identical.  This allows git to quickly\n\tdetermine the differences between two related tree objects,\n\tsince it can ignore any entries with identical object names.\n\n\tNote that the files all have mode 644 or 755: git actually only\n\tpays attention to the executable bit.\n\n\tEncoding\n\t~~~~~~~~\n\tEntries are of variable length and self-delimiting.  Each entry\n\tconsists of\n\n\t- a POSIX file mode in octal representation\n\t- exactly one space (ASCII SP)\n\t- filename for the entry, as a NUL-terminated string\n\t- 20-byte binary object name\n\n\tThe mode should be 100755 (executable file), 100644 (regular\n\tfile), 120000 (symlink), 40000 (subdirectory), or 160000\n\t(subproject), with no leading zeroes.  Modes with one leading\n\tzero and the synonym 100664 for 100644 are also accepted for\n\thistorical reasons.\n\n\tThe filename may be an arbitrary nonempty string of bytes, as\n\tlong as it contains no '/' or NUL character.\n\n\tThe associated object must be a valid blob if the mode indicates\n\ta file or symlink, tree if it indicates a subdirectory, or\n\tcommit if it indicates a subproject.  The blob associated to a\n\tsymlink entry indicates the link target and its content not\n\thave any embedded NULs.\n\nBy the way, git fsck seems to tolerate the old \"flat tree\" format\n(i.e., that condition is FSCK_WARN and not FSCK_ERROR), but I don't\nsee any code supporting it elsewhere in git.  Bug?\n\n\tSorting\n\t~~~~~~~\n\t... no duplicates, sort order, etc ...\n\n[...]\n> +Tag object is ascii plain text in a format similar to email format\n> +(RFC 822). It consists of a header and a body, separated by a blank\n> +line.\n\nThe above description makes me worry that the reader might try some\nthings that are allowed by RFC 822: rearranging header fields,\ncontinuation lines, and so on.\n\n>        The header includes exactly four fields in the following order:\n> +\n> +1. \"object\" field, followed by SHA-1 in ascii of the tagged object\n> +2. \"type\" field, followed by the type in ascii of the tagged object\n> +   (either \"commit\", \"tag\", \"blob\" or \"tree\" without quotes,\n> +   case-sensitive)\n> +3. \"tag\" field, followed by the tag name\n> +4. \"tagger\" field, followed by the <XXX, to be named>\n> +\n> +The tag body contains the tag's message and possibly GPG signature.\n\nThis part looks good.  Stealing from the user manual again, maybe:\n\n\tTAG OBJECTS\n\t-----------\n\tA tag object contains an object, object type, tag name, the name\n\tof the person (\"tagger\") who created the tag, and a message,\n\twhich may contain a signature.\n\n\t------------------------------------------------\n\t$ git cat-file tag v1.5.0\n\tobject 437b1b20df4b356c9342dac8d38849f24ef44f27\n\ttype commit\n\ttag v1.5.0\n\ttagger Junio C Hamano <junkio@cox.net> 1171411200 +0000\n\n\tGIT 1.5.0\n\t-----BEGIN PGP SIGNATURE-----\n\tVersion: GnuPG v1.4.6 (GNU/Linux)\n\n\tiD8DBQBF0lGqwMbZpPMRm5oRAuRiAJ9ohBLd7s2kqjkKlq1qqC57SbnmzQCdG4ui\n\tnLE/L9aUXdWeTFPron96DLA=\n\t=2E+0\n\t-----END PGP SIGNATURE-----\n\t------------------------------------------------\n\n\tMore precisely, a tag contains at least five lines:\n\n\t1. \"object\", followed by a space, followed by the 40-character\n\t   textual object name of the tagged object\n\t2. \"type\" + SP + the type of the tagged object (\"commit\", \"tag\",\n\t   \"blob\", or \"tree\")\n\t3. \"tag\" + SP + the name of the tag\n\t4. \"tagger\" + SP + an ident string\n\t5. a blank line\n\n\tAny remaining text after these lines forms the tag message.\n\n\tThe object field must point to a valid object of type indicated\n\tby the type field.  The tag name can be an arbitrary string\n\twithout NUL bytes or embedded newlines; in practice it usually\n\tfollows the restrictions described in git-check-ref-format(1).\n\n[...]\n> +\n> +Commit object is in similar format to tag object. The commit body is\n> +in plain text of the chosen encoding (by default UTF-8). The commit\n> +header has the following fields in listed order\n\nSame considerations apply here --- I'd suggest stealing text from the\ncommit-object section of the user manual and from commit logs.\n\nHope that helps,\nJonathan\n\n> +\n> +1. One \"tree\" field, followed by the commit's tree's SHA-1 in ascii\n> +2. Zero, one or more \"parent\" field\n> +3. One \"author\" field, in <XXX to be named> format\n> +3. One \"committer\" field, in <XXX to be named> format\n> +4. Optionally one \"encoding\" field, followed by the encoding used for\n> +   commit body\n> +5. GPG signature (fixme)\n> +\n> +More headers after these fields are allowed. Unrecognized header\n> +fields must be kept untouched if the commit is rewritten. However, a\n> +compliant Git implementation produces the above header fields only.\n> +\n>  GIT\n>  ---\n>  Part of the linkgit:git[1] suite\n"},{"id":"184781","messageId":"7v4nur2806.fsf@alter.siamese.dyndns.org","threadId":"29632","inReplyTo":"1329312140-24089-1-git-send-email-pclouds@gmail.com","subject":"Re: [PATCH/RFC] Document format of basic Git objects","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-02-15T19:48:09Z","receivedAt":"2012-02-15T19:48:09Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Nguyễn Thái Ngọc Duy  <pclouds@gmail.com> writes:\n\n> This is just a draft text with a bunch of fixmes. But I'd like to hear\n> from the community if this is a worthy effort. If so, then whether\n> git-cat-file is a proper place for it. Or maybe we put relevant text\n> in commit-tree, write-tree and mktag, then refer to them in cat-file\n> because cat-file can show raw objects.\n>\n> So comments?\n\nThis _only_ describes the payload (i.e. without the 'blob <size>\\n' header\nused in loose object, in other words, what read_object() may return).\nThere should be a sentence to stress this.  As many Git intros (including\nmy book) begin with the \"a short header 'blob <size>\\n' concatenated with\nthe contents is hashed to compute the object name\" picture, it would be\nconfusing unless you explicitly say that you are only describing the\n\"contents\" part.\n\nIt makes sense to mention that the cat-file subcommand is used to obtain\nthis raw data somewhere in the documentation, but I would say the content\nof this patch belongs to Documentation/technical/ somewhere.\n\n>  PS. This also makes me wonder if tag object supports \"encoding\".\n\nI do not think so.\n\n> +OBJECT FORMAT\n> +-------------\n> +\n> +Tree object consists of a series of tree entries sorted in memcmp()\n> +order by entry name. Each entry consists of:\n> +\n> +- POSIX file mode encoded in octal ascii\n\nAdd \", no 0 padding to the right\" at the end, as I heard that every\nimitation of Git gets this wrong in its first version.\n\n> +- One space character\n> +- Entry name terminated by one character NUL\n> +- 20 byte SHA-1 of the entry\n\n> +Tag object is ascii plain text in a format similar to email format\n> +(RFC 822). ...\n\nDo not mention \"email format (RFC 822)\" at all.  The differences are\nsignificant enough that it only confuses the readers.\n\nWe do not have colon at the end of the header, we do not promise to parse\nfield names case insensitively, and the way continuation lines are parsed\nis totally different (a \"similar\" construct in RFC 2822 is \"folded header\nlines\", but it is signalled by \"folding white space\", it discards the\nend-of-line from the previous line and makes the result a logical single\nline. Our continuation lines are introduced by a single SP and the result\nof concatenation keeps the end-of-line from the previous lines, making the\nresult multiple lines).\n\nAlso we do not promise that the lines in the header part are always\n<field,value> pairs.  So rephrase this while carefully distinguishing\nbetween \"a line in header\" and \"field\".\n\n    A commit or a tag object begins with the \"header\" that consists of one\n    or more lines delimited by LF. The end of the header is signalled by\n    an empty line.\n\n    A \"continuation line\" in the header begins with a SP.  The remainder\n    of the line, after removing that SP, is concatenated to the previous\n    line, while retaining the LF at the end of the previous line.\n\n    When a line in the header begins with a letter other than SP, and has\n    at least one SP in it, it is called a \"field\".  A field consists of\n    the \"field name\", which is the string before the first SP on the line,\n    and its \"value\", which is everything after that SP.  When the value\n    consists of multiple lines, continuation lines are used.\n\n    More than one field with the same name can appear in the header of an\n    object, and the order in which they appear is significant.\n\n    In a commit object, the header begins with the following fields that\n    have such and such meaning.\n\n    In a tag object, the header begins with the following fields...\n\n    After these defined fields, newer versions of git may add more lines\n    in the header. Some of them may be fields, others might not be. The\n    implementations to parse commit and tag objects must ignore lines in\n    the header that it does not understand without triggering an error.\n\n>  ... It consists of a header and a body, separated by a blank\n> +line. The header includes exactly four fields in the following order:\n> +\n\nIf you hand-craft a tag-like object that has unknown field after these\nfour, how badly the current implementations behave?\n\n> +1. \"object\" field, followed by SHA-1 in ascii of the tagged object\n> +2. \"type\" field, followed by the type in ascii of the tagged object\n> +   (either \"commit\", \"tag\", \"blob\" or \"tree\" without quotes,\n> +   case-sensitive)\n> +3. \"tag\" field, followed by the tag name\n> +4. \"tagger\" field, followed by the <XXX, to be named>\n\n> +The tag body contains the tag's message and possibly GPG signature.\n> +\n> +Commit object is in similar format to tag object. The commit body is\n\nIt is strange that you introduce tag and then commit.  I would think that\nreaders expect to see them presented in the usual blob/tree/commit/tag\norder.\n\n> +in plain text of the chosen encoding (by default UTF-8). The commit\n> +header has the following fields in listed order\n> +\n> +1. One \"tree\" field, followed by the commit's tree's SHA-1 in ascii\n> +2. Zero, one or more \"parent\" field\n> +3. One \"author\" field, in <XXX to be named> format\n> +3. One \"committer\" field, in <XXX to be named> format\n> +4. Optionally one \"encoding\" field, followed by the encoding used for\n> +   commit body\n> +5. GPG signature (fixme)\n> +\n> +More headers after these fields are allowed. Unrecognized header\n> +fields must be kept untouched if the commit is rewritten.\n\nReplace the first sentence with \"New kinds of fields may be added in later\nversions of git.\" and drop the second one entirely.  Depending on the\nreason and nature of the \"rewrite\", we may or may not want to keep these\nunknown header lines, so it is best to leave the behaviour unspecified.\nFor example, it makes sense to retain \"mergetag\" because it is about the\nparent, not the resulting commit.  It does not make sense to keep \"gpgsig\"\nbecause it is about the commit you are rewriting to invalidate that old\nsignature.\n"},{"id":"184805","messageId":"7vwr7nz1xx.fsf@alter.siamese.dyndns.org","threadId":"29632","inReplyTo":"7v4nur2806.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH/RFC] Document format of basic Git objects","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-02-16T07:12:42Z","receivedAt":"2012-02-16T07:12:42Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano <gitster@pobox.com> writes:\n\n>> +- POSIX file mode encoded in octal ascii\n>\n> Add \", no 0 padding to the right\" at the end, as I heard that every\n> imitation of Git gets this wrong in its first version.\n\nEhh, of course no 0 padding on the LEFT hand side.\n\nRice-bowl with left hand, chopsticks with right hand. I always mix these\nup.  Sorry.\n"},{"id":"184940","messageId":"1329624946-32173-1-git-send-email-pclouds@gmail.com","threadId":"29632","inReplyTo":"1329312140-24089-1-git-send-email-pclouds@gmail.com","subject":"[PATCH/RFC v2] Document format of basic Git objects","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2012-02-19T04:15:46Z","receivedAt":"2012-02-19T04:15:46Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"Still draft for discussion. Of three people who participated on this\nthread, two favor a man page (me and Jonathan), one techincal/\n(Junio), so let's put it as a man page for now.\n\nSome notes:\n\n - I'm tempted to include pack-format.txt because I also document\n   loose object format here. If it's included and\n   gitrepository-layout.txt links to this, we have a quite complete\n   documentation of what's inside $GIT_DIR (assuming rebase-apply and\n   such are of private use)\n\n - Not sure if we fix the order of gpgsig and mergetag, or they can be\n   mixed together. Also not sure if we can have multiple gpgsig, I\n   haven't checked the code.\n\n - I skipped the experimental loose object format (it's what it's\n   called in sha1_file.c). I think we can call it deprecated and move\n   on.\n\n - Do we assume tag/commit header in utf-8 or ascii?\n\n - We don't do any encoding on ident strings, right?\n\nMostly-written-by: Jonathan Nieder <jrnieder@gmail.com>\nMostly-written-by: Junio C Hamano <gitster@pobox.com>\nRemaining-stolen-from: Documentation/user-manual.txt\nSigned-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n---\n Documentation/git-object.txt |  273 ++++++++++++++++++++++++++++++++++++++++++\n 1 files changed, 273 insertions(+), 0 deletions(-)\n create mode 100644 Documentation/git-object.txt\n\ndiff --git a/Documentation/git-object.txt b/Documentation/git-object.txt\nnew file mode 100644\nindex 0000000..359af37\n--- /dev/null\n+++ b/Documentation/git-object.txt\n@@ -0,0 +1,273 @@\n+git-object(5)\n+=============\n+\n+NAME\n+----\n+git-object - Git object format\n+\n+SYNOPSIS\n+--------\n+$GIT_DIR/objects/*\n+\n+OBJECT ON-DISK REPRESENTATIONS\n+------------------------------\n+Objects can be stored on disk as loose (unpacked) objects or\n+in packs. Loose objects are in $GIT_DIR/objects/[0-9a-f][0-9a-f]\n+directories. Packs are $GIT_DIR/objects/pack/pack-*.pack. Each pack\n+has a corresponding index file to speed up pack access.\n+\n+Object SHA-1\n+~~~~~~~~~~~~\n+An object SHA-1 is calculated on its header and payload. The content\n+to be consumed by SHA-1 calculation is:\n+\n+- Object type in ascii, either \"commit\", \"tree\", \"tag\" or \"blob\"\n+  (without quotes)\n+- One space (ASCII SP)\n+- The payload length in ascii canonical decimal format\n+- ASCII NUL\n+- Object payload\n+\n+Loose objects\n+~~~~~~~~~~~~~\n+Loose objects are simply a compressed form using zlib(3) of the\n+object's header and payload, as described in Object SHA-1 section\n+above.\n+\n+Packed objects\n+~~~~~~~~~~~~~~\n+FIXME maybe include Documentation/pack-format.txt\n+\n+BLOB OBJECTS\n+------------\n+Blob object payload is file data.\n+\n+TREE OBJECTS\n+------------\n+Tree object payload contains a list of entries, each with a mode,\n+object type, object name, and filename, sorted by filename. It\n+represents the contents of a single directory tree.\n+\n+The object type may be a blob, representing the contents of a file,\n+another tree, representing the contents of a subdirectory, or a commit\n+(representing a subproject). Since trees and blobs, like all other\n+objects, are named by a hash of their contents, two trees have the\n+same object name if and only if their contents (including,\n+recursively, the contents of all subdirectories) are identical. This\n+allows git to quickly determine the differences between two related\n+tree objects, since it can ignore any entries with identical object\n+names.\n+\n+Note that the files all have mode 644 or 755: git actually only pays\n+attention to the executable bit.\n+\n+Encoding\n+~~~~~~~~\n+Entries are of variable length and self-delimiting. Each entry\n+consists of\n+\n+- a POSIX file mode in octal ascii representation, no 0 padding to the\n+  left\n+- exactly one space (ASCII SP)\n+- filename for the entry, as a NUL-terminated string\n+- 20-byte binary object name\n+\n+The mode should be 100755 (executable file), 100644 (regular file),\n+120000 (symlink), 40000 (subdirectory), or 160000 (subproject), with\n+no leading zeroes. Modes with one leading zero and the synonym 100664\n+for 100644 are also accepted for historical reasons. Other modes are\n+not accepted.\n+\n+The filename may be an arbitrary nonempty string of bytes, as long as\n+it contains no '/' or NUL character.\n+\n+The associated object must be a valid blob if the mode indicates a\n+file or symlink, tree if it indicates a subdirectory, or commit if it\n+indicates a subproject. The blob associated to a symlink entry\n+indicates the link target and its content not have any embedded NULs.\n+\n+Sorting\n+~~~~~~~\n+Entries are sorted by memcmp(3) on file name. No duplicate file names\n+allowed.\n+\n+COMMIT OBJECT\n+-------------\n+The commit object links a physical state of a tree with a description\n+of how we got there and why. Commit object payload contains the\n+associated tree SHA-1, parent commits's SHA-1, author and comitter\n+information.\n+\n+------------------------------------------------\n+$ git cat-file commit 81d48f0aee54\n+tree 093f37084c133795e4ce71befa57185328737171\n+parent f5e4e20faa1eee3feaa0394897bbd1aca544e809\n+parent 661db794eb8179c7bea02f159bb691a2fff4a8e0\n+parent 14c173eb63432ba5d0783b6c4b23a8fe0c76fb0f\n+author Linus Torvalds <torvalds@linux-foundation.org> 1326576355 -0800\n+committer Linus Torvalds <torvalds@linux-foundation.org> 1326576355 -0800\n+mergetag object 661db794eb8179c7bea02f159bb691a2fff4a8e0\n+ type commit\n+ tag devicetree-for-linus\n+ tagger Grant Likely <grant.likely@secretlab.ca> 1326520038 -0700\n+ \n+ 2nd set of device tree changes for v3.3\n+ -----BEGIN PGP SIGNATURE-----\n+ Version: GnuPG v1.4.11 (GNU/Linux)\n+ \n+ iQIcBAABAgAGBQJPERbzAAoJEEFnBt12D9kBmDIP/R9Vspc6yhjSAEvdp/VET2gi\n+ TgAQfdp4VuYjjIt4cUPO5UQU9kw478GjTuP2blZEC9DlG1jSf/L8U+A7FHJIVVzU\n+ QfjwV1Lqaqk+sQQ1bsp2ixbesKECmqU9IweOIFmn0U2ZD+xlPFIpE2iTKEqymejf\n+ PVZsFlkVmhQZgudPNieyZMjQpQ9hEb6UcSfXT//nmoRRxCL/PiMHGRx3UdS3eRe7\n+ FApSW0Mty/PD07QXPsDjg1GvK59Gf6R1/4Bd31+rXEz9yaxf4I4I02fL553NDVIt\n+ tAPfo/4YKW1rLMWQRkAUqCaMk9v/DWxeWYbbiJNZ2R3kys9o8k26XXxvcuYnecS2\n+ G8DDJpmOikbN3Gvlskh40Tn3TJb5Wlgc7o/10L/fq6FovS4Uk7yUeFMqXUYfl8TU\n+ ziIlrlt9IGabXBN4JKJl3OabgkeO+Oz9DKhTQFJLY4/121LAtFVk3xd316mY+wpX\n+ mI83VmWMlp3sK+OLr+UdMTCXZvSIpu3KlGKMpAssHKUKxIV20NHLFNbm94/ywXBn\n+ Zb8arjcv7+WzwhSqQJj851cq4/sEYx5HB4wU5Nm5SXBwcO3ixiij6lHCoHU+NudR\n+ eyPIFLfrzwnUu3yTRgUfAnkgOce+2I+vUsU4pXUR6FyK73wSmm0+4WXQfB+OBlwD\n+ 2O1RjZedZCb6zzf17H2k\n+ =mup8\n+ -----END PGP SIGNATURE-----\n+mergetag object 14c173eb63432ba5d0783b6c4b23a8fe0c76fb0f\n+ type commit\n+ tag spi-for-linus\n+ tagger Grant Likely <grant.likely@secretlab.ca> 1326520366 -0700\n+ \n+ SPI bug fixes for v3.3\n+ -----BEGIN PGP SIGNATURE-----\n+ Version: GnuPG v1.4.11 (GNU/Linux)\n+ \n+ iQIcBAABAgAGBQJPERgyAAoJEEFnBt12D9kBRMsP/RBv6kWIb/qD7yJhrdbzJ4Tv\n+ 1f7coSytuHupZVpxJstELKPugRmp2R6YeFbKw8P4P/12233Q0FcdKTF6ZE2h3cBp\n+ bfCtyyzlFeY/nMfJKkwh37x2fHxNHynCCJEjHhecLday7NKQoTmmafivTfVmolWK\n+ /MGjDarTAzC1FaP1xpBnuiI8eCr5WIgb4WmtvOmxIntVT077xggdJLL/Co7fBCqn\n+ iibz3U/VyC68kQTGw6ELhnW1d7doHp7H3DJ2gPsh6lzpbv8JAnOMPpD+3Me1DVHE\n+ Ay0kxPHV4bqnDyB+uEGppUiNoaTd5InrMAw+udDad60TMwOZzIvMkgxo0PIVM9Mm\n+ k6mCcE2+TSnJetueX3cfrS5bRTPxUX7KRDC/WSp67/QPmelbYeRDLR7hrrQVqOPq\n+ 5hIKMfz/kTBXcaXk643TEveaZlMuOZxHBYAvsbu5BX/3SQqYFS4POdxdeZVnUf54\n+ ITHhftBtrXacCsjKujp0xmKCIpF+8v3yKRxGEQssByv8v+CaymNrEls2vTF8tn5P\n+ sAIjPFJYG+IHtDMIsTHOvSPA7uwWYsOVHFEYsbC1758esiBD8+qtfvFS3jAH99z+\n+ v2/aGsfMnjYEIsRtSm7PVTybJAo22Gr62yE/Q+rP//O0JaDahgdm009MjUo6BSgg\n+ XNhZjQRYAYEExMTjJ2TK\n+ =q39P\n+ -----END PGP SIGNATURE-----\n+\n+Merge tags 'devicetree-for-linus' and 'spi-for-linus' of git://git.secretlab.ca/git/linux-2.6\n+\n+2nd set of device tree changes and SPI bug fixes for v3.3\n+\n+* tag 'devicetree-for-linus' of git://git.secretlab.ca/git/linux-2.6:\n+  of/irq: Add interrupts-names property to name an irq resource\n+  of/address: Add reg-names property to name an iomem resource\n+\n+* tag 'spi-for-linus' of git://git.secretlab.ca/git/linux-2.6:\n+  spi/tegra: depend instead of select TEGRA_SYSTEM_DMA\n+------------------------------------------------\n+\n+More precisely, a commit object begins with of one or more lines\n+delimited by ASCII LF. The end of the header is signalled by an empty\n+line. Any remaining text after the empty line forms the commit\n+message. The header must not contain NUL.\n+\n+A \"continuation line\" in the header begins with an SP. The remainder\n+of the line, after removing that SP, is concatenated to the previous\n+line, while retaining the LF at the end of the previous line.\n+\n+When a line in the header begins with a letter other than SP, and has\n+at least one SP in it, it is called a \"field\". A field consists of the\n+\"field name\", which is the string before the first SP on the line, and\n+its \"value\", which is everything after that SP. When the value\n+consists of multiple lines, continuation lines are used.\n+\n+More than one field with the same name can appear in the header of an\n+object, and the order in which they appear is significant. A commit\n+object can contain these fields in the listed order:\n+\n+1. one \"tree\" field with the 40-character textual object name of the\n+   associated tree object\n+2. zero or more \"parent\" fields, each with 40-character textual object\n+   name of the parent commit object\n+3. one \"author\" field with an ident string\n+4. one \"committer\" field with an ident string\n+5. zero or one \"encoding\" field with an ascii string\n+6. zero or more \"mergetag\" fields with associated tag object content\n+7. zero or one \"gpgsig\" field with gpg signature content\n+\n+New kinds of fields may be added in later versions of git.\n+\n+Ident strings\n+~~~~~~~~~~~~~\n+Ident strings record who's responsible of doing something at what\n+time. For a commit, the ident string in \"author\" line records who is\n+the author of the associated changes and when the changes are\n+made. The ident string in \"committer\" line records who commits the\n+changes to the repository and at what time.\n+\n+An ident string consists of an email address and a timestamp. More\n+precisely:\n+\n+1. Optionally, a name\n+2. An email address wrapped around by `<` and `>`, followed by one\n+   space (ASCII SP)\n+3. The number of seconds since Epoch (00:00:00 UTC, January 1, 1970)\n+   followed by a space (ASCII SP)\n+4. Timezone: either plus or minus sign, followed by 4 decimal digits\n+\n+Name and email are encoded in UTF-8 and must must not contain ASCII\n+NUL characters.\n+\n+Commit encoding\n+~~~~~~~~~~~~~~~\n+Encoding field describes that encoding that the commit message is\n+encoded in. Encoding names must be recognized by iconv(3). By default,\n+commit message is in UTF-8. It's discouraged to use encodings that can\n+generate ASCII NUL characters.\n+\n+TAG OBJECTS\n+-----------\n+Tag object payload contains an object, object type, tag name, the name\n+of the person (\"tagger\") who created the tag, and a message, which may\n+contain a signature.\n+\n+------------------------------------------------\n+$ git cat-file tag v1.5.0\n+object 437b1b20df4b356c9342dac8d38849f24ef44f27\n+type commit\n+tag v1.5.0\n+tagger Junio C Hamano <junkio@cox.net> 1171411200 +0000\n+\n+GIT 1.5.0\n+-----BEGIN PGP SIGNATURE-----\n+Version: GnuPG v1.4.6 (GNU/Linux)\n+\n+iD8DBQBF0lGqwMbZpPMRm5oRAuRiAJ9ohBLd7s2kqjkKlq1qqC57SbnmzQCdG4ui\n+nLE/L9aUXdWeTFPron96DLA=\n+=2E+0\n+-----END PGP SIGNATURE-----\n+------------------------------------------------\n+\n+Tag object format resembles commit format. A tag commit may have the\n+following fields in listed order:\n+\n+1. one \"object\" field with 40-character textual object name of the\n+   tagged object\n+2. one \"type\" field with type of the tagged object (\"commit\", \"tag\",\n+   \"blob\", or \"tree\")\n+3. one \"tag\" field with the name of the tag\n+4. one \"tagger\" with an ident string\n+\n+New kinds of fields may be added in later versions of git.\n+\n+Any remaining text after the header forms the tag message. Tag message\n+has no specified encoding. Anything that does not contain ASCII NUL\n+characters are accepted.\n+\n+The object field must point to a valid object of type indicated by the\n+type field. The tag name can be an arbitrary string without NUL bytes\n+or embedded newlines; in practice it usually follows the restrictions\n+described in linkgit:git-check-ref-format[1].\n+\n+GIT\n+---\n+Part of the linkgit:git[1] suite\n-- \n1.7.8.36.g69ee2\n"},{"id":"184943","messageId":"7vlinzp67m.fsf@alter.siamese.dyndns.org","threadId":"29632","inReplyTo":"1329624946-32173-1-git-send-email-pclouds@gmail.com","subject":"Re: [PATCH/RFC v2] Document format of basic Git objects","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-02-19T08:39:41Z","receivedAt":"2012-02-19T08:39:41Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Nguyễn Thái Ngọc Duy <pclouds@gmail.com> writes:\n\n> Still draft for discussion. Of three people who participated on this\n> thread, two favor a man page (me and Jonathan), one technical/\n> (Junio), so let's put it as a man page for now.\n\nPersonally I do not have strong preference either way.\n\nThe original motivation of technical/ was that we wanted to have a place\nto keep documentation that would help ourselves, the people who write the\ninternals of git, even though we did not yet know and did not want to have\nto decide if it is a good idea to expose the end users, who may not care\nabout the gory details of the internal, with reams of such documents.\n\n>  - Not sure if we fix the order of gpgsig and mergetag, or they can be\n>    mixed together. Also not sure if we can have multiple gpgsig.\n\nYou can merge a signed tag and then sign the resulting commit yourself,\nand the order of the mixing would not matter. Technically a gpgsig is a\nsignature over the commit object payload without gpgsig lines, so you\ncould have two or more gpgsigs on the same commit object but from a larger\nworkflow point of view it would not be so useful, as it would involve\nsteps like this:\n\n * You prepare a commit object, you may perhaps sign it yourself;\n\n * You expose this commit object to chosen others from whom you want their\n   signature on it;\n\n * They sign it with \"commit -S --amend\", but when they do so they make\n   sure the resulting commit has the same committer/author header as the\n   original.  Note that the resulting commits will all have different\n   object name, as the object name is over all payload including their\n   gpgsigs.\n\n * You grab the gpgsig lines from these commits, paste them into the\n   header part of the original, and then re-hash the result with\n   \"hash-object -w -t commit\".  The result will have all valid gpgsigs\n   over the payload in the commit without its gpgsig lines, because the\n   gpgsig lines from all the signers were generated that way.\n\n * Then you give the general public the resulting commit.\n\n>  - I skipped the experimental loose object format (it's what it's\n>    called in sha1_file.c). I think we can call it deprecated and move\n>    on.\n\nGood.\n\n>  - Do we assume tag/commit header in utf-8 or ascii?\n\nAuthor-ident is typically utf-8 already, so you cannot assume \"ASCII\".\n\n\n> +Object SHA-1\n> +~~~~~~~~~~~~\n> +An object SHA-1 is calculated on its header and payload. The content\n> +to be consumed by SHA-1 calculation is:\n> +\n> +- Object type in ascii, either \"commit\", \"tree\", \"tag\" or \"blob\"\n> +  (without quotes)\n> +- One space (ASCII SP)\n> +- The payload length in ascii canonical decimal format\n\n\"canonical\" may make it sound as if the document is more formal, but then\nyou would have to define what is canonical and what is not somewhere else,\nso I would suggest dropping it.\n\n\tThe length of the payload in bytes, represented as a decimal integer.\n\nAlso if you spell ASCII, consistently spell it in all-caps.        \n\n> +- ASCII NUL\n> +- Object payload\n\n----------------------------------------------------------------\n\n> +BLOB OBJECTS\n> +------------\n> +Blob object payload is file data.\n\nWhat's the significance of saying \"file data\" here?  In a document that\ndescribes the structure, saying \"is uninterpreted sequence of bytes\" is\nmore accurate (the important point is that git does not care what it is)\nand covers cases where blob was recorded with \"hash-object -w --stdin\"\nwhere no such \"file data\" has ever existed in a 'file\". Also a blob may\nrecord contents of a symbolic link ;-).\n\n> +TREE OBJECTS\n> +------------\n> +Tree object payload contains a list of entries, each with a mode,\n> +object type, object name, and filename, sorted by filename. It\n> +represents the contents of a single directory tree.\n\nDrop \"object type,\" from this list.  It is inferred from the mode.  I\npersonally would prefer to say \"path\" or \"pathname\" when the entity\nreferred to may not be a regular file.  I am not sure the last sentence is\nnecessary, but if you must say something, say \"It represents a\ndirectory\". It is by definition redundant to say that a tree represents a\n\"tree\".  Replace the above with something line this:\n\n\t... entries, each with a mode, object name and path. The type of\n\tthe object is encoded in the \"mode\":\n\n        - 100644 or 100755: the object is a \"blob\" that records the\n          contents of a regular non-executable or executable file,\n          respectively, that exists at the path.\n\n\t- 120000: the object is a \"blob\" that records the contents of a\n          symbolic link that exists at the path.\n\n\t- 40000: the object is a \"tree\" that represents a subdirectory\n          that exists at the path.\n\n\t- 160000: the object is a \"commit\" that records the state of a\n          submodule that exists at the path.\n\n> +The object type may be a blob, representing the contents of a file,\n> +another tree, representing the contents of a subdirectory, or a commit\n> +(representing a subproject).\n\nand drop the above line.\n\n> +Since trees and blobs, like all other\n> +objects, are named by a hash of their contents, two trees have the\n> +same object name if and only if their contents (including,\n> +recursively, the contents of all subdirectories) are identical. This\n> +allows git to quickly determine the differences between two related\n> +tree objects, since it can ignore any entries with identical object\n> +names.\n\nIt does not make sense to say 'trees and blobs' when you explain that a\nsingle top-level tree object defines the entire tree's state.  Just say\n'trees'.  I know you would say \"I wanted to say if tree A and tree B are\nthe same except for the content of a single blob recorded at path P, the\nresult of hash for A and B would be different\", but the same can be said\nfor a submodule, so singling out 'blob' is incomplete.  Also these trees\nmay record the same set of blobs but tree B may record what tree A had at\npath P at path Q, so it is not like the only thing that matter in the tree\nis the object names.\n\nI personally do not think it is necessary to have the above paragraph at\nall in this object.\n\n> +Note that the files all have mode 644 or 755: git actually only pays\n> +attention to the executable bit.\n\nSaying 644 or 755 here is misleading as it does not match any reality\n(except for very early incarnation of git).  By rewriting the first\nparagraph, these two lines can be safely eliminated.\n\n> +Encoding\n> +~~~~~~~~\n\n\"Encoding\" is such a loaded word and does not help clarify what this\nsection is really about, which is \"format of a tree entry\", or simply\n\"Entries\".\n\n> +Entries are of variable length and self-delimiting. Each entry\n> +consists of\n> +\n> +- a POSIX file mode in octal ascii representation, no 0 padding to the\n> +  left\n\nThis is not \"a POSIX file mode\" at all.  The mode in a tree entry was\nmodelled after that, but there is no need to mention it, especially\nbecause POSIX does not define the exact bit assignment for types (the\npermission are defined from S_IXOTH to S_IRWXU and S_ISUID/S_ISGID with\nexact bit locations) and because of S_IFGITLINK which is clearly not\nPOSIX.  As we have enumerated them in the first paragraph,\n\n\tThe \"mode\" (see above).\n\nis sufficient here.\n\n> +- exactly one space (ASCII SP)\n> +- filename for the entry, as a NUL-terminated string\n\nAgain, \"pathname\" or just \"path\" for this entire document.\n\n> +- 20-byte binary object name\n> +\n> +The mode should be 100755 (executable file), 100644 (regular file),\n> +120000 (symlink), 40000 (subdirectory), or 160000 (subproject), with\n> +no leading zeroes. Modes with one leading zero and the synonym 100664\n> +for 100644 are also accepted for historical reasons. Other modes are\n> +not accepted.\n\nThis is made redundant by the first paragraph above.\n\n> +The filename may be an arbitrary nonempty string of bytes, as long as\n> +it contains no '/' or NUL character.\n\ns/, as long as it contains no/; it cannot contain any/\n\n> +The associated object must be a valid blob if the mode indicates a\n> +file or symlink, tree if it indicates a subdirectory, or commit if it\n> +indicates a subproject. The blob associated to a symlink entry\n> +indicates the link target and its content not have any embedded NULs.\n\nI doubt that we should even mention \"and its content not have ...\".  It is\nfor readlink(2) and symlink(2) to decide.\n\n> +Sorting\n> +~~~~~~~\n> +Entries are sorted by memcmp(3) on file name. No duplicate file names\n> +allowed.\n\nA sentence without a verb seen at the end of this paragraph.\n\n\n> +COMMIT OBJECT\n> +-------------\n> +The commit object links a physical state of a tree with a description\n> +of how we got there and why.\n\nWhat is the intended audience and the purpose of this document?  If this\nwere to strictly define and describe the \"structure\", then \"and why\" is\ninappropriate.  It is merely the best-current-practice at the human level\nto describe the \"why\" in their commit log messages---it does not break the\nstructure if nobody explains \"why\".\n\nOn the other hand, \"how we got there\" is a good phrase to explain that by\nrefering to its immediate parents, all the previous histories are also\ndescribed.\n\n> +... Commit object payload contains the\n> +associated tree SHA-1, parent commits's SHA-1, author and comitter\n> +information.\n\ns/.$/, among other things./; as the log message is also part of the\npayload.\n\nStart by labeling what the large block of example you are going to throw\nat the reader here.\n\n> +------------------------------------------------\n> +$ git cat-file commit 81d48f0aee54\n> +tree 093f37084c133795e4ce71befa57185328737171\n> +parent f5e4e20faa1eee3feaa0394897bbd1aca544e809\n> +parent 661db794eb8179c7bea02f159bb691a2fff4a8e0\n> +parent 14c173eb63432ba5d0783b6c4b23a8fe0c76fb0f\n> +author Linus Torvalds <torvalds@linux-foundation.org> 1326576355 -0800\n> +committer Linus Torvalds <torvalds@linux-foundation.org> 1326576355 -0800\n> +mergetag object 661db794eb8179c7bea02f159bb691a2fff4a8e0\n> + type commit\n> + tag devicetree-for-linus\n> + tagger Grant Likely <grant.likely@secretlab.ca> 1326520038 -0700\n> + \n> + 2nd set of device tree changes for v3.3\n> + -----BEGIN PGP SIGNATURE-----\n> + Version: GnuPG v1.4.11 (GNU/Linux)\n> + \n> + iQIcBAABAgAGBQJPERbzAAoJEEFnBt12D9kBmDIP/R9Vspc6yhjSAEvdp/VET2gi\n> + TgAQfdp4VuYjjIt4cUPO5UQU9kw478GjTuP2blZEC9DlG1jSf/L8U+A7FHJIVVzU\n\nElide the above like so:\n\n        -----BEGIN PGP SIGNATURE-----\n        Version: GnuPG v1.4.11 (GNU/Linux)\n\n        iQIcBAABAgAGBQJPERbzAAoJEEFnBt12D9kBmDIP/R9Vspc6yhjSAEvdp/VET2gi\n        TgAQfdp4VuYjjIt4cUPO5UQU9kw478GjTuP2blZEC9DlG1jSf/L8U+A7FHJIVVzU\n\t...\n        =mup8\n        -----END PGP SIGNATURE-----\n\n> +Merge tags 'devicetree-for-linus' and 'spi-for-linus' of git://git.secretlab.ca/git/linux-2.6\n> +\n> +2nd set of device tree changes and SPI bug fixes for v3.3\n> +\n> +* tag 'devicetree-for-linus' of git://git.secretlab.ca/git/linux-2.6:\n> +  of/irq: Add interrupts-names property to name an irq resource\n> +  of/address: Add reg-names property to name an iomem resource\n> +\n> +* tag 'spi-for-linus' of git://git.secretlab.ca/git/linux-2.6:\n> +  spi/tegra: depend instead of select TEGRA_SYSTEM_DMA\n> +------------------------------------------------\n> +\n> +More precisely, a commit object begins with of one or more lines\n> +delimited by ASCII LF. The end of the header is signalled by an empty\n> +line. Any remaining text after the empty line forms the commit\n\nDrop \"More precisely, \".  Also notice that you abruptly said \"end of the\nheader\" without mentioning anything about \"header\" in the previous\nsentence.\n\n\tA commit object begins with the \"header\" part, that consists of\n\tone or more lines delimited by LF, and the \"body\" part, that\n\trecords the commit log message.  The first empty line delimits the\n\theader and the body.\n\n> +The header must not contain NUL.\n\nI vaguely recall that you made sure neither the header nor the body\ncontains NUL.\n\n> +A \"continuation line\" in the header begins with an SP. The remainder\n> +of the line, after removing that SP, is concatenated to the previous\n> +line, while retaining the LF at the end of the previous line.\n> +\n> +When a line in the header begins with a letter other than SP, and has\n> +at least one SP in it, it is called a \"field\". A field consists of the\n> +\"field name\", which is the string before the first SP on the line, and\n> +its \"value\", which is everything after that SP. When the value\n> +consists of multiple lines, continuation lines are used.\n> +\n> +More than one field with the same name can appear in the header of an\n> +object, and the order in which they appear is significant. A commit\n> +object can contain these fields in the listed order:\n\ns/can contain/contains/; as you are marking optional ones with \"zero or\".\n\n> +1. one \"tree\" field with the 40-character textual object name of the\n> +   associated tree object\n> +2. zero or more \"parent\" fields, each with 40-character textual object\n> +   name of the parent commit object\n> +3. one \"author\" field with an ident string\n> +4. one \"committer\" field with an ident string\n> +5. zero or one \"encoding\" field with an ascii string\n\ns/zero or one/optionally, one/ (not a strong preference--I just felt that\nwould be easier to read).\n\n\tAfter the above fields, other fields may follow, and new types of\n\tfields may be added in later versions of git.  Example of these\n\toptional fields are:\n\n\t- \"mergetag\" that copies the contents of a signed tag on one of\n          the parent commit;\n        - \"gpgsig\" that records a GPG signature for this commit object.\n\n> +6. zero or more \"mergetag\" fields with associated tag object content\n> +7. zero or one \"gpgsig\" field with gpg signature content\n\nand exclude these two from the numbering above to make it clear they are\noptional.\n\n> +Ident strings\n> +~~~~~~~~~~~~~\n> +Ident strings record who's responsible of doing something at what\n> +time. For a commit, the ident string in \"author\" line records who is\n> +the author of the associated changes and when the changes are\n\ns/are/were/, perhaps?  Again, what the purpose of this document?  If this\nwere more than to strictly describe the \"structure\", it is OK and even\npreferable to leave the meaning the \"author\" as vague, but if this were\nalso to suggest the best current practice interpretation, it may be worth\nto add something like\n\n\tThere may be a case where it is difficult to attribute a commit to\n        a single author; think of it as recording the primary contact, the\n        person to ask any questions about the commit if needed later.\n\n> +made. The ident string in \"committer\" line records who commits the\n\ns/commits/committed/, perhaps?\n\n> +changes to the repository and at what time.\n> +\n> +An ident string consists of an email address and a timestamp. More\n> +precisely:\n\ns/of an email/of a name, an email/;\ns/. More precisely:/:/;\n\n> +1. Optionally, a name\n> +2. An email address wrapped around by `<` and `>`, followed by one\n> +   space (ASCII SP)\n\nThe above makes it sound as if \"A U Thor<author@example.xz>\" is usual and\nvalid.  How about \n\n\t1. A name, followed by one ASCII SP\n\nand after this enumeration, say something like:\n\n\tName may be missing in commit objects produced by repository\n\tconversion from other SCMs that do not have it.  Name and email\n\tare typically encoded in UTF-8.\n\neven though I am not sure the last sentence should be in this document.\n\n> +3. The number of seconds since Epoch (00:00:00 UTC, January 1, 1970)\n> +   followed by a space (ASCII SP)\n> +4. Timezone: either plus or minus sign, followed by 4 decimal digits\n> +\n> +Name and email are encoded in UTF-8 and must must not contain ASCII\n> +NUL characters.\n\nDrop \" and must must ...characters\"; you already said that the header does\nnot have any NUL.  As I already said, I am not sure if you should mention\n\"UTF-8\" at all in this document.\n\n> +Commit encoding\n> +~~~~~~~~~~~~~~~\n> +Encoding field describes that encoding that the commit message is\n> +encoded in.\n\ns/that encoding that/the character encoding in which/;\ns/encoded in/recorded/;\n\n> +... Encoding names must be recognized by iconv(3). By default,\n> +commit message is in UTF-8. It's discouraged to use encodings that can\n> +generate ASCII NUL characters.\n\nHere we would probably want to have a paragraph each for \"mergetag\" and\n\"gpgsig\".\n\n> +TAG OBJECTS\n> +-----------\n> +Tag object payload contains an object, object type, tag name, the name\n> +of the person (\"tagger\") who created the tag, and a message, which may\n> +contain a signature.\n\ns/a signature/a signature at the end/;\n\n> +------------------------------------------------\n> +$ git cat-file tag v1.5.0\n> +object 437b1b20df4b356c9342dac8d38849f24ef44f27\n> +type commit\n> +tag v1.5.0\n> +tagger Junio C Hamano <junkio@cox.net> 1171411200 +0000\n> +\n> +GIT 1.5.0\n> +-----BEGIN PGP SIGNATURE-----\n> +Version: GnuPG v1.4.6 (GNU/Linux)\n> +\n> +iD8DBQBF0lGqwMbZpPMRm5oRAuRiAJ9ohBLd7s2kqjkKlq1qqC57SbnmzQCdG4ui\n> +nLE/L9aUXdWeTFPron96DLA=\n> +=2E+0\n> +-----END PGP SIGNATURE-----\n> +------------------------------------------------\n> +\n> +Tag object format resembles commit format. A tag commit may have the\n> +following fields in listed order:\n> +\n> +1. one \"object\" field with 40-character textual object name of the\n> +   tagged object\n> +2. one \"type\" field with type of the tagged object (\"commit\", \"tag\",\n> +   \"blob\", or \"tree\")\n> +3. one \"tag\" field with the name of the tag\n> +4. one \"tagger\" with an ident string\n> +\n> +New kinds of fields may be added in later versions of git.\n> +\n> +Any remaining text after the header forms the tag message. Tag message\n> +has no specified encoding. Anything that does not contain ASCII NUL\n> +characters are accepted.\n> +\n> +The object field must point to a valid object of type indicated by the\n> +type field. The tag name can be an arbitrary string without NUL bytes\n> +or embedded newlines; in practice it usually follows the restrictions\n> +described in linkgit:git-check-ref-format[1].\n\nA description of how the signature part is formed needs to come here.\n\n> +GIT\n> +---\n> +Part of the linkgit:git[1] suite\n"},{"id":"184945","messageId":"7vboovp4lv.fsf@alter.siamese.dyndns.org","threadId":"29632","inReplyTo":"7vlinzp67m.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH/RFC v2] Document format of basic Git objects","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-02-19T09:14:20Z","receivedAt":"2012-02-19T09:14:20Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano <gitster@pobox.com> writes:\n\n> Nguyễn Thái Ngọc Duy <pclouds@gmail.com> writes:\n>\n>> Still draft for discussion.\n\nSmall clarifications and corrections.\n\n> ... from a larger\n> workflow point of view it would not be so useful, as it would involve\n> steps like this:\n>\n> ...\n>  * Then you give the general public the resulting commit.\n\nThe point of the above is *not* that it involves hash-object or having to\npreserve both author and committer dates when secondary signers sign the\ncommit---these are something the tool *could* learn to assist. The point\nis that adding more signatures *must* change the resulting commit object\nname, making it necessary not to expose it to the general public in order\nto avoid history rewinding, which *is* what makes it \"not so useful\".\n\nAnd that is why I didn't add such a tool support to help producing the end\nresult that wouldn't be so useful anyway.\n\n>> +TREE OBJECTS\n>> +------------\n>> +Tree object payload contains a list of entries, each with a mode,\n>> +object type, object name, and filename, sorted by filename. It\n>> +represents the contents of a single directory tree.\n>\n> Drop \"object type,\" from this list.  It is inferred from the mode.  I\n> personally would prefer to say \"path\" or \"pathname\" when the entity\n> referred to may not be a regular file.\n\nThe principle is not to say \"filename\" to give an incorrect impression\nthat we are only talking about a regular file. This principle applies to\npathnames in general (i.e. covers what is recorded in the index, too), but\nbecause we are talking about an entry in a tree, \"pathname component\" is\neven better than \"path\" or \"pathname\", because it has a specific meaning:\none part of pathname delimited by a slash.\n\nPOSIX does use \"filename\" for this purpose (and mentions \"pathname\ncomponent\" as a synonym), but if we use the word, without clarifying that\nthis document uses it in the strict POSIX sense, the reader can easily\nmisunderstand that we mean a more general \"name of a regular file\".\n\n>> +The object type may be a blob, representing the contents of a file,\n>> +another tree, representing the contents of a subdirectory, or a commit\n>> +(representing a subproject).\n>\n> and drop the above line.\n\nShould be obvious from the context, but I meant \"drop the above three\nlines\".\n\n> I personally do not think it is necessary to have the above paragraph at\n> all in this object.\n\ns/in this object/in this document/;\n\n>> +Encoding\n>> +~~~~~~~~\n>\n> \"Encoding\" is such a loaded word and does not help clarify what this\n> section is really about, which is \"format of a tree entry\", or simply\n> \"Entries\".\n>\n>> +Entries are of variable length and self-delimiting. Each entry\n>> +consists of\n\nActually, title this section as \"Tree Entries\", and begin the paragraph\nwith\n\n\tTre entries are of ...delimiting. Each entry consists of...\n\n>> +Ident strings\n>> +~~~~~~~~~~~~~\n>> +Ident strings record who's responsible of doing something at what\n>> +time. For a commit, the ident string in \"author\" line records who is\n>> +the author of the associated changes and when the changes are\n>\n> s/are/were/, perhaps?  Again, what the purpose of this document?  If this\n> were more than to strictly describe the \"structure\", it is OK and even\n\ns/ more than to/to/;\n\n> preferable to leave the meaning the \"author\" as vague, but if this were\n> also to suggest the best current practice interpretation, it may be worth\n> to add something like\n>\n> \tThere may be a case where it is difficult to attribute a commit to\n>         a single author; think of it as recording the primary contact, the\n>         person to ask any questions about the commit if needed later.\n"},{"id":"184949","messageId":"A716A27D407F401DAAC373027D5CFF2A@PhilipOakley","threadId":"29632","inReplyTo":"1329624946-32173-1-git-send-email-pclouds@gmail.com","subject":"Manually decoding a git object","fromName":"Philip Oakley","fromEmail":"philipoakley@iee.org","sentAt":null,"receivedAt":"2012-02-19T15:29:57Z","isPatch":false,"sender":{"key":"philipoakley@iee.email","avatar":"https://avatars.githubusercontent.com/u/914343?v=4"},"body":"If I have a renamed file which is a git object, such a \"Git_Object\", was\n8c-something-or-other, what is the easiest way of examining / decoding /\nrecreating the original file (either as its sha1, or a cat-file).\n\nI don't appear to be able to unzip the file in its raw format...  I'm using\nMsysgit on windows XP.\n\nPhilip\n"},{"id":"184965","messageId":"CAMocUqRNNXpGOJ2ZG8bWpkieKkVfoo0=Lkevjw+E3Tssw68Hwg@mail.gmail.com","threadId":"29632","inReplyTo":"A716A27D407F401DAAC373027D5CFF2A@PhilipOakley","subject":"Re: Manually decoding a git object","fromName":"徐迪","fromEmail":"xudifsd@gmail.com","sentAt":"2012-02-20T04:45:46Z","receivedAt":"2012-02-20T04:45:46Z","isPatch":false,"sender":{"key":"xudifsd@gmail.com","avatar":"https://gravatar.com/avatar/260814589d486b8dba6ff105f193d831bf9de466d688a20c53f7a516c7ffe146?d=mp&s=160"},"body":"2012/2/20 Philip Oakley <philipoakley@iee.org>:\n> If I have a renamed file which is a git object, such a \"Git_Object\", was\n> 8c-something-or-other, what is the easiest way of examining / decoding /\n> recreating the original file (either as its sha1, or a cat-file).\n>\nI don't think I fully understood what you mean, I assume you just move\nan object file from $GIT_DIR/objects/ to somewhere and rename it,\nlet's call it \"obj\", so if you want to exam its content you can just\nsimply call \"git cat-file -p obj\". And you can also use \"git cat-file\n-t obj\" to exam its object type. If it's a blob you can use \"git\ncat-file -p obj > original\" to recreate it, else it's meaningless to\nrecreate it.\n"},{"id":"184971","messageId":"B268B28A80B446BBADED9B3B6BA4B205@PhilipOakley","threadId":"29632","inReplyTo":"CAMocUqRNNXpGOJ2ZG8bWpkieKkVfoo0=Lkevjw+E3Tssw68Hwg@mail.gmail.com","subject":"Re: Manually decoding a git object","fromName":"Philip Oakley","fromEmail":"philipoakley@iee.org","sentAt":null,"receivedAt":"2012-02-20T07:58:17Z","isPatch":false,"sender":{"key":"philipoakley@iee.email","avatar":"https://avatars.githubusercontent.com/u/914343?v=4"},"body":"From: \"??\" <xudifsd@gmail.com> Sent: Monday, February 20, 2012 4:45 AM\n> 2012/2/20 Philip Oakley <philipoakley@iee.org>:\n>> If I have a renamed file which is a git object, such a \"Git_Object\", was\n>> 8c-something-or-other, what is the easiest way of examining / decoding /\n>> recreating the original file (either as its sha1, or a cat-file).\n>>\n> I don't think I fully understood what you mean, I assume you just move\n> an object file from $GIT_DIR/objects/ to somewhere and rename it,\n> let's call it \"obj\", so if you want to exam its content you can just\n> simply call \"git cat-file -p obj\". And you can also use \"git cat-file\n> -t obj\" to exam its object type. If it's a blob you can use \"git\n> cat-file -p obj > original\" to recreate it, else it's meaningless to\n> recreate it.\n\nWhen I tried it from my home directory (not in a git directory):\n$ git cat-file -p Git-Object\nfatal: Not a git repository (or any of the parent directories): .git\n\nBecause its sha1 isn't yet known I can't put it into the correct \n.git/objects/xx/ subdirectory of an fresh 'git init', and I have located an \nunzip programme that will take the plain git object and decode it - they all \nexpect archives.\n\nI've described the background use-case at \nhttp://stackoverflow.com/questions/9341278/how-to-track-the-git-directory-in-git-in-its-own-store - \nthe edit links to a typical corporate scenario.\n\nEven just locating a zlib implementation that simply confirms the file \nstream is compressesd and deflates it would be a start.\nPhilip\n"},{"id":"184973","messageId":"87mx8dj4at.fsf@thomas.inf.ethz.ch","threadId":"29632","inReplyTo":"A716A27D407F401DAAC373027D5CFF2A@PhilipOakley","subject":"Re: Manually decoding a git object","fromName":"Thomas Rast","fromEmail":"trast@inf.ethz.ch","sentAt":"2012-02-20T08:29:46Z","receivedAt":"2012-02-20T08:29:46Z","isPatch":false,"sender":{"key":"tr@thomasrast.ch","avatar":"https://avatars.githubusercontent.com/u/153510?v=4"},"body":"\"Philip Oakley\" <philipoakley@iee.org> writes:\n\n> If I have a renamed file which is a git object, such a \"Git_Object\", was\n> 8c-something-or-other, what is the easiest way of examining / decoding /\n> recreating the original file (either as its sha1, or a cat-file).\n>\n> I don't appear to be able to unzip the file in its raw format...  I'm using\n> Msysgit on windows XP.\n\nThe SHA1 is over the decompressed object contents.  The file simply\nholds a zlib-compressed stream of those contents.  (It's pretty much\nlike gzip without the file header.)\n\nYou can use any bindings to zlib and something that does sha1, e.g. in\npython:\n\n  $ cd g/.git/objects/aa/  # my git.git\n  $ ls\n  592bda986a8380b64acd8cbb3d5bdfcbc0834d  6322a757bee31919f54edcc127608a3d724c99\n  $ python\n  Python 2.7.2 (default, Aug 19 2011, 20:41:43) [GCC] on linux2\n  Type \"help\", \"copyright\", \"credits\" or \"license\" for more information.\n  >>> import hashlib\n  >>> hashlib.sha1(open('592bda986a8380b64acd8cbb3d5bdfcbc0834d').read().decode('zlib')).digest().encode('hex')\n  'aa592bda986a8380b64acd8cbb3d5bdfcbc0834d'\n\nNotice that the first byte of the hash goes into the directory name.\n\n-- \nThomas Rast\ntrast@{inf,student}.ethz.ch\n"},{"id":"184980","messageId":"5D290031F60941368FB1A36464201DCF@PhilipOakley","threadId":"29632","inReplyTo":"87mx8dj4at.fsf@thomas.inf.ethz.ch","subject":"Re: Manually decoding a git object","fromName":"Philip Oakley","fromEmail":"philipoakley@iee.org","sentAt":null,"receivedAt":"2012-02-20T09:05:43Z","isPatch":false,"sender":{"key":"philipoakley@iee.email","avatar":"https://avatars.githubusercontent.com/u/914343?v=4"},"body":"From: \"Thomas Rast\" <trast@inf.ethz.ch> Sent: Monday, February 20, 2012 8:29 \nAM\n> \"Philip Oakley\" <philipoakley@iee.org> writes:\n>\n>> If I have a renamed file which is a git object, such a \"Git_Object\", was\n>> 8c-something-or-other, what is the easiest way of examining / decoding /\n>> recreating the original file (either as its sha1, or a cat-file).\n>>\n>> I don't appear to be able to unzip the file in its raw format...  I'm \n>> using\n>> Msysgit on windows XP.\n\nCorrection to reply to  xu's message: and I have /NOT/ located an\nunzip programme that will take the plain git object and decode it.\n\n>\n> The SHA1 is over the decompressed object contents.  The file simply\n> holds a zlib-compressed stream of those contents.  (It's pretty much\n> like gzip without the file header.)\n>\n> You can use any bindings to zlib and something that does sha1, e.g. in\n> python:\n>\n>  $ cd g/.git/objects/aa/  # my git.git\n>  $ ls\n>  592bda986a8380b64acd8cbb3d5bdfcbc0834d \n> 6322a757bee31919f54edcc127608a3d724c99\n>  $ python\n>  Python 2.7.2 (default, Aug 19 2011, 20:41:43) [GCC] on linux2\n>  Type \"help\", \"copyright\", \"credits\" or \"license\" for more information.\n>  >>> import hashlib\n>  >>> \n> hashlib.sha1(open('592bda986a8380b64acd8cbb3d5bdfcbc0834d').read().decode('zlib')).digest().encode('hex')\n>  'aa592bda986a8380b64acd8cbb3d5bdfcbc0834d'\n>\n> Notice that the first byte of the hash goes into the directory name.\n>\n\nAt the moment I'm in a Catch 22 situation where I can't make the first step \nof examining the deflated contents, so I can't do all those next steps to \nget the sha1 etc.. Have I misunderstood your suggestions?\n\nPhilip \n"},{"id":"184982","messageId":"871uppbwnu.fsf@thomas.inf.ethz.ch","threadId":"29632","inReplyTo":"5D290031F60941368FB1A36464201DCF@PhilipOakley","subject":"Re: Manually decoding a git object","fromName":"Thomas Rast","fromEmail":"trast@inf.ethz.ch","sentAt":"2012-02-20T10:56:37Z","receivedAt":"2012-02-20T10:56:37Z","isPatch":false,"sender":{"key":"tr@thomasrast.ch","avatar":"https://avatars.githubusercontent.com/u/153510?v=4"},"body":"Philip Oakley <philipoakley@iee.org> writes:\n\n> From: \"Thomas Rast\" <trast@inf.ethz.ch> Sent: Monday, February 20,\n> 2012 8:29 AM\n>>\n>> The SHA1 is over the decompressed object contents.  The file simply\n>> holds a zlib-compressed stream of those contents.  (It's pretty much\n>> like gzip without the file header.)\n>>\n>> You can use any bindings to zlib and something that does sha1, e.g. in\n>> python:\n>>\n>>  $ cd g/.git/objects/aa/  # my git.git\n>>  $ ls\n>>  592bda986a8380b64acd8cbb3d5bdfcbc0834d\n>> 6322a757bee31919f54edcc127608a3d724c99\n>>  $ python\n>>  Python 2.7.2 (default, Aug 19 2011, 20:41:43) [GCC] on linux2\n>>  Type \"help\", \"copyright\", \"credits\" or \"license\" for more information.\n>>  >>> import hashlib\n>>  >>>\n>> hashlib.sha1(open('592bda986a8380b64acd8cbb3d5bdfcbc0834d').read().decode('zlib')).digest().encode('hex')\n>>  'aa592bda986a8380b64acd8cbb3d5bdfcbc0834d'\n>>\n>> Notice that the first byte of the hash goes into the directory name.\n>>\n>\n> At the moment I'm in a Catch 22 situation where I can't make the first\n> step of examining the deflated contents, so I can't do all those next\n> steps to get the sha1 etc.. Have I misunderstood your suggestions?\n\nHuh?  The method I showed does not rely on knowing the SHA1.  The fact\nthat I used it on a properly filed away (by its SHA1) object file is\nimmaterial, if perhaps confusing.\n\nI can untangle that python expression for you:\n\nhashlib.sha1(foo).digest()      gives the SHA1 digest of the string foo, as a (binary) string\nfoo.encode('hex')               turns foo from (binary) string into its hex representation\nopen('filename').read()         opens the file called filename, and returns its whole contents\nfoo.decode('zlib')              applies the zlib decompressor to foo, and returns the resulting data\n\nSo that trick works for any file[*], and you can then use its results to\nfile it back where it needs to go.\n\n\n[*] that is sufficiently small for Python to hold it in memory, but git\nshares the same problems in that department.\n\n-- \nThomas Rast\ntrast@{inf,student}.ethz.ch\n"},{"id":"184983","messageId":"CAMocUqQdBH3BhWGfFCiCLXhtw=-tRVdL9RBjma_tdoVdj2-Crw@mail.gmail.com","threadId":"29632","inReplyTo":"871uppbwnu.fsf@thomas.inf.ethz.ch","subject":"Re: Manually decoding a git object","fromName":"徐迪","fromEmail":"xudifsd@gmail.com","sentAt":"2012-02-20T11:39:32Z","receivedAt":"2012-02-20T11:39:32Z","isPatch":false,"sender":{"key":"xudifsd@gmail.com","avatar":"https://gravatar.com/avatar/260814589d486b8dba6ff105f193d831bf9de466d688a20c53f7a516c7ffe146?d=mp&s=160"},"body":"2012/2/20 Thomas Rast <trast@inf.ethz.ch>:\n> Philip Oakley <philipoakley@iee.org> writes:\n>\n>> From: \"Thomas Rast\" <trast@inf.ethz.ch> Sent: Monday, February 20,\n>> 2012 8:29 AM\n>>>\n>>> The SHA1 is over the decompressed object contents.  The file simply\n>>> holds a zlib-compressed stream of those contents.  (It's pretty much\n>>> like gzip without the file header.)\n>>>\n>>> You can use any bindings to zlib and something that does sha1, e.g. in\n>>> python:\n>>>\n>>>  $ cd g/.git/objects/aa/  # my git.git\n>>>  $ ls\n>>>  592bda986a8380b64acd8cbb3d5bdfcbc0834d\n>>> 6322a757bee31919f54edcc127608a3d724c99\n>>>  $ python\n>>>  Python 2.7.2 (default, Aug 19 2011, 20:41:43) [GCC] on linux2\n>>>  Type \"help\", \"copyright\", \"credits\" or \"license\" for more information.\n>>>  >>> import hashlib\n>>>  >>>\n>>> hashlib.sha1(open('592bda986a8380b64acd8cbb3d5bdfcbc0834d').read().decode('zlib')).digest().encode('hex')\n>>>  'aa592bda986a8380b64acd8cbb3d5bdfcbc0834d'\n>>>\n>>> Notice that the first byte of the hash goes into the directory name.\n>>>\nI think Thomas got the point.\n\n> When I tried it from my home directory (not in a git directory):\n> $ git cat-file -p Git-Object\n> fatal: Not a git repository (or any of the parent directories): .git\n\nthis is because git will first do a git-dir-search, if you're current\nwork dir is not within git repo, it will die.\nI really do not know how you get thing that mess. From the link[1] you\ngive, i think you just want to clone a repo across computer not by\nnetwork, if so this[2] will be helpful.\n\n[1]:http://stackoverflow.com/questions/9343260/what-after-git-unpack-objects-to-get-the-actual-file\n[2]:http://progit.org/2010/03/10/bundles.html\n"},{"id":"184988","messageId":"CACsJy8CEeZPf55idLB9NE-rf--ySmZh_9gkMc_zo0VmiVftMUg@mail.gmail.com","threadId":"29632","inReplyTo":"7vlinzp67m.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH/RFC v2] Document format of basic Git objects","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2012-02-20T13:55:28Z","receivedAt":"2012-02-20T13:55:28Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"2012/2/19 Junio C Hamano <gitster@pobox.com>:\n>>  - Do we assume tag/commit header in utf-8 or ascii?\n>\n> Author-ident is typically utf-8 already, so you cannot assume \"ASCII\".\n\nI wonder if anyone puts non utf-8 strings in there, or could we\nenforce utf-8 (i.e. validate and reject non utf-8 strings) and accept\nencoded word syntax (rfc 2047) with the help of the new\n$GIT_IDENT_ENCODING variable. The \"accept ...\" part can wait until\nsomeone is hit by \"utf-8 only\" check and steps up.\n\nBy the same reasoning, maybe we should declare tag content is utf-8\nonly, until someone needs and adds \"encoding\" support for it.\n\n>> +The filename may be an arbitrary nonempty string of bytes, as long as\n>> +it contains no '/' or NUL character.\n>\n> s/, as long as it contains no/; it cannot contain any/\n\nPathname also cannot be \".\" nor \"..\", I suppose.\nSince we also support Windows, should '\\\\' be banned too? ... probably\nnot worth it.\n\n>> +The header must not contain NUL.\n>\n> I vaguely recall that you made sure neither the header nor the body\n> contains NUL.\n\nOne of the purposes of this document is to note all constraints and\nlimitations (another one is a reference for users who want to dig deep\nin git data structure without looking at the code). The problem with\nhandling NUL probably only exists in C Git (and maybe libgit2). I'll\nturn that to \"should not contain NUL\".\n-- \nDuy\n"},{"id":"185001","messageId":"20120220161101.GA27105@sigill.intra.peff.net","threadId":"29632","inReplyTo":"CACsJy8CEeZPf55idLB9NE-rf--ySmZh_9gkMc_zo0VmiVftMUg@mail.gmail.com","subject":"Re: [PATCH/RFC v2] Document format of basic Git objects","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2012-02-20T16:11:01Z","receivedAt":"2012-02-20T16:11:01Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Feb 20, 2012 at 08:55:28PM +0700, Nguyen Thai Ngoc Duy wrote:\n\n> > Author-ident is typically utf-8 already, so you cannot assume \"ASCII\".\n> \n> I wonder if anyone puts non utf-8 strings in there, or could we\n> enforce utf-8 (i.e. validate and reject non utf-8 strings) and accept\n> encoded word syntax (rfc 2047) with the help of the new\n> $GIT_IDENT_ENCODING variable. The \"accept ...\" part can wait until\n> someone is hit by \"utf-8 only\" check and steps up.\n\nI was just having a similar discussion with libgit2 folks, who were\nwondering if there would ever be non-utf8 in there. When we call\n\"reencode_commit_message\", it looks like we do the whole object. In\nother words, your author name _must_ match any encoding you specify in\nthe \"encoding\" header.\n\nI.e., if you do:\n\n  # latin1 é\n  e=`printf '\\xe9'`\n\n  export GIT_AUTHOR_NAME=\"P${e}ff King\"\n  git init\n  git config i18n.commitencoding iso8859-1\n  touch foo && git add foo &&\n  git commit --allow-empty -m \"more latin1 ${e}ncoding\"\n\nboth the name and the message should show fine on your utf8 terminal if\nyou do this:\n\n  git config i18n.logoutputencoding utf8\n  git show\n\nAnd similarly, we do the right thing in format-patch, both with and\nwithout logoutputencoding set:\n\n  $ git format-patch --root --stdout | grep -Ei \"^(from|subject):\"\n  From: =?iso8859-1?q?P=E9ff=20King?= <peff@peff.net>\n  Subject: [PATCH] =?iso8859-1?q?more=20latin1=20=E9ncoding?=\n\n  $ git config i18n.logoutputencoding utf8\n  $ git format-patch --root --stdout | grep -Ei \"^(from|subject):\"\n  From: =?utf8?q?P=C3=A9ff=20King?= <peff@peff.net>\n  Subject: [PATCH] =?utf8?q?more=20latin1=20=C3=A9ncoding?=\n\n(where 0xc3a9 is the utf8 equivalent of latin1 0xe9).\n\nSo I have no idea if people are using it or not, but it is actually\nusable.\n\n-Peff\n"},{"id":"185003","messageId":"EF3174E8E5D5433CBC44E912309946A8@PhilipOakley","threadId":"29632","inReplyTo":"871uppbwnu.fsf@thomas.inf.ethz.ch","subject":"Re: Manually decoding a git object","fromName":"Philip Oakley","fromEmail":"philipoakley@iee.org","sentAt":null,"receivedAt":"2012-02-20T18:27:08Z","isPatch":false,"sender":{"key":"philipoakley@iee.email","avatar":"https://avatars.githubusercontent.com/u/914343?v=4"},"body":"From: \"Thomas Rast\" <trast@inf.ethz.ch> Sent: Monday, February 20, 2012\n10:56 AM\n> Philip Oakley <philipoakley@iee.org> writes:\n>\n>> From: \"Thomas Rast\" <trast@inf.ethz.ch> Sent: Monday, February 20,\n>> 2012 8:29 AM\n>>>\n>>> The SHA1 is over the decompressed object contents.  The file simply\n>>> holds a zlib-compressed stream of those contents.  (It's pretty much\n>>> like gzip without the file header.)\n>>>\n>>> You can use any bindings to zlib and something that does sha1, e.g. in\n>>> python:\n>>>\n>>>  $ cd g/.git/objects/aa/  # my git.git\n>>>  $ ls\n>>>  592bda986a8380b64acd8cbb3d5bdfcbc0834d\n>>> 6322a757bee31919f54edcc127608a3d724c99\n>>>  $ python\n>>>  Python 2.7.2 (default, Aug 19 2011, 20:41:43) [GCC] on linux2\n>>>  Type \"help\", \"copyright\", \"credits\" or \"license\" for more information.\n>>>  >>> import hashlib\n>>>  >>>\n>>> hashlib.sha1(open('592bda986a8380b64acd8cbb3d5bdfcbc0834d').read().decode('zlib')).digest().encode('hex')\n>>>  'aa592bda986a8380b64acd8cbb3d5bdfcbc0834d'\n>>>\n>>> Notice that the first byte of the hash goes into the directory name.\n>>>\n>>\n>> At the moment I'm in a Catch 22 situation where I can't make the first\n>> step of examining the deflated contents, so I can't do all those next\n>> steps to get the sha1 etc.. Have I misunderstood your suggestions?\n>\n> Huh?  The method I showed does not rely on knowing the SHA1.  The fact\n> that I used it on a properly filed away (by its SHA1) object file is\n> immaterial, if perhaps confusing.\n>\n> I can untangle that python expression for you:\n>\n> hashlib.sha1(foo).digest()      gives the SHA1 digest of the string foo,\n> as a (binary) string\n> foo.encode('hex')               turns foo from (binary) string into its\n> hex representation\n> open('filename').read()         opens the file called filename, and\n> returns its whole contents\n> foo.decode('zlib')              applies the zlib decompressor to foo, and\n> returns the resulting data\n>\n> So that trick works for any file[*], and you can then use its results to\n> file it back where it needs to go.\n>\n>\n> [*] that is sufficiently small for Python to hold it in memory, but git\n> shares the same problems in that department.\n>\n\nI see what you mean now. I'll need to work out how to get Python in\nmsysgit - the 'minimal' part of msys keeps on biting... That is, I didn't\nsee it (Python) in the 1.7.8 full install bash - I didn't see it anyway.\n\nI was hopeful that unzip/gunzip would have an option to simply deflate a\n(file)stream, rather than it expecting the normal file archive.\n"}]}