{"thread":{"id":"6706","subject":"git-fast-import","startedAt":"2007-02-06T02:31:11Z","lastAt":"2007-02-09T15:47:40Z","messageCount":49,"participants":["Shawn O. Pearce","Nicolas Pitre","Aneesh Kumar K.V","Andy Parkins","Jakub Narebski","Alex Riesen","Linus Torvalds","Horst H. von Brand","Daniel Barkalow","Karl Hasselström","Raimund Bauer","David Woodhouse","Johannes Schindelin"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"33683","messageId":"20070206023111.GB9222@spearce.org","threadId":"6706","inReplyTo":null,"subject":"git-fast-import","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2007-02-06T02:31:11Z","receivedAt":"2007-02-06T02:31:11Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"I'm starting to get gfi (git-fast-import) prepared for a merge into\nthe main git.git tree.  For those who don't know, gfi is the result\nof my work with Jon Smirl on trying to *quickly* import the massive\nMozilla CVS repository into Git.  Recently its been getting a lot\nof attention from the KDE, OOo, Dragonfly BSD, and Qt projects.\n\nWhen exactly we merge it in will depend a lot on Junio.  It should\nbe safe to merge before 1.5.0 as its strictly new source files,\nbut we may still want to wait until after 1.5.0 is out.\n\nI'm mainly worried about breaking compliation on odd architectures.\ngfi builds, runs and has been used for production level imports\non Mac OS X, Linux and Dragonfly BSD, using both 32 bit and 64 bit\narchitectures, but some of Git's other targets (e.g. AIX) haven't\nseen any testing.\n\nThe gfi code is quite stable and has been getting a lot of use\n(and discussion) lately.  A new test (t/t9300-fast-import.sh)\nhas been added and now, finally, documentation\n(Documentation/git-fast-import.txt).\n\nAs gfi is 1962 lines of C and its development history consists of\n74 commits made over the span of 7 months (first commit was Aug 5,\n2006) and several versions of core Git code (which gfi calls into,\nand which has gone through some non-trivial changes during that\ntime), I'm going to ask Junio to directly pull the topic branch\ninto git.git, rather than submitting it as patches.\n\nMy topic branch is published on repo.or.cz (thanks Pasky!).  I would\nencourage all parties who would have otherwise been interested in\nreviewing the patches on the mailing list to clone/fetch the topic\nand review it locally instead.\n\n\tgitweb: http://repo.or.cz/w/git/fastimport.git\n\tclone:  git://repo.or.cz/git/fastimport.git\n\nI'm particularly interested in feedback on the documentation,\nso I am attaching it below.\n\n-------\ngit-fast-import(1)\n==================\n\nNAME\n----\ngit-fast-import - Backend for fast Git data importers.\n\n\nSYNOPSIS\n--------\nfrontend | 'git-fast-import' [options]\n\nDESCRIPTION\n-----------\nThis program is usually not what the end user wants to run directly.\nMost end users want to use one of the existing frontend programs,\nwhich parses a specific type of foreign source and feeds the contents\nstored there to git-fast-import (gfi).\n\ngfi reads a mixed command/data stream from standard input and\nwrites one or more packfiles directly into the current repository.\nWhen EOF is received on standard input, fast import writes out\nupdated branch and tag refs, fully updating the current repository\nwith the newly imported data.\n\nThe gfi backend itself can import into an empty repository (one that\nhas already been initialized by gitlink:git-init[1]) or incrementally\nupdate an existing populated repository.  Whether or not incremental\nimports are supported from a particular foreign source depends on\nthe frontend program in use.\n\n\nOPTIONS\n-------\n--max-pack-size=<n>::\n\tMaximum size of each output packfile, expressed in MiB.\n\tThe default is 4096 (4 GiB) as that is the maximum allowed\n\tpackfile size (due to file format limitations). Some\n\timporters may wish to lower this, such as to ensure the\n\tresulting packfiles fit on CDs.\n\n--depth=<n>::\n\tMaximum delta depth, for blob and tree deltification.\n\tDefault is 10.\n\n--active-branches=<n>::\n\tMaximum number of branches to maintain active at once.\n\tSee ``Memory Utilization'' below for details.  Default is 5.\n\n--export-marks=<file>::\n\tDumps the internal marks table to <file> when complete.\n\tMarks are written one per line as `:markid SHA-1`.\n\tFrontends can use this file to validate imports after they\n\thave been completed.\n\n--branch-log=<file>::\n\tRecords every tag and commit made to a log file.  (This file\n\tcan be quite verbose on large imports.)  This particular\n\toption has been primarily intended to facilitate debugging\n\tgfi and has limited usefulness in other contexts.  It may\n\tbe removed in future versions.\n\n\nPerformance\n-----------\nThe design of gfi allows it to import large projects in a minimum\namount of memory usage and processing time.  Assuming the frontend\nis able to keep up with gfi and feed it a constant stream of data,\nimport times for projects holding 10+ years of history and containing\n100,000+ individual commits are generally completed in just 1-2\nhours on quite modest (~$2,000 USD) hardware.\n\nMost bottlenecks appear to be in foreign source data access (the\nsource just cannot extract revisions fast enough) or disk IO (gfi\nwrites as fast as the disk will take the data).  Imports will run\nfaster if the source data is stored on a different drive than the\ndestination Git repository (due to less IO contention).\n\n\nDevelopment Cost\n----------------\nA typical frontend for gfi tends to weigh in at approximately 200\nlines of Perl/Python/Ruby code.  Most developers have been able to\ncreate working importers in just a couple of hours, even though it\nis their first exposure to gfi, and sometimes even to Git.  This is\nan ideal situation, given that most conversion tools are throw-away\n(use once, and never look back).\n\n\nParallel Operation\n------------------\nLike `git-push` or `git-fetch`, imports handled by gfi are safe to\nrun alongside parallel `git repack -a -d` or `git gc` invocations,\nor any other Git operation (including `git prune`, as loose objects\nare never used by gfi).\n\nHowever, gfi does not lock the branch or tag refs it is actively\nimporting.  After EOF, during its ref update phase, gfi blindly\noverwrites each imported branch or tag ref.  Consequently it is not\nsafe to modify refs that are currently being used by a running gfi\ninstance, as work could be lost when gfi overwrites the refs.\n\n\nTechnical Discussion\n--------------------\ngfi tracks a set of branches in memory.  Any branch can be created\nor modified at any point during the import process by sending a\n`commit` command on the input stream.  This design allows a frontend\nprogram to process an unlimited number of branches simultaneously,\ngenerating commits in the order they are available from the source\ndata.  It also simplifies the frontend programs considerably.\n\ngfi does not use or alter the current working directory, or any\nfile within it.  (It does however update the current Git repository,\nas referenced by `GIT_DIR`.)  Therefore an import frontend may use\nthe working directory for its own purposes, such as extracting file\nrevisions from the foreign source.  This ignorance of the working\ndirectory also allows gfi to run very quickly, as it does not\nneed to perform any costly file update operations when switching\nbetween branches.\n\nInput Format\n------------\nWith the exception of raw file data (which Git does not interpret)\nthe gfi input format is text (ASCII) based.  This text based\nformat simplifies development and debugging of frontend programs,\nespecially when a higher level language such as Perl, Python or\nRuby is being used.\n\ngfi is very strict about its input.  Where we say SP below we mean\n*exactly* one space.  Likewise LF means one (and only one) linefeed.\nSupplying additional whitespace characters will cause unexpected\nresults, such as branch names or file names with leading or trailing\nspaces in their name, or early termination of gfi when it encounters\nunexpected input.\n\nCommands\n~~~~~~~~\ngfi accepts several commands to update the current repository\nand control the current import process.  More detailed discussion\n(with examples) of each command follows later.\n\n`commit`::\n\tCreates a new branch or updates an existing branch by\n\tcreating a new commit and updating the branch to point at\n\tthe newly created commit.\n\n`tag`::\n\tCreates an annotated tag object from an existing commit or\n\tbranch.  Lightweight tags are not supported by this command,\n\tas they are not recommended for recording meaningful points\n\tin time.\n\n`reset`::\n\tReset an existing branch (or a new branch) to a specific\n\trevision.  This command must be used to change a branch to\n\ta specific revision without making a commit on it.\n\n`blob`::\n\tConvert raw file data into a blob, for future use in a\n\t`commit` command.  This command is optional and is not\n\tneeded to perform an import.\n\n`checkpoint`::\n\tForces gfi to close the current packfile, generate its\n\tunique SHA-1 checksum and index, and start a new packfile.\n\tThis command is optional and is not needed to perform\n\tan import.\n\n`commit`\n~~~~~~~~\nCreate or update a branch with a new commit, recording one logical\nchange to the project.\n\n....\n\t'commit' SP <ref> LF\n\tmark?\n\t('author' SP <name> SP LT <email> GT SP <time> SP <tz> LF)?\n\t'committer' SP <name> SP LT <email> GT SP <time> SP <tz> LF\n\tdata\n\t('from' SP <committish> LF)?\n\t('merge' SP <committish> LF)?\n\t(filemodify | filedelete)*\n\tLF\n....\n\nwhere `<ref>` is the name of the branch to make the commit on.\nTypically branch names are prefixed with `refs/heads/` in\nGit, so importing the CVS branch symbol `RELENG-1_0` would use\n`refs/heads/RELENG-1_0` for the value of `<ref>`.  The value of\n`<ref>` must be a valid refname in Git.  As `LF` is not valid in\na Git refname, no quoting or escaping syntax is supported here.\n\nA `mark` command may optionally appear, requesting gfi to save a\nreference to the newly created commit for future use by the frontend\n(see below for format).  It is very common for frontends to mark\nevery commit they create, thereby allowing future branch creation\nfrom any imported commit.\n\nThe `data` command following `committer` must supply the commit\nmessage (see below for `data` command syntax).  To import an empty\ncommit message use a 0 length data.  Commit messages are free-form\nand are not interpreted by Git.  Currently they must be encoded in\nUTF-8, as gfi does not permit other encodings to be specified.\n\nZero or more `filemodify` and `filedelete` commands may be\nincluded to update the contents of the branch prior to the commit.\nThese commands can be supplied in any order, gfi is not sensitive\nto pathname or operation ordering.\n\n`author`\n^^^^^^^^\nAn `author` command may optionally appear, if the author information\nmight differ from the committer information.  If `author` is omitted\nthen gfi will automatically use the committer's information for\nthe author portion of the commit.  See below for a description of\nthe fields in `author`, as they are identical to `committer`.\n\n`committer`\n^^^^^^^^^^^\nThe `committer` command indicates who made this commit, and when\nthey made it.\n\nHere `<name>` is the person's display name (for example\n``Com M Itter'') and `<email>` is the person's email address\n(``cm@example.com'').  `LT` and `GT` are the literal less-than (\\x3c)\nand greater-than (\\x3e) symbols.  These are required to delimit\nthe email address from the other fields in the line.  Note that\n`<name>` is free-form and may contain any sequence of bytes, except\n`LT` and `LF`.  It is typically UTF-8 encoded.\n\nThe time of the change is specified by `<time>` as the number of\nseconds since the UNIX epoc (midnight, Jan 1, 1970, UTC) and is\nwritten in base-10 notation using US-ASCII digits.  The committer's\ntimezone is specified by `<tz>` as a positive or negative offset\nfrom UTC, in minutes.  For example EST would be expressed in `<tz>`\nby ``-0500''.\n\n`from`\n^^^^^^\nOnly valid for the first commit made on this branch by this\ngfi process.  The `from` command is used to specify the commit\nto initialize this branch from.  This revision will be the first\nancestor of the new commit.\n\nOmitting the `from` command in the first commit of a new branch will\ncause gfi to create that commit with no ancestor. This tends to be\ndesired only for the initial commit of a project.  Omitting the\n`from` command on existing branches is required, as the current\ncommit on that branch is automatically assumed to be the first\nancestor of the new commit.\n\nAs `LF` is not valid in a Git refname or SHA-1 expression, no\nquoting or escaping syntax is supported within `<committish>`.\n\nHere `<committish>` is any of the following:\n\n* The name of an existing branch already in gfi's internal branch\n  table.  If gfi doesn't know the name, its treated as a SHA-1\n  expression.\n\n* A mark reference, `:<idnum>`, where `<idnum>` is the mark number.\n+\nThe reason gfi uses `:` to denote a mark reference is this character\nis not legal in a Git branch name.  The leading `:` makes it easy\nto distingush between the mark 42 (`:42`) and the branch 42 (`42`\nor `refs/heads/42`), or an abbreviated SHA-1 which happened to\nconsist only of base-10 digits.\n+\nMarks must be declared (via `mark`) before they can be used.\n\n* A complete 40 byte or abbreviated commit SHA-1 in hex.\n\n* Any valid Git SHA-1 expression that resolves to a commit.  See\n  ``SPECIFYING REVISIONS'' in gitlink:git-rev-parse[1] for details.\n\nThe special case of restarting an incremental import from the\ncurrent branch value should be written as:\n----\n\tfrom refs/heads/branch^0\n----\nThe `^0` suffix is necessary as gfi does not permit a branch to\nstart from itself, and the branch is created in memory before the\n`from` command is even read from the input.  Adding `^0` will force\ngfi to resolve the commit through Git's revision parsing library,\nrather than its internal branch table, thereby loading in the\nexisting value of the branch.\n\n`merge`\n^^^^^^^\nIncludes one additional ancestor commit, and makes the current\ncommit a merge commit.  An unlimited number of `merge` commands per\ncommit are permitted by gfi, thereby establishing an n-way merge.\nHowever Git's other tools never create commits with more than 15\nadditional ancestors (forming a 16-way merge).  For this reason\nit is suggested that frontends do not use more than 15 `merge`\ncommands per commit.\n\nHere `<committish>` is any of the commit specification expressions\nalso accepted by `from` (see above).\n\n`filemodify`\n^^^^^^^^^^\nIncluded in a `commit` command to add a new file or change the\ncontent of an existing file.  This command has two different means\nof specifying the content of the file.\n\nExternal data format::\n\tThe data content for the file was already supplied by a prior\n\t`blob` command.  The frontend just needs to connect it.\n+\n....\n\t'M' SP <mode> SP <dataref> SP <path> LF\n....\n+\nHere `<dataref>` can be either a mark reference (`:<idnum>`)\nset by a prior `blob` command, or a full 40-byte SHA-1 of an\nexisting Git blob object.\n\nInline data format::\n\tThe data content for the file has not been supplied yet.\n\tThe frontend wants to supply it as part of this modify\n\tcommand.\n+\n....\n\t'M' SP <mode> SP 'inline' SP <path> LF\n\tdata\n....\n+\nSee below for a detailed description of the `data` command.\n\nIn both formats `<mode>` is the type of file entry, specified\nin octal.  Git only supports the following modes:\n\n* `100644` or `644`: A normal (not-executable) file.  The majority\n  of files in most projects use this mode.  If in doubt, this is\n  what you want.\n* `100755` or `755`: A normal, but executable, file.\n* `140000`: A symlink, the content of the file will be the link target.\n\nIn both formats `<path>` is the complete path of the file to be added\n(if not already existing) or modified (if already existing).\n\nA `<path>` string must use UNIX-style directory seperators (forward\nslash `/`), may contain any byte other than `LF`, and must not\nstart with double quote (`\"`).\n\nIf an `LF` or double quote must be encoded into `<path>` shell-style\nquoting should be used, e.g. `\"path/with\\n and \\\" in it\"`.\n\nThe value of `<path>` must be in canoncial form. That is it must not:\n\n* contain an empty directory component (e.g. `foo//bar` is invalid),\n* end with a directory seperator (e.g. `foo/` is invalid),\n* start with a directory seperator (e.g. `/foo` is invalid),\n* contain the special component `.` or `..` (e.g. `foo/./bar` and\n  `foo/../bar` are invalid).\n\nIt is recommended that `<path>` always be encoded using UTF-8.\n\n\n`filedelete`\n^^^^^^^^^^\nIncluded in a `commit` command to remove a file from the branch.\nIf the file removal makes its directory empty, the directory will\nbe automatically removed too.  This cascades up the tree until the\nfirst non-empty directory or the root is reached.\n\n....\n\t'D' SP <path> LF\n....\n\nhere `<path>` is the complete path of the file to be removed.\nSee `filemodify` above for a detailed description of `<path>`.\n\n`mark`\n~~~~~~\nArranges for gfi to save a reference to the current object, allowing\nthe frontend to recall this object at a future point in time, without\nknowing its SHA-1.  Here the current object is the object creation\ncommand the `mark` command appears within.  This can be `commit`,\n`tag`, and `blob`, but `commit` is the most common usage.\n\n....\n\t'mark' SP ':' <idnum> LF\n....\n\nwhere `<idnum>` is the number assigned by the frontend to this mark.\nThe value of `<idnum>` is expressed in base 10 notation using\nUS-ASCII digits.  The value 0 is reserved and cannot be used as\na mark.  Only values greater than or equal to 1 may be used as marks.\n\nNew marks are created automatically.  Existing marks can be moved\nto another object simply by reusing the same `<idnum>` in another\n`mark` command.\n\n`tag`\n~~~~~\nCreates an annotated tag referring to a specific commit.  To create\nlightweight (non-annotated) tags see the `reset` command below.\n\n....\n\t'tag' SP <name> LF\n\t'from' SP <committish> LF\n\t'tagger' SP <name> SP LT <email> GT SP <time> SP <tz> LF\n\tdata\n\tLF\n....\n\nwhere `<name>` is the name of the tag to create.\n\nTag names are automatically prefixed with `refs/tags/` when stored\nin Git, so importing the CVS branch symbol `RELENG-1_0-FINAL` would\nuse just `RELENG-1_0-FINAL` for `<name>`, and gfi will write the\ncorresponding ref as `refs/tags/RELENG-1_0-FINAL`.\n\nThe value of `<name>` must be a valid refname in Git and therefore\nmay contain forward slashes.  As `LF` is not valid in a Git refname,\nno quoting or escaping syntax is supported here.\n\nThe `from` command is the same as in the `commit` command; see\nabove for details.\n\nThe `tagger` command uses the same format as `committer` within\n`commit`; again see above for details.\n\nThe `data` command following `tagger` must supply the annotated tag\nmessage (see below for `data` command syntax).  To import an empty\ntag message use a 0 length data.  Tag messages are free-form and are\nnot interpreted by Git.  Currently they must be encoded in UTF-8,\nas gfi does not permit other encodings to be specified.\n\nSigning annotated tags during import from within gfi is not\nsupported.  Trying to include your own PGP/GPG signature is not\nrecommended, as the frontend does not (easily) have access to the\ncomplete set of bytes which normally goes into such a signature.\nIf signing is required, create lightweight tags from within gfi with\n`reset`, then create the annotated versions of those tags offline\nwith the standard gitlink:git-tag[1] process.\n\n`reset`\n~~~~~~~\nCreates (or recreates) the named branch, optionally starting from\na specific revision.  The reset command allows a frontend to issue\na new `from` command for an existing branch, or to create a new\nbranch from an existing commit without creating a new commit.\n\n....\n\t'reset' SP <ref> LF\n\t('from' SP <committish> LF)?\n\tLF\n....\n\nFor a detailed description of `<ref>` and `<committish>` see above\nunder `commit` and `from`.\n\nThe `reset` command can also be used to create lightweight\n(non-annotated) tags.  For example:\n\n====\n\treset refs/tags/938\n\tfrom :938\n====\n\nwould create the lightweight tag `refs/tags/938` referring to\nwhatever commit mark `:938` references.\n\n`blob`\n~~~~~~\nRequests writing one file revision to the packfile.  The revision\nis not connected to any commit; this connection must be formed in\na subsequent `commit` command by referencing the blob through an\nassigned mark.\n\n....\n\t'blob' LF\n\tmark?\n\tdata\n....\n\nThe mark command is optional here as some frontends have chosen\nto generate the Git SHA-1 for the blob on their own, and feed that\ndirectly to `commit`.  This is typically more work than its worth\nhowever, as marks are inexpensive to store and easy to use.\n\n`data`\n~~~~~~\nSupplies raw data (for use as blob/file content, commit messages, or\nannotated tag messages) to gfi.  Data can be supplied using an exact\nbyte count or delimited with a terminating line.  Real frontends\nintended for production-quality conversions should always use the\nexact byte count format, as it is more robust and performs better.\nThe delimited format is intended primarily for testing gfi.\n\nExact byte count format:\n\n....\n\t'data' SP <count> LF\n\t<raw> LF\n....\n\nwhere `<count>` is the exact number of bytes appearing within\n`<raw>`.  The value of `<count>` is expressed in base 10 notation\nusing US-ASCII digits.  The `LF` on either side of `<raw>` is not\nincluded in `<count>` and will not be included in the imported data.\n\nDelimited format:\n\n....\n\t'data' SP '<<' <delim> LF\n\t<raw> LF\n\t<delim> LF\n....\n\nwhere `<delim>` is the chosen delimiter string.  The string `<delim>`\nmust not appear on a line by itself within `<raw>`, as otherwise\ngfi will think the data ends earlier than it really does.  The `LF`\nimmediately trailing `<raw>` is part of `<raw>`.  This is one of\nthe limitations of the delimited format, it is impossible to supply\na data chunk which does not have an LF as its last byte.\n\n`checkpoint`\n~~~~~~~~~~~~\nForces gfi to close the current packfile and start a new one.\nAs this requires a significant amount of CPU time and disk IO\n(to compute the overall pack SHA-1 checksum and generate the\ncorresponding index file) it can easily take several minutes for\na single `checkpoint` command to complete.\n\n....\n\t'checkpoint' LF\n\tLF\n....\n\nPackfile Optimization\n---------------------\nWhen packing a blob gfi always attempts to deltify against the last\nblob written.  Unless specifically arranged for by the frontend,\nthis will probably not be a prior version of the same file, so the\ngenerated delta will not be the smallest possible.  The resulting\npackfile will be compressed, but will not be optimal.\n\nFrontends which have efficient access to all revisions of a\nsingle file (for example reading an RCS/CVS ,v file) can choose\nto supply all revisions of that file as a sequence of consecutive\n`blob` commands.  This allows gfi to deltify the different file\nrevisions against each other, saving space in the final packfile.\nMarks can be used to later identify individual file revisions during\na sequence of `commit` commands.\n\nThe packfile(s) created by gfi do not encourage good disk access\npatterns.  This is caused by gfi writing the data in the order\nit is received on standard input, while Git typically organizes\ndata within packfiles to make the most recent (current tip) data\nappear before historical data.  Git also clusters commits together,\nspeeding up revision traversal through better cache locality.\n\nFor this reason it is strongly recommended that users repack the\nrepository with `git repack -a -d` after gfi completes, allowing\nGit to reorganize the packfiles for faster data access.  If blob\ndeltas are suboptimal (see above) then also adding the `-f` option\nto force recomputation of all deltas can significantly reduce the\nfinal packfile size (30-50% smaller can be quite typical).\n\nMemory Utilization\n------------------\nThere are a number of factors which affect how much memory gfi\nrequires to perform an import.  Like critical sections of core\nGit, gfi uses its own memory allocators to ammortize any overheads\nassociated with malloc.  In practice gfi tends to ammoritize any\nmalloc overheads to 0, due to its use of large block allocations.\n\nper object\n~~~~~~~~~~\ngfi maintains an in-memory structure for every object written in\nthis execution.  On a 32 bit system the structure is 32 bytes,\non a 64 bit system the structure is 40 bytes (due to the larger\npointer sizes).  Objects in the table are not deallocated until\ngfi terminates.  Importing 2 million objects on a 32 bit system\nwill require approximately 64 MiB of memory.\n\nThe object table is actually a hashtable keyed on the object name\n(the unique SHA-1).  This storage configuration allows gfi to reuse\nan existing or already written object and avoid writing duplicates\nto the output packfile.  Duplicate blobs are surprisingly common\nin an import, typically due to branch merges in the source.\n\nper mark\n~~~~~~~~\nMarks are stored in a sparse array, using 1 pointer (4 bytes or 8\nbytes, depending on pointer size) per mark.  Although the array\nis sparse, frontends are still strongly encouraged to use marks\nbetween 1 and n, where n is the total number of marks required for\nthis import.\n\nper branch\n~~~~~~~~~~\nBranches are classified as active and inactive.  The memory usage\nof the two classes is significantly different.\n\nInactive branches are stored in a structure which uses 96 or 120\nbytes (32 bit or 64 bit systems, respectively), plus the length of\nthe branch name (typically under 200 bytes), per branch.  gfi will\neasily handle as many as 10,000 inactive branches in under 2 MiB\nof memory.\n\nActive branches have the same overhead as inactive branches, but\nalso contain copies of every tree that has been recently modified on\nthat branch.  If subtree `include` has not been modified since the\nbranch became active, its contents will not be loaded into memory,\nbut if subtree `src` has been modified by a commit since the branch\nbecame active, then its contents will be loaded in memory.\n\nAs active branches store metadata about the files contained on that\nbranch, their in-memory storage size can grow to a considerable size\n(see below).\n\ngfi automatically moves active branches to inactive status based on\na simple least-recently-used algorithm.  The LRU chain is updated on\neach `commit` command.  The maximum number of active branches can be\nincreased or decreased on the command line with `--active-branches=`.\n\nper active tree\n~~~~~~~~~~~~~~~\nTrees (aka directories) use just 12 bytes of memory on top of the\nmemory required for their entries (see ``per active file'' below).\nThe cost of a tree is virtually 0, as its overhead ammortizes out\nover the individual file entries.\n\nper active file entry\n~~~~~~~~~~~~~~~~~~~~~\nFiles (and pointers to subtrees) within active trees require 52 or 64\nbytes (32/64 bit platforms) per entry.  To conserve space, file and\ntree names are pooled in a common string table, allowing the filename\n``Makefile'' to use just 16 bytes (after including the string header\noverhead) no matter how many times it occurs within the project.\n\nThe active branch LRU, when coupled with the filename string pool\nand lazy loading of subtrees, allows gfi to efficiently import\nprojects with 2,000+ branches and 45,114+ files in a very limited\nmemory footprint (less than 2.7 MiB per active branch).\n\n\nAuthor\n------\nWritten by Shawn O. Pearce <spearce@spearce.org>.\n\nDocumentation\n--------------\nDocumentation by Shawn O. Pearce <spearce@spearce.org>.\n\nGIT\n---\nPart of the gitlink:git[7] suite\n"},{"id":"33687","messageId":"Pine.LNX.4.64.0702052211360.19212@xanadu.home","threadId":"6706","inReplyTo":"20070206023111.GB9222@spearce.org","subject":"Re: git-fast-import","fromName":"Nicolas Pitre","fromEmail":"nico@cam.org","sentAt":"2007-02-06T03:18:46Z","receivedAt":"2007-02-06T03:18:46Z","isPatch":false,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Mon, 5 Feb 2007, Shawn O. Pearce wrote:\n\n> When exactly we merge it in will depend a lot on Junio.  It should\n> be safe to merge before 1.5.0 as its strictly new source files,\n> but we may still want to wait until after 1.5.0 is out.\n\nFor that reason I think it should go in now.\n\n> I'm mainly worried about breaking compliation on odd architectures.\n\nWell, if it doesn't build then just don't make it a fatal build error.  \nThat won't be worse than not having it included at all.\nAnd if it compiles then consider it as a bonus!\n\n\nNicolas\n"},{"id":"33688","messageId":"Pine.LNX.4.64.0702052248070.19212@xanadu.home","threadId":"6706","inReplyTo":"20070206023111.GB9222@spearce.org","subject":"Re: git-fast-import","fromName":"Nicolas Pitre","fromEmail":"nico@cam.org","sentAt":"2007-02-06T04:06:23Z","receivedAt":"2007-02-06T04:06:23Z","isPatch":false,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Mon, 5 Feb 2007, Shawn O. Pearce wrote:\n\n> I'm particularly interested in feedback on the documentation,\n> so I am attaching it below.\n> \n[...]\n> \n> The time of the change is specified by `<time>` as the number of\n> seconds since the UNIX epoc (midnight, Jan 1, 1970, UTC) and is\n> written in base-10 notation using US-ASCII digits.  The committer's\n> timezone is specified by `<tz>` as a positive or negative offset\n> from UTC, in minutes.  For example EST would be expressed in `<tz>`\n> by ``-0500''.\n\nI think this is quite error prone, demonstrated by the fact that we \nscrewed that up ourselves on a few occasions.  I think that the frontend \nshould be relieved from this by letting it provide the time of change in \na more natural format amongst all possible ones(like RFC2822 for \nexample) and gfi should simply give it to parse_date().\n\nOtherwise I think this is pretty nice.\n\n\nNicolas\n"},{"id":"33690","messageId":"20070206054808.GA10508@spearce.org","threadId":"6706","inReplyTo":"Pine.LNX.4.64.0702052248070.19212@xanadu.home","subject":"Re: git-fast-import","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2007-02-06T05:48:08Z","receivedAt":"2007-02-06T05:48:08Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Nicolas Pitre <nico@cam.org> wrote:\n> On Mon, 5 Feb 2007, Shawn O. Pearce wrote:\n> > The time of the change is specified by `<time>` as the number of\n> > seconds since the UNIX epoc (midnight, Jan 1, 1970, UTC) and is\n> > written in base-10 notation using US-ASCII digits.  The committer's\n> > timezone is specified by `<tz>` as a positive or negative offset\n> > from UTC, in minutes.  For example EST would be expressed in `<tz>`\n> > by ``-0500''.\n> \n> I think this is quite error prone, demonstrated by the fact that we \n> screwed that up ourselves on a few occasions.  I think that the frontend \n> should be relieved from this by letting it provide the time of change in \n> a more natural format amongst all possible ones(like RFC2822 for \n> example) and gfi should simply give it to parse_date().\n\nThis is a really good point.  Its a little bit of work to switch\nto parse_date(); I'll try to get it done tomorrow night.\n\n-- \nShawn.\n"},{"id":"33692","messageId":"45C81C33.6010704@gmail.com","threadId":"6706","inReplyTo":"20070206023111.GB9222@spearce.org","subject":"Re: git-fast-import","fromName":"Aneesh Kumar K.V","fromEmail":"aneesh.kumar@gmail.com","sentAt":"2007-02-06T06:12:03Z","receivedAt":"2007-02-06T06:12:03Z","isPatch":false,"sender":{"key":"aneesh.kumar@gmail.com","avatar":"https://gravatar.com/avatar/0621fc0b2f14ead1e9024382f16053a808c148596da30c1b92572fa075621f68?d=mp&s=160"},"body":"Shawn O. Pearce wrote:\n> I'm starting to get gfi (git-fast-import) prepared for a merge into\n> the main git.git tree.  For those who don't know, gfi is the result\n> of my work with Jon Smirl on trying to *quickly* import the massive\n> Mozilla CVS repository into Git.  Recently its been getting a lot\n> of attention from the KDE, OOo, Dragonfly BSD, and Qt projects.\n> \n> When exactly we merge it in will depend a lot on Junio.  It should\n> be safe to merge before 1.5.0 as its strictly new source files,\n> but we may still want to wait until after 1.5.0 is out.\n> \n> \n.....\n\n\n> SYNOPSIS\n> --------\n> frontend | 'git-fast-import' [options]\n> \n\nDo we have example frontend  that can be added along with gfi ?\n\n-aneesh\n"},{"id":"33693","messageId":"20070206061817.GB10508@spearce.org","threadId":"6706","inReplyTo":"45C81C33.6010704@gmail.com","subject":"Re: git-fast-import","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2007-02-06T06:18:17Z","receivedAt":"2007-02-06T06:18:17Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"\"Aneesh Kumar K.V\" <aneesh.kumar@gmail.com> wrote:\n> >SYNOPSIS\n> >--------\n> >frontend | 'git-fast-import' [options]\n> >\n> \n> Do we have example frontend  that can be added along with gfi ?\n\nNot yet.  Some frontends are available here on repo.or.cz:\n\n  gitweb: http://repo.or.cz/w/fast-export.git\n  clone:  git://repo.or.cz/fast-export.git\n\nBut both lack branch support, for example, so they probably aren't\nnearly as complete as the existing non-gfi based importers.\n\n-- \nShawn.\n"},{"id":"33705","messageId":"200702060928.54440.andyparkins@gmail.com","threadId":"6706","inReplyTo":"20070206023111.GB9222@spearce.org","subject":"Re: git-fast-import","fromName":"Andy Parkins","fromEmail":"andyparkins@gmail.com","sentAt":"2007-02-06T09:28:52Z","receivedAt":"2007-02-06T09:28:52Z","isPatch":false,"sender":{"key":"andyparkins@gmail.com","avatar":null},"body":"On Tuesday 2007 February 06 02:31, Shawn O. Pearce wrote:\n\n> The time of the change is specified by `<time>` as the number of\n> seconds since the UNIX epoc (midnight, Jan 1, 1970, UTC) and is\n> written in base-10 notation using US-ASCII digits.  The committer's\n> timezone is specified by `<tz>` as a positive or negative offset\n> from UTC, in minutes.  For example EST would be expressed in `<tz>`\n> by ``-0500''.\n\nIs <tz> /really/ expressed in minutes?  500 minutes is 8 hours 20 minutes.\n\nI know what you mean, of course; and so would anyone reading it - so I suggest \njust dropping the \", in minutes\" - as it's not true.\n\n\nAndy\n-- \nDr Andy Parkins, M Eng (hons), MIEE\nandyparkins@gmail.com\n"},{"id":"33706","messageId":"eq9hvs$l9m$1@sea.gmane.org","threadId":"6706","inReplyTo":"20070206023111.GB9222@spearce.org","subject":"Re: git-fast-import","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2007-02-06T09:34:16Z","receivedAt":"2007-02-06T09:34:16Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Shawn O. Pearce wrote:\n\n> --depth=<n>::\n>         Maximum delta depth, for blob and tree deltification.\n>         Default is 10.\n\nDoes it support --window=<n> option?\n-- \nJakub Narebski\nWarsaw, Poland\nShadeHawk on #git\n"},{"id":"33707","messageId":"20070206093907.GD10508@spearce.org","threadId":"6706","inReplyTo":"eq9hvs$l9m$1@sea.gmane.org","subject":"Re: git-fast-import","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2007-02-06T09:39:07Z","receivedAt":"2007-02-06T09:39:07Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Jakub Narebski <jnareb@gmail.com> wrote:\n> Shawn O. Pearce wrote:\n> \n> > --depth=<n>::\n> >         Maximum delta depth, for blob and tree deltification.\n> >         Default is 10.\n> \n> Does it support --window=<n> option?\n\nNo.  It probably never will.  I don't see a reason to add it.\nHeck, --depth is probably not a great knob to have either.  At\nleast not in gfi.\n\n-- \nShawn.\n"},{"id":"33708","messageId":"20070206094048.GE10508@spearce.org","threadId":"6706","inReplyTo":"200702060928.54440.andyparkins@gmail.com","subject":"Re: git-fast-import","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2007-02-06T09:40:48Z","receivedAt":"2007-02-06T09:40:48Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Andy Parkins <andyparkins@gmail.com> wrote:\n> On Tuesday 2007 February 06 02:31, Shawn O. Pearce wrote:\n> \n> > The time of the change is specified by `<time>` as the number of\n> > seconds since the UNIX epoc (midnight, Jan 1, 1970, UTC) and is\n> > written in base-10 notation using US-ASCII digits.  The committer's\n> > timezone is specified by `<tz>` as a positive or negative offset\n> > from UTC, in minutes.  For example EST would be expressed in `<tz>`\n> > by ``-0500''.\n> \n> Is <tz> /really/ expressed in minutes?  500 minutes is 8 hours 20 minutes.\n> \n> I know what you mean, of course; and so would anyone reading it - so I suggest \n> just dropping the \", in minutes\" - as it's not true.\n\nHeh, right you are!\n\nNico's point about using parse_date() here is a really good one.\nI'm going to modify that section of gfi to use parse_date(), which\nwould change the language here anyway.  I'll try to not to make a\nsilly mistake such as the above in the updated docs.  :)\n\n-- \nShawn.\n"},{"id":"33709","messageId":"eq9j3k$sn1$1@sea.gmane.org","threadId":"6706","inReplyTo":"20070206023111.GB9222@spearce.org","subject":"Re: git-fast-import","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2007-02-06T09:53:20Z","receivedAt":"2007-02-06T09:53:20Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Shawn O. Pearce wrote:\n\n> `filemodify`\n> ^^^^^^^^^^\n[...]\n> `filedelete`\n> ^^^^^^^^^^\n\nShouldn't it be:\n\n`filemodify`\n^^^^^^^^^^^^\n\nand:\n\n`filedelete`\n^^^^^^^^^^^^\n\n-- \nJakub Narebski\nWarsaw, Poland\nShadeHawk on #git\n"},{"id":"33721","messageId":"81b0412b0702060550u3d4064dal43e37593b013f82e@mail.gmail.com","threadId":"6706","inReplyTo":"20070206023111.GB9222@spearce.org","subject":"Re: git-fast-import","fromName":"Alex Riesen","fromEmail":"raa.lkml@gmail.com","sentAt":"2007-02-06T13:50:48Z","receivedAt":"2007-02-06T13:50:48Z","isPatch":false,"sender":{"key":"raa.lkml@gmail.com","avatar":"https://avatars.githubusercontent.com/u/324101?v=4"},"body":"On 2/6/07, Shawn O. Pearce <spearce@spearce.org> wrote:\n> I'm mainly worried about breaking compliation on odd architectures.\n> gfi builds, runs and has been used for production level imports\n> on Mac OS X, Linux and Dragonfly BSD, using both 32 bit and 64 bit\n> architectures, but some of Git's other targets (e.g. AIX) haven't\n> seen any testing.\n\nCompilation errors are the simplest to fix, just send it in.\nI have to import lots of data from perforce spaghetti, so I'm very\nlikely to try it out.\n"},{"id":"33737","messageId":"Pine.LNX.4.64.0702060829310.8424@woody.linux-foundation.org","threadId":"6706","inReplyTo":"20070206054808.GA10508@spearce.org","subject":"Re: git-fast-import","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2007-02-06T16:35:04Z","receivedAt":"2007-02-06T16:35:04Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Tue, 6 Feb 2007, Shawn O. Pearce wrote:\n> Nicolas Pitre <nico@cam.org> wrote:\n> >\n> > I think this is quite error prone, demonstrated by the fact that we \n> > screwed that up ourselves on a few occasions.  I think that the frontend \n> > should be relieved from this by letting it provide the time of change in \n> > a more natural format amongst all possible ones(like RFC2822 for \n> > example) and gfi should simply give it to parse_date().\n> \n> This is a really good point.  Its a little bit of work to switch\n> to parse_date(); I'll try to get it done tomorrow night.\n\nActually, I disagree. We've traditionally have had _less_ bugs with the \npure integer format than we ever had with RFC2822 format.\n\nThe original (first seven days) date format inside git objects was \nrfc2822, and it was *horrible*. Not only does it take time to parse, \npeople get it constantly wrong, and it's ambiguous what summer-time means \netc. It's basically impossible to get anything that is totally repeatable \nfrom it, and you have to be so lax as to effectively accept even buggy \ninput. And yes, buggy input exists.\n\nSo I would strongly suggest that gfi keeps to the standard git date format \nwhich is easy to parse, and totally unambiguous. Yes, you can get it \nwrong, but at least then it's very clear *who* gets it wrong: it's \nwhatever feeds data to gfi. If gfi accepts a \"soft\" format, you get into \nall these gray areas of whether you want to be strictly rfc2822 only, or \nwhether you actually want to accept stuff that everybody accepts \n(including the git date functions, that try very hard to turn anything \nsensible into a date). And DST. And odd timezone names, etc etc.\n\nHaving a hard format, set in stone, and totally unambiguous, is really a \ngood thing. It actually ends up resulting in fewer bugs in the end, \nbecause it just makes sure that everybody is on the same page.\n\n\t\tLinus\n"},{"id":"33738","messageId":"Pine.LNX.4.64.0702060836180.8424@woody.linux-foundation.org","threadId":"6706","inReplyTo":"200702060928.54440.andyparkins@gmail.com","subject":"Re: git-fast-import","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2007-02-06T16:37:58Z","receivedAt":"2007-02-06T16:37:58Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Tue, 6 Feb 2007, Andy Parkins wrote:\n> \n> Is <tz> /really/ expressed in minutes?  500 minutes is 8 hours 20 minutes.\n> \n> I know what you mean, of course; and so would anyone reading it - so I suggest \n> just dropping the \", in minutes\" - as it's not true.\n\nAgreed. It _is_ \"in minutes\", but it's in an oddish human-readable base-60 \nformat. It's certainly *not* decimal, it's more like \"two decimal digits \nencode each base-60 digit in the obvious way\".\n\n\t\tLinus\n"},{"id":"33740","messageId":"20070206164441.GA4949@spearce.org","threadId":"6706","inReplyTo":"Pine.LNX.4.64.0702060836180.8424@woody.linux-foundation.org","subject":"Re: git-fast-import","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2007-02-06T16:44:41Z","receivedAt":"2007-02-06T16:44:41Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Linus Torvalds <torvalds@linux-foundation.org> wrote:\n> On Tue, 6 Feb 2007, Andy Parkins wrote:\n> > \n> > Is <tz> /really/ expressed in minutes?  500 minutes is 8 hours 20 minutes.\n> > \n> > I know what you mean, of course; and so would anyone reading it - so I suggest \n> > just dropping the \", in minutes\" - as it's not true.\n> \n> Agreed. It _is_ \"in minutes\", but it's in an oddish human-readable base-60 \n> format. It's certainly *not* decimal, it's more like \"two decimal digits \n> encode each base-60 digit in the obvious way\".\n\nWhat about this language?\n\n\tThe time of the change is specified by `<time>` as the number of\n\tseconds since the UNIX epoc (midnight, Jan 1, 1970, UTC) and is\n\twritten in base-10 notation using US-ASCII digits.  The committer's\n\ttimezone is specified by `<tz>` as a positive or negative offset\n\tfrom UTC.  For example EST (which is typically 5 hours behind GMT)\n\twould be expressed in `<tz>` by ``-0500'' while GMT is ``+0000''.\n\n-- \nShawn.\n"},{"id":"33741","messageId":"20070206165630.GB4949@spearce.org","threadId":"6706","inReplyTo":"Pine.LNX.4.64.0702060829310.8424@woody.linux-foundation.org","subject":"Re: git-fast-import","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2007-02-06T16:56:30Z","receivedAt":"2007-02-06T16:56:30Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Linus Torvalds <torvalds@linux-foundation.org> wrote:\n> On Tue, 6 Feb 2007, Shawn O. Pearce wrote:\n> > Nicolas Pitre <nico@cam.org> wrote:\n> > > I think this is quite error prone, demonstrated by the fact that we \n> > > screwed that up ourselves on a few occasions.  I think that the frontend \n> > > should be relieved from this by letting it provide the time of change in \n> > > a more natural format amongst all possible ones(like RFC2822 for \n> > > example) and gfi should simply give it to parse_date().\n> > \n> > This is a really good point.  Its a little bit of work to switch\n> > to parse_date(); I'll try to get it done tomorrow night.\n> \n> Actually, I disagree. We've traditionally have had _less_ bugs with the \n> pure integer format than we ever had with RFC2822 format.\n\nHmm.  Actually I think it depends on the source data.  :-)\n\nIf the source is only supplying RFC2822 date format and is reliable\nin its formatting of such, having gfi parse that rather than\nthe frontend is probably more reliable.  (Git already has a well\ntested date parsing routine.)  But if the source is easily able\nto get a time_t then that is just as easily formatted out to gfi,\nand reading that without error is child's play.\n\nAfter reading your email I'm now contemplating making this a command\nline flag, like `--date-format=rfc2822`, so a frontend could ask\ngfi to use parse_date() and whatever error that might bring, or\nstick with the pure integer format.\n \n> Having a hard format, set in stone, and totally unambiguous, is really a \n> good thing. It actually ends up resulting in fewer bugs in the end, \n> because it just makes sure that everybody is on the same page.\n\nWhich is why gfi is very strict about its handling of whitespace.\nIt assumes *exactly* one space between input fields, or *exactly*\none LF between commands.  Anything else is assumed to be part of\nthe next field.  If spaces show up in the imported data, its the\nfrontend that is sending stuff incorrectly.\n\nRight now however gfi is not validating the author or committer\ncommand arguments.  At all.  Which means that although the\ndocumentation says the format must be such-and-such, gfi doesn't\ncare.  Whatever comes in on the `author` or `committer` line is\ncopied verbatim into the commit object.  gfi probably should at\nleast verify that the timestamp part of the line actually contains\ndigits.  :)\n\n-- \nShawn.\n"},{"id":"33742","messageId":"Pine.LNX.4.64.0702060913220.8424@woody.linux-foundation.org","threadId":"6706","inReplyTo":"20070206165630.GB4949@spearce.org","subject":"Re: git-fast-import","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2007-02-06T17:20:40Z","receivedAt":"2007-02-06T17:20:40Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Tue, 6 Feb 2007, Shawn O. Pearce wrote:\n> \n> If the source is only supplying RFC2822 date format and is reliable\n> in its formatting of such, having gfi parse that rather than\n> the frontend is probably more reliable.  (Git already has a well\n> tested date parsing routine.)\n\nI'm not so worried about the git date parsing routines (which are fairly \nsolid) as about the fact that absolutely *tons* of people get rfc2822 \nwrong.\n\nAnd we'd never even see it, because gits date-parsing routines are very \nforgiving, and allow pretty much anything (and no, I'm not talking about \napproxidate(), which really *does* allow anything, I'm talking about the \n\"strict\" date parser). \n\nThey allow pretty much any half-way valid date, exactly because people \ndon't do rfc2822 right anyway (and because they are also meant to work \neven if you write the date by hand, like \"12:34 2005-06-07\").\n\nAnd *particularly* when it comes to timezones, it just guesses. The whole \ndaylight savings time thing is just too hard. And if no timezone exists, \nit will just take the current one, so things may *seem* like they work, \nbut then two different people importing the *same* archive in two \ndifferent locations will actually get different results!\n\nTHAT'S A BAD THING!\n\nIt's much better to specify the date so exactly that you simply cannot get \ndifferent results with the same input.\n\nSure, you can still mess up the program that actually generates the data \nfor gfi, and have bugs like that *there*, but at least they'd have to \nthink a bit about it.\n\nAnd the TZ problem is actually less likely if you have a strict TZ format. \nFor example, when importing from CVS, the natural thing to do is to just \nalways set TZ to +0000. Which gets you something reliable, and it won't \ndepend on who did the import.\n\nBut hey, especially if it's a flag, and especially if it's *documented* \nthat the date parsing will depend on the current timezone etc, then maybe \nit's all ok. It's certainly convenient to be able to give the date in any \nformat. It's just very easy to get bugs when you allow any random crud..\n\n\t\tLinus\n"},{"id":"33743","messageId":"20070206172052.GA5135@spearce.org","threadId":"6706","inReplyTo":"eq9j3k$sn1$1@sea.gmane.org","subject":"Re: git-fast-import","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2007-02-06T17:20:52Z","receivedAt":"2007-02-06T17:20:52Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Jakub Narebski <jnareb@gmail.com> wrote:\n> Shouldn't it be:\n> `filemodify`\n> ^^^^^^^^^^^^\n> and:\n> `filedelete`\n> ^^^^^^^^^^^^\n\nYes, thanks, its fixed.\n\n-- \nShawn.\n"},{"id":"33744","messageId":"Pine.LNX.4.64.0702060921260.8424@woody.linux-foundation.org","threadId":"6706","inReplyTo":"20070206164441.GA4949@spearce.org","subject":"Re: git-fast-import","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2007-02-06T17:24:51Z","receivedAt":"2007-02-06T17:24:51Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Tue, 6 Feb 2007, Shawn O. Pearce wrote:\n> \n> What about this language?\n> \n> \tThe time of the change is specified by `<time>` as the number of\n> \tseconds since the UNIX epoc (midnight, Jan 1, 1970, UTC) and is\n> \twritten in base-10 notation using US-ASCII digits.  The committer's\n> \ttimezone is specified by `<tz>` as a positive or negative offset\n> \tfrom UTC.  For example EST (which is typically 5 hours behind GMT)\n> \twould be expressed in `<tz>` by ``-0500'' while GMT is ``+0000''.\n\nI doubt it would confuse anybody. Although usually we'd not say\n\n\t\"in base-10 notation using US-ASCII digits\"\n\nthe normal way to do that is to just saying \"as an ASCII decimal integer\".\n\nSure, people could try to do \"10,200,300\" and claim it's \"decimal \ninteger\", but at that point, you can just tell them they're crazy, and \nignore them ;)\n\nBut your text certainly isn't wrong. I just think it overspecifies a bit, \nat the expense of readability.\n\n\t\tLinus\n"},{"id":"33746","messageId":"20070206174336.GB5135@spearce.org","threadId":"6706","inReplyTo":"81b0412b0702060550u3d4064dal43e37593b013f82e@mail.gmail.com","subject":"Re: git-fast-import","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2007-02-06T17:43:36Z","receivedAt":"2007-02-06T17:43:36Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Alex Riesen <raa.lkml@gmail.com> wrote:\n> On 2/6/07, Shawn O. Pearce <spearce@spearce.org> wrote:\n> >I'm mainly worried about breaking compliation on odd architectures.\n> >gfi builds, runs and has been used for production level imports\n> >on Mac OS X, Linux and Dragonfly BSD, using both 32 bit and 64 bit\n> >architectures, but some of Git's other targets (e.g. AIX) haven't\n> >seen any testing.\n> \n> Compilation errors are the simplest to fix, just send it in.\n\nTrue.\n\nBut it really is annoying when you download the latest-and-greatest\nrelease of a package only to find out it doesn't compile on your\nOS of choice, and even worse when you find out it is because of\nnew code that you will never use which was added in just before\nthe release went final!\n\n> I have to import lots of data from perforce spaghetti, so I'm very\n> likely to try it out.\n\nI can't help you with spaghetti, but the Qt folks did make their\nPerforce importer available.  Chris Lee put it in the fast-export\nproject on repo.or.cz.  Its a relatively short Python program.\nMight help you get started.\n\nThey created annotated tags (with no message) for every p4 changeset.\nI think its just because they didn't realize you can use (abuse?) the\n`reset` command in gfi to create lightweight tags instead.\n\n\nI actually implemented a \"data <path\" command in gfi to tell gfi\nto load data from a file, for this type of case where the foreign\nsystem has dropped the files in your working directory and you just\nwant Git to read them.\n\nBut there's no synchronization between gfi and the frontend (aside\nfrom the pipe buffer throttling the frontend), so there is no way\nfor the frontend to know that gfi has finished a batch of files\nand its safe to ask p4 for the next revision.\n\nSo I threw it away.  It was only a 10 line patch anyway.  :)\n\n-- \nShawn.\n"},{"id":"33747","messageId":"81b0412b0702061002q2a6359cdwa1369ba8eab6b492@mail.gmail.com","threadId":"6706","inReplyTo":"20070206174336.GB5135@spearce.org","subject":"Re: git-fast-import","fromName":"Alex Riesen","fromEmail":"raa.lkml@gmail.com","sentAt":"2007-02-06T18:02:40Z","receivedAt":"2007-02-06T18:02:40Z","isPatch":false,"sender":{"key":"raa.lkml@gmail.com","avatar":"https://avatars.githubusercontent.com/u/324101?v=4"},"body":"On 2/6/07, Shawn O. Pearce <spearce@spearce.org> wrote:\n> Alex Riesen <raa.lkml@gmail.com> wrote:\n> > On 2/6/07, Shawn O. Pearce <spearce@spearce.org> wrote:\n> > >I'm mainly worried about breaking compliation on odd architectures.\n> > >gfi builds, runs and has been used for production level imports\n> > >on Mac OS X, Linux and Dragonfly BSD, using both 32 bit and 64 bit\n> > >architectures, but some of Git's other targets (e.g. AIX) haven't\n> > >seen any testing.\n> >\n> > Compilation errors are the simplest to fix, just send it in.\n>\n> True.\n>\n> But it really is annoying when you download the latest-and-greatest\n> release of a package only to find out it doesn't compile on your\n> OS of choice, and even worse when you find out it is because of\n> new code that you will never use which was added in just before\n> the release went final!\n\nThan send it now! :)\n\n> > I have to import lots of data from perforce spaghetti, so I'm very\n> > likely to try it out.\n>\n> I can't help you with spaghetti, but the Qt folks did make their\n> Perforce importer available.  Chris Lee put it in the fast-export\n> project on repo.or.cz.  Its a relatively short Python program.\n> Might help you get started.\n\nYes, I saw their code. That's how I started thinking of using gfi\nin my p4 imports.\n\n> They created annotated tags (with no message) for every p4 changeset.\n> I think its just because they didn't realize you can use (abuse?) the\n> `reset` command in gfi to create lightweight tags instead.\n\nI found it's useless to do anything with p4 changes. They lack\nthe most important part of history: parent. The comments get\nuseless too, because they refer to the most recent change,\nwith no practical way to extract anything in between. Not much\nof a problem, nobody writes anything sensible in perforce\ncomments anyway.\n"},{"id":"33748","messageId":"Pine.LNX.4.64.0702061329090.19212@xanadu.home","threadId":"6706","inReplyTo":"Pine.LNX.4.64.0702060913220.8424@woody.linux-foundation.org","subject":"Re: git-fast-import","fromName":"Nicolas Pitre","fromEmail":"nico@cam.org","sentAt":"2007-02-06T18:53:37Z","receivedAt":"2007-02-06T18:53:37Z","isPatch":false,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Tue, 6 Feb 2007, Linus Torvalds wrote:\n\n> I'm not so worried about the git date parsing routines (which are fairly \n> solid) as about the fact that absolutely *tons* of people get rfc2822 \n> wrong.\n> \n> They allow pretty much any half-way valid date, exactly because people \n> don't do rfc2822 right anyway (and because they are also meant to work \n> even if you write the date by hand, like \"12:34 2005-06-07\").\n> \n> Sure, you can still mess up the program that actually generates the data \n> for gfi, and have bugs like that *there*, but at least they'd have to \n> think a bit about it.\n\nWell, exactly because GIT already has fairly solid date parsing \nroutines, and the fact that we needed solid date parsing routines in the \nfirst place, exactly because people don't do rfc2822 right anyway, \nshould be a hell of a big clue why we should parse date information for \nthe gfi frontend.  Because the date is for sure most likely in a screwed \nup format already and it is counter productive to have to deal with that \nin a duplicated piece of code.  And the bare reality is that people will \njust not care to parse it right themselves.\n\nQuoting from the gfi manual:\n\n|A typical frontend for gfi tends to weigh in at approximately 200\n|lines of Perl/Python/Ruby code.  Most developers have been able to\n|create working importers in just a couple of hours, even though it\n|is their first exposure to gfi, and sometimes even to Git.  This is\n|an ideal situation, given that most conversion tools are throw-away\n|(use once, and never look back).\n\nThis is therefore a damn good idea if gfi can make things right out of \ncrap because frontends will not get much attention after the first \"hey \nit works\" level.  And the GIT date format, albeit being perfectly \nunambigous, is not inline with the statement above.\n\nWith the GIT date format a conversion _will_ be necessary in the \nfrontend, while if gfi shove it to parse_date() instead then no \nconversion is even likely to be needed by the frontend.  I'd much prefer \nif frontend writers didn't have to care (and most probably manage to \nbotch it if they have to) about date conversion.  We even botched it a \nfew times ourselves despite the fact that we're damn good.\n\nAnd because our date parsing code is damn good (hey we're just damn good \naren't we?) I would bet that there will be much less conversion errors \nif gfi used parse_date() on provided data than if the frontend tries to \nparse the date itself.  This is wat we feed email submission through \neveryday anyway, so we must trust it to do a good job for imports as \nwell.\n\n\nNicolas\n"},{"id":"33753","messageId":"20070206200924.GA5352@spearce.org","threadId":"6706","inReplyTo":"Pine.LNX.4.64.0702061329090.19212@xanadu.home","subject":"Re: git-fast-import","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2007-02-06T20:09:24Z","receivedAt":"2007-02-06T20:09:24Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Nicolas Pitre <nico@cam.org> wrote:\n> This is therefore a damn good idea if gfi can make things right out of \n> crap because frontends will not get much attention after the first \"hey \n> it works\" level.  And the GIT date format, albeit being perfectly \n> unambigous, is not inline with the statement above.\n\nDone.  I just pushed a change to gfi which adds `--date-format=<fmt>`.\nFor <fmt> you have the choice of:\n\n  raw: Standard Git format.  This is the default, as its what\n  the existing frontends by Chris Lee, Simon Hausmann, Jon Smirl,\n  and Simon 'corecode' Schubert expect.\n\n  rfc2822: Run whatever crap you give us through parse_date(),\n  and cross your fingers.  If parse_date() returns < 0 we bomb\n  out, but otherwise take it at its word.\n\n  now: This is a toy, but useful if you really want now, dammit.\n  We just call datestamp() and tack that in.  Note that the frontend\n  must also supply the literal string `now` in the committer line\n  (e.g. \"committer A U Thor <at@example.com> now\") to prevent us\n  from bombing out.\n\nThe last one will probably get more useful when I fix gfi so it can\nsafely commit against active refs without losing commits (make it\ndo a strict fast-forward check before updating).  In this case it\nmay be useful for something like git-cvsserver, as it avoids the\nneed for a temporary directory, index, etc.\n \n-- \nShawn.\n"},{"id":"33754","messageId":"Pine.LNX.4.64.0702061559160.13009@xanadu.home","threadId":"6706","inReplyTo":"20070206200924.GA5352@spearce.org","subject":"Re: git-fast-import","fromName":"Nicolas Pitre","fromEmail":"nico@cam.org","sentAt":"2007-02-06T21:03:41Z","receivedAt":"2007-02-06T21:03:41Z","isPatch":false,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Tue, 6 Feb 2007, Shawn O. Pearce wrote:\n\n> Done.  I just pushed a change to gfi which adds `--date-format=<fmt>`.\n> For <fmt> you have the choice of:\n> \n>   raw: Standard Git format.  This is the default, as its what\n>   the existing frontends by Chris Lee, Simon Hausmann, Jon Smirl,\n>   and Simon 'corecode' Schubert expect.\n> \n>   rfc2822: Run whatever crap you give us through parse_date(),\n>   and cross your fingers.  If parse_date() returns < 0 we bomb\n>   out, but otherwise take it at its word.\n\nI think you should call it something else than rfc2822.  Because \nparse_date() accepts much more than just rfc2822.  What about \"cooked\"?\n\n\nNicolas\n"},{"id":"33755","messageId":"20070206211547.GB5820@spearce.org","threadId":"6706","inReplyTo":"Pine.LNX.4.64.0702061559160.13009@xanadu.home","subject":"Re: git-fast-import","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2007-02-06T21:15:47Z","receivedAt":"2007-02-06T21:15:47Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Nicolas Pitre <nico@cam.org> wrote:\n> I think you should call it something else than rfc2822.  Because \n> parse_date() accepts much more than just rfc2822.  What about \"cooked\"?\n\nIt does accept a lot more than that, but straying away from rfc2822\ngets into the grey areas of parse_date().  E.g. it matches crap such\nas 'yyyy-mm-dd' or 'yyyy-dd-mm'.  But that is completely ambiguous!\n\nI don't really want to advertise that it is accepting non-RFC 2822\ninput here.  I was thinking of doing an `iso` (yyyy-mm-dd hh:mm:ss)\nformat, which may just defer into parse_date(), but again encourage\nthe frontend to *only* feed that ISO style format.\n\n-- \nShawn.\n"},{"id":"33757","messageId":"Pine.LNX.4.64.0702061641040.13009@xanadu.home","threadId":"6706","inReplyTo":"20070206211547.GB5820@spearce.org","subject":"Re: git-fast-import","fromName":"Nicolas Pitre","fromEmail":"nico@cam.org","sentAt":"2007-02-06T21:42:48Z","receivedAt":"2007-02-06T21:42:48Z","isPatch":false,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Tue, 6 Feb 2007, Shawn O. Pearce wrote:\n\n> Nicolas Pitre <nico@cam.org> wrote:\n> > I think you should call it something else than rfc2822.  Because \n> > parse_date() accepts much more than just rfc2822.  What about \"cooked\"?\n> \n> It does accept a lot more than that, but straying away from rfc2822\n> gets into the grey areas of parse_date().\n\nOK that makes sense.\n\n\nNicolas\n"},{"id":"33763","messageId":"200702070117.l171HGxr023213@laptop13.inf.utfsm.cl","threadId":"6706","inReplyTo":"20070206164441.GA4949@spearce.org","subject":"Re: git-fast-import","fromName":"Horst H. von Brand","fromEmail":"vonbrand@inf.utfsm.cl","sentAt":"2007-02-07T01:17:16Z","receivedAt":"2007-02-07T01:17:16Z","isPatch":false,"sender":{"key":"vonbrand@inf.utfsm.cl","avatar":"https://avatars.githubusercontent.com/u/211384?v=4"},"body":"Shawn O. Pearce <spearce@spearce.org> wrote:\n\n[...]\n\n> What about this language?\n> \n> \tThe time of the change is specified by `<time>` as the number of\n> \tseconds since the UNIX epoc (midnight, Jan 1, 1970, UTC) and is\n> \twritten in base-10 notation using US-ASCII digits.  The committer's\n> \ttimezone is specified by `<tz>` as a positive or negative offset\n> \tfrom UTC.  For example EST (which is typically 5 hours behind GMT)\n> \twould be expressed in `<tz>` by ``-0500'' while GMT is ``+0000''.\n\nThat is /not/ a timezone! Maybe an offset from UTC.\n-- \nDr. Horst H. von Brand                   User #22616 counter.li.org\nDepartamento de Informatica                    Fono: +56 32 2654431\nUniversidad Tecnica Federico Santa Maria             +56 32 2654239\nCasilla 110-V, Valparaiso, Chile               Fax:  +56 32 2797513\n"},{"id":"33766","messageId":"Pine.LNX.4.64.0702061832020.8424@woody.linux-foundation.org","threadId":"6706","inReplyTo":"200702070117.l171HGxr023213@laptop13.inf.utfsm.cl","subject":"Re: git-fast-import","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2007-02-07T02:50:07Z","receivedAt":"2007-02-07T02:50:07Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Tue, 6 Feb 2007, Horst H. von Brand wrote:\n>\n> Shawn O. Pearce <spearce@spearce.org> wrote:\n> \n> [...]\n> \n> > What about this language?\n> > \n> > \tThe time of the change is specified by `<time>` as the number of\n> > \tseconds since the UNIX epoc (midnight, Jan 1, 1970, UTC) and is\n> > \twritten in base-10 notation using US-ASCII digits.  The committer's\n> > \ttimezone is specified by `<tz>` as a positive or negative offset\n> > \tfrom UTC.  For example EST (which is typically 5 hours behind GMT)\n> > \twould be expressed in `<tz>` by ``-0500'' while GMT is ``+0000''.\n> \n> That is /not/ a timezone! Maybe an offset from UTC.\n\nBtw, one thing that might be a good idea to document very clearly:\n\n - in the native git format, the offset from UTC has *nothing* to do with \n   the actual time itself. The time in native git is always in UTC, and \n   the offset from UTC does not change \"time\" - it's purely there to tell \n   in which timezone the event happened.\n\n   So 12345678 +0000 and 12345678 -0700 are *exactly*the*same*date*, \n   except event one happened in UTC, and the other happened in UTC-7.\n\n - in rfc2822 format, the offset from UTC actually *changes* the date. The \n   date \"Oct 12, 2006 20:00:00\" will be two _different_ times when you say \n   it is in PST or in UTC.\n\nAnd yes, for all I know we might get this wrong inside git too. It's easy \nto get confused, because they really do mean different things.\n\nFor an example of this, do\n\n\tmake test-date\n\nin git (which parses the argument using the \"exact date\" and \"approxidate\" \nversions respectively, and the exact date parsing will give the internal \ngit representation on the first line in the middle column), and then:\n\n\t./test-date \"1234567890 -0800\"\n\t./test-date \"1234567890 +0000\"\n\nand then try\n\n\t./test-date \"Fri Feb 13 15:31:30 2009 PST\"\n\t./test-date \"Fri Feb 13 15:31:30 2009 UTC\"\n\nand notice how the first two (numeric) dates that differ in UTC offset \nwill still return the exact same seconds since the epoch:\n\n\t1234567890 -0800\n\t1234567890 +0000\n\nbut the second example (with a rfc2822-like date), will show how the \nseconds-since-epoch changes, and gives:\n\n\t1234567890 -0800\n\t1234539090 +0000\n\nrespectively for those two dates.\n\nLogical? It actually is, but you have to understand how git represents \ndate to see the logic. To git, the \"timezone\" is really totally \nirrelevant. It doesn't really affect the \"date\" at all. At most, it \naffects how you _print_ the date, and you can tell what timezone the \ncomputer was set to when the commit was made.\n\nAnd yes, I would not be at all surprised if we had some bug here where we \ngot it wrong occasionally.\n\n\t\tLinus\n"},{"id":"33768","messageId":"Pine.LNX.4.64.0702062337090.20138@iabervon.org","threadId":"6706","inReplyTo":"20070206164441.GA4949@spearce.org","subject":"Re: git-fast-import","fromName":"Daniel Barkalow","fromEmail":"barkalow@iabervon.org","sentAt":"2007-02-07T04:45:14Z","receivedAt":"2007-02-07T04:45:14Z","isPatch":false,"sender":{"key":"barkalow@iabervon.org","avatar":"https://avatars.githubusercontent.com/u/55364219?v=4"},"body":"On Tue, 6 Feb 2007, Shawn O. Pearce wrote:\n\n> What about this language?\n> \n> \tThe time of the change is specified by `<time>` as the number of\n> \tseconds since the UNIX epoc (midnight, Jan 1, 1970, UTC) and is\n> \twritten in base-10 notation using US-ASCII digits.  The committer's\n> \ttimezone is specified by `<tz>` as a positive or negative offset\n> \tfrom UTC.  For example EST (which is typically 5 hours behind GMT)\n> \twould be expressed in `<tz>` by ``-0500'' while GMT is ``+0000''.\n\nEST is always 5 hours behind GMT. During the summer, EST is still 5 hours \nbehind GMT, but the clocks which use ET are set to EDT (-0400) instead. \n\n\t-Daniel\n*This .sig left intentionally blank*\n"},{"id":"33769","messageId":"Pine.LNX.4.64.0702062347060.20138@iabervon.org","threadId":"6706","inReplyTo":"20070206061817.GB10508@spearce.org","subject":"Re: git-fast-import","fromName":"Daniel Barkalow","fromEmail":"barkalow@iabervon.org","sentAt":"2007-02-07T04:55:46Z","receivedAt":"2007-02-07T04:55:46Z","isPatch":false,"sender":{"key":"barkalow@iabervon.org","avatar":"https://avatars.githubusercontent.com/u/55364219?v=4"},"body":"On Tue, 6 Feb 2007, Shawn O. Pearce wrote:\n\n> \"Aneesh Kumar K.V\" <aneesh.kumar@gmail.com> wrote:\n> > >SYNOPSIS\n> > >--------\n> > >frontend | 'git-fast-import' [options]\n> > >\n> > \n> > Do we have example frontend  that can be added along with gfi ?\n> \n> Not yet.  Some frontends are available here on repo.or.cz:\n> \n>   gitweb: http://repo.or.cz/w/fast-export.git\n>   clone:  git://repo.or.cz/fast-export.git\n> \n> But both lack branch support, for example, so they probably aren't\n> nearly as complete as the existing non-gfi based importers.\n\nIt might be nice to have a git-fast-export, which could actually be \npotentially useful for generating a repository with systematic differences \nfrom the original. (E.g., to make a repository of git's Documentation \ndirectory, with just the commits that affect it)\n\nThat might also be a big help to projects that find they should have been \nusing more, fewer, or different repositories through their history.\n\nAlso, I'd guess that it would be pretty straightforward and easy to \nunderstand, plus easy to verify correctness on large examples with.\n\n\t-Daniel\n*This .sig left intentionally blank*\n"},{"id":"33771","messageId":"20070207054614.GA7138@spearce.org","threadId":"6706","inReplyTo":"200702070117.l171HGxr023213@laptop13.inf.utfsm.cl","subject":"Re: git-fast-import","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2007-02-07T05:46:14Z","receivedAt":"2007-02-07T05:46:14Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"\"Horst H. von Brand\" <vonbrand@inf.utfsm.cl> wrote:\n> Shawn O. Pearce <spearce@spearce.org> wrote:\n> > \twritten in base-10 notation using US-ASCII digits.  The committer's\n> > \ttimezone is specified by `<tz>` as a positive or negative offset\n> > \tfrom UTC.  For example EST (which is typically 5 hours behind GMT)\n> > \twould be expressed in `<tz>` by ``-0500'' while GMT is ``+0000''.\n> \n> That is /not/ a timezone! Maybe an offset from UTC.\n\nIndeed.  Thank you for the correction.  I'll push out fixed docs\nshortly.\n\n-- \nShawn.\n"},{"id":"33772","messageId":"20070207055352.GB7138@spearce.org","threadId":"6706","inReplyTo":"Pine.LNX.4.64.0702061832020.8424@woody.linux-foundation.org","subject":"Re: git-fast-import","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2007-02-07T05:53:52Z","receivedAt":"2007-02-07T05:53:52Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Linus Torvalds <torvalds@linux-foundation.org> wrote:\n> Btw, one thing that might be a good idea to document very clearly:\n> \n>  - in the native git format, the offset from UTC has *nothing* to do with \n>    the actual time itself. The time in native git is always in UTC, and \n>    the offset from UTC does not change \"time\" - it's purely there to tell \n>    in which timezone the event happened.\n> \n>    So 12345678 +0000 and 12345678 -0700 are *exactly*the*same*date*, \n>    except event one happened in UTC, and the other happened in UTC-7.\n> \n>  - in rfc2822 format, the offset from UTC actually *changes* the date. The \n>    date \"Oct 12, 2006 20:00:00\" will be two _different_ times when you say \n>    it is in PST or in UTC.\n\nHere is the current language relating to date parsing in gfi:\n\nDate Formats\n~~~~~~~~~~~~\nThe following date formats are supported.  A frontend should select\nthe format it will use for this import by passing the format name\nin the `--date-format=<fmt>` command line option.\n\n`raw`::\n\tThis is the Git native format and is `<time> SP <offutc>`.\n\tIt is also gfi's default format, if `--date-format` was\n\tnot specified.\n+\nThe time of the event is specified by `<time>` as the number of\nseconds since the UNIX epoch (midnight, Jan 1, 1970, UTC) and is\nwritten as an ASCII decimal integer.\n+\nThe local offset is specified by `<offutc>` as a positive or negative\noffset from UTC.  For example EST (which is 5 hours behind UTC)\nwould be expressed in `<tz>` by ``-0500'' while UTC is ``+0000''.\nThe local offset does not affect `<time>`; it is used only as an\nadvisement to help formatting routines display the timestamp.\n+\nIf the local offset is not available in the source material, use\n``+0000'', or the most common local offset.  For example many\norganizations have a CVS repository which has only ever been accessed\nby users who are located in the same location and timezone.  In this\ncase the offset from UTC can be easily assumed.\n+\nUnlike the `rfc2822` format, this format is very strict.  Any\nvariation in formatting will cause gfi to reject the value.\n\n`rfc2822`::\n\tThis is the standard email format as described by RFC 2822.\n+\nAn example value is ``Tue Feb 6 11:22:18 2007 -0500''.  The Git\nparser is accurate, but a little on the lenient side.  Its the\nsame parser used by gitlink:git-am[1] when applying patches\nreceived from email.\n+\nSome malformed strings may be accepted as valid dates.  In some of\nthese cases Git will still be able to obtain the correct date from\nthe malformed string.  There are also some types of malformed\nstrings which Git will parse wrong, and yet consider valid.\nSeriously malformed strings will be rejected.\n+\nUnlike the `raw` format above, the timezone/UTC offset information\ncontained in an RFC 2822 date string is used to adjust the date\nvalue to UTC prior to storage.  Therefore it is important that\nthis information be as accurate as possible.\n+\nIf the source material is formatted in RFC 2822 style dates,\nthe frontend should let gfi handle the parsing and conversion\n(rather than attempting to do it itself) as the Git parser has\nbeen well tested in the wild.\n+\nFrontends should prefer the `raw` format if the source material\nis already in UNIX-epoch format, or is easily convertible to\nthat format, as there is no ambiguity in parsing.\n\n`now`::\n\tAlways use the current time and timezone.  The literal\n\t`now` must always be supplied for `<when>`.\n+\nThis is a toy format.  The current time and timezone of this system\nis always copied into the identity string at the time it is being\ncreated by gfi.  There is no way to specify a different time or\ntimezone.\n+\nThis particular format is supplied as its short to implement and\nmay be useful to a process that wants to create a new commit\nright now, without needing to use a working directory or\ngitlink:git-update-index[1].\n+\nIf separate `author` and `committer` commands are used in a `commit`\nthe timestamps may not match, as the system clock will be polled\ntwice (once for each command).  The only way to ensure that both\nauthor and committer identity information has the same timestamp\nis to omit `author` (thus copying from `committer`) or to use a\ndate format other than `now`.\n"},{"id":"33778","messageId":"20070207091351.GA1632@diana.vm.bytemark.co.uk","threadId":"6706","inReplyTo":"Pine.LNX.4.64.0702062347060.20138@iabervon.org","subject":"Re: git-fast-import","fromName":"Karl Hasselström","fromEmail":"kha@treskal.com","sentAt":"2007-02-07T09:13:51Z","receivedAt":"2007-02-07T09:13:51Z","isPatch":false,"sender":{"key":"kha@treskal.com","avatar":"https://gravatar.com/avatar/f0120c734b5279b345075a28521e1ac66acb20c9913ffe9bf6ae97e53f7f3f13?d=mp&s=160"},"body":"On 2007-02-06 23:55:46 -0500, Daniel Barkalow wrote:\n\n> It might be nice to have a git-fast-export, which could actually be\n> potentially useful for generating a repository with systematic\n> differences from the original. (E.g., to make a repository of git's\n> Documentation directory, with just the commits that affect it)\n\nOr to solve problems like\n\n  Gaaah! This file we've had in the repository for the last 17 months\n  has copyright problems and we can't distribute it!\n\nor\n\n  Wouldn't it be nice to permanently include all that old Linux\n  history that's currently grafted onto the \"real\" history?\n\nIn other words, general history rewriting, but fast.\n\n(Disclaimer: I've never tried to use the history rewrite tool that\nCogito has, so I don't know its limitations, or how fast it is.)\n\n-- \nKarl Hasselström, kha@treskal.com\n      www.treskal.com/kalle\n"},{"id":"33779","messageId":"20070207092103.GB1632@diana.vm.bytemark.co.uk","threadId":"6706","inReplyTo":"20070207055352.GB7138@spearce.org","subject":"Re: git-fast-import","fromName":"Karl Hasselström","fromEmail":"kha@treskal.com","sentAt":"2007-02-07T09:21:03Z","receivedAt":"2007-02-07T09:21:03Z","isPatch":false,"sender":{"key":"kha@treskal.com","avatar":"https://gravatar.com/avatar/f0120c734b5279b345075a28521e1ac66acb20c9913ffe9bf6ae97e53f7f3f13?d=mp&s=160"},"body":"On 2007-02-07 00:53:52 -0500, Shawn O. Pearce wrote:\n\n> Its the same parser used by gitlink:git-am[1] when applying patches\n> received from email.\n\nShould be \"It's\" or \"It is\".\n\n-- \nKarl Hasselström, kha@treskal.com\n      www.treskal.com/kalle\n"},{"id":"33781","messageId":"000c01c74a9a$82fc99c0$0b0aa8c0@abf.local","threadId":"6706","inReplyTo":"Pine.LNX.4.64.0702062347060.20138@iabervon.org","subject":"RE: git-fast-import","fromName":"Raimund Bauer","fromEmail":"ray007@gmx.net","sentAt":"2007-02-07T09:29:47Z","receivedAt":"2007-02-07T09:29:47Z","isPatch":false,"sender":{"key":"ray007@gmx.net","avatar":null},"body":"> It might be nice to have a git-fast-export, which could actually be \n> potentially useful for generating a repository with \n> systematic differences \n> >from the original. (E.g., to make a repository of git's Documentation\n> directory, with just the commits that affect it)\n\nSearch the list-archives for \"git-split\", that may be what you're looking\nfor.\n\n-- \nbest regards\n\n  Ray\n"},{"id":"33791","messageId":"1170845880.29759.1132.camel@pmac.infradead.org","threadId":"6706","inReplyTo":"Pine.LNX.4.64.0702061329090.19212@xanadu.home","subject":"Re: git-fast-import","fromName":"David Woodhouse","fromEmail":"dwmw2@infradead.org","sentAt":"2007-02-07T10:58:00Z","receivedAt":"2007-02-07T10:58:00Z","isPatch":false,"sender":{"key":"dwmw2@infradead.org","avatar":"https://gravatar.com/avatar/7afd4f07e0cf7d7e046ae2d23678296b37777c96488e6f3451e78a5514154ebd?d=mp&s=160"},"body":"On Tue, 2007-02-06 at 13:53 -0500, Nicolas Pitre wrote:\n> On Tue, 6 Feb 2007, Linus Torvalds wrote:\n> \n> > I'm not so worried about the git date parsing routines (which are fairly \n> > solid) as about the fact that absolutely *tons* of people get rfc2822 \n> > wrong.\n> > \n> > They allow pretty much any half-way valid date, exactly because people \n> > don't do rfc2822 right anyway (and because they are also meant to work \n> > even if you write the date by hand, like \"12:34 2005-06-07\").\n> > \n> > Sure, you can still mess up the program that actually generates the data \n> > for gfi, and have bugs like that *there*, but at least they'd have to \n> > think a bit about it.\n> \n> Well, exactly because GIT already has fairly solid date parsing \n> routines, and the fact that we needed solid date parsing routines in the \n> first place, exactly because people don't do rfc2822 right anyway, \n> should be a hell of a big clue why we should parse date information for \n> the gfi frontend.  Because the date is for sure most likely in a screwed \n> up format already and it is counter productive to have to deal with that \n> in a duplicated piece of code.  And the bare reality is that people will \n> just not care to parse it right themselves. \n\nNevertheless, they _should_. The principle is simple -- wherever there\nis ambiguity, you should seek to resolve that as _close_ to the point of\norigin as possible. Your 'best guess' gets worse and worse the further\nyou go from the source of the data.\n\nIf you're exporting from a legacy repository in one part of the world,\nthen transferring the raw data to a machine elsewhere to be imported\ninto git, you _really_ want to be making your guesses about timezones\nand character sets in the _export_ stage; not the subsequent import.\n\nSo there's a lot to be said for nailing down gfi's intermediate format\nand removing _all_ the ambiguity from it -- using git format dates\n(which I did that way precisely for the lack of ambiguity), and using\nUTF-8 (or some other _specified_ but not assumed character set).\n\n-- \ndwmw2\n"},{"id":"33793","messageId":"Pine.LNX.4.63.0702071216330.22628@wbgn013.biozentrum.uni-wuerzburg.de","threadId":"6706","inReplyTo":"20070207091351.GA1632@diana.vm.bytemark.co.uk","subject":"Re: git-fast-import","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2007-02-07T11:17:45Z","receivedAt":"2007-02-07T11:17:45Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Wed, 7 Feb 2007, Karl Hasselström wrote:\n\n> On 2007-02-06 23:55:46 -0500, Daniel Barkalow wrote:\n> \n> > It might be nice to have a git-fast-export, which could actually be\n> > potentially useful for generating a repository with systematic\n> > differences from the original. (E.g., to make a repository of git's\n> > Documentation directory, with just the commits that affect it)\n> \n> Or to solve problems like\n> \n>   Gaaah! This file we've had in the repository for the last 17 months\n>   has copyright problems and we can't distribute it!\n> \n> or\n> \n>   Wouldn't it be nice to permanently include all that old Linux\n>   history that's currently grafted onto the \"real\" history?\n> \n> In other words, general history rewriting, but fast.\n\nFor this, it would be better to use a different approach: fast-import \nstill hashes all the objects, which would not be necessary when rewriting. \nI guess that is what cogito's tool is doing.\n\nCiao,\nDscho\n"},{"id":"33803","messageId":"1170855490.29759.1155.camel@pmac.infradead.org","threadId":"6706","inReplyTo":"Pine.LNX.4.64.0702062347060.20138@iabervon.org","subject":"Re: git-fast-import","fromName":"David Woodhouse","fromEmail":"dwmw2@infradead.org","sentAt":"2007-02-07T13:38:10Z","receivedAt":"2007-02-07T13:38:10Z","isPatch":false,"sender":{"key":"dwmw2@infradead.org","avatar":"https://gravatar.com/avatar/7afd4f07e0cf7d7e046ae2d23678296b37777c96488e6f3451e78a5514154ebd?d=mp&s=160"},"body":"On Tue, 2007-02-06 at 23:55 -0500, Daniel Barkalow wrote:\n> It might be nice to have a git-fast-export, which could actually be \n> potentially useful for generating a repository with systematic differences \n> from the original. (E.g., to make a repository of git's Documentation \n> directory, with just the commits that affect it) \n\nThat kind of thing isn't hard to do. See the scripts which create the\n'JFFS2 for eCos' git tree or the 'exported kernel headers' git tree,\ndirectly from Linus' git tree.\n\n-- \ndwmw2\n"},{"id":"33862","messageId":"200702072218.l17MIeSD005420@laptop13.inf.utfsm.cl","threadId":"6706","inReplyTo":"20070207055352.GB7138@spearce.org","subject":"Re: git-fast-import","fromName":"Horst H. von Brand","fromEmail":"vonbrand@inf.utfsm.cl","sentAt":"2007-02-07T22:18:40Z","receivedAt":"2007-02-07T22:18:40Z","isPatch":false,"sender":{"key":"vonbrand@inf.utfsm.cl","avatar":"https://avatars.githubusercontent.com/u/211384?v=4"},"body":"Shawn O. Pearce <spearce@spearce.org> wrote:\n> Linus Torvalds <torvalds@linux-foundation.org> wrote:\n> > Btw, one thing that might be a good idea to document very clearly:\n> > \n> >  - in the native git format, the offset from UTC has *nothing* to do with \n> >    the actual time itself. The time in native git is always in UTC, and \n> >    the offset from UTC does not change \"time\" - it's purely there to tell \n> >    in which timezone the event happened.\n> > \n> >    So 12345678 +0000 and 12345678 -0700 are *exactly*the*same*date*, \n> >    except event one happened in UTC, and the other happened in UTC-7.\n> > \n> >  - in rfc2822 format, the offset from UTC actually *changes* the date. The \n> >    date \"Oct 12, 2006 20:00:00\" will be two _different_ times when you say \n> >    it is in PST or in UTC.\n> \n> Here is the current language relating to date parsing in gfi:\n> \n> Date Formats\n> ~~~~~~~~~~~~\n\n[...]\n\n> +\n> If the local offset is not available in the source material, use\n> ``+0000'', or the most common local offset.  For example many\n> organizations have a CVS repository which has only ever been accessed\n> by users who are located in the same location and timezone.  In this\n> case the offset from UTC can be easily assumed.\n\nNo, it can't. There are summer/winter times, etc.\n\n> +\n> Unlike the `rfc2822` format, this format is very strict.  Any\n> variation in formatting will cause gfi to reject the value.\n> \n> `rfc2822`::\n> \tThis is the standard email format as described by RFC 2822.\n> +\n> An example value is ``Tue Feb 6 11:22:18 2007 -0500''.  The Git\n> parser is accurate, but a little on the lenient side.  Its the\n> same parser used by gitlink:git-am[1] when applying patches\n> received from email.\n> +\n> Some malformed strings may be accepted as valid dates.  In some of\n> these cases Git will still be able to obtain the correct date from\n> the malformed string.  There are also some types of malformed\n> strings which Git will parse wrong, and yet consider valid.\n> Seriously malformed strings will be rejected.\n> +\n> Unlike the `raw` format above, the timezone/UTC offset information\n> contained in an RFC 2822 date string is used to adjust the date\n> value to UTC prior to storage.  Therefore it is important that\n> this information be as accurate as possible.\n\nSay what? If I use the \"raw\" format with UTC offset, the offset is just\nignored then?\n\n> +\n> If the source material is formatted in RFC 2822 style dates,\n\n\"uses RFC 2822 style dates\" would be better\n\n> the frontend should let gfi handle the parsing and conversion\n> (rather than attempting to do it itself) as the Git parser has\n> been well tested in the wild.\n> +\n> Frontends should prefer the `raw` format if the source material\n> is already in UNIX-epoch format, or is easily convertible to\n\n\"already uses Unix-epoch format, can be coaxed to give dates in that\nformat, or its format is easily convertible to it\" sounds better to me\n\n> that format, as there is no ambiguity in parsing.\n> \n> `now`::\n> \tAlways use the current time and timezone.  The literal\n> \t`now` must always be supplied for `<when>`.\n\n[...]\n\n> +\n> If separate `author` and `committer` commands are used in a `commit`\n> the timestamps may not match, as the system clock will be polled\n> twice (once for each command).\n\nBetter fix that. It can't be that costly to call gettimeofday(2) once and\nsquirrel the result away for later use.\n\n>                                 The only way to ensure that both\n> author and committer identity information has the same timestamp\n> is to omit `author` (thus copying from `committer`) or to use a\n> date format other than `now`.\n\nSee?\n-- \nDr. Horst H. von Brand                   User #22616 counter.li.org\nDepartamento de Informatica                    Fono: +56 32 2654431\nUniversidad Tecnica Federico Santa Maria             +56 32 2654239\nCasilla 110-V, Valparaiso, Chile               Fax:  +56 32 2797513\n"},{"id":"33864","messageId":"eqdk0t$jo5$2@sea.gmane.org","threadId":"6706","inReplyTo":"200702072218.l17MIeSD005420@laptop13.inf.utfsm.cl","subject":"Re: git-fast-import","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2007-02-07T22:31:59Z","receivedAt":"2007-02-07T22:31:59Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Horst H. von Brand wrote:\n> Shawn O. Pearce <spearce@spearce.org> wrote:\n\n>> Unlike the `raw` format above, the timezone/UTC offset information\n>> contained in an RFC 2822 date string is used to adjust the date\n>> value to UTC prior to storage.  Therefore it is important that\n>> this information be as accurate as possible.\n> \n> Say what? If I use the \"raw\" format with UTC offset, the offset is just\n> ignored then?\n\nIt is saved, and used only when _displaying_ human readable date.\n-- \nJakub Narebski\nWarsaw, Poland\nShadeHawk on #git\n"},{"id":"33866","messageId":"Pine.LNX.4.64.0702071428110.8424@woody.linux-foundation.org","threadId":"6706","inReplyTo":"200702072218.l17MIeSD005420@laptop13.inf.utfsm.cl","subject":"Re: git-fast-import","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2007-02-07T22:39:02Z","receivedAt":"2007-02-07T22:39:02Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Wed, 7 Feb 2007, Horst H. von Brand wrote:\n> \n> Say what? If I use the \"raw\" format with UTC offset, the offset is just\n> ignored then?\n\nThe offset that git maintaines is basically always ignored by git except \nfor pure printout purposes.\n\nFor example, when you traverse commits, git normally picks the next \nreachable commit to show by using the date. The UTC offset has no effect \non anything.\n\nIn fact, when we parse a commit, we don't even *parse* the timezone info. \nLook in commit.c: parse_commit_date. The timezone really doesn't even \nexist as far as any \"real\" git operation is concerned. It's just saved \naway, and it's _shown_ in \"git log\", but it has no real meaning apart from \nthat.\n\nSo git very much only works on UTC time internally, and the only thing \nthat actually matters in a string like \"1234567890 -0700\" is the first \npart. The \"-0700\" is _literally_ just a comment that is only ever even \nparsed by \"pretty_print_commit()\".\n\nBtw, CVS doesn't have any TZ info at all, so CVS also internally always \nsaves in UTC. It then tends to print out logs in whatever timezone you \nhappen to be in at the time of printout, afaik. \n\n\t\tLinus\n"},{"id":"33869","messageId":"20070207225532.GA1970@spearce.org","threadId":"6706","inReplyTo":"Pine.LNX.4.63.0702071216330.22628@wbgn013.biozentrum.uni-wuerzburg.de","subject":"Re: git-fast-import","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2007-02-07T22:55:32Z","receivedAt":"2007-02-07T22:55:32Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:\n> On Wed, 7 Feb 2007, Karl Hasselström wrote:\n> > In other words, general history rewriting, but fast.\n> \n> For this, it would be better to use a different approach: fast-import \n> still hashes all the objects, which would not be necessary when rewriting. \n> I guess that is what cogito's tool is doing.\n\ngfi doesn't require that it rehash blob objects.\n\nIf the blobs in question are already available in the repository\ngfi is running against (say, from the old branch history) you\ncan just feed those blob SHA-1s at gfi in its 'M' commands when\nmaking commits.  Yes gfi will need to recompute the tree hashes\nfrom scratch, but those are certainly smaller and faster to create\nthan blobs.\n\nSo you probably could make a faster history rewriter by taking\nthe output of say `git log --pretty=raw --raw -z`, filter that and\nreverse it, and stream it into gfi.  It probably would kick Cogito's\ncg-admin-rewritefilter thing in the teeth, as you are forking just\none gfi process rather than a thousand git-commit-tree processes.\n\nAnd if you are doing more complex pathname translations than just\npicking out a subtree, it also completely avoids needing to read and\nwrite index files via update-index, or tree objects by write-tree.\n\n-- \nShawn.\n"},{"id":"33888","messageId":"Pine.LNX.4.63.0702080054180.22628@wbgn013.biozentrum.uni-wuerzburg.de","threadId":"6706","inReplyTo":"20070207225532.GA1970@spearce.org","subject":"Re: git-fast-import","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2007-02-07T23:55:06Z","receivedAt":"2007-02-07T23:55:06Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Wed, 7 Feb 2007, Shawn O. Pearce wrote:\n\n> Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:\n> > On Wed, 7 Feb 2007, Karl Hasselström wrote:\n> > > In other words, general history rewriting, but fast.\n> > \n> > For this, it would be better to use a different approach: fast-import \n> > still hashes all the objects, which would not be necessary when rewriting. \n> > I guess that is what cogito's tool is doing.\n> \n> gfi doesn't require that it rehash blob objects.\n> \n> If the blobs in question are already available in the repository\n> gfi is running against (say, from the old branch history) you\n> can just feed those blob SHA-1s at gfi in its 'M' commands when\n> making commits.\n\nAh! I overlooked that feature. Certainly, this makes gfi (could we please \ncall it \"fast-import\", please?) very useful for history rewriting \npurposed.\n\nCiao,\nDscho"},{"id":"33893","messageId":"20070208001257.GB2469@spearce.org","threadId":"6706","inReplyTo":"Pine.LNX.4.63.0702080054180.22628@wbgn013.biozentrum.uni-wuerzburg.de","subject":"Re: git-fast-import","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2007-02-08T00:12:57Z","receivedAt":"2007-02-08T00:12:57Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:\n> Ah! I overlooked that feature. Certainly, this makes gfi (could we please \n> call it \"fast-import\", please?) very useful for history rewriting \n> purposed.\n\nHeh.  I was actually sort of thinking of renaming it git-gfi.  :)\n\ngit-fast-import is just too long to write.  And for some reason I\nhave been writing it a lot lately.  #git, email, git-fast-export's\nmanual page (which is now also the largest manual page in all\nof Git!).\n\nBut of course the better name is git-fast-import.  Stealing a\nthree-letter non-hypen-containing name for a tool the user never\nis meant to run by hand is just evil.\n\n\nI haven't even tried to use fast-import for general history\nrewriting, let alone benchmarked it against something like git-split\nor Cogito's rewriting tool, but I'd be willing to be that fast-import\nis faster.  The internal ``cache'' that it uses for the tree\nconstruction is lightweight enough that gfi can probably recreate\nonly the modified trees, compress and hash them, and output what\nit needs to, in the time it takes to fork+exec git-commit-tree.\n\n-- \nShawn.\n"},{"id":"33943","messageId":"Pine.LNX.4.64.0702080851140.8424@woody.linux-foundation.org","threadId":"6706","inReplyTo":"Pine.LNX.4.63.0702080054180.22628@wbgn013.biozentrum.uni-wuerzburg.de","subject":"Re: git-fast-import","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2007-02-08T16:56:08Z","receivedAt":"2007-02-08T16:56:08Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Thu, 8 Feb 2007, Johannes Schindelin wrote:\n> \n> Ah! I overlooked that feature. Certainly, this makes gfi (could we please \n> call it \"fast-import\", please?) very useful for history rewriting \n> purposed.\n\nYeah, I think fast-import is great. And I'd also like to echo that call to \nnot call it \"gfi\". Maybe it's just me, and maybe it's just because I'm a \nhome-owner who does things like add in-wall ethernet cables, but to me, \ngfi is about an electrical outlet.\n\nSo to me, gfi means \"ground fault interrupter\": the kind of outlet that \nbreaks the circuit if there is current leaking to the ground pin. All your \nelectrical outlets in \"wet areas\" (bathroom, kitchen within a certain \ndistance of a sink, outside, near swimming pools etc) are supposed to be \nGFI's.\n\nI realize that there's not a lot of chance of confusion in the git world, \nbut still.\n\n\t\t\tLinus\n"},{"id":"33954","messageId":"20070208191024.GA537@spearce.org","threadId":"6706","inReplyTo":"Pine.LNX.4.64.0702080851140.8424@woody.linux-foundation.org","subject":"Re: git-fast-import","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2007-02-08T19:10:24Z","receivedAt":"2007-02-08T19:10:24Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Linus Torvalds <torvalds@linux-foundation.org> wrote:\n> On Thu, 8 Feb 2007, Johannes Schindelin wrote:\n> > \n> > Ah! I overlooked that feature. Certainly, this makes gfi (could we please \n> > call it \"fast-import\", please?) very useful for history rewriting \n> > purposed.\n> \n> Yeah, I think fast-import is great. And I'd also like to echo that call to \n> not call it \"gfi\". Maybe it's just me, and maybe it's just because I'm a \n> home-owner who does things like add in-wall ethernet cables, but to me, \n> gfi is about an electrical outlet.\n\nOK.  There happen to be 78 uses of `gfi` in the manpage.\nI'll correct the spelling to fast-import.  :-)\n\n-- \nShawn.\n"},{"id":"33979","messageId":"Pine.LNX.4.63.0702082227170.22628@wbgn013.biozentrum.uni-wuerzburg.de","threadId":"6706","inReplyTo":"Pine.LNX.4.64.0702061832020.8424@woody.linux-foundation.org","subject":"Re: git-fast-import","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2007-02-08T21:34:17Z","receivedAt":"2007-02-08T21:34:17Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Tue, 6 Feb 2007, Linus Torvalds wrote:\n\n> [Talks about timestamps being in UTC, even if augmented by a timezone]\n>\n> And yes, for all I know we might get this wrong inside git too. It's \n> easy to get confused, because they really do mean different things.\n\nFWIW I just grepped git for tz, and looked at the results. The place I had \nto think a bit more about was in builtin-blame.c:format_time(). Probably a \nspecial date format is needed to stay compatible with cvsserver, otherwise \nshow_date() or even show_rfc2822_date() could be used.\n\nThe code actually adds the timezone in minutes to the timestamp, and then \ncalls gmtime() to be able to format the date with strftime() (something \nsimilar, without strftime() is done in show_[rfc2822_]date()). The result \nis correct AFAICT, although it would be cleaner IMHO to add yet another \nfunction to date.c which formats the time according to cvsserver's wishes.\n\nPost 1.5.0.\n\nCiao,\nDscho\n"},{"id":"34046","messageId":"20070209084912.GA12952@diana.vm.bytemark.co.uk","threadId":"6706","inReplyTo":"20070208191024.GA537@spearce.org","subject":"Re: git-fast-import","fromName":"Karl Hasselström","fromEmail":"kha@treskal.com","sentAt":"2007-02-09T08:49:12Z","receivedAt":"2007-02-09T08:49:12Z","isPatch":false,"sender":{"key":"kha@treskal.com","avatar":"https://gravatar.com/avatar/f0120c734b5279b345075a28521e1ac66acb20c9913ffe9bf6ae97e53f7f3f13?d=mp&s=160"},"body":"On 2007-02-08 14:10:24 -0500, Shawn O. Pearce wrote:\n\n> Linus Torvalds <torvalds@linux-foundation.org> wrote:\n>\n> > Yeah, I think fast-import is great. And I'd also like to echo that\n> > call to not call it \"gfi\". Maybe it's just me, and maybe it's just\n> > because I'm a home-owner who does things like add in-wall ethernet\n> > cables, but to me, gfi is about an electrical outlet.\n>\n> OK. There happen to be 78 uses of `gfi` in the manpage. I'll correct\n> the spelling to fast-import. :-)\n\nDidn't you listen to what Linus said? Near porcelain and plumbing is\nprecisely where you _need_ gfi!\n\n-- \nKarl Hasselström, kha@treskal.com\n      www.treskal.com/kalle\n"},{"id":"34064","messageId":"Pine.LNX.4.64.0702090746180.8424@woody.linux-foundation.org","threadId":"6706","inReplyTo":"20070209084912.GA12952@diana.vm.bytemark.co.uk","subject":"Re: git-fast-import","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2007-02-09T15:47:40Z","receivedAt":"2007-02-09T15:47:40Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Fri, 9 Feb 2007, Karl Hasselström wrote:\n> \n> Didn't you listen to what Linus said? Near porcelain and plumbing is\n> precisely where you _need_ gfi!\n\nGroan.\n\nThe whole git project is apparently infected with a terminal case of the \npuns. \n\nAnd they keep on getting more and more obscure.\n\n\t\tLinus"}]}