{"thread":{"id":"26474","subject":"[PATCH v3] fast-import: add 'ls' command","startedAt":"2011-02-11T22:43:39Z","lastAt":"2011-02-12T01:36:43Z","messageCount":7,"participants":["Jonathan Nieder","Sverre Rabbelier","Junio C Hamano"],"isPatch":true,"patchVersion":3,"patchTotal":null},"messages":[{"id":"160913","messageId":"20110211224339.GA9973@elie","threadId":"26474","inReplyTo":null,"subject":"[PATCH v3] fast-import: add 'ls' command","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2011-02-11T22:43:39Z","receivedAt":"2011-02-11T22:43:39Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"From: David Barr <david.barr@cordelta.com>\nDate: Thu, 2 Dec 2010 21:40:20 +1100\n\nIntroduce an \"ls\" command to read directory entries from the active\ncommit or a named commit.  This allows printing a blob from the active\ncommit or copying a blob or tree from a previous commit for use in the\ncurrent one.\n\nThere are two forms of the 'ls' command: the two-argument form prints\nthe entry at <path> for the tree underlying the tree, commit, or tag\nnamed by <dataref>:\n\n\t'ls' SP <dataref> SP <path> LF\n\nThe one-argument form prints the entry at <path> in fast-import's\nactive commit.\n\n\t'ls' SP <path> LF\n\nDirty hack: for now, git fast-import will treat missing paths as empty\nsubtrees and print them as\n\n 040000 tree 4b825dc642cb6eb9a060e54bf8d69288fbee4904\tpath/to/nowhere\n\nto avoid confusing frontends that inserted such a path before.  But\nan alternate output format\n\n\t'missing' SP <path> LF\n\nis also allowed, so other backends (and perhaps git some day) can\nconvey the difference between an empty subtree and no subtree at all.\n\nProposed updates to svn-fe use the \"ls\" command to delegate\nresponsibility for knowledge about paths to git.  Reading such\ninformation from the target repository instead of a data structure\nmaintained by the fast-import frontend simplifies the frontend a great\ndeal and means support for resuming an import in a separate\nfast-import run (i.e., incremental import) is basically free.\n\nSigned-off-by: David Barr <david.barr@cordelta.com>\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\nLast seen at [1].\n\nChanges from v2:\n - clearer documentation\n - has a corresponding 'feature'\n - tiny clarification to parse_ls\n - rebased against master (also applies on maint fwiw).\n\nThe vcs-fast-import-devs don't seem to dislike it, at least. :)\n\nThanks for the help and patience.\n\n[1] http://thread.gmane.org/gmane.comp.version-control.git/162698/focus=165553\n\n Documentation/git-fast-import.txt |   65 ++++++++++++++-\n fast-import.c                     |  164 ++++++++++++++++++++++++++++++++++++-\n t/t9300-fast-import.sh            |  112 +++++++++++++++++++++++--\n 3 files changed, 327 insertions(+), 14 deletions(-)\n\ndiff --git a/Documentation/git-fast-import.txt b/Documentation/git-fast-import.txt\nindex c3a2766..4f92954 100644\n--- a/Documentation/git-fast-import.txt\n+++ b/Documentation/git-fast-import.txt\n@@ -196,7 +196,8 @@ especially when a higher level language such as Perl, Python or\n Ruby is being used.\n \n fast-import is very strict about its input.  Where we say SP below we mean\n-*exactly* one space.  Likewise LF means one (and only one) linefeed.\n+*exactly* one space.  Likewise LF means one (and only one) linefeed\n+and HT one (and only one) horizontal tab.\n Supplying additional whitespace characters will cause unexpected\n results, such as branch names or file names with leading or trailing\n spaces in their name, or early termination of fast-import when it encounters\n@@ -334,6 +335,11 @@ and control the current import process.  More detailed discussion\n \tformat to the file descriptor set with `--cat-blob-fd` or\n \t`stdout` if unspecified.\n \n+`ls`::\n+\tCauses fast-import to print a directory entry in 'ls-tree'\n+\tformat to the file descriptor set with `--cat-blob-fd` or\n+\t`stdout` if unspecified.\n+\n `feature`::\n \tRequire that fast-import supports the specified feature, or\n \tabort if it does not.\n@@ -919,6 +925,57 @@ This command can be used anywhere in the stream that comments are\n accepted.  In particular, the `cat-blob` command can be used in the\n middle of a commit but not in the middle of a `data` command.\n \n+`ls`\n+~~~~\n+Prints a directory entry to a file descriptor previously arranged with\n+the `--cat-blob-fd` argument.  This allows printing a blob from the\n+active commit (with `cat-blob`) or copying a blob or tree from a\n+previous commit for use in the current one (with `filemodify`).\n+\n+The `ls` command can be used anywhere in the stream that comments are\n+accepted, including the middle of a commit.\n+\n+Reading from the active commit::\n+\tThis form can only be used in the middle of a `commit`.\n+\tThe path names a directory entry within fast-import's\n+\tactive commit.  The path must be quoted in this case.\n++\n+....\n+\t'ls' SP <path> LF\n+....\n+\n+Reading from a named tree::\n+\tThe `<dataref>` can be a mark reference (`:<idnum>`) or the\n+\tfull 40-byte SHA-1 of a Git tag, commit, or tree object,\n+\tpreexisting or waiting to be written.\n+\tThe path is relative to the top level of the tree\n+\tnamed by `<dataref>`.\n++\n+....\n+\t'ls' SP <dataref> SP <path> LF\n+....\n+\n+See `filemodify` above for a detailed description of `<path>`.\n+\n+Output uses the same format as `git ls-tree <tree> -- <path>`:\n+\n+====\n+\t<mode> SP ('blob' | 'tree' | 'commit') SP <dataref> HT <path> LF\n+====\n+\n+The <dataref> represents the blob, tree, or commit object at <path>\n+and can be used in later 'cat-blob', 'filemodify', or 'ls' commands.\n+\n+Git repositories do not distinguish between missing paths and empty\n+subtrees.  If a path is not found, 'git fast-import' will report it as\n+an empty tree.  Backends that do have a notion of empty trees may write\n+\n+====\n+\tmissing SP <path> LF\n+====\n+\n+for paths that do not correspond to a blob or subtree.\n+\n `feature`\n ~~~~~~~~~\n Require that fast-import supports the specified feature, or abort if\n@@ -946,8 +1003,10 @@ import-marks::\n \tany \"feature import-marks\" command in the stream.\n \n cat-blob::\n-\tIgnored.  Versions of fast-import not supporting the\n-\t\"cat-blob\" command will exit with a message indicating so.\n+ls::\n+\tRequire that the backend support the 'cat-blob' or 'ls' command.\n+\tVersions of fast-import not supporting the specified command\n+\twill exit with a message indicating so.\n \tThis lets the import error out early with a clear message,\n \trather than wasting time on the early part of an import\n \tbefore the unsupported command is detected.\ndiff --git a/fast-import.c b/fast-import.c\nindex 3886a1b..cef2b8c 100644\n--- a/fast-import.c\n+++ b/fast-import.c\n@@ -24,10 +24,12 @@ Format of STDIN stream:\n     commit_msg\n     ('from' sp committish lf)?\n     ('merge' sp committish lf)*\n-    file_change*\n+    (file_change | ls)*\n     lf?;\n   commit_msg ::= data;\n \n+  ls ::= 'ls' sp '\"' quoted(path) '\"' lf;\n+\n   file_change ::= file_clr\n     | file_del\n     | file_rnm\n@@ -132,7 +134,7 @@ Format of STDIN stream:\n   ts    ::= # time since the epoch in seconds, ascii base10 notation;\n   tz    ::= # GIT style timezone;\n \n-     # note: comments and cat requests may appear anywhere\n+     # note: comments, ls and cat requests may appear anywhere\n      # in the input, except within a data command.  Any form\n      # of the data command always escapes the related input\n      # from comment processing.\n@@ -141,7 +143,9 @@ Format of STDIN stream:\n      # must be the first character on that line (an lf\n      # preceded it).\n      #\n+\n   cat_blob ::= 'cat-blob' sp (hexsha1 | idnum) lf;\n+  ls_tree  ::= 'ls' sp (hexsha1 | idnum) sp path_str lf;\n \n   comment ::= '#' not_lf* lf;\n   not_lf  ::= # Any byte that is not ASCII newline (LF);\n@@ -374,6 +378,7 @@ static int cat_blob_fd = STDOUT_FILENO;\n \n static void parse_argv(void);\n static void parse_cat_blob(void);\n+static void parse_ls(struct branch *b);\n \n static void write_branch_report(FILE *rpt, struct branch *b)\n {\n@@ -2614,6 +2619,8 @@ static void parse_new_commit(void)\n \t\t\tnote_change_n(b, prev_fanout);\n \t\telse if (!strcmp(\"deleteall\", command_buf.buf))\n \t\t\tfile_change_deleteall(b);\n+\t\telse if (!prefixcmp(command_buf.buf, \"ls \"))\n+\t\t\tparse_ls(b);\n \t\telse {\n \t\t\tunread_command_buf = 1;\n \t\t\tbreak;\n@@ -2837,6 +2844,155 @@ static void parse_cat_blob(void)\n \tcat_blob(oe, sha1);\n }\n \n+static struct object_entry *dereference(struct object_entry *oe,\n+\t\t\t\t\tunsigned char sha1[20])\n+{\n+\tunsigned long size;\n+\tvoid *buf = NULL;\n+\tif (!oe) {\n+\t\tenum object_type type = sha1_object_info(sha1, NULL);\n+\t\tif (type < 0)\n+\t\t\tdie(\"object not found: %s\", sha1_to_hex(sha1));\n+\t\t/* cache it! */\n+\t\toe = insert_object(sha1);\n+\t\toe->type = type;\n+\t\toe->pack_id = MAX_PACK_ID;\n+\t\toe->idx.offset = 1;\n+\t}\n+\tswitch (oe->type) {\n+\tcase OBJ_TREE:\t/* easy case. */\n+\t\treturn oe;\n+\tcase OBJ_COMMIT:\n+\tcase OBJ_TAG:\n+\t\tbreak;\n+\tdefault:\n+\t\tdie(\"Not a treeish: %s\", command_buf.buf);\n+\t}\n+\n+\tif (oe->pack_id != MAX_PACK_ID) {\t/* in a pack being written */\n+\t\tbuf = gfi_unpack_entry(oe, &size);\n+\t} else {\n+\t\tenum object_type unused;\n+\t\tbuf = read_sha1_file(sha1, &unused, &size);\n+\t}\n+\tif (!buf)\n+\t\tdie(\"Can't load object %s\", sha1_to_hex(sha1));\n+\n+\t/* Peel one layer. */\n+\tswitch (oe->type) {\n+\tcase OBJ_TAG:\n+\t\tif (size < 40 + strlen(\"object \") ||\n+\t\t    get_sha1_hex(buf + strlen(\"object \"), sha1))\n+\t\t\tdie(\"Invalid SHA1 in tag: %s\", command_buf.buf);\n+\t\tbreak;\n+\tcase OBJ_COMMIT:\n+\t\tif (size < 40 + strlen(\"tree \") ||\n+\t\t    get_sha1_hex(buf + strlen(\"tree \"), sha1))\n+\t\t\tdie(\"Invalid SHA1 in commit: %s\", command_buf.buf);\n+\t}\n+\n+\tfree(buf);\n+\treturn find_object(sha1);\n+}\n+\n+static struct object_entry *parse_treeish_dataref(const char **p)\n+{\n+\tunsigned char sha1[20];\n+\tstruct object_entry *e;\n+\n+\tif (**p == ':') {\t/* <mark> */\n+\t\tchar *endptr;\n+\t\te = find_mark(strtoumax(*p + 1, &endptr, 10));\n+\t\tif (endptr == *p + 1)\n+\t\t\tdie(\"Invalid mark: %s\", command_buf.buf);\n+\t\tif (!e)\n+\t\t\tdie(\"Unknown mark: %s\", command_buf.buf);\n+\t\t*p = endptr;\n+\t\thashcpy(sha1, e->idx.sha1);\n+\t} else {\t/* <sha1> */\n+\t\tif (get_sha1_hex(*p, sha1))\n+\t\t\tdie(\"Invalid SHA1: %s\", command_buf.buf);\n+\t\te = find_object(sha1);\n+\t\t*p += 40;\n+\t}\n+\n+\twhile (!e || e->type != OBJ_TREE)\n+\t\te = dereference(e, sha1);\n+\treturn e;\n+}\n+\n+static void print_ls(int mode, const unsigned char *sha1, const char *path)\n+{\n+\tstatic struct strbuf line = STRBUF_INIT;\n+\n+\t/* See show_tree(). */\n+\tconst char *type =\n+\t\tS_ISGITLINK(mode) ? commit_type :\n+\t\tS_ISDIR(mode) ? tree_type :\n+\t\tblob_type;\n+\n+\t/* mode SP type SP object_name TAB path LF */\n+\tstrbuf_reset(&line);\n+\tstrbuf_addf(&line, \"%06o %s %s\\t\",\n+\t\t\tmode, type, sha1_to_hex(sha1));\n+\tquote_c_style(path, &line, NULL, 0);\n+\tstrbuf_addch(&line, '\\n');\n+\tcat_blob_write(line.buf, line.len);\n+}\n+\n+static void parse_ls(struct branch *b)\n+{\n+\tconst char *p;\n+\tstruct tree_entry *root = NULL;\n+\tstruct tree_entry leaf = {0};\n+\n+\t/* ls SP (<treeish> SP)? <path> */\n+\tp = command_buf.buf + strlen(\"ls \");\n+\tif (*p == '\"') {\n+\t\tif (!b)\n+\t\t\tdie(\"Not in a commit: %s\", command_buf.buf);\n+\t\troot = &b->branch_tree;\n+\t} else {\n+\t\tstruct object_entry *e = parse_treeish_dataref(&p);\n+\t\troot = new_tree_entry();\n+\t\thashcpy(root->versions[1].sha1, e->idx.sha1);\n+\t\tload_tree(root);\n+\t\tif (*p++ != ' ')\n+\t\t\tdie(\"Missing space after tree-ish: %s\", command_buf.buf);\n+\t}\n+\tif (*p == '\"') {\n+\t\tstatic struct strbuf uq = STRBUF_INIT;\n+\t\tconst char *endp;\n+\t\tstrbuf_reset(&uq);\n+\t\tif (unquote_c_style(&uq, p, &endp))\n+\t\t\tdie(\"Invalid path: %s\", command_buf.buf);\n+\t\tif (*endp)\n+\t\t\tdie(\"Garbage after path in: %s\", command_buf.buf);\n+\t\tp = uq.buf;\n+\t}\n+\ttree_content_get(root, p, &leaf);\n+\t/*\n+\t * A directory in preparation would have a sha1 of zero\n+\t * until it is saved.  Save, for simplicity.\n+\t */\n+\tif (S_ISDIR(leaf.versions[1].mode))\n+\t\tstore_tree(&leaf);\n+\n+\tif (!leaf.versions[1].mode) {\n+\t\t/*\n+\t\t * Missing path?  Must be an empty subtree!\n+\t\t *\n+\t\t * When git learns to track empty directories, we can report\n+\t\t * this by saying 'missing \"path/to/directory\"' instead.\n+\t\t */\n+\t\tprint_ls(S_IFDIR, (const unsigned char *) EMPTY_TREE_SHA1_BIN, p);\n+\t} else {\n+\t\tprint_ls(leaf.versions[1].mode, leaf.versions[1].sha1, p);\n+\t}\n+\tif (!b || root != &b->branch_tree)\n+\t\trelease_tree_entry(root);\n+}\n+\n static void checkpoint(void)\n {\n \tcheckpoint_requested = 0;\n@@ -3001,7 +3157,7 @@ static int parse_one_feature(const char *feature, int from_stream)\n \t\trelative_marks_paths = 0;\n \t} else if (!prefixcmp(feature, \"force\")) {\n \t\tforce_update = 1;\n-\t} else if (!strcmp(feature, \"notes\")) {\n+\t} else if (!strcmp(feature, \"notes\") || !strcmp(feature, \"ls\")) {\n \t\t; /* do nothing; we have the feature */\n \t} else {\n \t\treturn 0;\n@@ -3142,6 +3298,8 @@ int main(int argc, const char **argv)\n \twhile (read_next_command() != EOF) {\n \t\tif (!strcmp(\"blob\", command_buf.buf))\n \t\t\tparse_new_blob();\n+\t\telse if (!prefixcmp(command_buf.buf, \"ls \"))\n+\t\t\tparse_ls(NULL);\n \t\telse if (!prefixcmp(command_buf.buf, \"commit \"))\n \t\t\tparse_new_commit();\n \t\telse if (!prefixcmp(command_buf.buf, \"tag \"))\ndiff --git a/t/t9300-fast-import.sh b/t/t9300-fast-import.sh\nindex 52ac0e5..f70d2a9 100755\n--- a/t/t9300-fast-import.sh\n+++ b/t/t9300-fast-import.sh\n@@ -42,6 +42,14 @@ echo \"$@\"'\n \n >empty\n \n+test_expect_success 'setup: have pipes?' '\n+\trm -f frob &&\n+\tif mkfifo frob\n+\tthen\n+\t\ttest_set_prereq PIPE\n+\tfi\n+'\n+\n ###\n ### series A\n ###\n@@ -898,6 +906,97 @@ test_expect_success \\\n \t git diff-tree -C --find-copies-harder -r N4^ N4 >actual &&\n \t compare_diff_raw expect actual'\n \n+test_expect_success PIPE 'N: read and copy directory' '\n+\tcat >expect <<-\\EOF\n+\t:100755 100755 f1fb5da718392694d0076d677d6d0e364c79b0bc f1fb5da718392694d0076d677d6d0e364c79b0bc C100\tfile2/newf\tfile3/newf\n+\t:100644 100644 7123f7f44e39be127c5eb701e5968176ee9d78b1 7123f7f44e39be127c5eb701e5968176ee9d78b1 C100\tfile2/oldf\tfile3/oldf\n+\tEOF\n+\tgit update-ref -d refs/heads/N4 &&\n+\trm -f backflow &&\n+\tmkfifo backflow &&\n+\t(\n+\t\texec <backflow &&\n+\t\tcat <<-EOF &&\n+\t\tcommit refs/heads/N4\n+\t\tcommitter $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE\n+\t\tdata <<COMMIT\n+\t\tcopy by tree hash, part 2\n+\t\tCOMMIT\n+\n+\t\tfrom refs/heads/branch^0\n+\t\tls \"file2\"\n+\t\tEOF\n+\t\tread mode type tree filename &&\n+\t\techo \"M 040000 $tree file3\"\n+\t) |\n+\tgit fast-import --cat-blob-fd=3 3>backflow &&\n+\tgit diff-tree -C --find-copies-harder -r N4^ N4 >actual &&\n+\tcompare_diff_raw expect actual\n+'\n+\n+test_expect_success PIPE 'N: read and copy \"empty\" directory' '\n+\tcat <<-\\EOF >expect &&\n+\tOBJNAME\n+\t:000000 100644 OBJNAME OBJNAME A\tgreeting\n+\tOBJNAME\n+\t:100644 000000 OBJNAME OBJNAME D\tunrelated\n+\tOBJNAME\n+\t:000000 100644 OBJNAME OBJNAME A\tunrelated\n+\tEOF\n+\tgit update-ref -d refs/heads/copy-empty &&\n+\trm -f backflow &&\n+\tmkfifo backflow &&\n+\t(\n+\t\texec <backflow &&\n+\t\tcat <<-EOF &&\n+\t\tcommit refs/heads/copy-empty\n+\t\tcommitter $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE\n+\t\tdata <<COMMIT\n+\t\tcopy \"empty\" (missing) directory\n+\t\tCOMMIT\n+\n+\t\tM 100644 inline src/greeting\n+\t\tdata <<BLOB\n+\t\thello\n+\t\tBLOB\n+\t\tC src/greeting dst1/non-greeting\n+\t\tC src/greeting unrelated\n+\t\t# leave behind \"empty\" src directory\n+\t\tD src/greeting\n+\t\tls \"src\"\n+\t\tEOF\n+\t\tread mode type tree filename &&\n+\t\tsed -e \"s/X\\$//\" <<-EOF\n+\t\tM $mode $tree dst1\n+\t\tM $mode $tree dst2\n+\n+\t\tcommit refs/heads/copy-empty\n+\t\tcommitter $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE\n+\t\tdata <<COMMIT\n+\t\tcopy empty directory to root\n+\t\tCOMMIT\n+\n+\t\tM $mode $tree X\n+\n+\t\tcommit refs/heads/copy-empty\n+\t\tcommitter $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE\n+\t\tdata <<COMMIT\n+\t\tadd another file\n+\t\tCOMMIT\n+\n+\t\tM 100644 inline greeting\n+\t\tdata <<BLOB\n+\t\thello\n+\t\tBLOB\n+\t\tEOF\n+\t) |\n+\tgit fast-import --cat-blob-fd=3 3>backflow &&\n+\tgit rev-list copy-empty |\n+\tgit diff-tree -r --root --stdin |\n+\tsed \"s/$_x40/OBJNAME/g\" >actual &&\n+\ttest_cmp expect actual\n+'\n+\n test_expect_success \\\n \t'N: copy root directory by tree hash' \\\n \t'cat >expect <<-\\EOF &&\n@@ -1861,6 +1960,11 @@ test_expect_success 'R: feature no-relative-marks should be honoured' '\n     test_cmp marks.new non-relative.out\n '\n \n+test_expect_success 'R: feature ls supported' '\n+\techo \"feature ls\" |\n+\tgit fast-import\n+'\n+\n test_expect_success 'R: feature cat-blob supported' '\n \techo \"feature cat-blob\" |\n \tgit fast-import\n@@ -1986,14 +2090,6 @@ test_expect_success 'R: print two blobs to stdout' '\n \ttest_cmp expect actual\n '\n \n-test_expect_success 'setup: have pipes?' '\n-\trm -f frob &&\n-\tif mkfifo frob\n-\tthen\n-\t\ttest_set_prereq PIPE\n-\tfi\n-'\n-\n test_expect_success PIPE 'R: copy using cat-file' '\n \texpect_id=$(git hash-object big) &&\n \texpect_len=$(wc -c <big) &&\n-- \n1.7.4\n"},{"id":"160914","messageId":"AANLkTimReMYu7QYZEuunmH6WqUz=C0nOnAZHzB6cddXt@mail.gmail.com","threadId":"26474","inReplyTo":"20110211224339.GA9973@elie","subject":"Re: [PATCH v3] fast-import: add 'ls' command","fromName":"Sverre Rabbelier","fromEmail":"srabbelier@gmail.com","sentAt":"2011-02-11T22:47:08Z","receivedAt":"2011-02-11T22:47:08Z","isPatch":true,"sender":{"key":"srabbelier@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3098?v=4"},"body":"Heya,\n\nOn Fri, Feb 11, 2011 at 23:43, Jonathan Nieder <jrnieder@gmail.com> wrote:\n> Introduce an \"ls\" command to read directory entries from the active\n> commit or a named commit.  This allows printing a blob from the active\n> commit or copying a blob or tree from a previous commit for use in the\n> current one.\n\nUseful addition I think. What happens if you ask it for a path that is\nnot a directory?\n\n-- \nCheers,\n\nSverre Rabbelier\n"},{"id":"160915","messageId":"20110211225904.GA22391@elie","threadId":"26474","inReplyTo":"AANLkTimReMYu7QYZEuunmH6WqUz=C0nOnAZHzB6cddXt@mail.gmail.com","subject":"Re: [PATCH v3] fast-import: add 'ls' command","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2011-02-11T22:59:04Z","receivedAt":"2011-02-11T22:59:04Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Sverre Rabbelier wrote:\n> On Fri, Feb 11, 2011 at 23:43, Jonathan Nieder <jrnieder@gmail.com> wrote:\n\n>> Introduce an \"ls\" command to read directory entries from the active\n>> commit or a named commit.  This allows printing a blob from the active\n>> commit or copying a blob or tree from a previous commit for use in the\n>> current one.\n>\n> Useful addition I think. What happens if you ask it for a path that is\n> not a directory?\n\nHmm, the documentation was not at all clear, then.  Good catch.\n\nThe answer is that it always prints a single dirent, whether the path\nsupplied names a file or a directory.\n\n\tFE> ls :1 Documentation\n\tgfi> 040000 tree 9e6c2b599341d28a2a375f8207507e0a2a627fe9\tDocumentation\n\tFE> ls :1 git.c\n\tgfi> 100644 blob 23610aa0366ebe36e65e78fb8c5fba3f2d0b8f77\tgit.c\n\tFE> ls :1 thisfiledoesnotexist\n\tgfi> 040000 tree 4b825dc642cb6eb9a060e54bf8d69288fbee4904\tthisfiledoesnotexist\n\tFE> ls :1 RelNotes\n\tgfi> 120000 blob b942e499449d97aeb50c73ca2bdc1c6e6d528743\tRelNotes\n\tFE> ls 9e6c2b599341d28a2a375f8207507e0a2a627fe9 git-fast-import.txt\n\tgfi> 100644 blob 4f92954396e3f0f97e75b6838a5635b583708870\tgit-fast-import.txt\n\tFE> cat-blob b942e499449d97aeb50c73ca2bdc1c6e6d528743\n\tgfi> b942e499449d97aeb50c73ca2bdc1c6e6d528743 blob 32\n\tgfi> Documentation/RelNotes/1.7.4.txt\n\nMaybe this patch (for squashing) would help.\n\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\ndiff --git a/Documentation/git-fast-import.txt b/Documentation/git-fast-import.txt\nindex 4f92954..495e01f 100644\n--- a/Documentation/git-fast-import.txt\n+++ b/Documentation/git-fast-import.txt\n@@ -336,9 +336,9 @@ and control the current import process.  More detailed discussion\n \t`stdout` if unspecified.\n \n `ls`::\n-\tCauses fast-import to print a directory entry in 'ls-tree'\n-\tformat to the file descriptor set with `--cat-blob-fd` or\n-\t`stdout` if unspecified.\n+\tCauses fast-import to print a line describing a directory\n+\tentry in 'ls-tree' format to the file descriptor set with\n+\t`--cat-blob-fd` or `stdout` if unspecified.\n \n `feature`::\n \tRequire that fast-import supports the specified feature, or\n@@ -927,10 +927,11 @@ middle of a commit but not in the middle of a `data` command.\n \n `ls`\n ~~~~\n-Prints a directory entry to a file descriptor previously arranged with\n-the `--cat-blob-fd` argument.  This allows printing a blob from the\n-active commit (with `cat-blob`) or copying a blob or tree from a\n-previous commit for use in the current one (with `filemodify`).\n+Prints information about the object at a path to a file descriptor\n+previously arranged with the `--cat-blob-fd` argument.  This allows\n+printing a blob from the active commit (with `cat-blob`) or copying a\n+blob or tree from a previous commit for use in the current one (with\n+`filemodify`).\n \n The `ls` command can be used anywhere in the stream that comments are\n accepted, including the middle of a commit.\n@@ -957,7 +958,7 @@ Reading from a named tree::\n \n See `filemodify` above for a detailed description of `<path>`.\n \n-Output uses the same format as `git ls-tree <tree> -- <path>`:\n+Output uses the same format as `git ls-tree <tree> {litdd} <path>`:\n \n ====\n \t<mode> SP ('blob' | 'tree' | 'commit') SP <dataref> HT <path> LF\n"},{"id":"160917","messageId":"AANLkTikfAmFJoEXK1OYE7mKpY6xw=Xw830xS4jzk5mnQ@mail.gmail.com","threadId":"26474","inReplyTo":"20110211225904.GA22391@elie","subject":"Re: [PATCH v3] fast-import: add 'ls' command","fromName":"Sverre Rabbelier","fromEmail":"srabbelier@gmail.com","sentAt":"2011-02-11T23:51:07Z","receivedAt":"2011-02-11T23:51:07Z","isPatch":true,"sender":{"key":"srabbelier@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3098?v=4"},"body":"Heya,\n\nOn Fri, Feb 11, 2011 at 23:59, Jonathan Nieder <jrnieder@gmail.com> wrote:\n> Hmm, the documentation was not at all clear, then.  Good catch.\n>\n> The answer is that it always prints a single dirent, whether the path\n> supplied names a file or a directory.\n\nOk, makes sense :).\n\n-- \nCheers,\n\nSverre Rabbelier\n"},{"id":"160920","messageId":"7vlj1mytjv.fsf@alter.siamese.dyndns.org","threadId":"26474","inReplyTo":"20110211224339.GA9973@elie","subject":"Re: [PATCH v3] fast-import: add 'ls' command","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2011-02-12T00:40:04Z","receivedAt":"2011-02-12T00:40:04Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Nieder <jrnieder@gmail.com> writes:\n\n> From: David Barr <david.barr@cordelta.com>\n> Date: Thu, 2 Dec 2010 21:40:20 +1100\n>\n> Introduce an \"ls\" command to read directory entries from the active\n> commit or a named commit.  This allows printing a blob from the active\n> commit or copying a blob or tree from a previous commit for use in the\n> current one.\n>\n> There are two forms of the 'ls' command: the two-argument form prints\n> the entry at <path> for the tree underlying the tree, commit, or tag\n> named by <dataref>:\n>\n> \t'ls' SP <dataref> SP <path> LF\n>\n> The one-argument form prints the entry at <path> in fast-import's\n> active commit.\n>\n> \t'ls' SP <path> LF\n\nIs this really \"ls\"?\n\nObviously, an extended SHA-1 that is accepted by the normal git does not\nhave a notion of \"marks\", but modulo that, the first one looks to me very\nsimilar to \"rev-parse <ref>:<path>\" in spirit, which suggests that \"the\npath in the current one\" might be better spelled as \"rev-parse :<path>\" to\nmake the syntax and the concept more consistent across parts of the\nsystem.  If it makes sense to allow arbitary <committish> (or <treeish>\nfor that matter) for the <dataref> part of the parameter, this observation\nbecomes even more true, no?\n\nHaving said that, I do not deeply care about the token \"ls\" itself.  I\njust reacted to \"<dataref> SP <path>\" part.\n\n> Dirty hack: for now, git fast-import will treat missing paths as empty\n> subtrees and print them as\n>\n>  040000 tree 4b825dc642cb6eb9a060e54bf8d69288fbee4904\tpath/to/nowhere\n>\n> to avoid confusing frontends that inserted such a path before.\n\nSorry, but I am not quite sure what this paragraph is trying to say.\n"},{"id":"160924","messageId":"20110212012153.GA24736@elie","threadId":"26474","inReplyTo":"7vlj1mytjv.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH v3] fast-import: add 'ls' command","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2011-02-12T01:21:53Z","receivedAt":"2011-02-12T01:21:53Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"(+cc: vcs-fast-import-devs)\n\nJunio C Hamano wrote:\n\n>> There are two forms of the 'ls' command: the two-argument form prints\n>> the entry at <path> for the tree underlying the tree, commit, or tag\n>> named by <dataref>:\n>>\n>>       'ls' SP <dataref> SP <path> LF\n[...]\n> Is this really \"ls\"?\n>\n> Obviously, an extended SHA-1 that is accepted by the normal git does not\n> have a notion of \"marks\", but modulo that, the first one looks to me very\n> similar to \"rev-parse <ref>:<path>\" in spirit, which suggests that \"the\n> path in the current one\" might be better spelled as \"rev-parse :<path>\" to\n> make the syntax and the concept more consistent across parts of the\n> system.  If it makes sense to allow arbitary <committish> (or <treeish>\n> for that matter) for the <dataref> part of the parameter, this observation\n> becomes even more true, no?\n\nI suppose so.  It is inspired by \"git ls-tree <ref> -- <path>\", and\nit is convenient that it prints the mode, too (otherwise how do we\ntell a regular file from a symlink?).\n\n>> Dirty hack: for now, git fast-import will treat missing paths as empty\n>> subtrees and print them as\n>>\n>>  040000 tree 4b825dc642cb6eb9a060e54bf8d69288fbee4904\tpath/to/nowhere\n>>\n>> to avoid confusing frontends that inserted such a path before.\n>\n> Sorry, but I am not quite sure what this paragraph is trying to say.\n\nWhere $empty is the empty tree, I might expect:\n\n\tM 040000 $emptytree some/subdirectory\n\tls \"some/subdirectory\"\n\nto result in\n\n\t040000 tree $empty\tsome/subdirectory\n\nOn the other hand, I also would expect\n\n\tD some/subdirectory\n\tls \"some/subdirectory\"\n\nto result in\n\n\tmissing some/subdirectory\n\nSince git fast-import (like git itself) does not track empty\nsubdirectories, the state being reported in both these cases is the\nsame.  How can it tell the difference between the two?  So the patch\narbitrarily uses the former behavior (reporting nonexistent dirents as\nthough they were the empty subtree), which seems to work well in\npractice.\n\nIt is probably be less confusing to use \"missing\" and let frontends\nhandle that rather shielding the frontend from reality.  Like so:\n\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\n Documentation/git-fast-import.txt |    7 +----\n fast-import.c                     |   32 ++++++++++++--------------\n t/t9300-fast-import.sh            |   44 ++++++++++--------------------------\n 3 files changed, 29 insertions(+), 54 deletions(-)\n\ndiff --git a/Documentation/git-fast-import.txt b/Documentation/git-fast-import.txt\nindex 495e01f..e1b7a0f 100644\n--- a/Documentation/git-fast-import.txt\n+++ b/Documentation/git-fast-import.txt\n@@ -967,16 +967,13 @@ Output uses the same format as `git ls-tree <tree> {litdd} <path>`:\n The <dataref> represents the blob, tree, or commit object at <path>\n and can be used in later 'cat-blob', 'filemodify', or 'ls' commands.\n \n-Git repositories do not distinguish between missing paths and empty\n-subtrees.  If a path is not found, 'git fast-import' will report it as\n-an empty tree.  Backends that do have a notion of empty trees may write\n+If there is no file or subtree at that path, 'git fast-import' will\n+instead report\n \n ====\n \tmissing SP <path> LF\n ====\n \n-for paths that do not correspond to a blob or subtree.\n-\n `feature`\n ~~~~~~~~~\n Require that fast-import supports the specified feature, or abort if\ndiff --git a/fast-import.c b/fast-import.c\nindex cef2b8c..6c37b84 100644\n--- a/fast-import.c\n+++ b/fast-import.c\n@@ -2931,12 +2931,20 @@ static void print_ls(int mode, const unsigned char *sha1, const char *path)\n \t\tS_ISDIR(mode) ? tree_type :\n \t\tblob_type;\n \n-\t/* mode SP type SP object_name TAB path LF */\n-\tstrbuf_reset(&line);\n-\tstrbuf_addf(&line, \"%06o %s %s\\t\",\n-\t\t\tmode, type, sha1_to_hex(sha1));\n-\tquote_c_style(path, &line, NULL, 0);\n-\tstrbuf_addch(&line, '\\n');\n+\tif (!mode) {\n+\t\t/* missing SP path LF */\n+\t\tstrbuf_reset(&line);\n+\t\tstrbuf_addstr(&line, \"missing \");\n+\t\tquote_c_style(path, &line, NULL, 0);\n+\t\tstrbuf_addch(&line, '\\n');\n+\t} else {\n+\t\t/* mode SP type SP object_name TAB path LF */\n+\t\tstrbuf_reset(&line);\n+\t\tstrbuf_addf(&line, \"%06o %s %s\\t\",\n+\t\t\t\tmode, type, sha1_to_hex(sha1));\n+\t\tquote_c_style(path, &line, NULL, 0);\n+\t\tstrbuf_addch(&line, '\\n');\n+\t}\n \tcat_blob_write(line.buf, line.len);\n }\n \n@@ -2978,17 +2986,7 @@ static void parse_ls(struct branch *b)\n \tif (S_ISDIR(leaf.versions[1].mode))\n \t\tstore_tree(&leaf);\n \n-\tif (!leaf.versions[1].mode) {\n-\t\t/*\n-\t\t * Missing path?  Must be an empty subtree!\n-\t\t *\n-\t\t * When git learns to track empty directories, we can report\n-\t\t * this by saying 'missing \"path/to/directory\"' instead.\n-\t\t */\n-\t\tprint_ls(S_IFDIR, (const unsigned char *) EMPTY_TREE_SHA1_BIN, p);\n-\t} else {\n-\t\tprint_ls(leaf.versions[1].mode, leaf.versions[1].sha1, p);\n-\t}\n+\tprint_ls(leaf.versions[1].mode, leaf.versions[1].sha1, p);\n \tif (!b || root != &b->branch_tree)\n \t\trelease_tree_entry(root);\n }\ndiff --git a/t/t9300-fast-import.sh b/t/t9300-fast-import.sh\nindex f70d2a9..6b1ba6c 100755\n--- a/t/t9300-fast-import.sh\n+++ b/t/t9300-fast-import.sh\n@@ -934,25 +934,22 @@ test_expect_success PIPE 'N: read and copy directory' '\n \tcompare_diff_raw expect actual\n '\n \n-test_expect_success PIPE 'N: read and copy \"empty\" directory' '\n+test_expect_success PIPE 'N: empty directory reads as missing' '\n \tcat <<-\\EOF >expect &&\n \tOBJNAME\n-\t:000000 100644 OBJNAME OBJNAME A\tgreeting\n-\tOBJNAME\n-\t:100644 000000 OBJNAME OBJNAME D\tunrelated\n-\tOBJNAME\n \t:000000 100644 OBJNAME OBJNAME A\tunrelated\n \tEOF\n-\tgit update-ref -d refs/heads/copy-empty &&\n+\techo \"missing src\" >expect.response &&\n+\tgit update-ref -d refs/heads/read-empty &&\n \trm -f backflow &&\n \tmkfifo backflow &&\n \t(\n \t\texec <backflow &&\n \t\tcat <<-EOF &&\n-\t\tcommit refs/heads/copy-empty\n+\t\tcommit refs/heads/read-empty\n \t\tcommitter $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE\n \t\tdata <<COMMIT\n-\t\tcopy \"empty\" (missing) directory\n+\t\tread \"empty\" (missing) directory\n \t\tCOMMIT\n \n \t\tM 100644 inline src/greeting\n@@ -965,33 +962,16 @@ test_expect_success PIPE 'N: read and copy \"empty\" directory' '\n \t\tD src/greeting\n \t\tls \"src\"\n \t\tEOF\n-\t\tread mode type tree filename &&\n-\t\tsed -e \"s/X\\$//\" <<-EOF\n-\t\tM $mode $tree dst1\n-\t\tM $mode $tree dst2\n-\n-\t\tcommit refs/heads/copy-empty\n-\t\tcommitter $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE\n-\t\tdata <<COMMIT\n-\t\tcopy empty directory to root\n-\t\tCOMMIT\n-\n-\t\tM $mode $tree X\n-\n-\t\tcommit refs/heads/copy-empty\n-\t\tcommitter $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE\n-\t\tdata <<COMMIT\n-\t\tadd another file\n-\t\tCOMMIT\n-\n-\t\tM 100644 inline greeting\n-\t\tdata <<BLOB\n-\t\thello\n-\t\tBLOB\n+\t\tread -r line &&\n+\t\tprintf \"%s\\n\" \"$line\" >response &&\n+\t\tcat <<-\\EOF\n+\t\tD dst1\n+\t\tD dst2\n \t\tEOF\n \t) |\n \tgit fast-import --cat-blob-fd=3 3>backflow &&\n-\tgit rev-list copy-empty |\n+\ttest_cmp expect.response response &&\n+\tgit rev-list read-empty |\n \tgit diff-tree -r --root --stdin |\n \tsed \"s/$_x40/OBJNAME/g\" >actual &&\n \ttest_cmp expect actual\n-- \n1.7.4\n"},{"id":"160925","messageId":"20110212013643.GB24736@elie","threadId":"26474","inReplyTo":"20110212012153.GA24736@elie","subject":"Re: [PATCH v3] fast-import: add 'ls' command","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2011-02-12T01:36:43Z","receivedAt":"2011-02-12T01:36:43Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Jonathan Nieder wrote:\n\n> It is probably be less confusing to use \"missing\" and let frontends\n> handle that rather shielding the frontend from reality.  Like so:\n                    ^\n\nThis should say \"rather than\", as in: \"It is confusing to lie to the\nfrontend; let's not do that.\"\n\nSide note: this adds some complication to frontends.  For example,\nto copy some/path from the commit called :5, without this change,\none could do\n\n\techo 'ls :5 some/path'\n\tread -r mode type dataref path\n\techo \"M $mode $dataref $path\"\n\nbut now one would have to do:\n\n\techo 'ls :5 some/path'\n\tread -r mode rest\n\tcase $mode in\n\tmissing)\n\t\techo \"D some/path\"\n\t\t;;\n\t*)\n\t\tdataref=${rest#* }\n\t\techo \"M $mode $dataref some/path\"\n\t\t;;\n\tesac\n\nThe former is simpler; the latter is more explicit and probably less\nconfusing.\n\nSorry for the nonsense.\nJonathan\n"}]}