git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH] Add keyword collapse support to convert.c

From
APAndy Parkins <andyparkins@gmail.com>
Date
Apr 17, 2007, 20:52 UTC
Message-ID
<200704172152.58870.andyparkins@gmail.com>
In-Reply-To
<200704172146.33665.andyparkins@gmail.com>

This patch adds expansion of keywords support. The collapse is only performed when the "keywords" attribute it found for a file. The check for this attribute is done in the same way as the "crlf" attribute check.

The actual collapse is performed by keyword_collapse_git() which is called from convert_to_git() when the "keywords" attribute is found.

keyword_collapse_git() finds strings of the form
 $KEYWORD: ARBITRARY STRING$
And collapses them into
 $KEYWORD:$

No parsing of the keyword itself is performed, the content is simply dropped.

Despite the fact that this doesn't do anything useful from the users perspective, this patch forms the more important half of keyword expansion support - because it prevents the expansion from entering the repository. It effectively creates blind spots that git tools won't see.

convert_to_git() has also been changed so that it no longer only does CRLF conversion. Instead, a flag is kept to say whether any conversion was done by the CRLF code, and then that converted buffer is passed to keyword_collapse_git() and the flag again updated. It then returns 1 if either of these conversion functions actually changed anything.

I've also included a test script to show that the keyword collapse is working. It particular demonstrates that the diff between a file with keywords and the repository is blind to the expanded keyword.

git-apply is patched to perform the collapse as well on each fragment.
Signed-off-by: Andy Parkins <andyparkins@gmail.com>
---
This is on top of 1ddfc1ad616550764056077b9e12a35533298c89.

I'm positing it for posterity. It's not going anywhere though, so I'm not submitting it for inclusion.

 builtin-apply.c     |    2 +
 convert.c           |  123 +++++++++++++++++++++++++++++++++++++++++++++++++-
 t/t0030-keywords.sh |   95 +++++++++++++++++++++++++++++++++++++++
 3 files changed, 217 insertions(+), 3 deletions(-)
 create mode 100755 t/t0030-keywords.sh
diff --git a/builtin-apply.c b/builtin-apply.c
index fd92ef7..212c7d4 100644
--- a/builtin-apply.c
+++ b/builtin-apply.c
@@ -1056,6 +1056,8 @@ static int parse_single_patch(char *line, unsigned long size, struct patch *patc
 	unsigned long oldlines = 0, newlines = 0, context = 0;
 	struct fragment **fragp = &patch->fragments;
 
+	convert_to_git( patch->new_name, &line, &size );
+
 	while (size > 4 && !memcmp(line, "@@ -", 4)) {
 		struct fragment *fragment;
 		int len;
diff --git a/convert.c b/convert.c
index d0d4b81..a18e7ea 100644
--- a/convert.c
+++ b/convert.c
@@ -230,16 +230,133 @@ static int git_path_check_crlf(const char *path)
 	return attr_crlf_check.isset;
 }
 
+/* ------------------ keywords -------------------- */
+
+static void setup_keyword_check(struct git_attr_check *check)
+{
+	static struct git_attr *attr_keyword;
+
+	if (!attr_keyword)
+		attr_keyword = git_attr("keywords", 8);
+	check->attr = attr_keyword;
+}
+
+static int git_path_check_keyword(const char *path)
+{
+	struct git_attr_check attr_keyword_check;
+
+	setup_keyword_check(&attr_keyword_check);
+
+	if (git_checkattr(path, 1, &attr_keyword_check))
+		return -1;
+	return attr_keyword_check.isset;
+}
+
+static int keyword_collapse_git(const char *path, char **bufp, unsigned long *sizep)
+{
+	char *buffer, *nbuf, *keyword;
+	unsigned long size, keywordlength;
+	int changes = 0;
+	enum {
+		IN_VOID,
+		PRE_KEYWORD,
+		IN_KEYWORD,
+		IN_EXPANSION,
+		END_KEYWORD
+	} parser_state = IN_VOID;
+
+	size = *sizep;
+	if (!size)
+		return 0;
+	buffer = *bufp;
+
+	/*
+	 * Allocate an identically sized buffer, keyword collapse can
+	 * only reduce the size so we'll never overflow (although we might
+	 * waste a few bytes
+	 */
+	nbuf = xmalloc(size);
+	*bufp = nbuf;
+
+	while (size) {
+		unsigned char c;
+
+		c = *buffer;
+
+		switch( parser_state ) {
+		case IN_VOID:        /* Normal characters, wait for '$' */
+			if (c == '$')
+				parser_state = PRE_KEYWORD;
+			break;
+		case PRE_KEYWORD:    /* Gap between '$' and keyword */
+			keywordlength = 0;
+			keyword = buffer;
+			if (!isspace(c))
+				parser_state = IN_KEYWORD;
+			else
+				break;
+		case IN_KEYWORD:     /* Keyword itself */
+			if (c == ':')
+				parser_state = IN_EXPANSION;
+			else if (c == '$' || c == '\n' || c == '\r' || c == '\0' )
+				parser_state = END_KEYWORD;
+			else
+				keywordlength++;
+			break;
+		case IN_EXPANSION:   /* The expansion gets silently removed */
+			if (c == '$' || c == '\n' || c == '\r' || c == '\0' )
+				parser_state = END_KEYWORD;
+			else {
+				changes = 1;
+				/* Every character we skip reduces the overall size */
+				(*sizep)--;
+				buffer++;
+				size--;
+			}
+			continue;
+		case END_KEYWORD:    /* End of keyword */
+			parser_state = IN_VOID;
+			break;
+		}
+
+		*nbuf++ = c;
+		buffer++;
+		size--;
+	}
+
+	return (changes != 0);
+}
+
+
+/* ------------------------------------------------ */
 int convert_to_git(const char *path, char **bufp, unsigned long *sizep)
 {
+	int changes = 0;
+
 	switch (git_path_check_crlf(path)) {
 	case 0:
-		return 0;
+		changes += 0;
+		break;
+	case 1:
+		changes += forcecrlf_to_git(path, bufp, sizep);
+		break;
+	default:
+		changes += autocrlf_to_git(path, bufp, sizep);
+		break;
+	}
+
+	switch (git_path_check_keyword(path)) {
+	case 0:
+		changes += 0;
+		break;
 	case 1:
-		return forcecrlf_to_git(path, bufp, sizep);
+		changes += keyword_collapse_git(path, bufp, sizep);
+		break;
 	default:
-		return autocrlf_to_git(path, bufp, sizep);
+		changes += 0;
+		break;
 	}
+	return (changes != 0);
 }
 
 int convert_to_working_tree(const char *path, char **bufp, unsigned long *sizep)
diff --git a/t/t0030-keywords.sh b/t/t0030-keywords.sh
new file mode 100755
index 0000000..5180b0e
--- /dev/null
+++ b/t/t0030-keywords.sh
@@ -0,0 +1,95 @@
+#!/bin/sh
+
+cd $(dirname $0)
+
+test_description='Keyword expansion'
+
+. ./test-lib.sh
+
+# Adding the attribute "keywords" turns the keyword expansion on
+# I've used "notkeywords" as an attribute as a placeholder attribute
+# but this is just "somerandomattribute", it has no meaning
+
+# Expect success because the keyword attribute should be found
+test_expect_success 'Keywords attribute present' '
+
+	echo "keywordsfile keywords" >.gitattributes &&
+
+	echo "\$keyword: anythingcangohere\$" > keywordsfile &&
+
+	git add keywordsfile &&
+	git add .gitattributes &&
+	git commit -m test-keywords &&
+
+	git check-attr keywords -- keywordsfile
+'
+
+# Expect failure because the repository version should be different from the
+# working tree version.
+#
+#  In repository : $keyword:$
+#  In working dir: $keyword: anythingcangohere$
+#
+test_expect_failure 'Keywords collapse active' '
+
+	git show HEAD:keywordsfile > keywordsfile.cmp &&
+	cmp keywordsfile keywordsfile.cmp
+
+'
+
+# expect success because we want to find the keyword line collapsed in the
+# and hence appearing unchanged in the output of git-diff
+test_expect_success 'git-diff with keywords present' '
+	echo "Non-keyword containing line" >> keywordsfile &&
+	git diff -- keywordsfile | grep -qs "^ \$keyword:\$$"
+'
+
+# Check git-apply blindness
+cat > keyword-patch.diff << EOF
+diff --git a/keywordsfile b/keywordsfile
+--- a/keywordsfile
++++ b/keywordsfile
+@@ -1,2 +1,2 @@
+ \$keyword:\$
+-Non-keyword containing line
++Another non-keyword containing line
+EOF
+
+test_expect_success 'patch application with keywords active' '
+	git-apply --check keyword-patch.diff
+'
+
+# Expect failure because the keywords attribute should NOT be found
+test_expect_failure 'Keywords attribute absent' '
+
+	echo "keywordsfile notkeywords" >.gitattributes &&
+
+	git add .gitattributes &&
+	git commit -m test-not-keywords &&
+
+	git check-attr keywords -- keywordsfile
+
+'
+
+# If keywords are later disabled on that file, then the keyword collapsed
+# will be ignored, so a diff should now show differences, because git is no
+# longer keyword blind
+test_expect_success 'git-diff with keywords in file but disabled' '
+	git diff -- keywordsfile | grep -qs "^diff"
+'
+
+# Expect success because the repository should be identical to the working tree
+test_expect_success 'Keywords collapse inactive' '
+
+	git add keywordsfile &&
+	git commit -m "test-not-keywords"
+
+	git show HEAD:keywordsfile > keywordsfile.cmp &&
+	cmp keywordsfile keywordsfile.cmp
+'
+
+test_expect_failure 'patch application without keywords active' '
+	git-apply --check keyword-patch.diff
+'
+
+test_done
-- 
1.5.1.1.822.g0049
Previous: Andy ParkinsNext: Linus Torvalds
Message 35 of 66 in “Add keyword unexpansion support to convert.c”
  1. 2/2 Add keyword unexpansion support to convert.cAndy Parkins, Apr 17, 2007
  2. Junio C HamanoApr 17, 2007
  3. Andy ParkinsApr 17, 2007
  4. Linus TorvaldsApr 17, 2007
  5. Andy ParkinsApr 17, 2007
  6. Linus TorvaldsApr 17, 2007
  7. Andy ParkinsApr 17, 2007
  8. Nicolas PitreApr 17, 2007
  9. David LangApr 17, 2007
  10. Nicolas PitreApr 17, 2007
  11. David LangApr 17, 2007
  12. Nicolas PitreApr 17, 2007
  13. David LangApr 17, 2007
  14. Andy ParkinsApr 17, 2007
  15. Junio C HamanoApr 17, 2007
  16. Nicolas PitreApr 18, 2007
  17. Junio C HamanoApr 18, 2007
  18. Nicolas PitreApr 18, 2007
  19. Johannes SchindelinApr 18, 2007
  20. Nicolas PitreApr 18, 2007
  21. Johannes SchindelinApr 19, 2007
  22. David LangApr 21, 2007
  23. Junio C HamanoApr 21, 2007
  24. Nicolas PitreApr 21, 2007
  25. David LangApr 21, 2007
  26. Rogan DawesApr 18, 2007
  27. Linus TorvaldsApr 18, 2007
  28. Nicolas PitreApr 18, 2007
  29. Rogan DawesApr 18, 2007
  30. Nicolas PitreApr 18, 2007
  31. Rogan DawesApr 18, 2007
  32. Alon ZivApr 18, 2007
  33. Linus TorvaldsApr 17, 2007
  34. Andy ParkinsApr 17, 2007
  35. Add keyword collapse support to convert.cAndy Parkins, Apr 17, 2007
  36. Linus TorvaldsApr 17, 2007
  37. Linus TorvaldsApr 17, 2007
  38. Johannes SchindelinApr 18, 2007
  39. Nikolai WeibullApr 20, 2007
  40. Martin LanghoffApr 17, 2007
  41. Junio C HamanoApr 17, 2007
  42. Jakub NarebskiApr 20, 2007
  43. David LangApr 21, 2007
  44. Linus TorvaldsApr 17, 2007
  45. Johannes SixtApr 17, 2007
  46. Linus TorvaldsApr 17, 2007
  47. Andy ParkinsApr 17, 2007
  48. Rogan DawesApr 17, 2007
  49. Linus TorvaldsApr 17, 2007
  50. Rogan DawesApr 17, 2007
  51. Robin H. JohnsonApr 17, 2007
  52. Junio C HamanoApr 18, 2007
  53. J. Bruce FieldsApr 18, 2007
  54. Linus TorvaldsApr 18, 2007
  55. Junio C HamanoApr 18, 2007
  56. Linus TorvaldsApr 18, 2007
  57. Robin H. JohnsonApr 18, 2007
  58. Junio C HamanoApr 18, 2007
  59. Junio C HamanoApr 18, 2007
  60. Robin H. JohnsonApr 18, 2007
  61. Daniel BarkalowApr 18, 2007
  62. Johannes SchindelinApr 18, 2007
  63. Martin LanghoffApr 18, 2007
  64. David KågedalApr 18, 2007
  65. Robin H. JohnsonApr 18, 2007
  66. Matthieu MoyApr 17, 2007

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.