git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3 04/19] convert: add conv_attrs classification

From
Matheus Tavares <matheus.bernardino@usp.br>
Date
Oct 29, 2020, 02:14 UTC
Message-ID
<18c3f4247e717a7766f13b4b33a0bbe31aee6b69.1603937110.git.matheus.bernardino@usp.br>
In-Reply-To
<cover.1603937110.git.matheus.bernardino@usp.br>
From: Jeff Hostetler <jeffhost@microsoft.com>

Create `enum conv_attrs_classification` to express the different ways that attributes are handled for a blob during checkout.

This will be used in a later commit when deciding whether to add a file to the parallel or delayed queue during checkout. For now, we can also use it in get_stream_filter_ca() to simplify the function (as the classifying logic is the same).

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
[matheus.bernardino: use classification in get_stream_filter_ca()]
Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br>
---
 convert.c | 26 +++++++++++++++++++-------
 convert.h | 33 +++++++++++++++++++++++++++++++++
 2 files changed, 52 insertions(+), 7 deletions(-)
diff --git a/convert.c b/convert.c
index bd4d3f01cd..c0b45149b5 100644
--- a/convert.c
+++ b/convert.c
@@ -1972,13 +1972,7 @@ struct stream_filter *get_stream_filter_ca(const struct conv_attrs *ca,
 {
 	struct stream_filter *filter = NULL;
 
-	if (ca->drv && (ca->drv->process || ca->drv->smudge || ca->drv->clean))
-		return NULL;
-
-	if (ca->working_tree_encoding)
-		return NULL;
-
-	if (ca->crlf_action == CRLF_AUTO || ca->crlf_action == CRLF_AUTO_CRLF)
+	if (classify_conv_attrs(ca) != CA_CLASS_STREAMABLE)
 		return NULL;
 
 	if (ca->ident)
@@ -2034,3 +2028,21 @@ void clone_checkout_metadata(struct checkout_metadata *dst,
 	if (blob)
 		oidcpy(&dst->blob, blob);
 }
+
+enum conv_attrs_classification classify_conv_attrs(const struct conv_attrs *ca)
+{
+	if (ca->drv) {
+		if (ca->drv->process)
+			return CA_CLASS_INCORE_PROCESS;
+		if (ca->drv->smudge || ca->drv->clean)
+			return CA_CLASS_INCORE_FILTER;
+	}
+
+	if (ca->working_tree_encoding)
+		return CA_CLASS_INCORE;
+
+	if (ca->crlf_action == CRLF_AUTO || ca->crlf_action == CRLF_AUTO_CRLF)
+		return CA_CLASS_INCORE;
+
+	return CA_CLASS_STREAMABLE;
+}
diff --git a/convert.h b/convert.h
index 262c1a1d46..523ba9b140 100644
--- a/convert.h
+++ b/convert.h
@@ -190,4 +190,37 @@ int stream_filter(struct stream_filter *,
 		  const char *input, size_t *isize_p,
 		  char *output, size_t *osize_p);
 
+enum conv_attrs_classification {
+	/*
+	 * The blob must be loaded into a buffer before it can be
+	 * smudged. All smudging is done in-proc.
+	 */
+	CA_CLASS_INCORE,
+
+	/*
+	 * The blob must be loaded into a buffer, but uses a
+	 * single-file driver filter, such as rot13.
+	 */
+	CA_CLASS_INCORE_FILTER,
+
+	/*
+	 * The blob must be loaded into a buffer, but uses a
+	 * long-running driver process, such as LFS. This might or
+	 * might not use delayed operations. (The important thing is
+	 * that there is a single subordinate long-running process
+	 * handling all associated blobs and in case of delayed
+	 * operations, may hold per-blob state.)
+	 */
+	CA_CLASS_INCORE_PROCESS,
+
+	/*
+	 * The blob can be streamed and smudged without needing to
+	 * completely read it into a buffer.
+	 */
+	CA_CLASS_STREAMABLE,
+};
+
+enum conv_attrs_classification classify_conv_attrs(
+	const struct conv_attrs *ca);
+
 #endif /* CONVERT_H */
-- 
2.28.0
Previous: Junio C HamanoNext: Junio C Hamano
Message 24 of 93 in “Parallel Checkout (part I)”
  1. 00/19 Parallel Checkout (part I)Matheus Tavares, Oct 29, 2020
  2. 19/19 ci: run test round with parallel-checkout enabledMatheus Tavares, Oct 29, 2020
  3. 18/19 parallel-checkout: add tests related to .gitattributesMatheus Tavares, Oct 29, 2020
  4. 17/19 parallel-checkout: add tests related to clone collisionsMatheus Tavares, Oct 29, 2020
  5. 16/19 parallel-checkout: add tests for basic operationsMatheus Tavares, Oct 29, 2020
  6. 15/19 checkout-index: add parallel checkout supportMatheus Tavares, Oct 29, 2020
  7. 13/19 make_transient_cache_entry(): optionally alloc from mem_poolMatheus Tavares, Oct 29, 2020
  8. 14/19 builtin/checkout.c: complete parallel checkout supportMatheus Tavares, Oct 29, 2020
  9. 10/19 unpack-trees: add basic support for parallel checkoutMatheus Tavares, Oct 29, 2020
  10. Junio C HamanoNov 2, 2020
  11. Matheus Tavares BernardinoNov 3, 2020
  12. 09/19 entry: add checkout_entry_ca() which takes preloaded conv_attrsMatheus Tavares, Oct 29, 2020
  13. Junio C HamanoOct 30, 2020
  14. 11/19 parallel-checkout: make it truly parallelMatheus Tavares, Oct 29, 2020
  15. 12/19 parallel-checkout: support progress displayingMatheus Tavares, Oct 29, 2020
  16. 08/19 entry: move conv_attrs lookup up to checkout_entry()Matheus Tavares, Oct 29, 2020
  17. Junio C HamanoOct 30, 2020
  18. 05/19 entry: extract a header file for entry.c functionsMatheus Tavares, Oct 29, 2020
  19. Junio C HamanoOct 30, 2020
  20. 07/19 entry: extract cache_entry update from write_entry()Matheus Tavares, Oct 29, 2020
  21. 06/19 entry: make fstat_output() and read_blob_entry() publicMatheus Tavares, Oct 29, 2020
  22. 03/19 convert: add get_stream_filter_ca() variantMatheus Tavares, Oct 29, 2020
  23. Junio C HamanoOct 29, 2020
  24. 04/19 convert: add conv_attrs classificationMatheus Tavares, Oct 29, 2020
  25. Junio C HamanoOct 29, 2020
  26. 02/19 convert: add [async_]convert_to_working_tree_ca() variantsMatheus Tavares, Oct 29, 2020
  27. Junio C HamanoOct 29, 2020
  28. 01/19 convert: make convert_attrs() and convert structs publicMatheus Tavares, Oct 29, 2020
  29. Junio C HamanoOct 29, 2020
  30. Matheus Tavares BernardinoOct 30, 2020
  31. Junio C HamanoOct 30, 2020
  32. Junio C HamanoOct 29, 2020
  33. Jeff HostetlerOct 30, 2020
  34. 00/19 Parallel Checkout (part I)Matheus Tavares, Nov 4, 2020
  35. 01/19 convert: make convert_attrs() and convert structs publicMatheus Tavares, Nov 4, 2020
  36. Christian CouderDec 5, 2020
  37. Matheus Tavares BernardinoDec 5, 2020
  38. 02/19 convert: add [async_]convert_to_working_tree_ca() variantsMatheus Tavares, Nov 4, 2020
  39. Christian CouderDec 5, 2020
  40. Matheus Tavares BernardinoDec 5, 2020
  41. 03/19 convert: add get_stream_filter_ca() variantMatheus Tavares, Nov 4, 2020
  42. Christian CouderDec 5, 2020
  43. 04/19 convert: add conv_attrs classificationMatheus Tavares, Nov 4, 2020
  44. Christian CouderDec 5, 2020
  45. Matheus Tavares BernardinoDec 5, 2020
  46. 05/19 entry: extract a header file for entry.c functionsMatheus Tavares, Nov 4, 2020
  47. Christian CouderDec 6, 2020
  48. 06/19 entry: make fstat_output() and read_blob_entry() publicMatheus Tavares, Nov 4, 2020
  49. 07/19 entry: extract cache_entry update from write_entry()Matheus Tavares, Nov 4, 2020
  50. Christian CouderDec 6, 2020
  51. 08/19 entry: move conv_attrs lookup up to checkout_entry()Matheus Tavares, Nov 4, 2020
  52. Christian CouderDec 6, 2020
  53. Matheus Tavares BernardinoDec 7, 2020
  54. 09/19 entry: add checkout_entry_ca() which takes preloaded conv_attrsMatheus Tavares, Nov 4, 2020
  55. Christian CouderDec 6, 2020
  56. Matheus Tavares BernardinoDec 7, 2020
  57. 10/19 unpack-trees: add basic support for parallel checkoutMatheus Tavares, Nov 4, 2020
  58. Christian CouderDec 6, 2020
  59. Matheus Tavares BernardinoDec 7, 2020
  60. 11/19 parallel-checkout: make it truly parallelMatheus Tavares, Nov 4, 2020
  61. Emily ShafferDec 16, 2020
  62. Matheus Tavares BernardinoDec 17, 2020
  63. 12/19 parallel-checkout: support progress displayingMatheus Tavares, Nov 4, 2020
  64. 13/19 make_transient_cache_entry(): optionally alloc from mem_poolMatheus Tavares, Nov 4, 2020
  65. 15/19 checkout-index: add parallel checkout supportMatheus Tavares, Nov 4, 2020
  66. 17/19 parallel-checkout: add tests related to clone collisionsMatheus Tavares, Nov 4, 2020
  67. 18/19 parallel-checkout: add tests related to .gitattributesMatheus Tavares, Nov 4, 2020
  68. 19/19 ci: run test round with parallel-checkout enabledMatheus Tavares, Nov 4, 2020
  69. 16/19 parallel-checkout: add tests for basic operationsMatheus Tavares, Nov 4, 2020
  70. 14/19 builtin/checkout.c: complete parallel checkout supportMatheus Tavares, Nov 4, 2020
  71. 0/9 Parallel Checkout (part I)Matheus Tavares, Dec 16, 2020
  72. 1/9 convert: make convert_attrs() and convert structs publicMatheus Tavares, Dec 16, 2020
  73. 2/9 convert: add [async_]convert_to_working_tree_ca() variantsMatheus Tavares, Dec 16, 2020
  74. 3/9 convert: add get_stream_filter_ca() variantMatheus Tavares, Dec 16, 2020
  75. 5/9 entry: extract a header file for entry.c functionsMatheus Tavares, Dec 16, 2020
  76. 6/9 entry: make fstat_output() and read_blob_entry() publicMatheus Tavares, Dec 16, 2020
  77. 4/9 convert: add classification for conv_attrs structMatheus Tavares, Dec 16, 2020
  78. 7/9 entry: extract update_ce_after_write() from write_entry()Matheus Tavares, Dec 16, 2020
  79. 9/9 entry: add checkout_entry_ca() taking preloaded conv_attrsMatheus Tavares, Dec 16, 2020
  80. 8/9 entry: move conv_attrs lookup up to checkout_entry()Matheus Tavares, Dec 16, 2020
  81. Christian CouderDec 16, 2020
  82. Junio C HamanoDec 17, 2020
  83. 0/9 Parallel Checkout (part 1)Matheus Tavares, Mar 23, 2021
  84. 1/9 convert: make convert_attrs() and convert structs publicMatheus Tavares, Mar 23, 2021
  85. 2/9 convert: add [async_]convert_to_working_tree_ca() variantsMatheus Tavares, Mar 23, 2021
  86. 4/9 convert: add classification for conv_attrs structMatheus Tavares, Mar 23, 2021
  87. 3/9 convert: add get_stream_filter_ca() variantMatheus Tavares, Mar 23, 2021
  88. 6/9 entry: make fstat_output() and read_blob_entry() publicMatheus Tavares, Mar 23, 2021
  89. 8/9 entry: move conv_attrs lookup up to checkout_entry()Matheus Tavares, Mar 23, 2021
  90. 5/9 entry: extract a header file for entry.c functionsMatheus Tavares, Mar 23, 2021
  91. 9/9 entry: add checkout_entry_ca() taking preloaded conv_attrsMatheus Tavares, Mar 23, 2021
  92. 7/9 entry: extract update_ce_after_write() from write_entry()Matheus Tavares, Mar 23, 2021
  93. Junio C HamanoMar 23, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.