git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v3 0/9] Incremental po/git.pot update and new l10n workflow

From
Jiang Xin <worldhello.net@gmail.com>
Date
May 23, 2022, 13:42 UTC
Message-ID
<CANYiYbF8NDawCijZ1Hf=2zdi329=WSdYqL05Tz8+NOyJAFgpLA@mail.gmail.com>
In-Reply-To
<220523.86ilpwzmne.gmgdl@evledraar.gmail.com>

On Mon, May 23, 2022 at 4:19 PM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:

Show 71 quoted lines
>
>
> On Mon, May 23 2022, Ævar Arnfjörð Bjarmason wrote:
>
> > On Mon, May 23 2022, Jiang Xin wrote:
> >
> >> From: Jiang Xin <zhiyou.jx@alibaba-inc.com>
> >>
> >> A workflow change for translators are being proposed.
> >>
> >> Changes since v2:
> >>
> >>  1. Patch 1/9: reword.
> >>  2. Patch 2/9: reword.
> >>  3. Patch 3/9: reword, and add "FORCE" to prerequisites of "po/git.pot".
> >>  4. Patch 6/9: remove "FORCE" from prerequisites of "po/git.pot".
> >>  5. Patch 8/9: reword, and reuse "$(gen_pot_header)" to prepare pot
> >>                header for "po/git-core.pot".
> >>  6. Patch 9/9: various updates on po/README.md.
> >
> > From skimming this the *.c.po v.s. *.c extension is still left in
> > comments. I'm not saying you need to go for my suggestions, but it would
> > be very useful in CL's to note things that were suggested but not
> > changed, such as that.
> >
> > Right now I haven't paged that v2 discussion into my brain again, so I
> > don't know if that was the only thing, it's the only thing I remember
> > right now...
>
> This fix-up below implements what I suggested on v2, so now the comments
> in the generated file are correct, and don't refer to our intermediate
> files:
>
>         $ grep '#-#' po/git.pot
>         #. #-#-#-#-#  git-add--interactive.perl  #-#-#-#-#
>         #. #-#-#-#-#  add-patch.c  #-#-#-#-#
>         #. #-#-#-#-#  git-add--interactive.perl  #-#-#-#-#
>         #. #-#-#-#-#  branch.c  #-#-#-#-#
>         #. #-#-#-#-#  object-name.c  #-#-#-#-#
>         #. #-#-#-#-#  grep.c  #-#-#-#-#
>
> I gathered that the reason you preferred the whole "grep -q PRItime" was
> because you wanted to mitigate the effects of your IDE discovering these
> files.
>
> With the below you can define AGGRESSIVE_INTERMEDIATE and when you "make
> pot" the generated *.c files will only exist for as long as they're
> needed to generate the next step.
>
> But if you do a subsequent "make pot" will be slower, as we'll of course
> need to generate them again.
>
> I think it's better to go in this direction, and rename that
> AGGRESSIVE_INTERMEDIATE to something like
> MAKE_AVOID_REAL_EXTENSIONS_IN_GITIGNORED_FILES or whatever.
>
> I.e. our correctness shouldn't suffer because we're trying to work
> around some issue in a specific (and optional) developer tooling.
>
> There's also the fix there for the "header" dependency, but as noted in
> another reply it should probably be dropped altogether...
>
> diff --git a/Makefile b/Makefile
> index d3eae150de9..0b96b55b63f 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -2736,6 +2736,7 @@ endif
>  ## "po/git.pot" file.
>  LOCALIZED_ALL_GEN_PO =
>
> +LOCALIZED_C_GEN_C = $(LOCALIZED_C:%=.build/pot/po-munged/%)

Intermediate C source files copied from the original location, and PRItime will be replaced by PRIuMAX, if any.

Show 13 quoted lines
>  LOCALIZED_C_GEN_PO = $(LOCALIZED_C:%=.build/pot/po/%.po)
>  LOCALIZED_ALL_GEN_PO += $(LOCALIZED_C_GEN_PO)
>
> @@ -2745,26 +2746,19 @@ LOCALIZED_ALL_GEN_PO += $(LOCALIZED_SH_GEN_PO)
>  LOCALIZED_PERL_GEN_PO = $(LOCALIZED_PERL:%=.build/pot/po/%.po)
>  LOCALIZED_ALL_GEN_PO += $(LOCALIZED_PERL_GEN_PO)
>
> -## Gettext tools cannot work with our own custom PRItime type, so
> -## we replace PRItime with PRIuMAX.  We need to update this to
> -## PRIdMAX if we switch to a signed type later.
> -$(LOCALIZED_C_GEN_PO): .build/pot/po/%.po: %
> +ifdef AGGRESSIVE_INTERMEDIATE
> +.INTERMEDIATE: $(LOCALIZED_C_GEN_C)
Intermediate files "$(LOCALIZED_C_GEN_C)" will be removed automatically.
Show 19 quoted lines
> +endif
> +$(LOCALIZED_C_GEN_C): .build/pot/po-munged/%: %
>         $(call mkdir_p_parent_template)
> -       $(QUIET_XGETTEXT) \
> -           if grep -q PRItime $<; then \
> -               (\
> -                       sed -e 's|PRItime|PRIuMAX|g' <$< \
> -                               >.build/pot/po/$< && \
> -                       cd .build/pot/po && \
> -                       $(XGETTEXT) --omit-header \
> -                               -o $(@:.build/pot/po/%=%) \
> -                               $(XGETTEXT_FLAGS_C) $< && \
> -                       rm $<; \
> -               ); \
> -           else \
> -               $(XGETTEXT) --omit-header \
> -                       -o $@ $(XGETTEXT_FLAGS_C) $<; \
> -           fi
> +       $(QUIET_GEN)sed -e 's|PRItime|PRIuMAX|g' <$< >$@

Copy each source files in $(LOCALIZED_C) to corresponding intermediate file ($(LOCALIZED_C_GEN_C)) in ".build/pot/po-munged/", replacing PRItime with PRIuMAX if any.

Show 7 quoted lines
> +
> +$(LOCALIZED_C_GEN_PO): .build/pot/po/%.po: .build/pot/po-munged/%
> +       $(call mkdir_p_parent_template)
> +       $(QUIET_XGETTEXT)( \
> +               cd $(<D) && \
> +               $(XGETTEXT) $(XGETTEXT_FLAGS_C) --omit-header -o - $(<F) \
> +       ) >$@

For each intermediate C source files in ".build/pot/po-munged/", call xgettext to create corresponding po file in ".build/pot/po/" directory.

Show 8 quoted lines
>  $(LOCALIZED_SH_GEN_PO): .build/pot/po/%.po: %
>         $(call mkdir_p_parent_template)
> @@ -2786,11 +2780,24 @@ sed -e 's|charset=CHARSET|charset=UTF-8|' \
>  echo '"Plural-Forms: nplurals=INTEGER; plural=EXPRESSION;\\n"' >>$@
>  endef
>
> -.build/pot/git.header: $(LOCALIZED_ALL_GEN_PO)
> +.build/pot/git.header:

No. We should rebuild the pot header if any po file need to be update, because we want to refresh the timestamp in the "POT-Creation-Date:" filed of the pot header.

Show 16 quoted lines
>         $(call mkdir_p_parent_template)
>         $(QUIET_GEN)$(gen_pot_header)
>
> -po/git.pot: .build/pot/git.header $(LOCALIZED_ALL_GEN_PO)
> +# We go through this dance of having a prepared
> +# e.g. .build/pot/po/grep.c.po and copying it to
> +# .build/pot/to-cat/grep.c only because some IDEs (e.g. VSCode) pick
> +# up on the "real" extension for the purposes of auto-completion, even
> +# if the .build directiory is in .gitignore.
> +LOCALIZED_ALL_GEN_TO_CAT = $(LOCALIZED_ALL_GEN_PO:.build/pot/po/%.po=.build/pot/to-cat/%)
> +ifdef AGGRESSIVE_INTERMEDIATE
> +.INTERMEDIATE: $(LOCALIZED_ALL_GEN_TO_CAT)
> +endif
> +$(LOCALIZED_ALL_GEN_TO_CAT): .build/pot/to-cat/%: .build/pot/po/%.po
> +       $(call mkdir_p_parent_template)
> +       $(QUIET_GEN)cat $< >$@

Copy each po file in ".build/pot/po/" to another location ".build/pot/to-cat/", but without the ".po" extension.

Let's take "date.c" as an example:
1. Copy "date.c" to an intermediate C source file
".build/pot/po-munged/date.c" and replace PRItime with PRIuMAX in it.
2. Call xgettext to create  ".build/pot/po/date.c.po" from the
intermediate C source file ".build/pot/po-munged/date.c".
3. Optionally remove intermediate C source files like
".build/pot/po-munged/date.c". To have two identical C source files in
the same worktree is not good, some software may break. So I choose to
remove them.
4. Copy the po file (".build/pot/po/date.c.po") created in step 2 to
an intermediate fake C source file ".build/pot/to-cat/date.c" which is
a file without the ".po" extension. Please note this intermediate fake
C source file ".build/pot/to-cat/date.c" is not a valid C file, but a
PO file.
5. Call msgcat to create "po/git.pot" from all the intermediate fake C
source files including  ".build/pot/to-cat/date.c".
6. Optionally remove all the intermediate fake C source files in
".build/pot/to-cat/". I choose to remove them, because leave lots of
invalid C source files in worktree is not good.
For example, ".build/pot/po/date.c.po" was created from
> +
> +po/git.pot: .build/pot/git.header $(LOCALIZED_ALL_GEN_TO_CAT)
>         $(QUIET_GEN)$(MSGCAT) $(MSGCAT_FLAGS) $^ >$@
7. "po/git.pot" depends on the intermediate fake C source files. If
any single C source file has been changed, will run step 6 to copy all
po files in ".build/pot/po" to corresponding fake C source files in
".build/pot/to-cat/", if we choose to remove these intermediate fake C
source files.

This implementation is too heavy to solve a trivial issue. I think we can push forward this patch series and leave these comments in "po/git.pot":

Show 7 quoted lines
>         $ grep '#-#' po/git.pot
>         #. #-#-#-#-#  git-add--interactive.perl.po  #-#-#-#-#
>         #. #-#-#-#-#  add-patch.c.po  #-#-#-#-#
>         #. #-#-#-#-#  git-add--interactive.perl.po  #-#-#-#-#
>         #. #-#-#-#-#  branch.c.po  #-#-#-#-#
>         #. #-#-#-#-#  object-name.c.po  #-#-#-#-#
>         #. #-#-#-#-#  grep.c.po  #-#-#-#-#
Previous: Ævar Arnfjörð BjarmasonNext: Ævar Arnfjörð Bjarmason
Message 31 of 110 in “Incremental po/git.pot update and new l10n workflow”
  1. 0/9 Incremental po/git.pot update and new l10n workflowJiang Xin, May 3, 2022
  2. 1/9 Makefile: sort "po/git.pot" by file locationJiang Xin, May 3, 2022
  3. 2/9 Makefile: generate "po/git.pot" from stable LOCALIZED_CJiang Xin, May 3, 2022
  4. 5/9 po/git.pot: don't check in result of "make pot"Jiang Xin, May 3, 2022
  5. 3/9 Makefile: have "make pot" not "reset --hard"Jiang Xin, May 3, 2022
  6. 7/9 Makefile: add "po-update" rule to update po/XX.poJiang Xin, May 3, 2022
  7. 9/9 l10n: Document the new l10n workflowJiang Xin, May 3, 2022
  8. 8/9 Makefile: add "po-init" rule to initialize po/XX.poJiang Xin, May 3, 2022
  9. 4/9 i18n CI: stop allowing non-ASCII source messages in po/git.potJiang Xin, May 3, 2022
  10. 6/9 po/git.pot: remove this now generated file, see preceding commitJiang Xin, May 3, 2022
  11. Peter KreftingMay 3, 2022
  12. Jiang XinMay 4, 2022
  13. Junio C HamanoMay 4, 2022
  14. Daniel SantosMay 4, 2022
  15. Jiang XinMay 5, 2022
  16. Daniel SantosMay 5, 2022
  17. Junio C HamanoMay 5, 2022
  18. Jiang XinMay 6, 2022
  19. Jiang XinMay 5, 2022
  20. Junio C HamanoMay 4, 2022
  21. 1/9 Makefile: sort "po/git.pot" by file locationJiang Xin, May 19, 2022
  22. Ævar Arnfjörð BjarmasonMay 19, 2022
  23. Jiang XinMay 19, 2022
  24. 0/9 Incremental po/git.pot update and new l10n workflowJiang Xin, May 19, 2022
  25. Ævar Arnfjörð BjarmasonMay 19, 2022
  26. Jiang XinMay 19, 2022
  27. Ævar Arnfjörð BjarmasonMay 19, 2022
  28. 0/9 Incremental po/git.pot update and new l10n workflowJiang Xin, May 23, 2022
  29. Ævar Arnfjörð BjarmasonMay 23, 2022
  30. Ævar Arnfjörð BjarmasonMay 23, 2022
  31. Jiang XinMay 23, 2022
  32. Ævar Arnfjörð BjarmasonMay 23, 2022
  33. Jiang XinMay 23, 2022
  34. Jiang XinMay 23, 2022
  35. 0/9 Incremental po/git.pot update and new l10n workflowJiang Xin, May 23, 2022
  36. Junio C HamanoMay 23, 2022
  37. 01/10 Makefile: sort source files before feeding to xgettextJiang Xin, May 26, 2022
  38. 00/10 Incremental po/git.pot update and new l10n workflowJiang Xin, May 26, 2022
  39. 02/10 Makefile: generate "po/git.pot" from stable LOCALIZED_CJiang Xin, May 26, 2022
  40. 10/10 l10n: Document the new l10n workflowJiang Xin, May 26, 2022
  41. 06/10 po/git.pot: this is now a generated fileJiang Xin, May 26, 2022
  42. Junio C HamanoMay 26, 2022
  43. 04/10 i18n CI: stop allowing non-ASCII source messages in po/git.potJiang Xin, May 26, 2022
  44. 05/10 Makefile: remove duplicate and unwanted files in FOUND_SOURCE_FILESJiang Xin, May 26, 2022
  45. 08/10 Makefile: add "po-update" rule to update po/XX.poJiang Xin, May 26, 2022
  46. 07/10 po/git.pot: don't check in result of "make pot"Jiang Xin, May 26, 2022
  47. 09/10 Makefile: add "po-init" rule to initialize po/XX.poJiang Xin, May 26, 2022
  48. 03/10 Makefile: have "make pot" not "reset --hard"Jiang Xin, May 26, 2022
  49. 2/9 Makefile: generate "po/git.pot" from stable LOCALIZED_CJiang Xin, May 23, 2022
  50. 7/9 Makefile: add "po-update" rule to update po/XX.poJiang Xin, May 23, 2022
  51. 1/9 Makefile: sort "po/git.pot" by file locationJiang Xin, May 23, 2022
  52. 4/9 i18n CI: stop allowing non-ASCII source messages in po/git.potJiang Xin, May 23, 2022
  53. 3/9 Makefile: have "make pot" not "reset --hard"Jiang Xin, May 23, 2022
  54. Junio C HamanoMay 25, 2022
  55. Junio C HamanoMay 25, 2022
  56. Jiang XinMay 26, 2022
  57. Makefile: dedup git-ls-files output to prevent duplicate targetsJiang Xin, May 26, 2022
  58. Junio C HamanoMay 26, 2022
  59. Jiang XinMay 26, 2022
  60. Junio C HamanoMay 26, 2022
  61. Jiang XinMay 26, 2022
  62. Ævar Arnfjörð BjarmasonMay 26, 2022
  63. Jiang XinMay 26, 2022
  64. Junio C HamanoMay 26, 2022
  65. Ævar Arnfjörð BjarmasonMay 26, 2022
  66. Junio C HamanoMay 26, 2022
  67. Ævar Arnfjörð BjarmasonMay 26, 2022
  68. 6/9 po/git.pot: don't check in result of "make pot"Jiang Xin, May 23, 2022
  69. 5/9 po/git.pot: this is now a generated fileJiang Xin, May 23, 2022
  70. 9/9 l10n: Document the new l10n workflowJiang Xin, May 23, 2022
  71. 8/9 Makefile: add "po-init" rule to initialize po/XX.poJiang Xin, May 23, 2022
  72. 1/9 Makefile: sort "po/git.pot" by file locationJiang Xin, May 23, 2022
  73. Junio C HamanoMay 23, 2022
  74. Jiang XinMay 23, 2022
  75. 2/9 Makefile: generate "po/git.pot" from stable LOCALIZED_CJiang Xin, May 23, 2022
  76. Junio C HamanoMay 23, 2022
  77. 9/9 l10n: Document the new l10n workflowJiang Xin, May 23, 2022
  78. 7/9 Makefile: add "po-update" rule to update po/XX.poJiang Xin, May 23, 2022
  79. 8/9 Makefile: add "po-init" rule to initialize po/XX.poJiang Xin, May 23, 2022
  80. 3/9 Makefile: have "make pot" not "reset --hard"Jiang Xin, May 23, 2022
  81. Ævar Arnfjörð BjarmasonMay 23, 2022
  82. Jiang XinMay 23, 2022
  83. Jiang XinMay 24, 2022
  84. Junio C HamanoMay 23, 2022
  85. Jiang XinMay 23, 2022
  86. 4/9 i18n CI: stop allowing non-ASCII source messages in po/git.potJiang Xin, May 23, 2022
  87. 6/9 po/git.pot: don't check in result of "make pot"Jiang Xin, May 23, 2022
  88. Ævar Arnfjörð BjarmasonMay 23, 2022
  89. Jiang XinMay 23, 2022
  90. Jiang XinMay 23, 2022
  91. Ævar Arnfjörð BjarmasonMay 23, 2022
  92. 5/9 po/git.pot: this is now a generated fileJiang Xin, May 23, 2022
  93. 2/9 Makefile: generate "po/git.pot" from stable LOCALIZED_CJiang Xin, May 19, 2022
  94. Ævar Arnfjörð BjarmasonMay 19, 2022
  95. Jiang XinMay 19, 2022
  96. 3/9 Makefile: have "make pot" not "reset --hard"Jiang Xin, May 19, 2022
  97. Ævar Arnfjörð BjarmasonMay 19, 2022
  98. Jiang XinMay 19, 2022
  99. Ævar Arnfjörð BjarmasonMay 19, 2022
  100. 5/9 po/git.pot: this is now a generated fileJiang Xin, May 19, 2022
  101. 7/9 Makefile: add "po-update" rule to update po/XX.poJiang Xin, May 19, 2022
  102. Ævar Arnfjörð BjarmasonMay 19, 2022
  103. 6/9 po/git.pot: don't check in result of "make pot"Jiang Xin, May 19, 2022
  104. 8/9 Makefile: add "po-init" rule to initialize po/XX.poJiang Xin, May 19, 2022
  105. Ævar Arnfjörð BjarmasonMay 19, 2022
  106. 9/9 l10n: Document the new l10n workflowJiang Xin, May 19, 2022
  107. Junio C HamanoMay 19, 2022
  108. Jiang XinMay 21, 2022
  109. 4/9 i18n CI: stop allowing non-ASCII source messages in po/git.potJiang Xin, May 19, 2022
  110. Ævar Arnfjörð BjarmasonMay 19, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.