git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH] specify encoding for sed command

From
SZEDER Gábor <szeder.dev@gmail.com>
Date
Apr 13, 2018, 00:01 UTC
Message-ID
<20180413000105.21708-1-szeder.dev@gmail.com>
In-Reply-To
<CBA1FB4A-C586-48E0-A64E-371CCD2F6AC4@1eanda.com>
> I did a little more digging into this issue today.
Show 21 quoted lines
> The issue isn't actually caused by `sed`: it's caused by the way that
> the `set` builtin outputs characters in the Unicode Private Use Area
> (PUA) in the build of Bash 3.2.57 that macOS Sierra ships with.
> 
> Powerline uses several PUA code points to make some of its pretty text
> UI elements:
> 
> Code point (hex value): description
> U+E0A0 (0xEE82A0): Version control branch
> U+E0A1 (0xEE82A1): LN (line) symbol
> U+E0A2 (0xEE82A2): Closed padlock
> U+E0B0 (0xEE82B0): Rightwards black arrowhead
> U+E0B1 (0xEE82B1): Rightwards arrowhead
> U+E0B2 (0xEE82B2): Leftwards black arrowhead
> U+E0B3 (0xEE82B3): Leftwards arrowhead
> 
> macOS Bash 3.2.57's `set` builtin has garbled output where Powerline's
> special symbols should be in the PS1 variable, but Bash 4.4.19
> (installed on macOS via homebrew) and Bash 4.3.38 (Ubuntu 16.04) both
> display it correctly in the output of `set`. `echo $PS1` does display
> the symbols correctly on these versions of Bash.
Thanks for digging.
Show 5 quoted lines
> I think the best way to move forward with this is a new patch that uses
> `awk` instead of `sed`: I tested several `awk` variants and the command
> was portable without requiring any changes to LANG or LC_ALL.
> 
> Does that sound like a good plan?

No ;) Could you please give the patch below a try?

I intended it as a fix for a minor performance regression introduced in the same commit that triggered this issue, but if it can work around this issue, too, all the better.

I'm just not sure whether we should burden this otherwise nice and short commit message with the details of this Powerline-macOS-Bash-sed issue...

  -- >8 --
Subject: [PATCH] completion: reduce overhead of clearing cached --options

To get the names of all '$__git_builtin_*' variables caching --options of builtin commands in order to unset them, 8b0eaa41f2 (completion: clear cached --options when sourcing the completion script, 2018-03-22) runs a 'set |sed s///' pipeline. This works both in Bash and in ZSH, but has a higher than necessasry overhead with the extra processes.

In Bash we can do better: run the 'compgen -v __gitcomp_builtin_' builtin command, which lists the same variables, but without a pipeline and 'sed' it can do so with lower overhead.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
TODO: as an unexpected bonus, this might work around the issue with
'sed' and invalid UTF-8 characters in the environment as well, at least
in Bash.
  https://public-inbox.org/git/0102016293c8dca7-6626fcde-548d-476e-b61f-c83ecdeedfe1-000000@eu-west-1.amazonses.com/
---
 contrib/completion/git-completion.bash | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index b09c8a2362..4ef59a51be 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -282,7 +282,11 @@ __gitcomp ()
 
 # Clear the variables caching builtins' options when (re-)sourcing
 # the completion script.
-unset $(set |sed -ne 's/^\(__gitcomp_builtin_[a-zA-Z0-9_][a-zA-Z0-9_]*\)=.*/\1/p') 2>/dev/null
+if [[ -n ${ZSH_VERSION-} ]]; then
+	unset $(set |sed -ne 's/^\(__gitcomp_builtin_[a-zA-Z0-9_][a-zA-Z0-9_]*\)=.*/\1/p') 2>/dev/null
+else
+	unset $(compgen -v __gitcomp_builtin_)
+fi
 
 # This function is equivalent to
 #
-- 
2.17.0.366.gbe216a3084
Previous: Matthew ColemanNext: Matthew Coleman
Message 8 of 38 in “specify encoding for sed command”
  1. specify encoding for sed commandStephon Harris, Apr 5, 2018
  2. Ævar Arnfjörð BjarmasonApr 5, 2018
  3. Eric SunshineApr 5, 2018
  4. SZEDER GáborApr 5, 2018
  5. Matt ColemanApr 10, 2018
  6. Matt ColemanApr 11, 2018
  7. Matthew ColemanApr 12, 2018
  8. SZEDER GáborApr 13, 2018
  9. Matthew ColemanApr 13, 2018
  10. completion: reduce overhead of clearing cached --optionsSZEDER Gábor, Apr 13, 2018
  11. Jakub NarebskiApr 13, 2018
  12. SZEDER GáborApr 13, 2018
  13. Jakub NarebskiApr 14, 2018
  14. Jacob KellerApr 16, 2018
  15. Matthew ColemanApr 16, 2018
  16. Junio C HamanoApr 16, 2018
  17. SZEDER GáborApr 16, 2018
  18. Jakub NarębskiApr 16, 2018
  19. Junio C HamanoApr 16, 2018
  20. completion: reduce overhead of clearing cached --optionsSZEDER Gábor, Apr 17, 2018
  21. Junio C HamanoApr 17, 2018
  22. Matthew ColemanMay 7, 2018
  23. Todd ZullingerMay 8, 2018
  24. Junio C HamanoMay 8, 2018
  25. Jonathan NiederJun 7, 2018
  26. Dave BorowitzJun 7, 2018
  27. Rick van HattemJun 7, 2018
  28. SZEDER GáborJun 8, 2018
  29. Jonathan NiederJun 8, 2018
  30. SZEDER GáborJun 8, 2018
  31. Jonathan NiederJun 8, 2018
  32. SZEDER GáborJun 8, 2018
  33. Junio C HamanoJun 11, 2018
  34. completion: correct zsh detection when run from git-completion.zsh (Re: [PATCH v2] completion: reduce overhead of clearing cached --options)Jonathan Nieder, Jun 11, 2018
  35. SZEDER GáborJun 12, 2018
  36. SZEDER GáborJun 12, 2018
  37. Rick van HattemJun 12, 2018
  38. Junio C HamanoApr 8, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.