git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH] ci: disallow directional formatting

From
Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date
Nov 2, 2021, 15:01 UTC
Message-ID
<211102.86tugu1svc.gmgdl@evledraar.gmail.com>
In-Reply-To
<pull.1071.git.1635857935312.gitgitgadget@gmail.com>
On Tue, Nov 02 2021, Johannes Schindelin via GitGitGadget wrote:
Show 24 quoted lines
> From: Johannes Schindelin <johannes.schindelin@gmx.de>
>
> As described in https://trojansource.codes/trojan-source.pdf, it is
> possible to abuse directional formatting (a feature of Unicode) to
> deceive human readers into interpreting code differently from compilers.
>
> It is highly unlikely that Git's source code wants to contain such
> directional formatting in the first place, so let's disallow it.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>     ci: disallow directional formatting
>     
>     I just stumbled over
>     https://siliconangle.com/2021/11/01/trojan-source-technique-can-inject-malware-source-code-without-detection/,
>     which details an interesting social-engineering attack: it uses
>     directional formatting in source code to pretend to human readers that
>     the code does something different than it actually does.
>     
>     It is highly unlikely that Git's source code wants to contain such
>     directional formatting in the first place, so let's disallow it.
>     
>     Technically, this is not exactly -rc material, but the paper was just
>     published, and I want us to be safe.

There's a parallel discussion about doing something to detect this in "git am", which for the git project seems like a better place to put this.

Show 27 quoted lines
> Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1071%2Fdscho%2Fcheck-for-utf-8-directional-formatting-v1
> Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1071/dscho/check-for-utf-8-directional-formatting-v1
> Pull-Request: https://github.com/gitgitgadget/git/pull/1071
>
>  .github/workflows/main.yml | 7 +++++++
>  1 file changed, 7 insertions(+)
>
> diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
> index 6ed6a9e8076..7b4b4df03c3 100644
> --- a/.github/workflows/main.yml
> +++ b/.github/workflows/main.yml
> @@ -289,6 +289,13 @@ jobs:
>      - uses: actions/checkout@v2
>      - run: ci/install-dependencies.sh
>      - run: ci/run-static-analysis.sh
> +    - name: disallow Unicode directional formatting
> +      run: |
> +        # Use UTF-8-aware `printf` to feed a byte pattern to non-UTF-8-aware `git grep`
> +        # (Ubuntu's `git grep` is compiled without support for libpcre, otherwise we
> +        # could use `git grep -P` with the `\u` syntax).
> +        ! LANG=C git grep -Il "$(LANG=C.UTF-8 printf \
> +          '\\(\u202a\\|\u202b\\|\u202c\\|\u202d\\|\u202e\\|\u2066\\|\u2067\\|\u2068\\|\u2069\\)')"
>    sparse:
>      needs: ci-config
>      if: needs.ci-config.outputs.enabled == 'yes'
>
> base-commit: 0cddd84c9f3e9c3d793ec93034ef679335f35e49

It would be easier to maintain this if were added to ci/run-static-analysis.sh instead, where we have some similar tests, and if it lives there we could do away with the double-escaping, then it can also be run manually.

Also, can't we just pipe "git ls-files -z" into "perl -0ne" here and use its unconditional support for e.g. unicode properties in regexes.

How will this change impact RTL languages being added to po/?
Previous: Johannes Schindelin via GitGitGadgetNext: Taylor Blau
Message 2 of 18 in “ci: disallow directional formatting”
  1. ci: disallow directional formattingJohannes Schindelin via GitGitGadget, Nov 2, 2021
  2. Ævar Arnfjörð BjarmasonNov 2, 2021
  3. Taylor BlauNov 2, 2021
  4. Ævar Arnfjörð BjarmasonNov 2, 2021
  5. Johannes SchindelinNov 2, 2021
  6. Ævar Arnfjörð BjarmasonNov 2, 2021
  7. Junio C HamanoNov 3, 2021
  8. ci: disallow directional formattingJohannes Schindelin via GitGitGadget, Nov 3, 2021
  9. Junio C HamanoNov 3, 2021
  10. Ævar Arnfjörð BjarmasonNov 3, 2021
  11. Junio C HamanoNov 3, 2021
  12. Johannes SchindelinNov 4, 2021
  13. ci: disallow directional formattingJohannes Schindelin via GitGitGadget, Nov 4, 2021
  14. Ævar Arnfjörð BjarmasonNov 4, 2021
  15. Junio C HamanoNov 4, 2021
  16. Ævar Arnfjörð BjarmasonNov 8, 2021
  17. Junio C HamanoNov 8, 2021
  18. Ævar Arnfjörð BjarmasonNov 9, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.