threads / discuss / 53352

git fast-export not preserving executable permissions?

Subject: git fast-export not preserving executable permissions?

## tl;dr

6 messages between Apr 29, 2020 and May 4, 2020.

replies: 5people: 4as markdown or json

Doug Glidden· Apr 29, 2020, 13:36 UTC · lore
Hello Git world!

I have run into an issue that I cannot seem to resolve with git fast-export. When running a fast-export on a repo that contains scripts with executable permissions (e.g. a gradlew script), the resulting export does not properly reflect the executable permissions on the script files.

To illustrate this issue, I created a small sample repo, with one executable file and one non-executable file. From the output below, you can see that the mode in the output from fast-export is the same for both files; according to the documentation for fast-import, the mode for the executable file should be 100755 instead of 100644.

    $ ls -gG
    total 2
    -rwxr-xr-x 1 106 Apr 29 09:13 executable_script.sh*
    -rw-r--r-- 1  63 Apr 29 09:12 non_executable_file.txt
    $ git fast-export --all
    blob
    mark :1
    data 106
    #!/bin/bash
    # This is a shell script that should be executable.
    echo 'The script executed successfully!'
    blob
    mark :2
    data 63
    This file is a simple text file that should not be executable.
    reset refs/heads/dev
    commit refs/heads/dev
    mark :3
    author Doug <41mortimer@gmail.com> 1588167102 -0400
    committer Doug <41mortimer@gmail.com> 1588167102 -0400
    data 25
    Adding some sample files
    M 100644 :1 executable_script.sh
    M 100644 :2 non_executable_file.txt

Please let me know if there is any further information I can provide about this issue.

Thank you, Doug

Taylor Blau· Apr 29, 2020, 18:49 UTC · re: Doug Glidden · lore

Re: git fast-export not preserving executable permissions?

Hi Doug,
On Wed, Apr 29, 2020 at 09:36:31AM -0400, Doug Glidden wrote:
Show 7 quoted lines
> Hello Git world!
>
> I have run into an issue that I cannot seem to resolve with git
> fast-export. When running a fast-export on a repo that contains
> scripts with executable permissions (e.g. a gradlew script), the
> resulting export does not properly reflect the executable permissions
> on the script files.

Interesting. fast-import and fast-export both understand executable modes (although Git only understands the modes 644 and 755 for blobs), so this should be working.

I can not reproduce the issue as-is. Round-tripping a fast-import and fast-export preserves executable bits for me:

  #!/bin/bash
  set -e
  rm -rf repo client
  git init -q repo
  git init -q client
  (
    cd repo
    printf "x" >x
    printf "y" >y
    chmod +x x
    git add x y
    git commit -q -m "initial commit"
  )
  git -C repo fast-export HEAD | git -C client fast-import
  diff -u <(git -C repo ls-tree HEAD) <(git -C client ls-tree HEAD)
Show 37 quoted lines
> To illustrate this issue, I created a small sample repo, with one
> executable file and one non-executable file. From the output below,
> you can see that the mode in the output from fast-export is the same
> for both files; according to the documentation for fast-import, the
> mode for the executable file should be 100755 instead of 100644.
>
>     $ ls -gG
>     total 2
>     -rwxr-xr-x 1 106 Apr 29 09:13 executable_script.sh*
>     -rw-r--r-- 1  63 Apr 29 09:12 non_executable_file.txt
>
>     $ git fast-export --all
>     blob
>     mark :1
>     data 106
>     #!/bin/bash
>
>     # This is a shell script that should be executable.
>     echo 'The script executed successfully!'
>
>     blob
>     mark :2
>     data 63
>     This file is a simple text file that should not be executable.
>
>     reset refs/heads/dev
>     commit refs/heads/dev
>     mark :3
>     author Doug <41mortimer@gmail.com> 1588167102 -0400
>     committer Doug <41mortimer@gmail.com> 1588167102 -0400
>     data 25
>     Adding some sample files
>     M 100644 :1 executable_script.sh
>     M 100644 :2 non_executable_file.txt
>
> Please let me know if there is any further information I can provide
> about this issue.

Does Git think that the file is executable? Please run 'git ls-tree HEAD' to find out.

> Thank you,
> Doug

Thanks, Taylor

Doug Glidden· May 1, 2020, 13:35 UTC · re: Taylor Blau · lore

Re: git fast-export not preserving executable permissions?

Taylor,

Thanks for your response! It looks like git does not actually recognize the file as executable:

    $ git ls-tree HEAD
    100644 blob 7d2f57b2381766924e1e4ffcc62615c637bbd784    executable_script.sh
    100644 blob d1d7cf309e091f54f268503b31653d8eba42fe88
non_executable_file.txt

Now you have me wondering if the real problem here is that I'm working in git-bash on a Windows machine, which means the file permissions aren't completely native. I'm going to run a similar experiment in a native Linux environment and see if I get the same results. I'll let you know what I find.

Thanks, Doug

On Wed, Apr 29, 2020 at 2:49 PM Taylor Blau <me@ttaylorr.com> wrote:
Show 87 quoted lines
>
> Hi Doug,
>
> On Wed, Apr 29, 2020 at 09:36:31AM -0400, Doug Glidden wrote:
> > Hello Git world!
> >
> > I have run into an issue that I cannot seem to resolve with git
> > fast-export. When running a fast-export on a repo that contains
> > scripts with executable permissions (e.g. a gradlew script), the
> > resulting export does not properly reflect the executable permissions
> > on the script files.
>
> Interesting. fast-import and fast-export both understand executable
> modes (although Git only understands the modes 644 and 755 for blobs),
> so this should be working.
>
> I can not reproduce the issue as-is. Round-tripping a fast-import and
> fast-export preserves executable bits for me:
>
>   #!/bin/bash
>
>   set -e
>
>   rm -rf repo client
>
>   git init -q repo
>   git init -q client
>
>   (
>     cd repo
>     printf "x" >x
>     printf "y" >y
>     chmod +x x
>     git add x y
>     git commit -q -m "initial commit"
>   )
>
>   git -C repo fast-export HEAD | git -C client fast-import
>
>   diff -u <(git -C repo ls-tree HEAD) <(git -C client ls-tree HEAD)
>
> > To illustrate this issue, I created a small sample repo, with one
> > executable file and one non-executable file. From the output below,
> > you can see that the mode in the output from fast-export is the same
> > for both files; according to the documentation for fast-import, the
> > mode for the executable file should be 100755 instead of 100644.
> >
> >     $ ls -gG
> >     total 2
> >     -rwxr-xr-x 1 106 Apr 29 09:13 executable_script.sh*
> >     -rw-r--r-- 1  63 Apr 29 09:12 non_executable_file.txt
> >
> >     $ git fast-export --all
> >     blob
> >     mark :1
> >     data 106
> >     #!/bin/bash
> >
> >     # This is a shell script that should be executable.
> >     echo 'The script executed successfully!'
> >
> >     blob
> >     mark :2
> >     data 63
> >     This file is a simple text file that should not be executable.
> >
> >     reset refs/heads/dev
> >     commit refs/heads/dev
> >     mark :3
> >     author Doug <41mortimer@gmail.com> 1588167102 -0400
> >     committer Doug <41mortimer@gmail.com> 1588167102 -0400
> >     data 25
> >     Adding some sample files
> >     M 100644 :1 executable_script.sh
> >     M 100644 :2 non_executable_file.txt
> >
> > Please let me know if there is any further information I can provide
> > about this issue.
>
> Does Git think that the file is executable? Please run 'git ls-tree
> HEAD' to find out.
>
> > Thank you,
> > Doug
>
> Thanks,
> Taylor
Taylor Blau· May 1, 2020, 22:32 UTC · re: Doug Glidden · lore

Re: git fast-export not preserving executable permissions?

On Fri, May 01, 2020 at 09:35:28AM -0400, Doug Glidden wrote:
> Taylor,

(My full response is below, but please in general do not top-quote mail here.)

Show 12 quoted lines
>
> Thanks for your response! It looks like git does not actually
> recognize the file as executable:
>
>     $ git ls-tree HEAD
>     100644 blob 7d2f57b2381766924e1e4ffcc62615c637bbd784    executable_script.sh
>     100644 blob d1d7cf309e091f54f268503b31653d8eba42fe88
> non_executable_file.txt
>
> Now you have me wondering if the real problem here is that I'm working
> in git-bash on a Windows machine, which means the file permissions
> aren't completely native.

I was wondering if that was the case ;-). If you are using NTFS or FAT32, neither of these filesystems support execute permission bits. (I am certainly not an expert here, but I know that Dscho (cc'd) would be able to answer authoritatively here.)

That said, *Git* understands executable permissions, even if your filesystem doesn't. You can tell Git to mark a file as executable by the following:

  $ git update-index --chmod=+x /path/to/file

and then committing the result. Round-tripping this through 'git fast-{im,ex}port' should preserve the permissions from Git's perspective, and ditto for checking out the contents of a repository on a filesystem that does support the executable permission bit.

> I'm going to run a similar experiment in a native Linux environment
> and see if I get the same results. I'll let you know what I find.
Sounds good. I'll be very surprised if it doesn't work as you expect.
Show 93 quoted lines
> Thanks,
> Doug
>
>
>
> On Wed, Apr 29, 2020 at 2:49 PM Taylor Blau <me@ttaylorr.com> wrote:
> >
> > Hi Doug,
> >
> > On Wed, Apr 29, 2020 at 09:36:31AM -0400, Doug Glidden wrote:
> > > Hello Git world!
> > >
> > > I have run into an issue that I cannot seem to resolve with git
> > > fast-export. When running a fast-export on a repo that contains
> > > scripts with executable permissions (e.g. a gradlew script), the
> > > resulting export does not properly reflect the executable permissions
> > > on the script files.
> >
> > Interesting. fast-import and fast-export both understand executable
> > modes (although Git only understands the modes 644 and 755 for blobs),
> > so this should be working.
> >
> > I can not reproduce the issue as-is. Round-tripping a fast-import and
> > fast-export preserves executable bits for me:
> >
> >   #!/bin/bash
> >
> >   set -e
> >
> >   rm -rf repo client
> >
> >   git init -q repo
> >   git init -q client
> >
> >   (
> >     cd repo
> >     printf "x" >x
> >     printf "y" >y
> >     chmod +x x
> >     git add x y
> >     git commit -q -m "initial commit"
> >   )
> >
> >   git -C repo fast-export HEAD | git -C client fast-import
> >
> >   diff -u <(git -C repo ls-tree HEAD) <(git -C client ls-tree HEAD)
> >
> > > To illustrate this issue, I created a small sample repo, with one
> > > executable file and one non-executable file. From the output below,
> > > you can see that the mode in the output from fast-export is the same
> > > for both files; according to the documentation for fast-import, the
> > > mode for the executable file should be 100755 instead of 100644.
> > >
> > >     $ ls -gG
> > >     total 2
> > >     -rwxr-xr-x 1 106 Apr 29 09:13 executable_script.sh*
> > >     -rw-r--r-- 1  63 Apr 29 09:12 non_executable_file.txt
> > >
> > >     $ git fast-export --all
> > >     blob
> > >     mark :1
> > >     data 106
> > >     #!/bin/bash
> > >
> > >     # This is a shell script that should be executable.
> > >     echo 'The script executed successfully!'
> > >
> > >     blob
> > >     mark :2
> > >     data 63
> > >     This file is a simple text file that should not be executable.
> > >
> > >     reset refs/heads/dev
> > >     commit refs/heads/dev
> > >     mark :3
> > >     author Doug <41mortimer@gmail.com> 1588167102 -0400
> > >     committer Doug <41mortimer@gmail.com> 1588167102 -0400
> > >     data 25
> > >     Adding some sample files
> > >     M 100644 :1 executable_script.sh
> > >     M 100644 :2 non_executable_file.txt
> > >
> > > Please let me know if there is any further information I can provide
> > > about this issue.
> >
> > Does Git think that the file is executable? Please run 'git ls-tree
> > HEAD' to find out.
> >
> > > Thank you,
> > > Doug
> >
> > Thanks,
> > Taylor

Thanks, Taylor

Johannes Schindelin· May 4, 2020, 14:23 UTC · re: Taylor Blau · lore

Re: git fast-export not preserving executable permissions?

Hi,
On Fri, 1 May 2020, Taylor Blau wrote:
> If you are using NTFS or FAT32, neither of these filesystems support
> execute permission bits. (I am certainly not an expert here, but I know
> that Dscho (cc'd) would be able to answer authoritatively here.)

On Windows, there are indeed no executable bits. A file is considered executable if it has a file extension indicating it, e.g. `.exe`.

In Git for Windows' Bash (which comes from MSYS2), `ls -l` will also consider files whose first line is a hash-bang one (e.g. `#!/bin/sh`) as executable. This information is however _not_ used by Git.

If you want to mark a file as executable in Git for Windows, you will have to use `git add --chmod=+x <file>`.

Ciao, Johannes

Torsten Bögershausen· Apr 30, 2020, 03:42 UTC · re: Doug Glidden · lore

Re: git fast-export not preserving executable permissions?

On Wed, Apr 29, 2020 at 09:36:31AM -0400, Doug Glidden wrote:
Show 18 quoted lines
> Hello Git world!
>
> I have run into an issue that I cannot seem to resolve with git
> fast-export. When running a fast-export on a repo that contains
> scripts with executable permissions (e.g. a gradlew script), the
> resulting export does not properly reflect the executable permissions
> on the script files.
>
> To illustrate this issue, I created a small sample repo, with one
> executable file and one non-executable file. From the output below,
> you can see that the mode in the output from fast-export is the same
> for both files; according to the documentation for fast-import, the
> mode for the executable file should be 100755 instead of 100644.
>
>     $ ls -gG
>     total 2
>     -rwxr-xr-x 1 106 Apr 29 09:13 executable_script.sh*
>     -rw-r--r-- 1  63 Apr 29 09:12 non_executable_file.txt

What does git ls-files -s give you here ? We need to know, how Git tracks the files, is it 100644 or 100755 ?

The following works for me:
user@pc:/tmp/fff> git init
Initialized empty Git repository in /private/tmp/fff/.git/
user@pc:/tmp/fff> echo file1 >file1
user@pc:/tmp/fff> git config core.filemode false
user@pc:/tmp/fff> echo file2 >file2
user@pc:/tmp/fff> git add file1 file2
user@pc:/tmp/fff> git ls-files -s
100644 e2129701f1a4d54dc44f03c93bca0a2aec7c5449 0       file1
100644 6c493ff740f9380390d5c9ddef4af18697ac9375 0       file2
user@pc:/tmp/fff> git update-index --chmod=+x file2
user@pc:/tmp/fff> git ls-files -s
100644 e2129701f1a4d54dc44f03c93bca0a2aec7c5449 0       file1
100755 6c493ff740f9380390d5c9ddef4af18697ac9375 0       file2
user@pc:/tmp/fff> git commit -m "Add file1 file2"
[master (root-commit) f75926e] Add file1 file2
 2 files changed, 2 insertions(+)
  create mode 100644 file1
  create mode 100755 file2

user@pc:/tmp/fff> git fast-export --all blob mark :1 data 6 file1

blob mark :2 data 6 file2

reset refs/heads/master commit refs/heads/master mark :3 author Torsten Bögershausen <tboegi@web.de> 1588217922 +0200 committer Torsten Bögershausen <tboegi@web.de> 1588217922 +0200 data 16 Add file1 file2 M 100644 :1 file1 M 100755 :2 file2

← back to recent threads