threads / discuss / 56326

Git Modifying DLL

Subject: Git Modifying DLL

## tl;dr

9 messages between Aug 19, 2021 and Sep 1, 2021.

replies: 8people: 5as markdown or json

Jonathon Anderson· Aug 19, 2021, 17:58 UTC · lore

I'm having an issue with git modifying a DLL file and corrupting it. When I download the original working file, it has a hash starting with 8FE400... I then commit the DLL and push it to our repo. When I download the file from the repo, the DLL can't be loaded, and it has a hash starting with E004FB...

Opening the DLL in a hex editor and using the compare feature, there's a single change to the file. In the original, the byte code starting at 0x0074 is 2E 0D 0D 0A 24. In the git file, the byte code starting at 0x0074 is 2E 0D 0A 24

A single carriage return character (0x0D) has been removed, and the file size has changed from 260,608 bytes to 260,607 bytes.

I ruled out the possibility that the repo server was doing anything to the file because I deleted the file in my local repository then ran "git reset --hard HEAD" to restore the file, and the hash had once again changed to E004FB...

OS: Windows 10.0.19043 pro
git: 2.32.0.windows.1

I have no settings configured for git behavior handling line endings.

The original DLL can be found here: https://www.powershellgallery.com/packages/PSWindowsUpdate/2.1.1.2

Navigate to "Manual Download", download the nuget package and unzip it. The file is PSWindowsUpdate.dll

Thank you, Jonathon

Randall S. Becker· Aug 19, 2021, 18:13 UTC · re: Jonathon Anderson · lore

RE: Git Modifying DLL

On August 19, 2021 1:59 PM, Jonathon Anderson wrote:

Show 23 quoted lines

>
>I'm having an issue with git modifying a DLL file and corrupting it.
>When I download the original working file, it has a hash starting with 8FE400... I then commit the DLL and push it to our repo. When I
>download the file from the repo, the DLL can't be loaded, and it has a hash starting with E004FB...
>
>Opening the DLL in a hex editor and using the compare feature, there's a single change to the file. In the original, the byte code starting
>at 0x0074 is 2E 0D 0D 0A 24.
>In the git file, the byte code starting at 0x0074 is 2E 0D 0A 24
>
>A single carriage return character (0x0D) has been removed, and the file size has changed from 260,608 bytes to 260,607 bytes.
>
>I ruled out the possibility that the repo server was doing anything to the file because I deleted the file in my local repository then ran "git
>reset --hard HEAD" to restore the file, and the hash had once again changed to E004FB...
>
>OS: Windows 10.0.19043 pro
>git: 2.32.0.windows.1
>
>I have no settings configured for git behavior handling line endings.
>
>The original DLL can be found here:
>https://www.powershellgallery.com/packages/PSWindowsUpdate/2.1.1.2
>
>Navigate to "Manual Download", download the nuget package and unzip it. The file is PSWindowsUpdate.dll

Have you set up an entry for *.dll as binary in your .gitattributes file?

-Randall

Jonathon Anderson· Aug 19, 2021, 18:21 UTC · re: Randall S. Becker · lore

Re: Git Modifying DLL

I had not. I tested that and it worked. I assumed that git would automatically treat dll files as binary. Thanks for the help!

On Thu, Aug 19, 2021 at 1:13 PM Randall S. Becker <rsbecker@nexbridge.com> wrote:

Show 30 quoted lines

>
> On August 19, 2021 1:59 PM, Jonathon Anderson wrote:
> >
> >I'm having an issue with git modifying a DLL file and corrupting it.
> >When I download the original working file, it has a hash starting with 8FE400... I then commit the DLL and push it to our repo. When I
> >download the file from the repo, the DLL can't be loaded, and it has a hash starting with E004FB...
> >
> >Opening the DLL in a hex editor and using the compare feature, there's a single change to the file. In the original, the byte code starting
> >at 0x0074 is 2E 0D 0D 0A 24.
> >In the git file, the byte code starting at 0x0074 is 2E 0D 0A 24
> >
> >A single carriage return character (0x0D) has been removed, and the file size has changed from 260,608 bytes to 260,607 bytes.
> >
> >I ruled out the possibility that the repo server was doing anything to the file because I deleted the file in my local repository then ran "git
> >reset --hard HEAD" to restore the file, and the hash had once again changed to E004FB...
> >
> >OS: Windows 10.0.19043 pro
> >git: 2.32.0.windows.1
> >
> >I have no settings configured for git behavior handling line endings.
> >
> >The original DLL can be found here:
> >https://www.powershellgallery.com/packages/PSWindowsUpdate/2.1.1.2
> >
> >Navigate to "Manual Download", download the nuget package and unzip it. The file is PSWindowsUpdate.dll
>
> Have you set up an entry for *.dll as binary in your .gitattributes file?
>
> -Randall
>

Torsten Bögershausen· Aug 20, 2021, 15:37 UTC · re: Jonathon Anderson · lore

Re: Git Modifying DLL

First of all, thanks for the report. (And please, no top-posting in this list, see my comments at the end)

On Thu, Aug 19, 2021 at 01:21:03PM -0500, Jonathon Anderson wrote:

Show 35 quoted lines

> I had not. I tested that and it worked. I assumed that git would
> automatically treat dll files as binary. Thanks for the help!
>
> On Thu, Aug 19, 2021 at 1:13 PM Randall S. Becker
> <rsbecker@nexbridge.com> wrote:
> >
> > On August 19, 2021 1:59 PM, Jonathon Anderson wrote:
> > >
> > >I'm having an issue with git modifying a DLL file and corrupting it.
> > >When I download the original working file, it has a hash starting with 8FE400... I then commit the DLL and push it to our repo. When I
> > >download the file from the repo, the DLL can't be loaded, and it has a hash starting with E004FB...
> > >
> > >Opening the DLL in a hex editor and using the compare feature, there's a single change to the file. In the original, the byte code starting
> > >at 0x0074 is 2E 0D 0D 0A 24.
> > >In the git file, the byte code starting at 0x0074 is 2E 0D 0A 24
> > >
> > >A single carriage return character (0x0D) has been removed, and the file size has changed from 260,608 bytes to 260,607 bytes.
> > >
> > >I ruled out the possibility that the repo server was doing anything to the file because I deleted the file in my local repository then ran "git
> > >reset --hard HEAD" to restore the file, and the hash had once again changed to E004FB...
> > >
> > >OS: Windows 10.0.19043 pro
> > >git: 2.32.0.windows.1
> > >
> > >I have no settings configured for git behavior handling line endings.
> > >
> > >The original DLL can be found here:
> > >https://www.powershellgallery.com/packages/PSWindowsUpdate/2.1.1.2
> > >
> > >Navigate to "Manual Download", download the nuget package and unzip it. The file is PSWindowsUpdate.dll
> >
> > Have you set up an entry for *.dll as binary in your .gitattributes file?
> >
> > -Randall
> >

I downloaded the dll.

I can not reproduce the issue here - so I wonder, what is going on.

Could you run the following experiment, please ? (in git bash or so)

mkdir dlltest cd dlltest git init cp whereveritis/PSWindowsUpdate.dll .

git ls-files -o --eol

# I get git ls-files --eol -o PSWindowsUpdate.dll i/ w/-text attr/ PSWindowsUpdate.dll

This means, that the file is "classified" as "-text" in the working tree. "-text" means the same as binary. And git should not change CRLF at all.

Do you have any more .gitattributes in you repo? Of course, you can run the command inside your repo. Omit the "-o" for other and simply run git ls-files --eol

Jeff King· Aug 20, 2021, 18:00 UTC · re: Jonathon Anderson · lore

Re: Git Modifying DLL

On Thu, Aug 19, 2021 at 01:21:03PM -0500, Jonathon Anderson wrote:

> I had not. I tested that and it worked. I assumed that git would
> automatically treat dll files as binary. Thanks for the help!

Git doesn't know about any file extensions by default. Its default "is it binary" test looks for NUL bytes in the first 8k or so of the file. I'd expect your DLL would probably have such a NUL byte.

Is it possible you have other .gitattributes set which are confusing things?

You might try:

  git check-attr --all <path>

or:

  git ls-files --stdin | git check-attr --stdin --all

-Peff

Jonathon Anderson· Aug 20, 2021, 18:47 UTC · re: Jeff King · lore

Re: Git Modifying DLL

On Fri, Aug 20, 2021 at 1:00 PM Jeff King <peff@peff.net> wrote:

Show 22 quoted lines

>
> On Thu, Aug 19, 2021 at 01:21:03PM -0500, Jonathon Anderson wrote:
>
> > I had not. I tested that and it worked. I assumed that git would
> > automatically treat dll files as binary. Thanks for the help!
>
> Git doesn't know about any file extensions by default. Its default "is
> it binary" test looks for NUL bytes in the first 8k or so of the file.
> I'd expect your DLL would probably have such a NUL byte.
>
> Is it possible you have other .gitattributes set which are confusing
> things?
>
> You might try:
>
>   git check-attr --all <path>
>
> or:
>
>   git ls-files --stdin | git check-attr --stdin --all
>
> -Peff

When I remove '*.dll binary" from .gitattributes, I get this:

$ git check-attr --all ./PSWindowsUpdate.dll ./PSWindowsUpdate.dll: text: set ./PSWindowsUpdate.dll: eol: lf

When I add "*.dll binary" to .gitattributes, I get this:

$ git check-attr --all ./PSWindowsUpdate.dll ./PSWindowsUpdate.dll: binary: set ./PSWindowsUpdate.dll: diff: unset ./PSWindowsUpdate.dll: merge: unset ./PSWindowsUpdate.dll: text: unset ./PSWindowsUpdate.dll: eol: lf

I can also confirm that there are null bytes in the first 8k bytes. These are the first 16 bytes

4D 5A 90 00 03 00 00 00 04 00 00 00 FF FF 00 00

Thanks, Jonathon

brian m. carlson· Aug 20, 2021, 19:14 UTC · re: Jonathon Anderson · lore

Re: Git Modifying DLL

On 2021-08-20 at 18:47:02, Jonathon Anderson wrote:

Show 29 quoted lines

> On Fri, Aug 20, 2021 at 1:00 PM Jeff King <peff@peff.net> wrote:
> >
> > On Thu, Aug 19, 2021 at 01:21:03PM -0500, Jonathon Anderson wrote:
> >
> > > I had not. I tested that and it worked. I assumed that git would
> > > automatically treat dll files as binary. Thanks for the help!
> >
> > Git doesn't know about any file extensions by default. Its default "is
> > it binary" test looks for NUL bytes in the first 8k or so of the file.
> > I'd expect your DLL would probably have such a NUL byte.
> >
> > Is it possible you have other .gitattributes set which are confusing
> > things?
> >
> > You might try:
> >
> >   git check-attr --all <path>
> >
> > or:
> >
> >   git ls-files --stdin | git check-attr --stdin --all
> >
> > -Peff
> 
> When I remove '*.dll binary" from .gitattributes, I get this:
> 
> $ git check-attr --all ./PSWindowsUpdate.dll
> ./PSWindowsUpdate.dll: text: set
> ./PSWindowsUpdate.dll: eol: lf

Yes, this is definitely not correct. The flag "text" being set tells Git to do line-ending conversion and "eol=lf" says to convert line endings into LF.

You should look for things in your .gitattributes file that say something like "* text", which you probably don't want. You could use "* text=auto", which should be fine for most cases, though. It's also possible those aren't in a .gitattributes file in your repository but one elsewhere on your system. You can check gitattributes(5) for the locations of other files that can affect it.

As a note, it is best practice not to check binary dependencies or build artifacts into the repo. Those are best stored elsewhere, such as an artifact server.

-- 
brian m. carlson (he/him or they/them)
Toronto, Ontario, CA

Randall S. Becker· Aug 20, 2021, 19:51 UTC · re: brian m. carlson · lore

RE: Git Modifying DLL

On August 20, 2021 3:15 PM, brian m. carlson wrote:

Show 40 quoted lines

>On 2021-08-20 at 18:47:02, Jonathon Anderson wrote:
>> On Fri, Aug 20, 2021 at 1:00 PM Jeff King <peff@peff.net> wrote:
>> >
>> > On Thu, Aug 19, 2021 at 01:21:03PM -0500, Jonathon Anderson wrote:
>> >
>> > > I had not. I tested that and it worked. I assumed that git would
>> > > automatically treat dll files as binary. Thanks for the help!
>> >
>> > Git doesn't know about any file extensions by default. Its default
>> > "is it binary" test looks for NUL bytes in the first 8k or so of the file.
>> > I'd expect your DLL would probably have such a NUL byte.
>> >
>> > Is it possible you have other .gitattributes set which are confusing
>> > things?
>> >
>> > You might try:
>> >
>> >   git check-attr --all <path>
>> >
>> > or:
>> >
>> >   git ls-files --stdin | git check-attr --stdin --all
>> >
>> > -Peff
>>
>> When I remove '*.dll binary" from .gitattributes, I get this:
>>
>> $ git check-attr --all ./PSWindowsUpdate.dll
>> ./PSWindowsUpdate.dll: text: set
>> ./PSWindowsUpdate.dll: eol: lf
>
>Yes, this is definitely not correct.  The flag "text" being set tells Git to do line-ending conversion and "eol=lf" says to convert line endings
>into LF.
>
>You should look for things in your .gitattributes file that say something like "* text", which you probably don't want.  You could use
>"* text=auto", which should be fine for most cases, though.  It's also possible those aren't in a .gitattributes file in your repository but one
>elsewhere on your system.  You can check gitattributes(5) for the locations of other files that can affect it.
>
>As a note, it is best practice not to check binary dependencies or build artifacts into the repo.  Those are best stored elsewhere, such as an
>artifact server.

OT: Unless you are using git as an artifact repository to retain the dependency relationship between objects and the source commits that built them for PCI and SWIFT compliance. git is exceptional when it comes to evidence-of-origin forensics of specific installation artifacts when release structures are more complex than "deploy everything" use cases. But that may be a discussion we can have elsewhere.

Just my $20.00. -Randall

Jonathon Anderson· Sep 1, 2021, 21:07 UTC · re: Randall S. Becker · lore

Re: Git Modifying DLL

So, I somehow didn't realize what this was when I was looking at it, but this is the first line in our .gitattributes

* text eol=lf

It all makes sense now.

And, yea, we're using git to retain dependency relationships and bring those files under version control so we can audit changes and go through the full peer review process, etc

On Fri, Aug 20, 2021 at 2:51 PM Randall S. Becker <rsbecker@nexbridge.com> wrote:

Show 48 quoted lines

>
> On August 20, 2021 3:15 PM, brian m. carlson wrote:
> >On 2021-08-20 at 18:47:02, Jonathon Anderson wrote:
> >> On Fri, Aug 20, 2021 at 1:00 PM Jeff King <peff@peff.net> wrote:
> >> >
> >> > On Thu, Aug 19, 2021 at 01:21:03PM -0500, Jonathon Anderson wrote:
> >> >
> >> > > I had not. I tested that and it worked. I assumed that git would
> >> > > automatically treat dll files as binary. Thanks for the help!
> >> >
> >> > Git doesn't know about any file extensions by default. Its default
> >> > "is it binary" test looks for NUL bytes in the first 8k or so of the file.
> >> > I'd expect your DLL would probably have such a NUL byte.
> >> >
> >> > Is it possible you have other .gitattributes set which are confusing
> >> > things?
> >> >
> >> > You might try:
> >> >
> >> >   git check-attr --all <path>
> >> >
> >> > or:
> >> >
> >> >   git ls-files --stdin | git check-attr --stdin --all
> >> >
> >> > -Peff
> >>
> >> When I remove '*.dll binary" from .gitattributes, I get this:
> >>
> >> $ git check-attr --all ./PSWindowsUpdate.dll
> >> ./PSWindowsUpdate.dll: text: set
> >> ./PSWindowsUpdate.dll: eol: lf
> >
> >Yes, this is definitely not correct.  The flag "text" being set tells Git to do line-ending conversion and "eol=lf" says to convert line endings
> >into LF.
> >
> >You should look for things in your .gitattributes file that say something like "* text", which you probably don't want.  You could use
> >"* text=auto", which should be fine for most cases, though.  It's also possible those aren't in a .gitattributes file in your repository but one
> >elsewhere on your system.  You can check gitattributes(5) for the locations of other files that can affect it.
> >
> >As a note, it is best practice not to check binary dependencies or build artifacts into the repo.  Those are best stored elsewhere, such as an
> >artifact server.
>
> OT: Unless you are using git as an artifact repository to retain the dependency relationship between objects and the source commits that built them for PCI and SWIFT compliance. git is exceptional when it comes to evidence-of-origin forensics of specific installation artifacts when release structures are more complex than "deploy everything" use cases. But that may be a discussion we can have elsewhere.
>
> Just my $20.00.
> -Randall
>