{"thread":{"id":"49296","subject":"Temporary git files for the gitdir created on a separate drive in workdir","startedAt":"2018-09-07T06:21:49Z","lastAt":"2018-09-11T16:10:16Z","messageCount":9,"participants":["Hultqvist","Junio C Hamano","Duy Nguyen"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"357608","messageId":"CAJmnt9Yfed1W7F=C+dzac3AEe7nRq2cNP335MepTczboKJNoEg@mail.gmail.com","threadId":"49296","inReplyTo":null,"subject":"Temporary git files for the gitdir created on a separate drive in workdir","fromName":"Hultqvist","fromEmail":"hultqvist@silentorbit.com","sentAt":"2018-09-07T06:24:01Z","receivedAt":"2018-09-07T06:21:49Z","isPatch":false,"sender":{"key":"hultqvist@silentorbit.com","avatar":"https://gravatar.com/avatar/7e58535ac43cad883a344e776736ad830963673ad20654f4d24f60920e0195e2?d=mp&s=160"},"body":"I'm having two types of git repos in separate drives, in this case G:\nis a Google Drive File Stream meaning a slow synchronized storage.\n\nFirst repo:\nWorkdir: G:\\Test1\nGitdir: C:\\Test1.git\nG:\\Test1\\.git containing \"gitdir: C:\\Test1.git\"\n\nSecond repo:\nWorkdir: G:\\Test2\nGitdir: G:\\Test2\\.git\n\nIn the first repo I see a lot of \"master\", \"config\" and \"index\" files\nbeing created in the root of the workdir.\nThe nature of the slow sync causes these files to duplicate - not gits fault.\n\nIn the second repo no artefacts are noticed.\n\nI understand that the files are first created \"somewhere else\" to make\nan atomic move later.\nIn the first repo, the nature of being two separate drives appear to\ncause the residue.\nIn the second repo it works right.\n\nConsidering that the gitdir could be located on a different drive than\nthe workdir wouldn't it make more sense to create the temporary files\nin a subdirectory inside the gitdir rather tan in the workdir?\n\nAre there any flags or environment variables to control where\ntemporary files are created?\n"},{"id":"357631","messageId":"xmqqbm99xngs.fsf@gitster-ct.c.googlers.com","threadId":"49296","inReplyTo":"CAJmnt9Yfed1W7F=C+dzac3AEe7nRq2cNP335MepTczboKJNoEg@mail.gmail.com","subject":"Re: Temporary git files for the gitdir created on a separate drive in workdir","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2018-09-07T16:48:51Z","receivedAt":"2018-09-07T16:48:57Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Hultqvist writes:\n\n> Considering that the gitdir could be located on a different drive than\n> the workdir wouldn't it make more sense to create the temporary files\n> in a subdirectory inside the gitdir rather tan in the workdir?\n\nI do not think we intend to create temporary files, whose final\ndestination is somewhere under $GIT_DIR/, in any working tree;\nrather, I think we try to create them inside $GIT_DIR (or possibly\nif the destination is a file in a subdirectory of $GIT_DIR, then in\nthe same subdirectory). What you are seeing definitely smells like\na bug in the worktree code, perhaps getting confused by the fact\nthat the full path to these places look \"unusual\" by starting with a\nsingle alphabet followed by a colon (IOW, this may manifest only in\nWindows port).\n\n\n"},{"id":"357679","messageId":"CACsJy8DymnKk+Xo6UPQyAsToNpkSX4Ae+jzc04+0qXrZ6bmb3w@mail.gmail.com","threadId":"49296","inReplyTo":"xmqqbm99xngs.fsf@gitster-ct.c.googlers.com","subject":"Re: Temporary git files for the gitdir created on a separate drive in workdir","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2018-09-08T06:08:09Z","receivedAt":"2018-09-08T06:08:37Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Fri, Sep 7, 2018 at 6:48 PM Junio C Hamano wrote:\n>\n> Hultqvist writes:\n>\n> > Considering that the gitdir could be located on a different drive than\n> > the workdir wouldn't it make more sense to create the temporary files\n> > in a subdirectory inside the gitdir rather tan in the workdir?\n>\n> I do not think we intend to create temporary files, whose final\n> destination is somewhere under $GIT_DIR/, in any working tree;\n> rather, I think we try to create them inside $GIT_DIR (or possibly\n> if the destination is a file in a subdirectory of $GIT_DIR, then in\n> the same subdirectory). What you are seeing definitely smells like\n> a bug in the worktree code, perhaps getting confused by the fact\n> that the full path to these places look \"unusual\" by starting with a\n> single alphabet followed by a colon (IOW, this may manifest only in\n> Windows port).\n\nI agree. Auditing the setup code did not reveal anything though. Our\ncode should recognize these unusual Windows paths as absolute and\nwhile I spotted an incorrect use of '/' (instead of is_dir_sep) it\ndoes not explain the problem here.\n\nHultqvist, if you set environment variable GIT_TRACE_SETUP to 1 and\nrun \"git status\" in G:\\Test1, what does it say?\n-- \nDuy\n"},{"id":"357686","messageId":"CAJmnt9ZOCJfGFJ98xD17BYU1P=nRZ=eG16Oj5DjTMaF7H6WTvg@mail.gmail.com","threadId":"49296","inReplyTo":"CACsJy8DymnKk+Xo6UPQyAsToNpkSX4Ae+jzc04+0qXrZ6bmb3w@mail.gmail.com","subject":"Re: Temporary git files for the gitdir created on a separate drive in workdir","fromName":"Hultqvist","fromEmail":"hultqvist@silentorbit.com","sentAt":"2018-09-08T09:31:09Z","receivedAt":"2018-09-08T09:28:54Z","isPatch":false,"sender":{"key":"hultqvist@silentorbit.com","avatar":"https://gravatar.com/avatar/7e58535ac43cad883a344e776736ad830963673ad20654f4d24f60920e0195e2?d=mp&s=160"},"body":"The bash commands are using a git and bash bundle that was installed\nin parallel with gitextensions(a gui for git)\n\nG:\\Min enhet> set GIT_TRACE_SETUP=1\nG:\\Min enhet> git st\n10:40:28.881927 trace.c:318 setup: git_dir:\nC:/Users/hultqvist/Drive.git\n10:40:28.881927 trace.c:319 setup: git_common_dir:\nC:/Users/hultqvist/Drive.git\n10:40:28.881927 trace.c:320 setup: worktree: G:/Min enhet\n10:40:28.881927 trace.c:321 setup: cwd: G:/Min enhet\n10:40:28.881927 trace.c:322 setup: prefix: (null)\n10:40:28.882930 chdir-notify.c:67 setup: chdir from 'G:/Min\nenhet' to 'G:/Min enhet'\nOn branch master\nYour branch is up to date with 'nas/master'.\n\nnothing to commit, working tree clean\n\n$ cat .git\ngitdir: C:\\Users\\hultqvist\\Drive.git\n\n$cat C:/Users/hultqvist/Drive.git/config\n[core]\n repositoryformatversion = 0\n filemode = false\n bare = false\n logallrefupdates = true\n ignorecase = true\n autocrlf = false\n...\n\nG:\\Min enhet> git version\n$ git version\ngit version 2.18.0.windows.1\n\n\n\n\nDen lör 8 sep. 2018 kl 08:08 skrev Duy Nguyen :\n>\n> On Fri, Sep 7, 2018 at 6:48 PM Junio C Hamano wrote:\n> >\n> > Hultqvist writes:\n> >\n> > > Considering that the gitdir could be located on a different drive than\n> > > the workdir wouldn't it make more sense to create the temporary files\n> > > in a subdirectory inside the gitdir rather tan in the workdir?\n> >\n> > I do not think we intend to create temporary files, whose final\n> > destination is somewhere under $GIT_DIR/, in any working tree;\n> > rather, I think we try to create them inside $GIT_DIR (or possibly\n> > if the destination is a file in a subdirectory of $GIT_DIR, then in\n> > the same subdirectory). What you are seeing definitely smells like\n> > a bug in the worktree code, perhaps getting confused by the fact\n> > that the full path to these places look \"unusual\" by starting with a\n> > single alphabet followed by a colon (IOW, this may manifest only in\n> > Windows port).\n>\n> I agree. Auditing the setup code did not reveal anything though. Our\n> code should recognize these unusual Windows paths as absolute and\n> while I spotted an incorrect use of '/' (instead of is_dir_sep) it\n> does not explain the problem here.\n>\n> Hultqvist, if you set environment variable GIT_TRACE_SETUP to 1 and\n> run \"git status\" in G:\\Test1, what does it say?\n> --\n> Duy\n"},{"id":"357687","messageId":"CACsJy8AMaUmWO2zi9+Dpr9_ZHE7A0rSs63h646w4uokyKRAdUg@mail.gmail.com","threadId":"49296","inReplyTo":"CAJmnt9ZOCJfGFJ98xD17BYU1P=nRZ=eG16Oj5DjTMaF7H6WTvg@mail.gmail.com","subject":"Re: Temporary git files for the gitdir created on a separate drive in workdir","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2018-09-08T13:09:09Z","receivedAt":"2018-09-08T13:09:38Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Sat, Sep 8, 2018 at 11:28 AM Hultqvist wrote:\n>\n> The bash commands are using a git and bash bundle that was installed\n> in parallel with gitextensions(a gui for git)\n>\n> G:\\Min enhet> set GIT_TRACE_SETUP=1\n> G:\\Min enhet> git st\n> 10:40:28.881927 trace.c:318 setup: git_dir:\n> C:/Users/hultqvist/Drive.git\n> 10:40:28.881927 trace.c:319 setup: git_common_dir:\n> C:/Users/hultqvist/Drive.git\n> 10:40:28.881927 trace.c:320 setup: worktree: G:/Min enhet\n> 10:40:28.881927 trace.c:321 setup: cwd: G:/Min enhet\n> 10:40:28.881927 trace.c:322 setup: prefix: (null)\n> 10:40:28.882930 chdir-notify.c:67 setup: chdir from 'G:/Min\n> enhet' to 'G:/Min enhet'\n\nUnfortunately this looks good. Whenever those files 'index',\n'config'... are created, they should always be created in\nC:\\Users\\hultqvist\\Drive.git, not G:\\Min enhet, including their\ntemporary versions. I don't know if there are any more changes on the\nwindows fork that could affect this though, I only checked git.git.\n-- \nDuy\n"},{"id":"357692","messageId":"CACsJy8DdxmVd_jBG7Tp1e7k-BUc6R1=oWzPZ3z6xUZs2XU9rMA@mail.gmail.com","threadId":"49296","inReplyTo":"CACsJy8AMaUmWO2zi9+Dpr9_ZHE7A0rSs63h646w4uokyKRAdUg@mail.gmail.com","subject":"Re: Temporary git files for the gitdir created on a separate drive in workdir","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2018-09-08T13:43:45Z","receivedAt":"2018-09-08T13:44:14Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Sat, Sep 8, 2018 at 3:09 PM Duy Nguyen wrote:\n>\n> On Sat, Sep 8, 2018 at 11:28 AM Hultqvist wrote:\n> >\n> > The bash commands are using a git and bash bundle that was installed\n> > in parallel with gitextensions(a gui for git)\n> >\n> > G:\\Min enhet> set GIT_TRACE_SETUP=1\n> > G:\\Min enhet> git st\n> > 10:40:28.881927 trace.c:318 setup: git_dir:\n> > C:/Users/hultqvist/Drive.git\n> > 10:40:28.881927 trace.c:319 setup: git_common_dir:\n> > C:/Users/hultqvist/Drive.git\n> > 10:40:28.881927 trace.c:320 setup: worktree: G:/Min enhet\n> > 10:40:28.881927 trace.c:321 setup: cwd: G:/Min enhet\n> > 10:40:28.881927 trace.c:322 setup: prefix: (null)\n> > 10:40:28.882930 chdir-notify.c:67 setup: chdir from 'G:/Min\n> > enhet' to 'G:/Min enhet'\n>\n> Unfortunately this looks good. Whenever those files 'index',\n> 'config'... are created, they should always be created in\n> C:\\Users\\hultqvist\\Drive.git, not G:\\Min enhet, including their\n> temporary versions. I don't know if there are any more changes on the\n> windows fork that could affect this though, I only checked git.git.\n\nBTW do you notice these files showing up after any particular command\nor they're always there after cloning? Perhaps some command got the\n\".git\" directory discovery wrong and assumed $GIT_DIR=$GIT_WORK_TREE.\nWe have a much bigger problem then.\n-- \nDuy\n"},{"id":"357761","messageId":"CAJmnt9YxGoQO40GNQY=nTjGGaGB5X079koZ0QvCOZ4wd7ubkaA@mail.gmail.com","threadId":"49296","inReplyTo":"CACsJy8DdxmVd_jBG7Tp1e7k-BUc6R1=oWzPZ3z6xUZs2XU9rMA@mail.gmail.com","subject":"Re: Temporary git files for the gitdir created on a separate drive in workdir","fromName":"Hultqvist","fromEmail":"hultqvist@silentorbit.com","sentAt":"2018-09-09T15:30:57Z","receivedAt":"2018-09-09T15:28:40Z","isPatch":false,"sender":{"key":"hultqvist@silentorbit.com","avatar":"https://gravatar.com/avatar/7e58535ac43cad883a344e776736ad830963673ad20654f4d24f60920e0195e2?d=mp&s=160"},"body":"Since this thread started I haven't seen a single file mentioned being created,\nUsually they appear during work days when there is more activity.\nI've never seen the files created directly, only a larger amount of\nthem once in a while.\n\nI will run process monitor and get back once I find out more.\nDen lör 8 sep. 2018 kl 15:44 skrev Duy Nguyen :\n>\n> On Sat, Sep 8, 2018 at 3:09 PM Duy Nguyen wrote:\n> >\n> > On Sat, Sep 8, 2018 at 11:28 AM Hultqvist wrote:\n> > >\n> > > The bash commands are using a git and bash bundle that was installed\n> > > in parallel with gitextensions(a gui for git)\n> > >\n> > > G:\\Min enhet> set GIT_TRACE_SETUP=1\n> > > G:\\Min enhet> git st\n> > > 10:40:28.881927 trace.c:318 setup: git_dir:\n> > > C:/Users/hultqvist/Drive.git\n> > > 10:40:28.881927 trace.c:319 setup: git_common_dir:\n> > > C:/Users/hultqvist/Drive.git\n> > > 10:40:28.881927 trace.c:320 setup: worktree: G:/Min enhet\n> > > 10:40:28.881927 trace.c:321 setup: cwd: G:/Min enhet\n> > > 10:40:28.881927 trace.c:322 setup: prefix: (null)\n> > > 10:40:28.882930 chdir-notify.c:67 setup: chdir from 'G:/Min\n> > > enhet' to 'G:/Min enhet'\n> >\n> > Unfortunately this looks good. Whenever those files 'index',\n> > 'config'... are created, they should always be created in\n> > C:\\Users\\hultqvist\\Drive.git, not G:\\Min enhet, including their\n> > temporary versions. I don't know if there are any more changes on the\n> > windows fork that could affect this though, I only checked git.git.\n>\n> BTW do you notice these files showing up after any particular command\n> or they're always there after cloning? Perhaps some command got the\n> \".git\" directory discovery wrong and assumed $GIT_DIR=$GIT_WORK_TREE.\n> We have a much bigger problem then.\n> --\n> Duy\n"},{"id":"357779","messageId":"CAJmnt9aCfMGejO3B5ctZuCjJnE4K-dA1dwyz1uxoZwRBOJJqpg@mail.gmail.com","threadId":"49296","inReplyTo":"CAJmnt9YgFNjzZfOW38WbyC-9AK4rtS3P6_dKN6hS1X+y932uiA@mail.gmail.com","subject":"Re: Temporary git files for the gitdir created on a separate drive in workdir","fromName":"Hultqvist","fromEmail":"hultqvist@silentorbit.com","sentAt":"2018-09-10T10:29:44Z","receivedAt":"2018-09-10T10:27:26Z","isPatch":false,"sender":{"key":"hultqvist@silentorbit.com","avatar":"https://gravatar.com/avatar/7e58535ac43cad883a344e776736ad830963673ad20654f4d24f60920e0195e2?d=mp&s=160"},"body":"Sending again without HTML\n\nDen mån 10 sep. 2018 kl 12:28 skrev Hultqvist :\n>\n> First I need to correct my previous observations.\n>\n> Today there appeared new set of config files in the root.\n> I looked into a few of them and found that their content doesn't match that of the repo at \"G:/Min enhet\".\n> Instead separate files had content from separate git repos within the G drive.\n> These repos are not like the one we're discussed previously, they are completely within G: using a classical .git directory.\n>\n> I guess git is creating the temporary files as close as possible to the root, since \"G:\\\" can't be written to, only \"G:\\Min enhet\". and then copy them to the final destination which in this case is the same drive.\n> If so then we can't get away from the duplicate files, the duplicates themselves are the fault of Google Drive File Stream.\n> If git would create the config/master/index files within the .git dir to start with that would hide the clutter from the root but not prevent the duplicates.\n> Is this observation correct, that git creates temporary files closer to the root than inside the .git directory or a sub directory thereof?\n>\n> I haven't experiences any issues working with these repos themselves.\n> One risk could be that the final \"index\" is an older version than \"index (45)\".\n> I'm certain this haven't been the case since I push every commit to a remote repo that doesn't accept changes in history.\n>\n> I understand that master and index needs to be updated with regular use.\n> Does config need the same amount of updates?\n> I've considered that file to only change when I make explicit changes, not during regular use(push, commit, status).\n>\n> Thanks for the help looking into this for me.\n>\n>\n> Den sön 9 sep. 2018 kl 17:30 skrev Hultqvist :\n>>\n>> Since this thread started I haven't seen a single file mentioned being created,\n>> Usually they appear during work days when there is more activity.\n>> I've never seen the files created directly, only a larger amount of\n>> them once in a while.\n>>\n>> I will run process monitor and get back once I find out more.\n>> Den lör 8 sep. 2018 kl 15:44 skrev Duy Nguyen :\n>> >\n>> > On Sat, Sep 8, 2018 at 3:09 PM Duy Nguyen wrote:\n>> > >\n>> > > On Sat, Sep 8, 2018 at 11:28 AM Hultqvist wrote:\n>> > > >\n>> > > > The bash commands are using a git and bash bundle that was installed\n>> > > > in parallel with gitextensions(a gui for git)\n>> > > >\n>> > > > G:\\Min enhet> set GIT_TRACE_SETUP=1\n>> > > > G:\\Min enhet> git st\n>> > > > 10:40:28.881927 trace.c:318 setup: git_dir:\n>> > > > C:/Users/hultqvist/Drive.git\n>> > > > 10:40:28.881927 trace.c:319 setup: git_common_dir:\n>> > > > C:/Users/hultqvist/Drive.git\n>> > > > 10:40:28.881927 trace.c:320 setup: worktree: G:/Min enhet\n>> > > > 10:40:28.881927 trace.c:321 setup: cwd: G:/Min enhet\n>> > > > 10:40:28.881927 trace.c:322 setup: prefix: (null)\n>> > > > 10:40:28.882930 chdir-notify.c:67 setup: chdir from 'G:/Min\n>> > > > enhet' to 'G:/Min enhet'\n>> > >\n>> > > Unfortunately this looks good. Whenever those files 'index',\n>> > > 'config'... are created, they should always be created in\n>> > > C:\\Users\\hultqvist\\Drive.git, not G:\\Min enhet, including their\n>> > > temporary versions. I don't know if there are any more changes on the\n>> > > windows fork that could affect this though, I only checked git.git.\n>> >\n>> > BTW do you notice these files showing up after any particular command\n>> > or they're always there after cloning? Perhaps some command got the\n>> > \".git\" directory discovery wrong and assumed $GIT_DIR=$GIT_WORK_TREE.\n>> > We have a much bigger problem then.\n>> > --\n>> > Duy\n"},{"id":"357871","messageId":"CACsJy8DHTbpOmK=cthus3BZ4BtS0YbJS8=K72BCb7=fFx6TPBw@mail.gmail.com","threadId":"49296","inReplyTo":"CAJmnt9aCfMGejO3B5ctZuCjJnE4K-dA1dwyz1uxoZwRBOJJqpg@mail.gmail.com","subject":"Re: Temporary git files for the gitdir created on a separate drive in workdir","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2018-09-11T16:09:47Z","receivedAt":"2018-09-11T16:10:16Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Mon, Sep 10, 2018 at 12:27 PM Hultqvist wrote:\n>\n> Sending again without HTML\n>\n> Den mån 10 sep. 2018 kl 12:28 skrev Hultqvist :\n> >\n> > First I need to correct my previous observations.\n\nPlease don't top-post.\n\n> > Today there appeared new set of config files in the root.\n> > I looked into a few of them and found that their content doesn't match that of the repo at \"G:/Min enhet\".\n> > Instead separate files had content from separate git repos within the G drive.\n> > These repos are not like the one we're discussed previously, they are completely within G: using a classical .git directory.\n> >\n> > I guess git is creating the temporary files as close as possible to the root, since \"G:\\\" can't be written to, only \"G:\\Min enhet\". and then copy them to the final destination which in this case is the same drive.\n\nNo. Those files should always be created inside the \".git\" directory,\nwherever it is. Failing to creating a file in there is usually a\nserious error and the command will abort. Unless there is a bug\nlurking around of course, but I can't nail it down with just code\naudit.\n\nSince the content of those files does not look like from \"G:\\Min\nenhet\" repo, do you know which repo they belong to (and where those\nrepo and worktree are)? I ask because if these are submodules of\n\"G:\\Min enhet\" for example, then we need to head another direction. Or\nif they are completely unrelated to \"G:\\Min enhet\", oh boy...\n-- \nDuy\n"}]}