{"thread":{"id":"62143","subject":"The maintenance tasks will never run if maintenance.lock is accidentally not deleted","startedAt":"2024-09-19T12:41:01Z","lastAt":"2024-09-25T13:51:01Z","messageCount":4,"participants":["刘钟博","Derrick Stolee"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"503058","messageId":"CAN477tHJnVnOKfUsG5G9QAVdzYvmUuC8A8Vxt8mtHB23fd=hAQ@mail.gmail.com","threadId":"62143","inReplyTo":null,"subject":"The maintenance tasks will never run if maintenance.lock is accidentally not deleted","fromName":"刘钟博","fromEmail":"liuzhongbo.gg@gmail.com","sentAt":"2024-09-19T12:40:46Z","receivedAt":"2024-09-19T12:41:01Z","isPatch":false,"sender":{"key":"liuzhongbo.gg@gmail.com","avatar":null},"body":"In my work, I found that the prefetch task of maintenance often\nfailed, causing the fetch command to take a long time to execute in\nmonorepo.\n\nAfter investigation, it was found that the maintenance.lock file was\nnot deleted correctly for various reasons, resulting in the inability\nto trigger subsequent maintenance tasks.\n\nThe maintenance task needs to be executed in the background for a long\ntime. Therefore, due to various reasons, the maintenance.lock file is\neasily not deleted when the process exits abnormally.\nUnlike ref.lock, which will have an error prompt in the fetch command,\nmaintenance will only fail silently, which makes it difficult for most\npeople to find out where the problem is.\n\nSo is it recommended to add some mechanism to ensure that\nmaintenance.lock can be correctly restored when it is not deleted? For\nexample, add pid information to maintenance.lock, or add a lock\ntimeout mechanism.\n\nI'm not sure if I missed any information, but if this is feasible, I\nwould be happy to contribute such a patch.\n\nThanks.\nliuzhongbo\n"},{"id":"503059","messageId":"cce1d054-911e-407e-bc26-1c0bac4dd8e4@gmail.com","threadId":"62143","inReplyTo":"CAN477tHJnVnOKfUsG5G9QAVdzYvmUuC8A8Vxt8mtHB23fd=hAQ@mail.gmail.com","subject":"Re: The maintenance tasks will never run if maintenance.lock is accidentally not deleted","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2024-09-19T12:56:33Z","receivedAt":"2024-09-19T12:56:37Z","isPatch":false,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 9/19/24 8:40 AM, 刘钟博 wrote:\n> In my work, I found that the prefetch task of maintenance often\n> failed, causing the fetch command to take a long time to execute in\n> monorepo.\n> \n> After investigation, it was found that the maintenance.lock file was\n> not deleted correctly for various reasons, resulting in the inability\n> to trigger subsequent maintenance tasks.\n\nThis is unfortunately a common occurrence. It seems to be related to\nthe Git background processes being killed in such a way that does not\nallow the standard lock cleanup mechanism to kick in.\n\nAt least, I haven't been able to find a reason why Git would be\nfailing with something like a segfault which would also cause leftover\n.lock files.\n\n> So is it recommended to add some mechanism to ensure that\n> maintenance.lock can be correctly restored when it is not deleted? For\n> example, add pid information to maintenance.lock, or add a lock\n> timeout mechanism.\n\nI can speak from experience of previously having a lock timeout\nthat this could cause problems where maintenance processes start\nrunning on the same repo concurrently. The reason for this in the\npast was due to being blocked on credential manager prompts.\n\nI was vaguely remembering fixing that issue with credential prompts,\nbut then realized the change was only made to microsoft/git [1]. That\nsame change reverted the removal of \"stale\" .lock files.\n\nI should put this together for an upstream patch series, finally.\n\n[1] https://github.com/microsoft/git/pull/598\n\n> I'm not sure if I missed any information, but if this is feasible, I\n> would be happy to contribute such a patch.\nI will CC you on my submission for the credential changes, as a way\nto help introduce you to the code in this area.\n\nThanks,\n-Stolee\n\n"},{"id":"503245","messageId":"CAN477tFqDM64NsoXYKww7Xh7rNajMGn0DK062AjxDOmp+_7Lig@mail.gmail.com","threadId":"62143","inReplyTo":"cce1d054-911e-407e-bc26-1c0bac4dd8e4@gmail.com","subject":"Re: The maintenance tasks will never run if maintenance.lock is accidentally not deleted","fromName":"刘钟博","fromEmail":"liuzhongbo.gg@gmail.com","sentAt":"2024-09-23T10:26:02Z","receivedAt":"2024-09-23T10:26:14Z","isPatch":false,"sender":{"key":"liuzhongbo.gg@gmail.com","avatar":null},"body":"Thank you for your response.\nOn Thu, Sep 19, 2024 at 8:56 PM Derrick Stolee <stolee@gmail.com> wrote:\n> At least, I haven't been able to find a reason why Git would be\n> failing with something like a segfault which would also cause leftover\n> .lock files.\nYes, it is not necessarily a problem caused by git failure. I think it\nis a natural\nshortcoming of file existence lock, which cannot guarantee that the lock will be\nreleased when the process exits abnormally.\n\n> I can speak from experience of previously having a lock timeout\n> that this could cause problems where maintenance processes start\n> running on the same repo concurrently.\n> [1] https://github.com/microsoft/git/pull/598\nI read your commit and explored more. Perhaps the file locks provided by the\nsystems are a better choice, such as fcntl() on POSIX and LockFileEx()\non Windows.\nThey can be automatically released when the process exits abnormally.\nIf there are\nno objections, I'll give it a try and send a patch in a few days.\n\nThanks\nLiuzhongbo\n"},{"id":"503468","messageId":"8b6394ef-9ff6-4fb0-bfdb-3bf02b2f4bdb@gmail.com","threadId":"62143","inReplyTo":"CAN477tFqDM64NsoXYKww7Xh7rNajMGn0DK062AjxDOmp+_7Lig@mail.gmail.com","subject":"Re: The maintenance tasks will never run if maintenance.lock is accidentally not deleted","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2024-09-25T13:50:58Z","receivedAt":"2024-09-25T13:51:01Z","isPatch":false,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 9/23/24 6:26 AM, 刘钟博 wrote:\n > Thank you for your response.\n > On Thu, Sep 19, 2024 at 8:56 PM Derrick Stolee <stolee@gmail.com> wrote:\n >> At least, I haven't been able to find a reason why Git would be\n >> failing with something like a segfault which would also cause leftover\n >> .lock files.\n > Yes, it is not necessarily a problem caused by git failure. I think it\n > is a natural\n > shortcoming of file existence lock, which cannot guarantee that the lock will be\n > released when the process exits abnormally.\n >\n >> I can speak from experience of previously having a lock timeout\n >> that this could cause problems where maintenance processes start\n >> running on the same repo concurrently.\n >> [1] https://github.com/microsoft/git/pull/598\n > I read your commit and explored more. Perhaps the file locks provided by the\n > systems are a better choice, such as fcntl() on POSIX and LockFileEx()\n > on Windows.\n > They can be automatically released when the process exits abnormally.\n > If there are\n > no objections, I'll give it a try and send a patch in a few days.\n\nI'm curious to see how this would be implemented across platforms.\n\nI think the maintenance.lock file is a particular case where such a\nchange would be welcome, because we are not writing to it and then doing\na rename over the non-.lock version (like we do with many other files).\n\nFor that reason, you would be creating a new API construct, not\nreplacing the existing lockfile API.\n\nThanks,\n-Stolee\n\n"}]}