{"thread":{"id":"61769","subject":"Issue : Writing commits into the git repository takes longer than expected","startedAt":"2024-07-12T04:10:48Z","lastAt":"2024-07-12T11:35:58Z","messageCount":2,"participants":["Arpit Gupta","rsbecker@nexbridge.com"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"498562","messageId":"DB9P190MB15008877ED559C3ED70050E3A7A62@DB9P190MB1500.EURP190.PROD.OUTLOOK.COM","threadId":"61769","inReplyTo":null,"subject":"Issue : Writing commits into the git repository takes longer than expected","fromName":"Arpit Gupta","fromEmail":"argupta@axway.com","sentAt":"2024-07-12T04:10:44Z","receivedAt":"2024-07-12T04:10:48Z","isPatch":false,"sender":{"key":"argupta@axway.com","avatar":null},"body":"Hi,\n\nWe are maintaining the different versions of data in git repository using jgit maven library. So, a commit is done on the repository containing properties such as author name, date and time, action, and the file path.\nThe file path refers the xml file which contains the action performed and is stored inside the repository.\n\nWe have a job running every 5 minutes that commits the information onto the repository and the XML file content is over-written every time. Usually, the commits and writing of XML file takes around 4-5 seconds but sometimes the time while committing as well as writing the data increases which also increase the overall CPU utilization of the machine. This behavior is inconsistent with respect to the process and occurs randomly but during this behavior, there is a time when the CPU utilization becomes high that all other running processes hangs up which demands the restart of the server.\n\nFurthermore,\n>> The XML files that are being written as content are multi-line. There are 2 tags present in the file and each tag are on their own line (one tag being the child of the other). The file size isn't large. It is hardly 2-3kb. Below is the sample structure of the XML file being added as a part of content:\n\n<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?> <ServiceName type=ServiceType> <property>PropertyValue</property>\n</ServiceName>\n\n>> There are no virus scans running in the repository. Also, the git LFS isn't involved in this scenario.\n>> There might be a case as when the commit time starts increasing (initially from 4-5s to 30s to 1min to 6-7min) and during that time another commit call also starts as there is a scheduler of 5 minutes which triggers this action. But this will only cause a certain amount of delay and it shouldn't be the factor to increase the CPU Utilization.\nAlso, the machine memory size is 32GB and the machine type is /dev/nvme2n1\n\nThe commit time starts increasing from 4-5s and goes up to 6-7mins, what could be the trigger for the commit to increase from 4-5s to 1min and so on in this scenario since before that there can't be any parallel commits ongoing onto the repository? Also, as I mentioned before, this issue is totally inconsistent.\nLet me know in case any other information is required.\n\nCould you please suggest which areas should we look for while identifying the cause of this issue? Also, does frequent commit of the content onto repository can trigger this issue? \nIn your view, what might be the trigger of this issue and how we can proceed to resolve it?\n\nThanks & Regards,\nArpit Gupta\n"},{"id":"498575","messageId":"011301dad44f$a80fbec0$f82f3c40$@nexbridge.com","threadId":"61769","inReplyTo":"DB9P190MB15008877ED559C3ED70050E3A7A62@DB9P190MB1500.EURP190.PROD.OUTLOOK.COM","subject":"RE: Issue : Writing commits into the git repository takes longer than expected","fromName":"","fromEmail":"rsbecker@nexbridge.com","sentAt":"2024-07-12T11:35:48Z","receivedAt":"2024-07-12T11:35:58Z","isPatch":false,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On Friday, July 12, 2024 12:11 AM, Arpit Gupta wrote:\n>We are maintaining the different versions of data in git repository using\njgit maven\n>library. So, a commit is done on the repository containing properties such\nas author\n>name, date and time, action, and the file path.\n>The file path refers the xml file which contains the action performed and\nis stored\n>inside the repository.\n\nCan you confirm you are using JGit rather than Git core commands? If this is\na JGit situation, you may be hitting Java garbage collection or there may be\na git garbage collection happening. If this is JGit, please raise this with\nthe JGit team rather than here.\n\n>We have a job running every 5 minutes that commits the information onto the\n>repository and the XML file content is over-written every time. Usually,\nthe commits\n>and writing of XML file takes around 4-5 seconds but sometimes the time\nwhile\n>committing as well as writing the data increases which also increase the\noverall CPU\n>utilization of the machine. This behavior is inconsistent with respect to\nthe process\n>and occurs randomly but during this behavior, there is a time when the CPU\n>utilization becomes high that all other running processes hangs up which\ndemands\n>the restart of the server.\n>\n>Furthermore,\n>>> The XML files that are being written as content are multi-line. There\nare 2 tags\n>present in the file and each tag are on their own line (one tag being the\nchild of the\n>other). The file size isn't large. It is hardly 2-3kb. Below is the sample\nstructure of\n>the XML file being added as a part of content:\n>\n><?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?> <ServiceName\n>type=ServiceType> <property>PropertyValue</property>\n></ServiceName>\n>\n>>> There are no virus scans running in the repository. Also, the git LFS\nisn't involved\n>in this scenario.\n>>> There might be a case as when the commit time starts increasing\n(initially from 4-\n>5s to 30s to 1min to 6-7min) and during that time another commit call also\nstarts as\n>there is a scheduler of 5 minutes which triggers this action. But this will\nonly cause a\n>certain amount of delay and it shouldn't be the factor to increase the CPU\n>Utilization.\n\nThere is a lock file (.git/index.lock) that acts as a semaphore to prevent\ntwo processes from writing to the repo at once. If your scheduler starts a\nlog-running job that interacts with the repo, this may cause essentially a\nlock wait. Check how long it is around during your delay.\n\n>Also, the machine memory size is 32GB and the machine type is /dev/nvme2n1\n>\n>The commit time starts increasing from 4-5s and goes up to 6-7mins, what\ncould be\n>the trigger for the commit to increase from 4-5s to 1min and so on in this\nscenario\n>since before that there can't be any parallel commits ongoing onto the\nrepository?\n>Also, as I mentioned before, this issue is totally inconsistent.\n>Let me know in case any other information is required.\n>\n>Could you please suggest which areas should we look for while identifying\nthe cause\n>of this issue? Also, does frequent commit of the content onto repository\ncan trigger\n>this issue?\n>In your view, what might be the trigger of this issue and how we can\nproceed to\n>resolve it?\n\n"}]}