# Seeing high CPU usage during git-upload-pack 4 messages from 2012-05-09 to 2012-05-10. Participants: David Ebbo, Nguyen Thai Ngoc Duy. Thread: https://gitlist.dev/t/30481 ## David Ebbo, 2012-05-09 07:01 Subject: Seeing high CPU usage during git-upload-pack Message-ID: URL: https://gitlist.dev/e/CAPeUw3EsFCxki6obAAeVBb0g4MKNWBteL3VueG5djbajkFiv3A%40mail.gmail.com ``` We’re using git-upload-pack in a simple git server. When the repo is on a slow share, we’re noticing that git-upload-pack is hugging the CPU during the whole pack-objects operation. More details: - git-upload-pack gets launched (with params git-upload-pack --stateless-rpc //path/to/repo) - it itself launches a git process (with params pack-objects --revs --all --stdout --progress --delta-base-offset). This git process is the one that does all the disk I/O, but it's not the one using up the CPU time. - git-upload-pack appears to be waiting for the git process, and is at high CPU during that time, suggesting that it's doing some kind of busy wait. To further test that, I stopped the git process in the debugger, preventing it from making progress. At that point, git-upload-pack pegs the CPU forever. Would someone familiar with the sources be able to comment on this busy wait behavior, and on whether it could conceivably be done in a way that doesn’t use up some much CPU? Thanks, David ``` ## Nguyen Thai Ngoc Duy, 2012-05-09 10:37 Subject: Re: Seeing high CPU usage during git-upload-pack Message-ID: URL: https://gitlist.dev/e/CACsJy8DMm0ZNiQYQ1CvOfDJu5ogP42Y06792iA_xyzeHDGLJaw%40mail.gmail.com In-Reply-To: ``` On Wed, May 9, 2012 at 2:01 PM, David Ebbo wrote: > We’re using git-upload-pack in a simple git server. When the repo is > on a slow share, we’re noticing that git-upload-pack is hugging the > CPU during the whole pack-objects operation. What OS is it run on? What git version? > More details: > > - git-upload-pack gets launched (with params git-upload-pack > --stateless-rpc //path/to/repo) > - it itself launches a git process (with params pack-objects --revs > --all --stdout --progress --delta-base-offset). This git process is > the one that does all the disk I/O, but it's not the one using up the > CPU time. > - git-upload-pack appears to be waiting for the git process, and is at > high CPU during that time, suggesting that it's doing some kind of > busy wait. To further test that, I stopped the git process in the > debugger, preventing it from making progress. At that point, > git-upload-pack pegs the CPU forever. How about stopping upload-pack and see where it stops? There's a main loop in upload-pack.c, create_pack_file() that is only active when data comes. I looked but failed to see how it becomes a busywait loop. Maybe you can put some debugging there. Setting env var GIT_TRACE_PACKET and GIT_DEBUG_SEND_PACK before running upload-pack might also help. > Would someone familiar with the sources be able to comment on this > busy wait behavior, and on whether it could conceivably be done in a > way that doesn’t use up some much CPU? -- Duy ``` ## Nguyen Thai Ngoc Duy, 2012-05-09 15:11 Subject: Re: Seeing high CPU usage during git-upload-pack Message-ID: URL: https://gitlist.dev/e/CACsJy8CbaM6UNCT3ZP0%3DVZE9GhH2eJpgKe-dnGU5ubn_4gS%3DPw%40mail.gmail.com In-Reply-To: ``` Hi, I don't see anything private in the mail, so I CC git@vger again. It's a good thing to keep git@vger in the loop because there will be more (experienced) people to help you out. On Wed, May 9, 2012 at 9:37 PM, David Ebbo wrote: > Thanks Duy for your looking at this. > > On Wed, May 9, 2012 at 3:37 AM, Nguyen Thai Ngoc Duy wrote: >> >> What OS is it run on? What git version? >> > > This is using 1.7.9 on Windows (msysgit). Ah... msysgit reimplements poll() function, in compat/win32/poll.c. If it goes rouge, you could get an active loop. But this is just my suspicion. I don't know anything about msysgit's poll implementation so I can be completely wrong here. >> How about stopping upload-pack and see where it stops? There's a main >> loop in upload-pack.c, create_pack_file() that is only active when >> data comes. I looked but failed to see how it becomes a busywait loop. >> Maybe you can put some debugging there. >> >> Setting env var GIT_TRACE_PACKET and GIT_DEBUG_SEND_PACK before >> running upload-pack might also help. >> > > I'll try those env vars, and will debug as needed (I haven't done that > yet, so I'll need to get set up ) > > Thanks for the pointers! -- Duy ``` ## David Ebbo, 2012-05-10 20:41 Subject: Re: Seeing high CPU usage during git-upload-pack Message-ID: URL: https://gitlist.dev/e/CAPeUw3HyCz921upWA3YBtJXTOKTcdP5tDxUv3drR7%3DaovJ07_g%40mail.gmail.com In-Reply-To: ``` On Wed, May 9, 2012 at 8:11 AM, Nguyen Thai Ngoc Duy wrote: > > Ah... msysgit reimplements poll() function, in compat/win32/poll.c. If > it goes rouge, you could get an active loop. But this is just my > suspicion. I don't know anything about msysgit's poll implementation > so I can be completely wrong here. > Sorry for the delayed replay, and thanks for pointing out that this is msysgit specific and not general to all platform. I guess the next step is to discuss it on their forum (msysgit@googlegroups.com). thanks! David ```