threads / patch / 6007

patchUse column indexes in git-cvsserver where necessary.

Subject: [PATCH] Use column indexes in git-cvsserver where necessary.

## tl;dr

2 messages between Oct 23, 2006 and Oct 23, 2006. Diffs are folded; open one to read it.

replies: 1people: 2as markdown or json

Shawn Pearce· Oct 23, 2006, 05:09 UTC · lore

Tonight I found a git-cvsserver instance spending a lot of time in disk IO while trying to process operations against a Git repository with >30,000 objects contained in it.

Blowing away my SQLLite database and rebuilding all tables with indexes on the attributes that git-cvsserver frequently runs queries against seems to have resolved the issue quite nicely.

Since the indexes shouldn't hurt performance on small repositories and always helps on larger repositories we should just always create them when creating the revision storage tables.

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
---
 git-cvsserver.perl |   14 +++++++++++++-
 1 files changed, 13 insertions(+), 1 deletions(-)

Show changes to git-cvsserver.perl +13 −1

diff --git a/git-cvsserver.perl b/git-cvsserver.perl
index 08ad831..8817f8b 100755
--- a/git-cvsserver.perl
+++ b/git-cvsserver.perl
@@ -2118,9 +2118,17 @@ sub new
                 mode       TEXT NOT NULL
             )
         ");
+        $self->{dbh}->do("
+            CREATE INDEX revision_ix1
+            ON revision (name,revision)
+        ");
+        $self->{dbh}->do("
+            CREATE INDEX revision_ix2
+            ON revision (name,commithash)
+        ");
     }
 
-    # Construct the revision table if required
+    # Construct the head table if required
     unless ( $self->{tables}{head} )
     {
         $self->{dbh}->do("
@@ -2134,6 +2142,10 @@ sub new
                 mode       TEXT NOT NULL
             )
         ");
+        $self->{dbh}->do("
+            CREATE INDEX head_ix1
+            ON head (name)
+        ");
     }
 
     # Construct the properties table if required

-- 
1.4.3.g4e51

Martin Langhoff (CatalystIT)· Oct 23, 2006, 06:02 UTC · re: Shawn Pearce · lore

Re: [PATCH] Use column indexes in git-cvsserver where necessary.

Shawn Pearce wrote:

Show 11 quoted lines

> Tonight I found a git-cvsserver instance spending a lot of time in
> disk IO while trying to process operations against a Git repository
> with >30,000 objects contained in it.
> 
> Blowing away my SQLLite database and rebuilding all tables with
> indexes on the attributes that git-cvsserver frequently runs queries
> against seems to have resolved the issue quite nicely.
> 
> Since the indexes shouldn't hurt performance on small repositories
> and always helps on larger repositories we should just always create
> them when creating the revision storage tables.

 >
 > Signed-off-by: Shawn O. Pearce <spearce@spearce.org>

Ack.

I am thinking we need a lightweight schema versioning mechanism to decide whether the DB schema needs changes such as this. Too much work for this though. ;-)

We have a simple one in Moodle I could port when the time comes.

cheers,

martin

-- 
-----------------------------------------------------------------------
Martin @ Catalyst .Net .NZ  Ltd, PO Box 11-053, Manners St,  Wellington
WEB: http://catalyst.net.nz/           PHYS: Level 2, 150-154 Willis St
OFFICE: +64(4)916-7224                              MOB: +64(21)364-017
       Make things as simple as possible, but no simpler - Einstein
-----------------------------------------------------------------------