threads / discuss / 11960

Adding gitweb.owner, last shot

Subject: Adding gitweb.owner, last shot

## tl;dr

12 messages between Feb 8, 2008 and Feb 8, 2008.

replies: 11people: 3as markdown or json

Bruno Ribas· Feb 8, 2008, 04:41 UTC · lore

After some study about the insertion of a new repository configuration, which sets repository owner, it was decided by the group that creating another small file inside .git/ is not a good idea. So I started to bench the viability to add just the gitweb.owner configuration, using gitweb.owner together with gitweb.description there is no major performance downgrade compared to $projects_list , as seen below:

8<------- These times i got with a 1000projects running 2 dd to generate disk IO. Here comes the resultm NO projects_list projects_list 16m30s69 15m10s74 default gitweb, using FS's owner 16m07s40 15m24s34 patched to get gitweb.owner 16m37s76 15m59s32 same above, but without gitweb.owner

Now results for a 1000projects on an idle machine. NO projects_list projects_list 1m19s08 1m09s55 default gitweb, using FS's owner 1m17s58 1m09s55 patched to get gitweb.owner 1m18s49 1m08s96 same above, but without gitweb.owner 8<-------

The idea of creating only the gitweb.owner can be a case of study to centralize all gitweb repository configuration in one file. Maybe even change the way $projects_list is formated, as gitweb.cgi needs to check repository configuration to get repository description to generate projecT_list page, we could just list repository's directories.

Bruno Ribas· Feb 8, 2008, 04:41 UTC · re: Bruno Ribas · lore

[PATCH] gitweb: Use the config file to set repository owner's name.

From: Git Managment for C3SL <git@git.c3sl.ufpr.br>

Now gitweb checks if gitweb.owner exists before trying to get filesystem's owner.

Allow to use configuration variable gitweb.owner set the repository owner, it checks the gitweb.owner, if not set it uses filesystem directory's owner.

Useful when we don't want to maintain project list file, and all repository directories have to have the same owner (for example when the same SSH account is shared for all projects, using ssh_acl to control access instead).

Signed-off-by: Git Managment for C3SL <git@git.c3sl.ufpr.br>
---
 gitweb/gitweb.perl |    7 ++++++-
 1 files changed, 6 insertions(+), 1 deletions(-)
diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index 8ef2735..e8a43b7 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -1767,7 +1767,12 @@ sub git_get_project_owner {
 	if (exists $gitweb_project_owner->{$project}) {
 		$owner = $gitweb_project_owner->{$project};
 	}
-	if (!defined $owner) {
+
+	if (!defined $owner){
+		$owner = git_get_project_config('owner');
+	}
+
+	if (!$owner) {
 		$owner = get_file_owner("$projectroot/$project");
 	}
 

-- 
1.5.4.34.g053d9-dirty
Bruno Ribas· Feb 8, 2008, 04:41 UTC · re: Bruno Ribas · lore

[PATCH] gitweb: Update gitweb/README to include the new per-repository gitweb.owner

From: Git Managment for C3SL <git@git.c3sl.ufpr.br>
Signed-off-by: Git Managment for C3SL <git@git.c3sl.ufpr.br>
---
 gitweb/README |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)
diff --git a/gitweb/README b/gitweb/README
index 4c8bedf..2163071 100644
--- a/gitweb/README
+++ b/gitweb/README
@@ -233,6 +233,10 @@ You can use the following files in repository:
    Displayed in the project summary page. You can use multiple-valued
    gitweb.url repository configuration variable for that, but the file
    takes precendence.
+ * gitweb.owner
+   You can use the gitweb.owner repository configuration variable to set
+   repository's owner. It is displayed in the project list and summary
+   page. If it's not set, filesystem directory's owner is used.
  * various gitweb.* config variables (in config)
    Read description of %feature hash for detailed list, and some
    descriptions.

-- 
1.5.4.34.g053d9-dirty
Jakub Narebski· Feb 8, 2008, 10:55 UTC · re: Bruno Ribas · lore

Re: [PATCH] gitweb: Use the config file to set repository owner's name.

Bruno Ribas <ribas@c3sl.ufpr.br> writes:
> Allow to use configuration variable gitweb.owner set the repository owner,
> it checks the gitweb.owner, if not set it uses filesystem directory's owner.
Show 18 quoted lines
> diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
> index 8ef2735..e8a43b7 100755
> --- a/gitweb/gitweb.perl
> +++ b/gitweb/gitweb.perl
> @@ -1767,7 +1767,12 @@ sub git_get_project_owner {
>  	if (exists $gitweb_project_owner->{$project}) {
>  		$owner = $gitweb_project_owner->{$project};
>  	}
> -	if (!defined $owner) {
> +
> +	if (!defined $owner){
> +		$owner = git_get_project_config('owner');
> +	}
> +
> +	if (!$owner) {
>  		$owner = get_file_owner("$projectroot/$project");
>  	}
>  
First, I think the empty lines added are not needed.

Second, git_get_project_config() subroutine _REQUIRES_ for $git_dir to be set. So you have to set $git_dir before checking repo config; then you can reuse $git_dir in checking file owner.

-- 
Jakub Narebski
Poland
ShadeHawk on #git
Bruno Cesar Ribas· Feb 8, 2008, 13:53 UTC · re: Jakub Narebski · lore

Re: [PATCH] gitweb: Use the config file to set repository owner's name.

On Fri, Feb 08, 2008 at 02:55:33AM -0800, Jakub Narebski wrote:
> Bruno Ribas <ribas@c3sl.ufpr.br> writes:
> 
><snip> 
> First, I think the empty lines added are not needed.

I made those empty lines because original code had same empty lines above, I just let it to have same pattern, but I can remove. Should I remove?! I'll resend without it, and with $git_dir set.

Show 9 quoted lines
> 
> Second, git_get_project_config() subroutine _REQUIRES_ for $git_dir to
> be set. So you have to set $git_dir before checking repo config; then
> you can reuse $git_dir in checking file owner.
> 
> -- 
> Jakub Narebski
> Poland
> ShadeHawk on #git
-- 
Bruno Ribas - ribas@c3sl.ufpr.br
http://web.inf.ufpr.br/ribas
C3SL: http://www.c3sl.ufpr.br
Bruno Cesar Ribas· Feb 8, 2008, 14:30 UTC · re: Bruno Cesar Ribas · lore

Re: [PATCH] gitweb: Use the config file to set repository owner's name.

On Fri, Feb 08, 2008 at 11:53:27AM -0200, Bruno Cesar Ribas wrote:
Show 21 quoted lines
> On Fri, Feb 08, 2008 at 02:55:33AM -0800, Jakub Narebski wrote:
> > Bruno Ribas <ribas@c3sl.ufpr.br> writes:
> > 
> diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
> index 8ef2735..e8a43b7 100755
> --- a/gitweb/gitweb.perl
> +++ b/gitweb/gitweb.perl
> @@ -1767,7 +1767,12 @@ sub git_get_project_owner {
>       if (exists $gitweb_project_owner->{$project}) {
>               $owner = $gitweb_project_owner->{$project};
>       }
> -     if (!defined $owner) {
> +
> +     if (!defined $owner){
> +             $owner = git_get_project_config('owner');
> +     }
> +
> +     if (!$owner) {
>               $owner = get_file_owner("$projectroot/$project");
>       }
>

I that last 3lines should be inside the block that we call git_get_project_config, don't you think?

Show 23 quoted lines
> > First, I think the empty lines added are not needed.
> 
> I made those empty lines because original code had same empty lines above, I
> just let it to have same pattern, but I can remove. Should I remove?! I'll
> resend without it, and with $git_dir set.
> > 
> > Second, git_get_project_config() subroutine _REQUIRES_ for $git_dir to
> > be set. So you have to set $git_dir before checking repo config; then
> > you can reuse $git_dir in checking file owner.
> > 
> > -- 
> > Jakub Narebski
> > Poland
> > ShadeHawk on #git
> 
> -- 
> Bruno Ribas - ribas@c3sl.ufpr.br
> http://web.inf.ufpr.br/ribas
> C3SL: http://www.c3sl.ufpr.br 
> -
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
-- 
Bruno Ribas - ribas@c3sl.ufpr.br
http://web.inf.ufpr.br/ribas
C3SL: http://www.c3sl.ufpr.br
Jakub Narebski· Feb 8, 2008, 15:33 UTC · re: Bruno Cesar Ribas · lore

Re: [PATCH] gitweb: Use the config file to set repository owner's name.

I have joined the two emails to reply only once.
On Fri, 8 Feb 2008, Bruno Cesar Ribas wrote:
Show 20 quoted lines
> On Fri, Feb 08, 2008 at 02:55:33AM -0800, Jakub Narebski wrote:
>> Bruno Ribas <ribas@c3sl.ufpr.br> writes:
>>> 
>>> diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
>>> index 8ef2735..e8a43b7 100755
>>> --- a/gitweb/gitweb.perl
>>> +++ b/gitweb/gitweb.perl
>>> @@ -1767,7 +1767,12 @@ sub git_get_project_owner {
>>>       if (exists $gitweb_project_owner->{$project}) {
>>>               $owner = $gitweb_project_owner->{$project};
>>>       }
>>> -     if (!defined $owner) {
>>> +
>>> +     if (!defined $owner){
>>> +             $owner = git_get_project_config('owner');
>>> +     }
>>> +
>>> +     if (!$owner) {
>>>               $owner = get_file_owner("$projectroot/$project");
>>>       }

Another comment: why did you change from checking of "!defined $owner" to checking "!$owner"? git_get_project_config('owner') returns undef if gitweb.owner is not defined. With checking for defined we can avoid false positives of owner being "0" (in practice I think this does not matter) or "" (this could happen if somebody doesn't want for project to have owner shown).

Show 5 quoted lines
>> First, I think the empty lines added are not needed.
> 
> I made those empty lines because original code had same empty lines
> above, I just let it to have same pattern, but I can remove. Should I
> remove?!

The idea was for empty lines to separate blocks of code: variables declaration, initialization, finding an owner, and return value. So I think that empty lines are not needed here. There were no empty lines between check for owner in the structure populated by git_get_project_list_from_file() and checking filesystem stat for project directory owner.

By the way, the git_get_project_list_from_file() interface is a bit strange...

> I that last 3lines should be inside the block that we call
> git_get_project_config, don't you think?

No. I think using "if (!defined $foo) { maybe define foo }..." sequence is a good flow.

> I'll resend [...] with $git_dir set.
And with signoff corrected, I assume?

Please try to check if the code works with and without gitweb.owner set before sending new version of the patch...

-- 
Jakub Narebski
Poland
Bruno Cesar Ribas· Feb 8, 2008, 16:07 UTC · re: Jakub Narebski · lore

Re: [PATCH] gitweb: Use the config file to set repository owner's name.

On Fri, Feb 08, 2008 at 04:33:54PM +0100, Jakub Narebski wrote:
Show 30 quoted lines
> I have joined the two emails to reply only once.
> 
> On Fri, 8 Feb 2008, Bruno Cesar Ribas wrote:
> > On Fri, Feb 08, 2008 at 02:55:33AM -0800, Jakub Narebski wrote:
> >> Bruno Ribas <ribas@c3sl.ufpr.br> writes:
> >>> 
> >>> diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
> >>> index 8ef2735..e8a43b7 100755
> >>> --- a/gitweb/gitweb.perl
> >>> +++ b/gitweb/gitweb.perl
> >>> @@ -1767,7 +1767,12 @@ sub git_get_project_owner {
> >>>       if (exists $gitweb_project_owner->{$project}) {
> >>>               $owner = $gitweb_project_owner->{$project};
> >>>       }
> >>> -     if (!defined $owner) {
> >>> +
> >>> +     if (!defined $owner){
> >>> +             $owner = git_get_project_config('owner');
> >>> +     }
> >>> +
> >>> +     if (!$owner) {
> >>>               $owner = get_file_owner("$projectroot/$project");
> >>>       }
> 
> Another comment: why did you change from checking of "!defined $owner"
> to checking "!$owner"? git_get_project_config('owner') returns undef
> if gitweb.owner is not defined. With checking for defined we can avoid
> false positives of owner being "0" (in practice I think this does not
> matter) or "" (this could happen if somebody doesn't want for project
> to have owner shown).

When I tested it returned empty, but i found out it was lack of NAME for the user i was running tests.

Show 5 quoted lines
> 
> 
> > I'll resend [...] with $git_dir set.
> 
> And with signoff corrected, I assume?
of course.
> 
> Please try to check if the code works with and without gitweb.owner set 
> before sending new version of the patch...
I always do that.
> -- 
> Jakub Narebski
> Poland
-- 
Bruno Ribas - ribas@c3sl.ufpr.br
http://web.inf.ufpr.br/ribas
C3SL: http://www.c3sl.ufpr.br
Junio C Hamano· Feb 8, 2008, 07:38 UTC · re: Bruno Ribas · lore

Re: Adding gitweb.owner, last shot

Bruno Ribas <ribas@c3sl.ufpr.br> writes:
Show 17 quoted lines
> ... there is no major performance downgrade
> compared to $projects_list , as seen below:
>
> 8<-------
> These times i got with a 1000projects running 2 dd to generate disk IO.
> Here comes the resultm
> NO projects_list  projects_list
> 16m30s69          15m10s74       default gitweb, using FS's owner
> 16m07s40          15m24s34       patched to get gitweb.owner
> 16m37s76          15m59s32       same above, but without gitweb.owner
>
> Now results for a 1000projects on an idle machine.
> NO projects_list  projects_list
> 1m19s08           1m09s55       default gitweb, using FS's owner
> 1m17s58           1m09s55       patched to get gitweb.owner
> 1m18s49           1m08s96       same above, but without gitweb.owner
> 8<-------

Large installations would maintain the project_list in the flat file format for performance reasons anyway. Benchmarking under a condition that yields unreasonably long response time is somewhat meaningless, I am afraid. Who sane would wait for 15 minutes for project list to come up?

So I think your patch makes sense. It would not help nor hurt large installations, and would help smaller installations that do not care much about performance but are more interested in the convenience of not having to worry about maintaining the project_list.

As the act of signing off patches is a legal statement, I'd prefer real person's name, not "Git Managment for C3SL", in the messages to be applied. The change that adds the feature, and the documentation update to describe that new feature, should be in the same single patch for a small change like this.

Bruno Cesar Ribas· Feb 8, 2008, 13:49 UTC · re: Junio C Hamano · lore

Re: Adding gitweb.owner, last shot

On Thu, Feb 07, 2008 at 11:38:44PM -0800, Junio C Hamano wrote:
Show 25 quoted lines
> Bruno Ribas <ribas@c3sl.ufpr.br> writes:
> 
> > ... there is no major performance downgrade
> > compared to $projects_list , as seen below:
> >
> > 8<-------
> > These times i got with a 1000projects running 2 dd to generate disk IO.
> > Here comes the resultm
> > NO projects_list  projects_list
> > 16m30s69          15m10s74       default gitweb, using FS's owner
> > 16m07s40          15m24s34       patched to get gitweb.owner
> > 16m37s76          15m59s32       same above, but without gitweb.owner
> >
> > Now results for a 1000projects on an idle machine.
> > NO projects_list  projects_list
> > 1m19s08           1m09s55       default gitweb, using FS's owner
> > 1m17s58           1m09s55       patched to get gitweb.owner
> > 1m18s49           1m08s96       same above, but without gitweb.owner
> > 8<-------
> 
> Large installations would maintain the project_list in the flat
> file format for performance reasons anyway.  Benchmarking under
> a condition that yields unreasonably long response time is
> somewhat meaningless, I am afraid.  Who sane would wait for 15
> minutes for project list to come up?

But it got 15minutes with project_list to! The real problem is geting last change on repository under high IO.

Show 8 quoted lines
> 
><snip>
> 
> As the act of signing off patches is a legal statement, I'd
> prefer real person's name, not "Git Managment for C3SL", in the
> messages to be applied.  The change that adds the feature, and
> the documentation update to describe that new feature, should be
> in the same single patch for a small change like this.
I'll resend it !! my mistake of names =(
Show 8 quoted lines
> 
> 
> 
> 
> -
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
-- 
Bruno Ribas - ribas@c3sl.ufpr.br
http://web.inf.ufpr.br/ribas
C3SL: http://www.c3sl.ufpr.br
Jakub Narebski· Feb 8, 2008, 10:34 UTC · re: Bruno Ribas · lore

Re: Adding gitweb.owner, last shot

Bruno Ribas <ribas@c3sl.ufpr.br> writes:
Show 21 quoted lines
> After some study about the insertion of a new repository configuration,
> which sets repository owner, it was decided by the group that creating
> another small file inside .git/ is not a good idea. So I started to bench
> the viability to add just the gitweb.owner configuration, using gitweb.owner
> together with gitweb.description there is no major performance downgrade
> compared to $projects_list , as seen below:
> 
> 8<-------
> These times i got with a 1000projects running 2 dd to generate disk IO.
> Here comes the resultm
> NO projects_list  projects_list
> 16m30s69          15m10s74       default gitweb, using FS's owner
> 16m07s40          15m24s34       patched to get gitweb.owner
> 16m37s76          15m59s32       same above, but without gitweb.owner
> 
> Now results for a 1000projects on an idle machine.
> NO projects_list  projects_list
> 1m19s08           1m09s55       default gitweb, using FS's owner
> 1m17s58           1m09s55       patched to get gitweb.owner
> 1m18s49           1m08s96       same above, but without gitweb.owner
> 8<-------

It looks like there is almost no difference between using only FS owner, and reading also repository config using "git config -z -l"... Could anyone using gitweb on MS Windows or MacOS X, where fork is mich slower, check those figures?

Pasky, could you please try to benchmark this (well, at least without
gitweb.owner set) on a _real_ large set of repositories?
Show 5 quoted lines
> The idea of creating only the gitweb.owner can be a case of study to
> centralize all gitweb repository configuration in one file. Maybe even
> change the way $projects_list is formated, as gitweb.cgi needs to check
> repository configuration to get repository description to generate
> project_list page, we could just list repository's directories.

I was thinking about git-config-like format (but simplified to make it easy to parse it in Perl, just like git-cvsserver configuration), in the form of

[gitweb "<repository path, relative to $projectroot>"]
        description = <project description>
        url = <first URL>
        url = <seconf URL>
        owner = <repository owner>

The problem with parsing lies (among others) in the in-line comments, novalue keys, and key after section, I think...

-- 
Jakub Narebski
Poland
ShadeHawk on #git
Bruno Cesar Ribas· Feb 8, 2008, 13:51 UTC · re: Jakub Narebski · lore

Re: Adding gitweb.owner, last shot

On Fri, Feb 08, 2008 at 02:34:55AM -0800, Jakub Narebski wrote:
Show 5 quoted lines
> Bruno Ribas <ribas@c3sl.ufpr.br> writes:
> 
><snip> 
> Pasky, could you please try to benchmark this (well, at least without
> gitweb.owner set) on a _real_ large set of repositories?

And I'd like to know what this machine runs! If it is git dedicated, or shares with something else.

If it is git dedicated, it makes time better as disk IO will not be severe.
Show 24 quoted lines
>  
> > The idea of creating only the gitweb.owner can be a case of study to
> > centralize all gitweb repository configuration in one file. Maybe even
> > change the way $projects_list is formated, as gitweb.cgi needs to check
> > repository configuration to get repository description to generate
> > project_list page, we could just list repository's directories.
> 
> I was thinking about git-config-like format (but simplified to make it
> easy to parse it in Perl, just like git-cvsserver configuration), in
> the form of
> 
> [gitweb "<repository path, relative to $projectroot>"]
>         description = <project description>
>         url = <first URL>
>         url = <seconf URL>
>         owner = <repository owner>
> 
> The problem with parsing lies (among others) in the in-line comments,
> novalue keys, and key after section, I think...
> 
> -- 
> Jakub Narebski
> Poland
> ShadeHawk on #git
-- 
Bruno Ribas - ribas@c3sl.ufpr.br
http://web.inf.ufpr.br/ribas
C3SL: http://www.c3sl.ufpr.br

← back to recent threads