threads / patch / 19986

patchLimited git-gui to displaying 5000 new files

Subject: [PATCH] Limited git-gui to displaying 5000 new files

## tl;dr

12 messages between Jun 30, 2009 and Aug 12, 2009. Diffs are folded; open one to read it.

replies: 11people: 4as markdown or json

Dan Zwell· Jun 30, 2009, 20:37 UTC · lore

When there is a large number of new or modified files, "display_all_files" takes a long time, and git-gui appears to hang. Limit the display to 5000 files, by default. This number is configurable as gui.maxfilesdisplayed.

Show a warning if the list of files is truncated.

Signed-off-by: Dan Zwell <dzwell@zwell.net>
---
 git-gui.sh |   17 ++++++++++++++++-
 1 files changed, 16 insertions(+), 1 deletions(-)

Show changes to git-gui.sh +16 −1

diff --git a/git-gui.sh b/git-gui.sh
index 14b92ba..5a20923 100755
--- a/git-gui.sh
+++ b/git-gui.sh
@@ -745,6 +745,8 @@ set default_config(gui.newbranchtemplate) {}
 set default_config(gui.spellingdictionary) {}
 set default_config(gui.fontui) [font configure font_ui]
 set default_config(gui.fontdiff) [font configure font_diff]
+# TODO: this option should be added to the git-config documentation
+set default_config(gui.maxfilesdisplayed) 5000
 set font_descs {
 	{fontui   font_ui   {mc "Main Font"}}
 	{fontdiff font_diff {mc "Diff/Console Font"}}
@@ -1702,6 +1704,8 @@ proc display_all_files {} {
 	global ui_index ui_workdir
 	global file_states file_lists
 	global last_clicked
+	global files_warning
+	global default_config
 
 	$ui_index conf -state normal
 	$ui_workdir conf -state normal
@@ -1713,7 +1717,18 @@ proc display_all_files {} {
 	set file_lists($ui_index) [list]
 	set file_lists($ui_workdir) [list]
 
-	foreach path [lsort [array names file_states]] {
+	set to_display [lsort [array names file_states]]
+	set display_limit $default_config(gui.maxfilesdisplayed)
+	if {[llength $to_display] > $display_limit} {
+		if {![info exists files_warning] || !$files_warning} {
+			set warning "Displaying only $display_limit of "
+			append warning "[llength $to_display] files."
+			info_popup [mc $warning]
+			set files_warning 1
+		}
+		set to_display [lrange $to_display 0 [expr {$display_limit-1}]]
+	}
+	foreach path $to_display {
 		set s $file_states($path)
 		set m [lindex $s 0]
 		set icon_name [lindex $s 1]

-- 
1.6.3.3

Shawn O. Pearce· Aug 10, 2009, 15:38 UTC · re: Dan Zwell · lore

Re: [PATCH] Limited git-gui to displaying 5000 new files

Dan Zwell <dzwell@gmail.com> wrote:

Show 6 quoted lines

> When there is a large number of new or modified files,
> "display_all_files" takes a long time, and git-gui appears to
> hang. Limit the display to 5000 files, by default. This number
> is configurable as gui.maxfilesdisplayed.
>
> Show a warning if the list of files is truncated.

Show 7 quoted lines

> @@ -1713,7 +1717,18 @@ proc display_all_files {} {
> 	set file_lists($ui_index) [list]
> 	set file_lists($ui_workdir) [list]
>
> -	foreach path [lsort [array names file_states]] {
> +	set to_display [lsort [array names file_states]]
> +	set display_limit $default_config(gui.maxfilesdisplayed)

This should use [get_config gui.maxfilesdisplayed] so that the user can actually set this property in a configuration file and have git-gui honor it. Reading from $default_config means you are only looking at the hardcoded value you set in git-gui.sh.

> +	if {[llength $to_display] > $display_limit} {
> +		if {![info exists files_warning] || !$files_warning} {

Wouldn't it be easier to just set files_warning to 0 at the start of the script, so that you don't need to do this info exists test?

> +			set warning "Displaying only $display_limit of "
> +			append warning "[llength $to_display] files."
> +			info_popup [mc $warning]

This needs to be in the translated strings.

-- 
Shawn.

Alex Riesen· Aug 10, 2009, 17:06 UTC · re: Shawn O. Pearce · lore

Re: [PATCH] Limited git-gui to displaying 5000 new files

On Mon, Aug 10, 2009 at 17:38, Shawn O. Pearce<spearce@spearce.org> wrote:

Show 6 quoted lines

>> +                     set warning "Displaying only $display_limit of "
>> +                     append warning "[llength $to_display] files."
>> +                     info_popup [mc $warning]
>
> This needs to be in the translated strings.
>

Will do, as soon as this hits git://repo.or.cz/git-gui.git. Or should I have looked at the internationalization repo git://repo.or.cz/git-gui/git-gui-i18n.git?

Shawn O. Pearce· Aug 10, 2009, 17:08 UTC · re: Alex Riesen · lore

Re: [PATCH] Limited git-gui to displaying 5000 new files

Alex Riesen <raa.lkml@gmail.com> wrote:

Show 10 quoted lines

> On Mon, Aug 10, 2009 at 17:38, Shawn O. Pearce<spearce@spearce.org> wrote:
> >> + ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? set warning "Displaying only $display_limit of "
> >> + ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? append warning "[llength $to_display] files."
> >> + ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? info_popup [mc $warning]
> >
> > This needs to be in the translated strings.
> 
> Will do, as soon as this hits git://repo.or.cz/git-gui.git.
> Or should I have looked at the internationalization repo
> git://repo.or.cz/git-gui/git-gui-i18n.git?

No, I meant that the patch needs to use [mc] or whatever it is to enable the string to be localized. After its marked with the necessary code change, I can apply the patch, regenerate the .pot, and let translators update their .po when they have time.

-- 
Shawn.

Dan Zwell· Aug 10, 2009, 15:15 UTC · re: Shawn O. Pearce · lore

Re: [PATCH] Limited git-gui to displaying 5000 new files

Shawn O. Pearce wrote:

Show 15 quoted lines

> Alex Riesen <raa.lkml@gmail.com> wrote:
>> On Mon, Aug 10, 2009 at 17:38, Shawn O. Pearce<spearce@spearce.org> wrote:
>>>> + ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? set warning "Displaying only $display_limit of "
>>>> + ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? append warning "[llength $to_display] files."
>>>> + ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? info_popup [mc $warning]
>>> This needs to be in the translated strings.
>> Will do, as soon as this hits git://repo.or.cz/git-gui.git.
>> Or should I have looked at the internationalization repo
>> git://repo.or.cz/git-gui/git-gui-i18n.git?
> 
> No, I meant that the patch needs to use [mc] or whatever it is
> to enable the string to be localized.  After its marked with the
> necessary code change, I can apply the patch, regenerate the .pot,
> and let translators update their .po when they have time.
>

I will change it to use [mc] (and make the other changes that Shawn mentioned) and resubmit the patch.

-Dan

Dan Zwell· Aug 11, 2009, 18:23 UTC · re: Shawn O. Pearce · lore

Re: [PATCH] Limit git-gui to display a maximum number of files

When there is a large number of new or modified files, "display_all_files" takes a long time, and git-gui appears to hang. This change limits the number of files that are displayed. This limit can be set as gui.maxfilesdisplayed, and is 5000 by default.

A warning is shown when the list of files is truncated.

Signed-off-by: Dan Zwell <dzwell@zwell.net>
---
By the way, is the right way to deal with strings to be
translated? See the end of the patch.

 git-gui.sh     |   18 +++++++++++++++++-
 po/git-gui.pot |    5 +++++
 2 files changed, 22 insertions(+), 1 deletions(-)

Show changes to 2 files +22 −1

git-gui.sh, po/git-gui.pot

diff --git a/git-gui.sh b/git-gui.sh
index 3c0ce26..a4dde9e 100755
--- a/git-gui.sh
+++ b/git-gui.sh
@@ -745,6 +745,8 @@ set default_config(gui.newbranchtemplate) {}
 set default_config(gui.spellingdictionary) {}
 set default_config(gui.fontui) [font configure font_ui]
 set default_config(gui.fontdiff) [font configure font_diff]
+# TODO: this option should be added to the git-config documentation
+set default_config(gui.maxfilesdisplayed) 5000
 set font_descs {
 	{fontui   font_ui   {mc "Main Font"}}
 	{fontdiff font_diff {mc "Diff/Console Font"}}
@@ -1698,10 +1700,12 @@ proc display_all_files_helper {w path icon_name m} {
 	$w insert end "[escape_path $path]\n"
 }
 
+set files_warning 0
 proc display_all_files {} {
 	global ui_index ui_workdir
 	global file_states file_lists
 	global last_clicked
+	global files_warning
 
 	$ui_index conf -state normal
 	$ui_workdir conf -state normal
@@ -1713,7 +1717,19 @@ proc display_all_files {} {
 	set file_lists($ui_index) [list]
 	set file_lists($ui_workdir) [list]
 
-	foreach path [lsort [array names file_states]] {
+	set to_display [lsort [array names file_states]]
+	set display_limit [get_config gui.maxfilesdisplayed]
+	if {[llength $to_display] > $display_limit} {
+		if {!$files_warning} {
+			# do not repeatedly warn:
+			set files_warning 1
+			set warning "Displaying only $display_limit of "
+			append warning "[llength $to_display] files."
+			info_popup [mc $warning]
+		}
+		set to_display [lrange $to_display 0 [expr {$display_limit-1}]]
+	}
+	foreach path $to_display {
 		set s $file_states($path)
 		set m [lindex $s 0]
 		set icon_name [lindex $s 1]
diff --git a/po/git-gui.pot b/po/git-gui.pot
index 53b7d36..fb60472 100644
--- a/po/git-gui.pot
+++ b/po/git-gui.pot
@@ -90,6 +90,11 @@ msgstr ""
 msgid "Ready."
 msgstr ""
 
+#: git-gui.sh:1725
+#, tcl-format
+msgid "Displaying only %s of %s files."
+msgstr ""
+
 #: git-gui.sh:1819
 msgid "Unmodified"
 msgstr ""

-- 
1.6.4


Shawn O. Pearce wrote:

> Dan Zwell <dzwell@gmail.com> wrote:
>   
>> When there is a large number of new or modified files,
>> "display_all_files" takes a long time, and git-gui appears to
>> hang. Limit the display to 5000 files, by default. This number
>> is configurable as gui.maxfilesdisplayed.
>>
>> Show a warning if the list of files is truncated.
>>     
>
>   
>> @@ -1713,7 +1717,18 @@ proc display_all_files {} {
>> 	set file_lists($ui_index) [list]
>> 	set file_lists($ui_workdir) [list]
>>
>> -	foreach path [lsort [array names file_states]] {
>> +	set to_display [lsort [array names file_states]]
>> +	set display_limit $default_config(gui.maxfilesdisplayed)
>>     
>
> This should use [get_config gui.maxfilesdisplayed] so that the
> user can actually set this property in a configuration file and
> have git-gui honor it.  Reading from $default_config means you are
> only looking at the hardcoded value you set in git-gui.sh.
>
>   
>> +	if {[llength $to_display] > $display_limit} {
>> +		if {![info exists files_warning] || !$files_warning} {
>>     
>
> Wouldn't it be easier to just set files_warning to 0 at the start
> of the script, so that you don't need to do this info exists test?
>
>   
>> +			set warning "Displaying only $display_limit of "
>> +			append warning "[llength $to_display] files."
>> +			info_popup [mc $warning]
>>     
>
> This needs to be in the translated strings.
>
>

Shawn O. Pearce· Aug 11, 2009, 20:29 UTC · re: Dan Zwell · lore

Re: [PATCH] Limit git-gui to display a maximum number of files

Dan Zwell <dzwell@zwell.net> wrote:

Show 12 quoted lines

> When there is a large number of new or modified files,
> "display_all_files" takes a long time, and git-gui appears to
> hang. This change limits the number of files that are displayed.
> This limit can be set as gui.maxfilesdisplayed, and is
> 5000 by default.
>
> A warning is shown when the list of files is truncated.
>
> Signed-off-by: Dan Zwell <dzwell@zwell.net>
> ---
> By the way, is the right way to deal with strings to be
> translated? See the end of the patch.

No.

> +			set warning "Displaying only $display_limit of "
> +			append warning "[llength $to_display] files."
> +			info_popup [mc $warning]

This should be:

info_popup [mc "Displaying only %s of %s files." $display_limit [llength $to_display]]

> +msgid "Displaying only %s of %s files."
> +msgstr ""

So that then the placeholders are available here...

-- 
Shawn.

Dan Zwell· Aug 11, 2009, 18:50 UTC · re: Shawn O. Pearce · lore

Re: [PATCH] Limit git-gui to display a maximum number of files

A warning is shown when the list of files is truncated.

Signed-off-by: Dan Zwell <dzwell@zwell.net>
---
 git-gui.sh     |   17 ++++++++++++++++-
 po/git-gui.pot |    5 +++++
 2 files changed, 21 insertions(+), 1 deletions(-)

Show changes to 2 files +21 −1

git-gui.sh, po/git-gui.pot

diff --git a/git-gui.sh b/git-gui.sh
index 3c0ce26..eae1f81 100755
--- a/git-gui.sh
+++ b/git-gui.sh
@@ -745,6 +745,8 @@ set default_config(gui.newbranchtemplate) {}
 set default_config(gui.spellingdictionary) {}
 set default_config(gui.fontui) [font configure font_ui]
 set default_config(gui.fontdiff) [font configure font_diff]
+# TODO: this option should be added to the git-config documentation
+set default_config(gui.maxfilesdisplayed) 5000
 set font_descs {
 	{fontui   font_ui   {mc "Main Font"}}
 	{fontdiff font_diff {mc "Diff/Console Font"}}
@@ -1698,10 +1700,12 @@ proc display_all_files_helper {w path icon_name m} {
 	$w insert end "[escape_path $path]\n"
 }
 
+set files_warning 0
 proc display_all_files {} {
 	global ui_index ui_workdir
 	global file_states file_lists
 	global last_clicked
+	global files_warning
 
 	$ui_index conf -state normal
 	$ui_workdir conf -state normal
@@ -1713,7 +1717,18 @@ proc display_all_files {} {
 	set file_lists($ui_index) [list]
 	set file_lists($ui_workdir) [list]
 
-	foreach path [lsort [array names file_states]] {
+	set to_display [lsort [array names file_states]]
+	set display_limit [get_config gui.maxfilesdisplayed]
+	if {[llength $to_display] > $display_limit} {
+		if {!$files_warning} {
+			# do not repeatedly warn:
+			set files_warning 1
+			info_popup [mc "Displaying only %s of %s files." \
+				$display_limit [llength $to_display]]
+		}
+		set to_display [lrange $to_display 0 [expr {$display_limit-1}]]
+	}
+	foreach path $to_display {
 		set s $file_states($path)
 		set m [lindex $s 0]
 		set icon_name [lindex $s 1]
diff --git a/po/git-gui.pot b/po/git-gui.pot
index 53b7d36..074582d 100644
--- a/po/git-gui.pot
+++ b/po/git-gui.pot
@@ -90,6 +90,11 @@ msgstr ""
 msgid "Ready."
 msgstr ""
 
+#: git-gui.sh:1726
+#, tcl-format
+msgid "Displaying only %s of %s files."
+msgstr ""
+
 #: git-gui.sh:1819
 msgid "Unmodified"
 msgstr ""

-- 
1.6.4

Shawn O. Pearce· Aug 12, 2009, 14:43 UTC · re: Dan Zwell · lore

Re: [PATCH] Limit git-gui to display a maximum number of files

Dan Zwell <dzwell@zwell.net> wrote:

Show 7 quoted lines

> When there is a large number of new or modified files,
> "display_all_files" takes a long time, and git-gui appears to
> hang. This change limits the number of files that are displayed.
> This limit can be set as gui.maxfilesdisplayed, and is
> 5000 by default.
>
> A warning is shown when the list of files is truncated.

Thanks, applied.

-- 
Shawn.

Alex Riesen· Aug 12, 2009, 15:24 UTC · re: Shawn O. Pearce · lore

[PATCH] git-gui: Update russian translation

Signed-off-by: Alex Riesen <raa.lkml@gmail.com>
---

Shawn O. Pearce, Wed, Aug 12, 2009 16:43:45 +0200:

Show 11 quoted lines

> Dan Zwell <dzwell@zwell.net> wrote:
> > When there is a large number of new or modified files,
> > "display_all_files" takes a long time, and git-gui appears to
> > hang. This change limits the number of files that are displayed.
> > This limit can be set as gui.maxfilesdisplayed, and is
> > 5000 by default.
> >
> > A warning is shown when the list of files is truncated.
> 
> Thanks, applied.
>

And the updated translation.

But now, when I really think about the change, it looks useless. What has the _number_ of files has to do with the files you actually have to handle? As the sorting of the file list cannot be changed (and it wouldn't be a big help anyway), you have no chance to get to your file if it happens to be past the limit!

Wouldn't a pathname/glob filter in the command-line (or file/path selection dialog) to limit the scope be more appropriate and useful? And have the file list reading to happen in background, as gitk does?

 po/ru.po |   28 ++++++++++++++++------------
 1 files changed, 16 insertions(+), 12 deletions(-)

Show changes to po/ru.po +16 −12

diff --git a/po/ru.po b/po/ru.po
index 0ffc4a4..364c074 100644
--- a/po/ru.po
+++ b/po/ru.po
@@ -90,12 +90,18 @@ msgstr "Вызов программы поддержки репозитория
 
 #: git-gui.sh:1384
 msgid "Commit declined by prepare-commit-msg hook."
-msgstr "Сохранение прервано программой поддержки репозитория prepare-commit-msg"
+msgstr ""
+"Сохранение прервано программой поддержки репозитория prepare-commit-msg"
 
 #: git-gui.sh:1542 lib/browser.tcl:246
 msgid "Ready."
 msgstr "Готово."
 
+#: git-gui.sh:1726
+#, tcl-format
+msgid "Displaying only %s of %s files."
+msgstr "Показано %s из %s файлов."
+
 #: git-gui.sh:1819
 msgid "Unmodified"
 msgstr "Не изменено"
@@ -1297,8 +1303,8 @@ msgid ""
 msgstr ""
 "Невозможно исправить состояние во время операции слияния.\n"
 "\n"
-"Текущее слияние не завершено. Невозможно исправить предыдущее "
-"сохраненное состояние, не прерывая эту операцию.\n"
+"Текущее слияние не завершено. Невозможно исправить предыдущее сохраненное "
+"состояние, не прерывая эту операцию.\n"
 
 #: lib/commit.tcl:48
 msgid "Error loading commit data for amend:"
@@ -1723,8 +1729,7 @@ msgid ""
 msgstr ""
 "Невозможно выполнить слияние во время исправления.\n"
 "\n"
-"Завершите исправление данного состояния перед выполнением операции "
-"слияния.\n"
+"Завершите исправление данного состояния перед выполнением операции слияния.\n"
 
 #: lib/merge.tcl:27
 msgid ""
@@ -1888,8 +1893,8 @@ msgstr ""
 #, tcl-format
 msgid "File %s seems to have unresolved conflicts, still stage?"
 msgstr ""
-"Файл %s кажется содержит необработаные конфликты. "
-"Продолжить подготовку к сохранению?"
+"Файл %s кажется содержит необработаные конфликты. Продолжить подготовку к "
+"сохранению?"
 
 #: lib/mergetool.tcl:60
 #, tcl-format
@@ -2213,8 +2218,8 @@ msgid ""
 "One or more of the merge tests failed because you have not fetched the "
 "necessary commits.  Try fetching from %s first."
 msgstr ""
-"Некоторые тесты на слияние не прошли, потому что Вы не "
-"получили необходимые состояния. Попытайтесь получить их из %s."
+"Некоторые тесты на слияние не прошли, потому что Вы не получили необходимые "
+"состояния. Попытайтесь получить их из %s."
 
 #: lib/remote_branch_delete.tcl:207
 msgid "Please select one or more branches to delete."
@@ -2381,8 +2386,8 @@ msgstr "Выполнение: %s"
 
 #: lib/tools.tcl:149
 #, tcl-format
-msgid "Tool completed succesfully: %s"
-msgstr "Программа %s успешно завершилась."
+msgid "Tool completed successfully: %s"
+msgstr "Программа %s завершилась успешно."
 
 #: lib/tools.tcl:151
 #, tcl-format
@@ -2538,4 +2543,3 @@ msgstr "Использовать thin pack (для медленных сетев
 #: lib/transport.tcl:179
 msgid "Include tags"
 msgstr "Передать метки"
-

-- 
1.6.4.140.gc6dfd

Shawn O. Pearce· Aug 12, 2009, 15:40 UTC · re: Alex Riesen · lore

Re: [PATCH] git-gui: Update russian translation

Alex Riesen <raa.lkml@gmail.com> wrote:

Show 21 quoted lines

> Shawn O. Pearce, Wed, Aug 12, 2009 16:43:45 +0200:
> > Dan Zwell <dzwell@zwell.net> wrote:
> > > When there is a large number of new or modified files,
> > > "display_all_files" takes a long time, and git-gui appears to
> > > hang. This change limits the number of files that are displayed.
> > > This limit can be set as gui.maxfilesdisplayed, and is
> > > 5000 by default.
> > >
> > > A warning is shown when the list of files is truncated.
> > 
> > Thanks, applied.
> 
> But now, when I really think about the change, it looks useless.
> What has the _number_ of files has to do with the files you actually
> have to handle? As the sorting of the file list cannot be changed (and
> it wouldn't be a big help anyway), you have no chance to get to your
> file if it happens to be past the limit!
> 
> Wouldn't a pathname/glob filter in the command-line (or file/path
> selection dialog) to limit the scope be more appropriate and useful?
> And have the file list reading to happen in background, as gitk does?

Good point. I suspect the problem wasn't so much with Tcl doing the list processing as it was with Tk actually creating the underlying icons and stuff for each file name. But with the list clipped, you are right, you are basically SOL. You can't do much beyond dropping back to the CLI and using the CLI tools.

IMHO, if we aren't going to handle 20k file names, we should at least punt and tell the user we aren't going to handle 20k file names, rather than just play Outlook wannabe and lockup the entire UI until the user gets bored and kill -9's us. So this patch is better than nothing, it at least lets the user know we have given up on them.

-- 
Shawn.

Dan Zwell· Aug 12, 2009, 15:51 UTC · re: Shawn O. Pearce · lore

Re: [PATCH] git-gui: Update russian translation

Show 6 quoted lines

> Alex Riesen <raa.lkml@gmail.com> wrote:
>> But now, when I really think about the change, it looks useless.
>> What has the _number_ of files has to do with the files you actually
>> have to handle? As the sorting of the file list cannot be changed (and
>> it wouldn't be a big help anyway), you have no chance to get to your
>> file if it happens to be past the limit!

Show 12 quoted lines

> Good point.  I suspect the problem wasn't so much with Tcl doing the
> list processing as it was with Tk actually creating the underlying
> icons and stuff for each file name.  But with the list clipped,
> you are right, you are basically SOL.  You can't do much beyond
> dropping back to the CLI and using the CLI tools.
> 
> IMHO, if we aren't going to handle 20k file names, we should at least
> punt and tell the user we aren't going to handle 20k file names,
> rather than just play Outlook wannabe and lockup the entire UI until
> the user gets bored and kill -9's us.  So this patch is better than
> nothing, it at least lets the user know we have given up on them.
>

A warning would be good, but this gives users more information. In my case, I only wanted to see a few files, but the rest were from a directory that should have been in .gitignore. If I had seen the filenames, I would have known which directory was the culprit. This way, you don't need to drop to the command line to fix the problem.

-Dan