threads / patch / 12674

patchpack-objects: proper pack time stamping with --max-pack-size

Subject: [PATCH] pack-objects: proper pack time stamping with --max-pack-size

## tl;dr

5 messages between Mar 13, 2008 and Mar 14, 2008. Diffs are folded; open one to read it.

replies: 4people: 2as markdown or json

Nicolas Pitre· Mar 13, 2008, 18:59 UTC · lore

Runtime pack access is done in the pack file mtime order since recent packs are more likely to contain frequently used objects than old packs. However the --max-pack-size option can produce multiple packs with mtime in the reversed order as newer objects are always written first.

Let's modify mtime of later pack files (when any) so they appear older than preceding ones when a repack creates multiple packs.

Signed-off-by: Nicolas Pitre <nico@cam.org>
---
Show changes to builtin-pack-objects.c +28 −0
diff --git a/builtin-pack-objects.c b/builtin-pack-objects.c
index f504cff..4c2ed70 100644
--- a/builtin-pack-objects.c
+++ b/builtin-pack-objects.c
@@ -17,6 +17,8 @@
 #include "progress.h"
 #include "refs.h"
 
+#include <utime.h>
+
 #ifdef THREADED_DELTA_SEARCH
 #include "thread-utils.h"
 #include <pthread.h>
@@ -454,6 +456,7 @@ static void write_pack_file(void)
 	struct pack_header hdr;
 	int do_progress = progress >> pack_to_stdout;
 	uint32_t nr_remaining = nr_result;
+	time_t last_mtime = 0;
 
 	if (do_progress)
 		progress_state = start_progress("Writing objects", nr_result);
@@ -504,6 +507,7 @@ static void write_pack_file(void)
 
 		if (!pack_to_stdout) {
 			mode_t mode = umask(0);
+			struct stat st;
 			char *idx_tmp_name, tmpname[PATH_MAX];
 
 			umask(mode);
@@ -511,6 +515,7 @@ static void write_pack_file(void)
 
 			idx_tmp_name = write_idx_file(NULL, written_list,
 						      nr_written, sha1);
+
 			snprintf(tmpname, sizeof(tmpname), "%s-%s.pack",
 				 base_name, sha1_to_hex(sha1));
 			if (adjust_perm(pack_tmp_name, mode))
@@ -519,6 +524,28 @@ static void write_pack_file(void)
 			if (rename(pack_tmp_name, tmpname))
 				die("unable to rename temporary pack file: %s",
 				    strerror(errno));
+
+			/*
+			 * Packs are runtime accessed in their mtime
+			 * order since newer packs are more likely to contain
+			 * younger objects.  So if we are creating multiple
+			 * packs then we should modify the mtime of later ones
+			 * to preserve this property.
+			 */
+			if (stat(tmpname, &st) < 0) {
+				warning("failed to stat %s: %s",
+					tmpname, strerror(errno));
+			} else if (!last_mtime) {
+				last_mtime = st.st_mtime;
+			} else {
+				struct utimbuf utb;
+				utb.actime = st.st_atime;
+				utb.modtime = --last_mtime;
+				if (utime(tmpname, &utb) < 0)
+					warning("failed utime() on %s: %s",
+						tmpname, strerror(errno));
+			}
+
 			snprintf(tmpname, sizeof(tmpname), "%s-%s.idx",
 				 base_name, sha1_to_hex(sha1));
 			if (adjust_perm(idx_tmp_name, mode))
@@ -527,6 +554,7 @@ static void write_pack_file(void)
 			if (rename(idx_tmp_name, tmpname))
 				die("unable to rename temporary index file: %s",
 				    strerror(errno));
+
 			free(idx_tmp_name);
 			free(pack_tmp_name);
 			puts(sha1_to_hex(sha1));
Junio C Hamano· Mar 13, 2008, 20:59 UTC · re: Nicolas Pitre · lore

Re: [PATCH] pack-objects: proper pack time stamping with --max-pack-size

Nicolas Pitre <nico@cam.org> writes:
Show 7 quoted lines
> Runtime pack access is done in the pack file mtime order since recent 
> packs are more likely to contain frequently used objects than old packs.
> However the --max-pack-size option can produce multiple packs with mtime 
> in the reversed order as newer objects are always written first.
>
> Let's modify mtime of later pack files (when any) so they appear older 
> than preceding ones when a repack creates multiple packs.

It is a very good companion to f7c22cc (always start looking up objects in the last used pack first, May 30, 2007) after 10 months ;-)

Junio C Hamano· Mar 14, 2008, 04:33 UTC · re: Nicolas Pitre · lore

Re: [PATCH] pack-objects: proper pack time stamping with --max-pack-size

Nicolas Pitre <nico@cam.org> writes:
Show 20 quoted lines
> Runtime pack access is done in the pack file mtime order since recent 
> packs are more likely to contain frequently used objects than old packs.
> However the --max-pack-size option can produce multiple packs with mtime 
> in the reversed order as newer objects are always written first.
>
> Let's modify mtime of later pack files (when any) so they appear older 
> than preceding ones when a repack creates multiple packs.
>
> Signed-off-by: Nicolas Pitre <nico@cam.org>
> ---
> diff --git a/builtin-pack-objects.c b/builtin-pack-objects.c
> index f504cff..4c2ed70 100644
> --- a/builtin-pack-objects.c
> +++ b/builtin-pack-objects.c
> @@ -17,6 +17,8 @@
>  #include "progress.h"
>  #include "refs.h"
>  
> +#include <utime.h>
> +
Hmmm.  Shouldn't this go to git-compat-util.h?
Nicolas Pitre· Mar 14, 2008, 04:39 UTC · re: Junio C Hamano · lore

Re: [PATCH] pack-objects: proper pack time stamping with --max-pack-size

On Thu, 13 Mar 2008, Junio C Hamano wrote:
Show 24 quoted lines
> Nicolas Pitre <nico@cam.org> writes:
> 
> > Runtime pack access is done in the pack file mtime order since recent 
> > packs are more likely to contain frequently used objects than old packs.
> > However the --max-pack-size option can produce multiple packs with mtime 
> > in the reversed order as newer objects are always written first.
> >
> > Let's modify mtime of later pack files (when any) so they appear older 
> > than preceding ones when a repack creates multiple packs.
> >
> > Signed-off-by: Nicolas Pitre <nico@cam.org>
> > ---
> > diff --git a/builtin-pack-objects.c b/builtin-pack-objects.c
> > index f504cff..4c2ed70 100644
> > --- a/builtin-pack-objects.c
> > +++ b/builtin-pack-objects.c
> > @@ -17,6 +17,8 @@
> >  #include "progress.h"
> >  #include "refs.h"
> >  
> > +#include <utime.h>
> > +
> 
> Hmmm.  Shouldn't this go to git-compat-util.h?
Maybe... if you say so.

The whole header file arrangement logic is somewhat escaping my mind, so I simply decided to do the same as in test-chmtime.c.

Nicolas
Junio C Hamano· Mar 14, 2008, 05:13 UTC · re: Nicolas Pitre · lore

Re: [PATCH] pack-objects: proper pack time stamping with --max-pack-size

Nicolas Pitre <nico@cam.org> writes:
Show 17 quoted lines
>> > diff --git a/builtin-pack-objects.c b/builtin-pack-objects.c
>> > index f504cff..4c2ed70 100644
>> > --- a/builtin-pack-objects.c
>> > +++ b/builtin-pack-objects.c
>> > @@ -17,6 +17,8 @@
>> >  #include "progress.h"
>> >  #include "refs.h"
>> >  
>> > +#include <utime.h>
>> > +
>> 
>> Hmmm.  Shouldn't this go to git-compat-util.h?
>
> Maybe... if you say so.
>
> The whole header file arrangement logic is somewhat escaping my mind, so 
> I simply decided to do the same as in test-chmtime.c.

The general idea is to isolate the place system headers are included at, so that we have to worry about the inclusion and feature-macro definition order at only one place (see for example the hack about _ALL_SOURCE to work around AIX 5.3, yuck).

← back to recent threads