threads / patch / 35937

patchclone: allow initial sparse checkouts

Subject: [PATCH] clone: allow initial sparse checkouts

## tl;dr

8 messages between Feb 23, 2014 and Feb 25, 2014. Diffs are folded; open one to read it.

replies: 7people: 3as markdown or json

Robin H. Johnson· Feb 23, 2014, 02:31 UTC · lore

Presently if you want to perform a sparse checkout, you must either do a full clone and then recheckout, or do a git init, manually set up sparse, and then fetch and checkout.

This patch implements easily accessible sparse checkouts during clone, in the --sparse-checkout option.

$ git clone REPO --sparse-checkout PATH
Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>
---
 Documentation/git-clone.txt |  5 +++++
 builtin/clone.c             | 24 ++++++++++++++++++++++++
 2 files changed, 29 insertions(+)
Show changes to 2 files +29 −0

Documentation/git-clone.txt, builtin/clone.c

diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index 0363d00..1c21207 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -14,6 +14,7 @@ SYNOPSIS
 	  [-o <name>] [-b <name>] [-u <upload-pack>] [--reference <repository>]
 	  [--separate-git-dir <git dir>]
 	  [--depth <depth>] [--[no-]single-branch]
+	  [--sparse-checkout <path>]
 	  [--recursive | --recurse-submodules] [--] <repository>
 	  [<directory>]
 
@@ -209,6 +210,10 @@ objects from the source repository into a pack in the cloned repository.
 	The result is Git repository can be separated from working
 	tree.
 
+--sparse-checkout <path>::
+	Perform the initial checkout as a sparse checkout, checking out only the
+	paths specified by this option. This option may occur multiple times, with
+	one path per instance.
 
 <repository>::
 	The (possibly remote) repository to clone from.  See the
diff --git a/builtin/clone.c b/builtin/clone.c
index 43e772c..1137371 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -48,6 +48,7 @@ static int option_verbosity;
 static int option_progress = -1;
 static struct string_list option_config;
 static struct string_list option_reference;
+static struct string_list option_sparse_checkout_path;
 
 static int opt_parse_reference(const struct option *opt, const char *arg, int unset)
 {
@@ -97,6 +98,8 @@ static struct option builtin_clone_options[] = {
 		   N_("separate git dir from working tree")),
 	OPT_STRING_LIST('c', "config", &option_config, N_("key=value"),
 			N_("set config inside the new repository")),
+	OPT_STRING_LIST(0, "sparse-checkout", &option_sparse_checkout_path, N_("path"),
+			N_("set path for sparse checkout")),
 	OPT_END()
 };
 
@@ -270,6 +273,24 @@ static void setup_reference(void)
 	for_each_string_list(&option_reference, add_one_reference, NULL);
 }
 
+static void setup_sparse_checkout(void)
+{
+	FILE *info_sparse_checkout_fp;
+	struct string_list_item *item;
+	char *path = git_pathdup("info/sparse-checkout");
+	git_config_set("core.sparsecheckout", "true");
+	safe_create_leading_directories(path);
+	info_sparse_checkout_fp = fopen(path, "w");
+	if (!info_sparse_checkout_fp)
+		die(_("unable to create %s"), path);
+	for_each_string_list_item(item, &option_sparse_checkout_path) {
+		fprintf(info_sparse_checkout_fp, "%s\n", item->string);
+	}
+	fclose(info_sparse_checkout_fp);
+	adjust_shared_perm(path);
+	free(path);
+}
+
 static void copy_alternates(struct strbuf *src, struct strbuf *dst,
 			    const char *src_repo)
 {
@@ -873,6 +894,9 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
 	init_db(option_template, INIT_DB_QUIET);
 	write_config(&option_config);
 
+	if(option_sparse_checkout_path.nr)
+		setup_sparse_checkout();
+
 	git_config(git_default_config, NULL);
 
 	if (option_bare) {

-- 
1.9.0.291.g027825b.dirty
Duy Nguyen· Feb 23, 2014, 02:52 UTC · re: Robin H. Johnson · lore

Re: [PATCH] clone: allow initial sparse checkouts

On Sun, Feb 23, 2014 at 9:31 AM, Robin H. Johnson <robbat2@gentoo.org> wrote:
> Presently if you want to perform a sparse checkout, you must either do a
> full clone and then recheckout, or do a git init, manually set up
> sparse, and then fetch and checkout.
I think you could do "clone -n" (no checkout), set up sparse, then checkout.
> This patch implements easily accessible sparse checkouts during clone,
> in the --sparse-checkout option.
>
> $ git clone REPO --sparse-checkout PATH
Or take a file as input if there are lots of paths/rules.
-- 
Duy
Robin H. Johnson· Feb 23, 2014, 07:32 UTC · re: Duy Nguyen · lore

Re: [PATCH] clone: allow initial sparse checkouts

On Sun, Feb 23, 2014 at 09:52:16AM +0700,  Duy Nguyen wrote:
Show 5 quoted lines
> On Sun, Feb 23, 2014 at 9:31 AM, Robin H. Johnson <robbat2@gentoo.org> wrote:
> > Presently if you want to perform a sparse checkout, you must either do a
> > full clone and then recheckout, or do a git init, manually set up
> > sparse, and then fetch and checkout.
> I think you could do "clone -n" (no checkout), set up sparse, then checkout.
Yes, I think there are a few more ways, but still would be nice to
Show 6 quoted lines
> 
> > This patch implements easily accessible sparse checkouts during clone,
> > in the --sparse-checkout option.
> >
> > $ git clone REPO --sparse-checkout PATH
> Or take a file as input if there are lots of paths/rules.

How much demand for taking a file of rules, and opinions of syntax to do that vs specify on the commandline?

--sparse-checkout-from FILE vs. --sparse-checkout '<PATH' or something other prefix character.

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Infrastructure Lead
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
Duy Nguyen· Feb 23, 2014, 08:43 UTC · re: Robin H. Johnson · lore

Re: [PATCH] clone: allow initial sparse checkouts

On Sun, Feb 23, 2014 at 2:32 PM, Robin H. Johnson <robbat2@gentoo.org> wrote:
Show 6 quoted lines
>> > This patch implements easily accessible sparse checkouts during clone,
>> > in the --sparse-checkout option.
>> >
>> > $ git clone REPO --sparse-checkout PATH
>> Or take a file as input if there are lots of paths/rules.
> How much demand for taking a file of rules,

I don't know. I guess it depends on each repo's layout. If the layout is simple, usually you would need one or two rules so it's ok to type again and again. If it's more complicated and starts using '!' rules, it's probably best to save in a file.

> and opinions of syntax to do
> that vs specify on the commandline?
>
> --sparse-checkout-from FILE

I think this one is better. But if you don't see a need for it, we can always delay implementing it until an actual use case comes up.

> vs.
> --sparse-checkout '<PATH'
> or something other prefix character.
-- 
Duy
Robin H. Johnson· Feb 23, 2014, 20:58 UTC · re: Duy Nguyen · lore

Re: [PATCH] clone: allow initial sparse checkouts

On Sun, Feb 23, 2014 at 03:43:47PM +0700,  Duy Nguyen wrote:
Show 19 quoted lines
> On Sun, Feb 23, 2014 at 2:32 PM, Robin H. Johnson <robbat2@gentoo.org> wrote:
> >> > This patch implements easily accessible sparse checkouts during clone,
> >> > in the --sparse-checkout option.
> >> >
> >> > $ git clone REPO --sparse-checkout PATH
> >> Or take a file as input if there are lots of paths/rules.
> > How much demand for taking a file of rules,
> I don't know. I guess it depends on each repo's layout. If the layout
> is simple, usually you would need one or two rules so it's ok to type
> again and again. If it's more complicated and starts using '!' rules,
> it's probably best to save in a file.
> 
> > and opinions of syntax to do
> > that vs specify on the commandline?
> >
> > --sparse-checkout-from FILE
> 
> I think this one is better. But if you don't see a need for it, we can
> always delay implementing it until an actual use case comes up.

I think I'd prefer to delay that part then. What I'm concerned about if we do have it, is what ordering semantics there should be, eg for something like: --sparse-checkout '!X' --sparse-checkout-from F --sparse-checkout Y

Should that be [!X, *F, Y], or [*F, !X, Y], or something else? Would the option parser need to be modified to handle this? Or do we just make them mutually exclusive?

The only other clean alternative would be implementing ONLY --sparse-checkout-from, and letting uses use fds creatively: --sparse-checkout-from <(echo X; echo Y) But the msysgit crowd would probably mumble complaints under their breath at me.

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Infrastructure Lead
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
Junio C Hamano· Feb 24, 2014, 17:47 UTC · re: Robin H. Johnson · lore

Re: [PATCH] clone: allow initial sparse checkouts

"Robin H. Johnson" <robbat2@gentoo.org> writes:
> The only other clean alternative would be implementing ONLY
> --sparse-checkout-from, and letting uses use fds creatively:
> --sparse-checkout-from <(echo X; echo Y)

Not all POSIX shells have such an abomination that is process substitution. You can easily work it around by adopting the usual convention to use "-" to read from the standasrd input, though.

	(echo X; echo Y) | cmd --sparse-checkout-from -
Robin H. Johnson· Feb 24, 2014, 21:48 UTC · re: Junio C Hamano · lore

Re: [PATCH] clone: allow initial sparse checkouts

On Mon, Feb 24, 2014 at 09:47:16AM -0800,  Junio C Hamano wrote:
Show 9 quoted lines
> "Robin H. Johnson" <robbat2@gentoo.org> writes:
> > The only other clean alternative would be implementing ONLY
> > --sparse-checkout-from, and letting uses use fds creatively:
> > --sparse-checkout-from <(echo X; echo Y)
> Not all POSIX shells have such an abomination that is process
> substitution.  You can easily work it around by adopting the usual
> convention to use "-" to read from the standasrd input, though.
> 
> 	(echo X; echo Y) | cmd --sparse-checkout-from -

Is that a vote that you'd like to see a --sparse-checkout-from variant of my patch?

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Infrastructure Lead
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
Junio C Hamano· Feb 25, 2014, 00:17 UTC · re: Robin H. Johnson · lore

Re: [PATCH] clone: allow initial sparse checkouts

"Robin H. Johnson" <robbat2@gentoo.org> writes:
Show 12 quoted lines
> On Mon, Feb 24, 2014 at 09:47:16AM -0800,  Junio C Hamano wrote:
>> "Robin H. Johnson" <robbat2@gentoo.org> writes:
>> > The only other clean alternative would be implementing ONLY
>> > --sparse-checkout-from, and letting uses use fds creatively:
>> > --sparse-checkout-from <(echo X; echo Y)
>> Not all POSIX shells have such an abomination that is process
>> substitution.  You can easily work it around by adopting the usual
>> convention to use "-" to read from the standasrd input, though.
>> 
>> 	(echo X; echo Y) | cmd --sparse-checkout-from -
> Is that a vote that you'd like to see a --sparse-checkout-from variant
> of my patch?

Honestly, I do not particularly care too much about this feature, regardless of the interface [*1*].

It is just a vote that says "if --something-from form is going to be implemented, it should be able to read from the standard input with '-', unless there is a compelling reason not to do so".

[Footnote]

*1* In the longer term, I think sparse checkout is broken as a concept and I view this "use sparse checkout from the get-go" merely a stop-gap measure to make the band-aid a bit less painful to use. What you really want is a narrow clone, which is conceptually cleaner but a lot harder implentation-wise.

Not that keeping a band-aid usable longer is necessarily bad in the real world, though---so even I said *I*'m not interested, that is different from saying I'm not taking a patch on this topic.

← back to recent threads