git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 01/13] dir: allow exclusions from blob in addition to file

From
JHJeff Hostetler <git@jeffhostetler.com>
Date
Oct 25, 2017, 14:54 UTC
Message-ID
<b063dec4-b9c4-7145-9b57-7df22ccbc655@jeffhostetler.com>
In-Reply-To
<xmqq60b3iv79.fsf@gitster.mtv.corp.google.com>
On 10/25/2017 2:43 AM, Junio C Hamano wrote:
Show 32 quoted lines
> Jeff Hostetler <git@jeffhostetler.com> writes:
> 
>> +static int add_excludes_from_buffer(char *buf, size_t size,
>> +				    const char *base, int baselen,
>> +				    struct exclude_list *el);
>> +
>>   /*
>>    * Given a file with name "fname", read it (either from disk, or from
>>    * an index if 'istate' is non-null), parse it and store the
>> @@ -754,9 +758,9 @@ static int add_excludes(const char *fname, const char *base, int baselen,
>>   			struct sha1_stat *sha1_stat)
>>   {
>>   	struct stat st;
>> -	int fd, i, lineno = 1;
>> +	int fd;
>>   	size_t size = 0;
>> -	char *buf, *entry;
>> +	char *buf;
>>   
>>   	fd = open(fname, O_RDONLY);
>>   	if (fd < 0 || fstat(fd, &st) < 0) {
> 
> The post-context of this hunk is quite interesting in that there is
> a call to read_skip_worktree_file_from_index(); which essentially
> pretends as if we read from the filesystem but in fact it grabs the
> blob object name registered in the index and reads from it.
> 
> The reason why it is interesting is because this patch adds yet
> nother "let's instead read from a blob object" function and there is
> no sign to make the existing one take advantage of the new function
> seen in this patch.
> 

The existing code handles use cases where you want to read the exclusion list from a pathname in the worktree -- or from blob named in the index when the pathname is not populated (presumably because of the skip-worktree bit).

I was wanting to add a more general case (and perhaps my commit message should be improved). I want to be able to read it from a blob not necessarily associated with the current commit or not necessarily available on the local client, but yet known to exist. I'm thinking of the case the client could ask the server to do a partial clone using a sparse-checkout specification stored in a well-known location on the server. The reason for this is that, in this case, the client is pre-clone and doesn't have a worktree or index.

With my "add_excludes_from_blob_to_list()", we can request a blob-ish expression, such as "master:enlistments/foo". In my later commits associated with clone and fetch, we can use this mechanism to let the client ask the server to filter using the blob associated with this blob-ish. If the client has the blob (such as during a later fetch) and can resolve it, then it can and send the server the OID, but it can also send the blob-ish to the server and let it resolve it.

Jeff
Previous: Junio C HamanoNext: Junio C Hamano
Message 22 of 37 in “WIP Partial clone part 1: object filtering”
  1. 00/13 WIP Partial clone part 1: object filteringJeff Hostetler, Oct 24, 2017
  2. 02/13 list-objects-filter-map: extend oidmap to collect omitted objectsJeff Hostetler, Oct 24, 2017
  3. Junio C HamanoOct 25, 2017
  4. Jeff HostetlerOct 25, 2017
  5. Junio C HamanoOct 26, 2017
  6. 06/13 list-objects-filter-sparse: add sparse filterJeff Hostetler, Oct 24, 2017
  7. 08/13 list-objects: add traverse_commit_list_filtered methodJeff Hostetler, Oct 24, 2017
  8. Jonathan TanOct 25, 2017
  9. Jeff HostetlerOct 25, 2017
  10. 11/13 t6112: rev-list object filtering testJeff Hostetler, Oct 24, 2017
  11. 05/13 list-objects-filter-blobs-limit: add large blob filteringJeff Hostetler, Oct 24, 2017
  12. 13/13 t5317: pack-objects object filtering testJeff Hostetler, Oct 24, 2017
  13. 12/13 pack-objects: add list-objects filteringJeff Hostetler, Oct 24, 2017
  14. 10/13 rev-list: add list-objects filtering supportJeff Hostetler, Oct 24, 2017
  15. Jonathan TanOct 25, 2017
  16. Jeff HostetlerOct 25, 2017
  17. 04/13 list-objects-filter-blobs-none: add filter to omit all blobsJeff Hostetler, Oct 24, 2017
  18. 09/13 extension.partialclone: introduce partial clone extensionJeff Hostetler, Oct 24, 2017
  19. 01/13 dir: allow exclusions from blob in addition to fileJeff Hostetler, Oct 24, 2017
  20. Eric SunshineOct 25, 2017
  21. Junio C HamanoOct 25, 2017
  22. Jeff HostetlerOct 25, 2017
  23. Junio C HamanoOct 26, 2017
  24. Jeff HostetlerOct 26, 2017
  25. 07/13 list-objects-filter-options: common argument parsingJeff Hostetler, Oct 24, 2017
  26. Jonathan TanOct 25, 2017
  27. Jeff HostetlerOct 25, 2017
  28. 03/13 list-objects: filter objects in traverse_commit_listJeff Hostetler, Oct 24, 2017
  29. Jonathan TanOct 25, 2017
  30. Jeff HostetlerOct 25, 2017
  31. Jonathan TanOct 25, 2017
  32. Junio C HamanoOct 25, 2017
  33. Jonathan TanOct 25, 2017
  34. Jeff HostetlerOct 25, 2017
  35. Junio C HamanoOct 26, 2017
  36. Junio C HamanoOct 26, 2017
  37. Jonathan NiederOct 30, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.