git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH] teach fast-export an --anonymize option

From
Jeff King <peff@peff.net>
Date
Aug 21, 2014, 22:41 UTC
Message-ID
<20140821224150.GA21105@peff.net>
In-Reply-To
<xmqqmwaxr44x.fsf@gitster.dls.corp.google.com>
On Thu, Aug 21, 2014 at 01:15:10PM -0700, Junio C Hamano wrote:
Show 28 quoted lines
> Jeff King <peff@peff.net> writes:
> 
> > +/*
> > + * We anonymize each component of a path individually,
> > + * so that paths a/b and a/c will share a common root.
> > + * The paths are cached via anonymize_mem so that repeated
> > + * lookups for "a" will yield the same value.
> > + */
> > +static void anonymize_path(struct strbuf *out, const char *path,
> > +			   struct hashmap *map,
> > +			   char *(*generate)(const char *, size_t *))
> > +{
> > +	while (*path) {
> > +		const char *end_of_component = strchrnul(path, '/');
> > +		size_t len = end_of_component - path;
> > +		const char *c = anonymize_mem(map, generate, path, &len);
> > +		strbuf_add(out, c, len);
> > +		path = end_of_component;
> > +		if (*path)
> > +			strbuf_addch(out, *path++);
> > +	}
> > +}
> 
> Do two paths sort the same way before and after anonymisation?  For
> example, if generate() works as a simple substitution, it should map
> a character that sorts before (or after) '/' with another that also
> sorts before (or after) '/' for us to be able to diagnose an error
> that comes from D/F sort order confusion.

No, the sort order is totally lost. I'd be afraid that a general scheme would end up leaking information about what was in the filenames. It might be acceptable to leak some information here, though, if it adds to the realism of the result.

I tried here to lay the basic infrastructure and do the simplest thing that might work, so we could evaluate proposals like that independently (and also because I didn't come up with a clever enough algorithm to do what you're asking). Patches welcome on top. :)

-Peff
Previous: Junio C HamanoNext: Junio C Hamano
Message 3 of 21 in “teach fast-export an --anonymize option”
  1. teach fast-export an --anonymize optionJeff King, Aug 21, 2014
  2. Junio C HamanoAug 21, 2014
  3. Jeff KingAug 21, 2014
  4. Junio C HamanoAug 21, 2014
  5. Jeff KingAug 21, 2014
  6. teach fast-export an --anonymize optionJeff King, Aug 21, 2014
  7. Duy NguyenAug 22, 2014
  8. Philip OakleyAug 22, 2014
  9. Jeff KingAug 23, 2014
  10. Junio C HamanoAug 27, 2014
  11. Jeff KingAug 27, 2014
  12. teach fast-export an --anonymize optionJeff King, Aug 27, 2014
  13. Duy NguyenAug 28, 2014
  14. Jeff KingAug 28, 2014
  15. Ramsay JonesAug 28, 2014
  16. Junio C HamanoAug 28, 2014
  17. Jeff KingAug 28, 2014
  18. Junio C HamanoAug 28, 2014
  19. Jeff KingAug 28, 2014
  20. Eric SunshineAug 31, 2014
  21. Jeff KingAug 31, 2014

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.