git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 2/2] trace2: randomize/timestamp trace2 targets

From
Josh Steadmon <steadmon@google.com>
Date
Mar 15, 2019, 20:43 UTC
Message-ID
<20190315204327.GE47591@google.com>
In-Reply-To
<87h8c6baif.fsf@evledraar.gmail.com>
On 2019.03.14 00:49, Ævar Arnfjörð Bjarmason wrote:
Show 9 quoted lines
> 
> On Thu, Mar 14 2019, Josh Steadmon wrote:
> 
> > When the value of a trace2 environment variable contains instances of
> > the string "%ISO8601%", expand them into the current UTC timestamp in
> > ISO 8601 format.
> 
> Any reason not to just support feeding the path to strbuf_addftime(), to
> e.g. support a daily/hourly log?
No reason not to. Seems reasonable to me.
Show 24 quoted lines
> > When the value of a trace2 environment variable is an absolute path
> > referring to an existing directory, write output to randomly-named
> > files under the given directory. If the value is an absolute path
> > referring to a non-existent file and ends with a dash, use the value as
> > a prefix for randomly named files.
> >
> > The random filenames will consist of the value of the environment
> > variable (after potential timestamp expansion), followed by a 6
> > character random string such as would be produced by mkstemp(3).
> >
> > This makes it more convenient to collect traces for every git
> > invocation by unconditionally setting the relevant trace2 envvar to a
> > constant directory name.
> 
> Hrm, api-trace2.txt already specifies that the "sid" is going to be
> unique, couldn't we just have some mode where we use that?
> 
> But then of course when we have nested processes will contain slashes,
> so we'd either run into deep nesting or need to munge the slashes, in
> which case we might bump against a file length limit (although I haven't
> seen process trees deeper than 3-4).
> 
> Just to pry about the use-case since I'm doing similar collecting, why
> are you finding this easier to process?

Basically, our collection setup prefers smaller files that are "finished" earlier, rather than long-lived files that are constantly appended to.

Show 7 quoted lines
> With the current O_APPEND semantics you're (unless I've missed
> something) guaranteed to get a single process tree in nested order,
> whereas with this they'll all end up in separate files and you'll need
> to slurp them up, sort the whole thing and stitch it together yourself
> without the benefit of stream-parsing it where you can cheat a bit
> knowing that e.g. a "reflog expire" entry is always coming after the
> corresponding "gc" that invoked it.

Yeah, that is not an issue for us, although I can see why others would prefer single file. I suppose we can just modify the envvar to point to our newly-generated file before we spawn any child processes?

Previous: Jeff HostetlerNext: Josh Steadmon
Message 8 of 34 in “Randomize / timestamp trace2 targets”
  1. 0/2 Randomize / timestamp trace2 targetsJosh Steadmon, Mar 13, 2019
  2. 1/2 date: make get_time() publicJosh Steadmon, Mar 13, 2019
  3. 2/2 trace2: randomize/timestamp trace2 targetsJosh Steadmon, Mar 13, 2019
  4. Ævar Arnfjörð BjarmasonMar 13, 2019
  5. Jeff HostetlerMar 15, 2019
  6. Ævar Arnfjörð BjarmasonMar 15, 2019
  7. Jeff HostetlerMar 15, 2019
  8. Josh SteadmonMar 15, 2019
  9. Josh SteadmonMar 15, 2019
  10. Junio C HamanoMar 18, 2019
  11. Jeff KingMar 19, 2019
  12. Jeff KingMar 14, 2019
  13. Junio C HamanoMar 14, 2019
  14. Johannes SchindelinMar 14, 2019
  15. Josh SteadmonMar 15, 2019
  16. Jeff HostetlerMar 15, 2019
  17. Josh SteadmonMar 15, 2019
  18. Jeff HostetlerMar 18, 2019
  19. 0/1 Write trace2 output to directoriesJosh Steadmon, Mar 21, 2019
  20. 1/1 trace2: write to directory targetsJosh Steadmon, Mar 21, 2019
  21. Junio C HamanoMar 21, 2019
  22. Jeff HostetlerMar 21, 2019
  23. Junio C HamanoMar 22, 2019
  24. Jeff HostetlerMar 22, 2019
  25. 0/1 Write trace2 output to directoriesJosh Steadmon, Mar 21, 2019
  26. 1/1 trace2: write to directory targetsJosh Steadmon, Mar 21, 2019
  27. Ævar Arnfjörð BjarmasonMar 23, 2019
  28. Junio C HamanoMar 24, 2019
  29. Ævar Arnfjörð BjarmasonMar 24, 2019
  30. Junio C HamanoMar 25, 2019
  31. Ævar Arnfjörð BjarmasonMar 25, 2019
  32. Jeff HostetlerMar 25, 2019
  33. Jeff HostetlerMar 21, 2019
  34. Junio C HamanoMar 22, 2019

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.