git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 08/11] streaming_write_entry(): support files with holes

From
Thiago Farina <tfransosi@gmail.com>
Date
May 16, 2011, 13:03 UTC
Message-ID
<BANLkTi=A_vJbJ1v9NDVdh5DMNgTqHQXg6w@mail.gmail.com>
In-Reply-To
<1305505831-31587-9-git-send-email-gitster@pobox.com>
On Sun, May 15, 2011 at 9:30 PM, Junio C Hamano <gitster@pobox.com> wrote:
Show 49 quoted lines
> One typical use of a large binary file is to hold a sparse on-disk hash
> table with a lot of holes. Help preserving the holes with lseek().
>
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
> ---
>  entry.c |   18 +++++++++++++++++-
>  1 files changed, 17 insertions(+), 1 deletions(-)
>
> diff --git a/entry.c b/entry.c
> index 7733a6b..d50e388 100644
> --- a/entry.c
> +++ b/entry.c
> @@ -123,6 +123,7 @@ static int streaming_write_entry(struct cache_entry *ce, char *path,
>        enum object_type type;
>        unsigned long sz;
>        int result = -1;
> +       ssize_t kept = 0;
>        int fd = -1;
>
>        st = open_istream(ce->sha1, &type, &sz);
> @@ -137,17 +138,32 @@ static int streaming_write_entry(struct cache_entry *ce, char *path,
>
>        for (;;) {
>                char buf[10240];
> -               ssize_t wrote;
> +               ssize_t wrote, holeto;
>                ssize_t readlen = read_istream(st, buf, sizeof(buf));
>
>                if (!readlen)
>                        break;
> +               for (holeto = 0; holeto < readlen; holeto++)
> +                       if (buf[holeto])
> +                               break;
> +               if (readlen == holeto) {
> +                       kept += holeto;
> +                       continue;
> +               }
>
> +               if (kept && lseek(fd, kept, SEEK_CUR) == (off_t) -1)
> +                       goto close_and_exit;
> +               else
> +                       kept = 0;
>                wrote = write_in_full(fd, buf, readlen);
>
>                if (wrote != readlen)
>                        goto close_and_exit;
>        }
> +       if (kept &&
> +           ((  lseek(fd, kept - 1, SEEK_CUR) == (off_t) -1)
There is an extra whitespace after ((
Show 13 quoted lines
> +            || write(fd, "", 1) != 1))
> +               goto close_and_exit;
>        *fstat_done = fstat_output(fd, state, statbuf);
>
>  close_and_exit:
> --
> 1.7.5.1.365.g32b65
>
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
Previous: Junio C HamanoNext: Junio C Hamano
Message 19 of 49 in “writing out a huge blob to working tree”
  1. 00/11 writing out a huge blob to working treeJunio C Hamano, May 16, 2011
  2. 01/11 packed_object_info_detail(): do not return a stringJunio C Hamano, May 16, 2011
  3. Thiago FarinaMay 17, 2011
  4. Junio C HamanoMay 17, 2011
  5. 02/11 sha1_object_info_extended(): expose a bit more infoJunio C Hamano, May 16, 2011
  6. 03/11 sha1_object_info_extended(): hint about objects in delta-base cacheJunio C Hamano, May 16, 2011
  7. Shawn PearceMay 16, 2011
  8. 04/11 unpack_object_header(): make it publicJunio C Hamano, May 16, 2011
  9. 05/11 write_entry(): separate two helper functions outJunio C Hamano, May 16, 2011
  10. 06/11 streaming: a new API to read from the object storeJunio C Hamano, May 16, 2011
  11. Jeff KingMay 18, 2011
  12. Junio C HamanoMay 19, 2011
  13. 07/11 streaming_write_entry(): use streaming API in write_entry()Junio C Hamano, May 16, 2011
  14. 08/11 streaming_write_entry(): support files with holesJunio C Hamano, May 16, 2011
  15. Nguyen Thai Ngoc DuyMay 16, 2011
  16. Junio C HamanoMay 16, 2011
  17. Nguyen Thai Ngoc DuyMay 17, 2011
  18. Junio C HamanoMay 17, 2011
  19. Thiago FarinaMay 16, 2011
  20. 09/11 streaming: read non-delta incrementally from a packJunio C Hamano, May 16, 2011
  21. Shawn PearceMay 16, 2011
  22. Junio C HamanoMay 16, 2011
  23. 10/11 sha1_file.c: expose helpers to read loose objectsJunio C Hamano, May 16, 2011
  24. 11/11 streaming: read loose objects incrementallyJunio C Hamano, May 16, 2011
  25. Shawn PearceMay 16, 2011
  26. Jeff KingMay 18, 2011
  27. Jeff KingMay 18, 2011
  28. Jeff KingMay 18, 2011
  29. Junio C HamanoMay 18, 2011
  30. Jeff KingMay 18, 2011
  31. 00/11 writing out a huge blob to working treeJunio C Hamano, May 19, 2011
  32. 01/11 packed_object_info_detail(): do not return a stringJunio C Hamano, May 19, 2011
  33. 02/11 sha1_object_info_extended(): expose a bit more infoJunio C Hamano, May 19, 2011
  34. 03/11 sha1_object_info_extended(): hint about objects in delta-base cacheJunio C Hamano, May 19, 2011
  35. René ScharfeMay 20, 2011
  36. Junio C HamanoMay 21, 2011
  37. 04/11 unpack_object_header(): make it publicJunio C Hamano, May 19, 2011
  38. 05/11 write_entry(): separate two helper functions outJunio C Hamano, May 19, 2011
  39. 06/11 streaming: a new API to read from the object storeJunio C Hamano, May 19, 2011
  40. René ScharfeMay 20, 2011
  41. Junio C HamanoMay 21, 2011
  42. 07/11 streaming_write_entry(): use streaming API in write_entry()Junio C Hamano, May 19, 2011
  43. Junio C HamanoMay 20, 2011
  44. 08/11 streaming_write_entry(): support files with holesJunio C Hamano, May 19, 2011
  45. 09/11 streaming: read non-delta incrementally from a packJunio C Hamano, May 19, 2011
  46. 10/11 sha1_file.c: expose helpers to read loose objectsJunio C Hamano, May 19, 2011
  47. 11/11 streaming: read loose objects incrementallyJunio C Hamano, May 19, 2011
  48. 02/11 interdiffJunio C Hamano, May 19, 2011
  49. Jeff KingMay 19, 2011

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.