git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 2/3] read-cache: use get_be32 instead of hand-rolled ntoh_l

From
Jeff King <peff@peff.net>
Date
Jan 23, 2014, 21:26 UTC
Message-ID
<20140123212642.GB21705@sigill.intra.peff.net>
In-Reply-To
<20140123212036.GA21299@sigill.intra.peff.net>

Commit d60c49c (read-cache.c: allow unaligned mapping of the index file, 2012-04-03) introduced helpers to access unaligned data. However, we already have get_be32, which has a few advantages:

  1. It's already written, so we avoid duplication.
  2. It's probably faster, since it does the endian
     conversion and the alignment fix at the same time.
  3. The get_be32 code is well-tested, having been in
     block-sha1 for a long time. By contrast, our custom
     helpers were probably almost never used, since the user
     needed to manually define a macro to enable them.

We have to add a get_be16 implementation to the existing get_be32, but that is very simple to do.

Signed-off-by: Jeff King <peff@peff.net>
---
This _might_ still suffer from the issue fixed in 5f6a112 (block-sha1:
avoid pointer conversion that violates alignment constraints,
2012-07-22), as we are taking the pointer of a uint32 in a struct. But
if that is the case, then the original did, as well. It's not clear to
me if the casting get_be32 does is sufficient, or if a sufficiently
clever compiler might make assumptions based on the original pointer
type.

I'm inclined to leave it for now, as we haven't made anything worse, and nobody has reported a problem.

 compat/bswap.h |  4 ++++
 read-cache.c   | 44 ++++++++++++--------------------------------
 2 files changed, 16 insertions(+), 32 deletions(-)
diff --git a/compat/bswap.h b/compat/bswap.h
index 7d17953..120c6c1 100644
--- a/compat/bswap.h
+++ b/compat/bswap.h
@@ -136,11 +136,15 @@ static inline uint64_t git_bswap64(uint64_t x)
     defined(__powerpc__) || defined(__powerpc64__) || \
     defined(__s390__) || defined(__s390x__)
 
+#define get_be16(p)	ntohs(*(unsigned short *)(p))
 #define get_be32(p)	ntohl(*(unsigned int *)(p))
 #define put_be32(p, v)	do { *(unsigned int *)(p) = htonl(v); } while (0)
 
 #else
 
+#define get_be16(p)	( \
+	(*((unsigned char *)(p) + 0) << 8) | \
+	(*((unsigned char *)(p) + 1) << 0) )
 #define get_be32(p)	( \
 	(*((unsigned char *)(p) + 0) << 24) | \
 	(*((unsigned char *)(p) + 1) << 16) | \
diff --git a/read-cache.c b/read-cache.c
index 33dd676..4221872 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -1313,26 +1313,6 @@ int read_index(struct index_state *istate)
 	return read_index_from(istate, get_index_file());
 }
 
-#ifndef NEEDS_ALIGNED_ACCESS
-#define ntoh_s(var) ntohs(var)
-#define ntoh_l(var) ntohl(var)
-#else
-static inline uint16_t ntoh_s_force_align(void *p)
-{
-	uint16_t x;
-	memcpy(&x, p, sizeof(x));
-	return ntohs(x);
-}
-static inline uint32_t ntoh_l_force_align(void *p)
-{
-	uint32_t x;
-	memcpy(&x, p, sizeof(x));
-	return ntohl(x);
-}
-#define ntoh_s(var) ntoh_s_force_align(&(var))
-#define ntoh_l(var) ntoh_l_force_align(&(var))
-#endif
-
 static struct cache_entry *cache_entry_from_ondisk(struct ondisk_cache_entry *ondisk,
 						   unsigned int flags,
 						   const char *name,
@@ -1340,16 +1320,16 @@ static struct cache_entry *cache_entry_from_ondisk(struct ondisk_cache_entry *on
 {
 	struct cache_entry *ce = xmalloc(cache_entry_size(len));
 
-	ce->ce_stat_data.sd_ctime.sec = ntoh_l(ondisk->ctime.sec);
-	ce->ce_stat_data.sd_mtime.sec = ntoh_l(ondisk->mtime.sec);
-	ce->ce_stat_data.sd_ctime.nsec = ntoh_l(ondisk->ctime.nsec);
-	ce->ce_stat_data.sd_mtime.nsec = ntoh_l(ondisk->mtime.nsec);
-	ce->ce_stat_data.sd_dev   = ntoh_l(ondisk->dev);
-	ce->ce_stat_data.sd_ino   = ntoh_l(ondisk->ino);
-	ce->ce_mode  = ntoh_l(ondisk->mode);
-	ce->ce_stat_data.sd_uid   = ntoh_l(ondisk->uid);
-	ce->ce_stat_data.sd_gid   = ntoh_l(ondisk->gid);
-	ce->ce_stat_data.sd_size  = ntoh_l(ondisk->size);
+	ce->ce_stat_data.sd_ctime.sec = get_be32(&ondisk->ctime.sec);
+	ce->ce_stat_data.sd_mtime.sec = get_be32(&ondisk->mtime.sec);
+	ce->ce_stat_data.sd_ctime.nsec = get_be32(&ondisk->ctime.nsec);
+	ce->ce_stat_data.sd_mtime.nsec = get_be32(&ondisk->mtime.nsec);
+	ce->ce_stat_data.sd_dev   = get_be32(&ondisk->dev);
+	ce->ce_stat_data.sd_ino   = get_be32(&ondisk->ino);
+	ce->ce_mode  = get_be32(&ondisk->mode);
+	ce->ce_stat_data.sd_uid   = get_be32(&ondisk->uid);
+	ce->ce_stat_data.sd_gid   = get_be32(&ondisk->gid);
+	ce->ce_stat_data.sd_size  = get_be32(&ondisk->size);
 	ce->ce_flags = flags & ~CE_NAMEMASK;
 	ce->ce_namelen = len;
 	hashcpy(ce->sha1, ondisk->sha1);
@@ -1389,14 +1369,14 @@ static struct cache_entry *create_from_disk(struct ondisk_cache_entry *ondisk,
 	unsigned int flags;
 
 	/* On-disk flags are just 16 bits */
-	flags = ntoh_s(ondisk->flags);
+	flags = get_be16(&ondisk->flags);
 	len = flags & CE_NAMEMASK;
 
 	if (flags & CE_EXTENDED) {
 		struct ondisk_cache_entry_extended *ondisk2;
 		int extended_flags;
 		ondisk2 = (struct ondisk_cache_entry_extended *)ondisk;
-		extended_flags = ntoh_s(ondisk2->flags2) << 16;
+		extended_flags = get_be16(&ondisk2->flags2) << 16;
 		/* We do not yet understand any bit out of CE_EXTENDED_FLAGS */
 		if (extended_flags & ~CE_EXTENDED_FLAGS)
 			die("Unknown index entry format %08x", extended_flags);
-- 
1.8.5.2.500.g8060133
Previous: Jonathan NiederNext: Jonathan Nieder
Message 39 of 68 in “pack bitmaps”
  1. 0/22 pack bitmapsJeff King, Dec 21, 2013
  2. 01/23 sha1write: make buffer const-correctJeff King, Dec 21, 2013
  3. Christian CouderDec 22, 2013
  4. 02/23 revindex: Export new APIsJeff King, Dec 21, 2013
  5. 03/23 pack-objects: Refactor the packing listJeff King, Dec 21, 2013
  6. 04/23 pack-objects: factor out name_hashJeff King, Dec 21, 2013
  7. 05/23 revision: allow setting custom limiter functionJeff King, Dec 21, 2013
  8. 06/23 sha1_file: export `git_open_noatime`Jeff King, Dec 21, 2013
  9. 07/23 compat: add endianness helpersJeff King, Dec 21, 2013
  10. 08/23 ewah: compressed bitmap implementationJeff King, Dec 21, 2013
  11. Jonathan NiederJan 23, 2014
  12. Jeff KingJan 23, 2014
  13. 1/2 compat: move unaligned helpers to bswap.hJeff King, Jan 23, 2014
  14. Jonathan NiederJan 23, 2014
  15. Jeff KingJan 23, 2014
  16. Jonathan NiederJan 23, 2014
  17. Jeff KingJan 23, 2014
  18. Jonathan NiederJan 23, 2014
  19. Jeff KingJan 23, 2014
  20. 2/2 ewah: support platforms that require aligned readsJeff King, Jan 23, 2014
  21. Jonathan NiederJan 23, 2014
  22. Jeff KingJan 23, 2014
  23. Jonathan NiederJan 23, 2014
  24. Jeff KingJan 23, 2014
  25. Jonathan NiederJan 23, 2014
  26. Jeff KingJan 23, 2014
  27. Jeff KingJan 23, 2014
  28. Shawn PearceJan 23, 2014
  29. Jeff KingJan 23, 2014
  30. brian m. carlsonJan 23, 2014
  31. Jeff KingJan 23, 2014
  32. Jonathan NiederJan 23, 2014
  33. Jeff KingJan 23, 2014
  34. Jonathan NiederJan 23, 2014
  35. Jonathan NiederJan 23, 2014
  36. 0/3 unaligned reads from .bitmap filesJeff King, Jan 23, 2014
  37. 1/3 block-sha1: factor out get_be and put_be wrappersJeff King, Jan 23, 2014
  38. Jonathan NiederJan 23, 2014
  39. 2/3 read-cache: use get_be32 instead of hand-rolled ntoh_lJeff King, Jan 23, 2014
  40. Jonathan NiederJan 23, 2014
  41. Jeff KingJan 24, 2014
  42. 3/3 ewah: support platforms that require aligned readsJeff King, Jan 23, 2014
  43. Jonathan NiederJan 23, 2014
  44. Vicent MartíJan 23, 2014
  45. Jonathan NiederJan 24, 2014
  46. Jonathan NiederJan 23, 2014
  47. 09/23 documentation: add documentation for the bitmap formatJeff King, Dec 21, 2013
  48. 10/23 pack-bitmap: add support for bitmap indexesJeff King, Dec 21, 2013
  49. 11/23 pack-objects: split add_object_entryJeff King, Dec 21, 2013
  50. 12/23 pack-objects: use bitmaps when packing objectsJeff King, Dec 21, 2013
  51. 13/23 rev-list: add bitmap mode to speed up object listsJeff King, Dec 21, 2013
  52. 14/23 pack-objects: implement bitmap writingJeff King, Dec 21, 2013
  53. 15/23 repack: stop using magic number for ARRAY_SIZE(exts)Jeff King, Dec 21, 2013
  54. 16/23 repack: turn exts array into array-of-structJeff King, Dec 21, 2013
  55. 17/23 repack: handle optional files created by pack-objectsJeff King, Dec 21, 2013
  56. 18/23 repack: consider bitmaps when performing repacksJeff King, Dec 21, 2013
  57. 19/23 count-objects: recognize .bitmap in garbage-checkingJeff King, Dec 21, 2013
  58. 20/23 t: add basic bitmap functionality testsJeff King, Dec 21, 2013
  59. 21/23 t/perf: add tests for pack bitmapsJeff King, Dec 21, 2013
  60. 22/23 pack-bitmap: implement optional name_hash cacheJeff King, Dec 21, 2013
  61. 23/23 compat/mingw.h: Fix the MinGW and msvc buildsJeff King, Dec 21, 2013
  62. Erik Faye-LundDec 25, 2013
  63. Jeff KingDec 28, 2013
  64. Vicent MartíDec 28, 2013
  65. Ramsay JonesDec 28, 2013
  66. Jeff KingDec 21, 2013
  67. Jeff KingDec 21, 2013
  68. Thomas RastDec 21, 2013

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.