{"thread":{"id":"49501","subject":"Understanding Index Header","startedAt":"2018-10-06T21:14:01Z","lastAt":"2018-10-06T22:32:56Z","messageCount":2,"participants":["Farhan Khan","Kevin Daudt"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"359773","messageId":"CAFd4kYC5psSkXxrHVU11b5bufFFVr-qP=S_Fn9AegCyADLw+HA@mail.gmail.com","threadId":"49501","inReplyTo":null,"subject":"Understanding Index Header","fromName":"Farhan Khan","fromEmail":"khanzf@gmail.com","sentAt":"2018-10-06T21:13:45Z","receivedAt":"2018-10-06T21:14:01Z","isPatch":false,"sender":{"key":"khanzf@gmail.com","avatar":null},"body":"Hi all,\n\nI am writing a program that parses out the .git/index file. I am\nreading the git index header documentation, but I seem to be getting\njibberish data.\nhttps://github.com/git/git/blob/master/Documentation/technical/index-format.txt\n\nThe first 12 bytes are the signature, version and entries. Great so far.\n\nAfterwards, I try to read the extension signature, which the\ndocumentation says: \"4-byte extension signature. If the first byte is\n'A'..'Z' the\nextension is optional and can be ignored.\". I am getting jibberish. Is\nthis expected?\n\nThen, the extension size comes back as 35560000 (after ntohl()). That\nseems extremely high. My structure is as follows:\n\nstruct _indexfile_hdr {\n        unsigned char   sig[4];         /* Always \"DIRC\" */\n        uint32_t        version;        /* Version Number */\n        uint32_t        entries;        /* Number of extensions */\n        unsigned char   extsig[4];      /* Extension signature */\n        uint32_t        extsize;        /* Size of the extension */\n        uint8_t         sha[8];         /* SHA1 of index before checksum */\n} __packed;\n\nAm I doing something wrong? Is there some offset or padding that I missed?\n\nThanks,\n--\nFarhan Khan\nPGP Fingerprint: B28D 2726 E2BC A97E 3854 5ABE 9A9F 00BC D525 16EE\n"},{"id":"359774","messageId":"20181006223126.GA3627@alpha","threadId":"49501","inReplyTo":"CAFd4kYC5psSkXxrHVU11b5bufFFVr-qP=S_Fn9AegCyADLw+HA@mail.gmail.com","subject":"Re: Understanding Index Header","fromName":"Kevin Daudt","fromEmail":"me@ikke.info","sentAt":"2018-10-06T22:31:26Z","receivedAt":"2018-10-06T22:32:56Z","isPatch":false,"sender":{"key":"me@ikke.info","avatar":"https://avatars.githubusercontent.com/u/135698?v=4"},"body":"On Sat, Oct 06, 2018 at 05:13:45PM -0400, Farhan Khan wrote:\n> Hi all,\n> \n> I am writing a program that parses out the .git/index file. I am\n> reading the git index header documentation, but I seem to be getting\n> jibberish data.\n> https://github.com/git/git/blob/master/Documentation/technical/index-format.txt\n> \n> The first 12 bytes are the signature, version and entries. Great so far.\n> \n> Afterwards, I try to read the extension signature, which the\n> documentation says: \"4-byte extension signature. If the first byte is\n> 'A'..'Z' the\n> extension is optional and can be ignored.\". I am getting jibberish. Is\n> this expected?\n> \n> Then, the extension size comes back as 35560000 (after ntohl()). That\n> seems extremely high. My structure is as follows:\n> \n> struct _indexfile_hdr {\n>         unsigned char   sig[4];         /* Always \"DIRC\" */\n>         uint32_t        version;        /* Version Number */\n>         uint32_t        entries;        /* Number of extensions */\n>         unsigned char   extsig[4];      /* Extension signature */\n>         uint32_t        extsize;        /* Size of the extension */\n>         uint8_t         sha[8];         /* SHA1 of index before checksum */\n> } __packed;\n> \n> Am I doing something wrong? Is there some offset or padding that I missed?\n> \n> Thanks,\n> --\n> Farhan Khan\n\nHello Farhan\n\nBetween the index header and the extensions there are index entries. So\nyou have 4 sections:\n\n1. Header (fixed size)\n2. Index entries (dynamic size, amount stored in the index)\n3. Extensions (dynamic size)\n4. hash\n\nSo you cannot capture the index with just a single struct. Also what you\ncall the 'entries' field is not the number of extensions, but the number\nof index entries.\n\nI'm not sure why you are seeing so many index entries. I've just tested\nit myself on the git repository and I got 3419 entries when reading in\nnetwork (big-endian) order.\n\nHope this helps a bit.\n\nKevin\n\n"}]}