{"thread":{"id":"63","subject":"[PATCH] Use libcurl to use HTTP to get repositories","startedAt":"2005-04-17T00:14:08Z","lastAt":"2005-04-17T04:55:59Z","messageCount":6,"participants":["Daniel Barkalow","Paul Jackson","Linus Torvalds","Ingo Molnar"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"400","messageId":"Pine.LNX.4.21.0504162008190.30848-100000@iabervon.org","threadId":"63","inReplyTo":null,"subject":"[PATCH] Use libcurl to use HTTP to get repositories","fromName":"Daniel Barkalow","fromEmail":"barkalow@iabervon.org","sentAt":"2005-04-17T00:14:08Z","receivedAt":"2005-04-17T00:14:08Z","isPatch":true,"sender":{"key":"barkalow@iabervon.org","avatar":"https://avatars.githubusercontent.com/u/55364219?v=4"},"body":"This enables the use of HTTP to download commits and associated objects\nfrom remote repositories. It now uses libcurl instead of local hack code.\n\nStill causes warnings for fsck-cache and rev-tree, due to unshared code.\n\nStill leaks a bit of memory due to bug copied from read-tree.\n\nNeeds libcurl post 7.7 or so.\n\nSigned-Off-By: Daniel Barkalow <barkalow@iabervon.org>\n\nIndex: Makefile\n===================================================================\n--- ed4f6e454b40650b904ab72048b2f93a068dccc3/Makefile  (mode:100644 sha1:b39b4ea37586693dd707d1d0750a9b580350ec50)\n+++ d332a8ddffb50c1247491181af458970bf639942/Makefile  (mode:100644 sha1:ca5dfd41b750cb1339128e4431afbbbc21bf57bb)\n@@ -14,7 +14,7 @@\n \n PROG=   update-cache show-diff init-db write-tree read-tree commit-tree \\\n \tcat-file fsck-cache checkout-cache diff-tree rev-tree show-files \\\n-\tcheck-files ls-tree merge-tree\n+\tcheck-files ls-tree merge-tree http-get\n \n all: $(PROG)\n \n@@ -23,6 +23,11 @@\n \n LIBS= -lssl -lz\n \n+http-get: LIBS += -lcurl\n+\n+http-get:%:%.o read-cache.o\n+\t$(CC) $(CFLAGS) -o $@ $^ $(LIBS)\n+\n init-db: init-db.o\n \n update-cache: update-cache.o read-cache.o\nIndex: http-get.c\n===================================================================\n--- /dev/null  (tree:ed4f6e454b40650b904ab72048b2f93a068dccc3)\n+++ d332a8ddffb50c1247491181af458970bf639942/http-get.c  (mode:100644 sha1:106ca31239e6afe6784e7c592234406f5c149e44)\n@@ -0,0 +1,126 @@\n+#include <fcntl.h>\n+#include <unistd.h>\n+#include <string.h>\n+#include <stdlib.h>\n+#include \"cache.h\"\n+#include \"revision.h\"\n+#include <errno.h>\n+#include <stdio.h>\n+\n+#include <curl/curl.h>\n+#include <curl/easy.h>\n+\n+static CURL *curl;\n+\n+static char *base;\n+\n+static int fetch(unsigned char *sha1)\n+{\n+\tchar *hex = sha1_to_hex(sha1);\n+\tchar *filename = sha1_file_name(sha1);\n+\n+\tchar *url;\n+\tchar *posn;\n+\tFILE *local;\n+\tstruct stat st;\n+\n+\tif (!stat(filename, &st)) {\n+\t\treturn 0;\n+\t}\n+\n+\tlocal = fopen(filename, \"w\");\n+\n+\tif (!local) {\n+\t\tfprintf(stderr, \"Couldn't open %s\\n\", filename);\n+\t\treturn -1;\n+\t}\n+\n+\tcurl_easy_setopt(curl, CURLOPT_FILE, local);\n+\n+\turl = malloc(strlen(base) + 50);\n+\tstrcpy(url, base);\n+\tposn = url + strlen(base);\n+\tstrcpy(posn, \"objects/\");\n+\tposn += 8;\n+\tmemcpy(posn, hex, 2);\n+\tposn += 2;\n+\t*(posn++) = '/';\n+\tstrcpy(posn, hex + 2);\n+\n+\tcurl_easy_setopt(curl, CURLOPT_URL, url);\n+\n+\tcurl_easy_perform(curl);\n+\n+\tfclose(local);\n+\t\n+\treturn 0;\n+}\n+\n+static int process_tree(unsigned char *sha1)\n+{\n+\tvoid *buffer;\n+        unsigned long size;\n+        char type[20];\n+\n+        buffer = read_sha1_file(sha1, type, &size);\n+\tif (!buffer)\n+\t\treturn -1;\n+\tif (strcmp(type, \"tree\"))\n+\t\treturn -1;\n+\twhile (size) {\n+\t\tint len = strlen(buffer) + 1;\n+\t\tunsigned char *sha1 = buffer + len;\n+\t\tunsigned int mode;\n+\t\tint retval;\n+\n+\t\tif (size < len + 20 || sscanf(buffer, \"%o\", &mode) != 1)\n+\t\t\treturn -1;\n+\n+\t\tbuffer = sha1 + 20;\n+\t\tsize -= len + 20;\n+\n+\t\tretval = fetch(sha1);\n+\t\tif (retval)\n+\t\t\treturn -1;\n+\n+\t\tif (S_ISDIR(mode)) {\n+\t\t\tretval = process_tree(sha1);\n+\t\t\tif (retval)\n+\t\t\t\treturn -1;\n+\t\t}\n+\t}\n+\treturn 0;\n+}\n+\n+static int process_commit(unsigned char *sha1)\n+{\n+\tstruct revision *rev = lookup_rev(sha1);\n+\tif (parse_commit_object(rev))\n+\t\treturn -1;\n+\t\n+\tfetch(rev->tree);\n+\tprocess_tree(rev->tree);\n+\treturn 0;\n+}\n+\n+int main(int argc, char **argv)\n+{\n+\tchar *commit_id = argv[1];\n+\tchar *url = argv[2];\n+\n+\tunsigned char sha1[20];\n+\n+\tget_sha1_hex(commit_id, sha1);\n+\n+\tcurl_global_init(CURL_GLOBAL_ALL);\n+\n+\tcurl = curl_easy_init();\n+\n+\tbase = url;\n+\n+\tfetch(sha1);\n+\tprocess_commit(sha1);\n+\n+\tcurl_global_cleanup();\n+\treturn 0;\n+}\nIndex: revision.h\n===================================================================\n--- ed4f6e454b40650b904ab72048b2f93a068dccc3/revision.h  (mode:100664 sha1:28d0de3261a61f68e4e0948a25a416a515cd2e83)\n+++ d332a8ddffb50c1247491181af458970bf639942/revision.h  (mode:100664 sha1:523bde6e14e18bb0ecbded8f83ad4df93fc467ab)\n@@ -24,6 +24,7 @@\n \tunsigned int flags;\n \tunsigned char sha1[20];\n \tunsigned long date;\n+\tunsigned char tree[20];\n \tstruct parent *parent;\n };\n \n@@ -111,4 +112,29 @@\n \t}\n }\n \n+static int parse_commit_object(struct revision *rev)\n+{\n+\tif (!(rev->flags & SEEN)) {\n+\t\tvoid *buffer, *bufptr;\n+\t\tunsigned long size;\n+\t\tchar type[20];\n+\t\tunsigned char parent[20];\n+\n+\t\trev->flags |= SEEN;\n+\t\tbuffer = bufptr = read_sha1_file(rev->sha1, type, &size);\n+\t\tif (!buffer || strcmp(type, \"commit\"))\n+\t\t\treturn -1;\n+\t\tget_sha1_hex(bufptr + 5, rev->tree);\n+\t\tbufptr += 46; /* \"tree \" + \"hex sha1\" + \"\\n\" */\n+\t\twhile (!memcmp(bufptr, \"parent \", 7) && \n+\t\t       !get_sha1_hex(bufptr+7, parent)) {\n+\t\t\tadd_relationship(rev, parent);\n+\t\t\tbufptr += 48;   /* \"parent \" + \"hex sha1\" + \"\\n\" */\n+\t\t}\n+\t\t//rev->date = parse_commit_date(bufptr);\n+\t\tfree(buffer);\n+\t}\n+\treturn 0;\n+}\n+\n #endif /* REVISION_H */\n\n"},{"id":"401","messageId":"20050416172952.3bf34bab.pj@sgi.com","threadId":"63","inReplyTo":"Pine.LNX.4.21.0504162008190.30848-100000@iabervon.org","subject":"Re: [PATCH] Use libcurl to use HTTP to get repositories","fromName":"Paul Jackson","fromEmail":"pj@sgi.com","sentAt":"2005-04-17T00:29:52Z","receivedAt":"2005-04-17T00:29:52Z","isPatch":true,"sender":{"key":"pj@sgi.com","avatar":null},"body":"> Needs libcurl post 7.7 or so.\n\nThat could be mentioned in the README, which has a list of 'Software\nrequirements.'  Actually, zlib-devel and openssl should be on this list\nas well.  My laziness got in the way of my sending in a patch for that.\n\n-- \n                  I won't rest till it's the best ...\n                  Programmer, Linux Scalability\n                  Paul Jackson <pj@engr.sgi.com> 1.650.933.1373, 1.925.600.0401\n"},{"id":"404","messageId":"Pine.LNX.4.21.0504162033060.30848-100000@iabervon.org","threadId":"63","inReplyTo":"20050416172952.3bf34bab.pj@sgi.com","subject":"Re: [PATCH] Use libcurl to use HTTP to get repositories","fromName":"Daniel Barkalow","fromEmail":"barkalow@iabervon.org","sentAt":"2005-04-17T00:42:13Z","receivedAt":"2005-04-17T00:42:13Z","isPatch":true,"sender":{"key":"barkalow@iabervon.org","avatar":"https://avatars.githubusercontent.com/u/55364219?v=4"},"body":"On Sat, 16 Apr 2005, Paul Jackson wrote:\n\n> > Needs libcurl post 7.7 or so.\n> \n> That could be mentioned in the README, which has a list of 'Software\n> requirements.'  Actually, zlib-devel and openssl should be on this list\n> as well.  My laziness got in the way of my sending in a patch for that.\n\nI'm working off of Linus's tree when not working on scripts, and it\ndoesn't have that section at all. Should I submit patches to the pasky\ntree? (I'm actually tempted to have my own public tree, now that I\nwouldn't have to set up anonymous rsync access, aside from the\nbootstrapping issue; then I could let people pull and merge instead of\napplying patches.)\n\n\t-Daniel\n*This .sig left intentionally blank*\n\n"},{"id":"406","messageId":"20050416174544.5ca28322.pj@sgi.com","threadId":"63","inReplyTo":"Pine.LNX.4.21.0504162033060.30848-100000@iabervon.org","subject":"Re: [PATCH] Use libcurl to use HTTP to get repositories","fromName":"Paul Jackson","fromEmail":"pj@sgi.com","sentAt":"2005-04-17T00:45:44Z","receivedAt":"2005-04-17T00:45:44Z","isPatch":true,"sender":{"key":"pj@sgi.com","avatar":null},"body":"Daniel wrote:\n> I'm working off of Linus's tree when not working on scripts, and it\n> doesn't have that section at all.\n\nAh so - nevermind my README comments then.\n\n-- \n                  I won't rest till it's the best ...\n                  Programmer, Linux Scalability\n                  Paul Jackson <pj@engr.sgi.com> 1.650.933.1373, 1.925.600.0401\n"},{"id":"431","messageId":"Pine.LNX.4.58.0504162135420.7211@ppc970.osdl.org","threadId":"63","inReplyTo":"20050416174544.5ca28322.pj@sgi.com","subject":"Re: [PATCH] Use libcurl to use HTTP to get repositories","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2005-04-17T04:36:46Z","receivedAt":"2005-04-17T04:36:46Z","isPatch":true,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Sat, 16 Apr 2005, Paul Jackson wrote:\n\n> Daniel wrote:\n> > I'm working off of Linus's tree when not working on scripts, and it\n> > doesn't have that section at all.\n> \n> Ah so - nevermind my README comments then.\n\nWell, actually, I suspect that something like this should go to Pasky. I\nreally see my repo as purely a \"internal git datastructures\", and when it\ngets to \"how do we interact with other peoples web-sites\", I suspect \nPasky's tree is better.\n\n\t\tLinus\n"},{"id":"436","messageId":"20050417045559.GA769@elte.hu","threadId":"63","inReplyTo":"Pine.LNX.4.21.0504162008190.30848-100000@iabervon.org","subject":"Re: [PATCH] Use libcurl to use HTTP to get repositories","fromName":"Ingo Molnar","fromEmail":"mingo@elte.hu","sentAt":"2005-04-17T04:55:59Z","receivedAt":"2005-04-17T04:55:59Z","isPatch":true,"sender":{"key":"mingo@elte.hu","avatar":null},"body":"\n* Daniel Barkalow <barkalow@iabervon.org> wrote:\n\n> Still leaks a bit of memory due to bug copied from read-tree.\n\nLinus, should i resend the 18 fixes i sent the other day? (as a GIT \nrepository perhaps?) I found roughly 6 common memory leaks, 8 \ntheoretical memory leaks, 2 overflows and did a couple of cleanups. One \nof the patches [the cache collision related thing] we agreed was not \nneeded, the rest is still very much valid i think. I did some basic \ntesting with the fixes applied, nothing seemed to break in any visible \nway in these tests.\n\n\tIngo\n"}]}