git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 15/23] http.c: new functions for the http API

From
Tay Ray Chuan <rctay89@gmail.com>
Date
Jun 6, 2009, 08:43 UTC
Message-ID
<20090606164353.967cce80.rctay89@gmail.com>
In-Reply-To
<cover.1244277116.git.rctay89@gmail.com>
From:	Mike Hommey <mh@glandium.org>
The new functions added are:
 - http_request() (internal function)
 - http_get_strbuf()
 - http_get_file()
 - http_error()

http_get_strbuf and http_get_file allow respectively to retrieve contents of an URL to a strbuf or an opened file handle.

http_error prints out an error message containing the URL and the curl error (in curl_errorstr).

Signed-off-by: Mike Hommey <mh@glandium.org>
Signed-off-by: Tay Ray Chuan <rctay89@gmail.com>
---
This patch is based on Mike's earlier patch:
Subject: [WIP Patch 03/12] Two new functions for the http API
Date:	Sun, 18 Jan 2009 09:04:28 +0100

I've added a new return status code for http_request, HTTP_START_FAILED, since some callers need to know when start_active_slot() fails.

http_request() will emit an error message when start_active_slot() fails; for other cases, no error message is emitted.

I've added http_error(). (see description above)

I've added the list of new functions to the commit message body, as suggested by Jakub.

 http.c |  104 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 http.h |   30 ++++++++++++++++++
 2 files changed, 134 insertions(+), 0 deletions(-)
diff --git a/http.c b/http.c
index 75fce9e..df22180 100644
--- a/http.c
+++ b/http.c
@@ -665,6 +665,110 @@ static char *quote_ref_url(const char *base, const char *ref)
 	return strbuf_detach(&buf, NULL);
 }

+/* http_request() targets */
+#define HTTP_REQUEST_STRBUF	0
+#define HTTP_REQUEST_FILE	1
+
+static int http_request(const char *url, void *result, int target, int options)
+{
+	struct active_request_slot *slot;
+	struct slot_results results;
+	struct curl_slist *headers = NULL;
+	struct strbuf buf = STRBUF_INIT;
+	int ret;
+
+	slot = get_active_slot();
+	slot->results = &results;
+	curl_easy_setopt(slot->curl, CURLOPT_HTTPGET, 1);
+
+	if (result == NULL) {
+		curl_easy_setopt(slot->curl, CURLOPT_NOBODY, 1);
+	} else {
+		curl_easy_setopt(slot->curl, CURLOPT_NOBODY, 0);
+		curl_easy_setopt(slot->curl, CURLOPT_FILE, result);
+
+		if (target == HTTP_REQUEST_FILE) {
+			long posn = ftell(result);
+			curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION,
+					 fwrite);
+			if (posn > 0) {
+				strbuf_addf(&buf, "Range: bytes=%ld-", posn);
+				headers = curl_slist_append(headers, buf.buf);
+				strbuf_reset(&buf);
+			}
+			slot->local = result;
+		} else
+			curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION,
+					 fwrite_buffer);
+	}
+
+	strbuf_addstr(&buf, "Pragma:");
+	if (options & HTTP_NO_CACHE)
+		strbuf_addstr(&buf, " no-cache");
+
+	headers = curl_slist_append(headers, buf.buf);
+
+	curl_easy_setopt(slot->curl, CURLOPT_URL, url);
+	curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, headers);
+
+	if (start_active_slot(slot)) {
+		run_active_slot(slot);
+		if (results.curl_result == CURLE_OK)
+			ret = HTTP_OK;
+		else if (missing_target(&results))
+			ret = HTTP_MISSING_TARGET;
+		else
+			ret = HTTP_ERROR;
+	} else {
+		error("Unable to start HTTP request for %s", url);
+		ret = HTTP_START_FAILED;
+	}
+
+	slot->local = NULL;
+	curl_slist_free_all(headers);
+	strbuf_release(&buf);
+
+	return ret;
+}
+
+int http_get_strbuf(const char *url, struct strbuf *result, int options)
+{
+	return http_request(url, result, HTTP_REQUEST_STRBUF, options);
+}
+
+int http_get_file(const char *url, const char *filename, int options)
+{
+	int ret;
+	struct strbuf tmpfile = STRBUF_INIT;
+	FILE *result;
+
+	strbuf_addf(&tmpfile, "%s.temp", filename);
+	result = fopen(tmpfile.buf, "a");
+	if (! result) {
+		error("Unable to open local file %s", tmpfile.buf);
+		ret = HTTP_ERROR;
+		goto cleanup;
+	}
+
+	ret = http_request(url, result, HTTP_REQUEST_FILE, options);
+	fclose(result);
+
+	if ((ret == HTTP_OK) && move_temp_to_file(tmpfile.buf, filename))
+		ret = HTTP_ERROR;
+cleanup:
+	strbuf_release(&tmpfile);
+	return ret;
+}
+
+int http_error(const char *url, int ret)
+{
+	/* http_request has already handled HTTP_START_FAILED. */
+	if (ret != HTTP_START_FAILED)
+		error("%s while accessing %s\n", curl_errorstr, url);
+
+	return ret;
+}
+
 int http_fetch_ref(const char *base, struct ref *ref)
 {
 	char *url;
diff --git a/http.h b/http.h
index 1ef7dc1..3d878d5 100644
--- a/http.h
+++ b/http.h
@@ -114,6 +114,36 @@ static inline int missing__target(int code, int result)

 #define missing_target(a) missing__target((a)->http_code, (a)->curl_result)

+/* Options for http_request_*() */
+#define HTTP_NO_CACHE		1
+
+/* Return values for http_request_*() */
+#define HTTP_OK			0
+#define HTTP_MISSING_TARGET	1
+#define HTTP_ERROR		2
+#define HTTP_START_FAILED	3
+
+/*
+ * Requests an url and stores the result in a strbuf.
+ *
+ * If the result pointer is NULL, a HTTP HEAD request is made instead of GET.
+ */
+int http_get_strbuf(const char *url, struct strbuf *result, int options);
+
+/*
+ * Downloads an url and stores the result in the given file.
+ *
+ * If a previous interrupted download is detected (i.e. a previous temporary
+ * file is still around) the download is resumed.
+ */
+int http_get_file(const char *url, const char *filename, int options);
+
+/*
+ * Prints an error message using error() containing url and curl_errorstr,
+ * and returns ret.
+ */
+int http_error(const char *url, int ret);
+
 extern int http_fetch_ref(const char *base, struct ref *ref);

 #endif /* HTTP_H */
--
1.6.3.1
Previous: Tay Ray ChuanNext: Tay Ray Chuan
Message 15 of 23 in “t5540-http-push: test fetching of loose objects”
  1. 01/23 t5540-http-push: test fetching of loose objectsTay Ray Chuan, Jun 6, 2009
  2. 02/23 t5540-http-push: test fetching of packed objectsTay Ray Chuan, Jun 6, 2009
  3. 03/23 http*: cleanup slot->local after fcloseTay Ray Chuan, Jun 6, 2009
  4. 04/23 http-push: send out fetch requests on queueTay Ray Chuan, Jun 6, 2009
  5. 05/23 http-push: fix missing "#ifdef USE_CURL_MULTI" around "is_running_queue"Tay Ray Chuan, Jun 6, 2009
  6. 06/23 t5550-http-fetch: test fetching of packed objectsTay Ray Chuan, Jun 6, 2009
  7. 07/23 http-push, http-walker: style fixesTay Ray Chuan, Jun 6, 2009
  8. 08/23 http-walker: verify remote packsTay Ray Chuan, Jun 6, 2009
  9. 09/23 http*: copy string returned by sha1_to_hexTay Ray Chuan, Jun 6, 2009
  10. 10/23 http-push: do not SEGV after fetching a bad pack idx fileTay Ray Chuan, Jun 6, 2009
  11. 11/23 Don't expect verify_pack() callers to set pack_sizeTay Ray Chuan, Jun 6, 2009
  12. 12/23 transport.c::get_refs_via_curl(): do not leak refs_urlTay Ray Chuan, Jun 6, 2009
  13. 13/23 http*: move common variables and macros to http.[ch]Tay Ray Chuan, Jun 6, 2009
  14. 14/23 http: create function end_url_with_slashTay Ray Chuan, Jun 6, 2009
  15. 15/23 http.c: new functions for the http APITay Ray Chuan, Jun 6, 2009
  16. 16/23 transport.c::get_refs_via_curl(): use the new http APITay Ray Chuan, Jun 6, 2009
  17. 17/23 http.c::http_fetch_ref(): use the new http APITay Ray Chuan, Jun 6, 2009
  18. 18/23 http-push.c::remote_exists(): use the new http APITay Ray Chuan, Jun 6, 2009
  19. 19/23 http-push.c::fetch_symref(): use the new http APITay Ray Chuan, Jun 6, 2009
  20. 20/23 http*: add http_get_info_packsTay Ray Chuan, Jun 6, 2009
  21. 21/23 http: use new http API in fetch_index()Tay Ray Chuan, Jun 6, 2009
  22. 22/23 http*: add helper methods for fetching packsTay Ray Chuan, Jun 6, 2009
  23. 23/23 http*: add helper methods for fetching objects (loose)Tay Ray Chuan, Jun 6, 2009

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.