threads / discuss / 63588

Fetch remote only if remote hasn't been fetched recently

Subject: Fetch remote only if remote hasn't been fetched recently

## tl;dr

8 messages between Jun 5, 2025 and Jul 1, 2025.

replies: 7people: 3as markdown or json

M Hickford· Jun 5, 2025, 11:16 UTC · lore

Hi. I'd like to fetch from a particular remote, but only if that remote hasn't been fetched in the last hour. How could I achieve this? Is there a relevant option for `git fetch`?

https://git-scm.com/docs/git-fetch
Konstantin Ryabitsev· Jun 5, 2025, 14:02 UTC · re: M Hickford · lore

Re: Fetch remote only if remote hasn't been fetched recently

On Thu, Jun 05, 2025 at 12:16:26PM +0100, M Hickford wrote:
> Hi. I'd like to fetch from a particular remote, but only if that
> remote hasn't been fetched in the last hour. How could I achieve this?
> Is there a relevant option for `git fetch`?

Not to my knowledge, but you can either look at the timestamp of FETCH_HEAD (if that's the only remote you're fetching), or wrap git-fetch in a small script that touches .git/last-fetched-foo file and you can then check that file and exit if it's newer than 1 hour ago.

-K
Junio C Hamano· Jun 5, 2025, 16:55 UTC · re: M Hickford · lore

Re: Fetch remote only if remote hasn't been fetched recently

M Hickford <mirth.hickford@gmail.com> writes:
Show 5 quoted lines
> Hi. I'd like to fetch from a particular remote, but only if that
> remote hasn't been fetched in the last hour. How could I achieve this?
> Is there a relevant option for `git fetch`?
>
> https://git-scm.com/docs/git-fetch

"git fetch" is "I want to fetch now". If you want to pace your fetches, you have to keep a record of when you fetched in the past and work from there.

I sense there perhaps is an XY problem?

If "git fetch" is done outside end-user's supervision and explicit intent, the remote-tracking branches will become much less useful to the human users. A good solusion that avoids this issue already exists as the "prefetch" task of the "git maintenance" suite.

M Hickford· Jun 7, 2025, 07:00 UTC · re: Junio C Hamano · lore

Re: Fetch remote only if remote hasn't been fetched recently

On Thu, 5 Jun 2025 at 17:55, Junio C Hamano <gitster@pobox.com> wrote:
Show 19 quoted lines
>
> M Hickford <mirth.hickford@gmail.com> writes:
>
> > Hi. I'd like to fetch from a particular remote, but only if that
> > remote hasn't been fetched in the last hour. How could I achieve this?
> > Is there a relevant option for `git fetch`?
> >
> > https://git-scm.com/docs/git-fetch
>
> "git fetch" is "I want to fetch now".  If you want to pace your
> fetches, you have to keep a record of when you fetched in the past
> and work from there.
>
> I sense there perhaps is an XY problem?
>
> If "git fetch" is done outside end-user's supervision and explicit
> intent, the remote-tracking branches will become much less useful to
> the human users.  A good solusion that avoids this issue already
> exists as the "prefetch" task of the "git maintenance" suite.
Interesting, thank you.

Suppose my network connection is intermittent. While the machine is online, `git fetch --prefetch` runs successfully. Later I am offline. How can I complete the fetch?

Junio C Hamano· Jun 9, 2025, 14:08 UTC · re: M Hickford · lore

Re: Fetch remote only if remote hasn't been fetched recently

M Hickford <mirth.hickford@gmail.com> writes:
Show 5 quoted lines
> Interesting, thank you.
>
> Suppose my network connection is intermittent. While the machine is
> online, `git fetch --prefetch` runs successfully. Later I am offline.
> How can I complete the fetch?

I _think_ that it uses "git fetch --prefetch" to store what is usually stored at refs/$R to refs/prefetch/$R (so your remote-tracking branch refs/remotes/origin/frotz is stored at refs/prefetch/remotes/origin/frotz instead), so I would presume that it should be the matter of doing something silly like this:

    $ git fetch . 'refs/prefetch/remotes/origin/*:refs/remotes/origin/*'
M Hickford· Jun 11, 2025, 07:00 UTC · re: Junio C Hamano · lore

Re: Fetch remote only if remote hasn't been fetched recently

On Mon, 9 Jun 2025 at 15:08, Junio C Hamano <gitster@pobox.com> wrote:
Show 17 quoted lines
>
> M Hickford <mirth.hickford@gmail.com> writes:
>
> > Interesting, thank you.
> >
> > Suppose my network connection is intermittent. While the machine is
> > online, `git fetch --prefetch` runs successfully. Later I am offline.
> > How can I complete the fetch?
>
> I _think_ that it uses "git fetch --prefetch" to store what is
> usually stored at refs/$R to refs/prefetch/$R (so your
> remote-tracking branch refs/remotes/origin/frotz is stored at
> refs/prefetch/remotes/origin/frotz instead), so I would presume that
> it should be the matter of doing something silly like this:
>
>     $ git fetch . 'refs/prefetch/remotes/origin/*:refs/remotes/origin/*'
>
Thanks, that worked.

To support Git users with sporadic connectivity, might it be useful to add an option to git fetch? Perhaps `git fetch --offline` or `git fetch --complete-prefetch`?

https://git-scm.com/docs/git-fetch
Junio C Hamano· Jun 11, 2025, 15:23 UTC · re: M Hickford · lore

Re: Fetch remote only if remote hasn't been fetched recently

M Hickford <mirth.hickford@gmail.com> writes:
Show 8 quoted lines
>>     $ git fetch . 'refs/prefetch/remotes/origin/*:refs/remotes/origin/*'
>>
>
> Thanks, that worked.
>
> To support Git users with sporadic connectivity, might it be useful to
> add an option to git fetch? Perhaps  `git fetch --offline` or `git
> fetch --complete-prefetch`?

"offline catch-up" is probably the phrase I would prefer to see in the documentation page to explain the concept of the operation than "complete prefetch".

"complete prefetch" sounds like an oxymoron, in that the prefetch has already been completed long ago, and the operation being proposed is more about using the result of that operation completed long ago to get yourself closer to the present state of the remote without any guarantees that you would end up being close enough.

In any case, to present it as a first-class option to end-users, there needs a lot more thinking and a bit of work need to go into the way "prefetch" works. For example, the longhand I gave would work as a good solution only when the user _knows_ that the prefetch is more recent than their actual remote-tracking branches. If refs/prefetch/remotes/R/* is older, then you wouldn't be bringing yourself closer to the present state of the remote at all with such a fetch, but as far as I know, a normal "git fetch R" from the remote R would not clear refs/prefetch/remotes/R/* when it completes. There is no "last time we fetched from there" record kept anywhere in the repository either. So offhand, you'd at least need to do these:

 - teach "git fetch" that updates remote-tracking branches to remove
   the corresponding ref in the refs/prefetch/ hierarchy;
 - decide on what option name to use for this new operation and
   document it;
 - implement the option "git fetch --that-option nickname" to
   - figure out the remote-tracking hierarchy for nickname and call
     it R
   - when refs/prefetch/remotes/R/* exists, do an
     $ git fetch . "+refs/prefetch/remotes/R/*:refs/remotes/R/*"
M Hickford· Jul 1, 2025, 07:00 UTC · re: Junio C Hamano · lore

Re: Fetch remote only if remote hasn't been fetched recently

On Wed, 11 Jun 2025 at 16:23, Junio C Hamano <gitster@pobox.com> wrote:
Show 10 quoted lines
>
> M Hickford <mirth.hickford@gmail.com> writes:
>
> > To support Git users with sporadic connectivity, might it be useful to
> > add an option to git fetch? Perhaps  `git fetch --offline` or `git
> > fetch --complete-prefetch`?
>
> "offline catch-up" is probably the phrase I would prefer to see in
> the documentation page to explain the concept of the operation than
> "complete prefetch".
That makes sense to me.
Show 28 quoted lines
> In any case, to present it as a first-class option to end-users,
> there needs a lot more thinking and a bit of work need to go into
> the way "prefetch" works.  For example, the longhand I gave would
> work as a good solution only when the user _knows_ that the prefetch
> is more recent than their actual remote-tracking branches.  If
> refs/prefetch/remotes/R/* is older, then you wouldn't be bringing
> yourself closer to the present state of the remote at all with such
> a fetch, but as far as I know, a normal "git fetch R" from the
> remote R would not clear refs/prefetch/remotes/R/* when it
> completes.  There is no "last time we fetched from there" record
> kept anywhere in the repository either.  So offhand, you'd at least
> need to do these:
>
>  - teach "git fetch" that updates remote-tracking branches to remove
>    the corresponding ref in the refs/prefetch/ hierarchy;
>
>  - decide on what option name to use for this new operation and
>    document it;
>
>  - implement the option "git fetch --that-option nickname" to
>
>    - figure out the remote-tracking hierarchy for nickname and call
>      it R
>
>    - when refs/prefetch/remotes/R/* exists, do an
>
>      $ git fetch . "+refs/prefetch/remotes/R/*:refs/remotes/R/*"
>
Thanks Junio for your insight.

I shall leave this as a feature request, perhaps suitable for a microproject #leftoverbits

← back to recent threads