git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2] packfile: freshen the mtime of packfile by configuration

From
MFMartin Fick <mfick@codeaurora.org>
Date
Jul 14, 2021, 20:20 UTC
Message-ID
<3112447.ymCj9SdLpg@mfick-lnx>
In-Reply-To
<87v95czn7q.fsf@evledraar.gmail.com>
On Wednesday, July 14, 2021 9:41:42 PM MDT you wrote:
Show 44 quoted lines
> On Wed, Jul 14 2021, Martin Fick wrote:
> > On Wednesday, July 14, 2021 8:19:15 PM MDT Ævar Arnfjörð Bjarmason wrote:
> >> The best way to get backups of git repositories you know are correct are
> >> is to use git's own transport mechanisms, i.e. fetch/pull the data, or
> >> create bundles from it.
> > 
> > I don't think this is a fair recommendation since unfortunately, this
> > cannot be used to create a full backup. This can be used to back up the
> > version controlled data, but not the repositories meta-data, i.e.
> > configs, reflogs, alternate setups...
> 
> *nod*
> 
> FWIW at an ex-job I helped systems administrators who'd produced such a
> broken backup-via-rsync create a hybrid version as an interim
> solution. I.e. it would sync the objects via git transport, and do an
> rsync on a whitelist (or blacklist), so pickup config, but exclude
> objects.
> 
> "Hybrid" because it was in a state of needing to deal with manual
> tweaking of config.
> 
> But usually someone who's needing to thoroughly solve this backup
> problem will inevitably end up with wanting to drive everything that's
> not in the object or refstore from some external system, i.e. have
> config be generated from puppet, a database etc., ditto for alternates
> etc.
> 
> But even if you can't get to that point (or don't want to) I'd say aim
> for the hybrid system.
> 
> This isn't some purely theoretical concern b.t.w., the system using
> rsync like this was producing repos that wouldn't fsck all the time, and
> it wasn't such a busy site.
> 
> I suspect (but haven't tried) that for someone who can't easily change
> their backup solution they'd get most of the benefits of git-native
> transport by having their "rsync" sync refs, then objects, not the other
> way around. Glob order dictates that most backup systems will do
> objects, then refs (which will of course, at that point, refer to
> nonexisting objects).
> 
> It's still not safe, you'll still be subject to races, but probably a
> lot better in practice.

It would be great if git provided a command to do a reliable incremental backup, maybe it could copy things in the order that you mention?

However, most people will want to use the backup system they have and not a special git tool. Maybe git fsck should gain a switch that would rewind any refs to an older point that is no broken (using reflogs)? That way, most backups would just work and be rewound to the point at which the backup started?

-Martin
-- 
The Qualcomm Innovation Center, Inc. is a member of Code 
Aurora Forum, hosted by The Linux Foundation


-Martin

-- 
The Qualcomm Innovation Center, Inc. is a member of Code 
Aurora Forum, hosted by The Linux Foundation


-- 
The Qualcomm Innovation Center, Inc. is a member of Code 
Aurora Forum, hosted by The Linux Foundation
Previous: Ævar Arnfjörð BjarmasonNext: Ævar Arnfjörð Bjarmason
Message 12 of 34 in “packfile: enhance the mtime of packfile by idx file”
  1. packfile: enhance the mtime of packfile by idx fileSun Chao via GitGitGadget, Jul 10, 2021
  2. Ævar Arnfjörð BjarmasonJul 11, 2021
  3. Sun ChaoJul 12, 2021
  4. packfile: freshen the mtime of packfile by configurationSun Chao via GitGitGadget, Jul 14, 2021
  5. Ævar Arnfjörð BjarmasonJul 14, 2021
  6. Taylor BlauJul 14, 2021
  7. Sun ChaoJul 14, 2021
  8. Taylor BlauJul 14, 2021
  9. Ævar Arnfjörð BjarmasonJul 14, 2021
  10. Martin FickJul 14, 2021
  11. Ævar Arnfjörð BjarmasonJul 14, 2021
  12. Martin FickJul 14, 2021
  13. Ævar Arnfjörð BjarmasonJul 20, 2021
  14. Son Luong NgocJul 15, 2021
  15. Ævar Arnfjörð BjarmasonJul 20, 2021
  16. Taylor BlauJul 14, 2021
  17. Ævar Arnfjörð BjarmasonJul 14, 2021
  18. Taylor BlauJul 14, 2021
  19. Junio C HamanoJul 14, 2021
  20. Sun ChaoJul 15, 2021
  21. Taylor BlauJul 15, 2021
  22. Sun ChaoJul 15, 2021
  23. Sun ChaoJul 14, 2021
  24. packfile: freshen the mtime of packfile by configurationSun Chao via GitGitGadget, Jul 19, 2021
  25. Taylor BlauJul 19, 2021
  26. Junio C HamanoJul 20, 2021
  27. Sun ChaoJul 20, 2021
  28. Ævar Arnfjörð BjarmasonJul 20, 2021
  29. Sun ChaoJul 20, 2021
  30. Sun ChaoJul 20, 2021
  31. Taylor BlauJul 20, 2021
  32. 0/2 packfile: freshen the mtime of packfile by configurationSun Chao via GitGitGadget, Aug 15, 2021
  33. 1/2 packfile: rename `derive_filename()` to `derive_pack_filename()`Sun Chao via GitGitGadget, Aug 15, 2021
  34. 2/2 packfile: freshen the mtime of packfile by bump fileSun Chao via GitGitGadget, Aug 15, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.