using git on flash media

I was wondering if anyone here could answer two silly questions arising from a lack of knowledge about the precise architecture of git (unfortunately websearching "using git flash" turns up pages about miserable people and Macromedia):

Is it reasonable to use git reasonably intensively keeping it's database on a flash media drive? (Ie, the flash drive is plugged into a standard desktop machine and the "actual versions" of the file that're being managed are on the machine's hard disc, but all of the git repository data are on a flash media drive. What I'm basically checking is that it doesn't, I dunno, rewrite files so frequently that on a modern flash drive it would wear out the entire drive unreasonably quickly.

Likewise, supposing that there are several machines with the same filesystem tree devoted to the git-chronicled project. Supposing you've got a careful user who checks out the latest tree from the flash drive into the machine's hard disk upon sitting down at one of these machines, and commits to the flash drive & properly umounts the flash before pulling it out. The user does this switching between several machines randomly. (So the git repository on flash acts to ensure that whenever I sit down at a machine I'm presented with my latest versions of everything.) Is there any obvious problem that could come up which could lead to git getting confused and somehow corrupting the archive contents on the flash drive? (Not the kind of catastrophic loss thing that'd be caught by taking regular backups but some corruption that'd silently make parts of the repository un-checkoutable in the main flash repository.) I'm trying to imagine some program somehow getting confused and somehow writing some vital piece of repository data to the "current" hard disc without realising that means it's not "always readable all the time", unlike the flash drive proper.

(I know the answer probably ought to be "just make all the machines networked and communicate via the network rather than a flash drive", but assume I'm not amenable to changing.)

Many thanks for any insight,

cheers, david tweed

Send instant messages to your online friends http://uk.messenger.yahoo.com 

On Thu, 20 Apr 2006, David Tweed wrote:

The largely write-once nature of git should mean that the only files that 
get rewritten a lot are
 - the directories get rewritten to, since git creates new objects at a 
   reasonable pace
 - the branch references get rewritten.

In general, I'd say that git probably does less writing than most other SCM's are likely to do.

That said, when you say "modern flash drive", I really suspect you shouldn't care deeply any more. Modern flash devices can be rewritten a lot more than old ones could (by an order of magnitude or more), and they almost always have wear levelling in hw, making it even less of an issue (but if they don't, your biggest issue will be that you should use a filesystem that does it for you).

That said, if you want to be safe, I think flash memory card vendors guarantee only up to 10,000 write cycles (and it used to be much less).

That's _complete_ rewrites, though, which is more than just a single sector write. They tend to guarantee 100,000 single-sector re-writes (ie more like the "directory update" things when you create a new object).

And assuming you'd count one commit as one "total rewrite" (which sounds unlikely - but it's certainly more than one sector - I don't know what they consider a total rewrite when they make up their numbers), that implies that to be really safe, you shouldn't do more than 10,000 commits before you replace your flash. Quite frankly, I suspect that's _way_ more conservative than you should be, but hey, since you asked..

10,000 commits is actually a fair number. The kernel has gotten 25,000 in a year, but the kernel is a pretty active and large project. I suspect that 10,000 commits is quite a lot of years for most projects.

One rule: NEVER mount your flash with the "sync" option, and use "noatime" to avoid unnecessary inode access time updates (that's especially true for git, where archive atimes aren't interesting, but it's usually a good idea for flash in general). Otherwise you'll get normal accesses ending up doign "writes" too and writes will do a lot more of them, and the above "one commit = one rewrite" rule-of-thumb is suddenly not at all conservative.

Btw, backups are still good. Flash or no flash, and whether you're very conservative in your flash usage or not.

		Linus

On 4/19/06, Linus Torvalds <torvalds@osdl.org> wrote:

Finding one that claims to have HW wear leveling is still hard, so you can never really tell.

When talking about flash, it's the erase cycles that are counted. Most NOR chips guarantee 100,000 erase cycles per eraseblock. NAND chips, which are what most flash drives use, can vary between 10,000 - 100,000 erases depending on the technology used.

A "total rewrite" is simply something that causes an eraseblock to be erased. That can vary alot, but it essentially boils down to needing to write new data to a location in an eraseblock that has already been written to.

To be really conservative, just don't do it ;). Most flash drives don't allow access to the raw flash chips, so while you can use something like JFFS2 on them still, the benefits of that are somewhat lost since you never really know what is going on underneath.

Right.

josh