# Limiting disk usage 7 messages from 2013-10-29 to 2013-10-30. Participants: Piotr Krukowiecki, Matthieu Moy, Jakub Narębski, Fredrik Gustafsson. Thread: https://gitlist.dev/t/35231 ## Piotr Krukowiecki, 2013-10-29 06:22 Subject: Limiting disk usage Message-ID: <0a4899b7-f75f-4016-9c99-5e6e3fb9603c@email.android.com> URL: https://gitlist.dev/e/0a4899b7-f75f-4016-9c99-5e6e3fb9603c%40email.android.com ``` Hi, What can be done to limit amount of space occupied on clone and checkout? I know about shallow clone and sparse checkout, anything else? Preferably I would split repository into multiple repositories but I worry that working with multiple directories would be much more troublesome. But maybe I am wrong? I don't have much experience with submodules, repo tool (what else?) Thanks -- Piotr Krukowiecki ``` ## Matthieu Moy, 2013-10-29 08:21 Subject: Re: Limiting disk usage Message-ID: URL: https://gitlist.dev/e/vpqob68ebcu.fsf%40anie.imag.fr In-Reply-To: <0a4899b7-f75f-4016-9c99-5e6e3fb9603c@email.android.com> ``` Piotr Krukowiecki writes: > Hi, > > What can be done to limit amount of space occupied on clone and > checkout? I know about shallow clone and sparse checkout, anything > else? Sorry if I'm stating the obvious, but "git gc" is a good start. If you did something wrong with your repo before, maybe "git gc --aggressive" can help. -- Matthieu Moy http://www-verimag.imag.fr/~moy/ ``` ## Piotr Krukowiecki, 2013-10-29 13:50 Subject: Re: Limiting disk usage Message-ID: URL: https://gitlist.dev/e/CAA01Cso%2ByTzatiAaQahx5h2N0nnOON7FsnWNSYrN-pV%3D8qzn7g%40mail.gmail.com In-Reply-To: ``` [resending as sent previous version as HTML, sorry about that] On Tue, Oct 29, 2013 at 9:21 AM, Matthieu Moy wrote: > > Piotr Krukowiecki writes: > > > Hi, > > > > What can be done to limit amount of space occupied on clone and > > checkout? I know about shallow clone and sparse checkout, anything > > else? > > Sorry if I'm stating the obvious, but "git gc" is a good start. If you > did something wrong with your repo before, maybe "git gc --aggressive" > can help. Yes, I put a lot of code into the repository. I know it is wrong from git POV ;) That's why I wrote about splitting repository into multiple smaller repositories, but AFAIK git is not very good at managing that too... (Background: we're thinking about migrating svn repository into git. Currently there is one svn repository containing many related directories. They are branched/merged/tagged together. Often work is done on all/many of them, but there are some use cases where we need only some of them) -- Piotr Krukowiecki ``` ## Jakub Narębski, 2013-10-29 14:45 Subject: Re: Limiting disk usage Message-ID: <526FCA13.4090902@gmail.com> URL: https://gitlist.dev/e/526FCA13.4090902%40gmail.com In-Reply-To: ``` On 2013-10-29 14:50, Piotr Krukowiecki wrote: > Yes, I put a lot of code into the repository. I know it is wrong from git POV ;) > That's why I wrote about splitting repository into multiple smaller > repositories, but AFAIK git is not very good at managing that too... Lot of code isn't problem, see Linux kernel or GCC, or LibreOffice.org > (Background: we're thinking about migrating svn repository into git. > Currently there is one svn repository containing many related > directories. They are branched/merged/tagged together. Often work is > done on all/many of them, but there are some use cases where we need > only some of them) If they are independent projects, they should get independent repositories; you can stitch them back together using git-submodule (or git-subtree). reposurgeon can hel you with that. -- Jakub Narębski ``` ## Piotr Krukowiecki, 2013-10-30 07:44 Subject: Re: Limiting disk usage Message-ID: URL: https://gitlist.dev/e/CAA01Cso8J%2BBBgskoRtxkPRrtM4KJ1O2FhcB4uh%2BQB2Dr%3DDZGyQ%40mail.gmail.com In-Reply-To: <526FCA13.4090902@gmail.com> ``` On Tue, Oct 29, 2013 at 3:45 PM, Jakub Narębski wrote: > On 2013-10-29 14:50, Piotr Krukowiecki wrote: >> > Lot of code isn't problem, see Linux kernel or GCC, or LibreOffice.org OK, it depends on how much code is too much for you. I'm not talking about performance of git, just disk usage. In my case I have too much code. > If they are independent projects, they should get independent repositories; > you can stitch them back together using git-submodule (or git-subtree). > reposurgeon can hel you with that. They are not totally independent projects (nor totally dependent). Normally you want them all, you want to create a branch/tag on all of them, if new directory is created you want to automatically have it too etc. But there are use cases when you need only some of them and then they can be seen as independent . git-submodule have several disadvantages for my use: - does not track branches (you have to specify hardcoded SHA1 instead of a branch name) - additional complexity / command layer (you have to do things like "submodule init", "submodule update") - you can't simply work on all submodules (you can't simply branch, merge, commit, log, diff, etc in all submodules using one command in top-level project) git-subtree looks much better in that regard, but for example it needs an explicit prefix (name of "subproject") on which it operates. So you can't say "update everything" or "commit everything" etc. With hundreds of subprojects you need another layer to automate this... -- Piotr Krukowiecki ``` ## Jakub Narębski, 2013-10-30 15:39 Subject: Re: Limiting disk usage Message-ID: <52712816.9010600@gmail.com> URL: https://gitlist.dev/e/52712816.9010600%40gmail.com In-Reply-To: ``` On 2013-10-30 08:44, Piotr Krukowiecki wrote: > On Tue, Oct 29, 2013 at 3:45 PM, Jakub Narębski wrote: >> On 2013-10-29 14:50, Piotr Krukowiecki wrote: >> If they are independent projects, they should get independent repositories; >> you can stitch them back together using git-submodule (or git-subtree). >> reposurgeon can hel you with that. > > They are not totally independent projects (nor totally dependent). > Normally you want them all, you want to create a branch/tag on all of > them, if new directory is created you want to automatically have it > too etc. But there are use cases when you need only some of them and > then they can be seen as independent . > > git-submodule have several disadvantages for my use: > - does not track branches (you have to specify hardcoded SHA1 instead > of a branch name) > - additional complexity / command layer (you have to do things like > "submodule init", "submodule update") > - you can't simply work on all submodules (you can't simply branch, > merge, commit, log, diff, etc in all submodules using one command in > top-level project) git-submodule was created (I think) to manage loosely coupled fairly independent projects, where you need nevertheless to ensure that both work together (so you use specific version of subproject for a given version of superproject). It looks like this is not what you have. From time to time you can find there ideas about adding "floating" mode to git-submodule, but as far as I know up till now without effect... > git-subtree looks much better in that regard, but for example it needs > an explicit prefix (name of "subproject") on which it operates. So you > can't say "update everything" or "commit everything" etc. With > hundreds of subprojects you need another layer to automate this... git-subtree is about helping merging / embedding (or re-embedding) of subproject history in superproject history, and extracting subproject history from superproject history. But embedded subproject is just subdirectory to git. It looks like it isn't what you want. There are many projects[1] that try to solve problem of managing interdependent repositories, e.g.: gitslave, repo, fgit. Maybe one of them would be good fit for your problem. [1]: https://git.wiki.kernel.org/index.php/InterfacesFrontendsAndTools -- Jakub Narębski ``` ## Fredrik Gustafsson, 2013-10-30 15:40 Subject: Re: Limiting disk usage Message-ID: <20131030154011.GB9691@paksenarrion.iveqy.com> URL: https://gitlist.dev/e/20131030154011.GB9691%40paksenarrion.iveqy.com In-Reply-To: <52712816.9010600@gmail.com> ``` On Wed, Oct 30, 2013 at 04:39:02PM +0100, Jakub Narębski wrote: > From time to time you can find there ideas about adding "floating" > mode to git-submodule, but as far as I know up till now without > effect... How about git submodule update --remote ? -- Med vänliga hälsningar Fredrik Gustafsson tel: 0733-608274 e-post: iveqy@iveqy.com ```