threads / discuss / 31210

Sync production with Git

Subject: Sync production with Git

## tl;dr

6 messages between Aug 8, 2012 and Aug 9, 2012.

replies: 5people: 5as markdown or json

kiranpyati· Aug 8, 2012, 13:11 UTC · lore
I am new to github,

Earlier we used to manually upload files on the production through FTP although git was present on the production. Due to this now git status shows many modified and untrack files.

To sync that with git we have downloaded all files from production and committed to git. Now git has all files same as production.

We have not pulled on production since last 6 months and because of this it shows modified and untracked files.

Now if we pull on the production there any 100% chances of the conflict happened on all modified files. As there are hundreds of modified files since last since month. Git pull will show conflict to all those files. In that case site will get down and we can not afford this.

We want a way to seamlessly sync production and Git.
Can anybody please help me on this?
Thanks in advance..!!

-- View this message in context: http://git.661346.n2.nabble.com/Sync-production-with-Git-tp7564617.html Sent from the git mailing list archive at Nabble.com.

Matthieu Moy· Aug 8, 2012, 14:20 UTC · re: kiranpyati · lore

Re: Sync production with Git

kiranpyati <kiran.pyati@infobeans.com> writes:
> We want a way to seamlessly sync production and Git.

You should be aware that Git was not designed for this scenario. The usual flow with Git (and actually with most revision control systems), is to do the development with Git, then use your build system to generate a package that can be used in production (e.g. generate a .tar.gz, or a .jar, or whatever your platform needs), and then install this package on your production server.

It can be tempting, however, to use your revision control system as a deployment tool, so that an update on the production server be as simple as "git pull". But in real-life applications, it usually has to be more complicated: do you need to generate some files after you fetch the latest version of the source? Do you need to update your database? Isn't the .git/ directory harmfull here (e.g. do I want the full history source of my project to be visible worldwide if this is a webapplication?) ...

If you insist in using Git for deployment, then you should absolutely stick to it. Whether for deployment or for anything else, trying to send changes using both Git and other mechanism (e.g. uploading files directly to a working tree as you did) puts you in trouble 99.9% of the cases.

In your case, the damage is already done. If I were you, I'd do something like

<do some backup> <make sure the backup is OK> <think twice "will I be able to restore the backup if it goes wrong?"> $ git fetch origin $ git reset --hard origin/master

(actually, if I were you, I'd try reproducing the situation on a non-production server first)

"git fetch" will download the revisions from the remote server, which should be the repository where the version you want to run is located. "git reset --hard" will discard any local change (committed or not) you may have, and set your local working tree to the latest version in the master branch of the remote repository. You may need a "git clean" to remove untracked files too.

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/
David Aguilar· Aug 8, 2012, 22:35 UTC · re: Matthieu Moy · lore

Re: Sync production with Git

On Wed, Aug 8, 2012 at 7:20 AM, Matthieu Moy <Matthieu.Moy@grenoble-inp.fr> wrote:

Show 10 quoted lines
> kiranpyati <kiran.pyati@infobeans.com> writes:
>
>> We want a way to seamlessly sync production and Git.
>
> You should be aware that Git was not designed for this scenario. The
> usual flow with Git (and actually with most revision control systems),
> is to do the development with Git, then use your build system to
> generate a package that can be used in production (e.g. generate a
> .tar.gz, or a .jar, or whatever your platform needs), and then install
> this package on your production server.
Obligatory link to Sitaram's very helpful docs:
http://sitaramc.github.com/the-list-and-irc/deploy.html
Thanks to Sitaram for this very helpful guide.

That said, please pay close attention to everything Matthieu wrote here. It seems like things will go smoothly once you "true things up" and then follow a sensible process for moving things from git to production going forward. Sitaram's write-up can help you discover what the right process is for you.

Show 41 quoted lines
> It can be tempting, however, to use your revision control system as a
> deployment tool, so that an update on the production server be as simple
> as "git pull". But in real-life applications, it usually has to be more
> complicated: do you need to generate some files after you fetch the
> latest version of the source? Do you need to update your database? Isn't
> the .git/ directory harmfull here (e.g. do I want the full history
> source of my project to be visible worldwide if this is a
> webapplication?) ...
>
> If you insist in using Git for deployment, then you should absolutely
> stick to it. Whether for deployment or for anything else, trying to send
> changes using both Git and other mechanism (e.g. uploading files
> directly to a working tree as you did) puts you in trouble 99.9% of the
> cases.
>
> In your case, the damage is already done. If I were you, I'd do
> something like
>
> <do some backup>
> <make sure the backup is OK>
> <think twice "will I be able to restore the backup if it goes wrong?">
> $ git fetch origin
> $ git reset --hard origin/master
>
> (actually, if I were you, I'd try reproducing the situation on a
> non-production server first)
>
> "git fetch" will download the revisions from the remote server, which
> should be the repository where the version you want to run is located.
> "git reset --hard" will discard any local change (committed or not) you
> may have, and set your local working tree to the latest version in the
> master branch of the remote repository. You may need a "git clean" to
> remove untracked files too.
>
> --
> Matthieu Moy
> http://www-verimag.imag.fr/~moy/
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
-- 
David
Drew Northup· Aug 9, 2012, 03:46 UTC · re: Matthieu Moy · lore

Re: Sync production with Git

On Wed, Aug 8, 2012 at 10:20 AM, Matthieu Moy <Matthieu.Moy@grenoble-inp.fr> wrote:

Show 10 quoted lines
> kiranpyati <kiran.pyati@infobeans.com> writes:
>
>> We want a way to seamlessly sync production and Git.
>
> You should be aware that Git was not designed for this scenario. The
> usual flow with Git (and actually with most revision control systems),
> is to do the development with Git, then use your build system to
> generate a package that can be used in production (e.g. generate a
> .tar.gz, or a .jar, or whatever your platform needs), and then install
> this package on your production server.

As I mentioned several days too late on a previous thread, the assumption of formal packaging does not fit more than a few scenarios. Certainly generating a tarfile sounds like a more reasonable solution in many cases, but that requires discipline and plausible deniability--you won't have a way of detecting if somebody didn't follow the "official" deployment path. It is a sad fact that revision control is probably the most sane way to detect this and deal with it appropriately (without loosing work).

Show 5 quoted lines
> It can be tempting, however, to use your revision control system as a
> deployment tool, so that an update on the production server be as simple
> as "git pull". But in real-life applications, it usually has to be more
> complicated: do you need to generate some files after you fetch the
> latest version of the source? Do you need to update your database?

If you are dealing with something updatable by just dropping a tar file on it and walking away then a simple "git pull" will do just fine (provided permissions and ACLs are set appropriately). If any of those other concerns apply then you aren't talking about a simple file update--you are talking about an application upgrade.

(Note: If you are using extended ACLs on a LINUX filesystem such as ext2, ext3, or ext4, you won't have to worry about the git pull flattening those--if you set them correctly--as that data is not part of the content and container information that GIT tracks.)

> Isn't
> the .git/ directory harmfull here (e.g. do I want the full history
> source of my project to be visible worldwide if this is a
> webapplication?) ...
If you have configured your webserver correctly this is not an issue.
Show 17 quoted lines
> If you insist in using Git for deployment, then you should absolutely
> stick to it. Whether for deployment or for anything else, trying to send
> changes using both Git and other mechanism (e.g. uploading files
> directly to a working tree as you did) puts you in trouble 99.9% of the
> cases.
>
> In your case, the damage is already done. If I were you, I'd do
> something like
>
> <do some backup>
> <make sure the backup is OK>
> <think twice "will I be able to restore the backup if it goes wrong?">
> $ git fetch origin
> $ git reset --hard origin/master
>
> (actually, if I were you, I'd try reproducing the situation on a
> non-production server first)

Before doing this, do a "git diff -b -w" (ignoring whitespace changes) to see how big the damage is and extract patches.

If you've planned things out well you won't be using the same branch in production that you use for general development and therefore will have the ability to use GIT to help get the changes made directly on the production system into a temporary branch on another system for proper merging, testing, and QA.

Show 6 quoted lines
> "git fetch" will download the revisions from the remote server, which
> should be the repository where the version you want to run is located.
> "git reset --hard" will discard any local change (committed or not) you
> may have, and set your local working tree to the latest version in the
> master branch of the remote repository. You may need a "git clean" to
> remove untracked files too.

Then you can safely hard reset to the commit just prior the divergence and pull the changes onto the production system--re-applying those changes you discovered in the "git diff -b -w" step on a developer's system first, in a temporary branch, before doing the reset, merging back to development, etc.

So, in conclusion, this is a situation that can be dealt with if one keeps his wits about him.

-- 
-Drew Northup
--------------------------------------------------------------
"As opposed to vegetable or mineral error?"
-John Pescatore, SANS NewsBites Vol. 12 Num. 59
demerphq· Aug 9, 2012, 04:21 UTC · re: kiranpyati · lore

Re: Sync production with Git

On 8 August 2012 15:11, kiranpyati <kiran.pyati@infobeans.com> wrote:
Show 22 quoted lines
> I am new to github,
>
> Earlier we used to manually upload files on the production through FTP
> although git was present on the production. Due to this now git status shows
> many modified and untrack files.
>
> To sync that with git we have downloaded all files from production and
> committed to git. Now git has all files same as production.
>
> We have not pulled on production since last 6 months and because of this it
> shows modified and untracked files.
>
> Now if we pull on the production there any 100% chances of the conflict
> happened on all modified files. As there are hundreds of modified files
> since last since month. Git pull will show conflict to all those files. In
> that case site will get down and we can not afford this.
>
> We want a way to seamlessly sync production and Git.
>
> Can anybody please help me on this?
>
> Thanks in advance..!!
Try git-deploy.
https://github.com/git-deploy
It contains a full work flow management for handling rollouts from git.
Yves
-- 
perl -Mre=debug -e "/just|another|perl|hacker/"
demerphq· Aug 9, 2012, 04:25 UTC · re: demerphq · lore

Re: Sync production with Git

On 9 August 2012 06:21, demerphq <demerphq@gmail.com> wrote:
Show 29 quoted lines
> On 8 August 2012 15:11, kiranpyati <kiran.pyati@infobeans.com> wrote:
>> I am new to github,
>>
>> Earlier we used to manually upload files on the production through FTP
>> although git was present on the production. Due to this now git status shows
>> many modified and untrack files.
>>
>> To sync that with git we have downloaded all files from production and
>> committed to git. Now git has all files same as production.
>>
>> We have not pulled on production since last 6 months and because of this it
>> shows modified and untracked files.
>>
>> Now if we pull on the production there any 100% chances of the conflict
>> happened on all modified files. As there are hundreds of modified files
>> since last since month. Git pull will show conflict to all those files. In
>> that case site will get down and we can not afford this.
>>
>> We want a way to seamlessly sync production and Git.
>>
>> Can anybody please help me on this?
>>
>> Thanks in advance..!!
>
> Try git-deploy.
>
> https://github.com/git-deploy
>
> It contains a full work flow management for handling rollouts from git.
Better link:
https://github.com/git-deploy/git-deploy
Yves
-- 
perl -Mre=debug -e "/just|another|perl|hacker/"

← back to recent threads