{"thread":{"id":"12890","subject":"symbolic link management in git-archive","startedAt":"2008-03-27T11:29:28Z","lastAt":"2008-03-31T22:12:09Z","messageCount":10,"participants":["Sergio Callegari","Miklos Vajna","Johannes Schindelin","Junio C Hamano","René Scharfe"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"73197","messageId":"loom.20080327T112740-539@post.gmane.org","threadId":"12890","inReplyTo":null,"subject":"symbolic link management in git-archive","fromName":"Sergio Callegari","fromEmail":"sergio.callegari@gmail.com","sentAt":"2008-03-27T11:29:28Z","receivedAt":"2008-03-27T11:29:28Z","isPatch":false,"sender":{"key":"sergio.callegari@gmail.com","avatar":"https://gravatar.com/avatar/c98f41317e0422c1e630385de0e3970227b8e5ad15f35ba8586066467cc833bc?d=mp&s=160"},"body":"Hi,\n\nI guess the answer is \"no\" or \"not yet\", but is there a way to tell the zip\nbackend of git-archive to follow symbolic links rather than to store them?\n\nSergio\n"},{"id":"73198","messageId":"20080327114024.GR14567@genesis.frugalware.org","threadId":"12890","inReplyTo":"loom.20080327T112740-539@post.gmane.org","subject":"Re: symbolic link management in git-archive","fromName":"Miklos Vajna","fromEmail":"vmiklos@frugalware.org","sentAt":"2008-03-27T11:40:24Z","receivedAt":"2008-03-27T11:40:24Z","isPatch":false,"sender":{"key":"vmiklos@frugalware.org","avatar":"https://gravatar.com/avatar/401c1cbbb3a5d13e650c691a2c71d6fd0b80df1a01bc74d9f1972675dd58f2bd?d=mp&s=160"},"body":"On Thu, Mar 27, 2008 at 11:29:28AM +0000, Sergio Callegari wrote:\n> I guess the answer is \"no\" or \"not yet\", but is there a way to tell the zip\n> backend of git-archive to follow symbolic links rather than to store them?\n\nhow would that handle a '. -> foo' symlink? following such a recursion\nwould lead to an infinite loop, i guess.\n"},{"id":"73199","messageId":"alpine.LSU.1.00.0803271244420.3802@racer.site","threadId":"12890","inReplyTo":"20080327114024.GR14567@genesis.frugalware.org","subject":"Re: symbolic link management in git-archive","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-03-27T11:44:57Z","receivedAt":"2008-03-27T11:44:57Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Thu, 27 Mar 2008, Miklos Vajna wrote:\n\n> On Thu, Mar 27, 2008 at 11:29:28AM +0000, Sergio Callegari wrote:\n> > I guess the answer is \"no\" or \"not yet\", but is there a way to tell the zip\n> > backend of git-archive to follow symbolic links rather than to store them?\n> \n> how would that handle a '. -> foo' symlink? following such a recursion\n> would lead to an infinite loop, i guess.\n\nDon't forget '/ -> foo'.\n\nCiao,\nDscho\n"},{"id":"73200","messageId":"loom.20080327T115110-82@post.gmane.org","threadId":"12890","inReplyTo":"alpine.LSU.1.00.0803271244420.3802@racer.site","subject":"Re: symbolic link management in git-archive","fromName":"Sergio Callegari","fromEmail":"sergio.callegari@gmail.com","sentAt":"2008-03-27T12:11:48Z","receivedAt":"2008-03-27T12:11:48Z","isPatch":false,"sender":{"key":"sergio.callegari@gmail.com","avatar":"https://gravatar.com/avatar/c98f41317e0422c1e630385de0e3970227b8e5ad15f35ba8586066467cc833bc?d=mp&s=160"},"body":"Johannes Schindelin gmx.de> writes:\n\n> \n> Hi,\n> \n> On Thu, 27 Mar 2008, Miklos Vajna wrote:\n> \n> > On Thu, Mar 27, 2008 at 11:29:28AM +0000, Sergio Callegari gmail.com> wrote:\n> > > I guess the answer is \"no\" or \"not yet\", but is there a way to tell the zip\n> > > backend of git-archive to follow symbolic links rather than to store them?\n> > \n> > how would that handle a '. -> foo' symlink? following such a recursion\n> > would lead to an infinite loop, i guess.\n> \n> Don't forget '/ -> foo'.\n> \n> Ciao,\n> Dscho\n> \n> \n\nMy question was inspired by the fact that the unix version of the zip program\nhas a switch, precisely to decide whether to follow or to store links.\nI believe that the reason why this switch exist is clear: for some mysterious\nreason the world is populated by OSs that do not understand symlinks and while\nsomeone finds out why it is sensible to have workarounds. Obviously zip archives\nwith stored links are completely useless on these OSs.\nBTW, note that also tar has a similar switch.\n \nI believe that cases like the foo -> . or the foo -> / (or even foo -> ..) can\nbe solved by either:\n\n- limiting the symlink dereferentiation only to symlinks pointing to plain files\n\nor\n\n- leaving it to the user not to ask for link following when he knows that he has\nsuch types of links\n\nPersonally I prefer the second. In the end:\n- If these types of link exist, it is clear that the git-managed-stuff is not\nmade for certain OSs, so the user should know that asking for link following\nmakes no sense\n- The user should know that there are some commands that might be dangerous\n(think rm -fr ~), just warn them in the manual.\n- The maximum risk here is to have the command never stop and fill the disk.\n\nSergio\n"},{"id":"73206","messageId":"7v3aqcb02s.fsf@gitster.siamese.dyndns.org","threadId":"12890","inReplyTo":"loom.20080327T112740-539@post.gmane.org","subject":"Re: symbolic link management in git-archive","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-03-27T16:31:23Z","receivedAt":"2008-03-27T16:31:23Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Sergio Callegari writes:\n\n> I guess the answer is \"no\" or \"not yet\", but is there a way to tell the zip\n> backend of git-archive to follow symbolic links rather than to store them?\n\nI am not sure what you mean. Are you tracking a symbolic link X that\npoints at Y in your revision and expecting git-archive to include whatever\nhappens to be at Y (which may or may not even exist) when you run the\ncommand?\n\nIf that is the case, the answer is \"no\" and \"will never happen\". If you\nare tracking a symbolic link X that points at Y, the information git\ntracks is the fact that there is a symbolic link X that points at Y, and\nnot what Y happens to look like at a random moment. Change to Y is not\ntracked by git so why should you get different output from git-archive of\nthe same revision before and after you modify Y which is not part of the\nrevision to begin with?\n\nIf that is not what you are asking, please restate the question.\n"},{"id":"73212","messageId":"loom.20080327T175844-199@post.gmane.org","threadId":"12890","inReplyTo":"7v3aqcb02s.fsf@gitster.siamese.dyndns.org","subject":"Re: symbolic link management in git-archive","fromName":"Sergio Callegari","fromEmail":"sergio.callegari@gmail.com","sentAt":"2008-03-27T18:34:30Z","receivedAt":"2008-03-27T18:34:30Z","isPatch":false,"sender":{"key":"sergio.callegari@gmail.com","avatar":"https://gravatar.com/avatar/c98f41317e0422c1e630385de0e3970227b8e5ad15f35ba8586066467cc833bc?d=mp&s=160"},"body":"Junio C Hamano pobox.com> writes:\n\n> \n> Sergio Callegari gmail.com> writes:\n> \n> > I guess the answer is \"no\" or \"not yet\", but is there a way to tell the\n> > zip\n> > backend of git-archive to follow symbolic links rather than to store \n> > them?\n> \n> I am not sure what you mean. Are you tracking a symbolic link X that\n> points at Y in your revision and expecting git-archive to include \n> whatever\n> happens to be at Y (which may or may not even exist) when you run the\n> command?\n\nYes, this is the case.\n\n> If that is the case, the answer is \"no\" and \"will never happen\". If you\n> are tracking a symbolic link X that points at Y, the information git\n> tracks is the fact that there is a symbolic link X that points at Y,\n\n... I obviously agree with this!\n\n> and\n> not what Y happens to look like at a random moment.\n\nWell, certainly if the zip backend of git-archive in addition to the -1 ... -9\nswitches also supported a --dereference switch, I would not run git-archive\n--format=zip --dereference at \"random moments\". I would run it when I am sure\nthat things pointed at by the symbolic links in the tracked project actually\npoint at sensible things.\n\n> Change to Y is not\n> tracked by git\n\nUnless Y is also in the tracked project...\n\n> so why should you get different output from git-archive of\n> the same revision before and after you modify Y which is not part of the\n> revision to begin with?\n\nBasically, because if I make a zip archive instead of a git bundle, I want to\nstore (and probably give to someone else) a copy of what _I see_ at a certain\ninstant in time which might not exactly coincide with the tracked state.\n\nNote that if Y is outside of the tracked project and I make an archive, and then\nI give the archive to my friend X, Mr. X will see the same symbolic link, but\nstill a completely and randomly different content than I do, depending on where\nhe is unpacking the archive.\n\nBut to get to a more practical case, my situation is this:\n\nI have a project where I need to have the same content in multiple places,\notherwise development tools are not happy.\nSince I do most of the development on Linux, I use symbolic links. This is very\ngood, not just because I save space, but particularly because I am sure that the\ncontent cannot loose coherency (which would be very bad) at the different\nplaces.\n\nEvery now and then, to distribute snapshots of the project, I run git archive to\nmake a zip file, that I give to other people. Unfortunately, some of them use\nWindows, where the symbolic links appear as one-liner text files. And obviously\nthey cannot compile anything and they complain.\n\nSo I would like git archive to be able to make zip archives with the symbolic\nlinks /resolved/.\n\nwhat I am asking is not to modify the standard behaviour of git archive to\nfollow links, rather to have a switch to activate the dereferencing behaviour\nonly when needed. In the end git archive is a nice shorthand for a checkout and\na successive run of zip or tar and both zip and tar have a switch to control\nthis dereferencing behaviour (BTW, zip on my distro dereferences by default, the\nswitch is to store symbolic links).\n\nSergio\n"},{"id":"73213","messageId":"7vd4pg9edm.fsf@gitster.siamese.dyndns.org","threadId":"12890","inReplyTo":"loom.20080327T175844-199@post.gmane.org","subject":"Re: symbolic link management in git-archive","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-03-27T19:05:25Z","receivedAt":"2008-03-27T19:05:25Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Sergio Callegari writes:\n\n> Unless Y is also in the tracked project...\n> ...\n> Note that if Y is outside of the tracked project and I make an archive,\n> and then I give the archive to my friend X, Mr. X will see the same\n> symbolic link, but still a completely and randomly different content\n> than I do, depending on where he is unpacking the archive.\n\nIf you _do_ keep track of Y in a separate repository, I think two archives\n(the one that has a pointer to Y, and the other that is taken from the\nrepository Y _at the revision you are using_), would solve that more\nnaturally. Then the version markers recorded in the archives would still\nbe valid.\n\n\tSide note. If we ever teach git-archive to create a recursive\n\ttarball that contains a submodule, we should be doing something\n\tlike that, not necessarily as two separate tarballs but possibly\n\twith a single tarball that has two comments that describe the\n\trevision of the toplevel and the submodule.\n\n> ... In the end git archive is a nice shorthand for a checkout and a\n> successive run of zip or tar and both zip and tar have a switch to\n> control this dereferencing behaviour (BTW, zip on my distro dereferences\n> by default, the switch is to store symbolic links).\n\nUnder such an option, at least the comment in the archive (both for zip\nand tar) that notes which revision the tarball was taken from should be\nomitted. As long as that is done, I think it is Ok to have such an\noptional behaviour.\n"},{"id":"73214","messageId":"loom.20080327T191845-374@post.gmane.org","threadId":"12890","inReplyTo":"7vd4pg9edm.fsf@gitster.siamese.dyndns.org","subject":"Re: symbolic link management in git-archive","fromName":"Sergio Callegari","fromEmail":"sergio.callegari@gmail.com","sentAt":"2008-03-27T19:20:16Z","receivedAt":"2008-03-27T19:20:16Z","isPatch":false,"sender":{"key":"sergio.callegari@gmail.com","avatar":"https://gravatar.com/avatar/c98f41317e0422c1e630385de0e3970227b8e5ad15f35ba8586066467cc833bc?d=mp&s=160"},"body":"Junio C Hamano pobox.com> writes:\n\n> \n> Under such an option, at least the comment in the archive (both for zip\n> and tar) that notes which revision the tarball was taken from should be\n> omitted. As long as that is done, I think it is Ok to have such an\n> optional behaviour.\n> \n\nyes, I overlooked this, but it makes perfect sense: if it is not an exact copy\nof the tracked state, it should not pretend to be so by indicating a revision.\n"},{"id":"73455","messageId":"47F14D4A.2020403@lsrfire.ath.cx","threadId":"12890","inReplyTo":"loom.20080327T175844-199@post.gmane.org","subject":"Re: symbolic link management in git-archive","fromName":"René Scharfe","fromEmail":"rene.scharfe@lsrfire.ath.cx","sentAt":"2008-03-31T20:44:58Z","receivedAt":"2008-03-31T20:44:58Z","isPatch":false,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Sergio Callegari schrieb:\n> I have a project where I need to have the same content in multiple places,\n> otherwise development tools are not happy.\n> Since I do most of the development on Linux, I use symbolic links. This is very\n> good, not just because I save space, but particularly because I am sure that the\n> content cannot loose coherency (which would be very bad) at the different\n> places.\n> \n> Every now and then, to distribute snapshots of the project, I run git archive to\n> make a zip file, that I give to other people. Unfortunately, some of them use\n> Windows, where the symbolic links appear as one-liner text files. And obviously\n> they cannot compile anything and they complain.\n> \n> So I would like git archive to be able to make zip archives with the symbolic\n> links /resolved/.\n\nWindows 2000 and up has support for symbolic links; it's just strangely\nrestricted, e.g. on Windows 2000 you can only link to directories and\nyou have to use tools that aren't shipped with the OS to create them.\nMicrosoft even calls them differently; here's a good starting point for\nmore information: http://en.wikipedia.org/wiki/NTFS_junction_point\n\nArguably, your unzip program should create junction points for symlinks\nin zip files. I wouldn't be surprised if none of the existing ones\nsupport that, though; junction points have been left undocumented for a\nlong time. It's also possible that they'd understand a different zip\ndialect for symlinks than the Info-Zip one produced by git-archive.\n\nWould it be practical for you to distribute a junction point creation\ntool like Mark Russinovich's Junction (except that Junction's EULA\nforbids redistribution under most circumstances; see here:\nhttp://www.microsoft.com/technet/sysinternals/FileAndDisk/Junction.mspx)\nand a script that creates these symlinks for your audience?\n\n\nIt's harder for git-archive to support following symlinks than for e.g.\nGNU tar. The reason is that the former operates on git objects, not\nfiles, directories or symlinks. In order to follow a symlink it would\nneed to evaluate the symlink, follow it and then add actual files and\ndirectories to the archive.\n\nFor your purposes, perhaps a slightly different implementation might be\nsufficient: namely to follow only relative symlinks that point to\ntracked objects. That way you still get a repeatable result and (most\nimportantly) git-archive doesn't need to look at files and directories,\nit can stay safely in git land. Would such a way of operation be useful\nto you?\n\nThanks,\nRené\n"},{"id":"73460","messageId":"47F161B9.1030408@gmail.com","threadId":"12890","inReplyTo":"47F14D4A.2020403@lsrfire.ath.cx","subject":"Re: symbolic link management in git-archive","fromName":"Sergio Callegari","fromEmail":"sergio.callegari@gmail.com","sentAt":"2008-03-31T22:12:09Z","receivedAt":"2008-03-31T22:12:09Z","isPatch":false,"sender":{"key":"sergio.callegari@gmail.com","avatar":"https://gravatar.com/avatar/c98f41317e0422c1e630385de0e3970227b8e5ad15f35ba8586066467cc833bc?d=mp&s=160"},"body":"René Scharfe wrote:\n\n> Windows 2000 and up has support for symbolic links; it's just strangely\n> restricted, [...] here's a good starting point for\n> more information: http://en.wikipedia.org/wiki/NTFS_junction_point\n>\n> Arguably, your unzip program should create junction points for symlinks\n> in zip files. I wouldn't be surprised if none of the existing ones\n> support that, though;\n>\n> Would it be practical for you to distribute a junction point creation\n> tool like Mark Russinovich's Junction (except that Junction's EULA\n> forbids redistribution under most circumstances; see here:\n> http://www.microsoft.com/technet/sysinternals/FileAndDisk/Junction.mspx)\n> and a script that creates these symlinks for your audience?\n> \nThank you very much for the detailed explanation. Unfortunately, I do \nnot think that Junction can be an option. My colleagues using Windows \ntend to be a bit \"conservative\" about the tools they use. If they \nnavigate filesystem contents with the graphical tool, and they \nlook-at/expand the content of a zip file from explorer they expect that \nit should be immediately right, otherwise there must be something wrong \nin the way /I/ create zip files. They would not appreciate having to \nunzip the file and then run an additional program on it to fix the \nunzipped stuff.\n> It's harder for git-archive to support following symlinks than for e.g.\n> GNU tar. The reason is that the former operates on git objects, not\n> files, directories or symlinks. In order to follow a symlink it would\n> need to evaluate the symlink, follow it and then add actual files and\n> directories to the archive.\n>\n> For your purposes, perhaps a slightly different implementation might be\n> sufficient: namely to follow only relative symlinks that point to\n> tracked objects. That way you still get a repeatable result and (most\n> importantly) git-archive doesn't need to look at files and directories,\n> it can stay safely in git land. Would such a way of operation be useful\n> to you?\n> \nAbsolutely positive. This would be already a great improvement, \nfulfilling 99.9% of needs.\n\nSergio\n"}]}