{"thread":{"id":"11262","subject":"Re: Something is broken in repack. Why not with fork and pipes?","startedAt":"2007-12-12T18:47:14Z","lastAt":"2007-12-12T19:41:30Z","messageCount":2,"participants":["J.C. Pizarro","Johannes Schindelin"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"62947","messageId":"998d0e4a0712121047m3cb09f37qc3157b96e5d171e7@mail.gmail.com","threadId":"11262","inReplyTo":null,"subject":"Re: Something is broken in repack. Why not with fork and pipes?","fromName":"J.C. Pizarro","fromEmail":"jcpiza@gmail.com","sentAt":"2007-12-12T18:47:14Z","receivedAt":"2007-12-12T18:47:14Z","isPatch":false,"sender":{"key":"jcpiza@gmail.com","avatar":null},"body":"At http://gcc.gnu.org/ml/gcc/2007-12/msg00360.html, Andreas Ericsson\n<ae@op5.se> wrote:\n> If it's still an issue next week, we'll have a 16 core (8 dual-core cpu's)\n> machine with some 32gb of ram in that'll be free for about two days.\n> You'll have to remind me about it though, as I've got a lot on my mind\n> these days.\n>\n>\n> --\n> Andreas Ericsson                   andreas.ericsson@op5.se\n> OP5 AB                             www.op5.se\n> Tel: +46 8-230225                  Fax: +46 8-230231\n\nIt's good idea if it's for 24/365.25 that it does\n autorepack-compute-again-again-again-those-unexplored-deltas of\n git repositories in realtime. :D\n\nSome body can do \"git clone\" that it could give smaller that one hour ago :D\n\n-----------------------------------------------------------------\n\nTo Linus, Why don't you forget the threaded implementation of your repo-pack?\n\nTo imagine a \"buggy bloated threading implementation originated to try it to\nwork only in HyperThreading Intel CPUs and 8 cores x 8 threads/core\nNiagara Sparcs\"\n\nIMHO, in multicored machine, multiprocessed implementation of repo-pack perfomes\nbetter than multithreaded implementation, although i've not their results.\n\nIt has not issue, not problem, etc. with memory allocation of threads,\nso monothreaded memory allocation is simple and fast!\n\nYou can see \"Why not with fork and pipes like in linux?\" at\nhttp://gcc.gnu.org/ml/gcc/2007-12/msg00203.html\nhttp://gcc.gnu.org/ml/gcc/2007-12/msg00209.html\n\nFor easy implementation, don't use threads due to complicated condition races\n between threads of multithreaded processes.\n\nTo use only condition races between monothreaded processes with select/epoll\n only in the parent process. It's due to the KISS principle works.\n\nThe children processes share almost readed-only memory due to COW\n (Copy On Write), so, before forking, the parent must to have a large\n plain data structures in C for children. The children use pipes to\n realize a complex intercommunication that the parent updates the\n results computated by the children almost of the time.\n\nAnother implementation is that the children can realize a locked\n load-and-store to/from unique filesystem's database if big memory to\n store data is a big problem.\n\nAnother implementation is to consider children processes as intensive-CPU\nslaves and parent process as the master that manipulates the big database.\n\nIf you want to measure the performance between multiprocessed vs multithreaded\nimplementation of repo-pack then you have to remember that\n\n   For same data input size and same data output size, to get the\n   seconds of your wall-clock or watch-clock as a measure of the benchmark\n   of this repo-pack.\n\nThe numeric data posted to mailing list about the timings dependently of # of\n threads are bad measured because they don't say how is small the result repo.\n and don't say if the results are the same independently of # of threads.\n\nFor good measures, we need \"to plot the curves\", e.g. based in\n( # of threads, elapsed time of wall-clock, data input size, data output size )\nand we can observe the intersection between above curves.\n\n   J.C.Pizarro\n"},{"id":"62954","messageId":"Pine.LNX.4.64.0712121939210.27959@racer.site","threadId":"11262","inReplyTo":"998d0e4a0712121047m3cb09f37qc3157b96e5d171e7@mail.gmail.com","subject":"Re: Something is broken in repack. Why not with fork and pipes?","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2007-12-12T19:41:30Z","receivedAt":"2007-12-12T19:41:30Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Wed, 12 Dec 2007, J.C. Pizarro wrote:\n\n> It's good idea if it's for 24/365.25 that it does\n>  autorepack-compute-again-again-again-those-unexplored-deltas of\n>  git repositories in realtime. :D\n\nThis sentence does not parse.\n\n> Some body can do \"git clone\" that it could give smaller that one hour ago :D\n\nNeither does this.\n\n> To Linus, Why don't you forget the threaded implementation of your \n> repo-pack?\n\nPlease do a little research before you ask such questions: it is neither \nLinus who did it, nor is it better to use processes than threads.\n\nBesides, your proposal has nothing to do with the issue of this thread \n(memory consumption).\n\nCiao,\nDscho\n"}]}