From: Jakub Narebski Date: Sun, 03 Oct 2010 09:20:59 GMT Subject: Re: Advertising the Git User's Survey 2011 Message-ID: <201010031121.03795.jnareb@gmail.com> In-Reply-To: <20101002062056.GB16626@kytes> On Sat, 2 Oct 2010, Ramkumar Ramachandra wrote: > Ævar Arnfjörð Bjarmason writes: > > On Mon, Sep 20, 2010 at 13:38, Jakub Narebski wrote: > > > > > Currenly the survey has more than 5000 responses (in a not whole month) > > > > That seems somewhat low, but maybe people just aren't that interested > > in taking surveys. > > Nice work on the survey! This is our best year by far. Some general > observations: > > Interesting statistic: 24508 people viewed it, 7821 people completed > it, but 0 people started filling out information and later decided not > to submit it. It could mean that many people clicked through and found > the survey, but probably left because it looked too long at a glance? Without the knowledge how those two numbers are calculated we can only speculate what do they mean. I think that '0' in 'Incomplete' statistics is here because this survey doesn't have compulsory questions: answering all questions are optional, so it might mean that even if one question in the survey is answered, then the survey is considered complete by Survs.com statistics. I personally do not like the "wizard" formatting of surveys, i.e. dividing survey into page so you are not presented with very long page, but are presented withc chunks of survey at glance. Even if you see how much survey did you fill in (how many pages there are in total), and even if you can go back to previous page. I'd prefer to create a better information about survey upfront. We say that all questions are optional ("Note that you may skip questions as you like"). It is also stated that you can fill only a part of survey, and later go back to finish it... hmmm, I wonder if those cases where one edited his/her survey responses multiple times are counted as one finished survey, but multiple views. We could also write how much time it takes on average to fil the survey. > The average time spent on the survery is 34 minutes It would be interesting to have more detailed statistics of time spent on the survey that only average time, a single number. When one is filling open-form essay-length question, it would obviously take much more time than for one who doesn't. But Survs.com currently doesn't provide it. I can try to ask for it, though (via feedback). > - I think we can bring that down to 10~15 minutes if we design > questions to extract more information. Also, there's little incentive > for taking the survey: while many companies actually give out > discounts/ coupons for taking surveys, the least we can do is present > real-time results in the most interesting manner possible ie. survey > takers should see the "results so far" immediately after taking the > survey; some visualizations such as pie charts? What we can do is after finishing the survey to redirect to the survey analysis page: https://www.survs.com/results/33Q0OZZE/MV653KSPI2 http://tinyurl.com/GitSurvey2010Analysis instead of IIRC currently used redirect to https://git.wiki.kernel.org/index.php/GitSurvey2010 As far as I know Survs.com doesn't provide any API for extracting data or survey statistics required for creating such visualization. Neither we have a place where such app could be created, I think. > > In questions 5, 10, 12, 13, 16, cut down out the options that have > very few respondents and let them all go into "other". It probably > doesn't actually save the survey taker any time, but I think seeing a > long page with many options can be scary. The "5. Which Git version(s) are you using?" is not that long. We could create a cut off a bit earlier, perhaps on 1.4 (i.e. have "pre 1.4") or even earlier, we could remove alternate implementations answers (git-bigfiles, JGit, other implementations), or even concatenate 'master', 'next', 'pu' into single response... but would it buy us much? The other side of removing choices, relying instead on "other, please specify" response is that it makes it harder to analyze results of survey: different people use different words for the same thing (and there are also spelling mistakes), and make results less reliable: people do not fill "other" if there is at least partial match, or do not know how to specify their version. In "10. What Git interfaces, implementations and frontends do you use?" we can remove tools marked as deprecated... if not for the fact that it is actively interesting to know how many people use such deprecated tools. Besides, the list of answers to this question is not overly long, I don't think. We can remove those choices in "12. What Git GUIs (graphical user interfaces) do you use?" that got less than 1% rounded, or less than 10 responses. On the other hand some people stated earlier that the list of possible choices in the survey (not necessarily about this question in specific) serve as reminder / information about possible choices. The other side of removing options from "13. Which git hosting site(s) do you use for your project(s)?" is that when sending requests to announce the survey to those git hosting sites that are not on this list, some of them requested to be added (which is impossible after starting the survey; and before survey begins it is little sense to send announcements). Besides all of those below 1% rounded (Codesion, GitFarm, The Chaw, CipherHive) are also those that I didn't get response to request for announcing Git User's Survey 2010... We could make it more organized though, e.g. by sorting list of options alphabetically, or something like that. Removing options from "16. Which of the following features do you use?" would make it harder to analyze and less reliable. Especially in this question different people consider different features important enough to mention, and describe feature in many different ways. Besides, each option except of "git cvsserver" got more than 1% rounded, and having "git cvsserver" is interesting on its own (perhaps in other question?). > 1. Country of residence: we can probably make this a nice click-on-map > interface as opposed to freeform text. It'll be more useful to us, > and more interesting to users when we advertise the results. It would be nice to have click-on-map (Google Maps or Bing Maps based), something like Ohloh provides, resulting in map of survey responders similar to the map of git users and git contributors on Ohloh http://www.ohloh.net/p/git/map it isn't something that Survs.com offers currently. I can only ask for it to be provided... Another solution would be to have pre-filled combo box (