{"thread":{"id":"40922","subject":"BuGit: File-less distributed issue tracking system with Git","startedAt":"2015-12-02T19:34:19Z","lastAt":"2016-02-02T16:13:34Z","messageCount":2,"participants":["Stefan Monnier"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"273933","messageId":"jwva8psr6vr.fsf-monnier+gmane.comp.version-control.git@gnu.org","threadId":"40922","inReplyTo":null,"subject":"BuGit: File-less distributed issue tracking system with Git","fromName":"Stefan Monnier","fromEmail":"monnier@iro.umontreal.ca","sentAt":"2015-12-02T19:34:19Z","receivedAt":"2015-12-02T19:34:19Z","isPatch":false,"sender":{"key":"monnier@iro.umontreal.ca","avatar":null},"body":"I've hacked on this for personal use, mostly, but I figured if there\ncould be interest in such a beast, this is probably one of the best\nplaces to find it.\n\nSo, see attached BuGit, an issue tracking system which stores its\ndatabase in Git to try and get \"distributed operation for free\".\n\nBy pushing all the hard work to Git, BuGit is able to implement\na distributed bug tracking system in a simple shell script (less than\n20KB so far).  It can of course do off-line operations, but it can also\nshare bugs between unrelated databases (e.g. if you forward a bug report\nto someone else using this same system, you can keep the two bugs\nsync'd, even if they may get different bug-numbers on each side).\n\nObviously, this is lacking in many respects.  You can get a barely\ntolerable web-based UI (but only to browse bugs, not to manipulate them)\nwith any Git front-end, but if you really need a web-based UI it'll take\nextra work.\n\n\n        Stefan\n\n\n#!/bin/sh\n\n###  BuGit  ---  File-less distributed bug tracking system with Git\n\n# The design is based on the idea of trying to represent the bug-database\n# in such a way that Git's merge takes care of our own merge needs.  IOW\n# Git's merge should only result in conflicts when there is a *real* conflict\n# that can only be resolved by hand (e.g. two concurrent changes to the title\n# of a bug).\n#\n# The general idea is as follows:\n# - Keep messages in the metadata (more specifically the commit log), so\n#   they can't generate conflicts, they're auto-merged, and the ordering\n#   automatically preserved.\n# - Most other data is kept in file *names* (i.e. the files themselves are\n#   empty, to avoid merge conflicts).\n\n# In BuGit, bugs can be identified in 3 ways:\n# - ID: Bugs have a unique and immutable identification called \"ID\", usually\n#   some kind of random hexadecimal number.  This is the only stable and\n#   unambiguous identification.\n# - NAME: Bugs have a \"NAME\", also known as their \"title\".  This is\n#   a human-readable text which is expected to describe the bug concisely.\n#   This property can change over time, and several bugs can have\n#   the same NAME, tho this should be unusual.\n# - NB: Bugs can have a number.  This is not a property of the bug, tho, in\n#   the sense that the same bug can be known under different numbers in\n#   different databases.  So a bug can have several different NBs, and\n#   initially a bug has no NB at all (since allocation of a bug NB tends to\n#   be a centralized operation).  But in a given database, a given bug\n#   has at most one NB.\n# If BuGit says it wants a \"BUG\", it means that you can give it any one of\n# those kinds of identifiers.\n\n# The database is layed out as follows:\n# - Every bug lives in its own branch named \"bugs/ID\".\n#   The branch's commit messages hold the bug's messages.\n#   The branch's files are as follows:\n#   - \"name\": simple file holding the current NAME of this bug.\n#   - \"attachments/<timestamp> - <name>\" are attachments.\n#   - \"followers/<email>\" are empty files whose name indicates that <email>\n#     would like to receive updates on this bug.\n#   - \"assigned-to/<email>\" are empty files whose name indicate that <email>\n#     is reponsible for this bug.\n#   - \"tags/<tag>\" are empty files indicating that <tag> is applied to the bug.\n# Additionally to those bug branches, there is a \"master\" branch whose\n# commit message are unimportant and whose files are:\n# - \"names/NAME/ID\": empty file indicating that bug ID has name NAME.\n#   This is a cache used to perform reverse lookups (from NAME to ID),\n#   and it currently is not always kept up-to-date.  FIXME!\n# - \"numbers/NB\": simple file holding the ID of bug number NB.\n#\n# By design, the only possible sources of conflicts when merging different\n# databases are:\n# - if different bugs have the same NB.\n# - if a given bug ID has different names.\n# - if a bug has two different attachments with the same name same timestamp\n#   [ the timestamp should make this very unlikely, tho ].\n\n## Helper functions ###########################################################\n\nbugit_upcase () {\n    echo \"$@\" | tr '[:lower:]' '[:upper:]'\n}\n\nbugit_make_optloop () {\n    echo 'done=\"\"'\n    echo 'while [ $# -gt 0 ] && [ \"\" = \"$done\" ]; do'\n    echo 'case $1 in'\n    for arg in $(echo $1); do\n        case $arg in\n            *\\=)\n                argname=$(echo \"$arg\" | sed 's/=$//')\n                echo \"--${arg}*) ${argname}=\\$(echo \\$1 | sed 's/^[^=]*=//')\"\n                echo \"shift ;;\"\n                echo \"--${argname})\"\n                echo \"[ \\$# -gt 1 ] || invalid \\\"Missing arg for \\$1 option\\\"\"\n                echo \"${argname}=\\$2; shift 2 ;;\" ;;\n            *)\n                echo \"--${arg}) ${arg}=true; shift ;;\" ;;\n        esac\n    done\n    echo '--) done=--; shift ;;'\n    echo '--*) invalid \"Invalid option $1\" ;;'\n    echo '*) done=-- ;;'\n    echo 'esac'\n    echo 'done'\n    shift\n    while [ $# -gt 0 ]; do\n        arg=$1\n        case $arg in\n            \"...\") [ $# = 1 ] || internal_error \"... can only be last\"\n                   rest=ok ;;\n            \"[\"*\"]\") argname=$(echo \"$arg\" | sed 's/[][]//g')\n                     echo \"[ \\$# = 0 ] || { $argname=\\$1 ; shift; }\" ;;\n            *)\n                echo \"[ \\$# -gt 0 ] ||\"\n                echo \"invalid 'Missing argument $(bugit_upcase \"$arg\")'\"\n                echo \"{ $arg=\\$1 ; shift; }\" ;;\n        esac\n        shift\n    done\n    [ \"ok\" = \"$rest\" ] ||\n        echo \"[ \\$# = 0 ] || invalid 'Unexpected extra args:' \\\"\\$@\\\"\"\n}\n\ninvalid () {\n    echo \"$@\"; echo\n    source=$(which \"$0\")\n    sed -ne 's|^bugit_cmd_\\([^ ()]*\\).*#|bugit \\1|p' <\"$source\"\n    exit 1\n}\n\nuser_error () {\n    echo \"$@\"\n    exit 1\n}\n\ninternal_error () { user_error \"Internal error!\" \"$@\"; }\n\nbugit_get_id () {\n    bug=\"$1\"\n    [ ! \"\" = \"$bug\" ] || user_error \"Empty bug identifer!\"\n    if git show-ref \"bugs/$bug\" >/dev/null; then id=\"$bug\"; else\n        bugit_checkout_master\n        if [ -f \"numbers/$bug\" ]; then id=$(cat \"numbers/$bug\"); else\n            name=$(bugit_to_filename \"$bug\")\n            ids=$(ls \"names/$name\" 2>/dev/null)\n            case $ids in\n                \"\") user_error \"No bug by that name\" ;;\n                *\" \"*) user_error \"Ambiguous name: $ids\" ;;\n                *) id=\"$ids\" ;;\n            esac\n        fi\n    fi\n}\n\nbugit_in_master_p () {\n    [ \"ref: refs/heads/master\" = \"$(cat .git/HEAD)\" ]\n}\n\nbugit_get_number () {\n    bugit_in_master_p || internal_error \"bugit_get_number while not in master!\"\n    set -- $(cd numbers 2>/dev/null && grep -l \"$1\" * 2>/dev/null)\n    if [ \"$#\" = 0 ]; then return 1; else number=$1; fi\n}\n\nbugit_get_name () {\n    id=$1\n    bugit_get_branch \"$id\" nomerge\n    name=$(git cat-file blob \"$branch:name\")\n    if bugit_in_master_p; then\n        # Let's just double check that the \"names\" subdir is up-to-date\n        filename=$(bugit_to_filename \"$name\")\n        [ -f \"names/$filename/$id\" ] || {\n            touch \"names/$filename/$id\"\n            git add \"names/$filename/$id\"\n            # FIXME: If bugit_get_name is called several times (as is the\n            # case for \"bugit list\"), we'd want to combine all these commits\n            # into a single one.\n            git commit -m \"Update name of $id\"\n        }\n    fi\n}\n\nbugit_assert_clean_p () {\n    [ \"\" = \"$(git status --porcelain)\" ] || user_error \"Uncommitted changes!\"\n}\n\nbugit_checkout_master () {\n    bugit_in_master_p || {\n        bugit_assert_clean_p\n        git checkout master\n    }\n}\n\nbugit_get_branch () {           # Find the branch of a given bug-id\n    id=$1\n    nomerge=$2\n    if git show-ref -q --verify \"refs/heads/bugs/$id\"; then\n        branch=\"refs/heads/bugs/$id\"\n    else\n        branches=$(git for-each-ref --format \"%(refname)\" \\\n                       \"refs/remotes/*/bugs/$id\")\n        set -- $branches\n        case $# in\n            0) user_error \"No bug with id '$id'\" ;;\n            1) branch=$1 ;;\n            *) if [ \"nomerge\" = \"$nomerge\" ]; then\n                   branch=$1\n               else\n                   bugit_checkout_id \"$id\"\n                   branch=\"refs/heads/bugs/$id\"\n               fi ;;\n        esac\n    fi\n}\n\nbugit_merge () {\n    bug=$1\n    branch=$2\n    git merge -m Merge \"$branch\" ||\n        user_error \"Merge conflict in bug '$bug'\"\n}\n\nbugit_checkout_id () {          # Checkout the branch for bug ID.\n    id=$1\n    bugit_assert_clean_p\n    if git show-ref -q --verify \"refs/heads/bugs/$id\"; then\n        git checkout \"bugs/$id\"\n    else\n        branches=$(git for-each-ref --format \"%(refname)\" \\\n                       \"refs/remotes/*/bugs/$id\")\n        [ ! \"\" = \"$branches\" ] || user_error \"No bug with id '$id'\"\n        set -- $branches\n        first=\"$1\"; shift\n        git checkout -b \"bugs/$id\" \"$first\"\n        for branch; do\n            bugit_merge \"$id\" \"$branch\"\n        done\n    fi\n}\n\nbugit_author () {\n    echo \"$(git config --get user.name) <$(git config --get user.email)>\"\n}\n\nbugit_to_branchname () {\n    echo \"$@\" | tr ' ' '_'\n}\n\nbugit_to_filename () {\n    echo \"$@\" | tr '/' '_'\n}\n\nbugit_to_bugname () {\n    tr '_' '/'\n}\n\nbugit_generate_id () {\n    uuidgen 2>/dev/null ||\n        dd bs=1 count=16 </dev/urandom 2>/dev/null |\n            md5sum |\n            sed 's/ .*//'\n}\n\nbugit_add_attachments () {\n    if [ $# -gt 0 ]; then\n        # Add a timestamp to reduce the risk of conflict.\n        date=$(date \"+%Y-%m-%d %H:%M\")\n        mkdir -p attachments\n        for f; do\n            filename=\"$(date \"+%Y-%m-%d %H:%M\") - $(basename \"$f\")\"\n            cp \"$f\" \"attachments/$filename\"\n        done\n        git add attachments/\n    fi\n}\n\n## Commands ###################################################################\n\nbugit_cmd_init () {             # : Initialize a new bug database\n    eval \"$(bugit_make_optloop '')\"\n    git init \"$@\"\n    git commit --allow-empty -m 'Initial commit'\n}\n\nbugit_cmd_new () {              # [--author AUTHOR] NAME [ATTACHMENTS...]\n    eval \"$(bugit_make_optloop 'author=' name ...)\"\n    [ ! \"\" = \"$author\" ] || author=$(bugit_author)\n    id=$(bugit_generate_id)\n    \n    bugit_assert_clean_p\n    git checkout --orphan bugs/\"$id\" ||\n        user_error \"Can't create branch 'bugs/$id'\"\n    git rm -rf .\n    mkdir -p followers\n    bugit_add_attachments \"$@\"\n    touch followers/\"$author\"\n    echo \"$name\" >name\n    git add .\n    git commit\n\n    # Now record the name->id mapping in the master branch.\n    git checkout master\n    filename=$(bugit_to_filename \"$name\")\n    mkdir -p names/\"$filename\"\n    touch names/\"$filename\"/\"$id\"\n    git add names/\"$filename\"/\"$id\"\n    git commit -m \"Add name->id mapping for $name\"\n}\n\nbugit_cmd_reply () {            # [--author AUTHOR] BUG [ATTACHMENTS...]\n    eval \"$(bugit_make_optloop 'author=' bug ...)\"\n    [ ! \"\" = \"$author\" ] || author=$(bugit_author)\n    bugit_get_id \"$bug\";\n    bugit_checkout_id \"$id\"\n    bugit_add_attachments \"$@\"\n    git commit --allow-empty\n}\n\nbugit_cmd_show () {             # BUG : Display the bug's content\n    eval \"$(bugit_make_optloop '' bug)\"\n    bugit_get_id \"$bug\"\n    git log --no-merges --reverse bugs/\"$id\"\n}\n\nbugit_cmd_list () {             # : List all bugs in the database\n    eval \"$(bugit_make_optloop '')\"\n    bugit_checkout_master\n    for id in $(git branch -a --list 'bugs/*' '*/bugs/*' |\n                       sed 's|^..\\(.*/\\)\\?bugs/||' | sort -u); do\n        bugit_get_name \"$id\" || name=\"\"\n        if bugit_get_number \"$id\"; then\n            echo \"bug#$number: $name\"\n        else\n            echo \"$id: $name\"\n        fi\n    done\n}\n\nbugit_cmd_number () {           # BUG... : Assign numbers to bugs\n    eval \"$(bugit_make_optloop '' ...)\"\n    # TODO: Allow several BUGs at a time, or \n    [ $# -gt 0 ] ||\n        # FIXME: when no BUG is specified, we should do it for all\n        # un-numbered bugs.\n        user_error \"Have to identify bugs explicitly\"\n    bugit_checkout_master\n    for bug; do\n        bugit_get_id \"$bug\"\n        nb=$(cd numbers 2>/dev/null && grep -l \"$id\" * 2>/dev/null)\n        [ \"\" = \"$nb\" ] || {\n            echo \"Already assigned number $nb to bug '$bug'\"\n            continue\n        }\n        mkdir -p numbers\n        # FIXME: Randomize this number somewhat, so that bug-numbering\n        # can be done offline as well!\n        last=$( (echo 0; ls numbers) | sort -n | tail -n 1)\n        nb=$(($last + 1))\n        echo \"$id\" >\"numbers/$nb\"\n        git add \"numbers/$nb\"\n    done\n    git commit -m \"Assign some bug numbers\"\n}\n\nbugit_cmd_push () {             # [--subset] REMOTE [BUG]\n    eval \"$(bugit_make_optloop 'subset' [remote] [bug])\"\n    [ ! \"\" = \"$remote\" ] || remote=origin\n    # FIXME: If we don't push \"master\", the remote \"master\" will have\n    # an incomplete \"names\" subdir!  Maybe we could fix it lazily (by improving\n    # the \"names\" cache so we can detect its staleness) or with a push-hook.\n    if [ \"true\" = \"$subset\" ]; then\n        [ \"\" = \"$bug\" ] || invalid \"Option --subset is redundant with $bug\"\n        git push \"$remote\" 'bugs/*:bugs/*'\n    elif [ \"\" = \"$bug\" ]; then\n        # FIXME: We can't \"--prune\" here since the remote may have some new\n        # branches.  That basically means there's no way to remove bugs/ID\n        # since the next \"bugit push\" will bring it right back!\n        git push --all \"$remote\"\n    else\n        bugit_get_id \"$bug\"\n        git push \"$remote\" \"bugs/$id:bugs/$id\"\n    fi\n}\n\nbugit_cmd_pull () {\n    eval \"$(bugit_make_optloop 'bugsonly' [remote] [bug])\"\n    # TODO: Same as push, with single bug, and matching bugs-only.\n    [ ! \"\" = \"$remote\" ] || remote=origin\n\n    if [ \"true\" = \"$subset\" ]; then\n        [ \"\" = \"$bug\" ] || invalid \"Option --subset is redundant with $bug\"\n        pattern=\"bugs/\"\n    elif [ \"\" = \"$bug\" ]; then\n        pattern=\"\"\n    else\n        bugit_get_id \"$bug\"\n        pattern=\"bugs/$id\"\n    fi\n    \n    git fetch \"$remote\"\n\n    for branch in $(git for-each-ref --format \"%(refname)\" \\\n                        \"refs/remotes/$remote/$pattern\"); do\n        local=$(echo \"$branch\" | sed 's|refs/remotes/[^/]*/||')\n        if [ \"HEAD\" = \"$local\" ]; then echo \"Skipping HEAD\"\n        elif ! git show-ref -q --verify \"refs/heads/$local\"; then\n            echo \"branch $branch has no local equivalent\"\n        elif [ \"\" = \"$(git rev-list \"$local..$branch\")\" ]; then\n            echo \"branch $branch has nothing new\"\n        else\n            bugit_assert_clean_p\n            git checkout \"$local\"\n            bugit_merge \"$local\" \"$branch\"\n        fi\n    done\n}\n\nbugit_cmd_id () {               # BUG : Return the id of BUG\n    eval \"$(bugit_make_optloop '' bug)\"\n    bugit_get_id \"$bug\"\n    echo \"$id\"\n}\n\nbugit_cmd_name () {             # BUG : Return the name of BUG\n    eval \"$(bugit_make_optloop '' bug)\"\n    bugit_get_id \"$bug\"\n    bugit_get_name \"$id\"\n    echo \"$name\"\n}\n\nbugit_cmd_rename () {           # BUG NEWNAME\n    eval \"$(bugit_make_optloop '' bug newname ...)\"\n    bugit_get_id \"$bug\"\n    bugit_get_name \"$id\"\n    bugit_checkout_id \"$id\"\n    echo \"$newname\" \"$@\" >name\n    git add name\n    git commit -m \"Rename\"\n    bugit_checkout_master\n    [ \"\" = \"$name\" ] ||\n        rm -f \"names/$(bugit_to_filename \"$name\")/$id\"\n    mkdir -p \"names/$(bugit_to_filename \"$newname\")\"\n    touch \"names/$(bugit_to_filename \"$newname\")/$id\"\n    git add names\n    git commit -m \"Rename $bug\"\n}\n\nbugit_cmd_tag () {              # BUG TAG\n    eval \"$(bugit_make_optloop '' bug tag)\"\n    set -- $tag\n    [ $# = 1 ] || user_error \"Invalid tag name '$tag'\"\n    bugit_get_id \"$bug\"\n    bugit_checkout_id \"$id\"\n    mkdir -p tags\n    touch \"tags/$tag\" || user_error \"Invalid tag name '$tag'\"\n    git add \"tags/$tag\"\n    git commit -m 'Add tag'\n}\n\nbugit_cmd_untag () {              # BUG TAG\n    eval \"$(bugit_make_optloop '' bug tag)\"\n    set -- $tag\n    [ $# = 1 ] || user_error \"Invalid tag name '$tag'\"\n    bugit_get_id \"$bug\"\n    bugit_checkout_id \"$id\"\n    rm -f \"tags/$tag\"\n    git add \"tags/$tag\"\n    git commit -m 'Remove tag'\n}\n\nbugit_cmd_taglist () {          # BUG\n    eval \"$(bugit_make_optloop '' bug)\"\n    bugit_get_id \"$bug\"\n    bugit_get_branch \"$id\"\n    tags=\"\"\n    echo $(git cat-file -p \"$branch:tags\" | while read mode type hash name; do\n                  echo $name\n              done)\n}\n\nbugit_cmd_assign () {           # [--only] BUG EMAIL\n    eval \"$(bugit_make_optloop 'only' bug email)\"\n    bugit_get_id \"$bug\"\n    bugit_checkout_id \"$id\"\n    mkdir -p assigned-to\n    if [ \"true\" = \"only\" ]; then rm -f assigned-to/*; fi\n    touch \"assigned-to/$email\" || user_error \"Invalid email name '$email'\"\n    git add \"assigned-to/\"\n    git commit -m 'Assign'\n}\n\nbugit_cmd_unassign () {         # BUG EMAIL\n    eval \"$(bugit_make_optloop '' bug email)\"\n    bugit_get_id \"$bug\"\n    bugit_checkout_id \"$id\"\n    rm -rf \"assigned-to/$email\"\n    git add \"assigned-to/\"\n    git commit -m 'Unassign'\n}\n\nbugit_cmd_assigned () {         # BUG\n    eval \"$(bugit_make_optloop '' bug)\"\n    bugit_get_id \"$bug\"\n    bugit_get_branch \"$id\"\n    git cat-file -p \"$branch:assigned-to\" | while read mode type hash name; do\n        echo \"$name\"\n    done\n}\n\nbugit_cmd_follow () {           # BUG EMAIL\n    eval \"$(bugit_make_optloop '' bug email)\"\n    bugit_get_id \"$bug\"\n    bugit_checkout_id \"$id\"\n    mkdir -p followers\n    touch \"followers/$email\" || user_error \"Invalid email name '$email'\"\n    git add \"followers/\"\n    git commit -m 'Add follower'\n}\n\n# TODO: Severity\n# \n\n[ \"$#\" -gt 0 ] || invalid \"BuGit usage:\"\ncmd=$1; shift\n\ntype \"bugit_cmd_$cmd\" >/dev/null ||\n    invalid \"Unknown BuGit command '$cmd'\"\n\n\"bugit_cmd_$cmd\" \"$@\"\n"},{"id":"277240","messageId":"jwvbn7zksha.fsf-monnier+gmane.comp.version-control.git@gnu.org","threadId":"40922","inReplyTo":"jwva8psr6vr.fsf-monnier+gmane.comp.version-control.git@gnu.org","subject":"Re: BuGit: File-less distributed issue tracking system with Git","fromName":"Stefan Monnier","fromEmail":"monnier@iro.umontreal.ca","sentAt":"2016-02-02T16:13:34Z","receivedAt":"2016-02-02T16:13:34Z","isPatch":false,"sender":{"key":"monnier@iro.umontreal.ca","avatar":null},"body":"> So, see attached BuGit, an issue tracking system which stores its\n> database in Git to try and get \"distributed operation for free\".\n\nIt's now hosted at https://gitlab.com/monnier/bugit\n\nIn the mean time it grew to 80KB, offers a read-only web UI, email\nnotifications, and a fairly complete command line UI.\n\nIt also changed its internal representation slightly, so it can now be\nused within the same Git repository as the project on which you're\nworking (e.g. I cloned the BuGit source code and BuGit's bug database\ninto the same repository, so I can use bugit directly from the source\ncode directory to manipulate its bugs).\n\nYou can even host the bug database and the source code of your project\nin the same repository (the post-receive hook script can be told to pass\nthe non-bugit-related changes to some other script).\n\n\n        Stefan\n"}]}