How does git follow branch history across a merge commit?

This question concerns git's representation of merge commits.

I understand that branches in git are references to commit vertices in a graph, that each of these vertices have zero or more reference edges to a parent (predecessor) vertex, and that merges create vertices with at least two parents. Walking back in time from a branch head involves traversing parent edges.

But what happens when such traversal reaches a merge commit? If one asks git to move back, say, in time by a few days, or so many predecessors, how does it know (or choose) which way to go upon reaching a merge commit?

I set up an experiment with two competing branches that require merging back together:

,----
| mkdir gitexp && cd gitexp && git init
| $EDITOR file1.txt
| git add file1.txt
| git commit -m'Initial impound.'
| $EDITOR file1.txt
| git commit -a -m'Added a second line.'
| 
| git checkout -b competition
| $EDITOR file1.txt
| git commit -a -m'Extended second line.'
| 
| git checkout master
| $EDITOR file1.txt
| git commit -a -m'Added third line.'
| 
| git checkout competition
| $EDITOR file1.txt
| git commit -a -m'Edited second line.'
| $EDITOR file1.txt
| git commit -a -m'Added an alternate third line.'
| 
| git checkout master
| git merge competition
| # Edit to resolve conflict:
| $EDITOR file1.txt
| git commit -a
| 
| git checkout 'HEAD^'
| # At this point, git picked the predecessor along branch "master".
`----

Was it just luck that "HEAD^" referred to the predecessor that came from branch "master" rather than branch "competition"?