We investigate the action of the first three levels of the Clifford hierarchy on sets of mutually unbiased bases comprising the Ivanovic mutually unbiased base (MUB) and the Alltop MUBs. Vectors in the Alltop MUBs exhibit additional symmetries when the dimension is a prime number equal to 1 modulo 3 and thus the set of all Alltop vectors splits into three Clifford orbits. These vectors form configurations with so-called Zauner subspaces, eigenspaces of order 3 elements of the Clifford group highly relevant to the SIC problem. We identify Alltop vectors as the magic states that appear in the context of fault- tolerant universal quantum computing, wherein the appearance of distinct Clifford orbits implies a surprising inequivalence between some magic states.