Our paper on multi-head routing strategies for MoEs has been accepted at NeurIPS 2023