We present a full analysis of the mode spectrum in a cavity formed by two parallel plates, one of which is magnetodielectric (metamaterial) while the other one is metallic, and obtain dispersion relations in closed form. The optical properties of the cavity walls are described in terms of realistic models for effective permittivity and permeability. Surface polaritons, i.e., electromagnetic modes that have at least partly an evanescent character, are shown to dominate the Casimir interaction at small separations. We analyze in detail the s-polarized polaritons, which are a characteristic feature of a magnetodielectric configuration, and discuss their role in the repulsive Casimir force.