We investigate the steady-state nonconservative open-system dynamics of an atom in a generic complex structured electromagnetic environment at finite temperature $T$. In such systems, when the atom moves along a translation-invariant axis of the environment, a frictional force acts on the particle. The effective viscosity due to friction results from the nonequilibrium interaction with the fluctuating (quantum) electromagnetic field, which effectively sets a privileged reference frame. We study the impact of both quantum and thermal fluctuations on the interaction, highlighting how they induce qualitatively different types of viscosity. To this end, we develop a self-consistent non-Markovian description that contains quantum and blackbody friction as special cases. In particular, we show how the interplay between the nonequilibrium dynamics, the quantum and the thermal properties of the radiation, as well as the confinement of light at the vacuum-material interface is responsible for several intriguing features. Our analysis is relevant for an experimental test of noncontact friction and the resulting electromagnetic viscosity