The Casimir-Polder force acts on polarizable particles due to quantum fluctuations of the electromagnetic field that are modified by the presence of material bodies. We investigate the Casimir-Polder interaction for atoms near cylindrical fibers with hollow cores. This geometry represents one of the archetypal configurations encountered in numerous experimental setups designed to control and manipulate atoms in fundamental and quantum technological applications. Specifically, we analyze how the interplay of both geometrical and material-related length scales characterize the interaction, emphasizing the impact of the shell thickness. We develop a flexible and fast-converging numerical scheme for evaluating the interaction over a wide range of atom-cylinder separations at both zero and finite temperature. Furthermore, we provide a detailed analytical investigation of how various material properties modify the Casimir-Polder potential. Finally, we analyze and discuss a number of limiting cases and compare numerical computations with corresponding analytical asymptotic expressions. In particular, in this geometry the Casimir-Polder potential is able to distinguish between an Ohmic and non-Ohmic description of conductors. One of the most significant outcomes of our work is that the shell thickness emerges as a useful parameter for controlling the interaction, opening avenues for both fundamental physics and applications in quantum technologies.