Their intrinsic properties render single quantum systems as ideal tools for quantum enhanced sensing and microscopy. As an additional benefit, their size is typically on an atomic scale that enables sensing with very high spatial resolution. Here, we report on utilizing a single nitrogen vacancy center in nanodiamond for performing three-dimensional scanning-probe fluorescence lifetime imaging microscopy. By measuring changes of the single emitter’s lifetime, information on the local density of optical states is acquired at the nanoscale. Three-dimensional ab initio discontinuous Galerkin time-domain simulations are used in order to verify the results and to obtain additional insights. This combination of experiment and simulations to gather quantitative information on the local density of optical states is of direct relevance for the understanding of fundamental quantum optical processes as well as for the engineering of novel photonic and plasmonic devices.