Electron energy loss spectroscopy (EELS) in a monochromated transmission electron microscope is applied to probe standing-wave-like cavity modes hybridized with surface plasmon polaritons (SPP) in rectangular submicron slits in a thin gold film. Coupling of hybridized SPP-cavity modes between two adjacent slits is studied by systematically varying the width of the metal bar d that separates the identical slits in a two-slit system. Measurements on two-slit systems with different slit lengths L and fixed width reveal energy shifts and mode splitting of the fundamental SPP cavity mode which can be generally described as a function of a dimensionless scaling parameter L/d. Numerical simulations with the Discontinuous Galerkin Time-Domain (DGTD) method confirm the experimental data and reveal insights into the underlying complex coupling mechanisms.