We take familiar inorganic oxide glasses and non-oxide glasses and the liquids from which they derive to review the current understanding of their atomic structure, ranging from the local environments of individual atoms to the long-range order which can cover many interatomic distances. The structural characteristics of important glasses and melts, like silicates, borates, alumino-silicates, halides and chalcogenides, are drawn from the results of recent spectroscopy and scattering experiments. The techniques include Nuclear Magnetic Resonance (NMR) and X-ray Absorption Fine Structure (XAFS), Neutron Scattering (NS) and Small- and Wide-angle X-ray Scattering measurements (SAXS/WAXS), and are often combined with computer simulation experiments in order to obtain detailed images of structure and diffusion in the glassy as well as in the molten state. We then review the current understanding of relaxation in glasses, liquids and polyamorphic states. This includes phenomenological models and theories of relaxation in different dynamical regimes, spectroscopic studies of atomic-scale mechanisms of viscous flow in inorganic glass-formers and the signatures of relaxational behaviour embedded in the low-frequency vibrational dynamics of glasses including the Boson peak and the Two-Level Systems (TLS) that control conformational transformation. We conclude this review by extending concepts of the dynamics of the glass transition from the supercooled liquid in order to understand the solid-state amorphization of crystals under temperature and pressure and to determine the thermodynamic limits of the crystalline and glassy state.