In this paper, we introduce Schwartz operators as a non-commutative analog of Schwartz functions and provide a detailed discussion of their properties. We equip them, in particular, with a number of different (but equivalent) families of seminorms which turns the space of Schwartz operators into a Fréchet space. The study of the topological dual leads to non-commutative tempered distributions which are discussed in detail as well. We show, in particular, that the latter can be identified with a certain class of quadratic forms, therefore making operations like products with bounded (and also some unbounded) operators and quantum harmonic analysis available to objects which are otherwise too singular for being a Hilbert space operator. Finally, we show how the new methods can be applied by studying operator moment problems and convergence properties of fluctuation operators.