There is growing interest in utilising virtual environments (VEs) in the context of sports. In particular there is a desire to be able to improve sensorimotor skills rather than just using a VE as a tool for strategy analysis, or entertainment. The range of skills required across all different sports is very large and varied. This review of the state-of-the-art, therefore focuses on just ball sports, and was carried out as part of a current project developing training tools for rugby. A VE needs to provide realistic rendering of the sports scene to achieve good perceptual fidelity. More important for a sport-themed VE is high functional fidelity, which requires an accurate physics model of a complex environment, real time response, and a natural user interface. The goal is to provide multiple scenarios to players at different levels of difficulty, providing them with improved skills that can be applied directly to the real sports arena. The typical hardware and software components needed are identified in the paper, and important psychological factors that should be considered are discussed. The challenges that need to be overcome are identified and illustrated with examples from recent projects in this area. (c) 2012 Elsevier Ltd. All rights reserved.