A description of the methods used to build a high quality, comprehensive reference library of electron-ionization mass spectra is presented. Emphasis is placed on the most challenging part of this project--the improvement of quality by expert evaluation. The methods employed for this task were developed over the course of a spectrum-by-spectrum review of a library containing well over 100,000 spectra. Although the effectiveness of this quality improvement task depended critically on the expertise of the evaluators, a number of guidelines are discussed which were found to be effective in performing this onerous and often subjective task. A number of specific examples of the particularly challenging task of spectrum editing are given.