Riley's Stuttering Severity Instrument (SSI) is widely used. The manuals allow SSI assessments to be made in different ways (e.g. from digital recordings or whilst listening to speech live). Digital recordings allow segments to be selected and listened to, whereas the entire recording has to be judged when listened to live. Comparison was made between expert judges when they used these digital and live procedures to establish whether one method was more sensitive and reliable than the other. Five expert judges assessed eight speakers four times each in two judgment conditions (digital vs. live). The eight speakers were chosen so that they spanned a wide range of stuttering severity. SSI version 3 (SSI-3) estimates were obtained on all occasions. An ANOVA showed a three-way interaction between sessions, speakers and condition that indicated that digital and live judgments varied across speakers and across sessions. The predictions that were upheld were (1) SSI-3 scores made from digital segments are more sensitive than SSI-3 scores made on the entire live signal; (2) digital and live judgments vary with respect to speaker's stuttering severity and across test sessions.