Caption UI/UX - Display Emotive and Paralinguistic Information in Captions

Emotive and paralinguistic information are typically missing from conventional closed captions. To include some of this information in closed captions, we generated graphical representations of this information that is normally represented with descriptive words. We evaluate them in captions and report on how participants perceive them. These viewers then provided verbal and written feedback regarding positive and negative aspects of the various captions. We found that hard of hearing viewers were significantly more positive about this style of captioning than deaf viewers and that some viewers believed that these augmentations were useful and enhanced their viewing experience.