To compare different joke generation systems, some basic quality measures are needed. However, until now a standard method of evaluating humor generation systems are still missing. Most approaches use humorousness of the output as an evaluation criterion, and let actual human rate the jokes. The main challenge of this method is that it’s highly subjective. As an alternative, Binsted and Ritchie (1997) suggest that to evaluate jokes based on whether people can tell it apart from a human-generated joke.