1 Does the summary build from sentence to sentence to a coherent
  body of information about the topic?
    A. Very coherently
    B. Somewhat coherently
    C. Neutral as to coherence
    D. Not so coherently
    E. Incoherent

2 If you were editing the summary to make it more concise and
  to the point, how much useless, confusing or repetitive text would
  you remove from the existing summary?
    A. None
    B. A little
    C. Some
    D. A lot
    E. Most of the text 
    
3 To what degree does the summary say the same thing over again?
    A. None; the summary has no repeated information
    B. Minor repetitions
    C. Some repetiton
    D. More than half of the text is repetitive
    E. Quite a lot; most sentences are repetitive

4 How much trouble did you have identifying the referents of noun
  phrases in this summary? Are there nouns, pronouns or
  personal names that are not well-specified? For example, a person is
  mentioned and it is not clear what his role in the story is, or any
  other entity that is referenced but its identity and relation with
  the story remains unclear
    A. No problems; it is clear who/what is being referred to throughout.
    B. Slight problems, mostly cosmetic/stylistic
    C. Somewhat problematic; some minor events/things/people/places
       are unclear,or a very few major ones, but overall the who and what
       are clear.
    D. Rather problematic; enough events/things/people/places are
       unclear that parts of the summary are hard to understand
    E. Severe problems; main events, characters or places are not 
       well-specified and/or it's difficult to say how they relate to
       the topic

5 To what degree do you think the entities (person/thing/event/place/...)
  were re-mentioned in an overly explicit way, so that readability
  was impaired? For example, a pronoun could have been used instead
  of a lengthy description, or a shorter discription would have been 
  more appropriate?
    A. None: references to entities were acceptably explicit
    B. A little: once or twice, an entity was over-described
    C. Somewhat: to a noticeable but not annoying degree, some
       entities were over-described
    D. Rather problematic: to a degree that became distracting,
       entities were over-described
    E. A lot: reintroduction of characters and entities made
        reading difficult/caused comprehension problems

6  Are there any obviously ungrammatical sentences, e.g.,
   missing components, unrelated fragments or any other
   grammar-related problem that makes the text diffcult to
   read.
     A. No noticeable grammatical problems 
     B. Minor grammar problems
     C. Some problems, but overall acceptable 
     D. A fair amount of grammatical errors 
     E. Too many problems, the summary is impossible to read	

7 Are there any datelines, system-internal formatting or
  capitalization errors that can make the reading of the summary
  difficult?
     A. No noticeable formatting problems
     B. Minor formatting problems
     C. Some, but they do not create any major difficulties
     D. A fair amount of formatting problems
     E. Many, to an extent that reading is difficult