IX, ALTERNATIVE TEST 1: NAMING THE MONTHS

PROCEDURE. Simply ask the subject to "name all the months of the year." Do not start him off by naming one month; give no look of approval or disapproval as the months are being named, and make no suggestions or comments of any kind.

When the months have been named, we "check up" the performance by asking: "What month comes before April?" "What month comes before July?" "What month comes before November?"

SCORING. Passed if the months are named in about fifteen or twenty seconds with no more than one error of omission, repetition, or displacement, and if two out of the three check questions are answered correctly. Disregard place of beginning.

REMARKS. Some are inclined to consider this test of little value, because of its supposed dependence on accidental training. With this opinion we cannot fully agree. The arguments already given in favor of the retention of naming the days of the week (year VII), apply equally well in the present case. It has been shown, however, that age, apart from intelligence, does have some effect on the ability to name the months. Defective adults of 9-year intelligence do about as well with it as normal children of 10-year intelligence.

The test appears in year X of Binet's 1908 scale and in year IX of the 1911 revision. Goddard places it correctly in year IX, while Kuhlmann and Bobertag have omitted it.

IX, ALTERNATIVE TEST 2: COUNTING THE VALUE OF STAMPS

PROCEDURE. Place before the subject a cardboard on which are pasted three 1-cent and three 2-cent stamps arranged as follows: 111222. Be sure to lay the card so that the stamps will be right side up for the child. Say: "You know, of course, how much a stamp like this costs (pointing to a 1-cent stamp). And you know how much one like this costs (pointing to a 2-cent stamp). Now, how much money would it take to buy all these stamps?"

Do not tell the individual values of the stamps if these are not known, for it is a part of the test to ascertain whether the child's spontaneous curiosity has led him to find out and remember their values. If the individual values are known, but the first answer is wrong, a second trial may be given. In such cases, however, it is necessary to be on guard against guessing.

If the child merely names an incorrect sum without saying anything to indicate how he arrived at his answer, it is well to tell him to figure it up aloud. "Tell me how you got it."

SCORING. Passed if the correct value is given in not over fifteen seconds.

REMARKS. The value of this test may be questioned on two grounds: (1) That it has an ambiguous significance, since failure to pass it may result either from incorrect addition or from lack of knowledge of the individual values of the stamps; (2) that familiarity with stamps and their values is so much a matter of accident and special instruction that the test is not fair.

Both criticisms are in a measure valid. The first, however, applies equally well to a great many useful intelligence tests. In fact, it is only a minority in which success depends on but one factor. The other criticism has less weight than would at first appear. While it is, of course, not impossible for an intelligent child to arrive at the age of 9 years without having had reasonable opportunity to learn the cost of the common postage stamps, the fact is that a large majority have had the opportunity and that most of those of normal intelligence have taken advantage of it. It is necessary once more to emphasize the fact that in its method of locating a test the Binet system makes ample allowance for "accidental" failures.

Like the tests of naming coins, repeating the names of the days of the week or the months of the year, giving the date, tying a bow-knot, distinguishing right and left, naming the colors, etc., this one also throws light on the child's spontaneous interest in common objects. It is mainly the children of deficient intellectual curiosity who do not take the trouble to learn these things at somewhere near the expected age.

The test was located in year VIII of the Binet scale. However, Binet used coins, three single and three double sous. Since we do not have either a half-cent or a 2-cent coin, it has been necessary to substitute postage stamps. This changes the nature of the test and makes it much harder. It becomes less a test of ability to do a simple sum, and more a test of knowledge as to the value of the stamps used. That the test is easy enough for year VIII when it can be given in the original form is indicated by all the French, German, and English statistics available, but four separate series of Stanford tests agree in finding it too hard for year VIII when stamps are substituted and the test is carried out according to the procedure described above.



CHAPTER XVI

INSTRUCTIONS FOR YEAR X

X, 1. VOCABULARY (THIRTY DEFINITIONS, 5400 WORDS)

PROCEDURE AND SCORING AS IN VIII, 6. At year X, thirty words should be correctly defined.

X, 2. DETECTING ABSURDITIES

PROCEDURE. Say to the child: "I am going to read a sentence which has something foolish in it, some nonsense. I want you to listen carefully and tell me what is foolish about it." Then read the sentences, rather slowly and in a matter-of-fact voice, saying after each: "What is foolish about that?" The sentences used are the following:—

(a) "A man said: 'I know a road from my house to the city which is downhill all the way to the city and downhill all the way back home.'" (b) "An engineer said that the more cars he had on his train the faster he could go." (c) "Yesterday the police found the body of a girl cut into eighteen pieces. They believe that she killed herself." (d) "There was a railroad accident yesterday, but it was not very serious. Only forty-eight people were killed." (e) "A bicycle rider, being thrown from his bicycle in an accident, struck his head against a stone and was instantly killed. They picked him up and carried him to the hospital, and they do not think he will get well again."

Each should ordinarily be answered within thirty seconds. If the child is silent, the sentence should be repeated; but no other questions or suggestions of any kind are permissible. Such questions as "Could the road be downhill both ways?" or, "Do you think the girl could have killed herself?" would, of course, put the answer in the child's mouth. It is even best to avoid laughing as the sentence is read.

Owing to the child's limited power of expression it is not always easy to judge from the answer given whether the absurdity has really been detected or not. In such cases ask him to explain himself, using some such formula as: "I am not sure I know what you mean. Explain what you mean. Tell me what is foolish in the sentence I read." This usually brings a reply the correctness or incorrectness of which is more apparent, while at the same time the formula is so general that it affords no hint as to the correct answer. Additional questions must be used with extreme caution.

SCORING. Passed if the absurdity is detected in four out of the five statements. The following are samples of satisfactory and unsatisfactory answers:—

(a) The road downhill

Satisfactory. "If it was downhill to the city it would be uphill coming back." "It can't be downhill both directions." "That could not be." "That is foolish. (Explain.) Because it must be uphill one way or the other." "That would be a funny road. (Explain.) No road can be like that. It can't be downhill both ways."

Unsatisfactory. "Perhaps he took a little different road coming back." "I guess it is a very crooked road." "Coming back he goes around the hill." "The man lives down in a valley." "The road was made that way so it would be easy." "Just a road. I don't see anything foolish." "He should say, 'a road which goes.'"

(b) What the engineer said

Satisfactory. "If he has more cars he will go slower." "It is the other way. If he wants to go faster he mustn't have so many cars." "The man didn't mean what he said, or else it was a slip of the tongue." "That's the way it would be if he was going downhill." "Foolish, because the cars don't help pull the train." "He ought to say slower, not faster."

Unsatisfactory. "A long train is nicer." "The engine pulls harder if the train has lots of cars." "That's all right. I suppose he likes a big train." "Nothing foolish; when I went to the city I saw a train that had lots of cars and it was going awfully fast." "He should have said, 'the faster I can run.'"

(c) The girl who was thought to have killed herself

Satisfactory. "She could not have cut herself into eighteen pieces." "She would have been dead before that." "She might have cut two or three pieces off, but she couldn't do the rest." (Laughing) "Well, she may have killed herself; but if she did it's a sure thing that some one else came along after and chopped her up." "That policeman must have been a fool. (Explain.) To think that she could chop herself into eighteen pieces."

Unsatisfactory. "Think that she killed herself; they know she did." "They can't be sure. Some one may have killed her." "It was a foolish girl to kill herself." "How can they tell who killed her?" "No girl would kill herself unless she was crazy." "It ought to read: 'They think that she committed suicide.'"

(d) The railroad accident

Satisfactory. "That was very serious." "I should like to know what you would call a serious accident!" "You could say it was not serious if two or three people were killed, but forty-eight,—that is serious."

Unsatisfactory. "It was a foolish mistake that made the accident." "They couldn't help it. It was an accident." "It might have been worse." "Nothing foolish; it's just sad."

(e) The bicycle rider

Satisfactory. "How could he get well after he was already killed?" "Why, he's already dead." "No use to take a dead man to the hospital." "They ought to have taken him to a grave-yard!"

Unsatisfactory. "Foolish to fall off of a bicycle. He should have known how to ride." "They ought to have carried him home. (Why?) So his folks could get a doctor." "He should have been more careful." "Maybe they can cure him if he isn't hurt very bad." "There's nothing foolish in that."

REMARKS. The detection of absurdities is one of the most ingenious and serviceable tests of the entire scale. It is little influenced by schooling, and it comes nearer than any other to being a test of that species of mother-wit which we call common sense. Like the "comprehension questions," it may be called a test of judgment, using this term in the colloquial and not in the logical sense. The stupid person, whether depicted in literature, proverb, or the ephemeral joke column, is always (and justly, it would seem) characterized by a huge tolerance for absurd contradictions and by a blunt sensitivity for the fine points of a joke. Intellectual discrimination and judgment are inferior. The ideas do not cross-light each other, but remain relatively isolated. Hence, the most absurd contradictions are swallowed, so to speak, without arousing the protest of the critical faculty. The latter, indeed, is only a name for the tendency of intellectually irreconcilable elements to clash. If there is no clash, if the elements remain apart, it goes without saying that there will be no power of criticism.

The critical faculty begins its development in the early years and strengthens pari passu with the growing wealth of inter-associations among ideas; but in the average child it is not until the age of about 10 years that it becomes equal to tasks like those presented in this test. Eight-year intelligence hardly ever scores more than two or three correct answers out of five. By 12, the critical ability has so far developed that the test is nearly always passed. It is an invaluable test for the higher grades of mental deficiency.

As a test of the critical powers Binet first used "trap questions"; as, for example, "Is snow red or black?" The results were disappointing, for it was found that owing to timidity, deference, and suggestibility normal children often failed on such questions. Deference is more marked in normal than in feeble-minded children, and it is because of the influence of this trait that it is necessary always to forewarn the subject that the sentence to be given contains nonsense.

Binet located the test in year XI of the 1908 scale, but changed it to year X in 1911. Goddard and Kuhlmann retain it in year XI. The large majority of the statistics, including those of Goddard and Kuhlmann, warrant the location of the test in year X. Not all have used the same absurdities, and these have not been worded uniformly. Most have required three successes out of five, but Bobertag and Kuhlmann require three out of four; Bobertag's procedure is also different in that he does not forewarn the child that an absurdity is to follow.

The present form of the test is the result of three successive refinements. It will be noted that we have made two substitutions in Binet's list of absurdities. Those omitted from the original scale are: "I have three brothers—Paul, Ernest, and myself," and, "If I were going to commit suicide I would not choose Friday, because Friday is an unlucky day and would bring me misfortune." The last has a puzzling feature which makes it much too hard for year X, and the other is objectionable with children who are accustomed to hear a foreign language in which the form of expression used in the absurdity is idiomatically correct.

The two we have substituted for these objectionable absurdities are, "The road downhill" and "What the engineer said." The five we have used, though of nearly equal difficulty, are here listed in the order from easiest to hardest. Our series as a whole is slightly easier than Binet's.

X, 3. DRAWING DESIGNS FROM MEMORY

PROCEDURE. Use the designs shown on the accompanying printed form. If copies are used they must be exact in size and shape. Before showing the card say: "This card has two drawings on it. I am going to show them to you for ten seconds, then I will take the card away and let you draw from memory what you have seen. Examine both drawings carefully and remember that you have only ten seconds."

Provide pencil and paper and then show the card for ten seconds, holding it at right angles to the child's line of vision and with the designs in the position given in the plate. Have the child draw the designs immediately after they are removed from sight.

SCORING. The test is passed if one of the designs is reproduced correctly and the other about half correctly. "Correctly" means that the essential plan of the design has been grasped and reproduced. Ordinary irregularities due to lack of motor skill or to hasty execution are disregarded. "Half correctly" means that some essential part of the design has been omitted or misplaced, or that parts have been added.

The sample reproductions shown on the scoring card will serve as a guide. It will be noted that an inverted design, or one whose right and left sides have been transposed, is counted only half correct, however perfect it many be in other respects; also that design b is counted only half correct if the inner rectangle is not located off center.

REMARKS. Binet states that the main factors involved in success are "attention, visual memory, and a little analysis." The power of rapid analysis would seem to be the most important, for if the designs are analyzed they may be reproduced from a verbal memory of the analysis. Without some analysis it would hardly be possible to remember the designs at all, as one of them contains thirteen lines and the other twelve. The memory span for unrelated objects is far too limited to permit us to grasp and retain that number of unrelated impressions. Success is possible only by grouping the lines according to their relationships, so that several of them are given a unitary value and remembered as one. In this manner, the design to the right, which is composed of twelve lines, may be reduced to four elements: (1) The outer rectangle; (2) the inner rectangle; (3) the off-center position of the inner rectangle; and (4) the joining of the angles. Of course the child does not ordinarily make an analysis as explicit as this; but analysis of some kind, even though it be unconscious, is necessary to success.

Ability to pass the test indicates the presence, in a certain definite amount, of the tendency for the contents of consciousness to fuse into a meaningful whole. Failure indicates that the elements have maintained their unitary character or have fused inadequately. It is seen, therefore, that the test has a close kinship with the test of memory for sentences. The latter, also, permits the fusion or grouping of impressions according to meaning, with the result that five or six times as many meaningful syllables as nonsense syllables or digits can be retained.

Binet had many more failures on design a than on design b. This was probably due to the fact that he showed the designs with our b to the left. A majority of subjects, probably because of the influence of reading habits, examine first the figure to the left, and because of the short time allowed for the inspection are unable to devote much time to the design at the right. We have placed the design of greater intrinsic difficulty at the left, with the result that the failures are almost equally divided between the two.

Binet used this test in his unstandardized series of 1905, omitted it in 1908, but included it in the 1911 revision, locating it in year X. Except for Goddard, who recommends year XI, there is rather general agreement that the test belongs at year X. Our own data show that it may be placed either at year X or year XI, according as the grading is rigid or lenient.

X, 4. READING FOR EIGHT MEMORIES

MATERIAL. We use Binet's selection, slightly adapted, as follows:—

New York, September 5th. A fire last night burned three houses near the center of the city. It took some time to put it out. The loss was fifty thousand dollars, and seventeen families lost their homes. In saving a girl, who was asleep in a bed, a fireman was burned on the hands.

The copy of the selection used by the subject should be printed in heavy type and should not contain the bars dividing it into memories. The Stanford record booklet contains the selection in two forms, one suitable for use in scoring, the other in heavy type to be read by the subject.

PROCEDURE. Hand the selection to the subject, who should be seated comfortably in a good light, and say: "I want you to read this for me as nicely as you can." The subject must read aloud.

Pronounce all the words which the subject is unable to make out, not allowing more than five seconds' hesitation in such a case.

Record all errors made in reading the selection, and the exact time. By "error" is meant the omission, substitution, transposition, or mispronunciation of one word.

The subject is not warned in advance that he will be asked to report what he has read, but as soon as he has finished reading, put the selection out of sight and say: "Very well done. Now, I want you to tell me what you read. Begin at the first and tell everything you can remember." After the subject has repeated everything he can recall and has stopped, say: "And what else? Can you remember any more of it?" Give no other aid of any kind. It is of course not permissible, when the child stops, to prompt him with such questions as, "And what next? Where were the houses burned? What happened to the fireman?" etc. The report must be spontaneous.

Now and then, though not often, a subject hesitates or even refuses to try, saying he is unable to do it. Perhaps he has misunderstood the request and thinks he is expected to repeat the selection word for word, as in the tests of memory for sentences. We urge a little and repeat: "Tell me in your own words all you can remember of it." Others misunderstand in a different way, and thinking they are expected to tell merely what the story is about, they say: "It was about some houses that burned." In such cases we repeat the instructions with special emphasis on the words all you can remember.

SCORING. The test is passed if the selection is read in thirty-five seconds with not more than two errors, and if the report contains at least eight "memories." By underscoring the memories correctly reproduced, and by interlineations to show serious departures from the text, the record can be made complete with a minimum of trouble.

The main difficulty in scoring is to decide whether a memory has been reproduced correctly enough to be counted. Absolutely literal reproduction is not expected. The rule is to count all memories whose thought is reproduced with only minor changes in the wording. "It took quite a while" instead of "it took some time" is satisfactory; likewise, "got burnt" for "was burned"; "who was sleeping" for "who was asleep"; "are homeless" for "lost their homes"; "in the middle" for "near the center"; "a big fire" for "a fire," etc.

Memories as badly mutilated as the following, however, are not counted: "A lot of buildings" for "three houses;" "a man" for "a fireman"; "who was sick" for "who was asleep"; etc. Occasionally we may give half credit, as in the case of "was seventeen thousand dollars" for "was fifty thousand dollars"; "and fifteen families" for "and seventeen families," etc.

REMARKS. Are we warranted in using at all as a measure of intelligence a test which depends as much on instruction as this one does? Many are inclined to answer this question in the negative. The test has been omitted from the revisions of Goddard, Kuhlmann, and Binet himself. As regards Binet's earlier test of reading for two memories, in year VIII, there could hardly be any difference of opinion. The ability to read at that age depends so much on the accident of environment that the test is meaningless unless we know all about the conditions which have surrounded the child.

The use of the test in year X, however, is a very different matter. There are comparatively few children of that age who will fail to pass it for lack of the requisite school instruction. Children of 10 years who have attended school with reasonable regularity for three years are practically always able to read the selection in thirty-five seconds and without over two mistakes unless they are retarded almost to the border-line of mental deficiency. Of our 10-year-olds who failed to meet the test, only a fourth did so because of inability to meet the reading requirements as regards time or mistakes. The remaining failures were caused by inadequate report, and most of these subjects were of the distinctly retarded group.

We may conclude, therefore, that given anything approaching normal educational advantages, the test is really a measure of intelligence. Used with due caution, it is perhaps as valuable as any other test in the scale. It is only necessary, in case of failure, to ascertain the facts regarding the child's educational opportunities. Even this precaution is superfluous in case the subject tests as low as 8 years by the remainder of the scale. A safe rule is to omit the test from the calculation of mental age if the subject has not attended school the equivalent of two or three years.

It has been contended by some that tests in which success depends upon language mastery cannot be real tests of intelligence. By such critics language tests have been set over against intelligence tests as contrasting opposites. It is easy to show, however, that this view is superficial and psychologically unsound. Every one who has an acquaintance with the facts of mental growth knows that language mastery of some degree is the sine qua non of conceptual thinking. Language growth, in fact, mirrors the entire mental development. There are few more reliable indications of a subject's stage of intellectual maturity than his mastery of language.

The rate of reading, for example, is a measure of the rate of association. Letters become associated together in certain combinations making words, words into word groups and sentences. Recognition is for the most part an associative process. Rapid and accurate association will mean ready recognition of the printed form. Since language units (whether letters, words, or word groups) have more or less preferred associations according to their habitual arrangement into larger units, it comes about that in the normal mind under normal conditions these preferred sequences arouse the apperceptive complex necessary to make a running recognition rapid and easy. It is reasonable to suppose that in the subnormal mind the habitual common associations are less firmly fixed, thus diminishing the effectiveness of the ever-changing apperceptive expectancy. Reading is, therefore, largely dependent on what James calls the "fringe of consciousness" and the "consciousness of meaning." In reading connected matter, every unit is big with a mass of tendencies. The smaller and more isolated the unit, the greater is the number of possibilities. Every added unit acts as a modifier limiting the number of tendencies, until we have finally, in case of a large mental unit, a fairly manageable whole. When the most logical and suitable of these associations arise easily from subconsciousness to consciousness, recognition is made easy, and their doing so will depend on whether the habitual relations of the elements have left permanent traces in the mind.

The reading of the subnormal subject bears a close analogy to the reading of nonsense matter by the normal person. It has been ascertained by experiment that such reading requires about twice as much time as the reading of connected matter. This is true for the reason that out of thousands of associations possible with each word, no particular association is favored. The apperceptive expectancy, practically nil in the reading of nonsense material, must be decidedly deficient in all poor reading.

Furthermore, in the case of the ordinary reader there is a feeling of rightness or wrongness about the thought sequences. That less intelligent subjects have this sense of fitness to a much less degree is evidenced by their passing over words so mutilated in pronunciation as to deprive them of all meaning. The transposition of letters and words, and the failure to observe marks of punctuation, point to the same thing. In other words, all the reading of the stupid subject is with material which to him is more or less nonsensical.[66]

[66] See "Genius and Stupidity," by Lewis M. Terman, in Pedagogical Seminary, September, 1906, p. 340 ff.

A little observation will convince one that mentally retarded subjects, even when they possess a reasonable degree of fluency in recognizing printed words, do not sense shades of meaning. Their reading is by small units. Words and phrases do not fuse into one mental content, but remain relatively unconnected. The expression is monotonous and the voice has more of the unnatural "schoolroom" pitch. They read more slowly, more often misplace the emphasis, and miscall more words. In short, one who has psychological insight and is acquainted with reading standards can easily detect the symptoms of intellectual inferiority by hearing a dull subject read a brief selection.

The giving of memories is also significant. Feeble-minded adults who have been well schooled are sometimes able to read the words of the text fairly fluently, but are usually unable to give more than a scanty report of what has been read. The scope of attention has been exhausted in the mere recognition and pronouncing of words. In general, the greater the mechanical difficulties which a subject encounters, the less adequate is his report of memories.

The test has, however, one real fault. School children have a certain advantage in it over older persons of the same mental age whose school experience is less recent. Adult subjects tend to give their report in less literal form. It is necessary, therefore, to give credit for the reproduction of the ideas of the passage rather than for strictly literal "memories."

The selection we have used is, with minor changes, the same as Binet's. His selection was divided into nineteen memories. The one here given has twenty-one memories. Binet used the test both in year VIII and year IX, requiring two memories at year VIII and six memories at year IX. When we require eight memories, as we have done, the test becomes difficult enough for non-selected school children of 10 years. Location in year X seems preferable, because it insures that the child will almost certainly have had the schooling requisite for learning to read a selection of this difficulty, even if he has started to school at a later age than is customary. Naturally, placing the test higher in the scale makes it more a test of report and less a test of ability to recognize and pronounce printed words.

X, 5. COMPREHENSION, FOURTH DEGREE

The questions for this year are:—

(a) "What ought you to say when some one asks your opinion about a person you don't know very well?" (b) "What ought you to do before undertaking (beginning) something very important?" (c) "Why should we judge a person more by his actions than by his words?"

The PROCEDURE is the same as for the previous comprehension tests. Each question may be repeated, but its form must not be changed. It is not permissible to make any explanation whatever as to the meaning of the question, except to substitute beginning for undertaking when (b) seems not to be comprehended.

SCORING. Two out of the three questions must be answered satisfactorily. Study of the following classified responses should make scoring fairly easy in most cases:—

(a) When some one asks your opinion

Satisfactory. "I would say I don't know him very well" (42 per cent of the correct answers). "Tell him what I know and no more" (34 per cent of correct answers). "I would say that I'd rather not express any opinion about him" (20 per cent of the correct answers). "Tell him to ask some one else." "I would not express any opinion."

Unsatisfactory. Unsatisfactory responses are due either to failure to grasp the import of the question, or to inability to suggest the appropriate action demanded by the situation.

The latter form of failure is the more common; e.g.: "I'd say they are nice." "Say you like them." "Say what I think." "Say it's none of their business." "Tell them I mind my own business." "Say I would get acquainted with them." "Say that I don't talk about people." "Say I didn't know how he looked." "Tell them you ought not to say such things; you might get into trouble." "I wouldn't say anything." "I would try to answer." "Say I did not know his name," etc.

The following are samples of failure due to mistaking the import of the question: "I'd say, 'How do you do?'" "Say,'I'm glad to meet you.'"

(b) Before undertaking something important

Satisfactory responses fall into the following classes:— (1) Brief statement of preliminary consideration; as: "Think about it." "Look it over." "Plan it all out." "Make your plans." "Stop and think," etc. (2) Special emphasis on preliminary preparation and correct procedure; as: "Find out the best way to do it." "Find out what it is." "Get everything ready." "Do every little thing that would help you." "Get all the details you can." "Take your time and figure it out," etc. (3) Asking help; as: "Ask some one to help you who knows all about it." "Pray, if you are a Christian." "Ask advice," etc. (4) Preliminary testing of ability, self-analysis, etc.; as: "Try something easier first." "Practice and make sure I could do it." "Learn how to do it," etc. (5) Consider the wisdom or propriety of doing it: "Think whether it would be best to do it." "See whether it would be possible."

About 65 per cent of the correct responses belong either to group (1) or (2), about 20 per cent to group (3), and most of the remainder to group (4).

Unsatisfactory responses are of the following types:— (1) Due to mistaking the import of the question; e.g.: "Ask for it." "Ought to say please." "Ask whose it is." Replies of this kind can be nearly all eliminated by repeating the question, using beginning instead of undertaking. (2) Replies more or less absurd or irrelevant; as: "Promise to do your best." "Wash your face and hands." "Get a lot of insurance." "Dress up and take a walk." "Tell your name." "Know whether it's correct." "Begin at the beginning." "Say you will do it." "See if it's a fake." "Go to school a long time." "Pass an examination." "Do what is right." "Add up and see how much it will cost." "Say I would do it." "Just start doing it." "Go away." "Consult a doctor." "See if you have time," etc.

(c) Why we should judge a person more by his actions than by his words

Satisfactory responses fall into the following classes:— (1) Words and deeds both mentioned and contrasted in reliability; as: "Actions speak louder than words" (this in 8 per cent of successes). "You can tell more by his actions than by his words." "He might talk nice and do bad things." "Sometimes people say things and don't do them." "It's not what you say but what you do that counts." "Talk is cheap; when he does a thing you can believe it." "People don't do everything they say." "A man might steal but talk like a nice man." Over 45 per cent of all correct responses belong to group (1). (2) Acts stressed without mention of words; as: "You can tell by his actions whether he is good or not." "If he acts nice he is nice." "Actions show for themselves." Group (2) contains about 25 per cent of the correct responses. (3) Emphasis on unreliability of words; as: "You can't tell by his words, he might lie or boast." "Because you can't always believe what people say." (Group (3) contains 15 per cent of the correct responses.) (4) Responses which state that a man's deeds are sometimes better than his words; as: "He might talk ugly and still not do bad things." "Some really kind-hearted people scold and swear." "A man's words may be worse than his deeds," etc. Group (4) contains over 10 per cent of the correct responses.

Unsatisfactory responses are usually due to inability to comprehend the meaning of the question. If there is a complete lack of comprehension the result is either silence or a totally irrelevant response. If there is partial comprehension of the question the response may be partially relevant, but fail to make the expected distinction.

The following are sample failures: "You could tell by his words that he was educated." "It shows he is polite if he acts nice." "Sometimes people aren't polite." "Actions show who he might be." "Acts may be foolish." "Words ain't right." "A man might be dumb." "A fellow don't know what he says." "Some people can talk, but don't have control of themselves." "You can tell by his acts whether he goes with bad people." "If he doesn't act right you know he won't talk right." "Actions show if he has manners." "Might get embarrassed and not talk good." "He may not know how to express his thoughts." "He might be a rich man but a poor talker." "He might say the wrong thing and afterwards be sorry for it," etc. (The last four are nearer correct than the others, but they fall just short of expressing the essential contrast.)

REMARKS. For discussion of the comprehension questions as a test of intelligence, see page 158.

Binet used eight questions, three "easy" and five "difficult," and required that five out of eight be answered correctly in year X. The eight were as follows:—

(1) What to do when you have missed your train. (2) When you have been struck by a playmate, etc. (3) When you have broken something, etc. (4) When about to be late for school. (5) When about to undertake something important. (6) Why excuse a bad act committed in anger more readily than a bad act committed without anger. (7) What to do if some one asks your opinion, etc. (8) Why can you judge a person better by his actions, etc.

As we have shown, questions 1, 2, 3, and 4 are much too easy for year X. Question 6 is hard enough for year XII. We have omitted it because it was not needed and is not entirely satisfactory.

X, 6. NAMING SIXTY WORDS

PROCEDURE. Say: "Now, I want to see how many different words you can name in three minutes. When I say ready, you must begin and name the words as fast as you can, and I will count them. Do you understand? Be sure to do your very best, and remember that just any words will do, like 'clouds,' 'dog,' 'chair,' 'happy'—Ready; go ahead!"

The instructions may be repeated if the subject does not understand what is wanted. As a rule the task is comprehended instantly and entered into with great zest.

Do not stare at the child, and do not say anything as the test proceeds unless there is a pause of fifteen seconds. In this event say: "Go ahead, as fast as you can. Any words will do." Repeat this urging after every pause of fifteen seconds.

Some subjects, usually rather intelligent ones, hit upon the device of counting or putting words together in sentences. We then break in with: "Counting (or sentences, as the case may be) not allowed. You must name separate words. Go ahead."

Record the individual words if possible, and mark the end of each half-minute. If the words are named so rapidly that they cannot be taken down, it is easy to keep the count by making a pencil stroke for each word. If the latter method is employed, repeated words may be indicated by making a cross instead of a single stroke. Always make record of repetitions.

SCORING. The test is passed if sixty words, exclusive of repetitions, are named in three minutes. It is not allowable to accept twenty words in one minute or forty words in two minutes as an equivalent of the expected score. Only real words are counted.

REMARKS. Scoring, as we have seen, takes account only of the number of words. It is instructive, however, to note the kind of words given. Some subjects, more often those of the 8- or 9-year intelligence level, give mainly isolated, detached words. As well stated by Binet, "Little children exhaust an idea in naming it. They say, for example, hat, and then pass on to another word without noticing that hats differ in color, in form, have various parts, different uses and accessories, and that in enumerating all these they could find a large number of words."

Others quickly take advantage of such relationships and name many parts of an object before leaving it, or name a number of other objects belonging to the same class. Hat, for example, suggests cap, hood, coat, shirt, shoes, stockings, etc. Pencil suggests book, slate, paper, desk, ink, map, school-yard, teacher, etc. Responses of this type may be made up of ten or a dozen plainly distinct word groups.

Another type of response consists in naming only objects present, or words which present objects immediately suggest. It is unfortunate that this occurs, since rooms in which testing is done vary so much with respect to furnishings. The subject who chooses this method is obviously handicapped if the room is relatively bare. One way to avoid this influence is to have all subjects name the words with eyes closed, but the distraction thus caused is sometimes rather disturbing. It is perhaps best for the present to adhere to the original procedure, and to follow the rule of making tests in a room containing few furnishings in addition to the necessary table and chairs.

A fourth type of response is that including a large proportion of unusual or abstract words. This is the best of all, and is hardly ever found except with subjects who are above the 11-year intelligence level.

It goes without saying that a response need not belong entirely to any one of the above types. Most responses, in fact, are characterized by a mixture of two or three of the types, one of them perhaps being dominant.

Though not without its shortcomings, the test is interesting and valuable. Success in it does not, as one might suppose, depend solely upon the size of the vocabulary. Even 8-year-olds ordinarily know the meaning of more than 3000 words, and by 10 years the vocabulary usually exceeds 5000 words, or eighty times as many as the child is expected to name in three minutes. The main factors in success are two, (1) richness and variety of previously made associations with common words; and (2) the readiness of these associations to reinstate themselves. The young or the retarded subject fishes in the ocean of his vocabulary with a single hook, so to speak. He brings up each time only one word. The subject endowed with superior intelligence employs a net (the idea of a class, for example) and brings up a half-dozen words or more. The latter accomplishes a greater amount and with less effort; but it requires intelligence and will power to avoid wasting time with detached words.

One is again and again astonished at the poverty of associations which this test discloses with retarded subjects. For twenty or thirty seconds such children may be unable to think of a single word. It would be interesting if at such periods we could get a glimpse into the subject's consciousness. There must be some kind of mental content, but it seems too vague to be crystallized in words. The ready association of thoughts with definite words connotes a relatively high degree of intellectual advancement. Language forms are the short-hand of thought; without facile command of language, thinking is vague, clumsy, and ineffective. Conversely, vague mental content entails language shortage.

Occasionally a child of 11- or 12-year intelligence will make a poor showing in this test. When this happens it is usually due either to excessive embarrassment or to a strange persistence in running down all the words of a given class before launching out upon a new series. Occasionally, too, an intelligent subject wastes time in thinking up a beautiful list of big or unusual words. As stated by Bobertag, success is favored by a certain amount of "intellectual nonchalance," a willingness to ignore sense and a readiness to break away from a train of associations as soon as the "point of diminishing returns" has been reached. This doubtless explains why adults sometimes make such a surprisingly poor showing in the test. They have less "intellectual nonchalance" than children, are less willing to subordinate such considerations as completeness and logical connection to the demands of speed. Knollin's unemployed men of 12- to 13-year intelligence succeeded no better than school children of the 10-year level.

We do not believe, however, that this fault is serious enough to warrant the elimination of the test. The fact is that in a large majority of cases the score which it yields agrees fairly closely with the result of the scale as a whole. Subjects more than a year or two below the mental age of 10 years seldom succeed. Those more than a year or two above the 10-year level seldom fail.

There is another reason why the test should be retained, it often has significance beyond that which appears in the mere number of words given. The naming of unusual and abstract words is an instance of this. An unusually large number of repetitions has symptomatic significance in the other direction. It indicates a tendency to mental stereotypy, so frequently encountered in testing the feeble-minded. The proportion of repetitions made by normal children of the 10- or 11-year intelligence level rarely exceeds 2 or 3 per cent of the total number of words named; those of older retarded children of the same level occasionally reach 6 or 8 per cent.

It is conceivable, of course, that a more satisfactory test of this general nature could be devised; such, for example, as having the subject name all the words he can of a given class (four-footed animals, things to eat, articles of household furniture, trees, birds, etc.). The main objection to this form of the test is that the performance would in all probability be more influenced by environment and formal instruction than is the case with the test of naming sixty words.

One other matter remains to be mentioned; namely, the relative number of words named in the half-minute periods. As would be expected, the rate of naming words decreases as the test proceeds. In the case of the 10-year-olds, we find the average number of words for the six successive half-minutes to be as follows:—

18, 121/2, 101/2, 9, 81/2, 7.

Some subjects maintain an almost constant rate throughout the test, others rapidly exhaust themselves, while a very few make a bad beginning and improve as they go. As a rule it is only the very intelligent who improve after the first half-minute. On the other hand, mentally retarded subjects and very young normals exhaust themselves so quickly that only a few words are named in the last minute.

Binet first located this test in year XI, but shifted it to year XII in 1911. Goddard and Kuhlmann retain it in year XI, though Goddard's statistics suggest year X as the proper location, and Kuhlmann's even suggest year IX. Kuhlmann, however, accepts fifty words as satisfactory in case the response contains a considerable proportion of abstract or unusual words. All the American statistics except Rowe's agree in showing that the test is easy enough for year X.

X, ALTERNATIVE TEST 1: REPEATING SIX DIGITS

The digit series used are 3-7-4-8-5-9; and 5-2-1-7-4-6.

The PROCEDURE and SCORING are the same as in VII, 3, except that only two trials are given, one of which must be correct. The test is somewhat too easy for year 10 when three trials are given.

The test of repeating six digits did not appear in the Binet scale and seems not to have been standardized until inserted in the Stanford series.

X, ALTERNATIVE TEST 2: REPEATING TWENTY TO TWENTY-TWO SYLLABLES

The sentences for this year are:—

(a) "The apple tree makes a cool, pleasant shade on the ground where the children are playing." (b) "It is nearly half-past one o'clock; the house is very quiet and the cat has gone to sleep." (c) "In summer the days are very warm and fine; in winter it snows and I am cold."

PROCEDURE and SCORING exactly as in VI, 6.

REMARKS. It is interesting to note that five years of mental growth are required to pass from the ability to repeat sixteen or eighteen syllables (year VI) to the ability to repeat twenty or twenty-two syllables. Similarly in memory for digits. Five digits are almost as easy at year VII as six at year X. Two explanations are available: (1) The increased difficulty may be accounted for by a relatively slow growth of memory power after the age of 6 or 7 years; or (2) the increase in difficulty may be real, expressing an inner law as to the behavior of the memory span in dealing with material of increasing length. Both factors are probably involved.

This is another of the Stanford additions to the scale. Average children of 10 years ordinarily pass it, but older, retarded children of 10-year mental age make a poorer showing. In the case of mentally retarded adults, especially, the verbal memory is less exact than that of school children of the same mental age.

X, ALTERNATIVE TEST 3: CONSTRUCTION PUZZLE A (HEALY AND FERNALD)

MATERIAL. Use the form-board pictured on page 279. This may be purchased of C. H. Stoelting & Co., Chicago, Illinois. A home-made one will do as well if care is taken to get the dimensions exact. Quarter-inch wood should be used. The inside of the frame should be 3 x 4 inches, and the dimensions of the blocks should be as follows: 1+3/16 x 3; 1 x 11/2; 1 x 23/4; 1 x 11/2; 11/4 x 2.

PROCEDURE. Place the frame on the table before the subject, the short side nearest him. The blocks are placed in an irregular position on the side of the frame away from the subject. Take care that the board with the blocks in place is not exposed to view in advance of the experiment.

Say: "I want you to put these blocks in this frame so that all the space will be filled up. If you do it rightly they will all fit in and there will be no space left over. Go ahead."

Do not tell the subject to see how quickly he can do it. Say nothing that would even suggest hurrying, for this tends to call forth the trial-and-error procedure even with intelligent subjects.



SCORING. The test is passed if the child succeeds in fitting the blocks into place three times in a total time of five minutes for the three trials.

The method of procedure is fully as important as the time, but is not so easily scored in quantitative terms. Nevertheless, the examiner should always take observations on the method employed, noting especially any tendency to make and to repeat moves which lead to obvious impossibilities; i.e., moves which leave a space obviously unfitted to any of the remaining pieces. Some subjects repeat an absurd move many times over; others make an absurd move, but promptly correct it; others, and these are usually the bright ones, look far enough ahead to avoid error altogether.

REMARKS. This test was devised by Professor Freeman, was adapted slightly by Healy and Fernald, and was first standardized by Dr. Kuhlmann. Miss Gertrude Hall has also standardized it, but on a different procedure from that described above.[67]

[67] Eugenics and Social Welfare Bulletin, No. 5, The State Board of Charities, Albany, New York.

The test has a lower correlation with intelligence than most of the other tests of the scale. Many bright children of 10-year intelligence adopt the trial-and-error method and have little success, while retarded older children of only 8-year intelligence sometimes succeed. Age, apart from intelligence, seems to play an important part in determining the nature of the performance. A favorable feature of the test, however, is the fact that it makes no demand on language ability and that it brings into play an aspect of intelligence which is relatively neglected by the remainder of the scale. For this reason it is at least worth keeping as an alternative test.



CHAPTER XVII

INSTRUCTIONS FOR YEAR XII

XII, 1. VOCABULARY (FORTY DEFINITIONS, 7200 WORDS)

PROCEDURE and SCORING as in previous vocabulary tests.[68] In this case forty words must be defined.

[68] See VIII, 6.

XII, 2. DEFINING ABSTRACT WORDS

PROCEDURE. The words to be defined are pity, revenge, charity, envy, and justice. The formula is, "What is pity? What do we mean by pity?" and so on with the other words. If the meaning of the response is not clear, ask the subject to explain what he means. If the definition is in terms of the word itself, as "Pity means to pity someone," "Revenge is to take revenge," etc., it is then necessary to say: "Yes, but what does it mean to pity some one?" or, "What does it mean to take revenge?" etc. Only supplementary questions of this kind are permissible.

SCORING. The test is passed if three of the five words are satisfactorily defined. The definition need not be strictly logical nor the language elegant. It is sufficient if the definition shows that the meaning of the word is known. Definitions which define by means of an illustration are acceptable. The following are samples of satisfactory and unsatisfactory responses:—

(a) Pity

Satisfactory. "To be sorry for some one." "To feel compassion." "To have sympathy for a person." "To feel bad for some one." "It means you help a person out and don't like to have him suffer." "To have a feeling for people when they are treated wrong." "If anybody gets hurt real bad you pity them." "It's when you feel sorry for a tramp and give him something to eat." "If some one is in trouble and you know how it feels to be in that condition, you pity him." "You see something that's wrong and have your feeling aroused."

Of 130 correct responses, 85, or 65 per cent, defined pity as "to feel sorry for some one," or words to that effect. Less than 10 per cent defined by means of illustration.

Unsatisfactory. "To think of the poor." "To be good to others." "To help." "It means sorrow." "Mercy." "To cheer people up." "It means 'What a pity!'" "To be ashamed." "To be sick or poor." "It's when you break something."

Apart from inability to reply, which accounts for nearly one fourth of the failures, there is no predominant type of unsatisfactory response.

(b) Revenge

Satisfactory. "To get even with some one." "To get back on him." "To do something to the one who has done something to you." "To hurt them back." "To pay it back," or "Do something back." "To do something mean in return." "To square up with a person." "When somebody slaps you, you slap back." "You kill a person if he does something to you."

The expression "to get even" was found in 42 per cent of 120 correct answers; "to pay it back," or "To do something back," in 20 per cent; "To get back on him," in 17 per cent. About 8 per cent were illustrations.

Unsatisfactory. "To be mad." "You try to hurt them." "To fight." "You hate a person." "To kill them." "It means hateful." "To try again." "To think evil of some one." "To hate some one who has done you wrong." "To let a person off." "To go away from something."

Inability to reply accounts for a little over 40 per cent of the failures.

(c) Charity

Satisfactory. "To give to the poor." "To help those who are needy." "It is charity if you are poor and somebody helps you." "To give to somebody without pay."

Of 110 correct replies, 72 per cent were worded substantially like the first or second given above.

Unsatisfactory. "A person who helps the poor." "A place where poor people get food and things." "It is a good life." "To be happy." "To be poor." "Charity is being treated good." "It is to be charitable." "Charity is selling something that is not worth much." "It means to be good" or "to be kind."

When the last named response is given, we should say: "Explain what you mean." If this brings an amplification of the response to "It means to do things for the poor," or the equivalent, the score is plus. "Charity means love" is also minus if the statement cannot be further explained and is merely rote memory of the passage in the 13th chapter of 1st Corinthians. Simply "To help" or "To give" is unsatisfactory. Half of the failures are due to inability to reply.

(d) Envy

Satisfactory. "You envy some one who has something you want." "It's the way you feel when you see some one with something nicer than you have." "It's when a poor girl sees a rich girl with nice dresses and things." "You hate some one because they've got something you want." "Jealousy" (satisfactory if subject can explain what jealousy means; otherwise it is minus). "It's when you see a person better off than you are."

Nearly three fourths of the correct responses say in substance, "You envy a person who has something you want." Most of the others are concrete illustrations.

Unsatisfactory. "To hate some one," or simply "To hate." "You don't like 'em." "Bad feeling toward any one." "To be a great man or woman." "Not to be nice to people." "What we do to our enemies."

Inability to respond accounts for 55 per cent of the failures.

(e) Justice

Satisfactory. "To give people what they deserve." "It means that everybody is treated the same way, whether he is rich or poor." "It's what you get when you go to court." "If one does something and gets punished, that's justice." "To do the square thing." "To give everybody his dues." "Let every one have what's coming to him." "To do the right thing by any one." "If two people do the same thing and they let one go without punishing, that is not justice."

Approximately 38 per cent of 102 correct responses referred to treating everybody the same way; 25 per cent to "doing the square thing", 12 per cent were concrete illustrations; and 4 per cent were definitions of what justice is not.

Unsatisfactory. "It means to have peace." "It is where they have court." "It's the Courthouse." "To be honest." "Where one is just" (minus, unless further explained). "To do right" (minus, unless in explaining right the subject gives a definition of justice).

It is very necessary, in case of such answers as "Justice is to do right," "To be just," etc., that the subject be urged to explain further what he means. "To do right" includes nearly 12 per cent of all answers, and is given by the very brightest children. Most of these are able, when urged, to complete the definition in a satisfactory manner.

REMARKS. The reader may be surprised that the ability to define common abstract words should develop so late. Most children who have had anything like ordinary home or school environment have doubtless heard all of these words countless times before the age of 12 years. Nevertheless, the statistics from the test show unmistakably that before this age such words have but limited and vague meaning. Other vocabulary studies confirm this fact so completely that we may say there is hardly any trait in which 12- to 14-year intelligence more uniformly excels that of the 9- or 10-year level.

This is readily understandable when we consider the nature of abstract meanings and the intellectual processes by which we arrive at them. Unlike such words as tree, house, etc., the ideas they contain are not the immediate result of perceptual processes, in which even childish intelligence is adept, but are a refined and secondary product of relationships between other ideas. They require the logical processes of comparison, abstraction, and generalization. One cannot see justice, for example, but one is often confronted with situations in which justice or injustice is an element; and given a certain degree of abstraction and generalization, out of such situations the idea of justice will gradually be evolved.

The formation and use of abstract ideas, of one kind or another, represent, par excellence, the "higher thought processes." It is not without significance that delinquents who test near the border-line of mental deficiency show such inferior ability in arriving at correct generalizations regarding matters of social and moral relationships. We cannot expect a mind of defective generalizing ability to form very definite or correct notions about justice, law, fairness, ownership rights, etc.; and if the ideas themselves are not fairly clear, the rules of conduct based upon them cannot make a very powerful appeal.[69]

[69] See also p. 298 ff.

Binet used the words charity, justice, and kindness, and required two successes. In the 1911 revision he shifted the test from year XI to year XII, where it more nearly belongs. Goddard also places it in year XII and uses Binet's words, translating bonte, however, as goodness instead of kindness. Kuhlmann retains the test in year XI and adds bravery and revenge, requiring three correct definitions out of five. Bobertag uses pity, envy, and justice, requires two correct definitions, and finds the test just hard enough for year XII.

After using the words goodness and kindness in two series of tests, we have discarded them as objectionable in that they give rise to so many doubtful definitions. Even intelligent children often say: "Goodness means to do something good," "Kindness means to be kind to some one," etc. These definitions in a circle occur less than half as often with pity, revenge, and envy, which are also superior to charity and justice in this respect.

The relative difficulty of our five words is indicated by the order in which we have listed them in the test (i.e., beginning with the easiest and ending with the hardest). On the standard of three correct definitions, these words fit very accurately in year XII.

XII, 3. THE BALL-AND-FIELD TEST (SUPERIOR PLAN)

PROCEDURE, as in year VIII, test 1.

SCORING. Score 3 (or superior plan) is required for passing in year XII.[70]

[70] See scoring card.

XII, 4. DISSECTED SENTENCES

The following disarranged sentences are used:—

FOR THE STARTED AN WE COUNTRY EARLY AT HOUR

TO ASKED PAPER MY TEACHER CORRECT I MY

A DEFENDS DOG GOOD HIS BRAVELY MASTER

These should be printed in type like that used above. The Stanford record booklet contains the sentences in convenient form.

It is not permissible to substitute written words or printed script, as that would make the test harder. All the words should be printed in caps in order that no clue shall be given as to the first word in a sentence. For a similar reason the period is omitted.

PROCEDURE. Say: "Here is a sentence that has the words all mixed up so that they don't make any sense. If the words were changed around in the right order they would make a good sentence. Look carefully and see if you can tell me how the sentence ought to read."

Give the sentences in the order in which they are listed in the record booklet. Do not tell the subject to see how quickly he can do it, because with this test any suggestion of hurrying is likely to produce a kind of mental paralysis. If the subject has no success with the first sentence in one minute, read it off correctly for him, somewhat slowly, and pointing to each word as it is spoken. Then proceed to the second and third, allowing one minute for each.

Give no further help. It is not permissible, in case an incorrect response is given, to ask the subject to try again, or to say: "Are you sure that is right?" "Are you sure you have not left out any words?" etc. Instead, maintain absolute silence. However, the subject is permitted to make as many changes in his response as he sees fit, provided he makes them spontaneously and within the allotted time. Record the entire response.

Once in a great while the subject misunderstands the task and thinks the only requirement is to use all the words given, and that it is permitted to add as many other words as he likes. It is then necessary to repeat the instructions and to allow a new trial.

SCORING. Two sentences out of three must be correctly given within the minute allotted to each. It is understood, of course, that if the first sentence has to be read for the subject, both the other responses must be given correctly.

A sentence is not counted correct if a single word is omitted, altered, or inserted, or if the order given fails to make perfect sense.

Certain responses are not absolutely incorrect, but are objectionable as regards sentence structure, or else fail to give the exact meaning intended. These are given half credit. Full credit on one, and half credit on each of the other two, is satisfactory. The following are samples of satisfactory and unsatisfactory responses:—

(a) Satisfactory. "We started for the country at an early hour." "At an early hour we started for the country." "We started at an early hour for the country." Unsatisfactory. "We started early at an hour for the country." "Early at an hour we started for the country." "We started early for the country." Half credit. "For the country at an early hour we started." "For the country we started at an early hour."

(b) Satisfactory. "I asked my teacher to correct my paper." Unsatisfactory. "My teacher asked to correct my paper." "To correct my paper I asked my teacher." Half credit. "My teacher I asked to correct my paper."

(c) Satisfactory. "A good dog defends his master bravely." "A good dog bravely defends his master." Unsatisfactory. "A dog defends his master bravely." "A bravely dog defends his master." "A good dog defends his bravely master." "A good brave dog defends his master." Half credit. "A dog defends his good master bravely." "A dog bravely defends his good master." "A good master bravely defends his dog."

REMARKS. This is an excellent test. It involves no knowledge which may not be presupposed at the age in which it is given, and success therefore depends very little on experience. The worst that can be urged against it is that it may possibly be influenced to a certain extent by the amount of reading the subject has done. But this has not been demonstrated. At any rate, the test satisfies the most important requirement of a test of intelligence; namely, the percentage of successes increases rapidly and steadily from the lower to the higher levels of mental age.

This experiment can be regarded as a variation of the completion test. Binet tells us, in fact, that it was directly suggested by the experiment of Ebbinghaus. As will readily be observed, however, it differs to a certain extent from the Ebbinghaus completion test. Ebbinghaus omits parts of a sentence and requires the subject to supply the omissions. In this test we give all the parts and require the formation of a sentence by rearrangement. The two experiments are psychologically similar in that they require the subject to relate given fragments into a meaningful whole. Success depends upon the ability of intelligence to utilize hints, or clues, and this in turn depends on the logical integrity of the associative processes. All but the highest grade of the feeble-minded fail with this test.

This test is found in year XI of Binet's 1908 series and in year XII of his 1911 revision. Goddard and Kuhlmann retain it in the original location. That it is better placed in year XII is indicated by all the available statistics with normal children, except those of Goddard. With this exception, the results of various investigators for year XII are in remarkably close agreement, as the following figures will show:—

Per cent passing at year XII

Binet 66 Kuhlmann 68 Bobertag 78 Dougherty 64 Strong 72 Leviste and Morle 70 Stanford series (1911) 62 Stanford series (1913) 57 Stanford series (1914) 62 Princeton data 61

This agreement is noteworthy considering that no two experiments seem to have used exactly the same arrangement of words, and that some have presented the words of a sentence in a single line, others in two or three lines. A single line would appear to be somewhat easier.

XII, 5. INTERPRETATION OF FABLES (SCORE 4)

The following fables are used:—

(a) Hercules and the Wagoner

A man was driving along a country road, when the wheels suddenly sank in a deep rut. The man did nothing but look at the wagon and call loudly to Hercules to come and help him. Hercules came up, looked at the man, and said: "Put your shoulder to the wheel, my man, and whip up your oxen." Then he went away and left the driver.

(b) The Milkmaid and her Plans

A milkmaid was carrying her pail of milk on her head, and was thinking to herself thus: "The money for this milk will buy 4 hens; the hens will lay at least 100 eggs; the eggs will produce at least 75 chicks; and with the money which the chicks will bring I can buy a new dress to wear instead of the ragged one I have on." At this moment she looked down at herself, trying to think how she would look in her new dress; but as she did so the pail of milk slipped from her head and dashed upon the ground. Thus all her imaginary schemes perished in a moment.

(c) The Fox and the Crow

A crow, having stolen a bit of meat, perched in a tree and held it in her beak. A fox, seeing her, wished to secure the meat, and spoke to the crow thus: "How handsome you are! and I have heard that the beauty of your voice is equal to that of your form and feathers. Will you not sing for me, so that I may judge whether this is true?" The crow was so pleased that she opened her mouth to sing and dropped the meat, which the fox immediately ate.

(d) The Farmer and the Stork

A farmer set some traps to catch cranes which had been eating his seed. With them he caught a stork. The stork, which had not really been stealing, begged the farmer to spare his life, saying that he was a bird of excellent character, that he was not at all like the cranes, and that the farmer should have pity on him. But the farmer said: "I have caught you with these robbers, and you will have to die with them."

(e) The Miller, His Son, and the Donkey

A miller and his son were driving their donkey to a neighboring town to sell him. They had not gone far when a child saw them and cried out: "What fools those fellows are to be trudging along on foot when one of them might be riding." The old man, hearing this, made his son get on the donkey, while he himself walked. Soon, they came upon some men. "Look," said one of them, "see that lazy boy riding while his old father has to walk." On hearing this, the miller made his son get off, and he climbed on the donkey himself. Farther on they met a company of women, who shouted out: "Why, you lazy old fellow, to ride along so comfortably while your poor boy there can hardly keep pace by the side of you!" And so the good-natured miller took his boy up behind him and both of them rode. As they came to the town a citizen said to them, "Why, you cruel fellows! You two are better able to carry the poor little donkey than he is to carry you." "Very well," said the miller, "we will try." So both of them jumped to the ground, got some ropes, tied the donkey's legs to a pole and tried to carry him. But as they crossed the bridge the donkey became frightened, kicked loose and fell into the stream.

PROCEDURE. Present the fables in the order in which they are given above. The method is to say to the subject:

"You know what a fable is? You have heard fables?" Whatever the answer, proceed to explain a fable as follows: "A fable, you know, is a little story, and is meant to teach us a lesson. Now, I am going to read a fable to you. Listen carefully, and when I am through I will ask you to tell me what lesson the fable teaches us. Ready; listen." After reading the fable, say: "What lesson does that teach us?" Record the response verbatim and proceed with the next as follows: "Here is another. Listen again and tell me what lesson this fable teaches us," etc.

As far as possible, avoid comment or commendation until all the fables have been given. If the first answer is of an inferior type and we express too much satisfaction with it, we thereby encourage the subject to continue in his error. On the other hand, never express dissatisfaction with a response, however absurd or malapropos it may be. Many subjects are anxious to know how well they are doing and continually ask, "Did I get that one right?" It is sufficient to say, "You are getting along nicely," or something to that effect. Offer no comments, suggestions, or questions which might put the subject on the right track. This much self-control is necessary if we would make the conditions of the test uniform for all subjects.

The only occasion when a supplementary question is permissible is in case of a response whose meaning is not clear. Even then we must be cautious and restrict ourselves to some such question as, "What do you mean?" or, "Explain; I don't quite understand what you mean." The scoring of fables is somewhat difficult at best, and this additional question is often sufficient to place the response very definitely in the right or wrong column.

SCORING. Give score 2, i.e., 2 points, for a correct answer, and 1 for an answer which deserves half credit. The test is passed in year XII if 4 points are earned; that is, if two responses are correct or if one is correct and two deserve half credit.

Score 2 means that the fable has been correctly interpreted and that the lesson it teaches has been stated in general terms.

There are two types of response which may be given half credit. They include (1) the interpretations which are stated in general terms and are fairly plausible, but are not exactly correct; and (2) those which are perfectly correct as to substance, but are not generalized.

We overlook ordinary faults of expression and regard merely the essential meaning of the response.

The only way to explain the method is by giving copious illustrations. If the following sample responses are carefully studied, a reasonable degree of expertness in scoring fables may be acquired with only a limited amount of actual practice. The sampling may appear to the reader needlessly prolix, but experience has taught us that in giving directions for the scoring of tests error always lies on the side of taking too much for granted.

(a) Hercules and the Wagoner

Full credit; score 2. "God helps those who help themselves." "Do not depend on others." "Help yourself before calling for help." "It teaches that we should rely upon ourselves."

The following are not quite so good, but are nevertheless considered satisfactory. "We should always try, even if it looks hard and we think we can't do it." "When in trouble try to get out of it yourself." "We've got to do things without help." "Not to be lazy."

Half credit; score 1. This is most often given for the response which contains the correct idea, but states it in terms of the concrete situation, e.g.: "The man ought to have tried himself first." "Hercules wanted to teach the man to help himself." "The driver was too much inclined to depend on others." "The man was too lazy. He should not have called for help until he had tried to get out by himself." "To get out and try instead of watching."

Unsatisfactory; score 0. Failures are mainly of five varieties: (1) generalized interpretations which entirely miss the point; (2) crude interpretations which not only miss the point, but are also stated in terms of the concrete situation; (3) irrelevant or incoherent remarks; (4) efforts to repeat the story; and (5) inability to respond.

Sample failures of type (1), entirely incorrect generalizations: "Teaches us to look where we are going." "Not to ask for anything when there is no one to help." "To help those who are in trouble." "Teaches us to be polite." "How to help others." "Not to be cruel to horses." "Always to do what people tell you" (or "obey orders," etc.). "Not to be foolish" (or stupid, etc.). "If you would have a thing well done, do it yourself."

Failures of type (2), crude interpretations stated in concrete terms: "How to get out of the mud." "Not to get stuck in the mud." "To carry a stick along to pry yourself out if you get into a mud-hole." "To help any one who is stuck in the mud." "Taught Hercules to help the horses along and not whip them too hard." "Not to be mean like Hercules."

Failures of type (3), irrelevant responses: "It was foolish not to thank him." "He should have helped the driver." "Hercules was mean." "If any one helps himself the horses will try." "The driver should have done what Hercules told him." "He wanted the man to help the oxen."

Type (4): Efforts to repeat the story.

Type (5): Inability to respond.

(b) The Maid and the Eggs

Full credit; score 2. "Teaches us not to build air-castles." "Don't count your chickens before they are hatched." "Not to plan too far ahead." Slightly inferior, but still acceptable: "Never make too many plans." "Don't count on the second thing till you have done the first."

Half credit; score 1. "It teaches us not to have our minds on the future when we carry milk on the head." "She was building air-castles and so lost her milk." "She was planning too far ahead."

The responses just given are examples of fairly correct interpretations in non-generalized terms. The following are examples of generalized interpretations which fall below the accuracy required for full credit: "Never make plans." "Not to be too proud." "To keep our mind on what we are doing." "Don't cross a bridge till you come to it." "Don't count your eggs before they are hatched." "Not to be wanting things; learn to wait." "Not to imagine; go ahead and do it."

Unsatisfactory; score 0. Type (1), entirely incorrect generalization: "That money does not buy everything." "Not to be greedy." "Not to be selfish." "Not to waste things." "Not to take risks like that." "Not to think about clothes." "Count your chickens before they are hatched."

Type (2), very crude interpretations stated in concrete terms: "Not to carry milk on the head." "Teaches her to watch and not throw down her head." "To carry her head straight." "Not to spill milk." "To keep your chickens and you will make more money."

Type (3), irrelevant responses: "She wanted the money." "Teaches us to read and write" (18-year-old of 8-year intelligence). "About a girl who was selling some milk."

Type (4), effort to repeat the story.

Type (5), inability to respond.

(c) The Fox and the Crow

Full credit; score 2. "Teaches us not to listen to flattery." "Don't let yourself be flattered." "It is not safe to believe people who flatter us." "We had better look out for people who brag on us."

Half credit; score 1. Correct idea in concrete terms: "The crow was so proud of herself that she lost all she had." "The crow listened to flattery and got left." "Not to be proud and let people think you can sing when you can't." "If anybody brags on you don't sing or do what he tells you."

Pertinent but somewhat inferior generalizations: "Not to be too proud." "Pride goes before a fall." "To be on our guard against people who are our enemies." "Not to do everything people tell you." "Don't trust every slick fellow you meet."

Unsatisfactory; score 0. Type (1), incorrect generalization: "Not to go with people you don't know." "Not to be selfish." "To share your food." "Look before you leap." "Not to listen to evil." "Not to steal." "Teaches honesty." "Not to covet." "Think for yourself." "Teaches wisdom." "Never listen to advice." "Never let any one get ahead of you." "To figure out what they are going to do." "Never try to do two things at once." "How to get what you want."

Type (2), very crude interpretation stated in terms of the concrete situation: "Not to sing before you eat." "Not to hold a thing in your mouth; eat it." "To eat a thing before you think of your beauty." "To swallow it before you sing." "To be on your watch when you have food in your mouth."

Type (3), irrelevant responses: "The fox was greedy." "The fox was slicker than what the crow was." "The crow ought not to have opened her mouth." "The crow should just have shaken her head." "It served the crow right for stealing the meat." "The fox wanted the meat and just told the crow that to get it." "Foolishness." "Guess that's where the old fox got his name—'Old Foxy'—Don't teach us anything."

Type (4), efforts to repeat the story.

Type (5), inability to respond.

(d) The Farmer and the Stork

Full credit; score 2. "You are judged by the company you keep." "Teaches us to keep out of bad company." "Birds of a feather flock together." "If you go with bad people you are counted like them." "We should choose our friends carefully." "Don't go with bad people." "Teaches us to avoid the appearance of evil."

Half credit; score 1. "The stork should not have been with the cranes." "Teaches him not to go with robbers." "Don't go with people who are not of your nation." "Not to follow others."

Unsatisfactory; score 0. Type (1), incorrect generalization: "Not to steal." "Not to tell lies." "Not to give excuses." "A poor excuse is better than none." "Not to trust what people say." "Not to listen to excuses." "Not to harm animals that do no harm." "To have pity on others." "Not to be cruel." "To be kind to birds." "Not to blame people for what they don't do." "Teaches that those who do good often suffer for those who do evil." "To tend to your own business." "Not to meddle with other people's things." "Not to trespass on people's property." "Not to think you are so nice." "To keep out of mischief."

Type (2), very crude interpretations in concrete terms: "Taught the stork to look where it stepped and not walk into a trap." "Taught the stork to keep out of the man's field." "Not to take the seeds."

Type (3), irrelevant responses: "The farmer was right; storks do eat grain." "Served the stork right, he was stealing too." "He should try to help the stork out of the field."

Type (4), efforts to repeat the story.

Type (5), inability to reply.

(e) The Miller, His Son, and the Donkey

Full credit; score 2. "When you try to please everybody you please nobody." "Don't listen to everybody; you can't please them all." "Don't take every one's advice." "Don't try to do what everybody tells you." "Use your own judgment." "Have a mind of your own." "Make up your mind and stick to it." "Don't be wishy-washy." "Have confidence in your own opinions."

Half credit; score 1. Interpretations which are generalized but somewhat inferior: "Never take any one's advice" (too sweeping a conclusion). "Don't take foolish advice." "Take your own advice." "It teaches us that people don't always agree."

Correct idea but not generalized: "They were fools to listen to everybody." "They should have walked or rode just as they thought best, without listening to other people."

Unsatisfactory; score 0. Type (1), incorrect generalization: "To do right." "To do what people tell you." "To be kind to old people." "To be polite." "To serve others." "Not to be cruel to animals." "To have sympathy for beasts of burden." "To be good-natured." "Not to load things on animals that are small." "That it is always better to leave things as they are." "That men were not made for beasts of burden."

Type (2), very crude interpretations stated in concrete terms: "Not to try to carry the donkey." "That walking is better than riding." "The people should have been more polite to the old man." "That the father should be allowed to ride."

Type (3), irrelevant responses: "The men were too heavy for the donkey." "They ought to have stayed on and they would not have fallen into the stream." "It teaches about a man and he lost his donkey."

Type (4), efforts to repeat the story.

Type (5), inability to respond.

REMARKS. The fable test, or the "test of generalization," as it may aptly be named, was used by the writer in a study of the intellectual processes of bright and dull boys in 1905,[71] and was further standardized by the writer and Mr. Childs in 1911.[72] It has proved its worth in a number of investigations. It has been necessary, however, to simplify the rather elaborate method of scoring which was proposed in 1911, not because of any logical fault of the method, but because of the difficulty in teaching examiners to use the system correctly. The method explained above is somewhat coarser, but it has the advantage of being much easier to learn.

[71] "Genius and Stupidity," in Pedagogical Seminary, vol. xiii, pp. 307-73.

[72] "A Tentative Revision and Extension of the Binet-Simon Measuring Scale of Intelligence," Journal of Educational Psychology (1912).

The generalization test presents for interpretation situations which are closely paralleled in the everyday social experience of human beings. It tests the subject's ability to understand motives underlying acts or attitudes. It gives a clue to the status of the social consciousness. This is highly important in the diagnosis of the upper range of mental defectiveness. The criterion of the subnormal's fitness for life outside an institution is his ability to understand social relations and to adjust himself to them. Failure of a subnormal to meet this criterion may lead him to break common conventions, and to appear disrespectful, sulky, stubborn, or in some other way queer and exceptional. He is likely to be misunderstood, because he so easily misunderstands others. The skein of human motives is too complex for his limited intelligence to untangle.

Ethnological studies have shown in an interesting way the social origin of the moral judgment. The rectitude of the moral life, therefore, depends on the accuracy of the social judgment. It would be interesting to know what proportion of offenders have transgressed moral codes because of continued failure to grasp the essential lessons presented by human situations.

For the intelligent child even the common incidents of life carry an endless succession of lessons in right conduct. On the average school playground not an hour passes without some happening which is fraught with a moral hint to those who have intelligence enough to generalize the situation. A boy plays unfairly and is barred from the game. One bullies his weaker companion and arouses the anger and scorn of all his fellows. Another vents his braggadocio and feels at once the withering scorn of those who listen. Laziness, selfishness, meanness, dishonesty, ingratitude, inconstancy, inordinate pride, and the countless other faults all have their social penalties. The child of normal intelligence sees the point, draws the appropriate lesson and (provided emotions and will are also normal) applies it more or less effectively as a guide to his own conduct. To the feeble-minded child, all but lacking in the power of abstraction and generalization, the situation conveys no such lesson. It is but a muddle of concrete events without general significance; or even if its meaning is vaguely apprehended, the powers of inhibition are insufficient to guarantee that right action will follow.

It is for this reason that the generalization test is so valuable in the mental examinations of delinquents. It presents a moral situation, imagined, to be sure, but none the less real to the individual of normal comprehension. It tells us quickly whether the subject tested is able to see beyond the incidents of the given situation and to grasp their wider relations—whether he is able to generalize the concrete.

The following responses made by feeble-minded delinquents from 16 to 21 years of age demonstrate sufficiently their inability to comprehend the moral situation:—

Hercules and the Wagoner. "Teaches you to look where you are going." "Not to help any one who is stuck in the mud." "Not to whip oxen." "Teaches that Hercules was mean." "Teaches us to carry a stick along to pry the wheels out."

The Fox and the Crow. "Not to sing when eating." "To keep away from strangers." "To swallow it before you sing." "Not to be stingy." "Not to listen to evil." "The fox was wiser than the crow." "Not to be selfish with food." "Not to do two things at once." "To hang on to what you've got."

The Farmer and the Stork. "Teaches the stork to look where he steps." "Not to be cruel like the farmer." "Not to tell lies." "Not to butt into other people's things." "To be kind to birds." "Teaches us how to get rid of troublesome people." "Never go with anything else."

The following are the responses of an 18-year-old delinquent (intelligence level 10 years) to the five fables:—

Maid and Eggs. "She was thinking about getting the dress and spilled the milk. Teaches selfishness."

Hercules and the Wagoner. "He wanted to help the oxen out."

Fox and Crow. "Guess that's where the fox got his name—'Old Foxy.' Don't teach us anything."

Farmer and Stork. "Try and help the stork out of the field."

Miller, Son, and Donkey. "They was all big fools and mean to the donkey."

One does not require very profound psychological insight to see that a person of this degree of comprehension is not promising material for moral education. His weakness in the ability to generalize a moral situation is not due to lack of instruction, but is inherent in the nature of his mental processes, all of which have the infantile quality of average 9- or 10-year intelligence. Well-instructed normal children of 10 years ordinarily succeed no better. The ability to draw the correct lesson from a social situation is little developed below the mental level of 12 or 13 years.

The test is also valuable because it throws light on the subject's ability to appreciate the finer shades of meaning. The mentally retarded often show marked inferiority in this respect. They sense, perhaps, in a general way the trend of the story, but they fail to comprehend much that to us seems clearly expressed. They do not get what is left for the reader to infer, because they are insensible to the thought fringes. It is these which give meaning to the fable. The dull subject may be able to image the objects and activities described, but taken in the rough such imagery gets him nowhere.

Finally, the test is almost free from the danger of coaching. The subject who has been given a number of fables along with twenty-five or thirty other tests can as a rule give only hazy and inaccurate testimony as to what he has been put through. Moreover, we have found that, even if a subject has previously heard a fable, that fact does not materially increase his chances of giving a correct interpretation. If the situation depicted in the fable is beyond the subject's power of comprehension even explicit instruction has little effect upon the quality of the response.

Incidentally, this observation raises the question whether the use of proverbs, mottoes, fables, poetry, etc., in the moral instruction of children may not often be futile because the material is not fitted to the child's power of comprehension. Much of the school's instruction in history and literature has a moral purpose, but there is reason to suspect that in this field schools often make precocious attempts in "generalizing" exercises.

XII, 6. REPEATING FIVE DIGITS REVERSED

The series are 3-1-8-7-9; 6-9-4-8-2; 5-2-9-6-1.

PROCEDURE and SCORING. Exactly as in years VII and IX.[73]

[73] See discussion, p. 207 ff.

XII, 7. INTERPRETATION OF PICTURES

PROCEDURE. Use the same pictures as in III, 1, and VII, 2, and the additional picture d. Present in the same order. The formula to begin with is identical with that in VII, 2: "Tell me what this picture is about. What is this a picture of?" This formula is chosen because it does not suggest specifically either description or interpretation, and is therefore adapted to show the child's spontaneous or natural mode of apperception. However, in case, this formula fails to bring spontaneous interpretation for three of the four pictures, we then return to those pictures on which the subject has failed and give a second trial with the formula: "Explain this picture." A good many subjects who failed to interpret the pictures spontaneously do so without difficulty when the more specific formula is used.

If the response is so brief as to be difficult to classify, the subject should be urged to amplify by some such injunction as "Go ahead," or "Explain what you mean."

One more caution. It is necessary to refrain from voicing a single word of commendation or approval until all the pictures have been responded to. A moment's thought will reveal the absolute necessity of adhering to this rule. Often a subject will begin by giving an inferior type of response (description, say) to the first picture, but with the second picture adjusts better to the task and responds satisfactorily. If in such a case the first (unsatisfactory) response were greeted with an approving "That's fine, you are doing splendidly," the likelihood of any improvement taking place as the test proceeds would be greatly lessened.

SCORING. Three pictures out of four must be satisfactorily interpreted. "Satisfactorily" means that the interpretation given should be reasonably plausible; not necessarily the exact one the artist had in mind, yet not absurd. The following classified responses will serve as a fairly secure guide for scoring:—

(a) Dutch Home

Satisfactory. "Child has spilled something and is getting a scolding." "The baby has hurt herself and the mother is comforting her." "The baby is crying because she is hungry and the mother has nothing to give her." "The little girl has been naughty and is about to be punished." "The baby is crying because she does not like her dinner." "There's bread on the table and the mother won't let the little girl have it and so she is crying." "The baby is begging for something and is crying because her mamma won't give it to her." "It's a poor family. The father is dead and they don't have enough to eat."

Unsatisfactory. "The baby is crying and the mother is looking at her" (description). "It's in Holland, and there's a little girl crying, and a mamma, and there's a dish on the table" (mainly description). "The mother is teaching the child to walk" (absurd interpretation).

(b) River Scene

Satisfactory. "Man and lady eloping to get married and an Indian to row for them." "I think it represents a honeymoon trip." "In frontier days and a man and his wife have been captured by the Indians." "It's a perilous journey and they have engaged the Indian to row for them."

Unsatisfactory. "They are shooting the rapids." "An Indian rowing a man and his wife down the river" (mainly description). "A storm at sea" (absurd interpretation). "Indians have rescued a couple from a shipwreck." "They have been up the river and are riding down the rapids."

The following responses are somewhat doubtful, but should probably be scored minus: "People going out hunting and have Indian for a guide." "The man has rescued the woman from the Indians." "It's a camping trip."

(c) Post-Office

Satisfactory. "It's a lot of old farmers. They have come to the post-office to get the paper, which only comes once a week, and they are all happy." "There's something funny in the paper about one of the men and they are all laughing about it." "They are reading about the price of eggs, and they look very happy so I guess the price has gone up." "It's a bunch of country politicians reading the election news."

Unsatisfactory. "A man has just come out of the post-office and is reading to his friends." "It's a little country town and they are looking at the paper." "A man is reading the paper and the others are looking on and laughing." "Some men are reading a paper and laughing, and the other man has brought some eggs to market, and it's in a little country town." (All the above are mainly description.)

Responses like the following are somewhat better, but hardly satisfactory: "They are reading something funny in the paper." "They are reading the ads." "They are laughing about something in the newspaper," etc.

(d) Colonial Home

Satisfactory. "They are lovers and have quarreled." "The man has to go away for a long time, maybe to war, and she is afraid he won't return." "He has proposed and she has rejected him, and she is crying because she hated to disappoint him." "The woman is crying because her husband is angry and leaving her." "The man is a messenger and has brought the woman bad news."

Unsatisfactory. "The husband is leaving and the dog is looking at the lady." "It's a picture to show how people dressed in colonial times." "The lady is crying and the man is trying to comfort her." "The man is going away. The woman is angry because he is going. The dog has a ball in its mouth and looks happy, and the man looks sad."

Such responses as the following are doubtful, but rather minus than plus: "A picture of George Washington's home." "They have lost their money and they are sad" (gratuitous interpretation). "The man has struck the woman."

Doubt sometimes arises as to the proper scoring of imaginative or gratuitous interpretations. The following are samples of such: (a) "The little girl is crying because she wants a new dress and the mother is telling her she can have one when Christmas comes if she will be good." (b) "The man and woman have gone up the river to visit some friends and an Indian guide is bringing them home." (c) "Some old Rubes are reading about a circus that's going to come." (d) "Napoleon leaving his wife."

Sometimes these imaginative responses are given by very bright subjects, under the impression that they are asked to "make up" a story based on the picture. We may score them plus, provided they are not too much out of harmony with the situation and actions represented in the picture. Interpretations so gratuitous as to have little or no bearing upon the scene depicted should be scored minus.