REMARKS. The test of picture interpretation has been variously located from 12 to 15 years. It cannot be too strongly emphasized that everything depends on the nature of the pictures used, the form in which the question is put, and the standard for scoring. The Jingleman-Jack pictures used by Kuhlmann are as easy to interpret at 10 years as the Stanford pictures at 12. Spontaneous interpretation ("What is this a picture of?" or "What do you see in this picture?") comes no more readily at 14 years than provoked interpretation ("Explain this picture") at 12. The standard of scoring is no less important. If with the Stanford pictures we require three satisfactory responses out of four, the test belongs at the 12-year level, but the standard of two correct out of four can be met a year or two earlier.

Even after we have agreed upon a given series of pictures, the formula for giving the test, and upon the requisite number of passes, there remains still the question as to the proper degree of liberality in deciding what constitutes interpretation. There is no single point in mental development where the "ability to interpret pictures" sweeps in with a rush. Like the development of most other abilities, it comes by slow degrees, beginning even as early as 6 years.

The question is, therefore, to decide whether a given response contains as much and as good interpretation as we have a right to expect at the age level where the test has been placed. It is imperative for any one who would use the scale correctly to acquaint himself thoroughly with the procedure and standards described above.

XII, 8. GIVING SIMILARITIES, THREE THINGS

PROCEDURE. The procedure is the same as in VIII, 4, but with the following words:—

(a) Snake, cow, sparrow. (b) Book, teacher, newspaper. (c) Wool, cotton, leather. (d) Knife-blade, penny, piece of wire. (e) Rose, potato, tree.

As before, a little tactful urging is occasionally necessary in order to secure a response.

SCORING. Three satisfactory responses out of five are necessary for success. Any real similarity is acceptable, whether fundamental or superficial, although the giving of fundamental likenesses is especially symptomatic of good intelligence.

Failures may be classified under four heads: (1) Leaving one of the words out of consideration; (2) giving a difference instead of a similarity; (3) giving a similarity that is not real or that is too bizarre or far-fetched; and (4) inability to respond. Types (1), (3), and (4) are almost equally numerous, while type (2) is not often encountered at this level of intelligence.

This test provokes doubtful responses somewhat oftener than the earlier test of giving similarities. Those giving greatest difficulty are the indefinite statements like "All are useful," "All are made of the same material," etc. Fortunately, in most of these cases an additional question is sufficient to determine whether the subject has in mind a real similarity. Questions suitable for this purpose are: "Explain what you mean," "In what respect are they all useful?" "What material do you mean?" etc. Of course it is only permissible to make use of supplementary questions of this kind when they are necessary in order to clarify a response which has already been made.

While the amateur examiner is likely to have more or less trouble in deciding upon scores, this difficulty rapidly disappears with experience. The following samples of satisfactory and unsatisfactory responses will serve as a fairly adequate guide in dealing with doubtful cases:—

(a) Snake, cow, sparrow

Satisfactory. "All are animals" (or creatures, etc.). "All live on the land." "All have blood" (or flesh, bones, eyes, skin, etc.). "All move about." "All breathe air." "All are useful" (plus only if subject can give a use which they have in common). "All have a little intelligence" (or sense, instinct, etc.).

Unsatisfactory. "All have legs." "All are dangerous." "All feed on grain" (or grass, etc.). "All are much afraid of man." "All frighten you." "All are warm-blooded." "All get about the same way." "All walk on the ground." "All can bite." "All holler." "All drink water." "A snake crawls, a cow walks, and a sparrow flies" (or some other difference). "They are not alike."

(b) Book, teacher, newspaper

Satisfactory. "All teach." "You learn from all." "All give you information." "All help you get an education." "All are your good friends" (plus if subject can explain how). "All are useful" (plus if subject can explain how).

Unsatisfactory. "All tell you the news." "A teacher writes, and a book and newspaper have writing." "They are not alike." "All read." "All use the alphabet."

(c) Wool, cotton, leather

Satisfactory. "All used for clothing." "We wear them all." "All grow" (plus if subject can explain). "All have to be sent to the factory to be made into things." "All are useful" (plus if subject can give a use which all have in common). "All are valuable" (plus if explained).

Unsatisfactory. "All come from plants." "All grow on animals." "All came off the top of something." "All are things." "They are pretty." "All spell alike." "All are furry" (or soft, hard, etc.).

(d) Knife-blade, penny, piece of wire

Satisfactory. "All are made from minerals" (or metals). "All come from mines." "All are hard material."

Unsatisfactory. "All are made of steel" (or copper, iron, etc.). "All are made of the same metal." "All cut." "All bend easily." "All are used in building a house." "All are worthless." "All are useful in fixing things." "All have an end." "They are small." "All weigh the same." "Can get them all at a hardware store." "You can buy things with all of them." "You buy them with money." "One is sharp, one is round, and one is long" (or some other difference).

Such answers as "All are found in a boy's pocket," or "Boys like them," are not altogether bad, but hardly deserve to be called satisfactory. "All are useful" is minus unless the subject can give a use which they have in common, which in this case he is not likely to do. Bizarre uses are also minus; as, "All are good for a watch fob," "Can use all for paper weights," etc.

(e) Rose, potato, tree

Satisfactory. "All are plants." "All grow from the ground." "All have leaves" (or roots, etc.). "All have to be planted." "All are parts of nature." "All have colors."

Unsatisfactory. "All are pretty." "All bear fruit." "All have pretty flowers." "All grow on bushes." "All are valuable" (or useful). "They grow close to a house." "All are ornamental." "All are shrubbery."

REMARKS. The words of each series lend themselves readily to classification into a next higher class. This is the best type of response, but with most of the series it accounts for less than two thirds of the successes among subjects of 12-year intelligence. The proportion is less than one third for subjects of 10-year intelligence and nearly three fourths at the 14-year level. It would be possible and very desirable to devise and standardize an additional test of this kind, but requiring the giving of an essential resemblance or classificatory similarity.

For discussion of the psychological factors involved in the similarities test, see VII, 5.



CHAPTER XVIII

INSTRUCTIONS FOR YEAR XIV.

XIV, 1. VOCABULARY (FIFTY DEFINITIONS, 9000 WORDS)

PROCEDURE and SCORING, as in VIII, X, and XII. At year XIV fifty words must be correctly defined.

XIV, 2. INDUCTION TEST: FINDING A RULE

PROCEDURE. Provide six sheets of thin blank paper, say 81/2 x 11 inches. Take the first sheet, and telling the subject to watch what you do, fold it once, and in the middle of the folded edge tear out or cut out a small notch; then ask the subject to tell you how many holes there will be in the paper when it is unfolded. The correct answer, one, is nearly always given without hesitation. But whatever the answer, unfold the paper and hold it up broadside for the subject's inspection. Next, take another sheet, fold it once as before and say: "Now, when we folded it this way and tore out a piece, you remember it made one hole in the paper. This time we will give the paper another fold and see how many holes we shall have." Then proceed to fold the paper again, this time in the other direction, and tear out a piece from the folded side and ask how many holes there will be when the paper is unfolded. After recording the answer, unfold the paper, hold it up before the subject so as to let him see the result. The answer is often incorrect and the unfolded sheet is greeted with an exclamation of surprise. The governing principle is seldom made out at this stage of the experiment. But regardless of the correctness or incorrectness of the first and second answers, proceed with the third sheet. Fold it once and say: "When we folded it this way there was one hole." Then fold it again and say: "And when we folded it this way there were two holes." At this point fold the paper a third time and say: "Now, I am folding it again. How many holes will it have this time when I unfold it?" Record the answer and again unfold the paper while the subject looks on.

Continue in the same manner with sheets four, five, and six, adding one fold each time. In folding each sheet recapitulate the results with the previous sheets, saying (with the sixth, for example): "When we folded it this way there was one hole, when we folded it again there were two, when we folded it again there were four, when we folded it again there were eight, when we folded it again there were sixteen; now, tell me how many holes there will be if we fold it once more." In the recapitulation avoid the expression "When we folded it once, twice, three times," etc., as this often leads the subject to double the numeral heard instead of doubling the number of holes in the previously folded sheet. After the answer is given, do not fail to unfold the paper and let the subject view the result.

SCORING. The test is passed if the rule is grasped by the time the sixth sheet is reached; that is, the subject may pass after five incorrect responses, provided the sixth is correct and the governing rule can then be given. It is not permissible to ask for the rule until all six parts of the experiment have been given. Nothing must be said which could even suggest the operation of a rule. Often, however, the subject grasps the principle after two or three steps and gives it spontaneously. In this case it is unnecessary to proceed with the remaining steps.

REMARKS. This test was first used by the writer in a comparative study of the intellectual processes of bright and dull boys in 1905, but it was not standardized until 1914. Rather extensive data indicate that it is a genuine test of intelligence. Of 14-year-old school children testing between 96 and 105 I Q, 59 per cent passed this test; of 14-year-olds testing below 96 I Q, 41 per cent passed; of those testing above 105, 71 per cent passed. That is, the test agrees well with the results obtained by the scale as a whole. Of "average adults" only 10 per cent fail; and of "superior adults," fewer than 5 per cent. As a rule, the higher the grade of intelligence, the fewer the steps necessary for grasping the rule. Of the superior adults, only 35 per cent fail to get the rule as early as the end of the fourth step.

The test is little affected by schooling, and apart from differences in intelligence it is little influenced by age. Other advantages of the test are the keen interest it always arouses and its independence of language ability. It has been used successfully with immigrant subjects who had been in this country but a few months.

We have named the experiment an "induction test." It might be supposed that the solution would ordinarily be arrived at by deduction, or by an a-priori logical analysis of the principle involved. This, however, is rarely the case. Not one average adult out of ten reasons out the situation in this purely logical manner. It is ordinarily only after one or more mistakes have been made and have been exposed by the examiner holding up the unfolded paper to view that the correct principle is grasped. In the absence of deductive reasoning the subject must note that each unfolded sheet contains twice as many holes as the previous one, and must infer that folding the paper again will again double the number. The ability tested is the ability to generalize from particulars where the common element of the particulars can be discerned only by the selective action of attention, in this case attention to the fact that each number is the double of its predecessor.

XIV, 3. GIVING DIFFERENCES BETWEEN A PRESIDENT AND A KING

PROCEDURE. Say: "There are three main differences between a president and a king; what are they?" If the subject stops after one difference is given, we urge him on, if possible, until three are given.

SCORING. The three differences relate to power, tenure, and manner of accession. Only these differences are considered correct, and the successful response must include at least two of the three. We disregard crudities of expression and note merely whether the subject has the essential idea. As regards power, for example, any of the following responses are satisfactory: "The king is absolute and the president is not." "The king rules by himself, but the president rules with the help of the people." "Kings can have things their own way more than presidents can," etc.

It may be objected that the reverse of this is sometimes true, that the king of to-day often has less power than the average president. Sometimes subjects mention this fact, and when they do we credit them with this part of the test. As a matter of fact, however, this answer is seldom given.

Sometimes the subject does not stop until he has given a half-dozen or more differences, and in such cases the first three differences may be trivial and some of the later ones essential. The question then arises whether we should disregard the errors and pass the subject on his later correct responses. The rule in such cases is to ask the subject to pick out the "three main differences."

Sometimes accession and tenure are given in the form of a single contrast, as: "The president is elected, but the king inherits his throne and rules for life." This answer entitles the subject to credit for both accession and tenure, the contrast as regards tenure being plainly implied.

Unsatisfactory contrasts are of many kinds and are often amusing. Some of the most common are the following:—

"A king wears a crown." "A king has jewels." "A king sits on a throne." ("A king sets on a thorn" as one feeble-minded boy put it!) "A king lives in a palace." "A king has courtiers." "A king is very dignified." "A king dresses up more." "A president has less pomp and ceremony." "A president is more ready to receive the people." "A king sits on a chair all the time and a president does not." "No differences; it's just names." "A president does not give titles." "A king has a larger salary." "A king has royal blood." "A king is in more danger." "They have a different title." "A king is more cruel." "Kings have people beheaded." "A king rules in a monarchy and a president in a republic." "A king rules in a foreign country." "A president is elected and a king fights for his office." "A president appoints governors and a king does not." "A president lets the lawyers make the laws." "Everybody works for a king."

It is surprising to see how often trivial differences like the above are given. About thirty "average adults" out of a hundred, including high-school students, give at least one unsatisfactory contrast.

The test has been criticized as depending too much on schooling. The criticism is to a certain extent valid when the test is used with young subjects, say of 10 or 12 years. It is not valid, however, if the use of the test is confined to older subjects. With the latter, it is not a test of knowledge, but of the discriminative capacity to deal with knowledge already in the possession of the subject. It would be difficult to find an adult, not actually feeble-minded, who is ignorant of the facts called for: That the king inherits his throne, while the president is elected; that the tenure of the king is for life, and that of the president for a term of years; that kings ordinarily have, or are supposed to have, more power. Even the relatively stupid adult knows this; but he also knows that kings are different from presidents in having crowns, thrones, palaces, robes, courtiers, larger pay, etc., and he makes no discrimination as regards the relative importance of these differences.

The test is psychologically related to that of giving differences in year VII and to the two tests of finding similarities; but it differs from these in requiring a comparison based on fundamental rather than accidental distinctions. The idea is good and should be worked out in additional tests of the same type.

The test first appeared in the Binet revised scale of 1911. Kuhlmann omits it, and besides our own there are few statistics bearing on it. Our results show that if two essential differences are required, the test belongs where we have placed it, but that if only one essential difference is required, the test is easy enough for year XII.

XIV, 4. PROBLEM QUESTIONS

PROCEDURE. Say to the subject: "Listen, and see if you can understand what I read." Then read the following three problems, rather slowly and with expression, pausing after each long enough for the subject to find an answer:—

(a) "A man who was walking in the woods near a city stopped suddenly, very much frightened, and then ran to the nearest policeman, saying that he had just seen hanging from the limb of a tree a ... a what?" (b) "My neighbor has been having queer visitors. First a doctor came to his house, then a lawyer, then a minister (preacher or priest). What do you think happened there?" (c) "An Indian who had come to town for the first time in his life saw a white man riding along the street. As the white man rode by, the Indian said—'The white man is lazy; he walks sitting down.' What was the white man riding on that caused the Indian to say, 'He walks sitting down'?"

Do not ask questions calculated to draw out the correct response, but wait in silence for the subject's spontaneous answer. It is permissible, however, to re-read the passage if the subject requests it.

SCORING. Two responses out of three must be satisfactory. The following explanations and examples will make clear the requirements of the test:—

(a) What the man saw hanging

Satisfactory. The only correct answer for the first is "A man who had hung himself" (or who had committed suicide, been hanged, etc.). We may also pass the following answer: "Dead branches that looked like a man hanging."

A good many subjects answer simply, "A man." This answer cannot be scored because of the impossibility of knowing what is in the subject's mind, and in such cases it is always necessary to say: "Explain what you mean." The answer to this interrogation always enables us to score the response.

Unsatisfactory. There is an endless variety of failures: "A snake," "A monkey," "A robber," or "A tramp" being the most common. Others include such answers as "A bear," "A tiger," "A wild cat," "A cat," "A bird," "An eagle," "A bird's nest," "A hornet's nest," "A leaf," "A swing," "A boy in a swing," "A basket of flowers," "An egg," "A ghost," "A white sheet," "Clothes," "A purse," etc.

(b) My neighbor

Satisfactory. The expected answer is "A death," "Some one has died," etc. We must always check up this response, however, by asking what the lawyer came for, and this must also be answered correctly.

While it is expected that the subject will understand that the doctor came to attend a sick person, the lawyer to make his will, and the minister to preach the funeral, there are a few other ingenious interpretations which pass as satisfactory. For example, "A man got hurt in an accident; the doctor came to make him well, the lawyer to see about damages, and then he died and the preacher came for the funeral." Or, "A man died, the lawyer came to help the widow settle the estate and the preacher came for the funeral." We can hardly expect the 14-year-old child to know that it is not the custom to settle an estate until after the funeral.

The following excellent response was given by an enlightened young eugenist: "A marriage; the doctor came to examine them and see if they were fit to marry, the lawyer to arrange the marriage settlement, and the minister to marry them." The following logical responses occurred once each: "A murder. The doctor came to examine the body, the lawyer to get evidence, and the preacher to preach the funeral." "An unmarried girl has given birth to a child. The lawyer was employed to get the man to marry her and then the preacher came to perform the wedding ceremony." Perhaps some will consider this interpretation too far-fetched to pass. But it is perfectly logical and, unfortunately, represents an occurrence which is not so very rare.

If an incorrect answer is first given and then corrected, the correction is accepted.

Unsatisfactory. The failures again are quite varied, but are most frequently due to failure to understand the lawyer's mission. Of 66 tabulated failures, 26 are accounted for in this way, while only 6 are due to inability to state the part played by the minister. The most common incorrect responses are: "A baby born" (accounting for 5 out of 66 failures); "A divorce" (very common with the children tested by Dr. Ordahl, at Reno, Nevada!); "A marriage"; "A divorce and a remarriage"; "A dinner"; "An entertainment"; "Some friends came to chat," etc. In 20 failures out of 66, marriage was incorrectly connected with a will, a divorce, the death of a child, etc.

The following are not bad, but hardly deserve to pass: "Sickness and trouble; the lawyer and minister came to help him out of trouble." Or, "Somebody was sick; the lawyer wanted his money and the minister came to see how he was." A few present a still more logical interpretation, but so far-fetched that it is doubtful whether they should count as passes; for example: "A man and his wife had a fight. One got hurt and had to have the doctor, then they had a lawyer to get them divorced, then the minister came to marry one of them." Again, "Some one is dying and is getting married and making his will before he dies."

(c) What the man was riding on

The only correct response is "Bicycle." The most common error is horse (or donkey), accounting for 48 out of 71 tabulated failures. Vehicles, like wagon, buggy, automobile, or street car, were mentioned in 14 out of 71 failures. Bizarre replies are: "A cripple in a wheel chair"; "A person riding on some one's back," etc.

REMARKS. The experiment is a form of the completion test. Elements of a situation are given, out of which the entire situation is to be constructed. This phase of intelligence has already been discussed.[74]

[74] See IX, 5, and XII, 4.

While it is generally admitted that the underlying idea of this test is good, some have criticized Binet's selection of problems. Meumann thinks the lawyer element of the second is so unfamiliar to children as to render that part of the test unfair. Several "armchair" critics have mentioned the danger of nervous shock from the first problem. Bobertag throws out the test entirely and substitutes a completion test modeled after that of Ebbinghaus. Our own results are altogether favorable to the test. If it is used in year XIV, Meumann's objection hardly holds, for American children of that age do ordinarily know something about making wills. As for the danger of shock from the first problem, we have never once found the slightest evidence of this much-feared result. The subject always understands that the situation depicted is hypothetical, and so answers either in a matter-of-fact manner or with a laugh.

The bicycle problem is our own invention. Binet used the other two and required both to be answered correctly. The test was located in year XII of the 1908 scale, and in year XV of the 1911 revision. Goddard and Kuhlmann retain it in the original location. The Stanford results of 1911, 1912, 1914, and 1915 agree in showing the test too difficult for year XII, even when only two out of three correct responses are required. If the original form of the experiment is used, it is exceedingly difficult for year XV. As here given it fits well at year XIV.

XIV, 5. ARITHMETICAL REASONING

PROCEDURE. The following problems, printed in clear type, are shown one at a time to the subject, who reads each problem aloud and (with the printed problem still before him) finds the answer without the use of pencil or paper.

(a) If a man's salary is $20 a week and he spends $14 a week, how long will it take him to save $300? (b) If 2 pencils cost 5 cents, how many pencils can you buy for 50 cents? (c) At 15 cents a yard, how much will 7 feet of cloth cost?

Only one minute is allowed for each problem, but nothing is said about hurrying. While one problem is being solved, the others should be hidden from view. It is not permissible, if the subject gives an incorrect answer, to ask him to solve the problem again. The following exception, however, is made to this rule: If the answer given to the third problem indicates that the word yard has been read as feet, the subject is asked to read the problem through again carefully (aloud) and to tell how he solved it. No further help of any kind may be given.

SCORING. Two of the three problems must be solved correctly within the minute allotted to each. No credit is allowed for correct method if the answer is wrong.

REMARKS. We have selected these problems from the list used by Bonser in his Study of the Reasoning Ability of Children in the Fourth, Fifth, and Sixth School Grades.[75]

[75] Columbia University Contributions to Education, no. 37, 1910.

Our tests of 279 "at age" children between 12 and 15 years reveal the surprising fact that the test as here used and scored is not passed by much over half of the children of any age in the grades below the high-school age. Of the high-school pupils 19 per cent failed to pass, 21 per cent of ordinarily successful business men (!), and 27 per cent of Knollin's unemployed men testing up to the "average adult" level. To find average intelligence cutting such a sorry figure raises the question whether the ancient definition of man as "the rational animal" is justified by the facts. The truth is, average intelligence does not do a great deal of abstract, logical reasoning, and the little it does is done usually under the whip of necessity.

At first thought these problems will doubtless appear to the reader to be mere tests of schooling. It is true, of course, that in solving them the subject makes use of knowledge which is ordinarily obtained in school; but this knowledge (that is, knowledge of reading and of addition, subtraction, multiplication, and division) is possessed by practically all adults who are not feeble-minded, and by many who are. Success, therefore, depends upon the ability to apply this knowledge readily and accurately to the problems given—precisely the kind of ability in which a deficiency cannot be made good by school training. We can teach even morons how to read problems and how to add, subtract, multiply, and divide with a fair degree of accuracy; the trouble comes when they try to decide which of these processes the problem calls for. This may require intelligence of high or low order, according to the difficulty of the problem. As for the present test, we have shown that almost totally unschooled men of "average adult" intelligence pass this test as frequently as high-school seniors of the same mental level.

XIV, 6. REVERSING HANDS OF CLOCK

PROCEDURE. Say to the subject: "Suppose it is six twenty-two o'clock, that is, twenty-two minutes after six; can you see in your mind where the large hand would be, and where the small hand would be?" Subjects of 12- to 14-year intelligence practically always answer this in the affirmative. Then continue: "Now, suppose the two hands of the clock were to trade places, so that the large hand takes the place where the small hand was, and the small hand takes the place where the large hand was. What time would it then be?"

Repeat the test with the hands at 8.10 (10 minutes after 8), and again with the hands at 2.46 (14 minutes before 3).

The subject is not allowed to look at a clock or watch, or to aid himself by drawing, but must work out the problem mentally. As a rule the answer is given within a few seconds or not at all. If an answer is not forthcoming within two minutes the score is failure.

SCORING. The test is passed if two of the three problems are solved within the following range of accuracy: the first solution is considered correct if the answer falls between 4.30 and 4.35, inclusive; the second if the answer falls between 1.40 and 1.45, and the third if the answer falls between 9.10 and 9.15.

REMARKS. It appears that success in the test chiefly depends upon voluntary control over constructive visual imagery. Weakness of visual imagery may account for the failure of a considerable percentage of adults to pass the test. Visual imagery, however, is not absolutely necessary to success. One 8-year-old prodigy, who had 12-year intelligence, arrived in forty seconds at a strictly mathematical solution for the second problem, as follows: "If it is 2.46, and the hands trade places, then the little hand has gone about one fourth of the distance from 9 o'clock to 10 o'clock. One fourth of 60 minutes is 15 minutes, and so the time would be 15 minutes after 9 o'clock." Such a solution is certainly possible by the use of verbal imagery of any type.

The test shows a high correlation with mental age, but more than most others it is subject to the influence of cribbing. For this reason, other positions of the clock hands should be tried out for the purpose of finding substitute experiments of equal difficulty. Until such experiments have been made, it will be necessary to confine the experiment to the three positions here presented.

Schooling seems to have no influence whatever on the percentage of passes.

This test was first used by Binet in 1905, but was not included in either the 1908 or 1911 series. Goddard and Kuhlmann both include the test in their revisions, placing it in year XV. They give only two problems (our a and c) and require that both be answered correctly. Neither Goddard nor Kuhlmann, however, indicates the degree of error permitted.

Something depends upon original position of the hands. Binet used 6.20 and 2.46. For some reason the 2.46 arrangement is much more difficult than either 8.10 or 6.22, yielding almost twice as many failures as either of the other positions.

XIV, ALTERNATIVE TESTS: REPEATING SEVEN DIGITS

This time, as in year X, only two series are given, one of which must be repeated without error. The two series are: 2-1-8-3-4-3-9 and 9-7-2-8-4-7-5. Note that in none of the tests of repeating digits is it permissible to warn the subject of the number to be given.

REMARKS. Binet originally placed this test in year XII, giving three trials, but later moved it to year XV. Goddard and Kuhlmann retain it in year XII. Our data show that when three trials are given the test is too easy for year XIV, but that it fits this age when only two trials are allowed; that after the age of 12 or 14 years memory for relatively meaningless material, like digits or nonsense syllables, improves but little; and that above this level it does not correlate very closely with intelligence.



CHAPTER XIX

INSTRUCTIONS FOR "AVERAGE ADULT"

AVERAGE ADULT, 1: VOCABULARY (SIXTY-FIVE DEFINITIONS, 11,700 WORDS)

PROCEDURE and SCORING, as in previous vocabulary tests.[76] At the average adult level sixty-five words should be correctly defined.

[76] See VIII, 6.

AVERAGE ADULT, 2: INTERPRETATION OF FABLES (SCORE 8)

PROCEDURE. As in year XII, test 6. Use the same fables.

SCORING. The method of scoring is the same as for XII, but the total score must be 8 points to satisfy the requirements at this level.

REMARKS. For discussion of test, see XII, 5.

AVERAGE ADULT, 3: DIFFERENCES BETWEEN ABSTRACT TERMS

PROCEDURE. Say: What is the difference between:—

(a) Laziness and idleness? (b) Evolution and revolution? (c) Poverty and misery? (d) Character and reputation?

SCORING. Three correct contrasting definitions out of four are necessary for a pass. It is not sufficient merely to give a correct meaning for each word of a pair; the subject must point out a difference between the two words so as to make a real contrast. For example, if the subject defines evolution as a "growth" or "gradual change," and revolution as the turning of a wheel on its axis, the experimenter should say: "Yes, but I want you to tell me the difference between evolution and revolution." If the contrast is not then forthcoming the response is marked minus.

The following are sample definitions which may be considered acceptable:—

(a) Laziness and idleness. "It is laziness if you won't work, and idleness if you are willing to work but haven't any job." "Lots of men are idle who are not lazy and would like to work if they had something to do." "Laziness means you don't want to work; idleness means you are not doing anything just now." "Idle people may be lazy, or they may just happen to be out of a job." "It is laziness when you don't like to work, and idleness when you are not working." "An idle person might be willing to work; a lazy man won't work." "Laziness comes from within; idleness may be forced upon one." "Laziness is aversion to activity; idleness is simply the state of inactivity." "Laziness is idleness from choice or preference; idleness means doing nothing."

The essential contrast, accordingly, is that laziness refers to unwillingness to work; idleness to the mere fact of inactivity. This contrast must be expressed, however clumsily.

(b) Evolution and revolution. "Evolution is a gradual change; revolution is a sudden change." "Evolution is natural development; revolution is sudden upheaval." "Evolution means an unfolding or development; revolution means a complete upsetting of everything." "Evolution is the gradual development of a country or government; revolution is a quick change of government." "Evolution takes place by natural force; a revolution is caused by an outside force." "Evolution is growth; revolution is a quick change from existing conditions." "Evolution is a natural change; revolution is a violent change." "Evolution is growth step by step; revolution is more sudden and radical in its action." "Evolution is a change brought about by peaceful development, while revolution is brought about by an uprising."

The essential distinction, accordingly, is that evolution means a gradual, natural, or slow change, while revolution means a sudden, forced, or violent change. Non-contrasting definitions, even when the individual terms are defined correctly, are not satisfactory.

(c) Poverty and misery. "Poverty is when you are poor; misery means suffering." "Only the poor are in poverty, but everybody can be miserable." "Poverty is the lowest stage of poorness; misery means pain." "The poor are not always miserable, and the rich are miserable sometimes." "Poverty means to be in want; misery comes from any kind of suffering or anguish." "The poor are in poverty; the sick are in misery." "Poverty is the condition of being very poor financially; misery is a feeling which any class of people can have." "One who is poor is in poverty; one who is wretched or doesn't enjoy life is in misery." "Poverty comes from lack of money; misery, from lack of happiness or comfort." "Misery means distress. It can come from poverty or many other things."

(d) Character and reputation. "Character is what you are; reputation is what people say about you." "You have character if you are honest; but you might be honest and still have a bad reputation among people who misjudge you." "Character is your real self; reputation is the opinion people have about you." "Your character depends upon yourself; reputation depends on what others think of you." "Character means your real morals; reputation is the way you are known in the world." "A man has a good character if he would not do evil; but a man may have a good reputation and still have a bad character."

A little practice and a good deal of discrimination are necessary for the correct grading of responses to this test. Subjects are often so clumsy in expression that their responses are anything but clear. It is then necessary to ask them to explain what they mean. Further questioning, however, is not permissible. For uniformity in scoring it is necessary to bear in mind that the definitions given must, in order to be satisfactory, express the essential distinction between the two words.

REMARKS. What we have said regarding the psychological significance of test 2, year XII, applies equally well here. The test on the whole is a valuable one. Our statistics show that it is not, as some critics have thought, mainly a test of schooling.

The main criticism to be made is that it imposes a somewhat difficult task upon the power of language expression. For this reason it is necessary in scoring to disregard clumsiness of expression and to look only to the essential correctness or incorrectness of the thought.

This test first appeared in year XIII of Binet's 1908 scale. The terms used were "happiness and honor"; "evolution and revolution"; "event and advent"; "poverty and misery"; "pride and pretension." In the 1911 revision, "happiness and honor" and "pride and pretension" were dropped, and the other three pairs were moved up to the adult group, two out of three successes being required for a pass. Kuhlmann places it in year XV, using "happiness and honor" instead of our "character and reputation," and requires three successes out of five.

AVERAGE ADULT, 4: PROBLEM OF THE ENCLOSED BOXES

PROCEDURE. Show the subject a cardboard box about one inch on a side. Say: "You see this box; it has two smaller boxes inside of it, and each one of the smaller boxes contains a little tiny box. How many boxes are there altogether, counting the big one?" To be sure that the subject understands repeat the statement of the problem: "First the large box, then two smaller ones, and each of the smaller ones contains a little tiny box."

Record the response, and, showing another box, say: "This box has two smaller boxes inside, and each of the smaller boxes contains two tiny boxes. How many altogether? Remember, first the large box, then two smaller ones, and each smaller one contains two tiny boxes."

The third problem, which is given in the same way, states that there are three smaller boxes, each of which contains three tiny boxes.

In the fourth problem there are four smaller boxes, each containing four tiny boxes.

The problem must be given orally, and the solution must be found without the aid of pencil or paper. Only one half-minute is allowed for each problem. Note that each problem is stated twice.

A correction is permitted, provided it is offered spontaneously and does not seem to be the result of guessing. Guessing can be checked up by asking the subject to explain the solution.

SCORING. Three of the four problems must be solved correctly within the half-minute allotted to each.

REMARKS. Success depends, in the first place, upon ability to comprehend the statement of the problem and to hold its conditions in mind. Subjects much below the 12-year level of intelligence are often unable to do this.

Granting that the problem has been comprehended, success seems to depend chiefly upon the facility with which the constructive imagination manipulates concrete visual imagery. In this respect it resembles the problem of reversing the hands of a clock. With some subjects, however, verbal imagery alone is operative. Tactual imagery would, of course, serve the purpose as well.

This is as good a place as any to emphasize the fact that the introspective study of mental imagery has little to contribute to the measurement of intelligence. Intelligence tests are concerned with the total result of a thought process, rather than with the imagery supports of that process. Thought may be carried on almost equally well by various kinds of imagery. As Galton showed, a person can be taught to carry on arithmetical processes by the use of smell imagery. The kind of imagery employed is the product of slight, innate preferences complicated by the more or less accidental effects of habit.

We may say that imagery is to thinking what scaffolding is to architecture. The important thing is the completed building rather than the nature of the scaffolding employed in erecting it. No one thinks of blaming the ill construction of a building upon the kind of scaffolding used, for if the architect and builder are competent satisfactory scaffolding will be found. Just as little are deficiencies or peculiarities of imagery the real cause of low-order intelligence. We cannot increase intelligence by formal drill in the use of supposedly important kinds of mental imagery, any more than we can transform a plain carpenter into a Michael Angelo by instructing him in the use of scaffolding materials such as were employed in the construction of St. Peter's Cathedral.

This test is of our own invention and has been brought to its present form only after a good deal of preliminary experimentation. It correlates fairly well with mental age as determined by the scale as a whole. It was passed by 55 per cent of high-school pupils and by 65 per cent of unschooled business men. Success in it is thus seen not to depend upon schooling.

AVERAGE ADULT, 5: REPEATING SIX DIGITS REVERSED

The series used are: 4-7-1-9-5-2; 5-8-3-2-9-4; and 7-5-2-6-3-8.

PROCEDURE and SCORING, as in year VII, alternative 2.

REMARKS. The test is passed by approximately half of "average adults" and by three fourths of "superior adults." It shows no effect of schooling, the uneducated business men even surpassing our high-school students.

For the higher levels of intelligence, especially, the test is superior to that of repeating digits in the direct order. It is less mechanical and makes heavier demands upon higher intelligence.

AVERAGE ADULT, 6: USING A CODE

PROCEDURE. Show the subject the code given on the accompanying form. Say: "See these diagrams here. Look and you will see that they contain all the letters of the alphabet. Now, examine the arrangement of the letters. They go (pointing) a b c, d e f, g h i, j k l, m n o, p q r, s t u v, w x y z. You see the letters in the first two diagrams are arranged in the up-and-down order (pointing again), and the letters in the other two diagrams run in just the opposite way from the hands of a clock (pointing). Look again and you will see that the second diagram is drawn just like the first, except that each letter has a dot with it, and that the last diagram is like the third except that here, also, each letter has a dot. Now, all of this represents a code; that is, a secret language. It is a real code, one that was used in the Civil War for sending secret messages. This is the way it works: we draw the lines which hold a letter, but leave out the letter. Here, for example, is the way we would write 'spy?'" Then write the word spy, pointing out carefully where each letter comes from, and emphasizing the fact that the dot must be used in addition to the lines in writing any letter in the second or the fourth diagram. Illustrate also with war.

Then add: "I am going to have you write something for me; remember now, how the letters go, first (pointing, as before) a b c, d e f, g h i, then j k l, m n o, p q r, then s t u v, then w x y z. And don't forget the dots for the letters in this diagram and this one" (pointing). At this point, take away the diagrams and tell the subject to write the words come quickly. Say nothing about hurrying.

The subject is given a pencil, but is allowed to draw only the symbols for the words come quickly. He is not permitted to reproduce the entire code and then to copy the code letters from his reproduction.

SCORING. The test is passed if the words are written in six minutes and without more than two errors. Omission of a dot counts as only a half error.

REMARKS. It is not easy to analyze the mental functions which contribute to success in the code test. Contrary to what might be supposed, success does not necessarily depend upon getting and retaining a visual picture of the diagrams. Kinaesthetic imagery will answer the purpose just as well, or the original visual impression may even be translated at once into auditory-verbal imagery and remembered as such. The significance of the test must be expressed in other terms than the kind of imagery it may happen to bring into play.

Healy and Fernald describe the task of writing a code sentence without copy as one which requires "close attention and steadiness of purpose." They also emphasize the fact that the attention must be directed inward, since there is no object of interest before the senses and since no special stimulus to attention is offered by the experimenter. Observations we have made on subjects during the test confirm this view as to the factors involved.

That inability to remember the code as a whole is not a common cause of failure is shown by the fact that subjects above 12-year intelligence who have failed on the test are nearly always able to reproduce the diagrams and insert the letters in their proper places. To give the code form of a given letter without copy, however, makes a much heavier demand on attention. Nearly all subjects find it necessary to trace the code form, in imagination, from the beginning up to each letter whose code form is sought. Subjects of superior intelligence, however, sometimes hit upon the device of remembering the position of the individual key letters e.g. (the first letter of each figure) from which, as a base, any desired letter form may be quickly sought out.

The test correlates well with mental age, but for some reason not apparent it is passed by a larger percentage of high-school pupils than unschooled adults of the same mental level.

The code test was first described by Healy and Fernald in their "Tests for Practical Mental Classification."[77] The authors gave no data, however, which would indicate the mental level to which the test belongs. Dr. Goddard incorporated it in year XV of his revision of the Binet scale, but also fails to give statistics. The location given the test in the Stanford revision is based on tests of nearly 500 individuals ranging from a mental level of 12 years to that of "superior adult." It appears that the test is considerably more difficult than most had thought it to be.

[77] Psychological Review Monographs (1911), vol. XIII, no. 2, p. 51.

AVERAGE ADULT, ALTERNATIVE TEST 1: REPEATING TWENTY-EIGHT SYLLABLES

The sentences for this test are:—

(a) Walter likes very much to go on visits to his grandmother, because she always tells him many funny stories. (b) Yesterday I saw a pretty little dog in the street. It had curly brown hair, short legs, and a long tail.

PROCEDURE. Exactly as in VI, 6. Emphasize that the sentence must be repeated without a single change of any sort. Get attention before giving each sentence.

SCORING. Passed if one sentence is repeated without a single error. In VI and X we scored the response as satisfactory if one sentence was repeated without error, or if two were repeated with not more than one error each.

REMARKS. The test of repeating sentences is not as satisfactory in the higher intelligence levels as in the lower. It is too mechanical to tax very heavily the higher thought processes. It does, however, have a certain correlation with intelligence. Contrary to what one would have expected, uneducated adults of "average adult" intelligence surpassed our high-school students of the same mental level.

Binet located this test in year XII of the 1908 series, but shifted it to year XV in 1911. The American versions of the Binet scale have usually retained it in year XII, though Goddard admits that the sentences are somewhat too difficult for that year. Kuhlmann puts the test in year XII, but reduces the sentences to twenty-four syllables and permits one re-reading. We give only two trials and our sentences are considerably more difficult. With the procedure and scoring we have used, the test is rather easy for the "average adult" group, but a little too hard for year XIV.

AVERAGE ADULT, ALTERNATIVE TEST 2: COMPREHENSION OF PHYSICAL RELATIONS

(a) Problem regarding the path of a cannon ball

PROCEDURE. Draw on a piece of paper a horizontal line six or eight inches long. Above it, an inch or two, draw a short horizontal line about an inch long and parallel to the first. Tell the subject that the long line represents the perfectly level ground of a field, and that the short line represents a cannon. Explain that the cannon is "pointed horizontally (on a level) and is fired across this perfectly level field." After it is clear that these conditions of the problem are comprehended, we add: "Now, suppose that this cannon is fired off and that the ball comes to the ground at this point here (pointing to the farther end of the line which represents the field). Take this pencil and draw a line which will show what path the cannon ball will take from the time it leaves the mouth of the cannon till it strikes the ground."

SCORING. There are four types of response: (1) A straight diagonal line is drawn from the cannon's mouth to the point where the ball strikes. (2) A straight line is drawn from the cannon's mouth running horizontally until almost directly over the goal, at which point the line drops almost or quite vertically. (3) The path from the cannon's mouth first rises considerably from the horizontal, at an angle perhaps of between ten to forty-five degrees, and finally describes a gradual curve downward to the goal. (4) The line begins almost on a level and drops more rapidly toward the end of its course.

Only the last is satisfactory. Of course, nothing like a mathematically accurate solution of the problem is expected. It is sufficient if the response belongs to the fourth type above instead of being absurd, as the other types described are. Any one who has ever thrown stones should have the data for such an approximate solution. Not a day of schooling is necessary.

(b) Problem as to the weight of a fish in water

PROCEDURE. Say to the subject: "You know, of course, that water holds up a fish that is placed in it. Well, here is a problem. Suppose we have a bucket which is partly full of water. We place the bucket on the scales and find that with the water in it it weighs exactly 45 pounds. Then we put a 5-pound fish into the bucket of water. Now, what will the whole thing weigh?"

SCORING. Many subjects even as low as 9- or 10-year intelligence will answer promptly, "Why, 45 pounds and 5 pounds makes 50 pounds, of course." But this is not sufficient. We proceed to ask, with serious demeanor: "How can this be correct, since the water itself holds up the fish?" The young subject who has answered so glibly now laughs sheepishly and apologizes for his error, saying that he answered without thinking, etc. This response is scored failure without further questioning.

Other subjects, mostly above the 14-year level, adhere to the answer "50 pounds," however strongly we urge the argument about the water holding up the fish. In response to our question, "How can that be the case?" it is sufficient if the subject replies that "The weight is there just the same; the scales have to hold up the bucket and the bucket has to hold up the water," or words to that effect. Only some such response as this is satisfactory. If the subject keeps changing his answer or says that he thinks the weight would be 50 pounds, but is not certain, the score is failure.

(c) Difficulty of hitting a distant mark

PROCEDURE. Say to the subject: "You know, do you not, what it means when they say a gun 'carries 100 yards'? It means that the bullet goes that far before it drops to amount to anything." All boys and most girls more than a dozen years old understand this readily. If the subject does not understand, we explain again what it means for a gun "to carry" a given distance. When this part is clear, we proceed as follows: "Now, suppose a man is shooting at a mark about the size of a quart can. His rifle carries perfectly more than 100 yards. With such a gun is it any harder to hit the mark at 100 yards than it is at 50 yards?" After the response is given, we ask the subject to explain.

SCORING. Simply to say that it would be easier at 50 yards is not sufficient, nor can we pass the response which merely states that it is "easier to aim" at 50 yards. The correct principle must be given, one which shows the subject has appreciated the fact that a small deviation from the "bull's-eye" at 50 yards, due to incorrect aim, becomes a larger deviation at 100 yards. However, the subject is not required to know that the deviation at 100 yards is exactly twice as great as at 50 yards. A certain amount of questioning is often necessary before we can decide whether the subject has the correct principle in mind.

SCORING THE ENTIRE TEST. Two of the three problems must be solved in such a way as to satisfy the requirements above set forth.

REMARKS. These problems were devised by the writer. They yield interesting results, when properly given, but are not without their faults. Sometimes a very superior subject fails, while occasionally an inferior subject unexpectedly succeeds. On the whole, however the test correlates fairly well with mental age. At the 14-year level less than 50 per cent pass; of "average adults," from 60 to 75 per cent are successful. Few "superior adults" fail.

The test as here given is little influenced by the formal instruction given in the grades or the high school. In fact, 80 per cent of our uneducated business men, as contrasted with 65 per cent of high-school juniors and seniors, passed the test. Success probably depends in the main upon previous interest in physical relationships and upon the ability to understand phenomena of this kind which the subject has had opportunity to observe.

It would be interesting to standardize a longer series of problems designed to test a subject's comprehension of common physical relationships. In the first few months of life a normal child learns that objects unsupported fall to the ground. Later he learns that fire burns; that birds fly in the air; that fish do not sink in the water; that water does not run uphill; that it is easy to lift a leg or arm as one lies prone in the water; that mud is thrown from a rotating wheel (and always in the same direction); that a stone which is flying through the air swiftly is more dangerous than one which is moving slowly; that it is more dangerous to be run over by a train than by a buggy; that it is hard to run against a strong wind; that cyclones blow down trees and houses; that a rapidly moving train creates a stronger wind than a slower train; that a feather falls through the air with less speed than a stone; that a falling object gains momentum; that a heavy moving object is harder to stop than a light object moving at the same rate; that freezing water bursts pipes; that sounds sometimes give echoes; that rainbows cannot be approached; that a lamp seems dim by daylight; that by day the stars are not visible and the moon only barely visible; that the headlights of an approaching automobile or train are blinding; that if the room in which we are reading is badly lighted we must hold the book nearer to the eyes; that running makes the heart beat faster and increases the rate of breathing; that if we are cold we can get warm by running; that whirling rapidly makes us dizzy; that heat or exercise will cause perspiration, etc.

Although the causes of some of these phenomena are not understood even by intelligent adults without some instruction, the facts themselves are learned by the normal individual from his own experience. The higher the mental level and the greater the curiosity, the more observant one is about such matters and the more one learns. Many items of knowledge such as we have mentioned could and should be standardized for various mental levels. In devising tests of this kind we should, of course, have to look out for the influences of formal instruction.



CHAPTER XX

INSTRUCTIONS FOR "SUPERIOR ADULT"

SUPERIOR ADULT, 1: VOCABULARY (SEVENTY-FIVE DEFINITIONS, 13,500 WORDS)

PROCEDURE and SCORING, as in previous vocabulary tests. At the "superior adult" level seventy-five words should be known.

The test is passed by only one third of those at the "average adult" level, but by about 90 per cent of "superior adults." Ability to pass the test is relatively independent of the number of years the subject has attended school, our business men showing even a higher percentage of passes than high-school pupils.

SUPERIOR ADULT, 2: BINET'S PAPER-CUTTING TEST

PROCEDURE. Take a piece of paper about six inches square and say: "Watch carefully what I do. See, I fold the paper this way (folding it once over in the middle), then I fold it this way (folding it again in the middle, but at right angles to the first fold). Now, I will cut out a notch right here" (indicating). At this point take scissors and cut out a small notch from the middle of the side which presents but one edge. Throw the fragment which has been cut out into the waste-basket or under the table. Leave the folded paper exposed to view, but pressed flat against the table. Then give the subject a pencil and a second sheet of paper like the one already used and say: "Take this piece of paper and make a drawing to show how the other sheet of paper would look if it were unfolded. Draw lines to show the creases in the paper and show what results from the cutting."

The subject is not permitted to fold the second sheet, but must solve the problem by the imagination unaided.

Note that we do not say, "Draw the holes," as this would inform the subject that more than one hole is expected.

SCORING. The test is passed if the creases in the paper are properly represented, if the holes are drawn in the correct number, and if they are located correctly, that is, both on the same crease and each about halfway between the center of the paper and the side. The shape of the holes is disregarded.

Failure may be due to error as regards the creases or the number and location of the holes, or it may involve any combination of the above errors.

REMARKS. Success seems to depend upon constructive visual imagination. The subject must first be able to construct in imagination the creases which result from the folding, and secondly, to picture the effects of the cutting as regards number of holes and their location. It appears that a solution is seldom arrived at, even in the case of college students, by logical mathematical thinking. Our unschooled subjects even succeeded somewhat better than high-school and college students of the same mental level.

Binet placed this test in year XIII of the 1908 scale, but shifted it to the adult group in the 1911 revision. Goddard retains it in the adult group, while Kuhlmann places it in year XV. There have also been certain variations in the procedure employed. As given in the Stanford revision the test is passed by hardly any subjects below the 14-year level, but by about one third of "average adults" and by the large majority of "superior adults."

SUPERIOR ADULT, 3: REPEATING EIGHT DIGITS

PROCEDURE and SCORING, the same as in previous tests with digits reversed. The series used are: 7-2-5-3-4-8-9-6; 4-9-8-5-3-7-6-2; and 8-3-7-9-5-4-8-2.

Guard against rhythm and grouping in reading the digits and do not give warning as to the number to be given.

The test is passed by about one third of "average adults" and by over two thirds of "superior adults." The test shows no marked difference between educated and uneducated subjects of the same mental level.

SUPERIOR ADULT, 4: REPEATING THOUGHT OF PASSAGE

PROCEDURE. Say: "I am going to read a little selection of about six or eight lines. When I am through I will ask you to repeat as much of it as you can. It doesn't make any difference whether you remember the exact words or not, but you must listen carefully so that you can tell me everything it says." Then read the following selections, pausing after each for the subject's report, which should be recorded verbatim:—

(a) "Tests such as we are now making are of value both for the advancement of science and for the information of the person who is tested. It is important for science to learn how people differ and on what factors these differences depend. If we can separate the influence of heredity from the influence of environment, we may be able to apply our knowledge so as to guide human development. We may thus in some cases correct defects and develop abilities which we might otherwise neglect." (b) "Many opinions have been given on the value of life. Some call it good, others call it bad. It would be nearer correct to say that it is mediocre; for on the one hand, our happiness is never as great as we should like, and on the other hand, our misfortunes are never as great as our enemies would wish for us. It is this mediocrity of life which prevents it from being radically unjust."

Sometimes the subject hesitates to begin, thinking, in spite of our wording of the instructions, that a perfect reproduction is expected. Others fall into the opposite misunderstanding and think that they are prohibited from using the words of the text and must give the thought entirely in their own language. In cases of hesitation we should urge the subject a little and remind him that he is to express the thought of the selection in whatever way he prefers; that the main thing is to tell what the selection says.

SCORING. The test is passed if the subject is able to repeat in reasonably consecutive order the main thoughts of at least one of the selections. Neither elegance of expression nor verbatim repetition is expected. We merely want to know whether the leading thoughts in the selection have been grasped and remembered.

All grades of accuracy are found, both in the comprehension of the selection and in the recall, and it is not always easy to draw the line between satisfactory and unsatisfactory responses. The following sample performances will serve as a guide:—

Selection (a)

Satisfactory. "The tests which we are making are given for the advancement of science and for the information of the person tested. By scientific means we will be able to separate characteristics derived from heredity and environment and to treat each class separately. By doing so we can more accurately correct defects."

"Tests like these are for two purposes. First to develop a science, and second to apply it to the person to help him. The tests are to find out how you differ from another and to measure the difference between your heredity and environment."

"These tests are given to see if we can separate heredity and environment and to see if we can find out how one person differs from another. We can then correct these differences and teach people more effectively."

"The tests that we are now making are valuable along both scientific and personal lines. By using them it can be found out where a person is weak and where he is strong. We can then strengthen his weak points and remedy some things that would otherwise be neglected. They are of great benefit to science and to the person concerned."

"Tests such as we are now making are of great importance because they aim to show in what respects we differ from others and why, and if they do this they will be able to guide us into the right channel and bring success instead of failure."

Unsatisfactory. "Tests such as we are now making are of value both for the advancement of science and for the information of the person interested. It is necessary to know this."

"Such tests as we are now making show about the human mind and show in what channels we are fitted. It is the testing of each individual between his effects of inheritancy and environment."

"It is very interesting for us to study science for two reasons; first, to test our mental ability, and second for the further development of science."

"Tests such as we are now making help in two ways; it helps the scientists and it gives information to the people."

"Tests are being given to pupils to-day to better them and to aid science for generations to come. If each person knows exactly his own beliefs and ideas and faults he can find out exactly what kind of work he is fitted for by heredity. The tests show that environment doesn't count, for if you are all right you will get along anyway." (Note invention.)

Selection (b)

Satisfactory. "There are different opinions about life. Some call it good and some bad. It would be more correct to say that it is middling, because we are never as happy as we would like to be and we are never as sad as our enemies want us to be."

"One hears many judgments about life. Some say it is good, while others say it is bad. But it is really neither of the extremes. Life is mediocre. We do not have as much good as we desire, nor do we have as much misfortune as others want us to have. Nevertheless, we have enough good to keep life from being unjust."

"Some people have different views of life from others. Some say it is bad, others say it is good. It is better to class life as mediocre, as it is never as good as we wish it, and on the other hand, it might be worse."

"Some people think differently of life. Some think it good, some bad, others mediocre, which is nearest correct. It brings unhappiness to us, but not as much as our enemies want us to have."

Unsatisfactory. "Some say life is good, some say it is mediocre. Even though some say it is mediocre they say it is right."

"There are two sides of life. Some say it is good while others say it is bad. To some, life is happy and they get all they can out of life. For others life is not happy and therefore they fail to get all there is in life."

"One hears many different judgments of life. Some call it good, some call it bad. It brings unhappiness and it does not have enough pleasure. It should be better distributed."

"There are different opinions of the value of life. Some say it is good and some say it is bad. Some say it is mediocrity. Some think it brings happiness while others do not."

"Nowadays there is much said about the value of life. Some say it is good, while others say it is bad. A person should not have an ill feeling toward the value of life, and he should not be unjust to any one. Honesty is the best policy. People who are unjust are more likely to be injured by their enemies." (Note invention.)

REMARKS. Contrary to what the subject is led to expect, the test is less a test of memory than of ability to comprehend the drift of an abstract passage. A subject who fully grasps the meaning of the selection as it is read is not likely to fail because of poor memory. Mere verbal memory improves but little after the age of 14 or 15 years, as is shown by the fact that our adults do little better than eighth-grade children in repeating sentences of twenty-eight syllables. On the other hand, adult intelligence is vastly superior in the comprehension and retention of a logically presented group of abstract ideas.

There is nothing in which stupid persons cut a poorer figure than in grappling with the abstract. Their thinking clings tenaciously to the concrete; their concepts are vague or inaccurate; the interrelations among their concepts are scanty in the extreme; and such poor mental stores as they have are little available for ready use.

A few critics have objected to the use of tests demanding abstract thinking, on the ground that abstract thought is a very special aspect of intelligence and that facility in it depends almost entirely on occupational habits and the accidents of education. Some have even gone so far as to say that we are not justified, on the basis of any number of such tests, in pronouncing a subject backward or defective. It is supposed that a subject who has no capacity in the use of abstract ideas may nevertheless have excellent intelligence "along other lines." In such cases, it is said, we should not penalize the subject for his failures in handling abstractions, but substitute, instead, tests requiring motor cooerdination and the manipulation of things, tests in which the supposedly dull child often succeeds fairly well.

From the psychological point of view, such a proposal is naively unpsychological. It is in the very essence of the higher thought processes to be conceptual and abstract. What the above proposal amounts to is, that if the subject is not capable of the more complex and strictly human type of thinking, we should ignore this fact and estimate his intelligence entirely on the ability he displays to carry on mental operations of a more simple and primitive kind. This would be like asking the physician to ignore the diseased parts of his patient's body and to base his diagnosis on an examination of the organs which are sound!

The present test throws light in an interesting way on the integrity of the critical faculty. Some subjects are unwilling to extend the report in the least beyond what they know to be approximately correct, while others with defective powers of auto-criticism manufacture a report which draws heavily on the imagination, perhaps continuing in garrulous fashion as long as they can think of anything having the remotest connection with any thought in the selection. We have included, for each selection, one illustration of this type in the sample failures given above.

The worst fault of the test is its susceptibility to the influence of schooling. Our uneducated adults of even "superior adult" intelligence often fail, while about two thirds of high-school pupils succeed. The unschooled adults have a marked tendency either to give a summary which is inadequate because of its extreme brevity, or else to give a criticism of the thought which the passage contains.

This test first appeared in Binet's 1911 revision, in the adult group. Binet used only selection (b), and in a slightly more difficult form than we have given above. Goddard gives the test like Binet and retains it in the adult group. Kuhlmann locates it in year XV, using only selection (a). On the basis of over 300 tests of adults we find the test too difficult for the "average adult" level, even on the basis of only one success in two trials and when scored on the rather liberal standard above set forth.

SUPERIOR ADULT, 5: REPEATING SEVEN DIGITS REVERSED

PROCEDURE and SCORING, the same as in previous tests of this kind. The series are: 4-1-6-2-5-9-3; 3-8-2-6-4-7-5; and 9-4-5-2-8-3-7.

We have collected fewer data on this test than on any of the others, as it was added later to the test series. As far as we have used it we have found few "average adults" who pass, while about half the "superior adults" do so.

SUPERIOR ADULT, 6: INGENUITY TEST

PROCEDURE. Problem a is stated as follows:—

A mother sent her boy to the river and told him to bring back exactly 7 pints of water. She gave him a 3-pint vessel and a 5-pint vessel. Show me how the boy can measure out exactly 7 pints of water, using nothing but these two vessels and not guessing at the amount. You should begin by filling the 5-pint vessel first. Remember, you have a 3-pint vessel and a 5-pint vessel and you must bring back exactly 7 pints.

The problem is given orally, but may be repeated if necessary.

The subject is not allowed pencil or paper and is requested to give his solution orally as he works it out. It is then possible to make a complete record of the method employed.

The subject is likely to resort to some such method as to "fill the 3-pint vessel two thirds full," or, "I would mark the inside of the 5-pint vessel so as to show where 4 pints come to," etc. We inform the subject that such a method is not allowable; that this would be guessing, since he could not be sure when the 3-pint vessel was two thirds full (or whether he had marked off his 5-pint vessel accurately). Tell him he must measure out the water without any guesswork. Explain also, that it is a fair problem, not a "catch."

Say nothing about pouring from one vessel to another, but if the subject asks whether this is permissible the answer is "yes."

The time limit for each problem is 5 minutes. If the subject fails on the first problem, we explain the solution in full and then proceed to the next.

The second problem is like the first, except that a 5-pint vessel and a 7-pint vessel are given, to get 8 pints, the subject being told to begin by filling the 5-pint vessel.

In the third problem 4 and 9 are given, to get 7, the instruction being to "begin by filling the 4-pint vessel."

Note that in each problem we instruct the subject how to begin. This is necessary in order to secure uniformity of conditions. It is possible to solve all of the problems by beginning with either of the two vessels, but the solution is made very much more difficult if we begin in the direction opposite from that recommended.

Give no further aid. It is necessary to refrain from comment of every kind.

SCORING. Two of the three problems must be solved correctly within the 5 minutes allotted to each.

REMARKS. We have called this a test of ingenuity. The subject who is given the problem finds himself involved in a difficulty from which he must extricate himself. Means must be found to overcome an obstacle. This requires practical judgement and a certain amount of inventive ingenuity. Various possibilities must be explored and either accepted for trial or rejected. If the amount of invention called for seems to the reader inconsiderable, let it be remembered that the important inventions of history have not as a rule had a Minerva birth, but instead have developed by successive stages, each involving but a small step in advance.

It is unnecessary to emphasize at length the function of invention in the higher thought processes. In one form or another it is present in all intellectual activity; in the creation and use of language, in art, in social adjustments, in religion, and in philosophy, as truly as in the domains of science and practical affairs. Certainly this is true if we accept Mason's broad definition of invention as including "every change in human activity made designedly and systematically."[78] From the psychological point of view, perhaps, Mason is justified in looking upon the great inventor as "an epitome of the genius of the world." To develop a Krag-Joergensen from a bow and arrow, a "velvet-tipped" lucifer match from the primitive fire-stick, or a modern piano from the first crude, stringed, musical instrument has involved much the same intellectual processes as have been operative in transforming fetishism and magic into religion and philosophy, or scattered fragments of knowledge into science.

[78] Otis T. Mason: The Origins of Inventions. (London, 1902.)

Psychologically, invention depends upon the constructive imagination; that is, upon the ability to abstract from what is immediately present to the senses and to picture new situations with their possibilities and consequences. Images are united in order to form new combinations.

As we have several times emphasized, the decisive intellectual differences among human beings are not greatly dependent upon mere sense discrimination or native retentiveness. Far more important than the raw mass of sense data is the correct shooting together of the sense elements in memory and imagination. This is but another name for invention. It is the synthetic, or apperceptive, activity of the mind that gives the "seven-league boots" to genius. It is, however, a kind of ability which is possessed by all minds to a greater or less degree. Any test has its value which gives a clue, as this test does, to the subject's ability in this direction.

The test was devised by the writer and used in 1905 in a study of the intellectual processes of bright and dull boys, but it was not at that time standardized. It has been found to belong at a much higher mental level than was at first supposed. Only an insignificant number pass the test below the mental age of 14 years, and about two thirds of "average adults" fail. Of our "superior adults" somewhat more than 75 per cent succeed. Formal education influences the test little or not at all, the unschooled business men making a somewhat better showing than the high-school students.



SELECTED REFERENCES

The following classified lists include only the most important references under each topic. So many investigations have been made with the Binet-Simon tests in the last few years, and so many articles have been written in evaluation of the method, that a complete bibliography of the subject would require thirty or forty pages. Those who desire to make a more thorough study of the literature are referred to the admirable annotated bibliography compiled by Samuel C. Kohs, and published by Warwick & York, Baltimore. Kohs's Bibliography contains 254 references, and is complete to January 1, 1914.

BINET-SIMON TESTS OF NORMAL CHILDREN

1. Binet, A., et Simon, Th. "Le developpement de l'intelligence chez les enfants"; in Annee psychologique (1908), vol. 14, pp. 1-94.

Exposition of the original 1908 scale with results.

2. Binet, A. "Nouvelles recherches sur la mesure du niveau intellectuel chez les enfants d'ecole"; in Annee psychologique (1911), vol. 17, pp. 145-201.

Presents the 1911 revision.

3. Bobertag, O. "Ueber Intelligenzpruefungen (nach der Methode von Binet und Simon)"; in Zeitschrift fuer angewande Psychologie (1911), vol. 5, pp. 105-203; and (1912), vol. 6, pp. 495-537.

Analysis of 400 cases and criticism of method and results.

4. Dougherty, M. L. "Report on the Binet-Simon Tests given to Four Hundred and Eighty-three Children in the Public Schools of Kansas City, Kansas"; in Journal of Educational Psychology (1913), vol. 4, pp. 338-52.

5. Goddard, H. H. "The Binet-Simon Measuring Scale for Intelligence, Revised"; in Training School Bulletin (1911), vol. 8, pp. 56-62.

6. Hoffman, A. "Vergleichende Intelligenzpruefungen an Vorschuelern und Volksschuelern"; in Zeitschrift fuer angewande Psychologie (1913), vol. 8, pp. 102-20.

One hundred and fifty-six subjects. Ages seven, nine, and ten.

7. Johnston, Katherine L. "Binet's Method for the Measurement of Intelligence; Some Results"; in Journal of Experimental Pedagogy (1911), vol. 1, pp. 24-31.

Results of 200 tests of school children.

8. Kuhlmann, F. "Some Results of Examining 1000 Public-School Children with a Revision of the Binet-Simon Tests of Intelligence by Untrained Teachers"; in Journal of Psycho-Asthenics (1914), vol. 18, pp. 150-79, and 233-69.

9. Phillips, Byron A. "The Binet Tests applied to Colored Children"; in Psychological Clinic (1914), pp. 190-96.

A comparison of 86 colored and 137 white children.

10. Rogers, Agnes L., and McIntyre, J. L. "The Measurement of Intelligence in Children by the Binet-Simon Scale"; in British Journal of Psychology (1914), vol. 7, pp. 265-300.

11. Rowe, E. C. "Five Hundred Forty-Seven White and Two Hundred Sixty-Eight Indian Children tested by the Binet-Simon Tests"; in Pedagogical Seminary (1914), vol. 21, pp. 454-69.

12. Strong, Alice C. "Three Hundred Fifty White and Colored Children measured by the Binet-Simon Measuring Scale of Intelligence"; in Pedagogical Seminary (1913), vol. 20, pp. 485-515.

13. Terman, L. M., and Childs, H. G. "A Tentative Revision and Extension of the Binet-Simon Measuring Scale of Intelligence"; in Journal of Educational Psychology (1912), vol. 3, pp. 61-74, 133-43, 198-208, and 277-89.

Results of 396 tests of California school-children.

14. Terman, Lyman, Ordahl, Galbreath, and Talbert. The Stanford Revision and Extension of the Binet-Simon Measuring Scale of Intelligence. (1916.)

Detailed analysis of the results secured by testing 1000 unselected school-children within two months of a birthday.

15. Weintrob, J. and R. "The Influence of Environment on Mental Ability as shown by the Binet Tests"; in Journal of Educational Psychology (1912), pp. 577-86.

16. Winch, W. H. "Binet's Mental Tests: What They Are, and What We Can Do with Them"; in Child Study (London), 1913, 1914, 1915, and 1916.

An extended series of articles setting forth results of tests with normal children, and giving valuable criticisms and suggestions.

BINET-SIMON TESTS OF THE FEEBLE-MINDED

17. Chotzen, F. "Die Intelligenzpruefungsmethode von Binet-Simon bei schwachsinnigen Kindern"; in Zeitschrift fuer angewande Psychologie (1912), vol. 6, pp. 411-94.

A critical study of the results of 280 tests.

18. Goddard, H. H. "Four Hundred Feeble-Minded Children classified by the Binet Method"; in Pedagogical Seminary (1910), vol. 17, pp. 387-97; also in Journal of Psycho-Asthenics (1910), vol. 15, pp. 17-30.

Offers important evidence of the value of the Binet-Simon method.

19. Kuhlmann, F. "The Binet and Simon Tests of Intelligence in Grading Feeble-Minded Children"; in Journal of Psycho-Asthenics (1912), vol. 16, pp. 173-93.

Analysis of results from 1300 cases.

BINET-SIMON TESTS OF DELINQUENTS

20. Bluemel, C. S. "Binet Tests on Two Hundred Juvenile Delinquents"; in Training School Bulletin (1915), pp. 187-93.

21. Goddard, H. H. The Criminal Imbecile. The Macmillan Company. (1915.) 157 pages.

An analysis of the mentality of three murderers of moron or borderline intelligence.

22. Goddard, H. H. "The Responsibility of Children in the Juvenile Court"; in Journal of Criminal Law and Criminology (September, 1912).

Analysis of 100 tests of juvenile delinquents.

23. Healy, William. The Individual Delinquent. Little, Brown & Co. (1915.) 830 pages.

A textbook on delinquents. Gives results of many Binet-Simon tests.

24. Spaulding, Edith R. "The Results of Mental and Physical Examination of Four Hundred Women Offenders"; in Journal of Criminal Law and Criminology (1915), pp. 704-17.

25. Sullivan, W. C. "La mesure du developpement intellectuel chez les jeunes delinquantes"; in Annee psychologique (1912), vol. 18, pp. 341-61.

26. Williams, J. Harold. A Study of 150 Delinquent Boys. Bulletin no. 1, Research Laboratory of the Buckel Foundation. (1915.) 15 pages.

The Stanford revision used. Report of over 400 cases to follow.

BINET-SIMON TESTS OF SUPERIOR CHILDREN

27. Jeronutti, A. "Ricerche psicologiche sperimentali sugli alunni molto intelligenti"; in Lab. di Psicol. Sperim. della Reg. Univ. Roma. (1912)

Out of fifteen hundred school and kindergarten children, ages five to twelve, fourteen were selected by the teachers as the brightest. The Binet test showed them to be from one to three years in advance of their chronological ages.

28. Terman, L. M. "The Mental Hygiene of Exceptional Children"; in Pedagogical Seminary (1915), vol. 22, pp. 529-37.

Data on 31 children testing above 120 I. Q.

INSTRUCTIONS FOR GIVING THE BINET-SIMON TESTS

29. Binet, A., and Simon, Th. A Method of Measuring the Development of Intelligence in Young Children. Chicago Medical Book Company. (1915.) 82 pages.

Authorized translation of Binet's final instructions for giving the tests.

30. Goddard, H. H. "A Measuring Scale of Intelligence"; in Training School Bulletin (1910), vol. 6, pp. 146-55.

Condensed translation of Binet's 1908 Measuring Scale of Intelligence.

31. Goddard, H. H. "The Binet-Simon Measuring Scale for Intelligence, Revised"; in Training School Bulletin (1911), vol. 8, pp. 56-62.

32. Goddard, H. H. "Standard Method for Giving the Binet Test"; in Training School Bulletin (1913), vol. 10, pp. 23-30.

33. Kuhlmann, F. "A Revision of the Binet-Simon System for Measuring the Intelligence of Children"; Monograph Supplement of Journal of Psycho-Asthenics (September, 1912), 41 pages.

34. Wallin, J. E. W. "A Practical Guide for the Administration of the Binet-Simon Scale for Measuring Intelligence"; in The Psychological Clinic (1911), vol. 5, pp. 217-38.

CRITICISMS AND EVALUATIONS OF THE BINET-SIMON METHOD

35. Berry, C. S. "A Comparison of the Binet Tests of 1908 and 1911"; in Journal of Educational Psychology (1912), vol. 3, pp. 444-51.

36. Bobertag, O. "Ueber Intelligenzpruefungen (nach der Methode von Binet und Simon)"; in Zeitschrift fuer angewande Psychologie. (A, 1911), vol. 5, pp. 105-203; (B, 1912), vol. 6, pp. 495-537.

Accepts the method and gives valuable suggestions for improvement.

37. Brigham, Carl C. "An Experimental Critique of the Binet-Simon Scale"; in Journal of Educational Psychology (1914), pp. 439-48.

Finds the scale 96% efficient.

38. Goddard, H. H. "The Reliability of the Binet-Simon Measuring Scale of Intelligence"; in Proceedings of the Fourth International Congress of School Hygiene (1913), vol. 5, pp. 693-99.

Application of the theory of probability to the results proves the extremely small liability of error.

39. Kohs, Samuel C. "The Practicability of the Binet Scale and the Question of the Borderline Case"; in Training School Bulletin (1916), pp. 211-23.

Analysis of cases showing the reliability of the scale.

40. Kuhlmann, F. "Binet and Simon's System for Measuring the Intelligence of Children"; in Journal of Psycho-Asthenics (1911), vol. 15, pp. 79-92.

Finds the method of the greatest value.

41. Kuhlmann, F. "A Reply to Dr. L. P. Ayres's Criticism of the Binet and Simon System for Measuring the Intelligence of Children"; in Journal of Psycho-Asthenics (1911), vol. 16, pp. 58-67.

Many of the Ayres criticisms are shown to be unfounded.

42. Meumann, E. Vorlesungen zur Einfuehrung in die Experimentelle Paedagogik (1913), vol. 2, pp. 130-300.

Summary of the literature on Binet tests up to 1913. Accepts the method but gives suggestions for improvement. This summary and other writings of Meumann on the psychology of endowment are reviewed by Lewis M. Terman in a series of four articles in the Journal of Psycho-Asthenics for 1915.

43. Otis, A. S. "Some Logical and Mathematical Aspects of the Measurement of Intelligence by the Binet-Simon Method"; in The Psychological Review (April and June, 1916).

Considers the Binet-Simon method imperfect from the mathematical point of view.

44. Schmitt, Clara. Standardization of Tests for Defective Children. Psychological Monographs (1915), no. 83, 181 pages.

Contains (pp. 52-67) a discussion of the "Fallacies and Inadequacies of the Binet-Simon Series." Most of the criticisms here given are either superficial or unfair, some of them apparently being due to a lack of acquaintance with Binet's writings.

45. Stern, W. The Psychological Methods of Measuring Intelligence. Translated by G. M. Whipple. (1913.) 160 pages.

A splendid critical discussion of the Binet-Simon method. Should be read by every one who would use the scale.

46. Terman, L. M. "Suggestions for Revising, Extending, and Supplementing the Binet Intelligence Tests"; in Journal of Psycho-Asthenics (1913), vol. 18, pp. 20-33.

47. Terman, L. M. "Psychological Principles Underlying the Binet-Simon Scale and Some Practical Considerations for its Correct Use"; in Journal of Psycho-Asthenics (1913), vol. 18, pp. 93-104.

48. Terman, L. M. "A Report of the Buffalo Conference on the Binet-Simon Tests of Intelligence"; in Pedagogical Seminary (1913), vol. 20, pp. 549-54.

Abstracts of papers presented at the above conference.

49. Terman, Lyman, Ordahl, Galbreath, and Talbert. The Stanford Revision and Extension of the Binet-Simon Scale for Measuring Intelligence. (1916.)

Contains a chapter on the validity of the individual tests and on considerations relating to the formation of an intelligence scale.

50. Terman and Knollin. "The Detection of Borderline Deficiency by the Binet-Simon Method"; in Journal of Psycho-Asthenics (June, 1916).

A comparison of the accuracy of the Stanford and other revisions with borderline cases.

51. Treves and Saffiotti. "L'echelle metrique de l'intelligence modifiee selon la methode Treves-Saffiotti"; in Annee Psychologique (1912), pp. 327-40.

Criticize the age-grade method of measuring intelligence and propose a substitute.

52. Wallin, J. E. W. Experimental Studies of Mental Defectives. A Critique of the Binet-Simon Tests. Warwick & York. (1912.)

Criticism based on the use of the scale with epileptics.

53. Yerkes and Bridges. A Point Scale for Measuring Mental Ability. Warwick & York.

Authors think the point scale preferable to the Binet-Simon method.

BOOKS ON MENTAL DEFICIENCY

54. Binet, A., and Simon, Th. Mentally Defective Children. Translated from the French by W. B. Drummond. Longmans, Green & Co. (1914.) 171 pages.

Discusses the psychology, pedagogy, and medical examination of defectives.

55. Goddard, H. H. Feeble-Mindedness; Its Causes and Consequences. The Macmillan Company. (1913.) 599 pages.

The most important single volume on the subject. Extensive data on the causes of feeble-mindedness and excellent clinical pictures of all grades of mental defects.

56. Goddard, H. H. The Kallikak Family. The Macmillan Company. (1914.) 121 pages.

An epoch-making study of the hereditary transmission of mental deficiency in a degenerate family.

57. Holmes, Arthur. The Conservation of the Child. J. B. Lippincott Company. (1912.) 345 pages.

Methods of examination and treatment of defective children.

58. Holmes, Arthur. The Backward Child. Bobbs-Merrill Company. (1915.)

A popular treatment of the handling of backward children.

59. Huey, E. B. Backward and Feeble-Minded Children. Warwick & York. (1912.) 221 pages.

Devoted mainly to clinical accounts of borderline cases.

60. Lapage, C. P. Feeble-Mindedness in Children of School Age. The University Press, Manchester, England. (1911.) 359 pages.

61. Sherlock, E. B. The Feeble-Minded; A Guide to Study and Practice. The Macmillan Company. (1911.) 327 pages.

62. Tredgold, A. F. Mental Deficiency (Amentia). Bailliere, Tindall, and Cox. London, England. (2d ed. 1914.) 491 pages.

The best medical treatment of the subject.

STUDIES OF THE PROGRESS OF CHILDREN THROUGH THE GRADES

63. Ayres, Leonard P. Laggards in our Schools. The Russell Sage Foundation. (1909.) 236 pages.

Interesting and instructive discussion of school retardation and its causes.

64. Blan, Louis B. A Special Study of the Incidence of Retardation. Teachers College, Columbia University, Contributions to Education, no. 40. (1911.) 111 pages.

Review of the literature and a statistical study of the progress of 4579 children.

65. Keyes, C. H. Progress Through the Grades of City Schools. Teachers College, Columbia University, Contributions to Education, no. 42. (1911.) 79 pages.

Important study of the progress of several thousand children.

66. Strayer, George D. Age and Grade Census of Schools and Colleges. Bulletin no. 451, U.S. Bureau of Education. (1911.) 144 pages.

Statistics of the age-grade status of the children in 318 cities.

67. See also the Reports of leading school surveys, such as those of New York, Salt Lake City, Butte, Springfield (Mass.), Denver, Cleveland, etc.

REFERENCES ON THE SPECIAL CLASS FOR EXCEPTIONAL CHILDREN

68. Huey, E. B. "The Education of Defectives and the Training of Teachers for Special Classes"; in Journal of Educational Psychology (1913), pp. 545-50.

69. Goddard, H. H. School Training of Defective Children. World Book Company. (1914.) 97 pages.

Based on his survey of the treatment of backward children in the schools of New York City.

70. Holmes, W. H. School Organization and the Individual Child. The Davis Press, Worcester, Massachusetts. (1912.) 211 pages.

A comprehensive account of the efforts which have been made to adjust the school to the capacities of individual children.

71. Maennel, B. Auxiliary Education. Translated from the German by Emma Sylvester. Doubleday, Page & Co. (1909.) 267 pages.

72. Van Sickle, J. H., Witmer, L., and Ayres, L. P. Provision for Exceptional Children in Public Schools. Bulletin no. 461, U.S. Bureau of Education. (1911.) 92 pages.

73. Shaer, I. "Special Classes for Bright Children in an English Elementary School"; in Journal of Educational Psychology (1913), pp. 209-22.

74. Stern, W. "The Supernormal Child"; in Journal of Educational Psychology (1911), pp. 143-48 and 181-90.

A strong plea for special classes for superior children.

75. Vaney, V. Les classes pour enfants arrieres. Bulletin de la Societe libre pour l'etude psychologique de l'enfant (1911), pp. 53-152.

Report of the French National Commission appointed to investigate methods of treatment and training.

76. Witmer, L. The Special Class for Backward Children. The Psychological Clinic Press, Philadelphia. (1911.) 275 pages.

An account of the special class conducted in connection with the University of Pennsylvania Summer School.

LIST OF BINET'S MOST IMPORTANT CONTRIBUTIONS TO THE MEASUREMENT OF INTELLIGENCE

77. Binet, A. L'Etude experimentale de l'intelligence. Paris: Schleicher freres. (1903.)

78. Binet, A. "A Propos de la mesure de l'intelligence"; in Annee psychologique (1905), vol. 11, pp. 69-82.

79. Binet, A. Les enfants anormaux; guide pour l'admission des enfants anormaux dans les classes de perfectionnement. Paris: Colin (1907.)

80. Binet, A. Comment les instituteurs jugent-ils l'intelligence d'un ecolier? Bulletin de la Societe libre pour l'etude psychologique de l'enfant (1910), no. 10, pp. 172-82.

81. Binet, A. "Nouvelles recherches sur la mesure du niveau intellectuel chez les enfants d'ecole"; in Annee psychologique (1911), vol. 17, pp. 145-201.

82. Binet, A., et Simon, Th. "Sur la necessite d'etablir un diagnostique scientifique des etats inferieurs de l'intelligence"; in Annee psychologique (1905), vol. 11, pp. 163-90.

83. Binet, A., et Simon, Th. "Methodes nouvelles pour le diagnostique du niveau intellectuel des anormaux"; in Annee psychologique (1905), vol. 11, pp. 191-244.

84. Binet, A., et Simon, Th. "Application des Methodes nouvelles au diagnostique du niveau intellectuel chez des enfants normaux et anormaux d'hospice et d'ecole primaire"; in Annee psychologique (1905), vol. 11, pp. 245-336.