Evaluation

Whenever learning takes place, the result is a definable, observable, measurable change in behavior. The purpose of an evaluation is to determine how a student is progressing in the course. Evaluation is concerned with defining, observing, and measuring or judging this new behavior. Evaluation normally occurs before, during, and after instruction; it is an integral part of the learning process. During instruction, some sort of evaluation is essential to determine what the students are learning and how well they are learning it. The instructor's evaluation may be the result of observations of the students' overall performance, or it may be accomplished as either a spontaneous or planned evaluation, such as an oral quiz, written test, or skill performance test.

Oral Quizzes

The most used means of evaluation is the direct or indirect oral questioning of students by the instructor. Questions may be loosely classified as fact questions and thought questions. The answer to a fact question is based on memory or recall. This type of question usually concerns who, what, when, and where. Thought questions usually involve why or how, and require the student to combine knowledge of facts with an ability to analyze situations, solve problems, and arrive at conclusions. Proper quizzing by the instructor can have a number of desirable results.

Reveals the effectiveness of the instructor's training procedures.
Checks the student's retention of what has been learned.
Reviews material already covered by the student.
Can be used to retain the student's interest and stimulate thinking.
Emphasizes the important points of training.
Identifies points that need more emphasis.
Checks the student's comprehension of what has been learned.
Promotes active student participation, which is important to effective learning.

Characteristics of Effective Questions

An effective oral quiz requires some preparation. The instructor should devise and write pertinent questions in advance. One method is to place them in the lesson plan. Prepared questions merely serve as a framework, and as the lesson progresses, should be supplemented by such impromptu questions as the instructor considers appropriate. Usually an effective question has only one correct answer. This is always true of good questions of the objective type and generally will be true of all good questions, although the one correct answer to a thought question may sometimes be expressed in a variety of ways. To be effective, questions must apply to the subject of instruction. Unless the question pertains strictly to the particular training being conducted, it serves only to confuse the students and divert their thoughts to an unrelated subject. An effective question should be brief and concise, but also clear and definite. Enough words must be used to establish the conditions or significant circumstances exactly, so that instructor and students will have the same mental picture.

To be effective, questions must be adapted to the ability, experience, and stage of training of the students. Effective questions center on only one idea. A single question should be limited to who, what, when, where, how, or why, not a combination. Effective questions must present a challenge to the students. Questions of suitable difficulty serve to stimulate learning. Effective questions demand and deserve the use of proper English.

Types of Questions to Avoid

Asking, "Do you understand?" or "Do you have any questions?" has no place in effective quizzing. Assurance by the students that they do understand or that they have no questions provides no evidence of their comprehension, or that they even know the subject under discussion. Other typical types of questions that must be avoided are provided in the following list.

Puzzle- "What is the first action you should take if a conventional gear airplane with a weak right brake is swerving left in a right crosswind during a fullflap, power-on wheel landing?"
Oversize- "What do you do before beginning an engine overhaul?"
Toss-up- "In an emergency, should you squawk 7700 or pick a landing spot?"
Bewilderment- "In reading the altimeter-you know you set a sensitive altimeter for the nearest station pressure-if you take temperature into account, as when flying from a cold air mass through a warm front, what precaution should you take when in a mountainous area?"
Trick questions-These questions will cause the students to develop the feeling that they are engaged in a battle of wits with the instructor, and the whole significance of the subject of the instruction involved will be lost.

An example of a trick question would be where the alternatives are 1, 2, 3, and 4, but they are placed in the following form.

A. 4
B. 3
C. 2
D. 1

The only reason for reversing the order of choices is to trick the student to inadvertently answering incorrectly. Instructors often justify use of trick questions as testing for attention to detail. If attention to detail is an objective, detailed construction of alternatives is preferable to trick questions.
Irrelevant questions-The teaching process must be an orderly procedure of building one block of learning upon another in logical progression, until a desired goal is reached. Diversions, which introduce unrelated facts and thoughts, will only obscure this orderly process and slow the student's progress. Answers to unrelated questions are not helpful in evaluating the student's knowledge of the subject at hand. An example of an irrelevant question would be to ask a question about tire inflation during a test on the timing of magnetos.

Answering Questions from Students

Responses to student questions must also conform with certain considerations if answering is to be an effective teaching method. The question must be clearly understood by the instructor before an answer is attempted. The instructor should display interest in the student's question and frame an answer that is as direct and accurate as possible. After the instructor completes a response, it should be determined whether or not the student's reque st for information has been completely answered, and if the student is satisfied with the answer.

Sometimes it may be unwise to introduce the more complicated or advanced considerations necessary to completely answer a student's question at the current point in training. In this case, the instructor should carefully explain to the student that the question was good and pertinent, but that a detailed answer would, at this time, unnecessarily complicate the learning tasks. The instructor should advise the student to reintroduce the question later at the appropriate point in training, if it does not become resolved in the normal course of instruction.

Occasionally, a student asks a question that the instructor cannot answer. In such cases, the instructor should freely admit not knowing the answer, but should promise to get the answer or, if practicable, offer to help the student look it up in available references.

In all quizzing conducted as a portion of the instruction process, "yes" and "no" answers should be avoided. Questions should be framed so that the desired answers are specific and factual. Questions should also be constructed to avoid one-word answers, since such answers might be the product of a good guess and not be truly representative of student learning or ability. If a one-word answer is received, the instructor should follow up with additional questions to get a better idea of the student's comprehension of the material.

Written Tests

As evaluation devices, written tests are only as good as the knowledge and proficiency of the test writer. This section is intended to provide the aviation instructor with only the basic concepts of written test design. There are many excellent publications available to the aviation instructor on test administration, test scoring, grade assignment, whole test analysis, and test item analysis. Refer to the reference section at the end of this handbook for testing and test writing publications.

Characteristics of a Good Test

A test is a set of questions, problems, or exercises for determining whether a person has a particular knowledge or skill. A test can consist of just one test item, but it usually consists of a number of test items. A test item measures a single objective and calls for a single response. The test could be as simple as the correct answer to an essay question or as complex as completing a knowledge or practical test. Regardless of the underlying purpose, effective tests share certain characteristics.

Reliability is the degree to which test results are consistent with repeated measurements. If identical measurements are obtained every time a certain instrument is applied to a certain dimension, the instrument is considered reliable. An unreliable instrument cannot be depended upon to yield consistent results. An altimeter that has worn moving parts, a steel tape that expands and contracts with temperature changes, or cloth tapes that are affected by humidity cannot be expected to yield reliable measurements. While no instrument is perfectly reliable, it is obvious that some instruments are more reliable than others. For example, a laboratory balance is more reliable than a bathroom scale for measuring weight.

The reliability of an instrument can be estimated by numerous measurements of the same object. For example, a rough measure of the reliability of a thermometer can be obtained by taking several, consecutive readings of the temperature of a fluid held at a constant temperature. Except for the errors made by the person taking the readings, the difference between the highest and lowest readings can be considered a range of unreliability in the thermometer.

Reliability has the same meaning whether applied to written tests or to balances, thermometers, and altimeters. The reliability of a written test is judged by whether it gives consistent measurement to a particular individual or group. Measuring the reliability of a written test is, however, not as straightforward as it is for the measuring devices we have discussed. In an educational setting, knowledge, skills, and understanding do not remain constant. Students can be expected to improve their scores between attempts at taking the same test because the first test serves as a learning device. The student gains new knowledge and understanding. If a written test consistently rates the members of a group in a certain rank order, the reliability is probably acceptable, even though the scores of the students have increased overall.

Validity is the extent to which a test measures what it is supposed to measure. If a maintenance technician intends to measure the diameter of a bearing with a micrometer, it must be determined~,that the contacting surfaces of the bearing and the mwirometer are free of grease and dirt. Otherwise, the measurement will include the diameter of the bearing and the thickness of the extraneous matter, and it will be invalid.

A test used in educational evaluation follows the same principles of validity. Evaluations used in the classroom are valid only to the extent that they measure achievement of the objectives of instruction.

A rough estimate of the content validity of a classroom test may be obtained from the judgments of several competent instructors. To estimate validity, they should read the test critically and consider its content relative to the stated objectives of the instruction. Items that do not pertain directly to the objectives of the course should be modified or eliminated. Validity is the most important consideration in test evaluation. The instructor must carefully consider whether the test actually measures what it is supposed to measure.

Usability refers to the functionality of tests. A usable written test is easy to give if it is printed in a type size large enough for the students to read easily. The wording of both the directions for taking the test and of the test items themselves needs to be clear and concise. Graphics, charts, and illustrations, which are appropriate to the test items, must be clearly drawn, and the test should be easily graded.

Objectivity describes singleness of scoring of a test; it does not reflect the biases of the person grading the test. Later in the discussion, you will find that supplytype test items are very difficult to grade with complete objectivity. An example of this is essay questions. It is nearly impossible to prevent an instructor's own knowledge and experience in the subject area, writing style, or grammar from affecting the grade awarded. Selection-type test items, such as true-false or multiple-choice, are much easier to grade objectively.

Comprehensiveness is the degree to which a test measures the overall objectives. Suppose, for example, an aircraft maintenance technician wants to measure the compression of an aircraft engine. Measuring the compression on a single cylinder would not provide an indication of the entire engine. Only by measuring the compression of every cylinder would the test be comprehensive enough to indicate the compression condition of the engine.

In classroom evaluation, a test must sample an appropriate cross-section of the objectives of instruction. The comprehensiveness of a test is the degree to which the scope of the course objectives is tested. Sometimes it will not be possible to have test questions measuring all objectives of the course. At these times, the evaluation is but a sample of the entire course. Just as the owner of the wheat has to select samples of wheat from scattered positions in the car, the instructor has to make certain that the evaluation includes a representative and comprehensive sampling of the objectives of the course. In both instances, the evaluators must deliberately take comprehensive samples in order to realistically measure the overall achievement of the course objectives.

Discrimination is the degree to which a test distinguishes the difference between students. For example, a machinist wishes to measure six bearings that are slightly graduated in size. If a ruler is used to measure the diameters of the bearings, little difference will be found between the smallest bearing and the second smallest one. If the machinist compares the third bearing with the first bearing, slight differences in size might be detected, but the ruler could not be depended on for accurately assorting the six bearings. However, if the machinist measures with a micrometer, which can measure very fine graduations, the diameters of the first and second bearing, the second and third bearing, and so on, can be easily differentiated.

In classroom evaluation, a test must be able to measure small differences in achievement in relation to the objectives of the course. When a test is constructed to identify the difference in the achievement of students, it has three features.

There is a wide range of scores.
All levels of difficulty are included.
Each item distinguishes between the students who are low and those who are high in achievement of the course objectives.

Test Development

When testing aviation students, the instructor is usually concerned more with criterion-referenced testing than norm-referenced testing. Norm-referenced testing measures a student's performance against the performance of other students. Criterion-referenced testing evaluates each student's performance against a carefully written, measurable, standard or criterion. There is little or no concern about the student's performance in relation to the performance of other students. The FAA knowledge and practical tests for pilots and aircraft maintenance technicians are all criterion referenced because in aviation training, it is necessary to measure student performance agaitist a high standard of proficiency consistent with safety.

The aviation instructor constructs tests to measure progress toward the standards that will eventually be measured at the conclusion of the training. For example, during an early stage of flight training, the flight instructor must administer a presolo written exam to student pilots. Since tests are an integral part of the instructional process, it is important for the aviation instructor to be well informed about recommended testing procedures.

Aviation instructors can follow a four-step process when developing a test. This process is useful for tests that apply to the cognitive and affective domains of learning, and also can be used for skill testing in the psychomotor domain. The development process for criterion-referenced tests follows a general-to-specific pattern.

Determine Level-of-Learning Objectives

The first step in developing a test is to state the individual objectives as general, level-of-learning objectives. The objectives should measure one of the learning levels of the cognitive, affective, or psychornotor domains described in Chapter 1. The levels of cognitive learning include knowledge, comprehension, application, analysis, synthesis, and evaluation. For the comprehension or understanding level, an objective could be stated as, "Describe how to perform a compression test on an aircraft reciprocating engine." This objective requires a student to explain how to do a compression test, but not necessarily perform a compression test (application level). Further, the student would not be expected to compare the results of compression tests on different engines (analysis level), design a compression test for a different type of engine (synthesis or correlation level), or interpret the results of the compression test (evaluation level). A general level-of-learning objective is a good starting point for developing a test because it defines the scope of the learning task.

List Indicators/Samples of Desired Behavior

The second step is to list the indicators or samples of behavior that will give the best indication of the achievement of the objective. Some level-of-learning objectives often cannot be directly measured. As a result, behaviors that can be measured are selected in order to give the best evidence of learning. For example, if the instructor is expecting the student to display the comprehension level-of-learning on compression testing, some of the specific test question answers should describe appropriate tools and equipment, the proper equipment setup, appropriate safety procedures, and the steps used to obtain compression readings. The overall test must be comprehensive enough to give a true representation of the learning to be measured. It is not usually feasible to measure every aspect of a levelof-learning objective, but by carefully choosing samples of behavior, the instructor can obtain adequate evidence of learning.

Establish Criterion Objectives

The next step in the test development process is to define criterion (performance-based) objectives. In addition to the behavior expected, criterion objectives state the conditions under which the behavior is to be performed and the criteria that must be met. If the instructor developed performancebased objectives during the creation of lesson plans, criterion objectives have already been formulated. The criterion objective provides the framework for developing the test items used to measure the level-of-learning objectives. In the compression test example, a criterion objective to measure the comprehension level of learning might be stated as, "The student will demonstrate comprehension of compression test procedures for reciprocating aircraft engines by completing a quiz with a minimum passing score of 70%."

Develop Criterion-Referenced Test Items

The last step is to develop criterion-referenced test items. The actual development of the test questions is covered in the remainder of this chapter. While developing questions, the instructor should attempt to measure the behaviors described in the criterion objective(s). The questions in the exam for the compression test example should cover all of the areas necessary to give evidence of comprehending the procedure. The results of the test (questions missed) identify areas that were not adequately covered.

Performance-based objectives serve as a reference for the development of test items. If the test is the presolo knowledge test, the objectives are for the student to comprehend the regulations, the local area, the aircraft type, and the procedures to be used. The test should measure the student's knowledge in these specific areas. Individual instructors should develop their own tests to measure the progress of their students. If the test is to measure the readiness of a student to take a knowledge test, it should be based on the objectives of all the lessons the student has received.

Another source of test items includes FAA knowledge test guides for a particular knowledge test. These sample questions are designed to measure the level-of-leaming desired for pilots or aviation maintenance technicians. As a result, they are a good source of example questions to be used in measuring a student's preparedness to take the knowledge test.

However, care must be taken not to teach questions to ensure the student does not merely memorize answers or the letter of the answer. When using questions from any source, whether from a publisher or developed by individual instructors, periodically revising the questions used and changing the letters and positions of the answers will encourage learning the material rather than learning the test.

Written Test Items

Written questions include two general categories, the supply-type item and the selection-type item. Supplytype test items require the student to furnish a response in the form of a word, sentence, or paragraph. Selection-type test items require the student to select from two or more alternatives. See Appendix A for sample test items.

Supply Type

The supply-type item may be required where a selection-type cannot be devised to properly measure student knowledge. The supply-type requires the students to organize their knowledge. It demands an ability to express ideas that is not required for a selection-type item. This type item is valuable in measuring the students' generalized understanding of a subject.

On the other hand, a supply-type item may evaluate the students' ability to write rather than their specific knowledge of the subject matter. It places a premium on neatness and penmanship. The main disadvantage of supply-type tests is that they cannot be graded with uniformity. There is no assurance that the grade assigned is the grade deserved by the student. The same test graded by different instructors would probably be assigned different scores. Even the same test graded by the same instructor on consecutive days might be assigned altogether different scores. Still another disadvantage of a supply-type test is the time required by the student to complete it and the time required by the instructor to grade it. Everything considered, the disadvantages of the supply-type test appear to exceed the advantages to such an extent that instructors prefer to use the selection-type test. It should be noted that although selection-type tests are best in many cases, there are times where the supply-type is desirable. This would be when there is a need to thoroughly determine the knowledge of a person in a particular subject area. An example of this would be the presolo knowledge exam where it would be difficult to determine knowledge of procedures strictly with selection-type test items.

Selection Type

Written tests made up of selection-type items are highly objective. That is, the results of such a test would be graded the same regardless of the student taking the test or the person grading it. Tests that include only selection-type items make it possible to directly compare student accomplishment. For example, it is possible to compare the performance of students within one class to students in a different class, or students under one instructor with those under another instructor. By using selection-type items, the instructor can test on many more areas of knowledge in a given time than could be done by requiring the student to supply written responses. This increase in comprehensiveness can be expected to increase validity and discrimination. Another advantage is that selection-type tests are well adapted to statistical item analysis.

True-False

The true-false test item consists of a statement followed by an opportunity for the student to determine whether the statement is true or false. This itemtype, with all its variations, has a wide range of usage. It is well adapted for testing knowledge of facts and details, especially when there are only two possible answers. The chief disadvantage is that true-false questions create the greatest probability of guessing.

True-false test items are probably used and misused more than any other selection-type item. Frequently, instructors select sentences more or less at random from textual material and make half of them false by inserting negatives. When tests are constructed in this way, the principal attribute being measured is rote memory rather than knowledge of the subject. Such test construction has aroused antagonism toward selection tests in general and truefalse questions in particular. It has also decreased the validity of educational evaluations. Some of the principles that should be followed in the construction of true-false items are contained in the accompanying list.

Include only one idea in each statement.
Use original statements rather than verbatim text.
Statements should be entirely true or entirely false.
Avoid the unnecessary use of negatives. They tend to confuse the reader.
If negatives must be used, underline or otherwise emphasize the negative.
Avoid involved statements. Keep wording and sentence structure as simple as possible. Make statements both definite and clear.
Avoid the use of ambiguous words and terms (some, any, generally, most times, etc.)
Whenever possible, use terms which mean the same thing to all students.
Avoid absolutes (all, every, only, no, never, etc.) These words are known as determiners and provide clues to the correct answer. Since unequivocally true or false statements are rare, statements containing absolutes are usually false.
Avoid patterns in the sequence of correct responses because students can often identify the patterns. Instructors sometimes deliberately use patterns to make hand scoring easier. This is a poor practice.
Make statements brief and about the same length. Some instructors unconsciously make true statements longer than false ones. Students are quick to take advantage of this tendency.
If a statement is controversial (sources have differing information), the source of the statement should be listed.



Multiple Choice

A multiple-choice test item consists of two parts; the stem which includes the question, statement, or problem, and a list of alternatives or responses. Incorrect answers are called distractors. When properly devised and constructed, multiple-choice items offer several advantages that make this type more widely used and versatile than either the matching or the true-false items.

Multiple-choice test questions may be used to determine student achievement, ranging from acquisition of facts to understanding, reasoning, and ability to apply what has been learned. It is appropriate to use when the question, statement, or problem has the following characteristics.

Has a built-in and unique solution such as a specific application of laws or principles.
May be clearly limited by the wording of the item so that the student must choose the best of several offered solutions rather than a universal solution.
Is such that several options are plausible, or even scientifically accurate, but the student may be asked to identify the one most pertinent.
Has several pertinent solutions, and the student may be asked to identify the most appropriate solution.

Three major difficulties are common in the construction of multiple-choice test items. One is the development of a question or an item stem that must be expressed clearly and without ambiguity. Another requirement is that the statement of an answer or correct response cannot be refuted. Finally, the distractors must be written in such a way that they will be attractive to those students who do not possess the knowledge or understanding necessary to recognize the keyed response.

As mentioned previously, a multiple-choice item stem may take several basic forms.

It may be a direct question followed by several possible answers.
It may be an incomplete sentence followed by several possible phrases that complete the sentence.
It may be a stated problem based on an accompanying graph, diagram, or other artwork followed by the correct response and the distractors.

The student may be asked to select the one choice which is the correct answer or completion, the one choice that is an incorrect answer or completion, or the one choice which is best of the answers presented in the test item. Beginning test writers find it easier to write items in the question form. In general, the form with the options as answers to a question is preferable to the form that uses an incomplete statement as the stem. It is more easily phrased and is more natural for the student to read. Less likely to contain ambiguities, it usually results in more similarity between the options and gives fewer clues to the correct response. Samples of multiple-choice questions can be found in Appendix A.

When multiple-choice questions are used, three or four alternatives are generally provided. It is usually difficult to construct more than four convincing responses; that is, responses which appear to be correct to a person who has not mastered the subject matter.

Students are not supposed to guess the correct option; they should select an alternative only if they know it is correct. Therefore it is considered ethical to mislead the unsuccessful student into selecting an incorrect alternative. An effective and valid means of diverting the student from the correct response is to use common student errors as distractors. For example, if writing a question on the conversion of degrees Celsius to degrees Fahrenheit, providing alternatives derived by using incorrect formulas would be logical, since using the wrong formula is a common student error.

Items intended to measure the knowledge level of learning should have only one correct alternative; all other alternatives should be clearly incorrect. When items are to measure achievement at a higher level of learning, some or all of the alternatives should be acceptable responses-but one should be clearly better than the others. In either case, the instructions given should direct the student to select the best alternative. Some of the principles that should be followed in the construction of multiple-choice items are contained in the following list.

Make each item independent of every other item in the test. Do not permit one question to reveal, or depend on, the correct answer to another question. If items are to be interrelated, it becomes impossible to pinpoint specific deficiencies in either students or instructors.
Design questions that call for essential knowledge rather than for abstract background knowledge or unimportant facts.
State each question in language appropriate to the students. Failure to do so can result in decreased validity of the test, since the ability to understand the language will be measured as well as the subject-matter knowledge or achievement.
Include sketches, diagrams, or pictures when they can present a situation more vividly than words. They generally speed the testing process, add interest, and help to avoid reading difficulties and technical language. A common criticism of written tests is the reliance placed on the reading ability of the student. The validity of the examination may be decreased unless reading ability is an objective of the course or test.
When a negative is used, emphasize the negative word or phrase by underlining, bold facing, italicyzing, or printing in a different color. A student who is pressed for time may identify the wrong response simply because the negative form is overlooked. To whatever extent this occurs, the validity of the test is decreased.
Questions containing double negatives invariably cause confusion. If a word, such as "not" or "false ' " appears in the stem, avoid using another negative word in the stem or any of the responses.
Trick questions, unimportant details, ambiguities, and leading questions should be avoided, since they do not contribute to effective evaluation in any way. Instead, they tend to confuse and antagonize the student. Instructors often justify use of trick questions as testing for attention to detail. If attention to detail is an objective, detailed construction of alternatives is preferable to trick questions.

Stems

In preparing the stem of a multiple-choice item, the following general principles should be applied. These principles will help to ensure that the test item is valid.

The stem of the question should clearly present the central problem or idea. The function of the stem is to set the stage for the alternatives that follow.
The stem should contain only material relevant to its solution, unless the selection of what is relevant is part of the problem.
The stem should be worded in such a way that it does not give away the correct response. Avoid the use of determiners such as clue words or phrases.
Put everything that pertains to all alternatives in the stem of the item. This helps to avoid repetitious alternatives and saves time.
Generally avoid using "a" or "an" at the end of the stem. They may give away the correct choice. Every altemative should grammatically fit with the stem of the item.

Alternatives

The alternatives in a multiple-choice test item are as important as the stem. They should be formulated with care; simply being incorrect should not be the only criterion for the distracting alternatives. Some distractors which can be used are listed below.

An incorrect response which is related to the situation and which sounds convincing to the untutored.
A common misconception.
A statement which is true but does not satisfy the requirements of the problem.
A statement which is either too broad or too narrow for the requirements of the problem.

Research of instructor-made tests reveals that, in general, correct alternatives are longer than incorrect ones. When alternatives are numbers, they should generally be listed in ascending or descending order of magnitude or length.

Matching

A matching test item consists of two lists which may include a combination of words, terms, illustrations, phrases, or sentences. The student is asked to match alternatives in one list with related alternatives in a second list. In reality, matching exercises are a collection of related multiple-choice items. In a given period of time, more samples of a student's knowledge usually can be measured with matching rather than multiplechoice items. The matching item is particularly good for measuring a student's ability to recognize relationships and to make associations between terms, parts, words, phrases, clauses, or symbols listed in one column with related items in another column. Matching reduces the probability of guessing correct responses, especially if alternatives may be used more than once. The testing time can also be used more efficiently. Some of the principles that should be followed in the construction of matching items are included below.

Give specific and complete instructions. Do not make the student guess what is required.
Test only essential information; never test unimportant details.
Use closely related materials throughout an item. If students can divide the alternatives into distinct groups, the item is reduced to several multiplechoice items with few alternatives, and the possibility of guessing is distinctly increased.
Make all alternatives credible responses to each element in the first column, wherever possible, to minimize guessing by elimination.
Use language the student can understand. By reducing language barriers, both the validity and reliability of the test will be improved.
Arrange the alternatives in some sensible order. An alphabetical arrangement is common.

Matching-type test items are either equal column or unequal column. An equal column test item has the same number of alternatives in each column. When using this form, always provide for some items in the response column to be used more than once, or not at all, to preclude guessing by elimination. Unequal column type test items have more alternatives in the second column than in the first and are generally preferable to equal columns. Samples of the two forms of matching-item questions can be found in Appendix A.

Developing a Test Item Bank

Developing a test item bank is one of the instructor's most difficult tasks. Besides requiring considerable time and effort, this task demands a mastery of the subject, an ability to write clearly, and an ability to visualize realistic situations for use in developing problems. Because it is so difficult to develop good test items, a semipermanent record of items that have been developed is desirable. One way of preserving test items is to record the test item, along with the analysis of each question, on a set of cards. If questions are maintained on a computer, provisions could be made to include appropriate analysis gathered, thus creating a useful database. In either case, a pool of test questions is created after a large group of questions has been assembled. As long as precautions are taken to safeguard the security of items in the pool, the existence of the pool lightens the instructor's burden of continuously preparing new items.

Principles to Follow

Regardless of item type or form, the following principles should be followed in writing new items. The list also applies to reviewing and revising existing items.

Each item should test a concept or idea that is important for the student to know, understand, or be able to apply.
Each item must be stated so that everyone who is competent in the subject-matter area would agree on the correct response.
Each item should be stated in language the student will understand.
The wording of the item should be simple, direct, and free of ambiguity. The wording should be edited for brevity. Unnecessary words merely delay the student.
Sketches, diagrams, or pictures should be included when they are necessary for the student to visualize the problem correctly or when they will add realism.
Each item should present a problem that demands knowledge of the subject or course. No item that can be responded to solely on the basis of general knowledge should be included in an achievement test.

Presolo Knowledge Tests

Title 14 of the Code of Federal Regulations (14 CFR) part 61 requires the satisfactory completion of a presolo knowledge test prior to solo flight. The presolo knowledge test is required to be administered, graded, and all incorrect answers reviewed by the instructor providing the training prior to endorsing the student pilot certificate and logbook for solo flight. The regulation states that the presolo knowledge test must include questions applicable to 14 CFR parts 61 and 91 and on the flight characteristics and operational limitations of the make and model aircraft to be flown. This allows the flight instructor the flexibility to develop a presolo written test which not only evaluates the student's knowledge on general operating rules, but on the specific environment in which the student will be operating and on the particular make and model of aircraft to be flown.

The content and number of test questions are to be determined by the flight instructor. An adequate sampling of the general operating rules should be included. In addition, a sufficient number of specific questions should be asked to ensure the student has the knowledge to safely operate the aircraft in the local environment.

The regulation requires a presolo knowledge test for each make and model of aircraft to be soloed. Because of the varying complexity of aircraft and operating environments, the flight instructor will have to use good judgment in developing the test. For instance, a student who would be operating from a controlled airport located near a terminal control area or airport radar service area should have adequate knowledge to operate safely in the environment prior to solo. Likewise, a student operating from a high elevation airport might need emphasis placed on the effects of density altitude. Specific questions should be asked to fit the situation.

The specific procedures for developing test questions have been covered earlier in this chapter, but a review of some items as they apply to the presolo knowledge test are in order. Though selection-type test items are easier to grade, it is recommended that supply-type test items be used for the portions of the presolo knowledge test where specific knowledge is to be tested. One problem with supply-type test items is difficulty in assigning the appropriate grade. Since the purpose of this test is to determine if a student pilot is ready to solo, no specific grade is assigned. The purpose of the test is to determine fitness for solo and not to assign a grade relative to a student's peers. Since solo flight requires a thorough working knowledge of the different conditions likely to be encountered on the solo flight, it is important that the test properly evaluate this area. In this way, the instructor can see any areas that are not adequately understood and can then cover them in the review of the test. Selection-type test items do not allow the instructor to evaluate the student's knowledge beyond the immediate scope of the test items. An example of a supply-type test question would be to ask the student to, "Explain the procedures for entering the traffic pattern for Runway 26." The supply-type test item measures much more adequately the knowledge of the student, and lends itself very well to presolo testing.

Though supply-type test items allow broad questions to be asked, it is probably not possible to cover every conceivable circumstance to be encountered on a solo flight. The instructor must devise the test so the general operating rules are adequately sampled to ensure the overall objective of a safe solo flight is measured. The test also should ask a sufficient number of specific questions to determine that the student has the knowledge to safely operate the aircraft in the local area.

The instructor should keep a record of the test results for at least three (3) years. The record should at least include the date, name of the student, and the results of the test.

Performance Tests

The flight instructor does not administer the practical test for a pilot certificate, nor does the aviation maintenance instructor administer the oral and practical exam for certification as an aviation maintenance technician. Aviation instructors do get involved with the same skill or performance testing that is measured in these tests. Performance testing is desirable for evaluating training that involves an operation, a procedure, or a process. The job of the instructor is to prepare the student to take these tests. Therefore, each element of the practical test will have been evaluated prior to an applicant taking the practical exam.

Practical tests for maintenance technicians and pilots are criterion-referenced tests. The practical tests are criterion-referenced because the objective is for all successful applicants to meet the high standards of knowledge, skill, and safety required by the Federal Aviation Regulations.

The purpose of the practical test standards (PTS) is to delineate the standards by which FAA inspectors and designated pilot examiners conduct tests for ratings and certificates. The standards are in accordance with the requirements of 14 CFR parts 61, 91, and other FAA publications including the Aeronautical Information Manual and pertinent advisory circulars and handbooks. The objective of the PTS is to ensure the certification of pilots at a high level of performance and proficiency, consistent with safety.

The practical test standards for aeronautical certificates and ratings include AREAS OF OPERATION and TASKS that reflect the requirements of the FAA publications mentioned above. Areas of operation define phases of the practical test arranged in a logical sequence within each standard. They usually begin with Preflight Preparation and end with Postflight Procedures. Tasks are titles of knowledge areas, flight procedures, or maneuvers appropriate to an area of operation. Included are references to the applicable regulations or publications. Private pilot applicants are evaluated in all tasks of each area of operation. Flight instructor applicants are evaluated on one or more tasks in each area of operation. In addition, certain tasks are required to be covered and are identified by notes immediately following the area of operation titles.

An instructor is responsible for training the applicants to acceptable standards in all subject matter areas, procedures, and maneuvers included in the TASKS within each AREA OF OPERATION in the appropriate practical test standard. Because of the impact of their teaching activities in developing safe, proficient pilots, flight instructors should exhibit a high level of knowledge, skill, and the ability to impart that knowledge and skill to the students.

Since every task in the PTS may be covered on the check ride, the instructor must evaluate all of the tasks before certifying the applicant to take the practical test. While this evaluation will not be totally formal in nature, it should adhere to criterion -referenced testing. Practical test standards are available from several aviation publishers and are a good reference to use when preparing a student for the practical test. Although the instructor should always train the student to the very highest level possible, the evaluation of the student is only in relation to the standards listed in the PTS. The instructor, and the examiner, should also keep in mind that the standards are set at a level that is already very high. They are not minimum standards and they do not represent a floor of acceptability. In other words, the standards are the acceptable level that must be met and there are no requirements to exceed them.

Curriculum Overview
________________________________________
Page Contents
Curriculum
Curriculum & Instruction
Bases for Curriculum Planning
Curricula Criteria
How Values Influence Planning
Curriculum Foundation
Other Pages of Interest
Sources

________________________________________
Curriculum
The term curriculum is used in a number of different ways by parents, educators, and businesses. Some see curriculum as the "academic stuff that is done to children in school." Others view it as teacher directions and student activities that can be purchased from any number of curriculum publishers. Teachers themselves use the term in different ways depending on their views and needs. In any school staff room one may hear statements about curriculum such as the following:
"There's not enough time in the day to get through the curriculum!"
"The ____ reading curriculum's great but I hate that ____ math curriculum."
"I found this great curriculum website the other day that has all kinds of ideas for science lessons!"
"The kids are really making progress since I began modifying the curriculum to better meet their needs."
Webster's concisely defines curriculum as, "A course of study offered by a school" (Webster's II New Riverside Dictionary, 1984 p176). Curriculum is also often referred to as learning content, activities, and structures as experienced by students. Ronald C. Doll, in his book, Curriculum Improvement: Decision Making and Process, goes further, stating that:
The curriculum of a school is the formal and informal content and process by which learners gain knowledge and understanding, develop skills, and alter attitudes, appreciations, and values under he auspices of that school (Doll, 1996 p15).
It is this last definition that is perhaps the most useful to educators who wish to affect and improve student learning. Partially this is because it lacks the vagueness that many definitions have, and partially it is because curriculum, as Doll has defined it, can have outcomes that may be measured, allowing for the curriculum to be acted upon and improved.
back to top
The Relationship Between Curriculum and Instruction
Instruction is the creation and implementation of purposefully developed plans for the teaching of curriculum content. It is what teachers often concisely refer to as "planning" and "teaching." The relationship between curriculum and instruction is so intimate that "curriculumandinstruction" is frequently spoken as if it were one word (perhaps we should refer to it as "curstruction" or "instriculum"). With curriculum being the content of what is taught along with an overall process of how that content is to be taught, and instruction being the more detailed plans and the way those plans are implemented in order to teach the curriculum content, it becomes easy to understand that the two must be compatible in order to maximize student learning.
The case of multiage classrooms illustrates this close tie that exists between curriculum and instruction. Currently the most common classroom structure in American elementary schools is the single-grade classroom. This structure is meant to make instruction more efficient, allowing students of the same age to move through curriculum content at the same pace. In these classrooms the most prevalent teaching method is whole-class direct instruction. Because of the dominance of this structure nation-wide, commercially available curriculum and state learning standards are designed to be implemented in this type of learning environment. Some educators in their efforts to improve education have switched from a single-grade classroom structure to a multiage one. The multiage structure purposefully places students of different ages together in the same classroom while supporting an individualized continuous progress instructional model. While changing the structure of the classroom, multiage educators also change the instructional methods they use in order to better match the needs of their diverse group of students. They have found that "(c)urriculum designed for use in single-grade classrooms is not always adaptable to environments in which whole-class direct instruction is not the norm. Allowing for flexible groupings, academic diversity, and individual pacing are needs that are central to multiage practices" (Yates, Curriculum in Multiage Learning Environments, 2000). The instructional methods used by these teachers necessitate that curriculum be organized in a compatible manner.
back to top
The Bases for Curriculum Planning
When planning for curriculum improvement, two categories of bases should be understood, those that are institutional in nature and those that affect people directly. The institutional bases for curriculum planning include planning domains, the context or characteristics of the school situation, the impact of current trends and issues, and the use of strategic planning. Those bases of curriculum planning that affect people directly include student and teacher needs, local curriculum problems to be addressed, competencies of the planners, and pressures from inside and outside the school (Doll, 1996 p362-378). All of these bases affect the curriculum planning process in various ways and to differing degrees. They can also vary with each situation over time.
As of this writing, a current educational issue in the United States is that of student performance and preparation for the workplace. The trend is for state governments to create standards of competence that are tested at various points in students' educational careers and to make schools and students accountable for their performance on these tests. Test scores are frequently reported in the local media and this may lead to pressure from the local population being brought to bear on the school to improve its curricula. The context of the school may be that it is within a district that hasn't passed a school levy for a number of years and thus has not been able to budget money to work on improving the curricula during that time. This not-uncommon scenario shows how a combination of factors can become the bases for, and can influence the curriculum planning process.
back to top
Criteria to Plan, Develop, and Implement Curricula
Ronald Doll lists eleven principles of decision making and process as it relates to the evaluation of curricula and projects. These principles form the criteria of a quality curriculum development process that includes the stages of planning, development, and implementation.
Curriculum decisions should be made:
1. for valid educational reasons.
2. on the basis of the best available evidence.
3. in a context of broadly conceived aims of education
4. within a context of previously made decisions and of needs for additional decision making so that balance and other important curriculum considerations may be safeguarded.
5. by achieving a resolution of forces originating in the nature and development of learners, the nature of learning processes, demands of the society at large, requirements of the local community, and the nature and structure of subject matter to be learned.
6. cooperatively by persons who are legitimately involved in the effects of the decisions.
7. taking into account new facts of human life such as the proliferation of knowledge and a need for a new sense of unity within our diversity.
8. taking into account the many differences among learners.
9. with a realistic view of certain organizational or engineering matters that can affect the quality of the decisions themselves.
10. with some forethought about ways in which they may be communicated and shared.
11. only with reference to subject matter and pupil experiences that cannot be offered as satisfactorily outside the school (Doll, 1996 p293-296).
back to top
How Values Can Influence Curriculum Planning
Social forces that can influence curriculum planning come from far and wide. The ideas and values of various groups of people may include their social goals, ideas about cultural uniformity and diversity, social pressures, ideas about social change, their plans for the future, and their concepts of culture (Coutts, 1999). An example of this can be seen when contrasting the CYFERNet and the Catalyst: Voices of Chicago School Reform websites. The Children, Youth, and Families Education and Research Network website, CYFERNet is designed to provide "...program, evaluation and technology assistance for children, youth and family community-based programs". The website is a collaborative project that "...is funded (by) the U.S. Department of Agriculture's Cooperative State Research, Education, and Extension Service and the Cooperative Extension System" (Cooperative Extension System, 2000). Because of this, many of the curriculum links are to agricultural and 4-H educational activities. In contrast, the CATALYST: Voices of Chicago School Reform website sponsored by the Community Renewal Society, works "...to create racially and economically just communities" (Community Renewal Society, 2000). Its focus is on Chicago area urban educational issues, especially in regards to race and the economically disadvantaged. It provides information to help influence educational decision making as it relates to the organization's mission. On the one hand is a group that wishes to influence educational policy (and thus curriculum) to better meet the needs of children in an urban environment, and on the other a group trying to do the same for rural children. Although there may be a few communities where the two groups compete with one another, they do illustrate how the values and issues of various social groups can try to influence curriculum planning.
back to top
Curriculum Foundation
At the foundation to every curriculum, including the planning, design, and implementation stages, is the educational philosophy of those directly involved in the process. Often this can influence to a great extent the direction a school or school district takes with its curriculum and instruction. At the school district this writer has been employed with, the philosophy has allowed for a diversity of instructional styles as a way of meeting a diversity of children's learning styles. This has led in the elementary school to several educational options available for students and parents: single-grade, single-grade clusters, multiage, looping, and home-school hybrid educational environments. Because some of these educational structures have different instructional designs than others, there are available different curricular materials. Other nearby schools offer only a single choice and a single curriculum. The basis for these decisions can be found in the above mentioned factors as well as in the educational philosophies of the decision makers.
back to top
Other Pages of Interest
• To read about the educational philosophy of the writer, please follow this link to My Teaching Philosophy.
• To read a paper written by this author that discusses in more detail how the philosophical views of curriculum committee members can influence a curriculum decision, follow this link to Position Paper on Curriculum Priorities.
back to top
Sources:
All sources used in creating this website are cited on the Bibliography and Sources webpage. Also note that in-text citations used above are linked directly to the appropriate portion of the Bibliography and Sources webpage.
My Teaching Philosophy
Russell Yates
I am currently the teacher of children age 8 to 11 years old in an elementary multiage learning environment. Two principles are at the heart of all that I do as a teacher. First, I believe that education should be student-centered and second, that fostering an enjoyment of learning is probably the most important thing we can do with our students.
Learning and not teaching is the focus in my classroom. This simple statement is filled with implications. If teaching is the focus of a classroom, then control over student academic, social, and behavioral actions becomes the job of the teacher. Management of the details of individual students, of the classroom, of academic progress and pace, and even of recess behavior become the primary concern. An information delivery model in which knowledge is passed from the textbook company or state department of education through the teacher to the student would be the major route for knowledge acquisition in a teacher-centered classroom. This type of classroom supports a "culture of silence" as Paulo Freire has observed, a programming of conformity if looked at on a societal scale. It is also, as I see it, a Behaviorist way of managing instruction in which management of learning is reliant on forces external to the child, including reinforcement and an emphasis on consequences administered by the teacher.
In contrast, a learning focused classroom, as I see it, puts the primary emphasis on the process of learning and the use of knowledge. The teacher's job is then one of helping students acquire learning skills and the practicing of those skills in ways that are useful and meaningful to the student. The teacher does not act as some sort of all-important filter of information determining all of what each student needs to know, but rather becomes more of a guide, setting up learning and practice situations and showing or modeling new ways of acquiring and using information. For instance, I use many ideas from Gestalt Psychology when helping my students in math. My instruction is based on a problem-solving model in which I guide my students to find a variety of solutions to various problems, then set up experiences for them to transfer their new skills to similar but different situations. I also use Vygotsky's concepts of the zone of proximal development, scaffolding, and social interaction in my classroom. By using peer tutoring and modeling thoughtfully along with my own support of the individual student, children frequently show tremendous cognitive growth. Thus I believe that the learning environment is not limited to the classroom and to what I can always control but rather is made up of the tools of learning, including social interaction.
In looking at how I came to believe such a classroom is most conducive to student learning, it is important to first visit some of my past classrooms.
In the first 3 years of my teaching career I was learning how to survive each day. My task was to get through the curriculum while still keeping school as enjoyable for the students as possible. Thus I concentrated on orchestrating a classroom full of students, keeping them "on task," rewarding good academic and social behavior with material and nonmaterial rewards, and including negative consequences for "poor student choices." Overall students and their parents responded well to this. Part of this orchestration was to present instruction in a variety of teacher-determined ways, including the occasional learning project. I created many "systems" in the classroom for everything from independent reading to small group management. Although I did this with input from students (I've been known as a very democratic teacher), I still put an emphasis on my being in control of each and every situation. I believe that nearly every conflict or negative situation could be traced to my overemphasis on teacher control, on being the giver of consequences.
In the 1993-1994 school year I was fortunate enough to get a group of students that would push me and teach me about what they needed and wanted for their education. The student labels ranged from "severely gifted" to "How do you deal with him every day?" As the months passed I found it increasingly harder to always be in control of the students. Projects would work well, whether individual or group, but anything that had even a smattering of more traditional whole group instruction would be met by resistance and/or conflict. Students would even divide themselves into factions, those who sided with the teacher, those who wouldn't, and a few who would articulate a third or fourth stance. A number of students could not focus on instruction long enough to be able to successfully complete many or most individual learning tasks. For these students I would regularly give them individual instruction immediately after I had given whole class instruction, helping them get successfully started on the practice task. Others would be finished with the practice task at about the same time I completed the whole class instruction. They "got it" right away and began working early even when they were told not to (waiting didn't make sense to them, and they would usually complete the task with a high degree of accuracy). For some of the students, a great number of the "learning tasks" were just busy work, they already knew how to "do it" successfully. As you can probably tell from this brief description, I was teaching most often to the middle ground (and sometimes to the lower middle). This seemed to work best for me, as most students seemed to stay "on task." But I had to ask myself, "Am I really meeting the needs of all my students?"
After having survived that year, I reflected quite a bit on my beliefs about teaching and learning. I came to realize that children are complex and that they need to have some control over what and when they learn, that they are, and need to be, an active participant in the learning process. It finally occurred to me that I had been keeping the focus of my classroom on myself, it was a teacher-centered environment and that in order for students to really learn and to enjoy school, I had to refocus on the children, to make my classroom a child-centered one.
At about this time I discovered the ideas surrounding multiage educational practices. For me the two mixed quite well. My needs to change the underlying beliefs and structure of my classroom were a near perfect match with the beliefs and structure of multiage educational environments (see appendix for a short list of multiage practices).
In addition to "student-centeredness," I also believe that school should be enjoyable. I realize that not all learning tasks are going to be viewed as fun for all students, but that if school in general is considered fun, then learning is fun and the motivation to learn more is enhanced. I believe that engendering a love of learning is probably one of the most important things I can help elementary age children achieve. With that, every other learning task comes relatively easily to a student. As John Dewey stated in Democracy and Education, "Study of mental life has made evident the fundamental worth of native tendencies to explore, to manipulate tools and materials, to construct, to give expression to joyous emotion, etc. When exercises which are promoted by these instincts are a part of the regular school program, the whole pupil is engaged, the artificial gap between life in school and out is reduced, motives are afforded for attention to a large variety of materials and processes distinctly educative in effect play and work correspond, point for point, with the traits of the initial stage of knowing" (p. 195)
As a result of my classroom being both student-centered and a place where learning is enjoyable, the educational opportunities I give my students are varied. Learners and the learning process are very complex. I believe that there is no one formula for teaching that fits all situations or all students. For this reason I believe that a variety of approaches thoughtfully geared to both the content to be learned and individual students is the best way to structure a learning program. This matches Howard Gardner's Multiple Intelligences Theory, there are many different ways knowledge can be gained and learning can be assessed. Hands on learning activities will work well with many students and with a number of content areas. Research projects jig-sawed among the class members facilitate learning best for some students and some subject areas. A discovery approach in which the tools for a learning task are given to the students, a simple question is asked, and the procedure is left up to the students to discover works well in other situations and with some students. Again, variety of learning opportunities allows for the most number of children to both enjoy learning and, if choice is incorporated into the activity, to give children some control over their education.
I believe that structuring classroom management to allow for a wide diversity of children and for shared student control of the classroom environment is at least equally important as academic goals. The degree to which a classroom is student-centered or teacher-centered shows up in its management structure. During parts of each day I have students take over various management tasks. Students manage the opening portion of each day (attendance, reading the schedule, discussing the weather forecast). Students also run a discussion/transition time in which we discuss the various books we are reading. These are things in which I can monitor but that don't need to be done by the teacher. Students in my classroom also have the privilege of choosing their seat. It is not uncommon for a child to be seated at four or more different places in the classroom during the day. They are given the responsibility to choose a place where they can best learn and where they are most comfortable. What a difference from when I assigned seats and decided whom would sit where based on my needs and my perception of each student's needs. By giving students some control over their school lives, I am supporting a belief within the students that they, and not some external force, has control of their behavior and their learning. That through their own effort they can achieve success. This matches the idea from Attribution Theory that learning is acquired through constructive effort.
As you can see, the two principals of student-centeredness and enjoyment of the learning process underlie all aspects of my classroom and are two cornerstones of my philosophy of teaching.
Appendix - Multiage Education
The term "developmentally appropriate" is used a lot by multiage educators. The idea, from the work by Piaget, that certain educational practices are appropriate for children of certain age spans but not for those of different age spans, allows the educator to design lessons that will match the individual child. This becomes important in a classroom of children of diverse ages. The interesting thing about Piaget's developmental levels and multiage education is how the idea is used to determine the age spans appropriate to be placed in a single classroom, and how multiage teachers use the diversity to help children transfer from one cognitive level to another. Multiage programs are most often considered "primary" or "intermediate," roughly matching Piaget's Concrete Operational Level, divided in two. This recognizes that some children transition from one level to another at different times and that there is a substantial qualitative difference between cognition at the Preoperational Level and the Formal Operational Level, the levels bordering the Concrete Operational Level. The transition from one developmental level to another is supported through peer interaction and modeling and takes into account the four factors that allow movement from stage to stage, maturation, experience, social interaction, and equilibration. The heavy use of peer modeling by multiage educators is supported by Albert Bandura's Social-Cognitive Learning Theory. By using these "live models," the student in a multiage classroom learns vicariously through the success of the model. In this way instruction is enhanced and learning is engendered through more than simple reliance on the teacher as model.
The following is from a brochure I have produced that explains to parents what underlying beliefs I have and some of the general methods I use in the classroom.
A multiage educational program is a union of an organizational structure and unique combinations of teaching and learning strategies. The way learning occurs is made possible by the multiple age structure.
Why Multiage?
• Allows for flexibility in the grouping of children according to need, ability, or interest; not just by age.
• Problems associated with a yearly transition from one grade to another can be overcome. The teacher has a nucleus of children; trained in the details of the class organization who keep it going while newcomers absorb it.
• As the student-teacher-parent relationship develops over a longer period of time, students will receive greater support for their success in school.
• A more natural learning situation is established. Children work at their own pace. Their program is not geared to the work of a single year but can be adjusted over two or more years.
• Benefits come to the older children from the quality of leadership and responsibility they develop.
• Younger children are stimulated intellectually by older children.
• Children have a broader social experience with increased opportunities to lead and to follow, to collaborate and to make stable peer relationships.
It is my goal to use instructional strategies that:
• Expand the teacher's role to include that of a facilitator as well as a source of knowledge.
• Produce cooperation.
• Allow students to learn from each other through peer tutoring.
• Give students responsibility and independence in both learning and behavior.
• Build understanding of action-consequence relationship.
• Provide choice to students in different areas of learning that will reflect learning-style differences.
• Allow continuous learning through the use of learning centers, group instruction, projects, and individual pacing.
• Involve parents in classroom activities.
• Encourage student responsibility and ownership of the learning environment.
• Teach goal setting from an early age.
• Build leadership skills in all students.
I have a website that explains to interested educators the "whats" and "hows" of multiage education. Please visit it if you are interested in more of the details of how I match my philosophy of learning and teaching to the nuts and bolts of everyday life with a classroom of 3rd, 4th, and 5th graders. Its URL is: http://www.multiage-education.com.

Some preliminary question evaluation

1.how do we know that the students have learned what we intended or hoped they would?
2


Purposes for EVALUATION OF LEARNING

1.FEEDBACK TO STUDENTS
2. FEEDBACK TO TEACHERS
3. FEEDBACK TO PARENTS
4. INFORMATION FOR SELECTION AND CERTIFICATION
5. INFORMATION FOR ACCOUNTABILITY
6. INFORMATION TO INCREASE STUDENTS EFFORT

WHAT TOOLS DO WE HAVE?

1. Selected-response-format ( multiple choice, true-false) test and quizzes
2. written responses to academic prompts ( short answer format)
3. extended written products ( essays, papers, lab reports)
4. Visual products (PowerPoint, Mural)
5. Oral performances ( Oral report, world language dialogue, debate)
6. Students demonstrations ( athletic skills, music performance, role plays)
7. long term authentic assessment projects ( exhibits)
8. Portfolios
9. Reflective journals or learning logs
10. Informal observations of students


Assessment of learning
Start with what you want students to know and to be able to do at the eng of the lesson.
These are the learning or instructional objectives, and the assessment needs to be directly linked to them.

Knowledge and Skills

What we want students to know:

Vocabulary,Terminology, Definations
Key dactual information
Critical details
Sequences
Concepts



What do we want students to be able to do:

Basic skills – math, reading
Communication skills – listening, speaking, writing
Thinking skills- compare, infer, analyze, interpret
Research, investigation skills
Study skills
Interpersonal and group skills

Assessment of learning

Assessment and evaluation of learning is a snapshot of what is learned ( except alphabet and math facts) – there is no way to fully represent or measure all the learning that has occurred. So, of all the things learned, what do we want to demonstrate to selves students, parents was the outcome.

Assessment of learing : Prior to Implementation

Need to start with identification of what students already know. What are their existing understandings, skills, interests?

Can ask, “ why do you think this is important to know?” “ Why go you think we are studying this?” “ What do you already know about this?”

Diagnostic Assessment

Diagnostic assessment precedes instruction, and identifies:

1. students’ prior knowledge
2. misconceptions
3. interests
4. learning-style preferences
examples: pre-test, student surveys, skills check

Formative assessment

Formative assessment is ongoing, informal checks for understanding during the curriculum implementation, and provides:

1. Information to guide teaching
2. feedback for pacing
3. Information about how to improve student performance




Summative Assessment


Summative assessment follows instruction, and identifies:

1. Students’ level of mastery
2. student proficiency

examples: post-tests, performance task, culminating project, portfolio


Effective Assessment

1. Performance goals and standards are clear
2. diagnostic assessments check for prior knowledge, skill level, and misconceptions
3. students demonstrate their understanding though real-world applications
4. Assessment methods are matched to achievement targets.
5. Assessment is ongoing and timely
6. Learners have opportunities for trial and error, reflection and revision
7. self-assessment is expected.