The Baseline Survey is the first step in the project. A Baseline Survey gathers key information early in a project so that later judgments can be made about the quality and development results achieved of the project.
The project’s monitoring and evaluation plan is closely linked to each (objective) level of the log frame and includes indicators of achievement and means of verification. The Baseline Survey is an early element in the monitoring and evaluation plan and uses the log frame structure to systematically assess the circumstances in which the project commences.It provides the basis for subsequent assessment of how efficiently the activity is being implemented and the eventual results achieved. Subsequent monitoring of project progress also gathers and analyses data using the log frame and will be consistent with, but not repeat, the Baseline Survey. Mid-term reviews, project Completion reports and other evaluations will judge progress largely based on comparisons with the information from the Baseline Survey.A development activity entails change, so a good monitoring and evaluation system will: show whether change is occurring; indicate the results of the activity, including eventual impacts, whether these changes are intended or not intended, direct or indirect, positive or negative, primary or secondary; and suggest how to improve the efficiency of implementation, the extent of the desired results achieved and their sustainability.
The first stage in building an evaluation system typically involves design, execution and analysis of the baseline studies in order to establish the frame of reference for subsequent comparisons on which evaluation will be based. Since for these comparative purposes the data to be collected subsequently must be similar to those collected in the baseline studies, the methods of selecting and conducting these baseline studies and their content are extremely important.In effect, the principal conceptual work for the evaluation of a programme must occur at this stage, since the nature of the entire monitoring and evaluation system will be determined here. Moreover, the largest volume of data that is collected at any one time is obtained in the baseline studies. This stage will therefore involve the largest number of personnel and require the greatest amount of time.
As a result, it is the most costly stage in the design and implementation of the system.This cost factor led to the decision to make a double use of the baseline studies in each of the four cases under review here. On the one hand, data would be used for subsequent comparisons; on the other, data would be used. for a diagnosis of the existing situation of potential beneficiaries of the programmes for planning and programming purposes. In several cases in which programmes were already under way, the baseline data would affect implementation rather than formulation of the programme plan.A good baseline survey involves a careful specification of exactly what and how data is to be collected (through qualitative research), and a sampling frame representative of the project population (maybe with concentration on groups of particular interest within the population).
It ensures that survey information is accurate, that information collected is accurately and efficiently entered, and that final databases are user-friendly.Baseline surveys can take many forms, but their common characteristic is that they are conducted at the start or initial period of a project to: provide the information set for subsequently determining project impact, produce indicators for project monitoring and evaluation and form the basis for identifying specific project activities. The sampling frame should, budget permitting, include a treatment (in-project) and control (similar but outside-project) group.These two groups will constitute comparative points of reference from which the counterfactual question will be answered: in the absence of the project, what would have happened? Projects are designed to deliver measurable results, and a good baseline survey must answer the counterfactual so as to measure and distinguish those results from other variables.
Therefore conducting base line survey before a particular project has commenced has the following importance; It helps to determining what variables to measure, i. e. specifying the substantive content of the study. In academic research this usually means elaborating a set of hypotheses. By contrast, for the type of research necessary for a monitoring and evaluation system, the step consists in determining what information policy makers, programme planners and administrators require in order to ascertain whether or not the programme is functioning properly and why this is so. This step is perhaps the most difficult, since programmes do not always specify their objectives clearly and in measurable terms.
Indeed, many of the objectives of a given programme are not even stated formally. In addition, although most development programmes are multidisciplinary, the programme personnel often exhibit a particular professional bias towards obtaining one or another type of information deemed necessary for monitoring and evaluation. Thus economists may tend to be interested only in economic data, while sociologists may be interested only in social data. It helps in determination of the system structure.
In the case study approach, once the information to be obtained is known, the next step is to determine the number of community case studies which will comprise the monitoring and evaluation system. In the same way that the determination of variables is critical for the definition of the content of a system, so the number of cases is critical for the definition of the structure of a system. This number is determined by two distinct factors: methodological considerations of sample representativeness and resources available to implement the baseline case studies.Perfect representativeness would be guaranteed only if every community included in a programme is studied.
Statistical sampling basically seeks to select a sample of cases in such a way that it is possible to guarantee in advance that the sample will be representative of the entire population. The rule in sampling is that the degree to which representativeness can be guaranteed depends on the absolute number of cases in the sample A general rule of thumb is that the larger the number of cases in a sample, the greater the guaranteed representativeness.However, this general rule is conditioned by two other factors. First, the degree of representativeness of a sample does not increase proportionately as the size of a sample increases. That is, in an infinite population, assuming that 95 times out of 100 the sample accurately represents the population, a simple random sample of 196 yields a 7 per cent probable error, a sample of 384 yields a 5 per cent probable error, a sample of 600 yields a 14 per cent probable error, a sample of 1,061 yields a 3 per cent probable error and only with 9,604 cases can one achieve a 1 per cent probable error (AuSAID 2003 chap. ).
Thus, as sample size increases, each additional case contributes proportionately less to the reduction of probable error. There is, of course, a minimum sample size required for any given level of data precision. It helps to determine a sample of case studies. Having decided on the number of cases to be studied, based on the time and personnel available, it is necessary to determine how to draw the sample of cases. As has been noted earlier, representativeness of a sample depends on two factors: the absolute number of cases in the sample and the method by which the sample is drawn.
Since , time and personnel constraints limited the size of the sample, the method of drawing cases was the key factor. The first characteristic of the sampling method was that it should be random. In theory, this means that the selection process must contain at least one completely chance element not dependent on human judgement, which can be biased. In practice, this means that at some time during the procedure, usually at the point of selecting the specific case, a chance factor must be introduced. Without it, there is no possibility of guaranteeing that the sample will be free of bias.A second consideration in selecting the sampling method is that the more that is known about a population prior to drawing the sample, the fewer cases will be needed to achieve acceptable representativeness.
The logic behind this can be seen from the following example. Suppose it is known in advance that some communities in the programme area raise only cattle, some produce only cotton and some only corn, and that ethnic background, cultural patterns and types of local organization are strongly related to type of production. It is important that each type of community be represented in the sample.If the sample is drawn randomly without any further classification (a simple random sample), a rather large number of cases would be needed to ensure that the sample distribution is the same as the universe. If only a few cases were drawn, it is quite possible that none of the extreme cases would be represented or that one or another type of community would be overrepresented, thereby biasing the sample.
This could be avoided if a large number of cases are drawn, since, with a large number, it is likely that a few extreme cases would be included and that all types of communities would be represented more or less in proper proportion.On the other hand, if an estimate of the distribution of community types were known and the total set of communities could be classified in advance, therefore it is clear that fewer cases would be needed. By drawing cases according to community type, each type is represented in the sample in proportion to its existence in the population. If, for instance, 20 per cent of the total numbers of communities in the programme area depend on cattle raising, the same proportion of communities in the sample can be drawn from among cattle raising communities.In this way, it is possible to ensure in advance that the distribution of cases in the sample will approximate the distribution of cases in the population, according to the production variable. It helps to determining the number of individual interviews per community.
Much of the data to be obtained in each community has to be collected at the individual level. However, individual interviews with all families in each selected case are usually not possible. A sample of families was therefore required. The question was how many families to interview in each community.There are basically two methods of determining the number of interviews in each case: either a constant proportion of families (for example, 20 per cent) could be sampled in each case, or a fixed number (for example, 25 families) could be sampled. The decision on which procedure to use depended on whether individual data, as well as community data, would be generalized for the entire programme area, that is, whether it would be necessary to make such generalizations as "X per cent of the families living in the programme area have characteristic A".
AuSAID 2003 chap. 4).The actual sampling procedure in each case consisted of first obtaining a list of all families living in the community, either by means of official records, interviews with local leaders or conducting a census survey. Then, according to predetermined stratification criteria, each family is to be classified in a category and the required number of interviews will be selected from each category in proportion to the weight of each category in the same way as community cases were selected.For example, if 20 per cent of the families occupied the "landless labourer" category, 20 per cent of the sample would be drawn from that category. It helps in designing the questionnaire.
Converting information needs into measurement instruments constitutes the final, crucial conceptual step in developing a monitoring and evaluation system. Two overriding considerations influence this step: information must be collected as efficiently as possible, and measurements must have unambiguous interpretation.Questionnaires for monitoring and evaluation systems have by necessity rather broad coverage in terms of content. In designing them it is necessary to limit the number of questions for each type of subject to a maximum of one or two questions. To ensure that these few questions are valid, they should be defined almost exclusively in terms of behavior. Thus, for example, in obtaining information about nutrition, questions should be restricted to what people actually eat rather than to such othor ccrsieraticns as their attitude towards different foods.
This approach simplifies questionnaire construction in that all questions are straightforward and thus have a simple interpretation at the time of analysis. After determining this master list of questionnaire variables, derived from the list of information needed, the next step is to draft the specific questionnaires. These normally included one for interviews with families, another for interviews and observations at the community level and a third for the other levels. Since the greatest care must be exercised in drafting the individual questionnaires.
The procedure is to draft questions in working groups, and then submit them to a critical staff discussion to ensure that the questions were adequate for the specific measurement task. It helps in designing field work and pre tabulation of data. Since the base line studies consisted of a series of case studies, the model for field work procedures is not unlike that usually used in an academic case study. In each sampled community, the studies are made by a team utilizing the combination of survey, depth interviews and participant observation previously mentioned.The objective of the team is to obtain data in such a way that it is possible to understand in a total sense the dynamics of the communities being studied. In order to achieve this, it decided in all relevant areas where the tabulation and much of the analysis of the data would be made by the team which studied the community.
In academic research, it is not normal practice to delegate the task of tabulation and analysis to the interviewers because it is considered that they will code subjectively, make tabulation errors or be unable to analyse the results adequately.
