As numerous studies reveal, there is no single scientist c methodology that is exercised in science or in any other research practice. Rather, research methodologies are socially constructed. Since some constructions are better than others for did rent purposes, it becomes valuable to study did rent methodologies and their in ounce on research practice and results. Proposals for such studies are o red. 1 The state of design research methodology In many disciplines, research methodology Is seldom discussed by researchers. Such neglect may result from several attitudes towards research methodology Including
India Renee or Ignorance. Researchers may be India rent because their research Is well received by the community therefore they need not change or worry about it: or researchers may perceive their practice as science and wish to adopt as their methodology what they perceive to be the methodology used by scientists, henceforth referred to as the received scientist c methodology. Roughly, the received scientist c methodology consists of several steps: (1) observations or preliminary studies, (2) hypothesis formation, (3) hypothesis testing, (4) hypothesis evaluation, and (5) hypothesis acceptance or rejection.
It is asserted that results of research discovered by this methodology lead to applied research and subsequently, to practical Impact. In contrast to this assertion, It Is proclaimed that the goal of this methodology Is to advance knowledge for Its own sake and not address practical needs nor be responsible for delivering practical results. Most researchers would rarely question this methodology, but since It Is Impossible to follow or even hard to approximate, researchers who would claim to have adopted it, would not practice it. Details Retch (1995) of, and the controversies about, the received scientist c methodology. They are unaware of the alternatives of this methodology that we brie y mention later, their practice, and consequences. In fact, most researchers interpret methodology as a fancy synonym for method, while methodology is (or attempts to approximate) a compatible collection of assumptions and goals underlying methods, the methods, and the way the results of carrying the methods out are interpreted and evaluated.
The ability to validate the attainment of research assumptions and goals through the evaluations is a critical factor in making the above collection compatible. The did Renee in meanings assigned to the term methodology can be illustrated through an example from structural optimization. One research method of structural design involves the development of optimization procedures and their testing on benchmark problems. Most researchers will call this method
esearch methodology. " However, the assumptions underlying such work (e. G. , that optimization is a good model of structural design) and its testing (e. . , that simple benchmark problems are representatives of the complex structural designs performed by designers), or the believe that such research advances practice (e. G. , hat designers use optimization programs developed in research and that designers' practice been TTS from them), are rarely articulated thus rarely validated. If these issues would be addressed, the conclusions would probably contradict those implicit assumptions. First, independent of any discipline, optimization is a very restricted view of design (even with respect to Simony's (1981) restricted view).
Second, results obtained on simple benchmark problems do not necessarily transfer to real design problems nor do they re etc performance on other benchmark problems (Haft and Spikes, 1992); simple benchmark comparisons provide little understanding of the relative merit of did rent optimization procedures (Burns, 1989). Third, practitioners are very reluctant to use optimization procedures (Dolman, 1992; Haft and Spikes, 1992). This reluctance contradicts the implicit or stated research goals of improving structural design practice.
India Renee or ignorance towards research methodology relieve researchers from addressing such contradictions or exercising informed choices between methodologies in their research. Many researchers simply follow the method of their close senior peers without questioning or even knowing the assumptions that underlie it. In most cases, only the method I the actual research activity I is transferred to research apprentices. Thus, driven by social proximity, research assumptions become part of the implicit unarticulated research culture. Infrequently, this state of a airs had called the attention of researchers.
In 1987, two representative papers critical of the state of design research practice were published, one by Annotations (1987) and the other by Dixon (1987). Both papers advocated adopting the scientist c methodology in design research either for improving research quality or for improving design practice. These and other related publication, the state of design research methodology has remained virtually unchanged. Such reaction raises at least two questions: what may have caused this response and if this is an expected reaction, is the state of research methodology worth additional discussions?
Two plausible answers that originate from two did rent interpretations of Dixon and Annotation's papers Justify further discussions. First, Dixon and Annotation's positions may have been interpreted as criticizing the intellectual De science of research and demanding from researchers to exercise a theology did rent from the one they actually use and one that requires additional e rotor. In particular, the methodology Transactions of the SAME, Journal of Mechanical Design, 1995, in press 2 proposed demanded researchers to seriously test their hypotheses. It might have been expected that such requests would be opposed to or, worst, be ignored.
Second, researchers who are familiar with current views in the philosophy of science may have treated Dixon or Annotation's positions as being too simple De if they interpreted these positions as advocating for the received scientist c view. Since the stated goal of science is creating knowledge for the sake of knowing, but not necessarily knowledge that is relevant to practice, the received scientist c methodology may hinder improving practice by detaching the products of research (I. E. , design theories) from actual practice (Arises, 1980; Reich, 1992).
According to this interpretation and its limitation, previous calls for improving research methodology could not have impacted design practice even if researchers had adopted them. If design practice is indeed a goal of design research, did rent methodologies may be needed to establish a connection teen research and practice (Reich et al, 1992; Reich, AAA; Reich, Bibb). These methodologies can evolve in various ways including studying researchers' activities and the way these activities correlate with research progress, thereby identifying the relationships between did rent assumptions, methods, and consequences.
I have no intention to select between these two interpretations or to develop others but to explain how to improve research practice without assuming a axed methodology. To start with, we must acknowledge that there are did erring views about scientist c methodology (Koura, 1987). In addition, we must acknowledge studies on science and technology demonstrating that scientist c progress is in nuanced by social, cultural, and political factors. Researchers in various sciences are increasingly acknowledging that knowledge is socially constructed (Pickering, 1992), and knowledge of design, in particular (Condo et al, 1992; Monarch et al, 1993).
Moreover, the social in ounce on research practice includes aspects such as: shaping research goals according to available grants or unarticulated interests; publishing papers to receive tenure or to Justify traveling to conferences; and fraud (Bell, 1992; Broadband, 981). The rest studies on the social dimensions of science analyzed the progress of the recently, historical or re active studies in science and engineering have begun addressing the social aspects underlying research and the need for did rent methodologies if practical impact is sought.
These disciplines include: management science (Arises, 1980), education (Cuba, 1990), public policy (Pabulum and Classical, 1990), information systems (Beginners et al, 1987), cell biology (Grinning, 1982), design in general (Broadband, 1981), structural design (Addis, 1990; Diminishes, 1953), solid chains (Bucolically and Doorways, 1980), and even mathematics (Domicile et al, 1979).
Moreover, the social aspects manifested themselves in unexpected circumstances and in resolving seemingly trivial issues such as the implementation of computer arithmetic (MacKenzie, 1993)1 the most basic infrastructure for much engineering design research and practice. The importance of the aforementioned studies is twofold. First, they reject the received scientist c view as the means for formulating theories and as a means for improving practice.
Second, they acknowledge and demonstrate that research theology is a subject of study and constant improvement, and that gaining insight into the procedures of doing research can improve research itself. Since science is a social enterprise, the study of research methodology is mandatory for providing guidance in the maze of methodologies and in monitoring the quality of research. In order to sustain credibility, researchers must use and demonstrate that the techniques they develop in design research have some relevance to practice.
Moreover, since funding agencies Transactions of the SAME, Journal of Mechanical 3 press researchers to work towards improving design practice (National Research Council, 1991), researchers need to understand what kinds of studies are useful in practice, how are such studies conducted within budget limits, and which factors account for the did suasion of studies' results into practical engineering. Studying research methodology Researchers may ND it fruitful to study: the objectives or goals of engineering design research; how can these objectives be full Lied through research; how can progress towards research goals be tested; and how can this overall process be improved. Such study will evolve a repository of methods with their assumptions, interpretations, successes and failures. This is the essence of studying engineering design research methodology. This view does not advocate for nor lead to anarchy.
Furthermore, the evolving nature of methodology does not empty the usefulness of some principles for evaluating scientist c theories (e. G. , such as those acknowledged even by Kuhn, 1987), nor does it mean that methodology is merely an art (Beverage, 1957) that is not amenable to systematic study. It only acknowledges that the assumptions underlying types of research projects must be studied. We now illustrate the study of research methodology by elaborating some issues related to Annotation's six-step methodology 1987, p. 154).
Each of the steps raises issues that need further study. These issues are not startling; some are familiar while others are not. Unfortunately, most of them are neglected all too often. (1),(2) Propose/hypothesize that a set of rules for design can elucidate part of the design process and develop those rules. Several questions arise about the actual execution of this activity. What is a good source of such rules? Are (UN)successful designs (Petrol's, 1989; Such, 1990), patents previously issued (Researchers, 1988) or design textbooks (Acquire and Wallace, 1990) good sources?
Is studying human signers useful (Sublimation, 1992)? The answer is obviously a armature; nevertheless, rarely are these sources consulted. If studying human designers is useful, how do did rent ways of studying a etc the usefulness of the rules hypothesized? Inarguably, such studies bring to bear research methods from psychology and sociology into play in design research. For example, how are designers' activities being coded in observational studies? Is the coding scheme tested for reliability by using at lease two coders? Are the results statistically valid?
Which criteria may be used for selecting candidate hypotheses for further testing? Can the subjective bias in this selection be reduced? Note that the above questions raise a related question. Consider trading the quality of the design rules proposed with the resources to ND them. What kind of information is needed for making a sensible trade o and how can this information be collected and organized? (3) Have novice designers learn the rules and apply them. How is the above learning process taking place? Are the designers being taught thus introducing teachers' bias?
Or do they learn the rules on their own, potentially by solving Transactions of the SAME, Journal of Mechanical Design, 1995, in press 4 there design problems, thereby excluding the exercise of some measure of control? How are problems selected such that novice designers can solve them yet such that they are relevant to real practice. For that matter, how relevant is any laboratory experiment to real design? This critical question leads researchers in other disciplines as well as in design to use did rent methods such as ethnography and participatory research while studying designers.
See (Reich et al, 1992; Sublimation, 1992; Reich, AAA) for additional details. Are benchmark problems used by did rent researchers to allow for the replication of results? Is performance on benchmark problems indicative of performance on other problems or on real design? Is it possible to replicate results relevant to real design? Can rules for multidisciplinary design be hypothesized and tested in the same manner? If the common view of science is adopted, this study must be controlled to be valid.
One minimal requirement is that another group of designers participate in however, that since the rest group of novice designers are trained with the new rules, the second group must receive similar training with default or irrelevant rules. Furthermore, members of the groups must not know which group was trained with he new rules. A better study may also include two groups of expert designers, one that learns the rules and another that learns the default rules. The latter may provide better indication about the relative merit of the new design rules with respect to existing design practice.
In contrast, if the study follows a did rent methodology such as participatory research (Reich et al, 1992; White, 1991), the nature of the study would change sign scantly into long-term case studies where real design problems are addressed. Exercising common scientist c methods in this methodology may damage research (Plumber and Pricing, 1983). 4) Measure the design productivity of the rules. How is productivity being measured? Which criteria are included in the measurement: quality of design, time to design, or revenue of manufacturer?
Do the measures used adhere to the principles of measurement theory (Roberts, 1979; Reich, 1995), or are they ad hoc and meaningless? Do independent designers than those who created the designs, or do potential customers, participate in this measurement? Can the quality of design be assessed without manufacturing it and subjecting it to actual use? How relevant will abstract measurements be to practical design? Is the agreement quantitative or is qualitative information being gathered as well? (5) Evaluate the results to con arm or refute the hypothesis. How is the measured data evaluated?
What are the criteria that determine whether a hypothesis was con armed or refuted? Are these criteria general or context dependent? Note that most philosophers of science including Popper and Kuhn reject the existence of such criteria (Whimper, 1979). Are the criteria correlated with real design? That is, could not researchers ND designers successfully employing design rules that were refuted by researchers? For example, Fruits et al. 1990, p. 478) describe engineers using theories that produce erroneous results with respect to experiments but that have a pragmatic utility of did orientating between candidate designs.
Are hypotheses really refuted or con armed or are did rent hypotheses found to be useful Transactions of the SAME, Journal of Mechanical Design, 1995, in press 5 in did rent contexts? When is it possible to disregard experimental evidence in favor of keeping a hypothesis (Saga's, 1975)? When can experiments be harmful to progress (Trundles, 1982)? Does a failure of a hypothesis constitute a failure of a research project or can t provide useful information worth reporting? Will archival Journals publish such a report? (6) Re en the hypothesis. The comments on items (1) and (2) apply here. Jesting? When is re moment ins cent to address the failure of a hypothesis and a new worldview" must be adopted? The above expansion of Annotation's proposal re sects the complexity, richness, and necessity of studying research methodology. It illustrates that the design of a research activity is complex and did cult. It hints that some activities that lead to research successes may fail other research and that some activities may not be compatible with some methodologies. Furthermore, research failures (OR SUCCESSES) can lead to practical successes (or failures).
Therefore, it is critical to identify where methods fail or succeed and in relation to which assumptions. 3 Summary Science does not progress according to a distinctive methodology, nor could engineering design research; especially not if the goal is advancing design practice and not some abstract 'understanding. ' Did rent research scenarios consisting of did rent goals, disciplines, and cultural settings, may call for did rent research theologies for attaining the stated goals. Research involves design and therefore design researchers must be re active continuously.
This paper illustrated how researchers can be re active upon their research methodology. If researchers object to such re section, they risk losing credibility and, more importantly, lose the chance of discovering whether their work is meaningful. Acknowledgments The ideas expressed in this paper been Ted from discussions with Surest Condo, Sean Levy, Shouldn't Mulch-Reich, Air Monarch, and Cesarean Sublimation. This work was done partly while the author was with the Department of Civil and Environmental Engineering, Duke University, Durham, NC. And the Engineering Design Research Center, Carnegie Mellon University, Pittsburgh, PA.