PSYC 761 - Graduate Research Methods

Spring 2008; Brian Nosek; Tuesday and Thursday 9:30-10:45; Gilmer 081

Course Objective

Completion of this course will provide a foundation for the practice of science. We will wrestle with the fundamental issues for designing and executing a program of research, and in the interpretation and reporting of the research results.

17-Jan Introduction: Why do science? Sagan, 1996; Chaps 12, 13, 14, 17, 19
22-Jan Generating Hypotheses McGuire, 1989; McGuire, 1997
24-Jan Research Ethics Belmont Report, 1979; APA Ethics Code
29-Jan Question and Hypothesis Presentations  
31-Jan NO CLASS
5-Feb Rules of logic: Critical thinking Schick & Vaughn, 2002 [evidence, fallacies]; Pope, 2004; Lehman, 2001
7-Feb Internal and External Validity Shadish 2002 Chap 1; Sears, 1986 [part 1, 2]; Mook, 1983
12-Feb Construct Validity Cronbach & Meehl, 1955; Shadish 2002 Chapter 2
14-Feb Research Design Brewer, 2000; [[maybe Smith, 2000]]
19-Feb Measurement [[maybe John & Benet-Martinez, 2000 ]]
21-Feb NO CLASS  
26-Feb NO CLASS (may change)  
28-Feb Method and measures presentations  
4-Mar NO CLASS: Spring Break  
6-Mar NO CLASS: Spring Break  
11-Mar Experimental Setting Aronson et al., 1998
13-Mar Survey Methods Schwarz et al., 1998; Visser et al., 2000
18-Mar Interpreting Results I Abelson, 1995 Chap 1; Abelson, 1995 Chap 6
20-Mar Interpreting Results II [[maybe Cowles & Davis, 1982]; Rosnow & Rosenthal, 1989; Prentice & Miller, 1992
25-Mar Hypothesis testing Cohen, 1994; Greenwald, 1975; Wilkinson et al., 1999
27-Mar Power & Effect size Sedlmeier & Gigerenzer, 1989; Cohen, 1990
1-Apr Bias in Research Tetlock, 1994; Sears, 1994; [[maybe MacCoun, 1998]
3-Apr Study presentations 1  
8-Apr Study presentations 2  
10-Apr Study presentations 3  
15-Apr Moderators and Mediators Baron & Kenny, 1986; Spencer et al., 2006
17-Apr Theory and Research Progress Greenwald et al., 1986; [[maybe Kerr, 1998 [part 1, 2] ]]
22-Apr Writing a research report Bem, 2003; Spellman et al., 2008
24-Apr Methods or special topics 1  
29-Apr Methods or special topics 2  

Other topics: Sampling, reviewing, presenting data, giving talks, response latency as DV, specific methods, meta-analysis (Johnson & Eagly, 2000 – handbook), scale creation (Clark & Watson, 1995), challenges for scientific thinking and what to do about it (Dunning, 2006, prediction; others), philosophy of science

Course Requirements

Participation (25% of grade) : Participation in discussion is essential for the success of this class. Each member of the class should bring a discussion question to class that was inspired by the reading. Discussion questions should go beyond clarification questions (those should be brought as well), and raise methodological and conceptual issues concerning the class topic. Questions will be submitted via email to nosek@virginia.edu the night before class, and students will present the question for discussion during class.

Research design paper (25%) : Students will apply the principles covered in this course to design a study. Ideally, this will be an occasion for designing a study that will actually be conducted. It could be a follow-up to an ongoing research project, or a new line of research. The paper will take the form of a journal article (APA-style) with a brief introduction, well-elaborated methods, simple presentation of hypothesized results, and a general discussion that focuses on the research design decisions and the methodological advantages and shortcomings of the design for testing the hypotheses. Papers can be no more than 20 double-spaced pages of text – including references; less is not bad unless content is sacrificed. Papers should provide theoretical background and justification, but emphasize the design and analytic issues associated with the project. Papers are due on the first day of finals week.

Research design presentations (25%) : Research design proposals will be presented in class for discussion and critique. Students will present their research design and discuss the main design decisions with an emphasis on the issues covered in the course. The rest of the class will provide feedback and suggestions for design improvements.

Peer critiques (25%): Members of the course will provide critique and comments to all other class members in the context of discussion and presentation of research reports. In addition to this informal feedback, each student will provide formal feedback to two classmates on four occasions:

(1) following 1/29 presentations due 11:59pm on 2/1

(2) following 2/28 presentations due 11:59pm on 3/4

(3) following 4/3-10 presentations due 11:59pm on 4/13

(4) on final paper due 11:59pm on last day of finals

Formal evaluations should be no longer than 1 single-spaced page. Evaluations are to be submitted via email to nosek@virginia.edu with the reviewer’s name and the name of the reviewee at the top. Feedback should contain constructive suggestions about the advantages and challenges of the conceptual, methodological, and analytical features of the proposal. The first three reviews may also contain feedback about presentation style, but the review should focus on the presentation content. Reviewers and reviewees are also encouraged to discuss the project proposal outside of the formal review process. Nosek will collate the reviews after each round, write an action letter, and circulate the letter and reviews to the reviewers and reviewee. This process is intended to encourage collegial, substantive feedback and approximate the article/grant review process.

Brian's homepage