Methods
1. Discussing data analysis methods & procedures
2. Comprehension check
3. Language focus
4. Language activities

Discussing/ justifying data analysis methods and procedures (Move 6)

After the data have been cleaned, data analysis can begin. Depending on the method, specific procedures for identifying and explaining relationships between the study’s key variables should be outlined at the level of detail that would allow other researchers to replicate this study’s design (Swales & Feak, 2012). If there is more than one method to be employed, each can be presented under a subheading. If a method described in other studies is to be used, reference should be made to the relevant source(s), together with any necessary information on how this method will be modified to fit the current study. 

In quantitative and mixed methods research, data analysis will most often involve the use of mathematical and statistical tools to establish and explain correlations between variables and/ or identify the patterns of association between these variables. Since statistical analysis of data (e.g., correlation and regression analyses) is typically done with the help of software statistical packages and other analytical tools (e.g., STATA, SPSSSAS, etc.), these are often specified, and information on procedures for measuring the values assigned to given variables is provided. Specific ways to determine the statistical significance of the results are also explained. What is even more important, the rationale for employing these procedures to ensure the validity and reliability of the results should be given in connection to the study’s research questions or hypothesis. The level of detail at which this is done depends on the field and the method.

In qualitative research, the data analysis involves the researcher's subjective interpretation of concepts which are difficult to measure and evaluate (for example, attitudes, beliefs, and feelings about something). This kind of analysis is based on information which is in most cases presented as text - spoken or written (for example, interview transcripts, recollections, memos, etc.) and thus requires a careful definition and interpretation of the meaning of text. To ensure the reliability of these interpretations, procedures for analyzing qualitative data may have to be explained at greater length (Partridge & Starfield, 2007). This also implies that the units of data for coding are identified (for example, an artifact, a word, a group of words, such as a participant’s quote; the test scores of an individual student or a whole class of students, etc.) and explained. There can in fact be several units of analysis depending on the type of analyses employed in a single study.  

Examples:

Here are two unadapted extracts from the Methods section of two articles (not research proposals!) in the field of education. Note what kind of language the authors use to explain what the chosen data analysis methods and procedures are and to justify their use in each study. 

Extract 1

Openness to Theory and its Importance for Pre-Service Teachers’ Self-Efficacy, Emotions, and Classroom Behaviour in the Teaching Practicum

Descriptive statistics were carried out with SPSS 20. The model illustrated in Fig. 2 was analysed by means of structural equation modelling (SEM) using Mplus 7.1 (Muthén and Muthén, 1998–2012). SEM modelling entails the concurrent calculation of confirmatory factor analysis (modelling latent variables) and path analysis. Thus, it provides us with both a measurement model and a structural model at the same time (Kline, 2005). SEM modelling was chosen because it allows complex models to be tested in one step, including the simultaneous testing of direct and mediating effects. Missing data were imputed through the FIML procedure (Geiser, 2010). The data's goodness of fit was evaluated through various fit indexes: ...

Source: Hascher, T., & Hagenauer, G. (2016). Openness to theory and its importance for pre-service teachers’ self-efficacy, emotions, and classroom behaviour in the teaching practicum. International Journal of Educational Research, 77, 15–25.

Extract 2

Developing Pre-Service Teachers’ Global Understanding Through Computer-Mediated Communication Technology

Data analysis emphasizes holistic descriptions and interpretation of the social phenomenon created by online discussions and reflections. The primary goal was to provide as broad a presentation of participants’ understanding as possible. I employed the constant comparative method (Strauss & Corbin, 1990) to identify and develop patterns and themes in preservice teachers’ understanding of global education. At the first level of analysis, generally known as ‘‘open coding’’ stage, I read and coded all participants’ online messages and reflection essays to identify major categories of preservice teachers’ thinking and compared data for similarities and differences. Strauss and Corbin (1990) cautioned that during this early stage of data analysis, the research ‘‘may come up with dozens, even hundreds of conceptual labels,’’ thus requiring another stage of data analysis that leads to the discovery and naming of new categories (p. 65). The categorization process involved the reexamination of the data that I had coded and labeled as I collapsed categories into themes.

Source: Zong, G. (2009). Developing preservice teachers’ global understanding through computer-mediated communication technology. Teaching and Teacher Education, 25, 617–625.

 

Task 1
Task 2

Distinguishing between Methods steps

Read the extracts (1 to 3) from Methods sections to research articles in the fields of political science, language learning, and management. Match the extracts to the steps by dragging them into the spaces. More than one step can be matched to a step:

Text 1 (Source: Schuck, A., Vliegenthart, R., & De Vreese, C. (2016). Who's afraid of conflict? The mobilizing effect of conflict framing in campaign news. British Journal of Political Science, 46(1), 177–194.)
Thus we are able to conduct a multilevel analysis that assesses the impact of both individual- and country-level variables as well as their cross-level interaction on the mobilization of citizens in the 2009 EP elections across Europe in a single study.
The content analysis is used to investigate how the news media in the different EU member states have covered the campaign.
We rely on a multi-method research design including content analysis and a two-wave panel survey.
The panel survey is used to assess the impact of such coverage on voter turnout.
To study the conditional impact of conflict news framing on mobilization, we ...
This design enables us to assess the effect of campaign news more specifically by including the results of our media content analysis in our measure of individual news exposure with the same news outlets that are in our panel survey analysis.
Restating the study's focus
Presenting research design
Identifying specific data analysis/ collection methods
Justifying the choice of the methods
Note: Every hint in group field should be enclosed with curly braces.
Text 2 (Source: Locke, T., & Johnston, M. (2016). Developing an individual and collective self-efficacy scale for the teaching of writing in high schools. Academic Writing, 28, 1-14.)
Participants were 140 New Zealand high-school teachers, drawn from four schools. Three schools were from the Welling-ton region, and were selected because they reflected a cross-section of school types (co-ed and single-sex, and with varyingsocio-economic catchment areas). The fourth was an Auckland school participating in the “Culture of writing” project discussed previously.
In the first stage, a principal components analysis was used to identify the dimensionality of the questionnaire data; that is, the number and nature of the constructs represented in the participants’ responses, and which items were associated with which constructs. This kind of analysis is typical in work ofthis kind, although exploratory factor analysis is often used for the same purpose...
Each item in the teacher-of-writing self-efficacy questionnaire used in this study asked respondents to choose from four options related to the prompt: “How confident are you that you can?... A four-point Likert scale was selected on the basis that it might be difficult for participants to distinguish more than this number of levels...
Presenting the study's participants (sample)
Describing data collection tools
Describing data analysis methods and procedures
Note: Every hint in group field should be enclosed with curly braces.
Text 3 (Source: Bishop, K., Simsarian, S., & O’Neill, R. (2011). Preparation and prior experience in issue-selling success. Journal of Managerial Issues, 23(3), 323-340.)
Consistent with issue-selling moves that correspond to the temporal perspectives of preparatory and selling moves, survey items were developed integrating prior qualitative research and based on the issue-selling moves identified by Dutton et al. (2001). Questions were designed to assess three or four of the highest frequency moves noted by Dutton et al. (2001) for each of the different move patterns: packaging, involvement, and process.
The setting for this research was a mid-size hotel in a large northeastern city. The hotel is jointly owned by a private owner and a larger hotel management organization. The initial meeting with the hotel general manager (GM) and human resource director gave important insights into the current culture and context of the organization.
The hotel had three managerial levels: GM, department directors, and supervisory managers. All eight department directors and 34 supervisory managers were considered managers for this study. The final participants included 32 managers (76% response rate) who provided 84 issues. On average, participants had worked for the hotel for four years and in the hospitality industry for 15 years.
Based on exploratory and confirmatory factor analyses, items for the nine most frequent moves resulted in a fit of two factors. Based on the early theoretical work (Dutton & Ashford, 1993) these two factors were labeled as preparatory and selling moves. The preparatory moves and the selling moves scales are calculated using a mean item score.
Presenting the context
Presenting the study's participants (sample)
Describing data collection tools
Describing data analysis methods and procedures
Note: Every hint in group field should be enclosed with curly braces.
 
 
 

Data collection vs. data analysis

Choose ONE option that best illustrates what these extracts describe:

A two-wave panel survey was carried out in twenty-one European Union member states. Respondents were interviewed about one month prior to the EP elections and immediately afterwards. Fieldwork dates were 6–18 May and 8–19 June 2009. The survey was conducted using Computer Assisted Web Interviewing. (Source: Schuck, A., Vliegenthart, R., & De Vreese, C. (2016). Who’s afraid of conflict? The mobilizing effect of conflict framing in campaign news. British Journal of Political Science, 46(1), 177–194.)
The questionnaire was developed in English and translated into the different national languages. It was then translated back into English as an additional check of the accuracy of the translations. The translation was supervised by the research team and carried out by TNS (which also executes and translates the Eurobarometer surveys). Irregularities and problems arising from this process were resolved by deliberation. (Source: Schuck, A., Vliegenthart, R., & De Vreese, C. (2016). Who’s afraid of conflict? The mobilizing effect of conflict framing in campaign news. British Journal of Political Science, 46(1), 177–194.)
Our analysis relies on process-tracing through a mixture of primary documents, secondary accounts, and interviews with eight anonymous people (academics, civil servants, and elected politicians) involved with tobacco policy in Ireland (October 2013). Where possible, different interpretations have been checked against one another in an attempt to clarify the evidence. The emphasis is on developing both a detailed timeline of tobacco policy development in Ireland as well as applying the model of tobacco policy outlined above to help explain what was done and not done, both over the long term and at particular times. (Source: Studlar, D. (2015). Punching above their weight through policy learning: Tobacco control policies in Ireland. Irish Political Studies, 30(1), 41-78.)
We assess the predictive power of SPF [the Survey of Professional Forecasters] forecasts by measuring simply their performance relative to an estimate of the unconditional means of the variables analyzed. More specifically, we compute the average of each variable from 1982q4 through to 1984q1−h quarters, for h = 1, . . . , 5. (Source: Azevedo, J. V., & Jalles, J. (2016). Model-based vs. professional forecasts: Implications for models with nominal rigidities. Macroeconomic Dynamics, First View, 1–30.}
A three-step approach, as outlined by Churchill (1979) and Anderson (1987) was employed to assess the validity and reliability of the measures. First, Cronbach’s (1951) alpha coefficients were computed in order to assess the reliability of each scale. Coefficient alphas were evaluated relative to the minimum .70 level recommended by Nunnally (1978). The coefficient alphas meet the criteria set forth in the literature, with the lowest coefficient being .76 for local sourcing responsiveness, indicating adequate reliability for the measures employed in this study. Second, a confirmatory factor analysis (CFA) was carried out to investigate the convergent and discriminant validity of each construct... (Source: Yeniyurt, S., Henke, J., & Cavusgil, E. (2013). Integrating global and local procurement for superior supplier working relations. International Business Review, 22, 351–362.)
 
 
 
 
 

Language for describing data collection and analysis procedures

Tense use

In research articles the Past Simple is predominantly used when describing data collection and analysis methods. However, Present and Future Simple/ Would + Infinitive are more appropriate when describing proposed methods in research proposals. Both active and passive voice can be used, as in the examples below:

  • The experiment will be conducted in two phases. First, … Second, …
  • A questionnaire will be developed and translated into Italian.
  • I will draw on data from 50 subsidiaries to test the direct effects of top manager nationality on language policy.

Passive vs. active voice

The use of the passive voice is especially common in descriptions of procedures (e.g., steps to be completed in a certain order, under specific conditions, etc.). Note that the Present Simple Passive is common when conventional procedures established in a research area are described (especially in hard sciences, medicine, psychology, etc.):

Examples:

  • The metal is then heated to a temperature that is below the lower critical temperature.

It is advisable to switch to Future Simple when you want to show what you are planning to do in your study to separate this from the standard procedures applied in your area:

  • To account for these differences, two questionnaires will be developed …

On the other hand, although passive voice is a typical feature of academic writing, continuous use of the passive voice may result in writing in which the reader loses sight of who the agents are or may perceive the text as overly monotonous. To avoid this, the active voice can be used, especially when one’s own commentary on different aspects of their research is provided.

The use of the active voice is in fact quite frequent. It can alternate with the passive voice to ensure a better “flow,” as in this extract from a psychology article (note that Past Simple is used to describe what was done in the study).

Example:

The Emergent Effects of Psychological Intervention
During the administration of the intervention, students received sealed envelopes with writing materials from their teachers, who were blind to intervention condition and the research hypotheses. In silence, students in the interventioncondition wrote about their most important values, such as friendships or artistic ability, while students in the control condition wrote about their least important values (see the supplemental material in Cohen et al., 2006, for details). After approximately 15 min of writing, students placed the materials back in their envelopes, and the materials were returned to the researchers. Students believed they were participating in a regular classroom writing exercise and were unaware that the exercise was a psychological intervention or part of a research study (Psychology).
Source: Powers, J., Cook, J., Purdie-Vaughns, V., Garcia, J., Apfel, N., & Cohen, G. (2016). The Emergent effects of psychological intervention, Psychological Science, 27(2), 150-160.

Note that if there is a choice between I/ we pronouns when the active voice is used, “we” tends to sound more appropriate when one writes in collaboration with other authors rather than as a single author. The pronoun «I» may figure in published academic writing (especially in mathematics, psychology, etc.), however, it is safer to avoid the use of «I» and «my» so as not to sound too assertive and individualistic unless this is a norm in your field.  

Linking words

A variety of linking words are commonly employed to discuss data analysis and collection methods, and specific procedures.

Indicating order or sequence

Prior to/ before + noun/ V-ing, next, during, after (that), on + noun/ V-ing (on obtaining the samples)

  • The variable will be transformed logarithmically before the analysis.
  • Next, to test the hypothesis we will created a model .., which …
  • After the introductory stage, we will have several discussions with participants about ...
  • On obtaining the data, I will perform statistical analyses to evaluate …

Showing cause and effect

As a result, consequently, because, because of/ due to smth

  • As a result, the adjustment process takes longer in response to … than in response to …
  • Consequently, much of our knowledge of causal effects must come from non-randomized observational studies. / Research is inconsistent in uncovering relations between X, Y, and, consequently, Z.
  • Because of time constraints, we only study X.
  • This is justified because the amounts of … are not too different for the … campaigns.
  • Due to the limited amount of existing research about …, a qualitative inquiry is needed to explore this interrelationship.
Task 1
Task 2
Task 3
Task 4
Task 5

Past Tenses (active or passive voice) I

Choose the correct verb form (active or passive) in the text below:

To obtain feedback from students and teachers, a group of 25 freshmen and a group of 35 pre-service English teachers to use the website and then comment on their experience. The college students to use the website over a ten-week period. Each week they to work on various exercises available on the website, and each session for approximately two to three hours. At the end of the ten-week period, a questionnaire to solicit their feedback. The questionnaire of nine questions in the format of a five-level Likert scale and three open-ended questions which asked students about the strengths and weaknesses of the website and possible suggestions for improvement. The pre-service teachers to use the various interactive exercises based on the ASR (automatic speech recognition) system for about two hours in a computer laboratory. After working with these different exercises, they to submit an evaluation report in which they the strengths and weaknesses of the system.

Adapted from: Chen, H. (2011). Developing and evaluating an oral skills training website supported by automatic speech recognition technology. ReCALL, 23(1), 59–78 (p. 69).

Past tenses (active or passive voice)

Put the verbs in brackets into the correct form (active or passive). Type your answers into the gaps:

During the intervention, students (to receive) sealed envelopes with writing materials from their teachers, who (be) blind to the intervention condition and the research hypotheses. Students in the treatment group (ask) to write about their most important values, such as friendships or artistic ability, while students in the control group (instruct) to write about their least important values. After approximately 15 min of writing, students (place) the materials back in their envelopes, and the materials (return) to the researchers.

Adapted from: Powers, J., Cook, J., Purdie-Vaughns, V., Garcia, J., Apfel, N., & Cohen, G. (2016). Changing environments by changing individuals: The emergent effects of psychological intervention. Psychological Science, 27(2), 150–160.

Future tenses (active or passive voice)

Put the verbs in brackets into the correct future form (active or passive). Type your answers into the gaps:

1. Further investigations (have) to be carried out to solve the above problems and examine our findings.
2. The models that we (develop) in the study (base) on idealized mechanisms.
3. In this paper, I (argue) that software agents in the form of avatars (be) more effective in helping students learn about explanations than plain text.
4. The importance of this concrete dimension of administrative changes (discuss) in great detail.
5. It (demonstrate) in the study that leading global institutions are building an international financial architecture with law as a principal foundation.
6. We hypothesize that perceived usefulness of the online nursing courses (have) a positive effect on the behavioral intention to use the courses.
7. As (show) later, the new technique can help to test the validity of some important model assumptions.

Cause and effect phrases

Complete the sentences by dragging one of these words and phrases into the gaps:

because
consequently
due to
because of
As a result,
1. it is not surprising that songs from higher-selling albums are downloaded more often.
2. This assumption may again not be correct in practice lack of formal randomization.
3. Such a practice might seem less dangerous the public aspect of the data involved.
4. The third institution was included in the study it is one of the oldest and best-known tourism schools in Turkey.
5. It would be interesting to determine the importance of innovation for firms involved in the process of internationalization.

Revision

Decide if the following sentences sound correct. Click Yes or No next to each sentence:

1. Album weeks prior to the release date will be excluded from the sample.
2. This assumption may not be correct in practice due to lacking of formal randomization.
3. Such a practice might seem less dangerous because of the public aspect of the data involved.
4. Because of data limitations discussed below, these results are more speculative in nature than the other findings of the paper.
5. This concern arises because of there are two different real estate factors included in this study.
6. On obtaining financial data, we will gain a deeper understanding of why some of these government-backed schemes have not been successful.
7. Before presenting of the final evaluation results, we will summarize the major hypotheses that were investigated.