Yesterday I met with David Lancy, a Prof here at USU considered to be somewhat of an expert in qualitative research. He has been very generous with his time in visiting with me. His input has been very helpful. I shared with him where I am currently at in my research and asked for his input. Here is a summary of his comments:
- Look for big findings first.
- See if you can detect archetypes teachers from the data.
- The quantitative data I gathered is likely to be very noisy.
- I should work from a theory of learning and instruction, not technology.
- Finishing my dissertation needs to be a primary concern.
Look for big findings first
Clearly this only makes sense, though I probably would have not come up with that. Why give each bit of data and topic mentioned equal weight. He suggested that I just take each packet of information and sort them into four piles, not even being specific about what the piles are. I’m in the process of enterring all of the data into the computer and so I probably won’t go through the physical steps of what he described. I will do the equivalent electronically though. This leads to the next comment…
See if you can detect archetypes teachers from the data
My initial plan was to consider each of the tasks, barriers, and guidelines separately and look at the data I had gathered seeing what it had to say about each. In retrospect, this is probably a very quantitative approach. I would probably be able to report statistics about the percent of participant statements that gave evidence or support for each of the elements. The approach he suggests contrasts heavily with this. His approach is to look for archetypes into which I could group my participants. Each archetype description would obviously not completely describe each participant I placed in that category. However it is perhaps a good way to think about it.
Example archetypes could be: (a) teacher sees this and immediately envisions how it could be used, (b) teacher likes what they see, but needs some hand holding, (c) teacher wants something handed to them that they could use out of the box, (d) teacher would never touch this stuff.
The pragmatic usefulness of identifying archetypes might be to help give direction to where developers (I) should spend future development time and money. For example, I could focus on creating additional teacher support and integration with existing standards and textbooks, or I could just focus on developing newe mathlets.
After I have identified archetypes I could then move to analyzing parts of the archetypes.
I recently read a writeup of a closely related study. In doing that I was bit concerned about the lack of reporting of numerical data along side of provided anecdotal quotes. Because I’m pretty new to qualitative research, I was not clear as to how you should present your data. He indicated that he uses the following approach:
- Present a statement of a theme
- Give representative quotes to back it up
- Indicate some percentages about the number of participants or statements that concur or disagree
Another method I have seen mentioned in the literature is to present and subsequently discount alternative explanations of the data. One thing that Dr. Lancy recommended is to make sure that any claims I make can be supported the data. I guess this is obvious advice, but from some of my readings, not always followed.
The quantitative data I gathered is likely to be very noisy
I had a difficult time getting teachers to comment on issues like: In what ways would you like to reuse and adapt interactive online resources? and What features would you like in an authoring tool? I’m guessing that this is because they have had relatively little experience in these areas. So what I chose to do was present them with lists and descriptions and asked them to rate items. This resulted in likert scale and ranking survey data. Lancy thinks this data will likely be very noisy if I try to look at distributions across all participants. I agree, it will be interesting to see if looking at the data by archetype makes it less noisy.
I should work from a theory of learning and instruction, not technology
Dr. Lancy’s impression is that many of us Instructional Technology types tend to focus more on the technology than on the learning. He pointed out that there is already significant literature on teacher adoption of innovation and that we should not expect their approach to adopting this innovation to vary greatly from other types of innovation. He suggests grounding theory and interpretation of data in the context of that literature.
Finishing my dissertation needs to be a primary concern
A number of times during our discussion, I started talking about how I could gather some additional data or do this or that. Each time, Dr. Lancy brought me back to reality, reminding me that I need to finish. His most memorable statement was “Joel, we don’t want you to retire on a graduate student salary.” I think my wife would resonate to that statement 
Posted on August 13th, 2003 by joel
Filed under: research
Leave a Reply