The following post consists of the lightning round remarks I made as part of the OLC Accelerate 2018 Research Summit. I was asked to speak on systematic inquiry – what is it and why it is important – to a group of higher ed online instructional designers, faculty, and administrators.
I was an interesting choice as a panelist. The other members of the panel were CAOs, CEOs, and internationally known researchers. I am a designer, not a researcher. I am not a CAO or a CEO or a director. However, I am an online learning research super-user, a sometimes action-researcher, and a sympathetic critic who wants something more for our field and the people in and around it. These were the credentials I took up with me when I got on the stage.
This talk resonated with about 10% of the room and therefore cannot be considered well-received. However, I’m used to being different and I’m glad I participated – I said exactly what I needed to say.
Also, I owe much of what I said to this essay by RE Wyllys. I credited Dr. Wyllys throughout my talk, but you should read his essay for yourself.
When I was asked to speak on systematic inquiry, I think I probably did the same thing that you all would have done: I googled it, read some articles and essays from a variety of disciplines, and then checked in with my Twitter followers just to make sure I wasn’t missing something big in terms of a discipline-specific meaning or theoretical framework.
Ultimately, I found a definition that seemed to make everyone happy – which may be the first and only time that happens on Twitter. All sources seemed to agree that systematic inquiry is the careful, deliberate effort to understand a problem or an unknown. It is the primary way in which we develop formal knowledge or practice within a discipline or field of study. In general, systematic inquiry is what people typically mean when they say something is “good” research, scholarship, or evaluation.
In Zen and the Art of Motorcycle Maintenance, Robert Pirsig describes systematic inquiry as an artful combination of inductive and deductive logic that is best laid out formally to help promote reflection and sharing. He goes on to list the classic scientific method as what one should write down in their notebook.
The scientific method is a great example of systematic inquiry, but when we consider what systematic inquiry means beyond a single question – through time and across networks of people – we need an orientation that goes beyond the scientific method. Systematic inquiry can also be considered as a way of being in and interacting with a world that is relational, fluid, ethical, and emergent. It hinges on the act of questioning everything.
What are the values that drive our questions and designs? What are the politics of our descriptions? Why am we valuing these methods over others? What is our purpose and how does it align with our impact? For whom are we writing and why?
These are the sorts of questions that we must ask and answer as part of a longitudinal or systems-level vision of systematic inquiry.
There are reasons we strive for systematic inquiry. The first can be found in Zen and the Art of Motorcycle Maintenance (although Albert Einstein also said something very similar):
“The real purpose of the scientific method is to make sure Nature hasn’t misled you into thinking you know something you don’t actually know.”
The world is complicated and we must approach it methodically or else we will get lost in the web of independent, dependent, constant, confounding, mediating, co-variables, and other things that are currently unimaginable that make up most things worth understanding.
The second argument for systematic inquiry is deeply rooted in the social sciences. The act of reflecting on and seeking out diverse forms of feedback about every aspect of our work not only helps protect us from Nature, but also helps protect us from ourselves.
The cautionary tale of the streetlight effect is real; many of us make the mistake of focusing our attention on areas which are already familiar or easier to measure while ignoring other possibilities that are more difficult to pursue.
Finally, systematic inquiry provides an infrastructure of shared language and practices that supports other forms of knowledge discovery and propels them forward. Roald Hoffman, the Nobel-prize winning chemist, writes that serendipitous discovery will occur when the mind has been prepared by intense, preferably interdisciplinary study, our institutions have provided space and opportunity for knowledge sharing, we have cultivated our senses of curiosity and wonder, and we have committed ourselves to hard work.
In considering the state of systematic inquiry in our field’s research literature, I am grateful to the online learning researchers who engage in systematic inquiry. In fact, we have some of the best online learning researchers here in this room with us today.
However, as a field, online learning has arrived at a point in its development when it is time to reconsider how we approach online learning research – particularly at the level of teaching, learning, and instructional design – so that it is meaningful and useful to others in our field and beyond.
It would be impossible to cover all aspects of online learning in a lightning speech, so I am going to focus on educational social media research: the body of literature that addresses the use of social media – including wikis, blogs, Twitter, YouTube, etc. – in the higher ed classroom.
As part of an ongoing project, Drs. Jessica Knot, Laura Pasquini and I recently reviewed almost 500 empirical articles on the use of social media in higher ed.
Our initial analysis indicates that educational social media literature is dominated by perception and use surveys and descriptive studies. In general, these descriptive studies are written by practitioner-researchers who implement a social media tool in the classroom and are reporting on their experience and the learner’s perceptions of their experience. Less than 10% of studies involved quasi-experimental or experimental design. Relatively few of the descriptive studies identified a specific qualitative research methodology and those that did rarely met baseline standards for rigor (or even discussed rigor).
Our findings were consistent with Kimmerle et al (2015), which also looked at educational social media research: many of the articles written about social media use in the classroom focused on the technological aspects and specific functionalities of particular tools without consideration for their pedagogical purpose or associated learner outcomes.
As a field, our literature is hyperfocused on tools and often fails to place digital tools in the larger context of teaching, learning, and instructional design.
We cannot claim to be supporting systematic inquiry into online learning until we diversify our research questions and enhance the overall rigor of our research methodologies.
How do we fix this problem? How do we design our systems and foster a culture to promote a systematic understanding of online teaching and learning?
I have more questions than answers and I’m hoping we can continue the discussion through the breakout sessions, but here are a few observations and ideas to get us started.
Based on the hyperfocus on tool functionality, the limited discussions of instructional design, and thin applications and descriptions of research methodologies, I question how much pre-planning goes into the action research that accounts for a large portion of the online teaching and learning literature. I also question the support many of these faculty, who are first and foremost subject matter experts, are receiving from instructional designers and methodologists as they write these articles.
At the theoretical level, collective IQ plays a large role in the value system of digital pedagogies and participatory culture. So how can we operationalize collective IQ in a knowledge development context? How do we increase the inclusion of instructional designers, methodologists, learners, as well as instructors in the research process?
One of the biggest weaknesses in the online teaching and learning literature is a lack of communication; there appear to be major differences in how researchers are defining key terms, instructional approaches, and research methodologies. I think a lot of that comes from the interdisciplinary nature of our field.
How do we foster interdependency and community within our interdisciplinary field so that we might create a shared understanding of key terms and methodologies?
Alternatively, if this is not possible, how can we promote transparency and thick descriptions so that readers can understand what is being described and judge the results for themselves? Is that something that could be encouraged at the functional level of peer reviewers, editors, and journals?
And finally, if we enhance the rigor around what counts as research, how can we make room and promote value for other important types of knowledge creation? The sharing of design ideas and anecdotal experiences are valuable to other educators and designers, but perhaps they belong in other places or formats beyond the empirical journal article. How do we create those spaces and encourage others to value them as they do a peer-reviewed research journal?
These are some of the questions, I hope we will address today and through future conversations. Thank you.
