This is the first day of creating a blog. I wonder many times what I would write. I have reflected on would my ideas be worthy to spark conversation, challenge current ideologies, and etc. My hope, as I venture out to participate in the endeavors of OpenLearning, that I can offer different perspective on the field instructional design from the points of accreditation, career development for faculty, career develop for academic support staff, and course development.
Category Archives: Virginia Group
Virginia Faculty Collaborative Group
Dissertating in the Open (Visual Article Series)
To continue my series of visual articles on educational research, I have branched out (or perhaps come home) to the act of dissertating in the open. The following is meant to introduce the idea – to show the overlap and points of potential integration between open scholarship and the process of planning, implementing, and publishing a dissertation.
Much of what follows is inspired by the writings of Catherine Cronin and George Veletsianos and Royce Kimmons, conversations with Bonnie Stewart, Maha Bali, Virtually Connecting, and #OpenEd2016, and my own work with Jon Becker and Alec Couros. Special thanks again to Maha Bali and Tod Massa who reviewed this and offered their comments.
This is not meant to introduce the idea of open dissertations completely. There are additional things to be said and places to go. This is a part one.
Virginia Faculty Collaborative cMOOC Countdown
I’ve been working with a great group of higher ed folks affiliated with the AAC&U Faculty Collaboratives project for the last several months. The collaboratives are state-level efforts to enlist faculty in the far-reaching and essential challenges of re-imagining role of liberal education at this time of transformation across the higher education landscape. In January […]
Describing Qualitative Methods in Educational Research: The Annotated Infographic
As I described in my last post, Understanding Educational Research and Design, I have spent a significant amount of the last year reading and evaluating the educational research literature for its structure, themes, and methods. I found problems; too many articles lacked epistemological framing, glossed over faulty sampling and data collection, provided minimal information on analytical processes, and failed to advance the literature in any meaningful way. As this concerns me in terms of the future of educational research, I have started to develop a collection of downloadable, CC-licensed visual articles that address some of the issues I found in the literature.
One of the qualities I enjoy about educational research is its strong tradition of using qualitative, quantitative, and mixed study designs. However, too often I found articles that described qualitative data collection without information about participants or the collection process, qualitative findings without information on the analytic processes, or qualitative collection processes without information on analysis or findings. It was difficult to tell if the problems lie in the study design and implementation or whether these were shortcuts made in writing and reporting.
When researchers take short cuts in reporting research processes – qualitative or quantitative – they inevitably and irreparably devalue the work. The usefulness of a research article hinges on the reader’s ability to assess the quality and personal relevance of the research. The reader cannot evaluate the research if the author does not provide specific kinds of information about the work.
The following visual article reviews some of the critical methodological pieces that must make it into a research article (regardless of prescribed word counts), if a reader is to be able to make meaning of the work. While I have presented my argument through a lens of qualitative methodologies, at least one Twitter follower suggested that it was relevant to all research, regardless of methodological tradition. I tend to agree with him, but will be creating one specific to a quantitative lens soon. Also, a shout-out to all the Twitter followers who reviewed this visual article before I published it here. Special thanks go to Margy MacMillan and Chelsea O’Brien for their detailed feedback. Also, thanks to my friend and SCHEV colleague Tod Massa for looking at this about a dozen times over the last week and finally telling me today that it might be okay.
- For more information on criteria for quality in quantitative research:
Lincoln, Y. S. (1995). Emerging criteria for quality in qualitative and interpretive research. Qualitative Inquiry, 1(3), 275-289.
Mays, N., & Pope, C. (2000). Assessing quality in qualitative research. British Medical Journal, 320 (7226), 50.
Trochim, W. (2006). Qualitative Validity. Research Methods Knowledge Base
2. For more information on thematic analysis:
Charmaz, K. (2003). Grounded theory. Qualitative psychology: A practical guide to research methods, 81-110.
Fereday, J., & Muir-Cochrane, E. (2006). Demonstrating rigor using thematic analysis: A hybrid approach of inductive and deductive coding and theme development. International journal of qualitative methods, 5(1), 80-92.
Strauss, A., & Corbin, J. (1994). Grounded theory methodology. Handbook of Qualitative Research, 17, 273-285.
Vaismoradi, M., Turunen, H., & Bondas, T. (2013). Content analysis and thematic analysis: Implications for conducting a qualitative descriptive study. Nursing & health sciences, 15(3), 398-405.
3. For more information on qualitative research methods and strategies (general):
Bogdan, R., & Biklen, S. K. (1997). Qualitative research for education. Boston: Allyn & Bacon.
Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry (Vol. 75). Sage.
Strauss, A., & Corbin, J. (1998). Basics of qualitative research: Techniques and procedures for developing grounded theory . Sage Publications, Inc.
Giving the LMS a chance
I haven’t been shy over the years in criticizing the LMS. Students have consistently told me to stick with my website instead. Downside: they can’t see their grade up-to-the-minute. Upside: it’s organized to make it simple and easy to use. Plus all the stuff I’ve mentioned before (openness, availability outside semesters, etc.) Still, my critique …
Confessions of a Computer-Mediated Communications Addict
Yes, I’ve been going through the closets lately. Here’s something I caught that might be of interest: a closing keynote address I presented at the 2002 Mary Washington College Faculty Academy on Instructional Technology.
It was an interesting in-between time. Our Assistant VP for Instructional Technology, David Ayersman, had left MWC in 2001. Our first CIO, Chip German, would arrive and begin his work in the fall of 2002. I had led the Instructional Technology Advisory Committee for several years but had not yet become an administrator. That step occurred in the summer of 2003, at Chip’s instigation, and would change all that followed in my career.
And I had not yet read Doug Engelbart’s Augmenting Human Intellect: A Conceptual Framework, or Vannevar Bush’s As We May Think, or any of those writings. I had never even heard of those people. I had no clue whatsoever about the “resonant frequency” of this stream of thought. No blogging, no podcasting, no Bluehost Experiment, no domain of my own, nothing. As you’ll see from the talk, it was all listservs (surprisingly sturdy and useful then) and what turned out to be ugly, slow, and ultimately disappointing Blackboard discussion forums. (I don’t think I’d discovered the Steve Hoffman Music Forums at that point, either–a new moment in my experience of discussion forums that would eventually turn me away from Blackboard and its ilk for good.)
So this paper tumbles out of my closet, and I caught it. In this talk I see that nearly a decade of working with computer-mediated communication had brought me into the thought world of dreamers who’d imagined and helped to build my future–ironically, unmet dreamers whom I had yet to discover. I see that my discovery of those dreamers ended up radicalizing me, in a way, as what I would read just two years later awakened me to the scope of those dreams and their liberatory potential for teaching and learning. Those dreams, and those dreamers, seemed to me a secret history of shared yearning, flawed and stained as all human endeavors are, but one that asked the central questions about the most salient concerns. It was an intellectual rebirth for me, but also the beginning of a trial-by-fire in some respects.
I see how ready I was for my next teachers to appear. I see a discontinuous moment ahead that I could not have anticipated. (I see that I am already using the word “bootstrapping.”)
I am trying to draw some lessons from this moment that comes into a new focus, fourteen years later.
I also like the muddle-mull-meal idea, which I’ve never used since in any talk I’ve given or any paper I’ve written. Maybe you’ll like it too.
Confessions of a Computer Mediated Communications Addict_final-2
Understanding Educational Research Design: The Infographic
I.
This infographic represents a love letter, from me to the educational research community at large. Over the last year, I have engaged in two large, systematic literature reviews, designed an educational research instrument, co-written an educational research grant, and completed and defended my dissertation research in educational research and evaluation. These projects have not been limited to one field; rather, they have spanned subdisciplines (e.g. library, social media, student engagement), sectors (e.g. K-12, adult, higher education), stances (e.g. policy, psychology, teaching and learning) and funding structures (e.g. academic, professional, and independent).
I have been brought onto several of these projects as a methodologist. My role was to describe and critique the approaches and methods used by researchers within these different educational research communities. In this capacity, I have reviewed approximately 800 educational research articles this summer alone. What I have found is systematically alarming, but it is a story that will unfold through a series of published articles, conference presentations, and blog posts over the next year. This blog post and downloadable infographic marks a starting point for the story.
II.
In engaging with colleagues, authors, students, and casual readers of educational research literature, I have discovered that many people have a narrow perspective on how to talk about published research. Oftentimes, their discussion stops and starts with the terms “Qualitative,” “Quantitative,” or “Mixed Methods.” Alternatively, it will start with data collection techniques: “This is a survey” or “The authors did a bunch of interviews” are common ways to describe work.
The problem with this approach is that it ignores the fundamental truth that an author’s philosophical stance (critical, constructivist, postpositivist, etc) will impact every other aspect of their work. As my friend Maha Bali has argued, positivists and constructivists can both produce qualitative research, but it will have very different flavors to it. To call something “qualitative” is not enough to describe exactly what it is.
Furthermore, the pathway from constructivist to qualitative to data collection (think interviews) and analytic methods (think thematic coding) is not automatic and cannot be assumed. Some interviews are almost as prescriptive as questionnaires while others are free-flowing. Observations can yield qualitative or quantitative data. Thematic analysis can be inductive or deductive, emerging or compliant with a pre-existing framework.
All of this is to say that we must think and talk bigger than data collection and analytic methods when we describe research. If we fail to identify the underlying philosophies and assumptions of the work, we fail to understand the shortcomings, biases, and strengths inherent to it. We fail to ask the right questions and grasp the bigger picture.
III.
Every research design textbook I have ever read includes a single sentence: “Research questions drive the design, not the other way around.” This is true of good research. Therefore, when trying to describe the quality of a corpus of research literature, it behooves us to look to the nature of the questions being asked. Are they focused on describing the nature of what is? How things relate? What causes what?
Paying attention to these things is essential for critiquing the structure of the individual study, but there’s more – Identifying trends in research questions can tell you a lot about the field itself. For example, young or rapidly changing fields need to establish what is before they can move to why things happen. Funders need to learn to value strong descriptive research in these areas. However, a continued reliance on descriptive designs in the presence of previously established understanding suggests something may be broken in terms of the scholarship being performed, maybe a systematic lack of funding or support, vision or strategy, researcher understanding or training. Further investigation is warranted.
The point: Research questions matter. Pay attention.
IV.
Obviously, research methodology matters as well. Qualitative, quantitative, and mixed methodologies have definitions and criteria and support specific research frameworks or study designs. There are textbooks. There are articles. There are classes. There are teachers. There are methodologists who can help others do these things well when they don’t have an interest in learning it themselves.
I bring this up because of a tweeted conversation I had with my former dissertation advisor, Jon Becker, earlier today. He let me know – probably just to irk me, because this will always work – that he was peer reviewing an article that had self-identified as “mixed methods” because it had included one open-ended question on the end of a quantitative survey.
I doubt that Creswell and Plano Clark would approve any more than I did.
I trust Jon to be an impenetrable wall against this sort of behavior, but unfortunately Jon does not peer review every educational research article. Let me say this loudly:
The widespread sloppy or ignorant handling of the “mixed methods” or “qualitative” labels within the educational research literature is shocking. When critiquing articles or a body of literature as a whole, it is essential for all of us to question how authors are using these labels. Are they accurate? And if they aren’t accurate, what can we surmise about why or how they are being used?
- Can you really perform an “in-depth qualitative case study” based on the findings of one quantitative survey (true story; no)
- Can you report out a proper phenomenology or ethnography without a single quote or rich description of participant context (true story; no and no)
- Should you be able to read over 100 self-identified qualitative studies without a single reference to memberchecking or auditing? (true story; no)
- Can you claim mixed methods with just one open-ended question on the end of a survey (true story; no)
- Should you publish a paper on the basis of a single focus group conversation (30 minutes) with an n of three? (true story; no)
- Why did these examples get through peer- and editorial review? And what are we going to do about it?
V.
Therefore, I offer up this infographic as a place to start a conversation: a conversation with students, researchers, publishers, peer-reviewers, and ourselves. It is not comprehensive. It is not an expert piece (although it has gone through a form of peer-review on Twitter, led by the marvelous Maha Bali herself). However, this infographic serves as a reminder of the following facts:
The misuse of methods is rampant and labels are not to be blindly trusted.
Research questions tell a story oftentimes bigger than the article itself.
Data collection and analytic tools can be used in different ways by different research traditions to produce very different outcomes.
Philosophy shapes everything.
Been Wanting to Do This for a While
Edit 11/21/16 *for those of you just reading this on 11/21/16 or later, I’ve been working on another draft via Twitter with much help from everyone but especially Maha Bali. I’ll be putting out another, bigger better blog post. When I do, I’ll link it <here.>
Edit 11/22/16 But until the bigger better blog post comes out, I’ve added version 2.0 below version 1.0.
–LG
Since moving on from my postdoctoral fellowship I’ve been doing some educational research and development consulting. I’ve noticed a trend in some of the work I’ve been offered. People tend to bring me on to projects (paid and unpaid) for one of two reasons. First, I am usually a good bet to be a wildcard – if you want a different perspective or someone to blow audience/stakeholder expectations out of the water, I can do that sometimes.
Second, I have experience in synthesizing large amounts of information, leading me to a lot of work reading, organizing, analyzing, and critiquing entire fields and bodies of educational research. A lot of my work will be coming out in conference presentations, blog posts, and publications soon. However, through this process I’ve discovered that a lot of people have difficulties understanding how to describe research in terms of its underlying assumptions, design, and methods. This is a problem, because educational research is so decentralized and generally messed up that a systematic approach is essential to making sense of any of it. Making sense of something and being able to communicate what you’ve found is essential to critique and reform…which, in my opinion, the educational research desperately needs.
Therefore, I made an infographic to describe how I think about organizing research literature. It’s an infographic and not a power point slide or a set of bullet points because I’m trying to develop my infographic skills. Would love to hear your thoughts.
Work This Week – Week 1
Since I’ve been falling off on my weekly photo posts, I figure I’ll start a weekly work series. While that doesn’t seem to make sense . . . it does. I am really bad about keeping track of the good things, the progress, the little bits that add up. I am very good at looking at large, looming disasters. I’m hoping that documenting progress will help convince me that I am making progress, that I am doing small things to help people do a bit more.
VCU Qatar – Online Art History/Museology Courses
This is part of three online courses being created to help teach art history and two other arts/museology related courses for VCU Qatar. My role in this is more of a consultant (web design is being done externally by CHIPS NYC1) but I’ve built a few things to help get us past (or into) some sticking points.
I used one of the HTML5up templates to make this super quick example. I’m not thrilled to have a white guy doing the video but it was the highest production value I found after a quick search (that might be shaped by having to use English). The text is from one of the professors although I chopped it up a bit. There’s the ugly start to a Timeline JS project in there.2
Instagram Shortcode
After getting frustrated with the requirements for the Instagram API stuff, I made a plugin that uses the PHP scraper so I can grab a certain number of images by user or tag without having to authenticate. It’s an evolution of the page template version I made a while back. It’s one of those moves from a one-off item to something that’s more flexible in a variety of scenarios. I need to figure a better way to show that kind of progression of time/energy/tools.
Random Post
Dan wanted a random post from his Sports Law Spotting site. Facet WP made it easy for me to do that without doing any custom work. Just make a new template. In this case, it’s getting posts that are in the question category and the “orderby” => “rand” gets us the random post. Now we can stick it wherever via shortcode. Nothing fancy but far handier than creating a whole child theme . . .
<?php
return array(
"post_type" => "post",
"post_status" => "publish",
"orderby" => "rand",
"order" => "DESC",
"posts_per_page" => 1,
"tax_query" => array(
array(
"taxonomy" => "category",
"field" => "slug",
"terms" => "question"
)
)
);
H5P – Empty Library
Michael mentioned that after creating content using the H5P plugin his library remained empty. I hadn’t noticed that but it proved to be the case for me as well on another site in our install. On my local development install things worked correctly. I’m still trying to figure out why it’s not finding content (I suspect either our divided database or maybe NGINX) but in the meantime, I went to create a new H5P element and took note of the URL. Even with an empty library we can navigate through the content using the URLs. https://site.edu/site/wp-admin/admin.php?page=h5p_new&id=3 gets us to the edit page for our 3rd item. Not a solution but a temporary way to limp along and proof the items are saving, they just don’t show up on the library page for some reason.
Flux Tower & Other Rice Center Data
I had a meeting with the VCU Rice Center group and will be working with them around constructing some explorable explanation options for the data they’re capturing around all sorts of environmental variables. Some of this is driven by the installation of a Flux Tower but there’s also just a ton of data they gather that will be fun to figure out. There is much, much data. I’m looking at options like Crossfilter to deal with both scale and getting some of the interaction done without tons of custom work.
Digital Humanities History Syllabus
I’m co-teaching a digital humanities history course next semester with Dr. Ryan Smith. We’re working on the syllabus now. It’s going to be an interesting experience as it’s a new course for VCU. I was a history major as an undergrad but it’s been a good while for me. Just thinking about some general skill/disposition/tool relationships has been a bit intimidating. There is a lot to consider. I threw some hypothes.is notes on one of the readings Ryan suggested as a bit of practicing what I preach and trying to get a better handle on how digital humanities thinks about itself. It’s kind of weird in that I think about this stuff all the time but not really through the same lenses/words. At times “digital humanities” feels a bit confining.
Montessori
I’m reading Maria Montessori’s works from Project Gutenburg. I may re-read some Papert and, if I can take the depression, Illich.
Sports, Social Media, and Freedom of Speech
I did a guest spot in Dan’s Sports Law class to talk about issues involving student-athletes and social media restrictions. Once upon a time I played football in college and then went on to be an academic coordinator for student-athletes for a few years. That gives me a bit of credibility beyond the social media stuff. I did a little research getting ready and was really impressed by this article (and a bit dismayed by the comment). The presentation is embedded below. Click on it once and then you can navigate with your sideways arrow keys.
Sites in Progress
I’m working on Footsteps on the James and revising the Text Set site I did about 3 years ago. It’s always crazy to look back at how bad I was at things. I did all sorts of crazy stuff to make this particular site work and I won’t be able to easily undo them.
The one semi-interesting thing that I did to improve things was to deal with associating books with particular units. Originally, I’d sort of made this work by filling in a Gravity Form field via a URL parameter. So when you make a Unit that has many books associated with it, you’d end up with a URL that would fill in the right tag for the book. That works fine until you navigate away and come back to the naked URL (no ?tag=item777).
With a bit of javascript I am now able to set that field no matter what.
Luckily, I have the tag written in the body of the post (although this could be better and it could be in a specific div with an id).
So with some jQuery . . . I can get the item and then set the field.
jQuery(document).ready(function($) {
if (jQuery(".items").find("strong").html()){
var unique = jQuery(".items").find("strong").html(); //get item
document.getElementById('input_1_18').value=unique; //set field
}
});
Odds & Ends
I presented a one hour workshop on Rampages/WordPress. Not sure how effective that is with low attendance (~6 people). At some point it seems you’re so small there’s no size benefit and you just lose out on individualizing the conversation. I think targeting by discipline or something might make more sense. It also feels like any sessions ought to getting increasingly specific – move from a survey to more particular items with increasing specificity/sophistication.
Also sat through any number of meetings. Some beneficial. I continue to assure people that ALT Lab (my department) still exists and will continue to exist.
1 Who do seem pretty awesome. Their work is really nice and I love thinking about this move from a FileMaker Pro/AppleScript solution into WordPress.
2 I’ve had issues with that site loading very slowly in Chrome Canary so I’ve been linking to the version I keep on GitHub to increase speed.
Weekly Web Harvest for 2016-10-30
- Wiki Education Foundation – Wiki Education Foundation
- » BC Open Educational Technology Collaborative
- Of Course Westworld’s Costumes Are 3D-Printed, Too – Racked
With the exception of some places in Italy or maybe England, you honestly can’t find beautiful, intricate fabrics anymore. It’s really sad! So we had to hire 3D printers in Los Angeles to reprint all our vintage fabrics so we could have more — and then beg them to work around our crazy TV schedules.