We seem to be getting some consistent feedback from reviewers, and from friendly critics who have read the paper(s). Here are some of their points, both "big" (killer), and "small" (subtle but important). There are surely some other make-or-break issues, and they can be added to this list.
- Citing non-research materials makes people suspect they're reading a "student paper" not a "scholarly paper". I personally see at least two reasons for quoting and citing these kinds of works: (1) it's meant to make the writing accessible; (2) to some extent we are dealing with a "pop culture" phenomenon, and for this reason it is appropriate to mention popular items. However, academics aren't always likely to agree, and even academics working in the free/open world are somewhat likely to freak out about a footnote to Wikipedia. A possible middle ground: use some popular stuff if "popular" is relevant (Fast Company, maybe Infinite Jest), but make sure we use really solid academic stuff where it's needed (e.g. point to a survey paper about meta-learning instead of to a Thomas Friedman quote).
- To a lesser extent, citing secondary literature (the paper about Darwin) vs primary literature (Darwin) is a similar issue.
- People seem to stumble over the section about "the challenge". It's probably not sufficiently clear what problem paragogy is supposed to help solve. We say it has to do with people coming together to work on or learn about something, when none of them has a particularly strong expertise -- but this needs to be beefed up a bit. Similarly, in the following section on "What does paragogy offer", instead of just listing 5 principles, we should really say in detail how paragogy addresses the challenge brought up in the previous section (OK, through these principles, but still, concretely).
- Is it a theory or a critique? This is a big stumbling block. Personally, I almost feel like we could throw away everything but the "Key Message" (A Case Study in Paragogical Evaluation) and start over -- or perhaps better yet, split the paper into two papers, one where we develop the theory, one where we really develop this particular critique. In any case, I think we wouldn't have had anything to say here without the critique. (This is what made it so frustrating when one of the reviewers didn't even read that part!) So, it's probably not called for to throw the other things away, but let's set them aside for a minute, and look at how to re-build the paper around the key message. (And let's be more clear up front about what the key message is.)
- Pictures. Remember we're writing for an audience that includes people whose English skills aren't as good as ours (and probably some whose English skills, particularly when it comes to academic writing, are much better). If we can find a way to sum everything up with a few pictures, people are much less likely to become confused. We've been using the Raphael picture to great effect: we could potentially adapt the OddMuse "How Wiki Works" image, to make "How Paragogy Works".
- Examples. Academic reviewers didn't much like our examples (AAR, Darwin, AA); some friendly critics have said they like them (and we, obviously, like them, or we wouldn't have included them). Maybe part of the problem is that these examples only come in the middle of the paper, and people were hoping to have a working example earlier on. To some extent this was what referencing the Meno was supposed to provide, but because we can't quote all of it, people might be confused about that. Anyway, a good example functions like a picture (and might even be put in the form of a picture or cartoon!), giving people the take-home-point right away.
- Definition(s). People want to know what "peer learning", "P2PU", and "paragogy" are. Even if we were submitting to an education-specific conference, many people might not know these terms. For OKCon, most won't have a clue unless we're explicit. And (by the same token), we need to tell these people why they should care (we did this a bit in the first paper when we talked about "peer production".