visualizing the quant-qual divide in sociology

The blog CWTS has a post that uses bibliometric data to map how quantitative and qualitative research segregates in sociology:

Perhaps we can conclude from this small foray into the quantitative-qualitative divide that particular research topics are often confined to one method. A qualitative approach yields a richer, thicker description, and embeds an analysis in a wider context. At the same time, it may trigger questions that require a more quantitative answer, which in turn may require again a more qualitative analysis. We may thus continuously switch between qualitative and quantitative methods. Rather than trying to integrate the two, which is for example promoted under the heading of mixed methods, we should perhaps mostly keep challenging both views from the other perspective. We should not be blind to the challenges posed by the other perspective, but accept that the other perspective can supplement and nuance our conclusions, rather than invalidate them. Fortunately, when looking at the distribution of publications in journals, some of the more general journals, such as American Sociological Review and American Journal of Sociology do include publications from both perspectives (although the quantitative perspective seems more present). More specialised journals, such as Cultural Sociology and Social Forces, mainly focus on respectively qualitative and quantitative research. At least, there are some common fora for discussion, but there is room for improvement.

It seems as though the authors did not use any type of formal layout algorithm for these graphs- if the nodes/terms were moved around by hand to highlight the divide, might the quant/qual divide appear much worse than it actually is?

I’m with Chris. I can’t get their Java thing to work, but I suspect they forced the data into two clusters and then set their parameters to maximize cluster distance. Additionally, to the extent they are capturing different worlds, I think it is non-US vs US sociology. The words on the left look like a combination of qualitative words plus words associated with Giddens/Beck/Foucault/Bourdieu.

Hi all,
I’m one of the authors of the blogpost on the CWTS-site (https://www.cwts.nl/blog#). Just wanted to answer some of the questions here.
@olderwoman there is no paper (yet) but you should be able to find the entire text under ‘blog’ at the CWTS-site.
@Chris and Neal. I don’t think it answers all your questions but the blogpost includes a paragraph on methods at the end. I hope you will appreciate it a bit more when you know the methodology behind it:

Method
We collected all articles published in journals that belong to the subject category of Sociology in the Web of Science, published between 2010-2015. In total, there are 14,613 articles published in that period in 146 different journals. We input the titles and abstracts into the VOSviewer, which identifies terms from the text (technically known as noun phrases) using a natural language processing algorithm. We then exclude terms that occur less than 20 times in this corpus, which results in 2450 terms. Using these terms, the VOSviewer calculates a so-called relevance score to exclude generic terms. For example, a term such as “conclusion” appears frequently, but co-occurs with almost any other term. Such generic terms are thus excluded based on a low relevance score. We select the 60% most relevant terms that remain, which results in 1470 terms. See Van Eck et al. (2011) for more technical details on how these terms are extracted and how relevance scores are calculated, and see Van Eck & Waltman (2010) for a more general overview of the VOSviewer. The overall quantitative-qualitative divide seems quite robust with respect to slightly different choices of parameters. For example, using a cut-off of 15 occurrences and selecting the 1500 most relevant terms reveals a nearly identical map, and so does a cut-off of 25 occurrences and selecting the top 1000 most relevant terms.

The terms are visualised using a method that tries to position frequently co-occurring terms close to each other. For those with some technical background, the technique resembles a weighted variant of multidimensional scaling. More information can be found in Van Eck et al. (2010). Finally, the terms are clustered using the same mathematical framework as used for the visualisation, as explained in Waltman, Van Eck & Noyons (2010). The clustering technique resembles modularity, a well-known method for detecting communities in networks, but there are some subtle differences. In particular, different granularities of clustering can be found, and we need to set some resolution parameter. The default setting in the VOSviewer is 1, which still identifies some subtopics in the field of sociology, such as gender issues. The coarser division between quantitative-qualitative is revealed at a resolution somewhere in the range of about 0.55-0.85. Finally, we normalise the frequency of a term in a journal so that the average is equal to 1, to arrive at comparable scales for the colours (we visualize on a common scale of 0-2).
– See more at: https://www.cwts.nl/blog?article=n-q2v294&title=revealing-the-quantitative-qualitative-divide-in-sociology-using-bibliometric-visualization#sthash.WqCQHgoT.dpuf

It’s frustrating that you can’t get the Java to work, especially if you want to explore it further. If you could send us what error you get, we’ll look into that. You can fill out the form at http://www.vosviewer.com/contact, and send me a mail.

Thomas already quoted the method section, so everything should be clear from that. But I just like to emphasise that everything is done algorithmically: term extraction; layout; and clustering. We didn’t choose any parametrisation that influences the layout to emphasise the divide. There is no manual involvement or editing of the layout in any way. Nor are any words chosen manually, they are extracted through text mining techniques. The only thing is that some parameters need to be set, as explained in the method section.

@Chris, perhaps US vs. non-US sociology plays some role here. But I don’t believe that US sociology doesn’t use “practice”, “notion” or “discourse” and that non-US sociology doesn’t use “data”, “survey” or “association”. So, although there may be a distinction between US and non-US sociology, and it may play some part, I don’t think it’s the main explanation of the divide we are seeing.

Just a tip to the authors of the blog post: Thomson Reuters’ definition of the universe of sociology journals is very sloppy. It includes a few journals that no (OK, no more than one or two) sociologists would claim as sociology, consider publishing in, read, or cite: for example, The Annals of Tourism Research (one of the top ten “sociology” journals by impact factor!), and the Cornell Hospitality Quarterly. With a large n, it’s probably not going to make any difference, but that’s an empirical question. The more general point: the boundaries around the population matter.

Good point! I think the majority of the journals would still be considered sociology though, but I would have to check more in detail. It might be interesting to take only those journals that are uniquely assigned to the subject category of sociology, and not also to others, such as tourism. Then again, that also excludes other journals which some might consider to be part of sociology, especially those assigned also to anthropology. In general, delineating any field is likely to stay problematic and debatable, and boundaries are never as clear cut as you would like. But I think that the overall picture of quantitative-qualitative will remain by and large the same. Anyway, as you say, it remains an empirical question!