7 Comments

I have trouble referring to Kaufmanns study as scientific. Yes he did analyze some set of data. But is that all the data we have? Was the structure worthy of a scientific study (discerning between method, results and discussion at least)? Was the study peer reviewed? Was the study published in a journal?

Who funded the study? Where does it show transparency in possible conflicts of interests? We should not believe a sugar study that was funded by a sugar company, so does its CSPI funding not ruin tis credibility? Why are not the primary sources cited? At least one of the citations does not even work.

What method did he choose in selecting the data? Wher eis the transparency? From which theoretical standpoint did he operate? Which philosophical assumptions did he make?

I don't want to say this automatically makes the study useless (rememebr the fallacy fallacy). But it is kinda strange. There is too much structural and methodical critic to call this scientific. Why do you call it scinetific?

Expand full comment
author

This wasn't really a post about Eric's study?

Expand full comment
Jun 6, 2022·edited Jun 6, 2022

You mentioned it and rated it (at the very least you took it for face value), so i thought that is a good excuse talk about it. What do you think about it?

Expand full comment
author

Yes, as a segue to the most important example of peer contagion.

But since you ask, the bulk of the survey data seems to come from highly reputable sources, like the GSS. I don't need to peer review to see that LGBT identification has gone up from 2008-21 according to the General Social Survey. Peer review is also not a very good system when there's next to no viewpoint diversity on contentious issues.

That's all I can say for now.

Expand full comment

It is the best system we have atm. Even for the most unpopular research ideas this is a method that is working. Even while gatekeeping is existant, i would bet that there are enough journals that are open to this view. Especially if there were no methodical flaws.

Expand full comment
author

It's clearly not the best system in the disciplines where not one openly Republican professor can be found. The best system there is to write a report and have people scrutinise it openly. If you've been in academia, you know that it's relatively easy for a report like this to be spiked in peer review. So be thankful Eric was able to write it and get it out there so we can even have this conversation.

Expand full comment
Jun 6, 2022·edited Jun 6, 2022

Clearly? I would say its very far from clear. If you advocate that everyone can scrutinise it, then i ask you to show me how "clear" this really is.

If you limit it to peers: There are many types of peer reviews and we are clearly not sure which is the best. They all have they advantages and disadvantages. There are already some experimental forms that are similar to what you proposed. And generally the open peer review method comes very close to it too. Generally i find double blind much better since, you don't have to fear ruining your sympathy with the author. An double blind system that is openly sharing all the documents related to the review process to the reader seems to be ideal. From the studys i have read about this, there is still no evidence that this is more effective.

Anyways. That the comment section in the Kaufmann report is closed makes me worry! Not allowing any review is the worst you can do.

There aren't any republican professors in Politics?

Expand full comment