Fragile masculinity in adolescent boys: On "aggressive cognitions".
A perfect little exemplar of the kind of research that holds up entire worldviews and theories on masculinity.
I recently wrote a thread on X, about a great paper by Christine Reyna that highlights how the ideological homogeneity of social psychology, and the resulting liberal bias (or equalitarian bias) taints the construction and interpretation of psychological scales and other measurements.
She points out that this bias affects the questions social psychologists ask, how they ask them, and then how the responses get interpreted.
I then encountered perhaps a perfect example of this issue of measurement and the tendency for researchers to interpret results and data in a way that fits their worldview.
This paper, from a team of researchers whose lead-author’s website lays out his program of research:
The language in descriptions such as these, found at labs throughout academia, highlight how most academics have programs of research which beg the question or which have clear ideological commitments. You can read that description and know what his studies will conclude—the conclusions are made in advance, and the data are made to fit the conclusions, consciously or unconsciously.
And this brings me to the target article of this post, a recently published paper on “Adolescent boys’ aggressive responses to perceived threats to their gender typicality”.
First of all, before I get into the study and its methods, lets look at the big headline findings that the authors hit you with up front:
Remember terms like “responded with aggression” and “aggression was heightened”, or that the threat to masculinity “predicted an aggressive reaction”. We will see whether the data support such a conclusion.
The Design
The study was a simple between-subjects design, with no pre-test. What this means is that the boys were randomly assigned to one condition (no masculinity threat condition) or another (masculinity threat condition). The study involved 200 or so adolescent boys.
To start, all of the participants completed a simple quiz, and their experimental condition dictated what feedback they received. In the masculinity threat condition participants received feedback which was supposed to threaten the boys’ sense of masculinity with the feedback:
“Wow! You did REALLY WELL on the Girl Questions Quiz. You actually did better than most girls your age. Congrats!”
and
“Well… you didn't do too well on the Guy Questions Quiz. You missed more questions than other guys usually do. Based on your quiz results, it seems like you're more like the average girl than the average guy””
So this was the experimental manipulation. The “no threat” condition did not receive gender incongruent feedback.
Measuring… Aggression?
Now here is where things get silly.
Despite the headline findings that threatened boys responded with “heightened aggression” which clearly implies an aggressive behavioral response, we find out that they actually supposedly are measuring “aggressive cognition”. And I say “measuring” in the loosest possible terms.
This involved having the participants complete a timed word completion task in which:
participants saw 17 randomly ordered word fragments and had 3 min to complete the fragments. Each fragment could be completed with either aggressive or non-aggressive words... For example, the fragment GU__ could be completed as “gun” (aggressive) or “guy,” “gum,” and “gut” (non-aggressive). Because the task was speeded, participants had little time to reflect on each word; thus, their responses likely reflected implicit, uncontrolled cognitions
From the online materials, you can see the word-stems and the possible aggressive responses in brackets:
Okay, so we are measuring so-called “implicit” cognitions, a concept which has been the subject of heated debate in the social sciences, and whose foundational studies are falling to the replication crisis.
We are supposed to believe that a participant completing the word-stem “gu__” with an “n”, constitutes what the authors call “an aggressive response”.
Is a boy, playing a word game, who writes the letter “n” into a little box, “responding with aggression”? There are about 10 different explanations for why we might see some participants respond with one word or another—from personality, to life experience, to interests, recent relevant context, etc.
The word completion task doesn’t even have face validity. There is no measurement of emotion or arousal, which constitutes a fundamental facet of the construct “aggression”, nor is there any measurement of actual behavioral aggression. Just writing the letter “n”, or other letters into a little box.
The results…
This brings us to the results. Drum roll please, here is the “aggressive response”!
The regression model revealed a main effect of threat condition, b = 0.05, SE = 0.02, p = 0.013, which indicated that aggressive word-fragment completions were 4.7 percentage points more frequent among boys whose gender typicality had been threatened than among boys in the affirmation (control) condition.
A beta coefficient of 0.05. And a stated difference of 4.7% more “aggressive” word stem completions for the threatened condition.
If participants completed all 17 of the word stems, this 4.7% difference represents a difference of 0.85 words on average between the threatened and non-threatened condition. This supposedly represents an elicited “aggression response” in those boys who had their masculinity threatened.
This is an effect which could easily be accounted for by 2 or 3 participants in the threat condition responding with a higher number of so-called “aggressive” words. Without quantifying “person’s as effect sizes”, its impossible to know what this result really reflects.
Also, without a pre-test of the participants in each condition, it’s hard to know how much of this is just random noise. Given the low variability in scores in each condition, and the lack of pre-registration of clear exclusion criteria and strict design, it is highly likely that the statistical significance is just a false positive.
Motivated Cognition
It seems as if the authors here were determined to interpret these results in a way that supported their worldview and favored narratives around fragile masculinity, and how it is produced in the young by outside influence.
If you were an unbiased scientist intent on being rigorous and impartial, would you interpret those results in the way that the authors did? Even if we suspend incredulity and take the data and results at face value (that the difference was meaningful), and don’t believe they are just a result of a type 1 error, there is a much more reasonable explanation, given the issues with the validity of the measurement.
It seems more likely that upon completing the initial quiz and receiving such explicitly gendered feedback, participants likely assumed or guessed that they were taking tests of gender typicality.
As a result of their gender-incongruous feedback, the boys in the masculinity-threat feedback likely were motivated to display gender-typical traits (is a natural instinct to conform to norms and and maintain ones self perception, and also they probably thought they were “supposed” to be getting “boys” scores for the experiment). So they attempted to fill the subsequent word stems with words that conformed to a more male gender typicality (so “b_t_le” becomes “battle” rather than “bottle”).
But again, that’s assuming the difference was meaningful and not just noise.
More importantly, the word completion task can’t reasonably be considered to be a valid measure of aggression. A valid measure is one that measures what it claims to measure.
So for the word completion task to be considered a valid measure of aggressive cognition (it should be noted that the authors usually just refer to it as aggression, which seems different to “aggressive cognition”), it would need to have been shown to accurately and strongly predict actual aggression for it to be in anyway useful to make the claims made by the authors.
Regarding this word-completion measure of aggression, the authors say that
Past research has demonstrated that this measure of aggressive cognition is valid and relates to behavior
However, one paper they cite to support that statement does not even use the word-completion task as a measure of aggressive cognition—they use a timed word pronunciation task. The other is a review article which describes a past study that used the word completion task, but nowhere provides validity evidence.
Throughout the article the authors reference other works which used the word completion task, which in turn cite other works, and which all make assertions about the validity of the task, however I could not actually find a paper that showed that this task actually relates to aggressive behavior as the authors claim above.
There is no evidence anywhere, that I can see, which shows that the word completion task is a valid measure of aggression or aggressive cognition.
But remember, as I pointed out at the top of this post, the authors made broad claims about how adolescents “responded with aggression” and their “aggression was heightened” in response to a threat to their masculinity. These claims were then contextualized in terms of the “decades of experimental research has shown that gender typicality threats elicit heightened aggression among men” and of “the negative effects of gender conformity pressure”. Better deconstruct masculinity and any masculine social norms then, huh?
I was going to go further into the related aggression and masculinity literature here, but it as an overwhelming, exhausting gish-gallop of poor work, unjustified assumptions, and ignored alternative hypotheses. For every article like this one that you find flaws in, the authors cite 50 others. So you look into those studies, too, and find more flaws and more citations to other flawed work. And they’re full of unjustified assumptions and invalid measurement, weak manipulations, or suspicious analysis.
The vast majority of these papers in this area are from or rely heavily on research from the early 2000’s or even the 1990’s, a time of fraud, p-hacking, and just generally poor research, and they typically use small samples of 50 or so male and 50 or so female undergrads. Very often, as with this study, they find significant betas of 0.03, and interpret these as if they were meaningful. Almost always, the data are not available, nor detailed descriptions of the research methods.
There is no way to combat these huge bodies of literature built on flawed assumptions, flawed theory, and flawed methods, which all cite each other and support each other. There has been decades of accumulation here.
So much of psychology is like this—a clear gish-gallop technique used to build theories and support worldviews which don’t stand up to scrutiny, but which are nigh on impossible to refute because of the sheer amount of research, and the complexity of the citation laundering.
But these types of studies get attention, they earn the authors grants and awards, they make it through peer review, and they are used to support all sorts of theories and claims about masculinity, patriarchy, aggression, gender norms.
For example, the article discussed here was used as the basis of an article for The Conversation, with section headings like “The ‘problem’ with masculinity”.
They claim—following the lead of the study’s authors—that the young boys “demonstrated feelings of aggression” in response to threats to their masculinity. As we know, feelings were not measured. Nor are we even sure that the manipulation had any effect. And despite the authors claims, “aggression” was not validly measured.
The article goes on to explicitly write off the idea that masculinity and sex differences in aggression are based in biology, before moving on to a brief and shallow discussion about Andrew Tate and traditional masculinity, concluding that something must be done and children must be re-educated. The problem is that certain progressive people read shallow articles like this and then actually call for (and vote for) unnecessary interventions in schools and broad societal restructuring—all based on pretty shoddy research.
Remember, never EVER take at face value the findings of ANY study in social psychology as they are presented in research highlights or popular news publications like The Conversation. Almost the entire field is run by ideologues with clear bias.
It’s a house of cards. A progressive propaganda machine designed only to advance a certain orthodox worldview.
If you enjoy my writing and are inclined to do so, please buy me a a coffee or like share and subscribe!
Thanks for highlighting the issues with design of this study on masculinity. Another issue with the study's design is that girls weren't also tested. Thus, the study was biased from the start.
I have recently written about the current widespread use of biased methodologies in psychological research here:
Woke Academics Are Rigging Research Methods To Support Their Ideology
https://www.realityslaststand.com/p/woke-academics-are-rigging-their
@James L. Nuzzo The article Brandon mentions in his post could go into your Weekly Roundup's rubbish bin.