Welcome to History of Data Science. Discover the stories of heroes who transformed our daily lives!

BROUGHT TO YOU BY Dataiku Dataiku

xperiences-ico Xperiences
Charles Spearman: A Soldier Turns to Stats
Mathematics / Statistics

Charles Spearman: A Soldier Turns to Stats

4 min read
Charles Spearman got a late start, but he finished strong. Not only did he reshape the study of psychology and statistics, but the research tools he championed in those fields remain relevant in many others to this day.

From Soldier to Scholar

Spearman’s route to academia was unusual. Although he harbored interests of pursuing academia from a young age, he ended up joining the army at age 20. He remained in the army for 15 years, spending time in India and Burma and climbing to the rank of captain, before he left to get a PhD in psychology.

He chose to pursue his studies at the University of Leipzig in Germany, where an emerging psychology movement championed treating the field as a science, rather than as a branch of philosophy or metaphysics, as was still largely the case in Britain.

Factor Analysis

Spearman is often credited as the inventor of factor analysis. At the very least, he was the first to advocate its use in psychological research.

Factor analysis is critical to understanding the relationship between variables in a statistical analysis, particularly in psychology or social science. The general idea is for the researcher to propose a broad factor, such as “depression,” as well as a number of variables that may correlate with that factor to varying degrees (loss of appetite, suicidal thoughts, loneliness, etc). Upon gathering the data, the researcher uses “factor loading” to find out how much each variable correlates with the underlying factor.

“The general fact is that our limited intellects can only hope to deal with the infinite complexity of Nature after analyzing it down to its bare unaesthetic elements.”

Measuring Intelligence

Spearman’s most notable use of factor analysis led to his discovery of general intelligence, which he called “the g factor”. In a paper published in 1904, he observed that children’s performance in one subject had a strong correlation to their performance in another. While there are certainly kids who are lousy at math but excel in history, Spearman observed through factor analysis clear evidence of a general intelligence measure that applied across subjects.

To this day, the concept of a general intelligence factor is widely embraced in psychology, although there is disagreement about whether certain cognitive tests, such as IQ tests, fail to capture certain indicators of intelligence. In all cases, cognitive tests are regarded as correlating with intelligence, rather than aligning with it.