The Claws Come Out: The Syuzhet Debates

This week’s readings on big history and humanities computing were both informative and fun, as I enjoy watching scholars critique each other and, sometimes, get catty. The syuzhet debates read something like this:

1) Jockers introduces syuzhet, a package for R that studies plot shifts through sentiment analysis. In the first two blog posts, he details the Fourier transformation, Euclidean distance, how plot shape is derived, and the distance matrix.

2) Swafford writes a blog post in response that further describes the algorithm behind syuzhet, which works by splitting a novel into sentences; assigning a positive or negative number to each sentence; and smoothing out the numbers to get a foundation shape of the novel. She then discusses the various problems she ran into while working through syuzhet, including having the package incorrectly interpret multiple sentences as being one sentence; not graphing emotional valence of a text and instead creating graphs of word frequency groups by theme; and ringing artifacts creating the six or seven plot archetypes rather than those archetypes resulting from similarities between the emotional structures of the novel.

3) Jockers responds to these critiques in another blog post. He writes that the tool does not have to be perfect, just “good enough.” While he maintains that he is sympathetic to Annie’s position on the sentence level precision of syuzhet, he thinks that for this case it doesn’t really matter. As long as the overall shape is reminiscent of the known sense of the novel’s plot, then the tool is working.

4) Swafford responds to the “good enough” claim in a–yup, you guessed it–blog post. She makes two conclusions: foundation shapes are not the right tool to use since they do not always reveal the emotional valence of novels; and¬†benchmarks are needed in order to evaluate syuzhet properly. In another blog post, she points to specific examples that Jockers had provided and show how they are not true illustrations of syuzhet’s success.

5) Jockers responds with the final blog post, writing that he believes the true test of the method lies in assessing whether or not the shapes produced by the transformation are a good approximation of the known shape of the story.

Swafford pointed out several serious issues with the package, and Jockers responded by mansplaining and ultimately looked like a very sore loser. Despite this, several good points were brought up by others who blogged in response to this debate. Andrew Piper discussed the issue of validation and argued that humanists need their own particular form of validation. Scott Enderle wrote about Fourier transformations and believes that ringing artifacts are necessary. Ben Schmidt found that Fourier transformations are not the correct smoothing function to use for plots.

This debate really showcased how gender plays a role in digital humanities. In the first week’s readings, we read Miriam Posner’s article “Think Talk Make Do” about women and coding in DH, and I think we have come full circle. Jockers’s responses to Swafford came across as one of the best examples of mansplaining I have ever seen. I really felt that he was condescending and looking down at Swafford because of her gender. Women can code just as well as men can, and Swafford had every right to pick apart syuzhet and examine the algorithm for problems. She found some serious issues, which I felt that Jockers brushed off by saying the package was “good enough.” And speaking of “good enough,” is that what we really want our tools to be? Do we want to work with something that is “good enough” and use that to analyze large quantities of data?

Leave a Reply

Your email address will not be published. Required fields are marked *