The goal of our experiment is to assess the impact of music on cognitive function through the performance of activities that require logical reasoning. In our experiment, subjects were exposed to terrible music (the I Love You song by Barney and Friends) and white noise music during two Sodoku puzzle challenges back-to-back. Subjects were recruited through Mechnical Turk from different states / countries, with different levels of education, different levels expertise in logical games, and different ages ranging from 24 to 67 years old. The outcome measures we analyzed related to completion rate of the game as well as number of incomplete (white) spaces left after a given time limit. The results of our experiment did not find a significant casual relationship between different types of music and the ability to perform logic-related tasks.
What makes individuals productive? Does different noise affect this productivity? Some individuals seem to work better with complete silence and no interruptions, while others need music or some level of stimulation. Others may need to read in silence, but code with loud music. Thus, stimulation needed seems not only different from person to person, but differ within each task a person performs. As we thought about a research question, we wondered if there were any generalizations that can be applied to the role of stimulation, cognitive ability, and focus. What helps us focus? What makes it harder for us to focus? What makes humans more productive on tasks at hand? The research question, holistically, is: do different sensory stimuli impact cognitive ability?
Subjects were exposed to a randomized treatment and control protocol. All valid, non-attriting, compliant subjects experienced the treatment and control protocols and played one logical reasoning game with each one.
The control protocol involved listening to a Youtube video where the visual content was not important; rather, as described to the subject, the audio was only what the subject needed to focus on while completing the exercise. The control audio consisted of an approximately 30-minute track of one minute of randomly generated white noise and then a repeating two-digit number, 67.
The treatment protocol consisted of a Youtube video similar to the control video where the visual content was not important but consisted of a song associated with the children’s television show Barney & Friends. This song was selected due to its millennial pop-culture association with being a hypnotizingly “bad” song. This song along with another two-digit number, 83, composed the approximately 30 minute-long treatment track.
The logical reasoning component of the experiment that measured the degree of progress was the Sudoku website SudokuKingdom. The experiment leveraged SudokuKingdom’s randomly generated Sudoku game boards and the researchers selected the “very easy” mode, in which 45 of the 81 squares are filled in. We selected SudokuKingdom as our platform because of its intuitive means of filling in the number in the Sudoku boards, its ability to automatically load a “very easy” level board, and its clearly visible timer. Each participant had five minutes to complete each of the two games they played.
We used Mechanical Turk to recruit participants and conduct the experiment. The benefits for using Mechanical Turk included speed, consistency, secure payment, and a diversity of subjects. With respect to speed, the primary benefit of leveraging the Mechanical Turk platform was that, similar to a MapReduce process, we could map a job (experiment) across multiple subjects simultaneously, then the results are reduced by technology and verified by the human-in-the-loop.
Mechanical Turk also provided a more consistent experience as it was agnostic to the experimenter (as we would suspect that Danielle, Osmar, or Dan, even if following a script, would vary their instructions or environment slightly, which would require another blocked variable and analysis). Further, as the data was automatically collected, errors and omissions were reduced yielding more consistent data.
Additionally, remuneration for subjects was handled automatically via the Mechanical Turk system. This obviated the need to hold money in cash, which presents additional risk for the experimenters. Further, there is a more robust paper-trail in case of auditing in relation to the experiment.
Finally, Mechanical Turk presented the opportunity to recruit subjects from a diversity of backgrounds. Whereas we would previously have been limited to engaging with local subjects in the San Francisco, Miami, and New York areas, Mechanical Turk facilitated the reach into new geographies as the platform is location agnostic. If we were all recruiting among our friend or work groups in specific locations, the chances of spillover would be higher.
The primary drawback for using Mechanical Turk regarded compliance monitoring. As we were unable to visually confirm complete compliance of our experiment protocols, we compromised and instituted several compliance checks, described below. Our research team felt that this was an acceptable tradeoff and we would work to ensure compliance with a few procedures also described below.
We chose three key ways to measure the degrees of success in this experiment. Two of these outcomes were within subject experiments and one was between subject experiments:
-
Within-Subject Outcomes:
- Completion - Finished the game within five minutes in one Sudoku game and did not finish in another
- Number of Boxes Left Empty - Number of white spaces or empty boxes on the Sudoku board in an individual game. If a subject completed the entire game, no white spaces would be left and this value would be zero.
-
Between Subject Outcomes:
- Completion of the First Game - Finished the first game or did not finish the first game. We are blocking for the treatment vs. control represented in the first game (i.e. the Barney song vs. white noise). We are also controlling for various covariates such as level of education, knowledge of Sudoku, knowledge of logic games in general, time spent playing games, morning vs. night person, and all other variables that we asked about in the beginning of the survey on Mechnical Turk.
With the data we obtained from our experiment and subsequent evaluation running longitudinal and transversal analysis we found no evidence of the impact of music on the performace of Sodoku games. We found no evidence (in statistical or practical significance) that the I Love You song by Barney and Friends had any effect on the percentage of completion rate of a Sudoku game or partial completion of the game counting the number white spaces left on the game board, when compared to white noise.
One potential reason for our lack of effect is that Barney really may not be annoying in short doses. Participants were only listening to the song for five minutes at most, which may not have been long enough to observe an effect. Conversely, white noise may also have been irritating to many of our participants. While articles have shown that white noise is quite soothing and can help cognitive function, our sample of participants may not have felt that way.