In 2017–18, 2,500 volunteers counted tiny pores on ginkgo leaves to study the ancient atmosphere; 24,000 classifications helped process the images

Ginkgo trees have barely changed in 270 million years, which makes them a strange kind of time capsule. The leaves have small openings known as stomata, which serve as breathing points for the plants. According to astudy led by Laura Soul and colleagues at the Smithsonian Institution's National Museum of Natural History, the density of such breathing pores changes according to the amount of carbon dioxide in the air; hence, scientists can examine the fossilized leaves of ginkgo and get an idea of the state of the atmosphere millions of years back. The study noted that the slowest part of this work is simply counting the pores in each leaf.

How a leaf pore reveals old air

The measurement scientists rely on is called the stomatal index, which compares the number of stomata to the total number of surface cells on a leaf. Previous studies had indicated that ginkgo trees develop a smaller number of stomata compared to other cells under higher carbon dioxide concentration, and a larger number in cases of lower carbon dioxide concentration. This makes it possible for the researchers to work backwards from the leaf fossil to the level of carbon dioxide. This method was improved upon in a 2016 study by Richard Barclay and Scott Wing, the same Smithsonian scientists behind the Fossil Atmospheres project, who examined the accuracy of the ginkgo-based method for estimating past carbon dioxide.


Making a boring job into a public project

Counting stomata under a microscope can be a slow process. The lead scientist needed around 14.5 minutes for each image, and there was a huge pile of images waiting to be processed. To make the process faster, the Smithsonian scientists turned to Zooniverse, a website designed by scientists from the University of Oxford and Adler Planetarium in Chicago that allows citizens to participate in scientific research. Before opening the project to the public, the team ran an early trial round, in which the first participants had difficulties with certain images. So, the scientists added an "unsure" option and prepared a training set, so that by the time the official test phase began on December 20, 2017, volunteers already knew how to count. The volunteers didn't need to have a degree in science for this; just time and interest were enough.

Thousands of people, tens of thousands of counts

Researchers launched that official test phase with 100 test images on December 20, 2017. By March 18, 2018, each image had been classified by 50 different volunteers, offering 5,000 classifications to compare with the expert's own counts. By the time the paper was written, over 2,500 participants had been involved in total, with over 24,000 classifications made over 2,424 images. That’s a much lower average of roughly 10 classifications per image, well below the 50-per-image standard used in the initial test set, suggesting the requirement was relaxed as the project scaled up.

The data shows that volunteers took an average of 21 minutes per image, which was longer than the 14.5 minutes taken by the lead scientist, which researchers attributed to the fact that volunteers had no prior training in counting cells from plants. Even so, because thousands of volunteers could work on images simultaneously, the project processed the overall backlog far faster than a single expert working alone, even though each individual volunteer was slower per image.

How close did the volunteers come?

The results were a mixed bag. Volunteers' stomata counts specifically held up well against the expert's: they got exactly the correct number of stomata as did the expert in 36% of the classifications, and were accurate to within one stomatal pore, a category that includes those exact matches, in 64% of classifications overall. The bigger source of error came from a separate measurement: counting epidermal cells, the other component of stomatal index, proved difficult. In less than 1% of the classifications, the volunteers got exactly the same number as the expert and understated the number of these cells in 92% of the classifications. As the stomatal index is the ratio of the stomata to the total number of cells, understating the number of cells increased the index. On average, the stomatal index estimated by the volunteers was 7.81 units higher than that of the expert, based on the same image.

Fixing the numbers without wasting anyone's time

Rather than discarding the information, various ways of correcting the data were attempted by the group. Excluding classifications where the participant had marked either one stoma or one cell, basically flagging that they hadn't completed the task, cut the average error down to 4.46 units, a reduction of about 43%. Filtering for incomplete work proved more effective than any other method the scientists attempted, like considering only registered users or interpreting an uncertain mark as a cell, neither of which were particularly helpful.

This is quite an interesting observation about how ordinary volunteers can actually provide useful, but not necessarily perfect, data. This is something similar to what was mentioned by Joe Cox and colleagues in their 2015 research that explored factors for successful Zooniverse projects. The Smithsonian team said they planned to try methods that combine markings from many volunteers on the same image, because people tend to miss different cells.

Why this project matters beyond ginkgo leaves

The authors write that the primary objective is understanding Earth's climate history. Reliable CO2 measurements from ancient warm periods, 50 to 100 million years ago, including even short hyperthermal events, provide a natural means of comparing modern-day climate change with what happened before. Fossilization of Ginkgo leaves works well enough to give us a good record for the long run if we can rely on the counting method. Moreover, the project has also provided an opportunity for thousands of non-scientists to have a taste of climate science work and to turn a routine lab task into a shared public effort.