Google published a detailed explainer about how changes to search rankings are evaluated internally before being rolled out to users.
Danny Sullivan, Google’s Public Liaison for Search, goes over the role of search quality raters and how their evaluations are incorporated into algorithm updates.
Thousands of changes to Google’s search algorithms are made every year, and they all go through a process to determine whether they will be useful for searchers around the world.
Here’s are some ways that feedback from quality raters and regular people are used in algorithm updates.
Google’s research team talks to people around the world to understand how Search can be more useful:
“We invite people to give us feedback on different iterations of our projects and we do field research to understand how people in different communities access information online.”
Ultimately, Google’s mission is to make information universally accessible and helpful, with a commitment to serving all users in pursuit of that goal.
Search Quality Raters
Google has publicly available rater guidelines that describe how its algorithms intend to surface content.
The guidelines are over 160 pages long, but if it were to be narrowed down to single phrase then Sullivan puts it this way:
“… we like to say that Search is designed to return relevant results from the most reliable sources available.”
Many signals are picked up by Google’s algorithms automatically. But when it comes to signals like relevance and trustworthiness, those require human judgement.
Google has a group of more than 10,000 people all over the world who are