Just about the whole world now relies on online search engines to look up information, compare products and services, and search for … anything really!
To ‘Google’ has become a verb, the behaviour of looking something up online has become the norm for many in the quest for data and some 8.5 billion queries are made to Google every day.
But how do search engines work and how is best to take advantage of them to benefit your business?
What is a Search Engine?
When asked what a search engine actually is, many would simply respond with the name ‘Google’, but in fact there are many search engines and while all do slightly vary, their premise is the same.
A search engine is a software system designed to allow users to search the internet for specific information. This is usually done via a textual web search query, in which the user types keywords into the search engine and is presented with a line of results on a SERP (Search Engine Results Page). The results are usually provided ranked with the most likely to be accurate and useful for the user’s search queries.
While what we think of as search engines are often websites such as Google and Yahoo, such software also exists to search through the content of individual websites and databases.
What Happens when a Search is Performed?
When a search is performed, the system reviews its index of web pages to find those it believes are relevant to the search. This judgment is carried out by a program known as a search engine algorithm, which identifies the most relevant results to the search query and then presents these search results on a SERP.
In most cases, the search engine will review its index of web pages regularly in order to ensure it is always presenting the most up-to-date content possible. It will also update its algorithm often in order to maintain current technology and continuously improve the results presented.
The time between the search query being input and the web page index being scoured for relevant results usually takes just seconds; and often just a fraction of a second.
What is a Search Engine Algorithm and What is its Purpose?
Lots is spoken about the algorithms used across search engines and social media but it seems that genuine understanding of them is very low.
A search engine algorithm is a computer program that runs on a collection of formulas to determine the quality and relevancy of a particular web page or advert to a keyword or phrase that may form part of a user’s search query.
The purpose of an algorithm is to provide the most appropriate search results possible for the user, which in turn result in the users being more likely to return to the search engine again and it being able to gain competitive advantage above others. For this reason, search engine algorithms are frequently updated and improved: with Google updating theirs an estimated twice a week minimum!
What are the Most Popular Search Engines?
Google holds over 92% of the world’s search engine market share, but there are other search engines that hold significant share in particular geographic locations. Bing, owned and operated by Microsoft, is considered the nearest competition to Google and holds around an 8% market share. Baidu is the third largest globally, but this is largely due to its dominance in China, where it holds around 73% of the total market. Generally speaking, any business looking to market in China or the surrounding territories needs to ensure they have tailored their search marketing practices to Baidu rather than Google.
Other major search engines include Yahoo, Yandex, Ask, DuckDuckGo, Naver, AOL and Seznam.
As a result of Google’s market dominance, most business’ search marketing and SEO (Search Engine Optimisation) efforts focus toward it and it’s considered the status quo in the western world to tailor search strategies exclusively for this, with other search engines not considered a priority.
How Do Search Engines Differ?
While all search engines do run on the same basic premise of indexing websites and then carrying out the analysis of these websites using an algorithm to present appropriate results, they do vary in the way in which they carry out this analysis.
Google’s algorithm has developed hugely over the last ten or so years and now lands its focus on matching up the intent behind the search being made and the results, whereas Bing continues to spotlight on the keywords being used as a baseline.
Yahoo has been powered by Bing since 2011 and so displays search results using much of the same criteria as it. Yahoo places more importance on domain age than Bing, but similarly focuses on keywords and meta data.
In China, Google and all Google products are banned; leaving Baidu as the most important search engine in the country. While Baidu’s algorithms can be considered not entirely dissimilar to Google’s (there is a large amount of AI and intent-based calculations utilised), the system behind Baidu actually analyses web content to a higher degree than Google and so can take a little longer to update initially. Baidu really only works well in simplified Chinese and prioritises those domains registered in China above anything not local to the country or its borders; and is very heavy on censorship.
How Do Search Engines Provide Search Results So Fast?
Search engines are able to scour millions of sources for analysis and ranking in a fraction of a second in order to present adequate and appropriate search results, and the key to this speed is all based around its ability to index.
When a search engine crawlers ‘crawl’ and index the web, they essentially create a database for that search engine to use. This negates the need for them to search the entire internet every time a query is made and instead provides their own resource to search through.
Consider being asked for a chow mein recipe, rather than go and take out every book in the library on Asian cuisine and leaf through them, you can grab the most appropriate title, skip to the index and look up ‘chow mein’ under C. With everything already well categorised, it’s considerably easier and quicker for the search algorithms behind-the-scenes to categorise and analyse the information at hand.
Google won’t comment on the exact size of its indexed internet records but has admitted that it’s over 100 million gigabytes in size, and realistically, it’s probably several times that. To give you an idea of how big that is, it’s 100,000,000,000,000,000,000 bytes! However, it’s worth noting that only a handful of people employed by Google have a true idea of the exact technology behind the Google index, and many of those working on search algorithms aren’t informed on how all areas of the business work together to produce the results they do.
How to Tell Search Engines to Crawl your Website
For a website to appear on a search engine, it must be included in the system’s ongoing list of indexed pages; or it is not able to analyse it for relevant content when a search query is made.
Firstly, any ‘noindex’ tags from the coding and programming of the website must be removed. These tags direct search engines to bypass the domain’s content and so don’t allow search engine bots to ‘crawl’ the pages and add it to their index. Businesses may wish to keep these tags in place until their website content is complete, but at this point should remove them entirely.
In order to optimise the way search engine algorithms are able to crawl a site, it should have a site map and a full robot.txt file.
For Google indexing, websites can be submitted directly to the search engine using the Google Search Console. Domain ownership will need to be verified.
For Bing indexing, websites can also be submitted directly through their Bing Webmaster Tools: and if a Google Search Console entry already exists, this can be copied straight across. Submissions to Bing automatically index to Yahoo, too.
DuckDuckGo automatically indexes the internet and so doesn’t require any formal submission, although it prioritises those results on Bing, and so if a Bing submission has been made, it will be picked up quicker than those not listed on that system.
To index a site to Baidu, a localised site in simplified Chinese with either a .com or .cn domain will need to be created. This can then be submitted directly through Baidu Webmaster Tools.
Where to Get Help with Search Engine Optimization
If you’re not sure which search engine is best to target your SEO and search marketing practices toward, or think that you may need to branch out a little with your efforts, get in touch with Woya Digital. Our in-house team of SEO experts can help advise, guide and draw up a strategy to maximise exposure and success online.