First thing first. SERP means Search Engine Results Page.
SERP refers to the results shown by a search engine in response to a given keyword or key phrase. In the field of Search Engine Optimization (SEO) it is extremely important to keep an eye on your SERPs. These pages tell you a lot about how well formed is your SEO strategy as well as it gives you some idea about how users may reach your website.
Look at the following SERP from Google for the key phrase car insurance in new york. I took this screenshot on Jan 14, 2013. Look at the results and see how they are stacked. Also consider for a moment; if you had made this search which of the results you’ll prefer to click.
This search on Google will show all the pages of kipzer http://www.kipzone.com listed in Google.
When you look for your SERP you should notice the following items:
Title of your web page
These three things are key to success. SEO folks mostly optimize these three things for a given website. Title is very important because this is the first thing that a searcher notices in the search results. In the above given screenshot, you can see that Google makes the used keywords bold in title and description. Therefore, your title and description must contain the keywords you want to target.
Once a searcher fixes her eyes on a title –she tends to look at the description. Description also contains highlighted keywords. If the description is useful and catchy –the searcher might decide to click on your link. Description is very important but it is oft neglected by the webmasters.
The URL should also contain keywords. Most searchers won’t look at the URL but it is very important for search engines. They give preference and higher placement to the results where keywords are also present in the URL.
A search engine results page (SERP) is the list of results that a search engine returns in response to a specific word or phrase query. Each listing includes the linked Web page title, the linked page URL (Uniform Resource Locator), a brief description of the page content and, in some cases, links to points of interest within the website.
There are three main types of results on a SERP: Pages that the search engine spider has crawled and indexed; pages that have been manually added to the search engine’s directory; and pages that appear as a result of paid inclusion. The highest-ranking hits generally link to the most useful information; links grow less relevant as they move farther down the list.
Depending on the number of Web pages that contain a particular word or phrase, a SERP might show anywhere from zero (in the case of no matches at all) to millions of items. For example, entering the phrase “complex-number admittance” into the Google search engine yields few results. In contrast, entering the single word “hurricane” yields millions of results.
In most cases, search engine users will look at only the first one to three pages of hits (results). Web designers and site owners use search engine optimization (SEO) methods to make their sites and pages appear at or near the top of a SERP.
Your websites Search Engine Results Pages (SERPs) are determined by various SEO factors, on page content (your body text, the title tag etc…), anchor text of links to your web sites pages, the search engines algorithms tweak of the day and of course the number of websites/pages competing for that search phrase: are there a lot of webmasters targeting the phrase.
In this section of the SEO Tutorial we’ll deal with estimating the number of competitors for a Search Engine Results Page (SERP).
Determining your Search Engine Results Pages SEO Competition:
The first mistake a lot of amateur SEOs (and professional SEOs for that matter) make is looking at total number of pages found for a given SERP and from this limited information concluding the difficulty and number of competitors for that search phrase.
This method is inaccurate and should not be relied upon, for example the word HOME is found on a lot of pages (5,960,000,000 pages!!), not because they are targeting SERPs related to the word home, but because they use home as the anchor text of home page links and the word home is quite common: there’s almost 6 billion pages indexed in Google for home!
Check out the ‘SEO Home’ SERP, 318,000,000 pages found with a loose search (just type the keywords and search, no speech marks etc… so a loose search) for SEO Home, which would suggest a difficult SERP with quite a lot of competition. But this loose search alone tells us nothing about real competition for the SERP, what we need to know is of those millions of pages how many are targeting or at least mentioning the exact phrase SEO Home, those are your real SERPs competition.
The top SERP result is for a site about education: Sponsors for Educational Opportunity:
Sponsors for Educational Opportunity (SEO) provides superior educational and career programs to young people from underserved and underrepresented communities to maximize their opportunities for college and career success.
It’s interesting because no SEO expert would ever target a SERP like SEO Home, no one is going to build backlinks with anchor text “SEO Home”. Guess it’s a good thing it’s not an SEO site at number one.
Keyword Proximity Suggests True SEO SERP Competition
An exact phrase match search is where you enclose your search phrase within speech marks (“search phrase goes here”) when making a search in Google. An exact phrase search removes all loose matches (see below).
An exact match would be the bold text below-
1. ‘At SEO Home search engine services we deliver great results’
A loose search match would be text like this-
Keyword proximity (how close the keywords are to each other) is important to Google SEO, as you can appreciate an exact phrase match (like text #1 above) to a search engine is considered much more important than finding the keywords separated (you can’t get a better keyword proximity than next to one another), or a loose match (like text #2 above).
Two similar pages with the above strings of text and no other mention of the keywords SEO and Home you would expect to find the page with text number 1 way above the page with text number 2 for the SEO Home SERP.
We basically assume web pages with perfect keyword proximity for the given search are targeting the SERP either by choice or by luck (not all webmasters are SEO’s).
When you perform the exact phrase match “SEO Home” it gives 119,000 pages. This tells us the vast majority of the 318,000,000 pages found for SEO Home (loose search) are not interested in the SEO Home SERP; only 119,000 pages (less than 1%) are your true competition since they use the exact text on their page.
That is not to say a loose matching page or even a page with some or all of the keywords not present at all can’t rank highly since anchor text alone from high PageRank (PR) links or lots of links with the right anchor text (like the Miserable Failure Google bomb SERP many years ago) can beat exact phrase matches.
Interestingly the same Sponsors for Educational Opportunity site is ranked number one for the exact search and some of the top ranked SEO sites (they are ranked for loads of SEO phrases) are NOT in the top 10. Couldn’t find Search Engine Watch in the top 50 for this SERP, they were number 2 for the loose search because of a combination of a lot of links with anchor text SEO and the text home used within body content etc…
An Example SERPs Competition Calculation
We’ll use two SERPs that at first glance appear to have similar levels of competition, at least from just looking at the total number of pages found in Google with a loose search (the inaccurate measure of competitor pages). Then using exact phrase match searches to show how one is relatively easy and the others is relatively difficult.
This section was written in 2006, since then I think the MSN Today service has closed down (or at least changed: the original MSN site for MSN Today signup no longer exists), so no longer a good SERP for SEO analyses, will have to find a new one when I get the time.
The SEO concept and analysis is still valid, only problem is you can’t do the same SERP checking I did many years ago because the SERPs have changed significantly. Real shame as it was a really good example.
It’s very interesting to see the significant increase in pages found for all my example SERPs, I’ve edited the earlier ones (in June 2014) and some went up a lot. For example loose search for SEO Home was 5,360,000 pages, now it’s 318,000,000 pages (from 5 million to 300 million!!!. Exact search for “SEO Home” was 13,900 pages, now it’s 119,000.
That’s an enormous increase in pages Google is indexing today relative to 2005/2006: no wonder SEO is so much harder today, SEO competition is fierce. Even though it looks like MSN Today was closed years ago (was part of Windows Live Messenger, MSN Today was a website that appeared in a small browser or “trident” window.) there’s still almost 2,000 Google searches for MSN Today a month in 2014. So though there’s a lot more competition for all SERP, there’s a lot more visitors as well and they search for weird stuff 🙂
- MSN Today : 9,850,000 pages
- Psychology Today : 8,700,000 pages
Based on this information alone MSN Today should be the harder SERP by a small margin, more pages using those two words should mean more competition.
As you can appreciate an exact phrase match to a search engine is considered much more important than finding the words separated (loose match). So finding a text match like-
1. “At MSN Today you’ll find a search engine hardly anyone uses”
Will be considered more important for the SERP MSN Today than the loose matching text below-
2. “Earlier Today the search engine MSN crashed due to excessive use from the alt.internet.search-engines newsgroup community!”
Both have the two keywords, but number 1. will be considered more relevant (all other things being equal of course) than number 2. This doesn’t mean number 2 can’t show high for a search for MSN Today, but without supporting anchor text (links using the anchor text MSN Today or derivatives) it is highly unlikely.
Exact Phrase Match Searches
“MSN Today” : 10,600 pages
“Psychology Today” : 1,150,000 pages
- The exact phrase “MSN Today” is found on only 10,600 pages out of almost 10,000,000 (just 0.1%).
- The exact phrase “Psychology Today” is found on 1,150,000 pages out of almost 9,000,000 (~15%).
For the purpose of determining real SEO competition for a SERP you can all but ignore the loose search matches.
The real competition for MSN Today is just 10,600 pages (not that many) since the remaining 9.5 million won’t have enough content relevance to compete effectively (unless they have significant links using the right anchor text as well, which is unlikely).
So MSN Today is most likely a relatively easy SERP, especially in comparison to a SERP like Psychology Today where there are 1.15 million pages with good content relevance to that SERP (this could be a difficult SERP to obtain).
Other Factors Impacting on SERP Difficulty
As you saw with the MSN Today example above from the millions of pages found with a loose search only 10,600 are using the exact phrase. This information is vital to determining the difficulty of a SERP, but does not always reflect true difficulty since if there are just 10 pages online targeting a SERP and they all do better than your page you don’t get the prized top 10 listing.
For a search phrase to provide traffic to your site you ideally want one of the top 5 listings and no less than top 10 (first page result). Other than very high traffic SERPs after the 10th result traffic practically stops and so you need to look at what actually has the top 10 listings for a given SERP.
Keeping with our MSN Today example the first two pages are-
1. http://www.msn.com PR9 with 815,000 backlinks in Google with many using the anchor text MSN.
2. join.msn.com PR7 with 130 backlinks in Google with many using the anchor text MSN.
Even though there are only 10,600 pages listed for this SERP it would be very difficult to beat these two pages, they have high PR and more importantly a lot of the links will use one of the keywords of this SERP (MSN).
To beat these two pages you will need good on site optimization and quite a lot of links (probably enough to make your page PR6) using the anchor text MSN and Today (not necessarily together). You will notice you don’t need as many links as the two pages above since neither of them is optimized well for the phrase MSN Today. They are optimized for MSN though, so the lack of optimization for the Today part makes obtaining the SERP a little easier (but still difficult).
MSN Today SERP Update: July 1st 2005 this page is 6th for the MSN Today SERP. Not bad for no links using that as anchor text or even basic on page optimization. I’ve added a couple of links to the page now with anchor text MSN Today, top 3 is a real possibility.
Quick SEO Tip – If you want to ONLY see the loose matching pages of a SERP use this search format in Google:
Search Phrase -“Search Phrase”
This search will only show pages that have both keywords Search and Phrase, but not in the format Search Phrase (so NOT perfect keyword proximity).
A few SEO caveats about SERPs and how some SEO’s manipulate potential clients. You will find on a lot of SEO company websites lists of SERPs the site or their clients sites rank well for in Google and other search engines. Read through these SERPs lists carefully you will almost always find they consist of really easy search phrases, SERPs that you would get with little to no optimization.
One SEO firms web site I visited recently listed phrases like-
And stated they have top 20 listings in Google for them all. Sounds good must be pulling in thousands of targeted search engine traffic, but look closer only the 1st two (in bold) are of any real benefit to a SEO firm, the rest will not result in much if any traffic. They are listed to ‘bulk up’ the SERPs (make it look good to a potential client). The reality is the majority of the above SERPs will result in a handful of visits a month, not exactly the SEO benefits the firm wants to portray!
So when you see a list like this go through it and see if this really is a successful site or just flim flam.
Here’s a few worthless top 10 SERPs this site had in Google (many years ago as I update, not going to add new ones as they’ll tend to come and go over time since not specifically targeting them)-
Some of these sound fine, but I got them from the August 2014 web logs, specifically looking for phrases that have resulted in less than 10 visits that month. So as we’ve listed SERPs above that comes to less than 200 search engine visitors per month (closer to 80 visitors a month). Not very impressive when you do the maths.
Three key components of SERP :
- The User’s Search or Query – This is the thing a searcher types in. In the example above, someone is looking for information or products relating to Apple computers.
- Organic SERP Listings – These are the “natural” listings. To produce these results, Google uses a series of metrics to ensure that your site is relevant. To have your site show here, you have to effectively “score well” on that algorithmic test.
- Paid SERP Listings – These are advertisements, or “sponsored links.” You can have your ad displayed here by launching a pay-per-click search campaign, and by ensuring that your maximum CPC bids and Quality Score allow you to secure a high enough ad position for the keyword you’re targeting.
Why Do SERPs Matter ?
Because ranking your site in the proper SERP position will be the difference between success and failure in your search marketing campaigns. Consider the following insights provided by these studies:
- 68% of searchers select a result on the first page of search results
- There is a disproportionate number of clicks (around 40 percent) on the number one listing
- Reaching the first page in paid search is equally important
How To Improve Your Search Engine Ranking Page Positions
Improving your SERP position can be done in a few (often not so easy) steps:
- Research Keywords – You first need to discover search engine keywords that will drive traffic AND actual business.
- Organize (Group) the Keywords – You then have to create a logical hierarchy for these keywords; you need an effective information architecture for SEO, and you need to create tight Ad Groups and well-organized ad campaigns for PPC.
- Manage Your SEO Workflow – You need to be able to determine which areas will give you the best return on your time and resource investments; ensure you’re not spending months building out an SEO campaign to a set of keywords that won’t convert for you and your business.
- Manage Your PPC Workflow – This works in much the same way that SEO workflow does: you need to prioritize which Ad Groups and campaigns will be worth the effort of writing ad creative and developing landing pages for.
- Act On the Analytics – You then need to do something with all this keyword data and prioritization: you need to write the Web copy, create the ad campaigns, actually achieve the SERP positions you were after.
- Observe the Results, & Repeat – Finally, you need to develop processes for iteratively improving upon the results you’ve generated. You need to automate certain aspects of your campaigns, and ensure that you are continually having new keywords researched, workflow dynamically suggested, and that you are equipped with the tools to perpetually build on this initial “SERP improvement to-do list”.