Slide 1: http://www.firstgovsearch.gov/
Search Engine Optimization Basics
Bob Keating Search Program Manager FirstGov Technologies, GSA
1
Slide 2: http://www.firstgovsearch.gov/
SEO Topics
1. 2. 3. 4. Search Visibility Factors Basics of Search Friendly Design Site Maintenance Resources
2
Slide 3: http://www.firstgovsearch.gov/
Search Visibility Factors
3
Slide 4: http://www.firstgovsearch.gov/
Basic Page Components Text (Keyword) Component
Words and phrases that match what your target audience types into search engines
Link Component
Site navigation and URL structure that search crawlers can easily follow
Popularity Component
Are sites link to you?
4
Slide 5: http://www.firstgovsearch.gov/
Text: Keyword Density
Choose one or two keywords or phases to optimize for each page Do not over use - avoid keyword stuffing Over use of keywords can result in being penalized or ignored Incorporate other complimentary words and phrases Check Keyword Density:
http://www.webjectives.com/keyword.htm http://www.keyworddensity.com/
5
Slide 6: http://www.firstgovsearch.gov/
Text: Keyword Prominence
Search engines place “weight” on terms according to where they are used Place Keywords in … Title tags Headings and emphasized text Visible body text Description meta tags Alt text in images Title and body tags are most important Keyword in the URL is helpful, but not a significant factor in ranking
6
Slide 7: http://www.firstgovsearch.gov/
The Metadata Myth
Quote: “Metadata improves search relevancy” False. Except for title and description tags, Web search engines ignore other metadata. However, Metadata does matter to enterprise search. Quote: “Using standard metadata is a best practice.” False. It is only a best practice if the metadata is used by your agency for specific applications (e.g. enterprise search or content syndication). It is not a best practice among professional web designers.
7
Slide 8: http://www.firstgovsearch.gov/
The Metadata Myth
Quote: “If more agencies were diligent about adding metadata, Google would pay attention” False. Industry is focused on developing algorithms that determine relevancy based on content rather than what an author or metadata creator says about the content Key points Metadata is not the key to high rankings Only use metadata if it is important to your agency’s mission Paying more attention to keyword usage in other areas will have a greater impact on relevancy
8
Slide 9: http://www.firstgovsearch.gov/
Link Component
Pages will not rank well if your site does not have a navigation scheme Navigation scheme must please users and search engines Create a site map, but also plan how pages link to each other Avoid dangling pages
9
Slide 10: http://www.firstgovsearch.gov/
Problem Navigation Schemes
Poor HTML coding Image maps Frames JavaScript Dynamic Pages Flash
10
Slide 11: http://www.firstgovsearch.gov/
Creating Links
Keywords in links tell crawlers about the pages to which you are linking Keywords in links influence relevancy of the page to which you are linking Avoid “click here” links, instead create links like:
Bad: Click here for more info on famous admirals. Good: Visit our naval history site for more info on famous admirals.
11
Slide 12: http://www.firstgovsearch.gov/
Popularity Component
Based on the number pages that link to you The more popular pages that link to you, the higher your popularity All search engines have different popularity algorithms
Google’s algorithm is called Page Rank
Every page on the Web is given a calculation of it is popularity based on inbound links
12
Slide 13: http://www.firstgovsearch.gov/
Popularity Factors
Number and Popularity of Inbound Links
Get listed in Yahoo!, DMOZ Network with other agency and industry sites Make your site a link magnet
Anchor Text
Others’ use of keywords in a link to your site
Popularity is assigned per page, not for the entire site
Popularity is not inherited Need to deliberately link internal pages to pass on PR
13
Slide 14: http://www.firstgovsearch.gov/
Outbound Links
Your popularity is not determined by the sites you link to Your outbound links affect the popularity of the sites you’re linking to Internal links and inbound links have the most impact on your popularity Outbound links help identify you with a hub
14
Slide 15: http://www.firstgovsearch.gov/
Robots Exclusion
Meta-Tag Robots Exclusion
<head><meta name=“robots” content=“no index, nofollow”></head>
Robots.txt File
Place in server’s root directory Two elements: User-agent, Disallow Example:
User-agent: * Disallow: /cgi-bin/ Disallow: /scripts/ Disallow: /images/
15
Slide 16: http://www.firstgovsearch.gov/
Robots Exclusion
Not all search engines pay attention to robots.txt instructions MSN and Yahoo! obey robots exclusion more often than Google Never exclude msnbot … you won’t have site search Blocking bots is contrary to OMB’s guidance
16
Slide 17: http://www.firstgovsearch.gov/
Case Study: NOAA Fisheries
Term: Fisheries #1 on FirstGov #1 on MSN #2 on Google #3 on Yahoo Term: Fish Doesn’t show up on the first page or subsequent pages
17
Slide 18: http://www.firstgovsearch.gov/
18
Slide 19: http://www.firstgovsearch.gov/
Case Study: NOAA Fisheries
Text Component: page is well-optimized for “fisheries” but not “fish” Link Component: Fly-out navigation may be a problem for some crawlers. Add search friendly navigation (e.g., site map) Popularity Component: Lots of inbound links using “fisheries” (not “fish”) in anchor text
19
Slide 20: http://www.firstgovsearch.gov/
Basics of Search Friendly Design
20
Slide 21: http://www.firstgovsearch.gov/
Basic Concepts
Page Content: using content and text that target your audiences; and attract search engines and links from other sites Navigation: giving users and crawlers easy access to content Design Considerations: make sure bells and whistles don’t undermine SEM efforts Page Rank: link popularity
21
Slide 22: http://www.firstgovsearch.gov/
Managing Page Content
22
Slide 23: http://www.firstgovsearch.gov/
Types of Pages
Home Page About Contact Site Map News Forms Galleries FAQs Catalogs Product pages Shopping cart Search Results
How you write, design and optimize a page depends on the type of page
23
Slide 24: http://www.firstgovsearch.gov/
Text to Include
Keywords
Use language of your audience Use keyword selection tools Yahoo!/Overture: http://inventory.overture.com/ Google: https://adwords.google.com/select/KeywordSandbox
Page Content Make content appear focused
Title tag, headings, contextual links, cross-links Body text should be visible, i.e., should not have to do any action to view main page text
24
Slide 25: http://www.firstgovsearch.gov/
Primary vs. Secondary Text
Primary Text Title tag Body text Text near the top of the page Text in and around links (e.g. anchor text) Secondary Text Alt text Description tag Domain name and URL elements
25
Slide 26: http://www.firstgovsearch.gov/
What Kind of Content to Include?
Write your own
Pros: Original, unique content Cons: Time consuming, bureaucratic process
Use someone else’s
Pros: Easy way to bulk up content Cons: Still need to worry about how to add value in order to attract links
26
Slide 27: http://www.firstgovsearch.gov/
Use Syndicated Content
Tends to update frequently Crawlers visit frequently-updated pages more often Combine syndicated content to create a unique resource Syndicate your own content (e.g. RSS)
Increase site visibility and attracts traffic Another opportunity for inbound links
27
Slide 28: http://www.firstgovsearch.gov/
Managing Site Architecture
28
Slide 29: http://www.firstgovsearch.gov/
Site Navigation Scheme
Text Links Very search engine friendly Use for primary or secondary navigation Problems with Text Links Can negatively skew keyword density Crawlers tend to read text links first
29
Slide 30: http://www.firstgovsearch.gov/
Site Navigation Scheme
Navigation buttons
Okay as long as you include alt text Avoid JavaScript, unless you can provide navigation crawlers can follow Recommendation: use alt text and text navigation at the bottom of the page – allows you put keywords in multiple places
30
Slide 31: http://www.firstgovsearch.gov/
Site Navigation Scheme
Image Maps
Crawlers ignore links inside image maps User text links or navigation buttons elsewhere
Pull Down Menus
Generally not crawler friendly because they need JavaScript or a CGI program Always provide two forms of navigation: one for your users, and one for your crawlers
31
Slide 32: http://www.firstgovsearch.gov/
Help Crawlers Navigate
Create a site map Subscribe to Google Site Maps
A crawler enabling tool to assist the Google crawler Analyzes information about your site’s architecture to improve crawling
32
Slide 33: http://www.firstgovsearch.gov/
Design Considerations
33
Slide 34: http://www.firstgovsearch.gov/
Design Considerations
Bells and whistles (Flash, JavaScript, animation) enhance user experience but can hurt search visibility Implement features carefully in order to keep search engine ranking Ensure design team understands SEM concepts
34
Slide 35: http://www.firstgovsearch.gov/
Use External JavaScript and CSS
Using JavaScript on site navigation can greatly decrease site crawlability Crawlers do not follow links embedded inside JavaScript code; or they limit the types of embedded links crawled JavaScript in <body> can decrease page load time Long download times may indicate the site is spam, and crawlers could ignore
35
Slide 36: http://www.firstgovsearch.gov/
Use External JavaScript and CSS
External files decrease page load time for visitors External files decrease download time for crawlers
Remember to disallow crawling scripts in the robots.txt
External scripts are easier to re-use
36
Slide 37: http://www.firstgovsearch.gov/
Frames
Do not design in frames Crawlers have trouble getting from the frameset page to the actual web page Frameset does not provide crawler with keyword rich text and links Each page is indexed separately, so pages that only make sense as a frameset will be indexed individually in search engines <no frames> tag is ignored due to spam
37
Slide 38: http://www.firstgovsearch.gov/
Frames Workarounds
1. Add Navigation
Give all pages unique title and description tags Put navigation links on your pages
2. Add Java Script in your <head> tag
<script language=“javascript”> <! – if (top == self) self.location.href = “index.html”; // </script>
This will force the browser to always load the frameset However, browser will always load the home page, not the indexed page Back button will be disabled
38
Slide 39: http://www.firstgovsearch.gov/
Flash
Few search engines crawl links embedded inside a Flash navigation scheme Flash sites contain little text If you include Flash …
Include a “Skip” link so both the user and crawler can go to the real homepage Include title and description meta tags
39
Slide 40: http://www.firstgovsearch.gov/
Dynamic Pages
Database-driven, created on the fly by asp, cfm, php, jsp or cgi scripts Dynamic sites are comprised of templates, but usually without original content When a page is viewed, the template loads the content from the database Parameters are added to the URL, which tells the template to load specific content
40
Slide 41: http://www.firstgovsearch.gov/
Dynamic Pages
Example: URLs such as this are difficult for search engines to index because they do not know the parameters that define a unique page The more parameters, the less likely pages will be indexed A database may continually feed data, crashing your server and scaring off the crawler
http://smithsonianstore.com/catalog/product.jsp?productId=14273&parent
41
Slide 42: http://www.firstgovsearch.gov/
Search Friendly Dynamic Pages
Create static HTML pages Modify URLs so they don’t look like dynamic pages, fewer parameters Use URL re-write trick using mod_rewrite
42
Slide 43: http://www.firstgovsearch.gov/
Session IDs
The kiss of death if left unmanaged Same content is delivered to the crawler but as unique URLs Crawlers will ignore web pages with session IDs Omit session IDs if the requestor is a crawler … but no cloaking!
43
Slide 44: http://www.firstgovsearch.gov/
Optimizing PDFs
Make sure PDFs contain actual text, not images of text Same rules for use of keywords and phrases apply Put the most important text in the title, headlines Minimize document size (> 100K) Create optimized HTML pages for PDFs
44
Slide 45: http://www.firstgovsearch.gov/
Managing Page Rank
45
Slide 46: http://www.firstgovsearch.gov/
Understanding Page Rank
All search engines assign a value to your site based on inbound links Google calls this relevancy factor “Page Rank” (PR) - synonymous with popularity ranking An inbound link is a vote for your page An outbound link is a vote for the page you’re linking to
46
Slide 47: http://www.firstgovsearch.gov/
Understanding Page Rank
A page is assigned PR as soon as it’s indexed Make sure all your pages have at least one link back into your site The page receiving the most inbound links gets the highest page rank You can pass PR around to pages on your site
47
Slide 48: http://www.firstgovsearch.gov/
Can You Leak Page Rank?
Controversial topic among SEOs Supporting View: PR leaks in the sense that a page’s outbound links will decrease that page’s available PR for redistribution throughout the site Opposing View: Search engines analyze inbound and outbound links to determine your authority as hub. No site is an island
48
Slide 49: http://www.firstgovsearch.gov/
Outbound Link Strategy
Do not create pages with mostly outbound links Link to quality, related sites – helps to establish you as an authority or hub If a page contains several outgoing links, also include links to other pages on your site Don’t be afraid to link to sites with low page rank, but quality content
49
Slide 50: http://www.firstgovsearch.gov/
Getting Links to Your Site
Links from Yahoo! and DMOZ
Impact on popularity may vary Helps to get noticed by crawlers
Establish reciprocal link arrangements with agencies covering similar topics Reach out to state and local agencies Syndicate your content Reach out to professional communities of interest
50
Slide 51: http://www.firstgovsearch.gov/
Links that Aren’t Links
Link has been created in a manner that search engines can’t understand Link actually points elsewhere, and not directly to your site Example: FirstGov.gov
http://www.firstgov.gov/external/external.jsp?url=http://www.noaa.gov/
Link passes through a JSP program that logs the click and redirects the browser to noaa.gov Search engines read this as an internal link to FirstGov, not an outbound link to NOAA A listing on FirstGov.gov will not help boost your search ranking
51
Slide 52: http://www.firstgovsearch.gov/
Site Maintenance
52
Slide 53: http://www.firstgovsearch.gov/
Re-Designing Your Site
Plan BEFORE you embark on a re-design Will the site architecture change? Static to Dynamic? How is the content changing? Add SEO review as an activity to your project plan Make sure your contractor has SEO skills Poor planning and execution can kill your search rankings Do not wait until after you re-designed! Do not ask the FirstGov Search Team to re-crawl your site!
53
Slide 54: http://www.firstgovsearch.gov/
Changes to Site Architecture
Try to keep the same filenames and directory structure when redesigning Follow MSN’s “What to do when your site moves”:
http://search.msn.com/docs/siteowner.aspx?t=SEARCH_WEBMASTER_REF_Redirectcode.htm
Recommended: Set up HTTP 301 redirects that point to the new site or pages Not Recommended: Add a meta-refresh tag to your page header. This won’t remove your original page from the MSN index, and thus your site’s search engine.
54
Slide 55: http://www.firstgovsearch.gov/
Removing Content
Avoid using default 404 pages Custom 404 pages are more user-friendly Submit all 404 URLs to the search engines using their Add URL form – quickest way to get the 404 pages out of the index Remember to change the HTTP status code on custom error pages from 200 to 404 Do not ask the FirstGov Search Team to re-crawl your site!
55
Slide 56: http://www.firstgovsearch.gov/
Resources and Recommendations
56
Slide 57: http://www.firstgovsearch.gov/
Recommendations
Read Industry Forums
Webmaster World: http://www.webmasterworld.com/ Search Engine Watch: http://www.searchenginewatch.com/ Digital Point Forums: http://forums.digitalpoint.com/ High Rankings: http://www.highrankings.com/forum/
Go to Industry Events
Search Engine Strategies: http://www.searchenginestrategies.com/
57
Slide 58: http://www.firstgovsearch.gov/
Recommended Reading
Battelle, John. The Search: How Google and Its Rivals Rewrote the Rules of Business and Transformed our Culture. New York. Penguin, 2005. Kent, Peter. Search Engine Optimization for Dummies. Hoboken, NJ. Wiley, 2004. Moran, Mike, Hunt, Bill. Search Engine Marketing, Inc. Upper Saddle River, NJ. IBM Press, 2006. Thurow, Shari. Search Engine Visibility. Indianapolis, IN: New Riders, 2003.
58
Slide 59: http://www.firstgovsearch.gov/
Contact Information
Bob Keating Search Program Manager robert.keating@gsa.gov (202) 208-1047 Susan Fariss Search Analyst susan.fariss@gsa.gov (202) 219-1437
59
