Choosing keywords seems like something you could do with a pen and legal pad and a half hour of cogitating. In fact this is almost certain to fail unless you truly are the prototype for the ideal customer. And even then you’ll miss a lot.
Here are the best ways I know of. I use these approaches or elements of them almost every day:
Steal keywords from your competitors, especially the ones that are kicking your butt. I’ll assume you know who they are, but if you don’t, start with a guess at keywords. For Ke Nalu we’ll guess Stand Up Paddle surf, SUP surf, paddlesurf, stand up board. Now search Google for those terms:

This is a good time to explain a little about Google and search engine result pages (SERPs) in general. You probably know all this, but I’m committed to assuming very little. The left side of a Google page is like editorial content in a magazine–it’s the stuff they give you to get you to use the service. It’s called the organic search results and companies like Google have closely guarded secret ways of deciding who gets listed first.
It’s supposed to be in order of greatest relevance and popularity, but SEO (Search Engine Optimization) aims to defeat that purpose and game the SERP. That’s fine, Google and other search companies set the rules and enforce them, but they are far from perfect. Many sites with superior content languish with nearly no visitors because they haven’t done the dance according to Google. You may consider SEO to be sleazy snake oil (and a lot of it is) but you need to know the rules, and Goggle sure isn’t going to tell you. Their “SEO” instruction is certain to relegate you to page fifty.
I plan on doing several chapters just on doing research with Google, so this is just a taste.
You May Be Looking At The Web Through Beer Googles
Here’s an important sidebar. Your Goggle search results and mine may not be the same. You rush into your boss’ office to show that your efforts have somehow moved your website up to the #1 spot in Goggle for your most important keyword. She does a quick search on the keyword you gave her and your site shows up on page three. Why?
It’s the same reason that the screen shot above of a google search for Stand Up Paddle Surfing shows Ke Nalu at the top of the heap. Woo-Hoo! Not so fast, buckwheat. I was looking through beer googles.
It’s because you and I are logged into our Google account and have Google History turned on and she doesn’t. Her search is real, yours is wishful thinking (and so is mine). Some time ago Google added some interesting features for people who have the Google toolbar. They started keeping track of all your searches and modified the search results to reflect what you have been searching for previously. In theory this should make searches more relevant to you. In practice it means your searches are meaningless for determining your standing on search pages.
Here’s what my search history looks like:

You may not like this history feature running silently away, logging everything you do. I know I’m not enamored with it. Here’s how to clear your history. Click on Remove items, then click “clear entire web history”

Then click “clear history” and zap, it’s gone. Or is it?

Google doesn’t say too much about search history, the privacy advocates are tearing their hair over it. Google has moved it from a little surprise feature into a place of it’s own called Google History. But it might be active on your account. Go turn it off, and log out of your Google account when you’re doing searches that you want to reflect what the general population is seeing.
An interesting note is that you can clear your search history, and your beer googles remain. Your search will still reflect the searches you made before, and will show how often you seached for specific sites. Hmm, how could that be? But if you log out of your google account the search goes back to normal. Log back in and somehow it remembers your history.
Good thing these guys aren’t evil. Of course I suspect most people realize that Googles “don’t be evil” mantra really meant “don’t be Microsoft”.
So now our Google search shows no Ke Nalu on the first page. There is a fairly prominent listing for Ponohouse, which is also a blog I own. It was the first blog I started that included posts about stand up paddle surfing–it was one of the first SUP blogs on the web, and that history gives it some prominence that Ke Nalu doesn’t have.

In fact Ke Nalu is really relegated to page three. That’s very cool, we’ll be able to show the progress as we go. If it were page one I wouldn’t be able to show you how to do this magic. Or at least we wouldn’t have as clear a proof. Ke Nalu used to be on the first page of Google for the keyword “Stand Up Paddle” but my inattention has let it slip.

All better, end of sidebar.
Google is encroaching more and more on the organic search with sponsored links on the top of the left column and “shopping results” on the bottom. Google gets affiliate money for the shopping links and is paid directly for the sponsored links at the top of the page and in the right column.
For our current purposes we are interested in all the results, because we can scalp keywords from everywhere. I make this sound underhanded, but it’s legitimately available information, I’m just going to show you how to get and use it.
We click on the top link and go to Supsurfmag.com, an incredibly cluttered site that does well because the founders work on Google position constantly and have a lot of backlinks and fresh content. I think it’s a pretty homely site, but on the web, beauty is as beauty does, though good design still delivers value. I’ll explain all that later in the creative section, but for now let’s concentrate on our search for Keywords.

At the top of the Firefox menu–my preferred browser–I select View>>page source to see the html code for the home page. I’m not going to show it all here on this page–you can look at it for yourself.
Near the top of this code are a lot of housekeeping and structural statements that tell your browser certain things about the page it is going to display. In particular a set called Meta code that describes the nature of the page, including this code:

The “Keywords” meta name includes keywords that the owners of the site believe are important. Historically some search engines used these keywords to find relevant content, but they are generally ignored now–Google categorically states that they don’t use them. But almost every site has them since you simply don’t know for sure if they might help. They might not help with the search engines, but they let us know what the site owners probably think are most important.
Equally important is the “description” meta name, since site owners still believe that this is searched, and the words in the description are often (but not always) the ones that appear in the abstract that the search result displays. Remember that Keywords aren’t just important to robots. The human looking at the result needs to see that the site they click on is likely to have relevant content.
If you do this step for all the significant competitive sites in the first few pages of the search results you’ll have a pretty nice collection going.
You should use a spreadsheet to organize these keywords. If you do that you can copy and paste directly from the page source, just as I did to place that meta data onto this page. Later you’ll be separating your keywords into segments like company, product, product interest, product evaluate, product buy, etc. A fine place to start for your spreadsheet is to use the Export function of the Google Keywords tool and then add whatever you would like to it. If you look at the bottom of the screenshot below you can see where the tool offers to export the information as text, CSV for Excel, or bare CSV (Comma separated values). These can be imported directly into a spreadsheet.
Look at the paid advertisements that show up in search results for any keyword. These folks are paying for clicks associated with these particular keywords. This gives you very strong clues about the value of the keywords you are finding, especially if established online companies are using them. I’ll show you how to get an even better idea by looking at what they are willing to pay for these ads later, but for now, relevant ads are good. Go ahead and click on them and do the basic keyword analysis you’re learning. Don’t get carried away–you’re costing them money and it’s underhanded and petty to do that for the pure purpose of harm. But the advertising is out in the public. You should use all the information you can gather to inform your marketing.

Now that you have some basic keywords and you know where your competitors’ sites are, use keyword tools like Wordtracker or Google’s free tools to expand your set and evaluate their popularity. Popular keywords are important, but once again I warn you to consider other characteristics. for example, Starboard is a popular maker of SUP boards. A prospect searching for Starboard SUP boards might be interesting, but a prospect searching for Starboard SUP Board Prices reveals something entirely different.
I find Google’s keyword tool particularly handy for locating hordes of keywords. If you are trying to do SEO only, then it’s not for you–Google clearly designed it to encourage you to buy lots of adwords. But if you’re building what we are building–a template for conversations, content, marketing, customer understanding, etc. , then it’s just great.
You can use the tool for expansion and evaluation of keywords you already know, and you can use it to fully evaluate the competition’s site for keywords.
Notice that some of the words relate only peripherally the search terms that we’ve examined so far–you’ll get some chaff with this tool. But look also at the interesting variations.
Analyze Similar Sites. Here’s some magic. Check the radio button for “Website Content” instead of “Descriptive words or phrases” and enter the URL for your own or your competitors sites. Bonanza. A valuable analysis of the keywords found on the website and the search volume.

This analysis is for www.surpsurfmag.com. It doesn’t take too much imagination to understand how helpful this is. Cooler yet, you can export the results as text, CSV (comma-separated variables) formatted for Excel, or just CSV so it can go straight into your spreadsheet.
As you can imagine, using this tool you can flesh out your spreadsheet quickly. You can eliminate duplicates as you go, or write a macro to do the elimination. For research purposes I like to retain the duplicated keywords on a worksheet dedicated to a specific site or characteristic, and then eliminate the duplicates on the master page.
Add specificity. Work your way through the keywords and add modifiers that make your keywords better fit your products, company, customers and prospects. For example, Women’s Stand Up Paddle Boards or Stand Up Paddle Shortboards are more specific. Then roll these more specific keywords back through the tools to to rate them for popularity and add variations you may not have considered.
Keywords and the Long Tail
The long tail is a reversal of the 80/20 or Pareto rule–interpreted for marketers as 20 percent of a market collects the vast majority of the customers, or twenty percent of products get eighty percent of the sales, or twenty percent of books attract eighty percent of readers. It’s certainly true, and companies have run their businesses aiming to be in that small group, offering those few products that sell the most. But now there’s an alternative strategy.

Chris Anderson looked at the power curve that results and realized businesses could thrive in that skinny long tail of the power curve if there were a large volume of prospects, and the distribution and inventory costs allow them to realize significant profit out of selling small volumes of hard-to-find items to many customers, instead of only selling large volumes of a reduced number of popular items.
In other words, the long tail is a viable strategy to do business on the internet. Businesses were evolving into that model before Mr. Anderson explicated it, but his article in Wired Magazine sparked a revolution in how business can be conducted and what strategies bear the greatest risk.
Keywords should follow a Long Tail approach, in fact one of the keys to world domination is to start with the small volume, 80 percent, more specific keywords and work your way up to the 20 percent big mommas. As your site gains in volume and links from traffic resulting from searches for long tail keywords, you are in a better position to compete for the higher volume ones. It’s like gaining a beachead, which I consider a primary concern in marketing–both taking them offensively and defending them.
Long tail keywords tend to be highly specific, like “Luxury Home In Portland Oregon” rather than “Home” or even “Luxury Home”. They might still face considerable competition, but it is relatively easy to move to the top of such mini-markets both because you will probably be one of the few making a strong effort and because your content will naturally support the keyword. We will play with this specific example because I’m building a website to sell my “luxury home in Portland Oregon” which gives me another nice dual purpose example.
Always update your keywords. Your analytics will show which words and phrases are having the most effect. You are doing all this work so that prospects looking for a specific product can find it at your company. But even more important is that you can cross the cultural divide and get them to buy. Keywords change more rapidly then you might expect, influenced by everything that is going on in the space. All of the conversations on the web are ephemeral to some degree, so committing your research to paper is not as valuable as you might expect, unless you’re interested in the history and trends for some reason.
Rank your Keywords. Now you have a seed list of highly relevant keyword phrases from all your efforts. You should have hundreds of keyword phrases in a column on a spreadsheet. All the Keywords that have gone through a tracking tool (Google, wordtracker, wordze, etc.) should have the search counts in an accompanying column. Now you do a Google search for each keyword and record how many pages return for each keyword in the next column on your spreadsheet. Take a look at the competitors that emerge. You will find a lot of surprises. I helped a friend with this approach and found a company that was knocking off his products, being supplied by his chinese manufacturer. An all-too common tale these days, but before we did the keyword research he had no idea it was going on.
Let’s Get Competitive
I hate hearing someone say they don’t really have competitors–I’ve been there myself. It only means one of three possible things. You’re too early to the market, or you don’t know the market or there is no market. Usually it’s simply that you don’t know the market and there’s no excuse for that today. You need market intelligence to know your competitors. It used to be expensive and difficult. Now it’s free and easy.
Good marketing these days calls for some highly organized competitive intel. And of course we’re going to use keywords to do it. I promise that pretty soon now we’ll talk about marketing tactics that don’t have much to do with keywords, but for now we’re stuck with them–we need them to get our baseline strategy in place.
Let’s find our toughest competitors. We’ll define this as people who are using a strategy similar to ours, and doing a good job of it. They will have found the keywords, or at least the concepts that prospects find their products with, and they will have optimized their entire site, or sections of it, for those keywords. We’re going to get a little esoteric in doing this, and I apologize for not explaining everything, but if I did this would quickly devolve into an SEO textbook, and the world doesn’t really need another of those right now. If you don’t understand what I’m doing just view this as a recipe.
Pick a set of the keywords that seem most relevant–both in connection to your products and in popularity. For our example for Ke Nalu it’s going to be just Stand Up Paddle which got 60,500 searches in August 2009.
We start off just doing a google search for that phrase enclosed in quotes. Remember to turn off search history. If you don’t, the results will be skewed by the searches you have made–your site will figure large, even though if you go to someone else’s computer it may not show up at all.
Now we’re going to do an intitle search, which means we’re going to add a parameter to our search to find sites that include that phrase in their titles. Google parameters are fussy, you can’t have spaces between the colon following the parameter and the keyword, but the keyword must be surrounded with quotes if it’s a phrase.

Lots of sites are optimized for “Stand Up Paddle” in their title. Now let’s see who is really working this keyword hard. We’ll do an inanchor search along with intitle

Still lots of results. These are the most formidable competitors for the keyword: People who have optimized both the title and anchor text for the keyword I care about.
Now it’s time to see how tough these competitors really are. One easy measure is the number of backlinks. Links to the site are one of the key ways that Google sets pagerank. I generally take everything on the first few pages of the Google search and use Yahoo’s Site Explorer to look at all their backlinks. Here’s the first one:

I wander off to all these links to look at their quality–by that I mean how relevant they are to the page they link to and what kind of keywords are in the text. While I’m looking I also note the pagerank of the referring pages. If they have a high pagerank the links are worth more.
All of this is a lot of work, but it can really fill in the competitive landscape for you. If your business is new, or new to this space, it gives you a good idea of what you will need to do to be successful. Who the prospects and customers are, who the major secondary players are–the consultants, resellers, value-added resellers and dealers, etc.. And it can show you where all the influential editorial comes from, which pundits are active, where the blogs and forums are.
If you’re too pressed to do this kind of work, and it IS a great deal of work–the lists grow geometrically–your agency might be able to do it for you. B&J certainly can. It’s not inexpensive, but compared to old school competitive intel it is infinitely more actionable, infinitely more accurate, and much cheaper.
If you play around with these tools in your space you will probably be terrified initially. Even if you are the industry leader and a Fortune 500 or Russell 2000 company you may be getting your butt kicked online. The online world can look like you’re standing knee deep in Piranhas. Don’t sweat it too much, a lot of those Piranhas are toothless. If you bang away at things and build a plan you can do well in almost any space. I’m going to stop saying “we’ll cover that later” because I’m getting quite a backlog, but that’s pretty much the plan.
What we are fundamentally doing is the first steps in building a marketing platform. The platform is not just a website, or just a set of tools, it is the combination of all your marketing efforts once they have been integrated. TV, print, direct mail, and all your web stuff along with how you interact with the sales force and/or customers as prospects. You can build it all into a robust environment, or you can hold it together with string, duct tape and a lot of manual labor. You can leave out the components of your marketing that are tough to integrate, or get some good help and stitch it all together seamlessly.
But in the end, the better and more robust your platform is, the better your marketing will be. It’s not cheap, and its not easy, but it’s worth the cost and effort because you can run with it for a very long time.
Seth Godin wrote a post in his blog recently about this notion that really resonated with me. He said modern marketers don’t rent, they own. Generations of marketers rented attention–a magazine, a TV network, a billboard, a direct mail campaign mailed to rented lists–every advertising venue or media company developed an audience that they could rent to you. We didn’t care about the long term value or health of that media venue, we just wanted to know how many eyeballs we could attract to our brand or products.
Now the rules are different. We’re building our own audience and that means they are a long term asset. There are two important elements to this difference–the audience and the platform. If we are really smart we care about both and invest in both. We gain value both by making the platform more sticky, useful, engaging and by keeping the audience satisfied and involved.
Depending on your business and it’s sales model the value of either element can be higher. But the place to invest first and foremost is in the platform. The quality of the platform and it’s ability to bridge the cultural divide is the primary tool for maintaining and extending the audience. Your sales cycle can live in your platform, attracting, nurturing (maturing), converting and extending the prospects into customers and advocates.
As Seth Godin says: “If you don’t invest in the platform, you’ll be at a disadvantage, now and forever. The smart way to build a brand today is to invest in the elements of the platform… the product, the technology, the websites (plural) and the systems you need to make it easy for people to show up at your very own trade show. And then embrace these people and shoot for 90% conversion, not .5%. Like most good investments, it’s expensive and worth more than it costs.”