Yadabyte SEO Primer 4: Mommy, what's a search engine?

Guaranteed Spider Vision

Every night, while you are sleeping, spiders are crawling the World Wide Web. These "spiders" are computer programs sitting, en masse, in the server houses of the search engine companies. These spiders see the web in one medium and one medium only, and that is text. They don't see images or animations and they don't hear sounds. Now, you might be thinking, if you have read any of my other blogs, that I'm just showing off with the metaphor about spiders seeing in text here. Well, for once good reader, I am not. If you look at your web page like a spider you get a new insight into your site's level of optimization. But how...?

There is a free-to-use tool that lets you see like a spider: its called Lynx View. Take a look at the Yadabyte.com Website and now take a look at it as a web spider sees it with Lynx View. This is how Google sees it, more or less. Google adds extra layers such as if text is bold or not, but at heart, that's the Google view.

• Try this: Look at your website's Spider View here. What you should see, when you read down from the top through the page, are words that most represent what your site is about. If you don't, then you don't need an SEO expert to tell you that your site is critically badly optimized and unless you like it quiet, you need to take action. (If your site is totally unoptimized, in a few minutes you can do more good for your SEO than paying Yadabyte Websites or anyone else to optimize it).

The Long and The Crawl of It


Much of the work that search engines do is looking at the internet with Spider Vision. They do it do it constantly in what are called Crawls.The way a Crawl works is like this:

• The search engine has a list of websites. It gets these from the internet's Domain Name Servers (Google has its own DNS) as well as from its own historical database.
• When it crawls it will start off on a website - or tens of thousands at once - look at it with spider vision and save what it finds in its database (more on this later). It will then follow any links into and out of the site to other web pages and record what it finds at each one of those. This creates an explosion of searching spiders. Its an explosion you want your sight caught up in as many times and from as many directions as possible.

In an ideal world the internet would be searched like this constantly and simultaneously, but the computational power and internet bandwidth isn't available for this at the current time. So short-cuts are taken. We will discuss this in a later, more advanced section, of the series, Crawl Frequency and Depth..


So, we have the idea of this spider that reads web pages by crawling across the internet, but that's only part of the search engine; most of the really clever and innovative stuff happens after the spider has saved what it sees into the search engine database.



The Search Engine Database


All the text-based data that the spiders find is saved in the search engine company's big database. This is composed of essentially two parts: the data and the software. The data is the index and the page content - all the stuff the spider finds. The software is the absolutely crazily complex algorithms and functions that are made on the data. As a professional software architect, I can assure you, in case you didn't already know, what goes on in the engines houses of Google is some of the most sophisticated programming ever.

What's a real paradigm shift in how this software works is that it is dynamic in both directions. The search engine process and results change with what's on the web, and they change depending upon who, where and when the search is made. The reality of a search engine is that nobody, not even the people at Google or MSN, can say what any results could be.


Where we are in this series.

• So far we have seen the framework of what it's all about from an SEO point of view.
• We have seen what web presence is and how to get an idea of your site's web presence.
• We have seen how traffic entails a good search result and how good search results entail traffic.
  
• And in this last section we have seen how a search engine works in a conceptual, and, I hope, useful way.

The conclusion to this part of the series is that the web is so huge and complex, as are the search engine databases and softwares that represent the web, that nobody, not even Larry and Serge, can guarantee the results of any search. It's like the weather.

The rest of this series is about how to predict the weather better and, with a little effort, get a little more sunshine on your little patch of the internet. One thing's for sure, there will be more over-stretched metaphors.


SEO Primer Part 5 will be published shortly

Yadabyte SEO Primer 3: Traffic

We have some sites that we use for testing (or that we bought and never used) that are not connected to the internet. They sit there, not getting any traffic except yadabyte traffic. Google doesn't know about them. Nor yahoo. Nobody does really. These sites have zero web presence. That's what a new site is like.

So.. lets suppose you have a new website. Its built, its beautiful and it works exactly how you want it. If you now do nothing, your site will stay with zero web presence. It will have no visitors. It will not be found by google or yahoo. It will languish in a lonely internet solitude. Your new site nmeeds traffic.

You or someone needs to get it traffic because, as said, it wont happen by its self. Its pretty daunting with a virgin site... or just a quiet site to answer the question, "How can I get traffic?". And in essence this is what this series is all about; getting traffic to generate web presence so you can get more traffic.

I think a good vantage point to look at traffic is from a distance. Looking from here there are three ways traffic can come to your website:


The Three Types of Traffic

1. Search Engines - Most successful sites get most of their traffic from search engines. This is an irrefutable fact of the internet right now. Your site will need to generate search engine traffic.
2. Hyperlinks - Hyperlinks made the internet possible and they are still its prime matter. Hyperlinks not only allow people to come to your site from another site, but as we saw in the last section, they largely determine a site's web presence.
3. Address Bar Traffic - This is where someone enters your website's domain name in their browser and visit the site. That much is simple, but tehre are endless reasons why they might have done this. maybe they saw your ad in the paper, on tv, maybe a happy customer wrote it down on a beer mat. Maybe they saw your car decals. Some sites have lots of Address Bar Traffic (www.yadabytewebsites.com) others get hardly any (www.yadabyte.com).
   

Thats it.

There is no other way (I count email based links as hyperlinks) to get traffic to your site.

In the next post in this SEO Primer we will look at search engines as "things" so that you can understand their workings without much mystery. Following that we start to get into the mud and muck of search engine optimization, link building and, if we have time, how to get to the top slot in Google for the search phrase "yadabyte SEO primer".


Yadabyte SEO Primer 4: Mommy, what's a search engine? will be published shortly.

Yadabyte SEO Primer 2: Web Presence

The internet is the most complex and massive system of information ever built. You may have a website for your little holiday cottage on Orkney but its sitting in a vast... web... of connections and these connections are governed by hundreds of thousands of automatic process, human decisions and a fair dollop of chance. You need to know about your site's web presence, if you don't you are managing your website critically in the dark.

Although the notion of Web Presence is very abstract, there are a number of metrics available that can give you a very good idea of your site - or any site's - web presence. We will go through some of the following in posts all to themselves later in this series, but for now:

1. Google Page Rank - Unless your selling porn or BNP merchandise your website's Google Page Rank, or PR is, I think, the most important metric for web presence. Page Rank is a number between 1 and 10, simply put: the higher a site's PR the more important Google thinks that site is.
   
2. Alexa rank - Alexia is a subsidiary of Amazon and - for the last decade or so - it has been monitoring web traffic via its toolbar and other means. New sites wont have an Alexa rank but as a guide line, our www.tomeraider.com has a rank under half a million, Yahoo is 1 and Google is 3.
3. Backlinks - Backlinks are links from websites to your website. The quality of these links is important as if they come from "bad" sites then your site may be penalized in SEO terms.
   
4. Other Search Engines - Like it or hate it, right now Google is by far the most important search engine for UK websites and web users. Nonetheless, its always worth increasing your web presences by having your site optimized for other search engines.
   
5. Sundries - There are a bunch of other factors, like mentions in the blogosphere, forums, Usenet, email lists..... All of these factors can add, or detract in some cases, from a site's web presence. Some sundries, like getting a mention on a social news site such as digg.com can be worth an absolute fortune in terms relative value.
   

If you or your company owns a website, any website, then you need to get clued up with the website's web presence. It will take you 2 mins a week to do and is invaluable in so many ways. Its also totally non-teccy (Unlike, say webstats or Google Analytics) - my mum could find out this information - and make sense of it - whenever she wanted.


Know your Web Presence

There are a zillion tools and websites available to monitor all kinds of things to do with web site metrics. But lets start simple and use the free Tool Url at http://www.toolurl.com/

Visit this site, enter your website address in the box provided. You will instantly be told:

1. The Site's Google Page Rank
2. The Site's Alexa Rank
3. The quantity and source of Backlinks from varied search engines

If I was you I would date and write down these numbers (If your site is very new you might not have much to write down). And get into the habit of checking them regularly as they are so so important if you are serious and professional about your website.

Not knowing them is like being a stock broker who never checks the stock market.

SEO Primer 3: Traffic, will be published shortly.

Yadabyte SEO Primer 1: Kicking Up Trouble, a Confession and an Introduction

In the late mid 90s we were selling the first version of our TomeRaider Ebook software for the Psion Handheld. This was before Google, virals and mainstream SEO. There were various forums online for the Psion. In one of them, I got into a bit of a slanging match with a guy or guys. I can't remember what it was about (you would be amazed at what techies are prepared to tiff over), but what we noticed was that this forum argument had a direct correlation to downloads, and thus sales, of TomeRaider. If we let the tiff die down in the forum you could see sales and downloads slump. So, staying maximally friendly and equally antagonistic, we kept these arguments rolling and reaped the benefit of the traffic for a good while. Nobody got hurt.

You couldn't do that now, people are way too clued up. You would be branded a spammer and a flamer and in the long run your web presence would suffer. Since those "days of thunder" everything has changed. The internet crashed, Google arrived, social networks and blogs form the dominating substrata of the net, and there is a morality about what you can and can't do online. But one thing hasn't changed:

if your website is going to be a success, it needs good traffic.

Good traffic isn't just lots of traffic. Good traffic is visitors who want to be on your website. It's visitors who might need what you are offering.

This is the first part of a series I am writing on Web Presence, Traffic and Search Engine Optimisation. If you own a website and you're not an SEO Pro it might be worth reading. I want to achieve three key things with this SEO Primer:

1. To provide a clear no-nonsense and easy-to-understand account of the philosophy and practice of SEO. Something the reader can take away, for free, and make use of in tangible ways. Well, tangible for the internet at least - there is nothing in these posts that you shouldn't be able to do.
2. To show the myths of SEO - A lot of people make a lot of money from SEO, and often all they are selling is myth and ignorance. If I can prevent one person spending a bunch of cash on an SEO Company that merely carpet bombs the net with link farm submissions and SEO Spam then hey, my work here is done.
3. To get new SEO Clients. Although Yadabyte Websites have been improving our own site's web presence for years, we are now feeling confident enough that we can do beneficial SEO for our clients. Thoughtful, bespoke, holistic SEO that will work. So, if what you learn in this series sounds clear but you don't have the time or inclination to implement it, then feel free to hire Yadabyte Websites to manage your SEO.

SEO is a often punted as a "dark art". It isn't. Subscribe to this blog and see the light.

SEO Primer 2: Web Presence, will be published shortly.

Site launch: Burrows Estate Agents

We have just launched our latest website, www.burrowsestateagents.co.uk , an estate agent based in St Austell, Cornwall.

We did an estate agents site last year at www.duckworthpotter.co.uk and liked what we did on that one. Estate agent sites are notoriously hard to get right, and most estate agents seem to not really understand what their website is for, especially with the ubiquitous reliance on rightmove.

With both sites we wanted to get the visitors browsing houses in the shortest amount of time, without any sign up, distraction, or that all important "eye meandering."

With the www.burrowsestateagents.co.uk site we programmed a brand new control using AJAX that we call the "Show me" control, its pretty nifty. You select an option and in close to real time it displays a list of all the properties that match that option. Then when you select another option the list is filtered again but dependent upon your first option. it means visitors to the sight can hone in an what they are looking for really quickly and simply. The client loves it, so do we:)

In our original design we just had the blue sky on load, which looked dramatic and modern but, the client wanted to have the "about text" on the main page, so we did.

Meh, everyone is happy and thats all good.

Server Attack and Salt

We have three servers at the moment for the websites we host, both our sites and client sites. Two in the UK and one in the USA.

So... yesterday we found out that our main UK root server had been attacked. The hackers had installed a port scanner which was sniffing out the internet in an attempt to find some open ports on peoples PCs so it could then attack them.

We had no idea how they had managed to do this, and priding ourselves in our understanding of security concepts and principles, we were a bit shocked they had got through. In many years of running web servers, this was a first.

The mistake we made was a simple one, and its one most people will have made. One of our staff had set up a test site on the server to try something out, as we often do. An innocuous little site sitting unconnected to the internet. He chose the password "test123" and there was the mistake - the password was just weak.

We now know that it was guessed by a brute force attack from some automated bot that searches the net. Once that password was guessed the hacker could get into the system. (Note to any hosting clients, this would not have effected your site).

The problem is solved and we have learned our lesson, but it does go to show how you always have to be on your guard and how you really should use strong passwords.

Add some salt to your passwords

Incidentally one way to really strengthen your passwords against such attacks is to use what cryptographers call a "salt". In its simplest sense this is an "unusual" phrase that you can prefix of suffix to your passwords to increase their strength without making them impossible to remember.

EG

password="paris67"
salt="tenred45!"

Actual password=tenred45!pasris67

So you might have ten passwords that are strengthened by one salt.

by the way if you do have lots of passwords check out our free and portable Yadabyte Passwords. It allows you to store an unlimited number of passwords on a USB key, ipod, PC and even your Pocket PC. Its very very secure using AES encryption and its very easy to use.

Download Yadabyte Passwords for free from Yadabyte.com

Another blog!

So I have started another blog. I think that makes about 500 I have now. Some for work, some for fun and some secret ones documenting secret things that nobody must even know exits. Soon I will need a blog about the blog I have. It's getting complex.

So this blog is about Yadabyte Websites. I''m not sure what we will put on it but the first post will be about our main server being attacked this week. It is going to be exciting!

Maybe.