Servers for the Information Age

The never-ending need for File Servers in the modern world

Handling the ‘Data on Demand’ needs of a New Generation!
-
I was recently helping someone troubleshoot some problems at SearchEngineWatch forums and it struck me that I should write an article about this and other issues. Because sometimes when you run into a brick wall and can not figure out why your site is not getting listed or even crawled in the search engines it is not always a site issue.

In fact, sometimes it is a server issue which you may or may not have control over. In that case, it does not matter what you do to your site, you just aren’t going to get it listed. This article covers some things to look at and how you can go about fixing them.

Let me start by giving you an idea of what happened with the person on the forum. His site had been up for some time, in fact he recently put up 3 sites. Two of which were getting crawled and indexed and one wasn’t showing up in the engines.

We started by looking at the usual suspects - such as improperly coded robots.txt, poor navigation, banned domain and so on, yet none of these were an issue. The site was very search engine friendly, easily navigable and didn’t use a robots.txt file.

So the next thing I checked was the WHOIS information and on one site I found improper information and on another WHOIS lookup I found no information at all. This lead me to believe that perhaps there was a hosting/server problem at the root of the issue.

So I contacted the site owner and got him to verify with the web host that the DNS information was properly configured. It turns out that it wasn’t. Since the site wasn’t properly resolving via DNS it wasn’t getting crawled by the search engines.

This is just one of the many server issues which can affect the positioning and ranking of your site in the search engines. While DNS resolution and IP configuration are generally the two most common culprits there are several other server issues which can harm your search engine ranking and even cause indexing problems.

DNS issues

Let’s take a closer look at what happened with this site: What I found is that I could connect to the domain name through my browser. Further, the owner had enough foresight to register his site as the only site on the IP, so I was also able to connect to the site via IP in the browser.

I could also ping both the site and IP and receive proper results, indicating that at least at some level there was DNS resolution happening.

However, when I went to perform the WHOIS lookup at the different site I wasn’t seen the results I should have, which means the DNS for the site hadn’t been properly configured.

I like to use whois.webhosting.info to perform these lookups as you can also perform IP searches to see if the site resolves properly to the IP. Further, you can also see how many and what types of sites may also be hosted on a domain. This can give an indication of other issues which could cause a site to not be indexed.

For example, had I performed a WHOIS lookup and found a bunch of porn or casino sites, I could have assumed (probably quite rightly) that Google had greylisted or blacklisted that IP address because of the sites hosted on it. Then, any legitimate site would also suffer the flag, causing indexing problems by the engine.

But that wasn’t the case here. As I said, it was a single site on a unique IP address.

I gave the information about the DNS to the site owner, indicating my concerns that the site wasn’t properly resolving and I found out a few days later that one of the hosts name servers was indeed acting up. I don’t know what “acting up” means, but it could indicate that the resolver wasn’t always resolving the IP to the site for some visitors, or it was improperly resolving it. In either case, should a search engine come across such inconsistency it will, over time, quit coming to the site.

More issues which could affect rank

There are other issues, not DNS related, but server related which could affect your indexing and ranking possibilities. These include improperly configured servers, or overloaded servers among other things.

Improperly configured servers

As you read above, there was an improperly configured (or failing) name server causing problems with the site, but there are other server issues which could affect the site as well.

Once the site is resolved via a name server, the request is then forwarded to the webserver hosting the site. If the webserver isn’t properly configured however to accept the request, then it will deny the request to the site, even if the site exists on the server.

While this is a rare occurrence, it does happen. If the webmaster mistypes the domain name for example, or attaches the site to the wrong IP attached to a different server, this could cause issues.

Another example of an improperly configured server is one which denies requests to domains without the “www.” Your webserver should be configured to accept all requests for a domain whether they have the www or not.

In fact, the ideal situation is for the server to automatically issue a “301″ server site redirect to any request made for a non-www domain which then forwards users to the www domain. This is because sometimes search engines will see the www and non-www domains as 2 separate sites. Therefore link popularity and rankings could be split among the domains. In a worst case scenario, one or both of these domains could be banned by the search engines for duplicate content.

In addition, if you are fortunate enough to have your own IP address, you should check with your web host to ensure that the IP address resolves to the site. This way if a user (or search engine) tries to connect to the site via IP they will be served the site properly, and not receive a “page not found” error.

Alias issues

As mentioned above, it is best to have an non-www requests 301 redirected to the www domain. However, for ease of use (and because its quicker and easier to configure) most web hosts will allow all requests to a site to return that site, so that any request to the www, or non-www domain will return the same site. As I eluded to earlier, this could get the site penalized or banned. At the very least, the site won’t properly propagate link popularity, which will ultimately affect its ability to rank competitively in the search engines.

With some servers, the only way to combat this is to set up a unique site for each alias and have the non-www alias 301 to the www domain. Sometimes that requires the appearance of 2 sites per domain on the web server. While most web hosts won’t agree to this, perhaps they can help you come up with a solution that will work for you.

Overloaded servers

Sometimes, when you perform a WHOIS lookup you will find hundreds and hundreds of sites hosted on a single IP.

While in many cases this isn’t an issue, you can still run into problems in the search engines.

This is because, to save time, many engines will cache the IP of a domain and simply request the domain by IP rather than a full DNS request each time it wishes to index the site.

That means, every other site hosted on that IP has the potential to negatively impact your search engine rankings.

This is because every site which is associated with that IP will by default be associated with your site, because of the common IP. If one of the sites has done something to spam the engine in the past, there is a chance that your site could suffer the consequences.

Go back to the example I gave earlier - a legitimate site sharing an IP with porn or gambling sites. I can almost guarantee you that the legitimate site will not rank competitively.

Consider it guilt by association. In this case the association is the sharing of an IP address.

While search engines are getting much smarter at identifying these types of things, some issues still arise. Therefore it’s safer to take precautions now.

If you do find such a site on your shared IP, you could always request that your site be moved to a new IP so that it doesn’t suffer the consequences of a penalized IP.

Another problem with overloaded hosts is that sometimes there is a site (or a few sites) which use so much of a servers resources that other sites suffer. If this is the case, then your site may not respond in a timely manner which means search engine crawlers could stop coming to the site and it will fall from the index.

Again, the solution here is to move to a different webserver or change hosting providers altogether.

As you may see, there could be many issues outside of the site itself which can affect search engine rankings.

The best way you can identify these issues is to become a web detective and perform a little searching of your own using WHOIS servers and PING commands to test out your site to make sure it resolves properly to the web host and IP it is supposed to.

Many times when you’ve ensured that there isn’t an issue on the site itself, then its likely a hosting issue.

And in some cases, you may be able to get these resolved quickly while others take a little longer. If you find its taking too long to get hosting issues solved, might I suggest shopping around for a new host? One that will treat you like a customer and not a number or pay check? That way you can ensure that your site receives the attention it deserves, and problems are dealt with quickly
About the Author

Rob Sullivan - SEO Specialist and Internet Marketing Consultant. Any reproduction of this article needs to have an html link pointing to http://www.textlinkbrokers.com

The never-ending need for File Servers in the modern world

Handling the ‘Data on Demand’ needs of a New Generation!
-
Am I the only one, or does it seem harder and harder to remember things?

I remember long ago, somehow I made a barely-conscious decision to purge useless information from my brain. This was probably to make room for newer, more exciting information, such as the name of that cute girl over there or the latest interpretation of the Special Theory of Relativity.

It was a double edged strategy. I was able to process loads of new data, but the mundane, such as remembering to wear matching shoes, suffered.

I moved through life controlled by a CPU which more resembled RAM (random access, or temporary, memory) as opposed to archival hard-drive storage. But oddly, I was able to conjure up weird pieces of exhaustively researched triviata which held my rapt attention for an intense half-life until they decayed and lay abandoned on the trail behind me.

Such is the world inside my head. A place where fascination can be triggered by the profoundly mundane, held under laser and electron microscope until all secrets are revealed, only to be abandoned like a pregnant hillbilly girlfriend.

As an older fart, the strangeness of my mind-space becomes even more noticeable. Now I have old-timers disease as well as Einsteinitis. I know I should be drilling down to some essential truth, I just can’t remember what it is. I may even be drilling unconsciously without even knowing it.

So here I am, like millions in my demographic, who can’t remember sh*t.

I am sure I get amazing amounts accomplished, I just seem to have lost track of what these accomplishments actually are.

Which is why I was forced to develop memory workarounds. We all have them. Here are some of mine:

1. Leave things by the door so I remember to take them with me as I depart.

2. Write myself to-do lists (a no-brainer).

3. Write to-do items in my Palm(tm) calendar.

4. Write myself emails.

5. Leave out reminder objects, such as bills to pay, in conspicuous places.

In addition, I hired a really clever guy to build me what I call a “Bounceback Server”. It actually returns copies of my emails to me at intervals I specify. Think of it as Ginko Bilaboa on steroids, except that it can be validated.

If, for instance, I want my email to bounce back to me in one week, I copy the email to 1w@poingo.com. One week later, when I have thoroughly forgotten the incredible urgency of the project, my email reappears to refresh my decidedly unfresh memory.

At this point I will decide whether to issue yet another emailed nudge or hock-a-chinik in some other way. If I send out a reminder email, again I will copy the Bounceback Server for whatever interval I think will make me look incredibly efficient to my nudgee.

You might be surprised at the number of emails you could send out, even in one day, which would be be candidates for this type of reminder system. I send emails to the Bounceback Server between 5 and 10 times a day. In addition, I also open emails bounced back to me from the Bounceback Server another 5-10 times a day.

My projects maintain momentum. People know I care. Progress takes place. I appear to be at the top of my game even though in truth I am operating on maybe one last badly worn but highly experienced brain cell.

I have made my Bounceback Server available to you for a pitifully low price that is less than even a small handful of bootlegged Aderals, and it won’t even attack your liver. You can even send emails using up to three different email addresses, and the Bounceback Server will still recognize you as a friend. Try it before you forget.
About the Author

Mark Meshulam offers the Poingo Productivity Suite, a suite of simple software programs which make your work, easier and much more fun. Bounceback Server is one such application.

visit http://www.poingo.com

The never-ending need for File Servers in the modern world

Handling the ‘Data on Demand’ needs of a New Generation!
-
There are a lot of web servers out there. Whenever you go to a website, you’re downloading it from a web server. When you pay money to a web host, what you’re really doing is renting a space on their web server. The Internet consists of millions of computers networked together, but it’s the servers that are providing all the information that makes up the web - you can’t have a website unless it’s on a server.

What is a Web Server?

A web server is really just a powerful computer - they use the same kinds of processors and memory that normal computers use, but they have more of it. Servers usually run a Unix or Unix-like operating system like Linux or BSD, but they can just as easily run Windows.

What makes these computers servers isn’t their hardware - it’s the software they run. Web server software includes the HTTP server itself, as well as databases and other things that are needed to make a web server work however it needs to. This is why different hosts offer different features: they have different programs installed on their servers.

Web Servers Serve Files

The role of the web server, at its most basic level, is to send people your files over HTTP. It has a hard disk (often more than one) and stores your files like any other computer - if you don’t upload a file called ‘index.html’, many servers will list all your files for you instead of providing a web page. It’s the replacement of the index.html (named because it is supposed to be an index of files) that creates the illusion of everything on the server being one ‘web site’, instead of a set of files linked together.

Web Servers Run Scripts

Of course, web servers don’t always just serve the same files over and over again. Sometimes they need to insert other information into pages, especially information that comes from databases. This is done with scripting languages like PHP and Perl - the server is told that it should give files that end in .pl or .php to the appropriate script interpreters, and these interpreters then tell the server what to send to the browser. This means that dynamic websites can often be slow, as the server is having to produce a different page for each visitor.

Virtual Servers and Dedicated Servers

When you buy web hosting, though, you’re not necessarily getting a whole server to yourself - in fact, the chances are that you’re not, unless you’re paying lots of money. Instead, you’ll be sharing a server with the hosts’ other customers. You might not realise this, since the server doesn’t appear to have anything on it that isn’t yours, but the other customers are simply being hidden from you - you’re using what is known as a ‘virtual server’.

For small websites, there isn’t really any option other than virtual servers: they’re a great idea for letting resources be shared among lots of websites that don’t use much of the server’s power or space. If one of the sites does start growing, though, you might find your website slowing down. Oddly enough, this fact means that it’s often better to find a host that offers price plans with limits instead of one that offers ‘unlimited’ disk space and bandwidth to each customer - your website will be much faster at the ‘limited’ host.

More Than You’d Think

One thing that people don’t often think about is that there’s more than one web server program out there. It’s not really visible to visitors, since they all do basically the same thing, but there are lots of servers available, and they’re all quite different in the way they work. There are three main groups:

Apache. The open source Apache software is the most popular server software out there, with around 70% of the market share.

Microsoft servers. Microsoft are responsible for the various versions of IIS (Internet Information Server) and PWS (Personal Web Server), which altogether have around 20% of the market.

Sun servers. Sun produce lots of servers, most notably the Netscape-branded ones. The market share of these servers depends on whether you count all sites (making it 3%) or just the actively maintained ones (in which case it drops to less than 1%).

Source for statistics: netcraft.com.

Other servers available are mostly ’simple’ servers that don’t have all the somewhat unnecessary features of these servers, such as thttpd (the ‘t’ is for tiny or turbo). There are literally hundreds of them, but they have mostly negligible market share.

About the author:

Information supplied and written by Lee Asher of Eclipse Domain Services

Domain Names, Hosting, Traffic and Email Solutions.