So what is a robots.txt file and how do you modify it so it works well with Google.

What is a robots.txt File
Robots are unmanned Internet tools that visit websites and try to read the content of and index the content of the site. For search engines they are interested in grabbing keywords and links within the site so they can be included in search results for their users.

To do this they read in the HTML content of the page using algorithms to weed out fake content. They will read the title of the page and of headlines and weight that against the rest of the content. They may also look at meta keywords and descriptions located in the header but many robots actually read your pages like a human would. Or at least they try to.

Robots used for advertising banners will read the page and feed an ad banner based on the content of the page. This often happens real time and is transparent to your visitors. This can place an extra load on your server because each page view will require a second, third, fourth or more hits to your pages just to serve your ads.

Ok so what can you do to control these bots? Well a robots.txt file can contain instructions that NICE bots will read and this will let you control how they browse through your site. However even some large search engine companies ignore this file and in that case you will need to setup a .htaccess rule to control the bots access to your site.

Since we are trying to make a nice robots.txt file for Google you should understand that their bots can and do change. Google will give you specific instructions when they change their settings that you can view from your AdSense account.

Setting Up The Robots.txt File
All you need to write a robots.txt file is notepad or other another pure txt file editor.

Start by creating an empty file named robots.txt

Add to that file the following:

Now save this file and upload it to your root website directory.
This is the main directory with your index.html file where people are pointed when they type in www.yoursitename.com in their browser.

This command will allow Google’s Bots to see your whole site. It should be on the top lines of your robots.txt file if you have other commands in your file.

You may also want to limit access to certain directories on your site from all bots.

In other pages we will cover how to block bad bots and how to use meta tags to control bots on your site.

We will also cover how to block bad bots with a .htaccess file.

Remember that controling who indexes your site is never perfect but it is something that you can do to optimize the way your visitors see your site in search engine indexes.

The first and most common is a 404 Error.
This is when your visitor trys to visit a page that is not available on your site. This often happens when you change your directory structure or permalink rewrite structure for your pages and although it is an Error it does not really represent a problem occurring.

The best way to handle your 404 errors is to add a 404 error page to your root website directory. If you are using CPanel or another website management panel you can review the ways that your host wants this file written but for the most part it is as simple as writing a basic html page and naming it 404.shtml and placing it in your /public_html/ directory. Note that the file is named SHTML because it is a server scripted file.

Other common website errors include:

206 Partial Content
301 Moved permanently (redirect)
400 (Bad request)
401 (Authorization required)
403 (Forbidden)
404 (Wrong page)
500 (Internal server error)

If you read your website logs you should be concerned if you are getting a large number of any errors.

500 (Internal server error)
This is a very important Error because it means that the scripts on your site are not functioning properly. It is very common to get this error when you are first installing Perl or Php Scripts because during the install or writing process you are likely to mistake a location of a file or forget the dreaded ; at the end of a line.

Once you fix the initial install if you check that the script is working well then 500 server errors may be caused by a slow server where your processes are timing out before they are completed. This is very common on shared hosting and for some website scripts there really is no cure but to move to a dedicated server to gain you more power. However for most basic website software your host should be able to handle a decent amount of load if you do not go overboard with plugins and special features.

Remember if you are seeing these errors in your log files then your visitors are seeing them when they load your site. Installing Error Handling Pages will present your customer with specific information and if you desire a way to contact you.

Writing Your Error Handler Files
You should make a html file that looks like the rest of your site but the content area should announce the information they were looking for can not be found or that an error has happened.

The page should have all the basic navigation working so they can get back to the area they just were or get to your home or contact page.

An easy way is to view your website and then copy the pages web source code into a text document. This is important for Shopping Sites and Blogs whos sites are often split up in a template.

You can also include a Search Box.

Some ad companies frown on placing ads on error pages so check with your program guidelines. The reason behind this is some sites make temporary pages to get into search engines and then take them offline generating thousands of error pages just to get clicks. 

Site Maps for Search Engines
If you find your site is having a number of missing or moved page problems you may want to look into generating a Sitemap file that can be read by search engines. View our other Guides on Search Engine Site Maps.

This is very important for administrators who use Amazon Webstores and for anyone that feeds their content off of remote xml data. If the generation of your sites pages can be unlimited based on an external data resource you should examine more indepth ways to restrict access to any bot.

For WordPress users you want to restrict access to files and directories that are not part of your site. You may also want to restrict access to your images and cache files.

A basic WordPress robots.txt file could include the following:

Notice User Agent * is all…. and that disallows are at the top
followed by overrides for Google AdSense.
An included badbot is also banned based on their user agent info found in the webserver logs.

Remember that a robots.txt file is only as good as the bot that reads it.
If someone is using wget to grab your site or a bot designed to harvest emails and links you are better off stopping it with script level denial, .htaccess or firewall commands.

If your site is getting hammered then contact your Service provider for help.

This info is just for reducing unneeded or unnecessary load by having bots run around in circles not finding what they should be looking for.

If you only have a handful of clients you can probably get away with doing the work manually but once you get past about 25 accounts you endup working on things that make your job just that much more difficult.

WHM.AutoPilot is an automation system for hosting accounts.

To use it you need to have a reseller account or server that has cPanel V2 or cPanel and Plesk V3.

The product is designed for Unix/linux servers and requires MySql, PHP4.4 or higher and  Zend Optimizer to run.

Your reseller account needs to be able to Create and Edit Packages and Manage Accounts.

How It Works

The product is pretty straight forward with only a couple clunky (reading directions necessary) aspects. The part that will take the most time is setting up payment and domain registration accounts with each respective company so your system will work once it is configured.

The first thing you do is setup your account hosting packages in WHMcPanel like you normally would. Like most hosts you will want to offer your customers a few different plans. Each will have different quotas and features.

Once you are happy with your accounts you setup your WHM.AutoPilot settings to communicate with your WHMcPanel account and import your hosting packages.

Once your packages are imported you can manage pricing.

Domain Names
You can offer domain name sales through Enom or a few other companies that can be purchased at the same time hosting is bought.

You will need to have a plan with enom and setup your account info and maybe even open a port to allow WHM.AutoPilot to talk to the Domain Name Seller.

Billing Options
There are probably about 25 different payment systems that can work with your billing system. Paypal is one of them however there are merchant accounts you can use.

How can you learn this software?

The software is downloadable for free and can be used for up to 10 accounts.

You can setup  your system to work in demo mode that way you can test how the product works without making actual payments.

You can also watch some really nice demo videos on their website that will walk you through the whole process of setting up the software and managing your accounts. You should set aside a full day to watch the videos and setup your own demo server.

Conclusion

This is a really nice product that offers a number of good features. At the  time we reviewed the product we wished it had better domain name buying choices but this is probably a result of domain sellers not opening up a resource for automated sales.

If you are reselling this is one of a few products that you need to have around. Some reseller accounts will come with a copy of this software for free if not take a look at the demo and you will see it is definitely worth it especially for automated monthy re-billing.

For more info visit their site

http://www.whmautopilot.com

The software provided is a variety of different packages, scripts and templates that are useful for developing a website. The benefit is that your customers can use code that has been reviewed to work well with most installs and most of the featured products do not require payment or registration by you or your hosting customer.Applications are installed with a single click or a helpful wizard.

The variety of products includes products like
Wordpress, Drupal, PHPCoin, osTicket, phpBB, Open-Realty about 2 dozen other products and maybe 100 website templates.

If you are purchasing reseller hosting with cPanel then you probably already have it installed. If you are setting up your own server then the cost is very low considering the advantage.

License includes free support and regular updates.

Cost is from $0 if included in your reseller package up to about $200 a year.

The real advantage of this product is that it takes the guess work out of installing packages. It will even create databases if needed so customers only need to enter passwords for their administration accounts..  click and their products are installed.

If it circumvents one tech call about how to assign db rights to a MySql User you have made your money back.

For more information visit.

http://www.netenberg.com/fantastico.php

This product may be available for free if you are purchasing a reseller account or if your server sales company offers it along with their Plesk or cPanel installs.

The idea behind the software is to automate as much of the back end as posible while still allowing you to go in and override and modify accounts. This is very useful if you are offering a special discount for charities or other customers.

The software works with a number of reseller control panels.

cPanel/WHM License Defender
Plesk Teamspeak
DirectAdmin ISPManager
TCAdmin (beta) InterWorx-CP

Creating local or remote accounts is simple using our control panel plugins. Our plugins will create, delete, update, suspend and unsuspend accounts with a simple click of a button. No more logging into multiple control panels to handle these simple tasks.

Choose from a variety of payment systems

PayPal, World Pay, Google Checkout,  StormPay, Authorize.Net, PsiGate,BluePay Protx, CCAvenue, eWay, ChronoPay, Money Bookers, Quantum Gateway, 2Checkout, e-gold, eProcessing Network,Internet Secure, LinkPoint

Allow customers to register Domain Names from Directi, Enom, OpenSRS

The product is full featured and is used by many large hosting companies.

Branding of your forms is easy

.

For more information visit their website.

http://www.newedge.com

The debate was centered around whether or not to use www. in front of a domain name and if so why.

To be fair the people having the conversation were more likely to be concerned about how their site looks on the web and not so much its overall function either as a multi-server enviroment or as an eventual structure for a corporations internet services but even if they are just hosting one Blog its no excuse to throw away proper configuration of your website.

And in all honesty I was probably the only one in the debate old enough to remember a time before there were webservices or a need for www. to be taged in front of your domain to identify the webserver.

So, without going into how to setup the dns lets just look at some of the uses of subdomains.

Lets say you have one website and the company offers file downloads or an anonymous ftp server. Even if that server is the same physical computer as your website there are advantages in setting up a subdomain of: ftp.host.com and using that account to host your files or anonymous uploads.

First you are segmenting your accounts. You have a folder that is within your main account that is dedicated to transfer of files. Even if you just link to them from your website you still have a nice way to control where the files go. However if you decide to allow public ftp services you can make a ftp account just for that subdomain.

ftp.host.com user:anonymous pass:none
you could even dedicate that account to a Public folder within the ftp subdomain account

ftp.host.com user:anonymous FOLDER /public_html/ftp/uploads

and also have an admin account on that subdomain that has full access to all the files on that subdomain.

ftp.host.com user:admin FOLDER /public_html/ftp

Now that is a pretty easy concept but what if your service grows and you end up with 4 computers in your rack just because the load is so high.

One for www.host.com website services
One for mail.host.com mail services
One for ftp.host.com ftp services
One for dns1.host.com dns services

In all practicality it could end up that way in a remote rack for a popular site or hosting service but in a intranet or corporate setting the presence of many different servers that provide many needs that are not just failover or cluster computers is very likely.

Also in the situation of Mail to a domain you may want to set your dns to accept mail from your main domain.com and use other servers for outgoing smtp.host or reading pop3.host or spit internal and external mail or you may have a number of mail servers that all use the same domain but identify based on port …. it can get slightly tricky but the main premise is still there.

So although there are reasons for strange domains like del.icio.us

You should also setup your DNS server to address www.icio.us AND icio.us

Type either of those in your browser and see what you get. Better yet ping them and see if you are forwarded to the correct website.

Ping request could not find host icio.us. Please check the name and try again.

This is just a waste

and it is not proper use of the domain name naming system.

They could be losing lots of traffic but maybe they think its not worth their time.

However if you are wondering if you should follow proper naming conventions then think about why they were developed in the first place.

Anyway don’t be stupid and miss customers or visitors because you didn’t add a www. CNAME to your settings to catch people and shoot them to your website.

For the most part you will see 500′s for script execution problems, 404′s For files not found or 301′s for permanently moved files but every once in a while you come across a strange one.

Most servers follow the same Error Codes so it doesn’t matter if you are running Apache or IIS or another web server.

100 Continue

The client SHOULD continue with its request. This interim response is used to inform the client that the initial part of the request has been received and has not yet been rejected by the server. The client SHOULD continue by sending the remainder of the request or, if the request has already been completed, ignore this response. The server MUST send a final response after the request has been completed.

101 Switching Protocols

The server understands and is willing to comply with the client’s request, via the Upgrade message header field (section 14.42), for a change in the application protocol being used on this connection. The server will switch protocols to those defined by the response’s Upgrade header field immediately after the empty line which terminates the 101 response.

The protocol SHOULD be switched only when it is advantageous to do so. For example, switching to a newer version of HTTP is advantageous over older versions, and switching to a real-time, synchronous protocol might be advantageous when delivering resources that use such features.

Successful 2xx

This class of status code indicates that the client’s request was successfully received, understood, and accepted.

200 OK

The request has succeeded. The information returned with the response is dependent on the method used in the request, for example:

GET an entity corresponding to the requested resource is sent in the response;

HEAD the entity-header fields corresponding to the requested resource are sent in the response without any message-body;

POST an entity describing or containing the result of the action;

TRACE an entity containing the request message as received by the end server.

201 Created

The request has been fulfilled and resulted in a new resource being created. The newly created resource can be referenced by the URI(s) returned in the entity of the response, with the most specific URI for the resource given by a Location header field. The response SHOULD include an entity containing a list of resource characteristics and location(s) from which the user or user agent can choose the one most appropriate. The entity format is specified by the media type given in the Content-Type header field. The origin server MUST create the resource before returning the 201 status code. If the action cannot be carried out immediately, the server SHOULD respond with 202 (Accepted) response instead.

A 201 response MAY contain an ETag response header field indicating the current value of the entity tag for the requested variant just created.

202 Accepted

The request has been accepted for processing, but the processing has not been completed. The request might or might not eventually be acted upon, as it might be disallowed when processing actually takes place. There is no facility for re-sending a status code from an asynchronous operation such as this.

The 202 response is intentionally non-committal. Its purpose is to allow a server to accept a request for some other process (perhaps a batch-oriented process that is only run once per day) without requiring that the user agent’s connection to the server persist until the process is completed. The entity returned with this response SHOULD include an indication of the request’s current status and either a pointer to a status monitor or some estimate of when the user can expect the request to be fulfilled.

203 Non-Authoritative Information

The returned metainformation in the entity-header is not the definitive set as available from the origin server, but is gathered from a local or a third-party copy. The set presented MAY be a subset or superset of the original version. For example, including local annotation information about the resource might result in a superset of the metainformation known by the origin server. Use of this response code is not required and is only appropriate when the response would otherwise be 200 (OK).

204 No Content

The server has fulfilled the request but does not need to return an entity-body, and might want to return updated metainformation. The response MAY include new or updated metainformation in the form of entity-headers, which if present SHOULD be associated with the requested variant.

If the client is a user agent, it SHOULD NOT change its document view from that which caused the request to be sent. This response is primarily intended to allow input for actions to take place without causing a change to the user agent’s active document view, although any new or updated metainformation SHOULD be applied to the document currently in the user agent’s active view.

The 204 response MUST NOT include a message-body, and thus is always terminated by the first empty line after the header fields.

205 Reset Content

The server has fulfilled the request and the user agent SHOULD reset the document view which caused the request to be sent. This response is primarily intended to allow input for actions to take place via user input, followed by a clearing of the form in which the input is given so that the user can easily initiate another input action. The response MUST NOT include an entity.

206 Partial Content

The server has fulfilled the partial GET request for the resource. The request MUST have included a Range header field (section 14.35) indicating the desired range, and MAY have included an If-Range header field to make the request conditional.

The response MUST include the following header fields:

      - Either a Content-Range header field (section 14.16) indicating
        the range included with this response, or a multipart/byteranges
        Content-Type including Content-Range fields for each part. If a
        Content-Length header field is present in the response, its
        value MUST match the actual number of OCTETs transmitted in the
        message-body.

      - Date

      - ETag and/or Content-Location, if the header would have been sent
        in a 200 response to the same request

      - Expires, Cache-Control, and/or Vary, if the field-value might
        differ from that sent in any previous response for the same
        variant

If the 206 response is the result of an If-Range request that used a strong cache validator (see section 13.3.3), the response SHOULD NOT include other entity-headers. If the response is the result of an If-Range request that used a weak validator, the response MUST NOT include other entity-headers; this prevents inconsistencies between cached entity-bodies and updated headers. Otherwise, the response MUST include all of the entity-headers that would have been returned with a 200 (OK) response to the same request.

A cache MUST NOT combine a 206 response with other previously cached content if the ETag or Last-Modified headers do not match exactly,

A cache that does not support the Range and Content-Range headers MUST NOT cache 206 (Partial) responses.

Redirection 3xx

This class of status code indicates that further action needs to be taken by the user agent in order to fulfill the request. The action required MAY be carried out by the user agent without interaction with the user if and only if the method used in the second request is GET or HEAD. A client SHOULD detect infinite redirection loops, since such loops generate network traffic for each redirection.

      Note: previous versions of this specification recommended a
      maximum of five redirections. Content developers should be aware
      that there might be clients that implement such a fixed
      limitation.

300 Multiple Choices

The requested resource corresponds to any one of a set of representations, each with its own specific location, and agent- driven negotiation information (section 12) is being provided so that the user (or user agent) can select a preferred representation and redirect its request to that location.

Unless it was a HEAD request, the response SHOULD include an entity containing a list of resource characteristics and location(s) from which the user or user agent can choose the one most appropriate. The entity format is specified by the media type given in the Content- Type header field. Depending upon the format and the capabilities of

the user agent, selection of the most appropriate choice MAY be performed automatically. However, this specification does not define any standard for such automatic selection.

If the server has a preferred choice of representation, it SHOULD include the specific URI for that representation in the Location field; user agents MAY use the Location field value for automatic redirection. This response is cacheable unless indicated otherwise.

301 Moved Permanently

The requested resource has been assigned a new permanent URI and any future references to this resource SHOULD use one of the returned URIs. Clients with link editing capabilities ought to automatically re-link references to the Request-URI to one or more of the new references returned by the server, where possible. This response is cacheable unless indicated otherwise.

The new permanent URI SHOULD be given by the Location field in the response. Unless the request method was HEAD, the entity of the response SHOULD contain a short hypertext note with a hyperlink to the new URI(s).

If the 301 status code is received in response to a request other than GET or HEAD, the user agent MUST NOT automatically redirect the request unless it can be confirmed by the user, since this might change the conditions under which the request was issued.

      Note: When automatically redirecting a POST request after
      receiving a 301 status code, some existing HTTP/1.0 user agents
      will erroneously change it into a GET request.

302 Found

The requested resource resides temporarily under a different URI. Since the redirection might be altered on occasion, the client SHOULD continue to use the Request-URI for future requests. This response is only cacheable if indicated by a Cache-Control or Expires header field.

The temporary URI SHOULD be given by the Location field in the response. Unless the request method was HEAD, the entity of the response SHOULD contain a short hypertext note with a hyperlink to the new URI(s).

If the 302 status code is received in response to a request other than GET or HEAD, the user agent MUST NOT automatically redirect the request unless it can be confirmed by the user, since this might change the conditions under which the request was issued.

      Note: RFC 1945 and RFC 2068 specify that the client is not allowed
      to change the method on the redirected request.  However, most
      existing user agent implementations treat 302 as if it were a 303
      response, performing a GET on the Location field-value regardless
      of the original request method. The status codes 303 and 307 have
      been added for servers that wish to make unambiguously clear which
      kind of reaction is expected of the client.

303 See Other

The response to the request can be found under a different URI and SHOULD be retrieved using a GET method on that resource. This method exists primarily to allow the output of a POST-activated script to redirect the user agent to a selected resource. The new URI is not a substitute reference for the originally requested resource. The 303 response MUST NOT be cached, but the response to the second (redirected) request might be cacheable.

The different URI SHOULD be given by the Location field in the response. Unless the request method was HEAD, the entity of the response SHOULD contain a short hypertext note with a hyperlink to the new URI(s).

      Note: Many pre-HTTP/1.1 user agents do not understand the 303
      status. When interoperability with such clients is a concern, the
      302 status code may be used instead, since most user agents react
      to a 302 response as described here for 303.

304 Not Modified

If the client has performed a conditional GET request and access is allowed, but the document has not been modified, the server SHOULD respond with this status code. The 304 response MUST NOT contain a message-body, and thus is always terminated by the first empty line after the header fields.

The response MUST include the following header fields:

      - Date, unless its omission is required by section 14.18.1

If a clockless origin server obeys these rules, and proxies and clients add their own Date to any response received without one (as already specified by [RFC 2068], ), caches will operate correctly.

      - ETag and/or Content-Location, if the header would have been sent
        in a 200 response to the same request

      - Expires, Cache-Control, and/or Vary, if the field-value might
        differ from that sent in any previous response for the same
        variant

If the conditional GET used a strong cache validator (see section 13.3.3), the response SHOULD NOT include other entity-headers. Otherwise (i.e., the conditional GET used a weak validator), the response MUST NOT include other entity-headers; this prevents inconsistencies between cached entity-bodies and updated headers.

If a 304 response indicates an entity not currently cached, then the cache MUST disregard the response and repeat the request without the conditional.

If a cache uses a received 304 response to update a cache entry, the cache MUST update the entry to reflect any new field values given in the response.

305 Use Proxy

The requested resource MUST be accessed through the proxy given by the Location field. The Location field gives the URI of the proxy. The recipient is expected to repeat this single request via the proxy. 305 responses MUST only be generated by origin servers.

      Note: RFC 2068 was not clear that 305 was intended to redirect a
      single request, and to be generated by origin servers only.  Not
      observing these limitations has significant security consequences.

306 (Unused)

The 306 status code was used in a previous version of the specification, is no longer used, and the code is reserved.

307 Temporary Redirect

The requested resource resides temporarily under a different URI. Since the redirection MAY be altered on occasion, the client SHOULD continue to use the Request-URI for future requests. This response is only cacheable if indicated by a Cache-Control or Expires header field.

The temporary URI SHOULD be given by the Location field in the response. Unless the request method was HEAD, the entity of the response SHOULD contain a short hypertext note with a hyperlink to the new URI(s) , since many pre-HTTP/1.1 user agents do not understand the 307 status. Therefore, the note SHOULD contain the information necessary for a user to repeat the original request on the new URI.

If the 307 status code is received in response to a request other than GET or HEAD, the user agent MUST NOT automatically redirect the request unless it can be confirmed by the user, since this might change the conditions under which the request was issued.

Client Error 4xx

The 4xx class of status code is intended for cases in which the client seems to have erred. Except when responding to a HEAD request, the server SHOULD include an entity containing an explanation of the error situation, and whether it is a temporary or permanent condition. These status codes are applicable to any request method. User agents SHOULD display any included entity to the user.

If the client is sending data, a server implementation using TCP SHOULD be careful to ensure that the client acknowledges receipt of the packet(s) containing the response, before the server closes the input connection. If the client continues sending data to the server after the close, the server’s TCP stack will send a reset packet to the client, which may erase the client’s unacknowledged input buffers before they can be read and interpreted by the HTTP application.

400 Bad Request

The request could not be understood by the server due to malformed syntax. The client SHOULD NOT repeat the request without modifications.

401 Unauthorized

The request requires user authentication. The response MUST include a WWW-Authenticate header field (section 14.47) containing a challenge applicable to the requested resource. The client MAY repeat the request with a suitable Authorization header field). If the request already included Authorization credentials, then the 401 response indicates that authorization has been refused for those credentials. If the 401 response contains the same challenge as the prior response, and the user agent has already attempted authentication at least once, then the user SHOULD be presented the entity that was given in the response, since that entity might include relevant diagnostic information. HTTP access authentication is explained in “HTTP Authentication: Basic and Digest Access Authentication”.

402 Payment Required

This code is reserved for future use.

403 Forbidden

The server understood the request, but is refusing to fulfill it. Authorization will not help and the request SHOULD NOT be repeated. If the request method was not HEAD and the server wishes to make public why the request has not been fulfilled, it SHOULD describe the reason for the refusal in the entity. If the server does not wish to make this information available to the client, the status code 404 (Not Found) can be used instead.

404 Not Found

The server has not found anything matching the Request-URI. No indication is given of whether the condition is temporary or permanent. The 410 (Gone) status code SHOULD be used if the server knows, through some internally configurable mechanism, that an old resource is permanently unavailable and has no forwarding address. This status code is commonly used when the server does not wish to reveal exactly why the request has been refused, or when no other response is applicable.

405 Method Not Allowed

The method specified in the Request-Line is not allowed for the resource identified by the Request-URI. The response MUST include an Allow header containing a list of valid methods for the requested resource.

406 Not Acceptable

The resource identified by the request is only capable of generating response entities which have content characteristics not acceptable according to the accept headers sent in the request.

Unless it was a HEAD request, the response SHOULD include an entity containing a list of available entity characteristics and location(s) from which the user or user agent can choose the one most appropriate. The entity format is specified by the media type given in the Content-Type header field. Depending upon the format and the capabilities of the user agent, selection of the most appropriate choice MAY be performed automatically. However, this specification does not define any standard for such automatic selection.

      Note: HTTP/1.1 servers are allowed to return responses which are
      not acceptable according to the accept headers sent in the
      request. In some cases, this may even be preferable to sending a
      406 response. User agents are encouraged to inspect the headers of
      an incoming response to determine if it is acceptable.

If the response could be unacceptable, a user agent SHOULD temporarily stop receipt of more data and query the user for a decision on further actions.

407 Proxy Authentication Required

This code is similar to 401 (Unauthorized), but indicates that the client must first authenticate itself with the proxy. The proxy MUST return a Proxy-Authenticate header field  containing a challenge applicable to the proxy for the requested resource. The client MAY repeat the request with a suitable Proxy-Authorization header field  HTTP access authentication is explained in “HTTP Authentication: Basic and Digest Access Authentication”

408 Request Timeout

The client did not produce a request within the time that the server was prepared to wait. The client MAY repeat the request without modifications at any later time.

409 Conflict

The request could not be completed due to a conflict with the current state of the resource. This code is only allowed in situations where it is expected that the user might be able to resolve the conflict and resubmit the request. The response body SHOULD include enough

information for the user to recognize the source of the conflict. Ideally, the response entity would include enough information for the user or user agent to fix the problem; however, that might not be possible and is not required.

Conflicts are most likely to occur in response to a PUT request. For example, if versioning were being used and the entity being PUT included changes to a resource which conflict with those made by an earlier (third-party) request, the server might use the 409 response to indicate that it can’t complete the request. In this case, the response entity would likely contain a list of the differences between the two versions in a format defined by the response Content-Type.

410 Gone

The requested resource is no longer available at the server and no forwarding address is known. This condition is expected to be considered permanent. Clients with link editing capabilities SHOULD delete references to the Request-URI after user approval. If the server does not know, or has no facility to determine, whether or not the condition is permanent, the status code 404 (Not Found) SHOULD be used instead. This response is cacheable unless indicated otherwise.

The 410 response is primarily intended to assist the task of web maintenance by notifying the recipient that the resource is intentionally unavailable and that the server owners desire that remote links to that resource be removed. Such an event is common for limited-time, promotional services and for resources belonging to individuals no longer working at the server’s site. It is not necessary to mark all permanently unavailable resources as “gone” or to keep the mark for any length of time — that is left to the discretion of the server owner.

411 Length Required

The server refuses to accept the request without a defined Content- Length. The client MAY repeat the request if it adds a valid Content-Length header field containing the length of the message-body in the request message.

412 Precondition Failed

The precondition given in one or more of the request-header fields evaluated to false when it was tested on the server. This response code allows the client to place preconditions on the current resource metainformation (header field data) and thus prevent the requested method from being applied to a resource other than the one intended.

413 Request Entity Too Large

The server is refusing to process a request because the request entity is larger than the server is willing or able to process. The server MAY close the connection to prevent the client from continuing the request.

If the condition is temporary, the server SHOULD include a Retry- After header field to indicate that it is temporary and after what time the client MAY try again.

414 Request-URI Too Long

The server is refusing to service the request because the Request-URI is longer than the server is willing to interpret. This rare condition is only likely to occur when a client has improperly converted a POST request to a GET request with long query information, when the client has descended into a URI “black hole” of redirection (e.g., a redirected URI prefix that points to a suffix of itself), or when the server is under attack by a client attempting to exploit security holes present in some servers using fixed-length buffers for reading or manipulating the Request-URI.

415 Unsupported Media Type

The server is refusing to service the request because the entity of the request is in a format not supported by the requested resource for the requested method.

416 Requested Range Not Satisfiable

A server SHOULD return a response with this status code if a request included a Range request-header field (section 14.35), and none of the range-specifier values in this field overlap the current extent of the selected resource, and the request did not include an If-Range request-header field. (For byte-ranges, this means that the first- byte-pos of all of the byte-range-spec values were greater than the current length of the selected resource.)

When this status code is returned for a byte-range request, the response SHOULD include a Content-Range entity-header field specifying the current length of the selected resource (see section ). This response MUST NOT use the multipart/byteranges content- type.

417 Expectation Failed

The expectation given in an Expect request-header field (see section 14.20) could not be met by this server, or, if the server is a proxy, the server has unambiguous evidence that the request could not be met by the next-hop server.

Server Error 5xx

Response status codes beginning with the digit “5″ indicate cases in which the server is aware that it has erred or is incapable of performing the request. Except when responding to a HEAD request, the server SHOULD include an entity containing an explanation of the error situation, and whether it is a temporary or permanent condition. User agents SHOULD display any included entity to the user. These response codes are applicable to any request method.

500 Internal Server Error

The server encountered an unexpected condition which prevented it from fulfilling the request.

501 Not Implemented

The server does not support the functionality required to fulfill the request. This is the appropriate response when the server does not recognize the request method and is not capable of supporting it for any resource.

502 Bad Gateway

The server, while acting as a gateway or proxy, received an invalid response from the upstream server it accessed in attempting to fulfill the request.

503 Service Unavailable

The server is currently unable to handle the request due to a temporary overloading or maintenance of the server. The implication is that this is a temporary condition which will be alleviated after some delay. If known, the length of the delay MAY be indicated in a Retry-After header. If no Retry-After is given, the client SHOULD handle the response as it would for a 500 response.

      Note: The existence of the 503 status code does not imply that a
      server must use it when becoming overloaded. Some servers may wish
      to simply refuse the connection.

504 Gateway Timeout

The server, while acting as a gateway or proxy, did not receive a timely response from the upstream server specified by the URI (e.g. HTTP, FTP, LDAP) or some other auxiliary server (e.g. DNS) it needed to access in attempting to complete the request.

      Note: Note to implementors: some deployed proxies are known to
      return 400 or 500 when DNS lookups time out.

505 HTTP Version Not Supported

The server does not support, or refuses to support, the HTTP protocol version that was used in the request message. The server is indicating that it is unable or unwilling to complete the request using the same major version as the client, other than with this error message. The response SHOULD contain an entity describing why that version is not supported and what other protocols are supported by that server.

You may have a site that has a number of copyright images such as a news or gossip site but for whatever reason you choose you can at least reduce the reuse of your image by others simply linking to them.

Many sites will already have a .htaccess file if you don’t then you will need to create one.

The .htaccess file is used for many reasons to give instruction to your webserver on how to process files.

Whenever the webserver accesses your account directory it will look for one of these files. If it is in the top directory then it will apply the commands to all sub-directories unless only specific directories are named in the .htaccess file setup.

Because the windows file system is picky about files with no prefix ie no prefix.something you will most likely need to create the file with one then rename it once it is on your server.

Start by making a new NotePad file named x.htaccess or open your current one for editing

Add the following

lines with # are comment lines
the first line requests modrewrite to be ON you may not need that on your specific server.

RewriteEngine On
#Replace ?yoursite\.com/ with your website url
RewriteCond %{HTTP_REFERER} !^http://(.+\.)?yoursite\.com/ [NC]
RewriteCond %{HTTP_REFERER} !^$
#Replace /images/dontcopy.jpg with the url of your copyright image
RewriteRule .*\.(jpe?g|gif|bmp|png)$ /images/dontcopy.jpg [L]

This code will tell the server that only yoursite.com is allowed to use your images.

If someone else links to your images like

they will be sent your copyright.jpg image

You should make an image that says something like visit yoursite.com for stuff.

If you only want to stop a few specific sites use the following

RewriteEngine On
RewriteCond %{HTTP_REFERER} ^http://(.+\.)?myspace\.com/ [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://(.+\.)?blogspot\.com/ [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://(.+\.)?livejournal\.com/ [NC]
#Replace /images/dontcopy.jpg with the url of your copyright image
RewriteRule .*\.(jpe?g|gif|bmp|png)$ /images/dontcopy.jpg [L]

Add more sites by using the OR command and on the final site do not add OR.

So this Rule says stop Myspace OR blogspot OR livejournal and end site list.

Make sure you upload the file as Ascii and then rename it to .htaccess

Final Note

Doing this can cause trouble if you want to allow other specific sites access to your images.  And it will not stop people from manually downloading your images and using them.

If you have cPanel, Plesk or another control panel for your website there may be a feature to generate this file through a web interface.

Cloud Computing, Shared Computing and other type processes are nothing new to computer technology.  In the very early stages most companies that needed access to large and powerful processors would use a remote terminal to log into a server. The tasks at that time were relatively simplistic compared to today’s standards.

In the past 15 years we have seen computer network backends go from mainframes to individual servers then clustered servers then blade servers and now cloud computing. There was also a progression of single core single processor computers to multi processor motherboards now multi core and multi processor computers and finally cloud computing that supposedly strings all of this together in one large mass.

The idea is similar to the borg where computing resources are assimilated and the collective (cloud) works as a master server. heh well its something like that.

Is Amazon Alone?
Cloud computing is now making its way into standard hosting companies and under utilized equipment is now being put to good use. But does this mean that along with overselling we will now see a handful of clients stealing available processes from standard accounts? Will your website be knocked offline or perform at a lower level because large clients need your servers brain power? This is probably likely.

If you want to findout more about EC2 products visit Amazon’s Webservices Site

http://aws.amazon.com/

For more information about cloud computing for standard customers visit RackSpace

http://www.rackspace.com