Clues to improving the performance of your e-commerce Web site could lie right at your feet. The Web logs recorded by most Web servers contain a mother lode of valuable information. Unfortunately, this log data is too often ignored.
Your Web log can tell you how visitors find your site, the search terms they use, and the days and times your site is most popular, along with other information.
You can use this information to measure the performance of changes you make to your site, and to assess the impact of advertising and links from other sites.
What Is a Web Log?
A Web log records data viewed by each visitor to your Web site. The data ranges from the highly useful, such as the URL that referred your visitors, to the arcane, such as the Web browser and operating system they use.
A Web log records one line of information for each piece of data viewed from your site, which could be an image, HTML page, or something else. A busy Web site may generate thousands of lines in its Web log each day. That quantity of data is impossible to analyze efficiently without software to make sense of it.
Web Log Analysis Software
There are many different log analysis apps. I like AWStats. It's a free, open-source app that analyzes measurements that are useful for e-commerce sites. It works with logs generated by Apache and Microsoft IIS, the two most popular Web servers. The latest version of AWStats, released in July, adds support for new top-level domains such as .jobs and .mobi.
Many Web hosting services include AWStats free with the hosting package, which means you can run it directly from your Web site. If your hosting service does not provide AWStats, you can still use the app by downloading it and installing it on your PC. You'll also need to download your Web log file from your server.
Because AWStats is written in Perl, you'll need to install this programming language. A free Windows version, ActivePerl Standard Distribution, is available from ActiveState.
Analyzing Your Web Log
AWStats displays a wide array of information. A Web site traffic report summarizes visits by day and month, showing the number of visitors, visits, pages, and hits, and the bandwidth used. Numeric data is graphically displayed using simple bar charts that allow you to spot trends at a glance.
Counts of hits or pages aren't all that relevant for an e-commerce site. One can increase hits by adding more graphic images to a page. Adding pages to a site can also increase page views. For e-commerce sites, the number of visitors is key, because visitors are prospects for the products and services you offer online. If you increase visitors to your site, you display your wares to more potential buyers.
Another key AWStats analysis shows how visitors connect to your site: direct entry of your URL, a bookmark, referral from an Internet search engine, or a link from another site. A more detailed analysis shows the keywords that visitors use to find your site on a search engine.
If you aren't getting the number of visitors that you expect from search engines, AWStats can help you determine the problem. For example, the section headed Robots/Spider Visitors shows whether your site has been visited by Googlebot, Yahoo Slurp, and MSNBot, the online agents for the top three search engines. If you don't see these listed, then notify the search engines by adding your site's URL to Google, Yahoo, or Microsoft Live Search.
While AWStats offers a wealth of useful reports, particularly for a free product, it may not provide enough information for large e-commerce sites. For example, it does not show visitor's paths from the first page visited to the exit page. You can view a sample report at the Demo section on the AWStats page.
Alternatives to Web Log Analyzers
In an earlier column, I discussed a service called HitTail. This service analyzes the search keywords used by your visitors and offers suggestions for improving results.
However, Web analysis service may not be sufficiently flexible to provide all the information you want. These services do not analyze your Web log; rather, they analyze a proxy for it, as recorded by their code. While the results can be close, they're usually not identical.
Keeping Up With Analysis
It's hard to measure your progress if you don't know your starting point. A Web log analyzer can show you where you stand by accurately measuring your recent activity. I recommend you analyze your Web logs periodically, say once a month, to assess your progress.