When famous US department store Macy's built its ecommerce web site, it wanted it to look just like its flagship New York department store. The graphics would mimic the ornate nineteenth century architecture of its Herald Square store in Manhattan, and all the products would be separated into their respective departments.

But although the web site attracted millions of virtual visitors, the company quickly realised that few of them were actually buying anything, says Gary Beaverman, director of Macy's Internet initiatives. Further investigation was required.

Armed with analytics software from Accrue, Beaverman could see that visitors did not browse the site in the same way that they would look around a physical store. Instead, they would go straight to the search engine to look for specific products. Simply by improving its search engine, Macys.com more than doubled its conversion rate in just one month.

What the Internet boom-and-bust has taught the survivors is that attracting visitors to a web site is the easy part. But persuading them to buy goods or to interact with the organisation in any meaningful way is still a challenge that many need to overcome. "There's often a discrepancy between what a site designer wants them to do and what users actually do," says Malcolm Duckett, vice president of marketing at analytics software vendor Speed-trap.

Jonathan Becher, senior vice president of corporate development and strategy at Accrue, concurs: "Most of the time web sites are designed for the web site designers." Building a web site that is easy to navigate is common sense, but organisations cannot know beforehand precisely what their customers want. That is why many companies have turned to web analytics software.

Web analytics can help companies understand how visitors use their site and thereby offer insight into what they want. It should not be confused with web measurement software, which covers more mechanical tasks such as click stream analysis – often just using web logs.

Web measurement software monitors behaviour such as the number of 'hits' a web page receives, the number of unique and repeat visitors a site has and the number of page views per visit, says Frank Buytendijk of analyst group Gartner. "The aim is to gain insight on a technical and content level." This general information can be useful, but it offers little insight into how customers interact with a site or the experiences visitors have on a site.

The goal of web analytics is more sophisticated. Its aim is to improve visitor's experiences, increase an organisation's understanding of its customers and to exploit this knowledge to increase sales and profitability.

There are several ways for measurement and analytics systems to collect data. The most commonly used method is to extract data from web logs. "For most organisations the log files are informative and quite cost effective," says John Woods, CEO of web analytics company Site Intelligence. However, such analysis does not offer detailed information at the level of the individual visitor.

Packet-sniffing technologies examine the packets between the web client and web server for more complete information. However, as Gartner's Buytendijk points out, these are all server-side technologies and the results can be misleading. For example, most Internet service providers (ISPs) cache popular web pages so that pages may be sent to a visitor from the ISP, and not from the site owner. This can result in understated traffic figures.

A more advanced technique is to embed a piece of JavaScript code on a web page. This code can collect data about interactions with the embedded pages. This data can be as detailed as where the user's mouse is. In this way, "the page watches itself" says Duckett.

But according to Duckett, adding JavaScript to an entire web site can be time consuming, as every element on the page must be tagged. Instead, Speed-trap embeds only a single Java applet tag on each page. "It's much simpler to implement," says Duckett.

Early stages
The web analytics market is still in its infancy. Little more than a year ago, few potential customers even knew what it was, says Becher. Today, all Fortune 500 companies are experimenting with web analytics software. "Perhaps 10% have done something serious," says Woods.

According to Jason Burby, director of web analytics at web design company ZAAZ, organisations have become increasingly interested in the technology in just the last six months as the economic environment has toughened.

ZAAZ helps to redesign web sites and offers web analytics consultancy. In the past, a customer might go to the company with "subjective ideas about what was wrong with their site, but no real foundation for the information," he says.

More sophisticated companies would have asked a few customers for their opinions, but little more. "People have said 'we've got to find a better way to be successful online'," says Burby. Organisations are now demanding more detailed data about how their sites are used, particularly as CEOs now want to see demonstrable return on investment (ROI) figures to justify all corporate technology spending.

Some of the results have been surprising. One enterprise software vendor wanted to increase registrations on its site in order to provide more leads to its sales team. It went as far as planning a major overhaul of its web site until, using its web analytic system, it discovered that the main problem was simply that the link to its registration page was not clearly visible and that only 22% of visitors were clicking on it.

Equally worrying, of those 22% only 45% finished the registration process. The forms were too long and too difficult to fill in, and most people were being directed to low-end products, regardless of what suited their companies best.

The vendor then changed its registration process and improved the visibility of some of its links. The result? An increase in its visitor-to-lead ratio of 49% and, a month later, the number of sales leads generated for its most expensive products had also risen by 65%. Sales leads for its second highest product leapt by 144%.

Vendors such as Site Intelligence and Accrue promise that companies typically enjoy a pay back on web analytics in three to six months. But that kind of ROI is dependent on ensuring that the software is correctly configured. Poorly-configured software is undoubtedly a problem, says Martyn Jobber, a senior analytics consultant with NetGenesis.

For example, Woods recounts the time when Site Intelligence went to see a prospective customer that was already using a basic web analytics tool. "They were tracking the conversion rate on their site as a key performance indicator… and they had the tool wrongly configured. As a result, all their key performance indications were wrong," he says.

In fact, the company's conversion rates were wrong by a factor of seven. The company thought it was converting very few customers and wanted to redesign the entire site. However, the real problem was that the number of visitors to the site was much lower than estimated, and conversion rates were actually quite high.

In some instances, poor configuration or use of web analytics software is the result of inter-departmental friction within the organisation. This is partly because it is typically the marketing department that drives the use of web analytics, but the software is controlled by the IT department.

Web analytics software can be expensive to both purchase and implement, but marketing departments may either ignore it or be ignorant of its potential, says Burby. At the same time, IT departments are often overworked and the two departments may simply not get on.

But the potential of web analytics software is immense, says Burby, and those organisations that have implemented it "are barely scratching the surface of what they can do," he says.