Guide to Web Site Application and Performance Management
Get what you need from your Web management platform
Web development expert hands out best practices adviceBy Beth Schultz
Thomas Powell, founder of Web development firm PINT and a Network World tester, has been monkeying around with Web applications since the commercial 'Net's formative days. Over the years, he's accumulated tons of experience building Web applications and making sure their performance stays up to snuff. Here he provides some of his best-practices advice for dealing with Web site application and performance management.
1. When you're monitoring for performance, make sure you're truly seeing your application from a real user's perspective.
Unfortunately, most Web site owners today are in a state of "blissful ignorance" when it comes to understanding how their applications are performing, Powell says. This is especially the case at companies with public-facing Web sites.
"On public-facing Web sites, there really is a sense of 'Oh, we're doing it right because I see it all right.' … But they don't see it through the eyes of the people who are out on the 'Net. They might try, but they'll have a lot of monitoring systems and a full-blown NOC, so that's always from their perspective. They're not seeing it from the user perspective," he says.
When users experience a problem with a Web application, they rarely take the time to alert a company that something is amiss on its site, and the performance management tools available today mostly only provide one piece of the picture. Depending on the type of Web application and performance management tool in use, a company might have great information on how fast packets are traveling end to end, or how many page views a user generated while on the site. But really, those things don't amount to much because they're not telling you what the user is experiencing.
As an example, Powell recounts being called in to help one company figure out why users were experiencing performance problems with a new Flash animation program. The company was monitoring file-transfer speeds across the WAN, and the packets were flying across broadband links. But the company wasn't accounting for this reality: In some cases, those broadband links attached to older systems with CPUs that got bogged down in processing the animation.
The fix was easy enough, he says. Now when the application installs, it profiles the user system, determines what it can handle, and adjusts itself to deliver just the right amount of "richness."
"These types of problems are all solvable," Powell emphasizes. "But first people have to acknowledge that it doesn't matter what they see – it only matters what the user sees."
2. Get your network and Web application development teams on the same page, so to speak.
Enterprises need to engender a cross-understanding between the Web application development and network teams in order to address the performance management challenge adequately, Powell says. To get to the root of a problem quickly – or better yet, to identity and fix a problem before it crops up for users – you can't have the people responsible for network monitoring doing their thing over in one corner and the developers checking application status in another corner. This gets back to understanding the user. "Users don't see an infrastructure piece and an application piece. They just see it as a whole system."
Web application and performance management vendors like Coradiant and Tealeaf Technology are on the track with their efforts to provide a more holistic view through their tools. But they face a big challenge in having to knock down the application developer/network administrator divide, he says. If the application team receives a performance alert, it needs to be able to find out what's happening on the network or in the servers, and vice versa.
"You have to train your developers to be network- and security-aware. That's the reality of it," he says.
3. Figure out exactly what it is you're trying to determine about your Web site before you select a management tool.
Remember, there are no magic microscopes out there, Powell says. "People are looking for a box or software that's like a burglar alarm. It'll give you an alert and do stuff, but it doesn't tell you the questions you need to be asking …. If you're not asking the right questions, the technology doesn't matter."
So a company needs to figure out what information it needs in order to do the job well. "If you're a CEO with an e-commerce site or a person who runs an HR site internally, you have a problem, a concern, a worry. 'Why aren't they putting more stuff in my cart? Why aren't they filling out the résumé form? Why do people complain about the form?' These are the questions you want answered; design or find the technologies and tools to answer them."
4. Make it easy for users to let you know when they're having problems on your site.
"You've seen sites that simply ask, 'How do you feel about this page?' and ask you to rate it. That's great," Powell says. "Most sites don't allow people to complain easily enough. Maybe you can click to open an e-mail but you can't vote instantly."
Too often, companies use page views as a measure of success, he adds. Just because a site visitor generates a lot of page view doesn't mean the person is interested. "That person could be frustrated, spinning around in circles looking at the same thing over and over [but never finding what they need." And in such a case, "that person does not like you. But how would you know that? You have to provide an easy way for them to tell you."
5. Always remember, your job is never done.
One certainty about Web application and performance management is that the process is ongoing. Keep asking the right questions, Powell says, and you'll keep discovering things you otherwise wouldn't have.
Top five questions before buying into a Web management scheme
Experts say to home in on data gathering, correlating processesBy Beth Schultz
How does the tool gather performance data?
Tool vendors approach the monitoring challenge in different ways. Some monitor passively, at the server, while others use agents out in the infrastructure and run synthetic transactions, for example. Some have a network bent, providing insight across the network, including the last mile, while others are more application-centric. The best tool architecture most accurately reflects the user experience, says George Hamilton, director of enterprise infrastructure, Yankee Group.
How does the vendor handle data correlation?
A Web application and performance management tool that cannot hook into other internal monitoring systems isn't a whole lot of good. "If that data is in total isolation, it doesn't give you a clue as to where to look for the problem, and it doesn't help with diagnostics and troubleshooting," Hamilton says. You need to be able to correlate the data with other systems and assess whether the problem resides in the network, on a server or perhaps in a bad piece of code. "The more help a vendor gives me on the diagnostics piece, the higher it should rate in an evaluation," he says.
And don't forget to ask who's going to do the initial and ongoing integration work, adds Jasmine Noel, principal at Ptak, Noel & Associates. Some vendors, like Indicative Software, are great at providing adapters for ease of integration. But the same should be true of any vendor you select, she says. "That's how you future-proof."
How sophisticated is the modeling engine for service-level agreements?
Some of the tools' SLA modeling engines will look only at the transaction, while others will provide a deeper view, showing how different classes of users interact with an application, for example. Also important is finding out how the analysis engine will integrate with other IT systems. For example, a good engine will recognize when an SLA will be missed, identify where the problem likely resides and take action, such as alerting the help desk or Web application manager, or entering a trouble ticket, Noel says.
What does the installation process entail?
If you're looking for ease of installation, a Web application and performance management appliance may be your preferred technology choice. That's one of the factors that drew Patrick Gardella, director of online architecture for Discovery Communications, in Silver Spring, Md., to Coradiant's TrueSight product. You plug it in, attach a cable to the span ports and begin accessing data. Without agents to install, the amount of time it takes from plug in to data capture is only about 20 minutes, he says.
How much support can my team expect?
As Tom Eckfeld, senior project manager at OhioHealth, a nonprofit healthcare organization in Columbus, Ohio, evaluated Web application and performance management tools, one thing stood out about Indicative Software, which he ultimately selected: the company's thoroughness. "At the front end, they really wanted to make sure they truly understood our requirements," he says. The vendor team hasn't slacked off, either. "The CTO and that team are constantly looking at ways to enhance the product and make it more valuable to us, and they've tried to stay in tune with new things we are doing and look for ways to help us out."
Choices abound for tracking Web performance
Mature products hone in on providing meaningful statsBy Beth Schultz
Most any company operating an internal or public-facing Web site relies on one type of monitoring tool or another to determine whether its applications are performing up to snuff. The choices are plentiful, with vendors evolving their mature core products to deliver ever more meaningful data.
Major players include Citrix Systems (via its Reflectent Software buy) and PremiTech, which provide management software combined with distributed agents that are deployed on client machines to capture application performance; Compuware, ProactiveNet and Symphoniq, which offer monitoring software that collects performance metrics across an infrastructure; Coradiant, Indicative Software and Tealeaf Technology, with appliances for monitoring traffic and capturing response times and other metrics while users interact with applications; and Gomez and Keynote Systems, which provide synthetic Web application and site performance measurement tests – otherwise known as Web analytics.
One particular thrust of late is to provide a more holistic view of the application in context of the business and IT goals. For example, Tealeaf Technology now offers a dashboard application aimed at letting users of its flagship application performance-management product, Tealeaf CX, better understand why customers abandon online applications and Web sites. The dashboard application, Tealeaf cxView, offers IT and business managers a portal through which they can monitor real-time application performance from the user or customer perspective and pose what-if scenarios to improve performance going forward.
Coradiant, too, updated its Web-traffic-analysis product with business-intelligence capabilities. It offers the Web.I WI-2100 Performance Analytics appliance, which plugs into the network and works with Web traffic and transaction data collected by its TrueSight Web performance management appliances. Web.I then adds business-intelligence analytics to show IT managers how well Web applications perform based on several criteria, such as user, geographical location or branch office.
Coradiant and Tealeaf are on the right track with these developments, says Thomas Powell, founder of Web development firm PINT and a member of the Network World Lab Alliance. Yet, these vendors face a hurdle getting over the network and application divide that persists within most IT departments, he cautions (see best practices. Need to link to that./cb).
Jasmine Noel, principal at Ptak, Noel & Associates, sees many good things coming from the ability to tie together Web-oriented business analysis, such as whether a company is successfully converting browsers into paying customers, with the IT performance information. This type of Web application and performance management tool, for example, not only could enable a company to map out the Web applications needed to deliver a new service, but also could provide a better understanding of the resources needed to support it. "If you can bring those two worlds, those two sets of analysis, together, you have a stronger understanding of how profitable a particular service would be. If the service needs five-nines availability to be valuable to users but it's going to cost you gazillions of dollars to deliver those five-nines, you don't want to be in that business," she says.
Bringing together the business side, Web application analysis and IT performance availability "is the future of where these products will go," she adds.
The real-time monitoring now becoming available through these tools has been a huge plus for users. Financial-services provider Sallie Mae, for example, uses Tealeaf CX as part of the flexible, real-time platform it developed for spotting system events that cause Web bottlenecks. Sallie Mae earned a 2007 Network World Enterprise All-Star Award for the project.
At OhioHealth, in Columbus, Ohio, Indicative's real-time monitoring and holistic view have helped change the way the organization approaches Web application and performance management. "The nice thing is, we typically are aware of a problem before our end user is. We now understand that something is not performing as it should and can start problem resolution and escalation before we hear from users," says Tom Eckfeld, senior project manager at the nonprofit healthcare organization. "Also nice is being able to accumulate all the various items into one location. We don't have to bring up 10 different products to do problem resolution."
Eckfeld says he doesn't consider the Indicative tool a "be-all and end-all" product but says it's far better than nothing.
As the Web application and performance management mainstays adapt their products for more rigorous business demands, other types of companies are looking to participate in the action, too.
Noel points to companies like ClearApp, which tries to understand performance by correlating the services provided by applications to the underlying code components supporting those services, as providing an interesting twist on the traditional relationship mapping concept as well as a new spin on application performance management.
This approach to Web application and performance management could really heat up now that enterprises are truly beginning to embrace service-oriented architectures. Traditional tools don't jibe all that well with SOA because that application model uses no defined infrastructure, Noel says. "From a technology perspective, SOA is the one that will twist [the traditional vendors]."
Mixing methods for Web managementBy Thomas A. Powell, Network World Lab Alliance
Web applications are composed of a multitude of hardware and software technologies any of which may slow or break the entire system from the end user's perspective. Application owners struggle daily to understand the details of individual failures across an entire Web system well enough to spot, reproduce and correct such errors. Web management systems aim to solve this challenging puzzle by piecing together data from a wide range of server, client and network components to show what went wrong and help professionals figure out what to fix.
Interestingly the approaches to Web management systems vary nearly as much as the monitored components themselves. Offerings are clearly different in their philosophical approaches but can roughly be divided between those that are more systems oriented monitoring and those that are more application, or user, focused.
Systems-focused Web management offerings, which can be both standalone products or delivered as services (like that offered by Gomez) that attempt to mimic end user interactions, initially focused on basic availability and delivery rates of any Web-based application. Beyond watching simple availability parameters, more recent products can also monitor for HTTP response codes and expected response text messages.
While older, more traditional Web monitors can alert on many application failures, they unfortunately are limited by what the specific parameters they test for. And, even when they alert administrators to the problems they do pinpoint, they usually lack the kind of detail necessary to pinpoint or sometimes even reproduce the problem. Modern Web management systems, such as offerings from firms like PremiTech (premitech.com), add more detailed monitoring of the various layers of a Web application including database, Web server, load balancers and at least local network conditions. To attain such extra detail traditional SNMP facilities will be aggregated from the various layers, but very often agents will have to be installed on monitored systems or devices which makes deployment more difficult.
Agent less approaches do also exist and often collect data via network taps for the purpose of reconstructing user experience and measuring overall performance. Players such as Coradiant (coradiant.com) do an admirable job rebuilding application activity and measuring performance data by passively looking at network traffic. However, without correlating system data such as what might be collected via logs, SNMP, or agents a passive network-focused solution alone may tell only part of the picture.
Obviously the ultimate goal of Web management is to provide essentially a Tivo-like replay of the customer experience, outlining both the server and network state quickly and clearly. Even early to market players like TeaLeaf (tealeaf.com) aimed to meet this lofty goal, but this isn't a trivial undertaking.
Even if it were possible that a single approach or even a combination of available approaches could fully address client, server, and network data collection with perfect replay, significant challenges would still remain. The usefulness of total Web application awareness will certainly be obscured in an explosion of data thereby making the value of such management tools more knowing what to monitor, how to filter to that which is interesting, how to diagnose that data and then what to do to fix the related problem. Regardless of the bright future of Web management tools, no combination of speedometers, replay cams and dummy lights will replace the need for truly skilled Web management professional at the helm so invest in that first and foremost.