The Following thought piece was developed to help customers, partners and other internal groups at USWeb/CKS understand the need to design measurement into their websites and online marketing efforts.  This white paper also helped educate the audience as to our capabilities, methodologies and approach to solving their problems. Being the Director of Online Strategy at the time, I was the sole contributor and author of this piece.

Our Understanding
     The Web Measurement Holy Grail
     Today’s Problems
     Why Web Measurement Fails
Our Approach
     Key Questions
     Project Methodology
     Business Objectives
     Business Metrics
     Website Architecture and Design
     Results
     Measurement and Analysis
     Recommendations
About USWeb/CKS

Our Understanding

The Web Measurement Holy Grail         top

Once the Internet began to develop into a platform for business transactions, pundits and technology gurus quickly forecasted the development of “RealTime Marketing” and other similar revolutions that would shape the way we would market in the future. A key component of these projections was being able to measure the success of marketing and business decisions that transpire on the web.  Since everything on the web could be digitally captured, there was reason to think that this would lead to “data driven” business and marketing decisions.

The equation was simple, driven from the company’s business objectives, just apply the following equation,

Figure 1

However, this panacea of web measurement and return on investment has not yet materialized.

Today’s Problems         top

Even though there are a multitude of tools that measure different aspects of web traffic, usage, segmentation and behavior, most businesses fall drastically short of adequately measuring the effectiveness of their sites. Today’s problems stem from legacy off-line marketing (and for the most part business) methods and management.  In traditional marketing organizations, using experienced marketing professionals, how many marketing programs are explicitly measured to gauge success?  How many traditional advertising campaigns are explicitly measured or tracked to objectives?  These are rhetorical questions, but they point to today’s problem.  Marketing and business people, who have honed their stills in traditional roles, are not familiar with measurability or linking marketing efforts to business objectives.  Consequently, the holy grail of web measurement expressed in figure 1, is more likely to based on unclear and meandering business objectives, and expressed as,

Figure 2 

Why Web Measurement Fails          top

Web measurement efforts usually fail for two major reasons, web measurement is not an integrated process, and web measurement capabilities were developed as an afterthought.  While there are a multitude of subordinate reasons, these two reasons are the basis of most failures.

Web measurement is a process and not an event. The concept of web measurement as an ongoing iterative process is often lost in the throes of the design and development process. To compensate for the lack of a measurement process, random measurement exercises are sometimes deployed. However, random exercises in measurement and analysis provide only random data points and inconclusive results. Results from measurement and analysis need to provide a direct feedback mechanism into the development/improvement process.

The lack of considering web measurement and analysis as a process directly leads to today’s web measurements failures.  An effective web measurement process dictates that the development process is driven by business objectives.  This point is critical to the successful implementation of any website.  Keeping the business objectives in focus during the design and development process is the equivalent to the proverbial, “Keeping your eye on the prize.”  Without this business objectivity injected into the design and development process, projects often go astray based on errant technology decisions or tangential design considerations. 

In today’s world, time is the currency of success.  If issues and challenges can not be addressed quickly, competitors will win the day.  A flexible and efficient feedback mechanism is required to tune the development and marketing efforts to maximize the business objectives.  But flexibility is crucial.  The feedback process needs to have the capability of affecting the business objectives as well.  What if the target audience on-line is not the original audience targeted?  Should the acquired audience be abandoned, or should the business objectives be modified to serve this new group of target customers?  The process needs to be foreshortened to increase effectiveness.  Long involved feedback and analysis of direct mail and other offline marketing efforts do not work in the Internet world.  The feedback process on the web needs to based on substantial data, but in short iterative cycles.

A solid measurement process does not guarantee success.  Establishing clear and definable business objectives is not always an easy task. Websites, especially large ones, often have many functions and serve multiple constituencies within an organization.  Some websites are controlled and developed by different groups.  Even sites with seemingly clear business objectives, like “sell more stuff”, can become complicated by sales channels, distribution arrangements, compensation structures, and non-traditional customers.  Web measurability needs to integrate the complexity of these objectives, or it will be doomed for internal dispute and failure.

Even once business objectives are carefully defined, measurement and analysis systems often do not deliver on their promises.  Although many factors play a role, translating business objectives into site implementation is where most breakdowns occur.  A big problem is determining the business metrics.  Business metrics play a critical role in the translation process, and act as proxy for evaluating the business objectives.  Poorly defined business metrics can lead to ineffectual measurement, or the miss interpretation of results. Additionally, poor interpretation of business metrics from business objectives can lead to wrong application of technology and run up development costs. 

Finally, web measurement needs to be an integral part of the development process, not a kludged, appended afterthought.  Website technology and application choices made without a consideration of measurement, often lead to expensive redesign or ineffectual measurement results. Having measurement and analysis become a component in the technology and application selection process is key to successful website development.

Our Approach

USWeb/CKS views web measurement as an important part of our value proposition.  By helping clients measure the success of our web development projects, USWeb/CKS can easily demonstrate our value, and the importance of the web efforts. 

Creating a tight feedback loop and a flexible design approach, USWeb/CKS is able to leverage its broad range of our services, from strategy through creative to final implementation, to address the ever-changing situation and conditions of each client site.

Key Questions          top

When developing a new site or redesigning an old site there are a number of questions that need to be raised.  Answering these questions can are critical to setting the course of action of the measurement system, but more importantly, the design and development of the site.  While many of these questions seem like “big picture” strategic questions, they provide insight into the business objectives of the site, which are then used to drive down to tangible and measurable business metrics.  Many or all of the following questions can be part of a measurement and analysis system, if designed and implemented properly.

• Who is the target audience, and are they using the site?
• How does the visiting audience segment by behavior?  What are they doing on the site?
• How does this compare and contrast to expectations? To offline segmentation?
• What do you want visitors to do on the site? What is the “call to action”?
• Who are the best customers? Why?
• Is the focus on attracting new customers or strengthen the relationship to existing customers?
• What on-line ads generate the most “best customers”?
• What on-line and offline ads generate the greatest ROI?
• What on-line and offline ads generate the greatest number of new customers? By sement? By Ad? By location?
• What products and services are currently offered on-line?  In the future?
• What are the greatest opportunities for cross selling and up selling?
• What is the goal of on-line sales versus other sales channels?
• What are the sales objectives through all channels? By customer segment? By product?

Project Methodology          top

The project methodology used for web measurement is quite simple, but the implementation is more challenging.  The methodology follows a straight forward logical path: determine a comprehensive, prioritized set of business objectives; translate these objectives to measurable metrics; determining base line values for the metrics; architect, design and implement the site (or changes); capture results; measure against baseline expectations; suggest recommendations for next iteration of design. Illustrated in Figure 3 below.

Figure 3  Design for Measurability Methodology

Each of these components will be further discussed in the following sections.

Business Objectives          top

Although a difficult process, it is imperative that goals of the site be broken down to a comprehensive set of business objectives.  These high-level directives need to be more than just dollars and cents. They need to cover numerous aspects of the business.  It is equally important that the business objectives be as comprehensive as possible. To develop a comprehensive business objectives, a reasonable subset of the following business objective issues need to be addressed:

• Company, product and brand positioning
• Target customer capture and retention rates
• Customer service, support and satisfaction levels
• Products and services sales mix
• Product cross-selling and up-selling goals
• Sales and distribution channel goals
• Advertising ROI and effectiveness goals
• Resource allocation and effectiveness goals

It is more expensive and time consuming to add additional measurement capability after-the-fact, than to incorporate measurement capability at the time of initial development (or redesign).  However, since measurement and analysis is an iterative process, flexibility needs to planned into the system as well as in the business objectives themselves.

Business Metrics         top

Business metrics are the translation of business objectives into tangible, measurable indicators, and are the link between business objectives and results. Business metrics can range from very concrete numbers (e.g., sales volumes, ROI, numbers of visitors, downloads) to very soft soft indicators (e.g., % of visitors in key segments, % of satisfied customer responses, perception results of on-line brand audits).

Before the business metrics can be evaluated, baseline values must be established. Initially, these baseline values may need to be generated using methods outside the measurement and analysis system. New site design and redesigns of existing sites are somewhat different. There are many overlapping methods to generate the initial base lines for the business metrics between new sites and redesigned sites.  Regardless, there are numerous methods to determining base line values.  A sample of these methods are listed below

• Behavioral segmentation of logfiles, or other online segmentation technologies
• Offline primary research with target audience (surveys, questionnaires and interviews)
• Usability studies with target audience and/or specific user segments
• On-line research (surveys) with target audience segments or potential target audience segments
• Secondary research of similar competitors
• Review of marketing and advertising spending and results
• Competitive review of marketing and advertising spending

Website Architecture and Design          top

Website architecture, design and implementation is not a simple process, especially for sites of any significant size.  Injecting measurability and analysis into the process only complicates matters. Therefore, it is important that firms refining the business objectives and developing the business metrics work in concert with the web architects and designers.  USWeb/CKS’s advantage is that all of these constituents reside and work together in the same organization.  Our breadth and depth of service, covering all aspects of strategy and design, are unparalleled in the industry.

The process for website architecture and design is detailed and complicated and too extensive for a thorough discussion in this document.  However, a graphical representation depicted in Figure 4 illustrates how website architecture, design and development are driven from the same objectives as the measurement and analysis capability.  This is no coincidence.  As stated before, business objectives, site design and measurement and analysis all need to create a tight feedback loop for success.  For this to happen they all need to be generated from the same set of objectives.

Figure 4  Web Architecture and Design Methodology

The stacked representation of the three boxes within “Site Architecture and Design” represent how each layer must be constructed to support the requirements of the above layer. In this case, business applications need to support the requirements of the user interface.  Likewise, the technology infrastructure needs to be designed to support both the business applications and the user interface. This approach ensures that the details of what are specified are actually implemented.  In reality, there are many iterations between the layers, evaluating the requirements, possibilities, benefits, budgets, resources and time.  Similarly, web measurement needs to be designed from business requirements in order to ensure that results can be captured and measured, but with an eye on the costs, benefits, resources and time.

Results          top

Results are not simply the evaluation of hits, visits or dollars.  Results need to reflect the detailed translation of business objectives and business metrics.  Results in some cases may be a detailed behavior analysis, or the outcome of on-line primary research.  Whether the form of the results is a specific number or a “soft” value, it is important that the form is consistent with expectations and the base line values. If there is a mismatch in form between results and base line values, then the full value of the analysis and recommendations may not materialize.

Measurement and Analysis         top

In a perfect world, this stage of the process would be simple. Compare the known results with the predefined base line, and “Viola” – the answer.  While some of the business objectives, business metrics and results may deliver “near perfect world” results, much of the measurement and analysis phase will be making judgements on data to estimate results.  While this may seem like black magic or very soft and unreliable around the edges, it provides much more solid footing to make decisions than speculation and intuition.

Recommendations          top

Based upon the results of the measurement and analysis, recommendations then drive iterations of the process.  Recommendations are usually directed toward site design.  Recommendations are also directed toward the modifications or business objectives and business metrics.  Although business flexibility warrants openness to recommendations to change business objectives, normally these changes are not drastic. 

Examples of recommendations could fall under many different categories. Recommendations would be very specific and actionable, but would not make sense out of context of the project.  Therefore, the following are some grossly generalized recommendations to provide a flavor of the type of recommendations proffered (these are just a sample of potential recommendations):

• Design changes to enhance ease of use and customer experience: specific recommendations increase the ease of use in the buying or information search process, personalization to provide more relevant information to the user, content organization changes to simplify user information retrieval
• Design changes that reduce buying process mortality:  specific recommendations that reduce the steps and ambiguity of the buying process
• Design changes to enhance and extend brand perception: specific design changes to maintain consistency across the site and across other offline marketing programs. 
• Navigational changes to reflect the users’ on-line segmentation: specific content, design and navigational recommendations that reflect the users’ self-defined behavioral segmentation to aid in site usage and interaction
• On-line ad buy recommendations to enhance attract a specific user segment:  specific on-line ad buy recommendations to attract specific visitor segments to aid in customer acquisition
• On-line add buys to maximize return on investment:  specific ad buy recommendations to maximize return (either financial or customer acquisition) per ad dollar spent.
• Product and service pairs that represent the greatest opportunity for cross selling:  sales and promotional recommendations that reflect users’ behavioral preferences.
• Business applications or design changes to mitigate or avoid channel conflict:  specific recommendations that maximize efficiency and distribution to various sales channels, based upon channel results and analysis

About USWeb/CKS

Headquartered in Silicon Valley, USWeb/CKS is a professional services firm that works with clients to define strategies and implement innovative ways to build their businesses through the combination of expertise in strategy, Internet technology and marketing communications. With more than 2,200 professionals in 40 offices worldwide, we have helped hundreds of businesses advance their marketing communications programs as well as their traditional IT and Internet systems.  Our clients range from mid-sized businesses to large corporations, including 46 of the 1998 Fortune 100.  USWeb/CKS is a publicly traded company (NASDAQ:USWB) with annualized revenue of nearly $300M.

By blending industry-specific knowledge with expertise in strategy, marketing commu­nications and Internet technology, USWeb/CKS provides a uniquely integrated service offering to help clients define strategies and implement innovative methods for building their businesses. Our 4 core services are:

1.      Designing and implementing marketing communication programs that include corporate and product brand development, relationship marketing, and audience development.
2.      Constructing Internet standards-based systems that allow our cli­ents to establish one-on-one relationships with customers through customer intelligence programs, data warehousing, and customer response and service systems.
3.      Building internal technology systems that help employees access key information and share knowledge, accelerating the process of their work and building collaboration across the company.
4.      Designing systems and processes that improve the efficiency of our clients’ businesses by accelerating transactions, increasing accuracy, and streamlining the processing and manipulation of data.

         top