Yahoo's Challenge

By Thomas Claburn
February 2006, Issue 22

The world's in love with Google and search-based advertising. That leaves Yahoo, the Google of the '90s and still the most popular destination on the Web in terms of visitors, to prove there's a better way to deliver advertising. But to do that, Yahoo needs to translate the 10 terabytes of data a day its visitors create and turn that raw information into marketable insights.

Yahoo's chief data officer, Usama Fayyad, sells the story.

"Search advertising is great. I can match ads to intent," Fayyad says. "Well, guess what? When you're on the Yahoo network--whether it's travel, whether it's autos or researching universities--you're telling a lot more about yourself and your intent. And I can use that. I can turn that into a very powerful ad-matching machine, just like search is. In fact, in many cases, much more powerful than search. It's just that the market hasn't discovered it yet."

That's a pretty big "just." Yahoo's challenge is convincing advertisers and marketing companies that it has a data-driven model that creates a more effective means of reaching consumers than first-generation search-based advertising. It's a challenge the 12-year-old Web portal needs to overcome if it hopes to regain its position at the top of the Internet mountain after being shoved aside by Google, the darling of Wall Street and the favorite Web site of information searchers throughout the world.

Not that Yahoo is hurting. Its $5.26 billion in revenue last year was 47% more than 2004. Not bad. But not Google, which had $6.1 billion in 2005 revenue, a 92% increase from 2004. Meanwhile, Google has been gaining search market share at the expense of MSN and Yahoo. Between November 2004 and November 2005, Yahoo went from handling 32% of Internet searches to 29.5%, according to comScore Media Metrix. No wonder Google's market capitalization stood at around $106 billion earlier this month, more than double Yahoo's market cap of roughly $46 billion.

More Than Search

But Yahoo is much more than search. It has popular E-mail and instant messaging applications; social-networking, personals, and photo sites; extensive E-commerce operations; and news and other forms of content. Yahoo Music was the No. 1 music site on the Web, with more than 23 million unique visitors a month, according to comScore Media Metrix.

Yahoo also is more diversified than Google, with 12% of its revenue coming from user fees. As for advertising revenue, Marianne Wolk, a financial analyst for the Susquehanna Financial Group, estimates that 58% of Yahoo's ad revenue in 2005 came from search advertising, while 42% came from contextual ads designed to promote a brand. For Google, only 3% to 4% of its ad revenue came from advertising unrelated to search.

That's not all bad. Search advertising is growing faster than contextual advertising online. "I find it illogical to make a case that it's better to be diversified when search is growing so much faster online," Wolk says.

That's why Yahoo is laboring to find new ways to take advantage of its vast stores of content, computing horsepower, and employee brainpower. It's worth remembering that the Web site founded in January 1994 as Jerry and David's Guide to the World Wide Web by two Stanford University graduate students grew into one of the most valuable companies on the planet, with a market cap of more than $140 billion at its peak in January 2000.

It may be that the Google-Yahoo rivalry tends to be overstated. The online ad market is still young and both companies continue to grow. "It's not a zero-sum game," says Chris Sherman, executive editor at Search Engine Watch, an online search news site. Advertisers think they need to market through both Google and Yahoo, he notes. But it's Google that gets most of the attention now as it glides into new markets--news, shopping, images, maps--seemingly on a daily basis.

Other numbers highlight Yahoo's challenges: It generated revenue of $535,000 per employee last year, while MSN generated $648,000 and Google generated $1,484,000, according to market research firm Outsell. "Google's low head count and high productivity reflect its heavy reliance on technology to do the heavy lifting," says an Outsell report.

Beyond the metrics, Yahoo also has a perception problem. Research shows that advertisers rate Google higher than Yahoo on the effectiveness of its online advertising, says Chuck Richard, an analyst with Outsell. Google's advantage in search is often cited, Richard says, while it's rarely mentioned that Yahoo has more unique users and that they spend 10 times as much time on Yahoo.

The challenge is to translate what Yahoo has--more visitors spending more time on its sites--into something advertisers want. And that's where IT will help determine whether Yahoo is a contender or an also-ran. It involves leveraging its low-cost, high-performing IT infrastructure and its top experts in search and data mining to provide advertisers with more effective and relevant methods for reaching consumers.

The Difference Maker

One approach is to do more data mining to maximize the effectiveness of ads and show the world that Google's way of delivering advertising isn't the only one that works.

Yahoo is certainly within striking distance. In an Outsell survey of 1,200 advertisers released in January, some 70.9% of respondents rated Google "extremely/somewhat effective" for keyword search ads, compared with 61.9% for Yahoo and 46.1% for MSN. For contextual placement, those numbers were 46.8% for Google and 40.1% for Yahoo. MSN doesn't offer such ads.

That's why Fayyad is gunning for Google. Tall, with a shaved head and slight, hard-to-place accent (he's from Tunisia), he might be mistaken for a distant relative of the late actor Yul Brynner. He's a formidable presence, and with his increasingly impressive team of researchers, he aims to use Yahoo's massive knowledge of its users to improve the relevance of ads for users and the effectiveness of those ads for marketers.

Fayyad cites Yahoo's ability to define groups like "automobile purchase intenders." "Based on how you use Yahoo, you give me a lot of hints to the fact that you're in the market for a car," he says. "In fact, I can guess with very high reliability that you're interested in buying a car in the next 90 days." Armed with that knowledge, Yahoo can offer companies the ability to buy ads that will appear in front of the people they most want to reach.

It's an IT-intensive challenge. The 425 million users who visit Yahoo each month generate a data trail that amounts to 10 terabytes a day. And that's just usage data. It doesn't include E-mail or images. "We need to be able to take 10 terabytes of data every day, collecting it from hundreds of thousands of servers around the world, process it, reduce it, decide what to keep, what to get rid of, make sure we update all information we have about users, and age it correctly," he says. "And then drive a whole bunch of applications. That alone is a tremendous challenge."

Yahoo now argues that using a search keyword's bid price as the sole metric to determine the position of ads on a search result page isn't the best way to advertise. "That's not the optimal way to do it and, in fact, that's something we're changing," Fayyad says, signaling a shift toward Google's more lucrative approach of positioning ads using additional criteria like ad popularity.

Fayyad says his mission at Yahoo is to listen to what customers are saying through their actions and use that information to improve Yahoo's products, which will keep visitors on the site longer. "If I get 10% more usage from the average consumer, I have suddenly created 10% more inventory for my ads," he says.

Yahoo accomplished something like that last year when it made an effort to improve user retention with Yahoo Mail. As Fayyad tells it, while combing through data, Yahoo engineers noticed that users of the company's free E-mail service also read a lot of news. From conversations with the Yahoo Mail business unit, the engineers realized it wasn't as easy as it could be to switch between E-mail and news on the site. So they added a news preview module that let users read news from the Yahoo Mail screen. After two months, they found that it dramatically improved retention. "That went from noticing a data pattern to a real product in two months," Fayyad says. Company officials declined to discuss the size of Yahoo's IT staff or its budget.

Battle For Talent

Speedy enhancements like that can only happen with a lot of superior brain power. So Yahoo is competing just as fiercely for people as it is for ad dollars. Google has been wooing top scientists in its quest to organize the world's information and make it universally accessible. In early February, for instance, it poached Udi Manber, CEO of Amazon.com's search subsidiary, A9.com.

With its acquisition of Inktomi in March 2003, Yahoo not only got back into the search-engine race, but it also stepped up the pace of R&D. In 2004, it opened Yahoo Research Labs to work on Web search and information retrieval and last year partnered with UC Berkeley; this year it turned to university researchers in Spain and Chile. Last summer, it hired Prabhakar Raghavan, a former senior researcher at IBM and chief scientist at Verity, to run Yahoo Research. He joined noted researchers Andrei Broder and Jan Pedersen. In January, Yahoo hired data-mining scientist Ricardo Baeza-Yates to run its labs in Chile and Spain.

They're mining data collected in Yahoo's data centers, which are costly and, for the most part, hidden from view. There are 27 of them, more or less, around the world, filled with between 100,000 and 200,000 servers. InformationWeek was invited to one in Santa Clara, Calif. When you make your way past very tight security and step inside, you can feel the cost of cooling the company's machines as conditioned air beneath the raised floor blows up through perforated tiles. If data had a sound, it would be the drone of fans.

"All of our E-commerce activity takes place in here," Operations VP Kevin Timmons says. The majority--88%--of the company's $5.2 billion in revenue comes from ads, but the company has an active E-commerce business that it's trying to grow.

Keeping infrastructure costs as low as possible is a top priority. While that's a common theme at any well-run business, Yahoo's commitment to cost control is part of its DNA. And it's reflected in the company's choice of hardware: commodity servers running BSD Unix, along with a smattering of just about every other operating system that Yahoo gained through acquisitions.

But parsimony doesn't mean that Yahoo is willing to accept slow response times and a poor user experience. Yahoo must be available 24-by-7, and it must be fast, says Phu Hoang, senior VP of engineering, who oversees the company's apps. Yet, Yahoo still looks closely at every request for new equipment.

Cost consciousness is a necessary obsession. In a company so dependent on the ability to efficiently scale its computing and network infrastructure, even minor changes to the cost equation multiply into major problems. "We squeeze just about every penny out of every piece of hardware we can," Timmons explains.

Of course, as any business-technology manager who runs a data center knows, sometimes the hardware pushes back. Power consumption and the cost of electricity are a big challenge, says CIO Lars Rabbe, an amiable, unassuming blue-eyed Dane. Instead of going for the fastest processors, the cost of electricity and cooling in data centers is forcing companies like Yahoo to look at lower-power systems, like servers that use new dual-core chips from Intel and AMD, which consume less power and generate less heat. "That changes the whole dynamics of how you manage your data centers," he says.

A conservative estimate for the average annual utility cost for a 100,000-square-foot data center is around $5.9 million, says Edward Koplin, a principal at engineering firm Jack Dale Associates. Based on the square footage of its Santa Clara data center, Yahoo is probably paying close to that for that data center alone. And it has 26 others of various sizes.

Keeping costs in line as it battles the highly automated Google is a priority. It will take new services, features, and marketing to make Yahoo more competitive, enhance its image, change perceptions, and bring in more revenue. But the road back up the mountain contains many potholes and blind curves, things that can quickly change perceptions in the wrong way.

CIO Rabbe, whose first job was at a university data center back in the days of punch cards, knows that how Yahoo handles customer data can be the biggest pitfall of all. "Being responsible for the security of the world's largest Internet company can be pretty daunting," he says, explaining later that "we care deeply about personal information. This is something we really make a big deal out of."

Unfortunately, the governments of both the United States and China, and probably many more, also care deeply about personal information and have considerable interest in Yahoo's data. It's a situation also faced by AOL, MSN, and Google. And when interests of governments and citizens collide, governments usually win--to the detriment of personal privacy and, sometimes, personal liberty.

Yahoo is accused of supplying information to Chinese authorities that led to the 2003 imprisonment of an Internet writer who was charged with subverting state power and sentenced to eight years in prison. While surrendering data to government officials may be a legal requirement, it doesn't dovetail with public assertions that customer privacy is important. And in Yahoo's case, its compliance with Chinese authorities has resulted in the detention of journalists. That pretty much guarantees bad press. It's a no-win situation for Google, Microsoft, and Yahoo, which is why all three have asked the U.S. government to make free speech a free-trade issue. For Yahoo and other major Web sites, free speech has hidden costs.

The situation illustrates the risks Yahoo faces as more users worldwide rely on it for news, information, commerce, communities, and connections to like souls. But it also shows the important role the Web portal plays in the lives of many people--and the importance of the information it gathers on each of those people.

The challenge is for Yahoo to figure out a way to parley that information into more revenue, while maintaining the trust that keeps 425 million visitors returning to the site each month for more than just search.