Sphere Blog Search

Om Malik is writing about his early impressions of a new blog search engine called Sphere. I’m an advisor to Sphere and excited to see that they are about go beta. Back at Oddpost, shortly before we got acquired by Yahoo, we were working with two guys, Steve Nieker and Martin Remy of ThinkTank23, who were going to help us with feed search and discovery for our RSS reader. I really liked them and their technology, but we never got to build a product together because of the acquisition. I thought the least I could do was introduce them to one of Oddpost’s lead investors, Tony Conrad. It turns out the three of them hit it off and started Sphere together with the goal of building a second generation blog search engine (one that can tell you what authority a blogger has on a given search topic, rather than just list everything that’s being said in the blogosphere). I’ve been very impressed with their product to date. If you use blog search frequently, please sign up for their beta. They are looking for people who are able to tell the qualitative differences between various blog search results.


SF Ski Jump

Quite a few people thought that ski jumping in our San Francisco neighborhood was annoying or excessive. I found it really exciting. Above is a (bad) snapshot of the jump. Much better pictures here and here.

Web 2.0

What is Web 2.0?

In the build-up to next week’s Web 2.0 conference, technology observers are debating what web 2.0 is (I’m speaking at the conference on a couple of topics). I think the term web 2.0 has reached a tipping point in Silicon Valley in the last couple of months. People have been prognosticating a shift in the evolution of the web for a few years now. The shift is towards the web as a computing platform on which to build applications (and away from a publishing platform for web pages). This new computing platform will consist of open and easily accessible web services such as search results, news data feeds or people’s shared photos. Software developers will use these services as building blocks to rapidly create whatever new products they can think up. I think the tipping point for this idea was Housingmaps. Built by a single developer in his spare time over the course of a couple of months, it makes the concept immediately obvious to people. Take two web 2.0 building blocks, in this case an RSS feed for Craigslist real estate listings and a maps API from Google, mix them together and you get a cool new app. Now have a few hundred thousand web developers repeat this process and you get a explosion of new ideas and products.

Housingmaps was quickly noticed by bloggers and written up in the NYT. Since then, I hear web 2.0 everywhere. As is typical when Silicon Valley is in the grip of a new buzzword, lots of ideas and people are piling onto the bandwagon. All of a sudden, every company with a corporate blog and an RSS feed is “web 2.0”. Conferences and roundtables are popping up. VCs are working overtime trying to figure out where the money will be made. I guess this is all part of the process of vetting an idea and finding the core that will enable true long term value. In the end, I believe we’ll find that this is a real and important trend that will be driven by a handful of building block service providers such as Yahoo! and Google and thousands of software developers who will be building new products such as Housingmaps. Incidently, an important part of my job at Yahoo is to create the building blocks, so I’m obviously totally unbiased.

PS: If you couldn’t get tickets to Web 2.0, check out Web 1.0 to be held across the street!

Update: I like this web 2.0 definition by Tim O’Reilly: “Web 2.0 is the network as platform, spanning all connected devices; Web 2.0 applications are those that make the most of the intrinsic advantages of that platform: delivering software as a continually-updated service that gets better the more people use it, consuming and remixing data from multiple sources, including individual users, while providing their own data and services in a form that allows remixing by others, creating network effects through an “architecture of participation,” and going beyond the page metaphor of Web 1.0 to deliver rich user experiences.”


Yahoo Mail Reviews Coming In

Official reviews are coming in.

“The new interface is stunning in its simplicity and ease of use”

“The new Yahoo Mail is far superior to Gmail”


Yahoo Mail Beta

The former Oddpost team and many people on the Yahoo Mail team have been working overtime to launch the new Yahoo Mail to public beta users this week. It’s very exciting to see such a high quality product launch to one of the biggest audiences on the web. You can sign up to try it here (not sure how long it will take to get an invite). In the meantime, there are lots of good reviews popping up. One of my favorites so far is by Paul Kedrosky (who seems quite unimpeachable when it comes to product opinions – make sure you click through, he’s got one of the best review opening lines I’ve seen in a while).


Many Sidebars

So Google released Sidebar which is competitive with Konfabulator. What I find interesting about it is that they called it Sidebar. There is already a similar product called Desktop Sidebar. More importantly, Microsoft has been showing a similar feature as part of Vista, their next version of Windows, and they’ve been calling it Sidebar for ages as well. Obviously Google knows this. Are they using the same name as Microsoft to annoy them or is sidebar supposed to become a standard name for this kind of thing?



It’s been a while, and I’ve been waiting for something important to blog: Rockstar: INXS rules! Just kidding (though Jordis is going to win).

But seriously, I’ve been working on a very exciting project: Yahoo’s acquisition of Pixoria, the makers of the amazing Konfabulator software. As of today, Konfabulator is part of Yahoo and available for free (it previously cost $20 per user). If you haven’t already, you have to try it out. Konfabulator lets you run awesome, beautiful mini-applications called Widgets on your desktop.

There are widgets to display weather reports, stock portfolios, kitty cat clocks, anything you want them to be. Widgets are very easy to make using XML and javascript. There are over 1000 of them at Konfabulator was built by three very talented guys, Arlo Rose, Perry Clarke and Ed Voas. They are the latest testament to how a small group of engineers and designers can have a huge impact in Silicon Valley. The three of them will join my Developer Network team at Yahoo where they will continue to push the envelope of their widget platform and help widget developers from around the world create and distribute amazing widgets.

Jeffrey points to some early press coverage.


New Yahoo Mail Coming

The fruits of Yahoo’s acquisition of Oddpost are starting to emerge into public view. Yesterday, we offered a few people a sneak peak of a new version of Yahoo Mail that’s based on Oddpost’s technology. There’s some discussion on it on the web. I particularly like Ross’s commentary.

Comparisons between the new Yahoo Mail and Gmail will be inevitable. However, I think it’s important to point out that the two products are aimed at somewhat different audiences. Yahoo Mail serves well over 100 million mainstream internet users. Gmail is aimed at early adopters (and as far as I know has a couple million users).

I’m a little biased, but I think this new version of Yahoo Mail will have a significant impact on people’s use of web mail. It will accelerate the move away from desktop mail by making web mail so fast and convenient that most users will never look back.


Decentralization Podcast

Just listened to a great podcast by Dave Winer. He talks about the internet’s great ability to decentralize markets (eBay for p2p commerce, Expedia for travel, etc) and how RSS, blogging and podcasting are decentralizing the publishing business by offering low cost publishing channels to anyone with a computer and allowing us to route around newspapers and radio stations, the previous gatekeepers of publishing channels.

What struck me most about this is the relationship between decentralization and increased information flow. Every time we decentralize something, we end up on the receiving end of a lot more information. For example, we can now book airline tickets without going through a travel agent, but we’re also faced with lots of travel information to sort through. Similarly with RSS and blogging, we can get news directly from ‘real people’, but we’re faced with thousands of blog posts everyday to scan. This could be a bad thing (information overload) or a good one (Dave stipulates that humans are actually quite well suited to this type of information processing). In either case, it seems like a great opportunity for software developers. People seem to have an insatiable appetite for an ever increasing amount of information which the internet is glad to supply, but the software tools to process and interact with that information are still in their infancy. For example, RSS readers are still in their first generation and barely able to let us effectively interact with blogging content and already the next waves of content – photos, tags, geolocation data, etc – are building.


1 Million RSS Feeds

I learned last week that Yahoo publishes over 1 million RSS feeds (!), a summary of which can be found here. This includes about 800k feeds from public Yahoo Groups and about 60k company news feeds from Yahoo Finance, but it does not count the dynamically generated feeds from Yahoo News (scroll down to “Create your own RSS news feeds”). That’s a lot of data for consumers and developers to tap into.

BTW, here are some other places to tap into Yahoo: Music Engine plug-ins, Search APIs, Flickr APIs, Search Marketing advertiser APIs, Maps links.


Konfabulator countdown

Konfabulator is counting down to their 2.0 release. Looks like their site will reveal some clues every day leading up to the release. Also, I just saw a very promising product that Adaptive Path is working on. Can’t wait to try it.


The Peak of Oil

A few weeks ago, I read about the idea that we are nearing the peak of world wide oil production in this Rolling Stone article. Now I’m all of a sudden seeing the idea pop up in several places. They were talking about it last night on NPR, the Guardian had this story, Boing Boing mentioned it. The theory is that sometime in the next 5-10 years we will reach the peak of world wide oil production (just like US production peaked in 1970 and has been steadily declining since). After that, production will start to decrease a few percent per year which will have an immediate effect on our lifestyles, energy consumption patterns and the geopolitical competition for oil. Apparently, this theory goes back to the 50s when a geophysicist named Hubbert predicted that the exploitation of any oilfield follows a predictable “bell curve” trend (Wikipedia entry). I find this fascinating because rather than talk about how we have decades left before we run out of oil, we are forced to face the possibility that within a few short years we will have to start making changes to our energy consumption patterns (i.e. reversing them from growing to shrinking). This seems like a plausible outcome. What I don’t agree with in these articles is the alarmist predictions of the world wide chaos that will follow the peak of oil. It seems to me that once we put our minds to it, we will be able to solve this problem and change our patterns more rapidly than people think (a possible analogy is the surprising speed with which people reacted to the world population growth problem in the 70s after there was critical awareness of it).


A New Project

I’ve taken on a new role at Yahoo, moving from the Mail team where I helped integrate Oddpost into Yahoo, to the Platform team where I will be working on expanding It’s a very exciting project. We’re in the process of opening Yahoo up to third party developers and expanding Yahoo’s reach from the traditional * sites to Yahoo-enabled products across the web. More info will follow as we get closer to launching new things.


Paul Graham on Startups

I consider Paul Graham’s Plan for Spam essay to be a seminal piece (he popularized the idea of using Bayesian spam filters, which we found to be amazingly effective at Oddpost). His latest essay is How to Start a Startup. Having been part of 4 startups, I was interested to hear what he has to say.

Here’s a list of ‘startup tips’ I gathered from the essay:
* you don’t need a brilliant idea, just offer someone better technology than they currently have
* more important than the idea are good people
* good people = people who are ‘animals’ at what they do
* hire engineers who are smart, get things done and have bearable personalities
* have 2-4 founders
* the founder team should include technical people, business people are optional
* build products that customers actually want
* listen to and watch your customers
* do rapid prototyping
* go after niche markets instead of giant consumer brands
* focus on small, low-end customers first then grow to the high-end
* get your employment and IP ownership docs in place up front
* raise money from VCs if you can, but spend as little of it as possible
* deeply understand your business
* be frugal
* locate where you’d want to live, not in an office park
* hire as few people as possible to keep low financial and management overhead

Most of these points I find spot on. The core advice boils down to “hire great people, listen to your customers and be frugal”. That’s good advice for a startup to focus on. However, it’s also advice that applies to most teams at most companies, not just startups. This made me wonder what’s truly unique about startups versus big companies. Hiring great people or being frugal can be emulated by a big company, but what are some startups traits that are hard or impossible for big companies to have?

1. No legacy issues: One of the biggest weaknesses of big companies is the requirement to support legacy businesses, customers and technology. This weighs down product teams and forces them to be incremental and predictable. Startups have no legacy issues. They can start fresh and design a product from scratch, using the latest technologies and insights without worrying about existing customers and revenue streams. This is a big advantage and suggests that product areas with lots of legacy baggage are good targets for startups to attack.

2. Low complexity: Another big company weakness is lots of overhead. Big companies are often ridiculed for being slow and having too much overhead, but often there simply is no way around the complexity of dependencies between multiple products and development teams, complicated business partnerships, competing budget requests, diverging strategies and PR messages, etc. Startups have almost none of this complexity. Everyone is focused on one project and all decisions can be made within a single group and a single set of goals. That’s another unique advantage and suggests that startups should go after big, heavily distributed and interdependent organizations.

3. Money and fame: The more obvious startup trappings. Big companies simply can’t compete with the potential jackpot and recognition that comes from being with a startup that breaks through. While this should not be the primary reason for doing a startup, there’s nothing wrong with wanting it, playing it to your full advantage when hiring people and making sure everyone gets to participate.

4. Fun corporate lifestyle: Paul touches on this when he says “You want to live at the office in a startup, so why not have a place designed to be lived in as your office?” Startups can locate themselves in great areas where people want to live and hang out. People simply have more fun in downtown Berkeley than in one of those non-descript office parks that seem to attract the bigger companies. And when people have more fun they stick around doing more work and being more productive.

Here then are some additional ‘startup tips’ that I think go to the heart of what makes startups unique and different from established companies:
* find products/companies with lots of legacy issues and leapfrog them with a fresh, new product
* take on big, heavily distributed and interdependent organizations with lots of overhead
* maximize opportunities for fame and fortune for the entire team
* create and take advantage of a great corporate lifestyle


The Memory Prediction Framework

On Intelligence by Jeff Hawkins is the best book I’ve read in, oh, 5 or 10 years. It introduces a new unifying theory of how the human brain works and extrapolates it into a vision of how we can build smart machines (ones that unlike AI and neural networks exhibit true intelligence). I’m no expert in AI or neurobiology, but I found the book very exciting and very readable. It lays out how little we know about the workings of human intelligence, proposes a simple and elegant new theory to explain it and then steps through example after example of how the theory makes sense for all kinds of hard to grasp concepts like intelligence, perception, creativity and consciousness. The new theory, called the memory prediction framework, describes how the cortex, the outermost layer of our brain, acts as a giant hierarchical memory bank. We use it to store all the events we experience around us as abstract patterns. At the same time, we use previously stored patterns to constantly predict what to expect next. This leads to the book’s core stipulation that “intelligence is the capacity of the brain to predict the future by analogy to the past”.

You can read the prologue here. I highly recommend this book.