1 Million RSS Feeds

I learned last week that Yahoo publishes over 1 million RSS feeds (!), a summary of which can be found here. This includes about 800k feeds from public Yahoo Groups and about 60k company news feeds from Yahoo Finance, but it does not count the dynamically generated feeds from Yahoo News (scroll down to “Create your own RSS news feeds”). That’s a lot of data for consumers and developers to tap into.

BTW, here are some other places to tap into Yahoo: Music Engine plug-ins, Search APIs, Flickr APIs, Search Marketing advertiser APIs, Maps links.

The Peak of Oil

A few weeks ago, I read about the idea that we are nearing the peak of world wide oil production in this Rolling Stone article. Now I’m all of a sudden seeing the idea pop up in several places. They were talking about it last night on NPR, the Guardian had this story, Boing Boing mentioned it. The theory is that sometime in the next 5-10 years we will reach the peak of world wide oil production (just like US production peaked in 1970 and has been steadily declining since). After that, production will start to decrease a few percent per year which will have an immediate effect on our lifestyles, energy consumption patterns and the geopolitical competition for oil. Apparently, this theory goes back to the 50s when a geophysicist named Hubbert predicted that the exploitation of any oilfield follows a predictable “bell curve” trend (Wikipedia entry). I find this fascinating because rather than talk about how we have decades left before we run out of oil, we are forced to face the possibility that within a few short years we will have to start making changes to our energy consumption patterns (i.e. reversing them from growing to shrinking). This seems like a plausible outcome. What I don’t agree with in these articles is the alarmist predictions of the world wide chaos that will follow the peak of oil. It seems to me that once we put our minds to it, we will be able to solve this problem and change our patterns more rapidly than people think (a possible analogy is the surprising speed with which people reacted to the world population growth problem in the 70s after there was critical awareness of it).

A New Project

I’ve taken on a new role at Yahoo, moving from the Mail team where I helped integrate Oddpost into Yahoo, to the Platform team where I will be working on expanding developer.yahoo.net. It’s a very exciting project. We’re in the process of opening Yahoo up to third party developers and expanding Yahoo’s reach from the traditional *.yahoo.com sites to Yahoo-enabled products across the web. More info will follow as we get closer to launching new things.

Paul Graham on Startups

I consider Paul Graham’s Plan for Spam essay to be a seminal piece (he popularized the idea of using Bayesian spam filters, which we found to be amazingly effective at Oddpost). His latest essay is How to Start a Startup. Having been part of 4 startups, I was interested to hear what he has to say.

Here’s a list of ‘startup tips’ I gathered from the essay:
* you don’t need a brilliant idea, just offer someone better technology than they currently have
* more important than the idea are good people
* good people = people who are ‘animals’ at what they do
* hire engineers who are smart, get things done and have bearable personalities
* have 2-4 founders
* the founder team should include technical people, business people are optional
* build products that customers actually want
* listen to and watch your customers
* do rapid prototyping
* go after niche markets instead of giant consumer brands
* focus on small, low-end customers first then grow to the high-end
* get your employment and IP ownership docs in place up front
* raise money from VCs if you can, but spend as little of it as possible
* deeply understand your business
* be frugal
* locate where you’d want to live, not in an office park
* hire as few people as possible to keep low financial and management overhead

Most of these points I find spot on. The core advice boils down to “hire great people, listen to your customers and be frugal”. That’s good advice for a startup to focus on. However, it’s also advice that applies to most teams at most companies, not just startups. This made me wonder what’s truly unique about startups versus big companies. Hiring great people or being frugal can be emulated by a big company, but what are some startups traits that are hard or impossible for big companies to have?

1. No legacy issues: One of the biggest weaknesses of big companies is the requirement to support legacy businesses, customers and technology. This weighs down product teams and forces them to be incremental and predictable. Startups have no legacy issues. They can start fresh and design a product from scratch, using the latest technologies and insights without worrying about existing customers and revenue streams. This is a big advantage and suggests that product areas with lots of legacy baggage are good targets for startups to attack.

2. Low complexity: Another big company weakness is lots of overhead. Big companies are often ridiculed for being slow and having too much overhead, but often there simply is no way around the complexity of dependencies between multiple products and development teams, complicated business partnerships, competing budget requests, diverging strategies and PR messages, etc. Startups have almost none of this complexity. Everyone is focused on one project and all decisions can be made within a single group and a single set of goals. That’s another unique advantage and suggests that startups should go after big, heavily distributed and interdependent organizations.

3. Money and fame: The more obvious startup trappings. Big companies simply can’t compete with the potential jackpot and recognition that comes from being with a startup that breaks through. While this should not be the primary reason for doing a startup, there’s nothing wrong with wanting it, playing it to your full advantage when hiring people and making sure everyone gets to participate.

4. Fun corporate lifestyle: Paul touches on this when he says “You want to live at the office in a startup, so why not have a place designed to be lived in as your office?” Startups can locate themselves in great areas where people want to live and hang out. People simply have more fun in downtown Berkeley than in one of those non-descript office parks that seem to attract the bigger companies. And when people have more fun they stick around doing more work and being more productive.

Here then are some additional ‘startup tips’ that I think go to the heart of what makes startups unique and different from established companies:
* find products/companies with lots of legacy issues and leapfrog them with a fresh, new product
* take on big, heavily distributed and interdependent organizations with lots of overhead
* maximize opportunities for fame and fortune for the entire team
* create and take advantage of a great corporate lifestyle

The Memory Prediction Framework


On Intelligence by Jeff Hawkins is the best book I’ve read in, oh, 5 or 10 years. It introduces a new unifying theory of how the human brain works and extrapolates it into a vision of how we can build smart machines (ones that unlike AI and neural networks exhibit true intelligence). I’m no expert in AI or neurobiology, but I found the book very exciting and very readable. It lays out how little we know about the workings of human intelligence, proposes a simple and elegant new theory to explain it and then steps through example after example of how the theory makes sense for all kinds of hard to grasp concepts like intelligence, perception, creativity and consciousness. The new theory, called the memory prediction framework, describes how the cortex, the outermost layer of our brain, acts as a giant hierarchical memory bank. We use it to store all the events we experience around us as abstract patterns. At the same time, we use previously stored patterns to constantly predict what to expect next. This leads to the book’s core stipulation that “intelligence is the capacity of the brain to predict the future by analogy to the past”.

You can read the prologue here. I highly recommend this book.