Notes from the last panel of Vertical Leap
Moderator barney pell, mayfield
ofer ben-shacher, RawSugar
julia komissarchik, Glenbrook industries
Paul pangaro, snap.com
bob syman, pubsub concepts
Julia: five words
don’t search, question get answers
our company harvests fact form the web to give to users are answers.
bob, pubsub:
we monitor the weblogs real time + many others including FAA, earthquake warnings, specific focused streams
ofer, RawSugar
better search, courtesy of your friends
We define ourselves as a social search engine. you can see bookmarks of freinds, of community. many companies in the same area, our secret sauce is we can show tags underneath the first tags, but go beyond.
Paul, snap.com
an alternative experience for experienced searchers. all search is local search: me. now.
the idea is to put the power in the user’s hand to control in real time the ranking process.
Q: what’s the economics of creating a microvertical. How to extract the structure from millions of pages?
Julia– six challenges
1. Dynamic pages versus static is a problem. Not even passworded. Did you have enough intelligence to ask the right questions to get a reasonable result? (Job or flight search) complex queries in dynamic web.
2. Once you’ve extracted those pages, precision becomes a critical issue.
3. Once you have a fact, how do you get additional information? Once you have a flight how do I combine with car, restaurants.
4. Long tail– how do you find the smaller players? How do you get the local plumber’s website?
5. You have to have flexibility; one vertical may not be enough. Maybe market size is too small, or maybe one vertical leads to another such as travel and local.
6. Speed of processing. Facts are harder than pages.
We do that.
Q: is it always pull
Bob: it’s not always going to be pull. Pull just won’t do it. Low latency requires publishers inform engines. Push based models are back. That model responds to the publisher’s needs. When a new job, a new flight a new product arrives, people can know about it. A lot of search takes advantage of temporary and anecdotal limitations. Like push– we haven’t had an infrastructure that supported the changing web. The offers that appear and are only good for a day or two. Big sites take weeks to get around. but big search engines have different index cycles for different sites… As we layer in IM, blogging comunity efforts, letting blogs tell search engines when the y update, we’ll see changes in push. One place engines can own vertical is when they have structured information. The big guys are good with unstructured information. But structured data allows more. Technorati, semantic web will change things, in the future when someone wants to hire, they’ll blog it and the search engines will find it instantly. Another bug in search engines is that they can’t’ understand the content, but structured content will also change that.
getting tired. But in the last session. Must stay tough!
Q: will all content be decided by publishers
Ofer: we don’t think so. Sure you can have some publisher decide they want to go for the greyhound market, but more likely you’ll have a bunch of enthusiasts who get together. If you look on a search engine you’ll find directories made by enthusiasts, and you’ll then have to go through them. But with searching on shopping.com you can get listings, narrow by facets. Think about taking all of that and bringing it to everybody. Suppose everybody has access to that. The greyhound and tag their sites and suddenly everyone has a better search.
Q: will people will go to verticals?
Paul: no people will have to spend too much time trying to figure out where to go. On the business side– what does a vertical mean? But it’s so important to everyday life, portals and search engines feel they have to serve it. News is common, real estate is valuable… there is a range of value to help choose. Where will the traffic come form if I am a narrow vertical? If many sites have to build their traffic, it’s a problem. Snap thinks of itself as a general search engine, but verticals have promise– digital drive shaft. Something like a camera is something we understand. As UI, my expectations are clear on travel, or news, I have a cognitive map. But I go somewhere and suddenly the dropdowns don’t work and ajax isn’t there, a common UI becomes desirable. imagine a world in which whenever I go to an interface, no matter what the vertical is, the UI is always be the same. If you can harness quickly, provide a uniform experience for the user, verticals become like horizontals.
Paul: when we are searching, we don’t want to search. we are trying to comprehend things in order to act. when would we want different interfaces?
Bob: yes, genealogy… you have specific needs, and ways of talking. you don’t want a list, you want a graphical representation, and you want to see relationships.
Paul: the richness is not in listings, listings are out.
Bob: listings are always nice for basic stuff, but there are places where you want more.
Julia: we can never get semi-structured information that is rich enough to get via tags only. you will always need another layer to help you extract information.
Ofer: it’s amazing to me that we’re talking about search which is supposed to be about helping us find, but in real life it’s always about what people we know that is important. but in search we are not talking about that importance.
Q: why not rss, Bob?
Bob: we do use it. atom and rss, for those who want to pull. we ofer both for politics, since both will work with all the aggregators. we also use push feeds, we use the atom format. But it doesn’t really matter. Atom is designed for push. rss is not, which is why we use it in the push environment, but in pull we allow both.
Q: advice of young companies starting vertical search.
bob: unless you have really special access to unique stuff, don’t build a walled garden. walled gardens are going to be harder to defend, so build on a better experience not captured data.
Ofer: search is affecting every decision… searching the internet is like searching the world. this will grow in importance. one advice is that this market is still in infancy. think big. many things can be done that no one is doing, and in 0-20 years it’ll be different. don’t’ think of the next little improvement I can make on this o that engine, think of how you can change the paradigm.
Paul: watch out for tags. tags are not meaning. given all the disagreement, I say it’s early and we’re in trouble. don’t presume everything we do now is everything. the tech is young, the biology is old. technology is disappointing because it’s implicit and brittle, but biology is the opposite… start with the conversation.
Julia: find your niche.