Also of note: eRDF has dropped off the map. hAtom and hReview are continuing their climb.

-m

Photo by Micah

I know he left many things undone, for example only enhancing 60% of the delivered result pages. He never got a chance to finish his life’s ambition of promoting RDFa and microformats to the masses or to be the killer app of the (lower-case) semantic web. You could say he will live on as “some of this structured data processing will be supported natively by the Microsoft platform”. Part of the monkey we loved will live on as enhanced results continue to flow forth from the Yahoo/Bing alliance.

The SearchMonkey Alumni group on LinkedIn is filled with wonderful mourners. Micah Alpern wrote there

I miss the team, the songs, and the aspiration to solve a hard problem. Everything else is just code.

Isaac Asimov was reported to have said “If my doctor told me I had only six minutes to live, I wouldn’t brood. I’d type a little faster.” Today we can identify with that sentiment. Keep typing.

-m

I’ll have more thoughts and perhaps implementation notes on this later. -m

For example, to see every hAtom-bearing page that mentions ‘dubinko’ use the query [searchmonkeyid:com.yahoo.uf.hatom dubinko]. Works similarly for hCard, hCalendar, hReview, and XFN. I’m sure more are coming soon too. -m

So what exactly is it?

A few months ago I went through the pages on this very blog and added hAtom markup. As a result of this change…well, nothing happened. I had a good experience learning about exactly what is involved in retrofitting an existing site with microformats, but I didn’t get any tangible benefit. With the “SearchMonkey” platform, any site using microformats, or RDFa or eRDF, is exposed to developers who can enhance search results. An enhanced result won’t directly make my my site rank higher in search, it it most certainly make it prone to more clicks, and ultimately more readership, more inlinks, and better organic ranking.

How about some questions and answers:

Q: Is this Tim Berners-Lee‘s vision of the Semantic Web finally getting fulfilled?

A: No.

Q: Does this presuppose everybody rushing to change their sites to include microformats, RDF, etc?

A: No. After all, there is a developer platform. Naturally, developers will have an easier time with sites that use official and community standards for structuring data, but there is no obligation for any site to make changes in order to participate and benefit.

Q: Why would a site want to expose all its precious data in an easily-extractable way?

A: Because within a healthy ecosystem it results in a measurable increase in traffic and customer satisfaction. Data on the public web is already extractable, given enough eyeballs. An openness strategy pays off (of which SearchMonkey is an existence proof).

Q: What about metacrap? We can never trust sites to provide honest metadata.

A: The system does have significant spam deterrents built in, of which I won’t say more. But perhaps more importantly, the plugin nature of the platform uses the power of the community to shape itself. A spammy plugin won’t get installed by users. A site that mixes in fraudulent RDFa metadata with real content will get exposed as fraudulent, and users will abandon ship.

Q: Didn’t ask.com prove that having a better user interface doesn’t help gain search market share?

A: Perhaps. But this isn’t about user interface–it’s about data (which enables a much better interface.)

Q: Won’t (Google|Microsoft|some startup) just immediately clone this idea and take advantage of all the new metadata out there?

A: I’m sure these guys will have some kind of response, and it’s true that a rising tide lifts all boats. But I don’t see anyone else cloning this exactly. The way it’s implemented has a distinctly Yahoo! appeal to it. Nobody has cloned Yahoo! Answers yet, either. In some ways, this is a return to roots, since Yahoo! started off as a human-guided directory. SearchMonkey is similar, except a much broader group of people can now participate. And there are some specific human, technical and financial reasons why as well, but I suggest inviting me out for beers if you want specifics. :-)

Disclaimer: as always, I’m not speaking for my employer. See the standard disclaimer. -m

Update: more Q and A

Q: How is SearchMonkey related to the recently announced Yahoo! Microsearch?

A: In brief, Microsearch is a research project (and a very cool one) with far-reaching goals, while SearchMonkey is targeted as imminently shipping software. I frequently talk to and compare notes with Peter Mika, the lead researcher for Microsearch.

Does GRDDL offer a way out? Could, for instance, the namespace name for Dublin Core metadata be assigned to the prefix “dc:” in an external file, linked via transformation to the document in question? Then it would be simpler, from a producer or consumer viewpoint, to simply use names like “dc:title” with no problems or ambiguity.

This could be especially useful not that discussions are reopening around XML in HTML.

As usual, comments welcome. -m

One marker of a well-designed markup language is that it looks to the future. This doesn’t mean it’s an amorphous blob of abstract indirections mapped to tags. It can (and arguably should) be concrete and solid, but designed in such a way that keeps bigger things in mind.

HTML and XHTML are, I suppose, canonical examples of this, giving birth to microformats and many other uses outside of a browser. -m

Since this was my first visit to Berkeley, I finally got to tell the joke “thank you for your OS”. Maybe you had to be there.

The intentional web is a formalism for describing “why the font tag is evil”. I often work with 3rd party integration languages, and the markup design is, without exception, crap. I hypothesize that the reason for this is jumping into solution-space before fully understanding problem-space. This seems to apply to lots more than just font tags; I lumped in WML and about half the world’s ajax sites for good measure.

Microformats are a formalism for describing “why creating a new markup language for my CD collection” is evil. Could XForms have been done as a microformat? No, microformats require a strong intentional foundation language, and HTML forms ain’t it. Is the proposed W3C approach an instance of “a deadly two-pronged attack”, a la Yahoo! Photos + Flickr? We’ll see. It does seem like that road leads to a namespace apocalypse, highlighting the fundamental difficulty namespaces hoists on attempts to usably extend HTML and XHTML at the same time. A namespace apocalypse may not be a bad thing.

On namespaces, I went over most of the points from my recent article. I won’t rehash that here.

What are some practical and implementation issues around XForms or the lack thereof? Focusing on mobile, as reason #1 I gave the lack of commercial-grade java browsers, discussed here previously. The state of mobile browsers is appalling, other than Opera and S60. Terms like “model” and “field” are troublesome, because the confuse the problem domain (the real world) and the solution domain (the computer). Browser vendors have been too inwardly-focused, both now and during the first attempt at salvaging HTML forms, leading to a premature jump into solution-space. But perhaps XForms dwelled for too long in the problem space…

Maybe I’ve mellowed some, but increasingly I’m able to look at both sides of issues. A useful skill for Information School students, wouldn’t you agree? -m

To elaborate on my previous comments a bit, the concept of what people find usable differs between sitting at a desktop and sitting/standing/running/driving with mobile in hand. Desktop sites aren’t optimized for these kinds of use patterns. Ergo, fertile ground for lots of mashups. You were getting tired of the Maps API + X formula anyway, right? ;-) -m