Some thoughts worth considering on state of HTML development today. -m

MZFinance.NoPasswordToken_message is the apparently Google-unique error message my iPhone gave me today whilst purchasing a free app. Always one happy to leave a new mark on the web, I’m recording it here. If you’ve seen it too (and you come here within the 30-day window) please post a comment on your experience. -m

Hands down, the stupidest Science Friday segment evar. I want my 11 minutes back. -m

Excellent article in Wired, perhaps a good explanation of my career. :-)

Dunbar observed that the skeptical (and sometimes heated) questions asked during a group session frequently triggered breakthroughs, as the scientists were forced to reconsider data they’d previously ignored.

Which sounds like a fairly typical spec review at Mark Logic. Hint: we’re hiring–email me.

-m

I just ordered a beer appreciation kit from tasteyourbeer.com. I’m all for less swilling, more appreciating. This one includes little vials of 13 different kinds of hops to compare. Train your palate, but be warned: once you start down this road, forever will it dominate your soul. You’ll be picking out different flavors in everything you eat or drink, and some things you don’t (like toothpaste). -m

One particular conversation I’ve overheard several times, often in the context of web and standards development, has always intrigued me. It goes something like this:

You know, Ted Nelson’s hypertext system from the 60’s had unbreakable, two-way links. It was elegant. But then came along Tim Berners-Lee and HTML, with its crappy, one-way, breakable links, and it took over the world.

The general moral of the story is usually about avoiding over-thinking problems and striving for simplicity. This has been rolling around in the back of my mind ever since the first time I heard the story. Is it an accurate assessment of reality? And how exactly did Nelson’s system, called Xanadu (R), manage the trick of unbreakable super-links? Even if the web ended up going in a different direction, there still might be lessons to learn for the current generation of people building things that run (and run on) the web.

Nelson’s book Literary Machines describes the system in some detail, but it’s hard to come by in the usual channels like Amazon, or even local bookstores. One place does have it, and for a reasonable price too: Eastgate Systems. [Disclosure: I bought mine from there for full price. I'm not getting anything for writing this post on my blog.] The book has a versioning notation, with 93.1 being the most recent, describing the “1993 design” of the software.

Pause for a moment and think about the history here. 1993 is 16 years ago as I write this, about the same span of time between Vannevar Bush’s groundbreaking 1945 article As We May Think (reprinted in full in Literary Machines) and Nelson’s initial work in 1960 on what would become the Xanadu project. As far as software projects go, this one has some serious history.

So how does it work? The basic concepts, in no particular order, are:

• A heavier-weight publishing process: Other than inaccessible “privashed” (as opposed to “pub”lished) documents, once published, documents are forever, and can’t be deleted except in extraordinary circumstances and with some kind of waiting period.
• All documents have a specific owner, are royalty-bearing, and work through a micropayment system. Anyone can quote, transclude, or modify any amount of anything, with the payments sorting themselves out accordingly.
• Software called a “front end” (today we’d call it a “browser”) works on behalf of the user to navigate the network and render documents.
• Published documents can be updated at will, in which case unchanged pieces can remain unchanged, with inserted and deleted sections in between. Thus, across the history of a document, there are implicit links forward and backward in time through all the various editions and alternatives.
• In general, links can jump to a new location in the docuverse or transclude part of a remote document into another, and many more configurations, including multi-ended links, and are granular to the character level, as well as attached to particular characters.
• Document and network addressing are accomplished through a clever numbering system (somewhat reminiscent of organic versioning, but in a way infinitely extensible on multiple axes). These address, called tumblers, represent a Node+User+Document+Subdocument, and a minor variant to the syntax can express ranges between two points therein.
• The system uses its own protocol called FEBE (Front End Back End) which contains at several verbs including on page 4/61: RETRIEVEV (like HTTP GET), DELETEVSPAN, MAKELINK, FINDNUMOFLINKSTOTHREE, FINDLINKSFROMTOTHREE, and FINDDOCSCONTAINING [Note that "three" in this context is an unusual notation for a link type] Maybe 10 more verbs are defined in total.

A few common themes emerge. One is the grandiose scope: This really is intended as a system to encompass all of literature past, present, and future, and to thereby create a culture of intellect and reshape civilization. “We think that anyone who actually understands the problems will recognize ours approach as the unique solution.” (italics from original, 1993 preface)

Another theme is simple solutions to incredibly difficult problems. So the basic solution to unbreakable links is to never change documents.  Sometimes these solutions work brilliantly, sometimes they fall short, and many times they ends up somewhere in between. In terms of sheer vision, nobody else has come close to inspiring as many people working on the web. Descriptions of what today we’d call a browser would sound familiar, if a bit abstract, even to casual users of Firefox or IE.

Nothing like REST seems to have occurred to Nelson or his associates. It’s unclear how widely deployed Xanadu prototypes ever were, or how many nodes were ever online at any point. The set of verbs in the FEBE protocol reads like that a competent engineer would come up with. The benefits of REST, in particular of minimizing verbs and maximizing nouns, are non-obvious without a significant amount of web-scale experience.

Likewise Creative Commons seems like something the designers never contemplated.  “Ancient documents, no longer having a current owner, are considered to be owned by the system–or preferably by some high-minded literary body that oversees their royalties.” (page 2/29) While this sounds eerily like the Google Books settlement, this misses the implications of truly free-as-in-beer content, but equally misses the power of free-as-in-freedom documents. In terms of social impact there’s a huge difference between something that costs $0 and $0.000001.

In this system anyone can include any amount of any published document into their own without special permission. In a world where people writing Harry Potter Lexicons are getting sued by the copyright industry, it’s hard to imagine this coming to pass without kicking and screaming, but it is a nice world to think about. Anyway, in Xanadu per-byte royalties work themselves out according to the proportion of original vs. transcluded bytes.

Where is Google in this picture? “Two system directories, maintained by the system itself, are anticipated: author and title, no more” (page 2/49) For additional directories or search engines, it’s not clear how that would work: is a search results page a published or privashed document? Does every possible older version of every result page stick around in the system? (If not, links to/from might break) It’s part of a bigger question about how to represent and handle dynamic documents in the system.

On privacy: “The network will not, may not monitor what is written in private documents.” (page 2/59) A whole section in chapter 3 deals with these kinds of issues, as does Computer Lib, another of Nelson’s works.

He was early to recognize the framing problem: how in a tangle of interlinked documents, to make sense of what’s there, to discern between useful and extraneous chunks. Nelson admits to no general solution, but points at some promising directions, one of which is link typing–the more information there is on individual links, the more handles there are to make sense of the tangle. Some tentative link types include title, author, supersession, correction, comment, counterpart, translation, heading, paragraph, quote, footnote, jump-link, modal jump-link, suggested threading, expansion, citation, alternative version, comment, certification, and mail.

At several points, Nelson mentions algorithmic work that makes the system possible. Page 1/36 states “Our enfilade data structures and methods effectively refute Donald Knuth’s list of desirable features that he says you can’t have all at once (in his book Fundamental Algorithms: Sorting and Searching)”. I’m curious if anyone knows more about this, or if Knuth ever got to know enough details to verify that claim, or revise his.

So was the opening anecdote a valid description of reality? I have to say no, it’s not that simple. Nelson rightly calls the web a shallow imitation of his grand ideas, but those ideas are–in some ways literally–from a different world. It’s not a question of “if only things had unfolded a bit differently…”. To put it even more strongly, a system with that kind of scope cannot be designed all at once, in order to be embraced by the real world it has to be developed with a feedback loop to the real world. This in no way diminishes the value and influence of big ideas or the place that Roarkian stick-to-your-gunnedness has in our world, industry, and society. We may have gotten ourselves into a mess with the architecture of the present web, but even so, Nelson’s vision will keep us aspiring toward something better.

I intend to return to this posting and update it for accuracy as my understanding improves. Some additional topics to maybe address are: a more detailed linking example (page 2/45), comparing XLink to Xanadu, comparing URIs and tumblers, and mention the bizarre (and yet oddly familiar if you’ve ever been inside a FedEx Kinkos) notion of “SilverStands”.

For more on Nelson, there is the epic writeup in Wired. YouTube has some good stuff too.

Comments are welcome. -m

Xanadu is a registered trademark, here used for specific identifying purpose.

MarkLogic fans should check out Norm Walsh’s posting about his talk at the NY User Group. If you follow the right Twitter feeds, this is probably not too much of a surprise, but now the cat is officially disjoint with the volume inside the bag. Disclaimer: be sure to read the disclaimer there. -m

There are no limits, only quanta. There is no smooth, only lumpy.

Analog is a myth; the world is digital. We just haven’t found the extent of its mantissa. – SymbolismLost.

More collected Geek Thoughts at http://geekthoughts.info.

The central thesis of The Inmates are Running the Asylum by Alan Cooper is dead on: engineers get too wrapped up in their own worlds, and left entirely to their own whims can easily make a product incomprehensible to ordinary folks. For this reason alone, it’s worth reading.

But I do question parts of his thesis. He (with tongue in cheek) posits the existence of another species of human, called Homo Logicus. Stepping on to an airplane, Homo Logicus turns left into the cockpit with a million buttons but ultimate control over every aspect of the plane. Regular Homo Sapiens, on the other hand, turn right and tuck themselves into a chair–no control but at least they can relax.

But if there was only one “species” of Homo Logicus, members (like me) would never experience usability issues in software created by fellow Logicians. But ordinary fax machines give me fits. The touch-screen copier at work instills dread in my heart. And the software I need to use to file expense reports–written by enterprise software geeks probably very similar to me–is a usability nightmare. Words fail me in expressing my disdain for this steaming heap of fail.

The book is sub-titled “Why High-Tech Products Drive Us Crazy”, but one doesn’t have to look very far to find similar usability bugs in the low-tech world. Seth Godin, for example, likes to talk about different things in life that Just Don’t Work, along with reasons why. Some examples:

• airport cab stand (75 cabs, 75 people, and it takes an hour)
• “don’t operate heavy machinery” warning on dog’s prescription medicine
• excessive fine print on liability agreements–intentionally hard to read and figure out
• official “Vote for Pedro” shirts that look nothing like the ones in the movie
• more examples on the web site

If anything, I think Cooper’s work doesn’t go far enough. It is relatively short on good examples, stretching out only four examples over four chapters. If properly-designed software is so hard to come up with examples of, then there are bigger problems in play (that would need to be dealt with by something more manifesto than book).

The book now 5 years old. Perhaps it’s time for an update. Particularly in the world of web software, lots has happend in 5 years. Flickr. Gmail. Yahoo Pipes. Google Docs. Even SearchMonkey. Instead of focusing on pointing at crappy software, I’d like to see more emphasis on properly-done interfaces. More delving into nuance, and common factors behind why both high-tech and low-tech products miss the mark.

But maybe that’s just me. -m

This article states:

The analysts determined YouTube’s bandwidth costs by assuming that 375 million unique visitors would visit the site in 2009, with 20 percent of those users consuming 400 kilobits per second of video at any given time. That works out to 30 million megabits being served up per second. That’s a heck of a lot of bandwidth to devote to videos of sneezing pandas.

Do you honestly believe that YouTube is sending out 30 petabits per second (to put it another way, fully saturating over 200,000 OC3 connections)? That on average, every single user who counts as a unique visitor in 2009 spends 20% of 24hrs = 4.8 hours actually downloading video, every day of every week?

Gesundheit. -m

Update: the quoted article indeed gets it wrong, though it appears the original Credit Suisse analyst report was estimating peak usage, not a running average. Still doesn’t smell right. Updating the article and title to point the finger at the right people.

To celebrate the unlamented demise of Valleywag, use as many of the ten insulting words you should know (along with any other appropriate words) as you can in a single short paragraph. Post in the comments below. This site is for geeks of all ages, so keep things PG. All right, PG-13. My favorite will be announced later.

More collected Geek Thoughts at http://geekthoughts.info.

I’ve successfully completed the National Novel Writing Month challenge–to write 50,000 words during the month of November–for three years running, and now I have three draft novels sitting around. At some point, racking up mere drafts gets to be pathetic, so this November I’m picking one to dig into with a heavy editing pass. I’m stocked up on red pens and ready to go, and maybe a third of the way through at this moment. Other writing efforts, like say this blog, get back-burnered for a few weeks.

The key to editing long works, by the way, is momentum. If you get stuck in one place, you quickly get burned out on it, like melting celluloid in an old-skool movie projector. Onward. -m

He won an olympic silver medal in the much-publicized race where the gold winner came in 0.01 second faster. And he blogs. He writes:

On winning a SILVER medal: I am completely happy, and still in complete disbelief that I was able to achieve this feat! I’m not joking… It’s a tough loss, but I’m on cloud nine. I congratulated Phelps and his coach Bob Bowman. I’m just glad the race was fun to watch for everyone. It was a pleasure for me, really.

What a fantastic attitude. I wish I had my head on that straight soemtimes. Milorad, if you are ever in the South Bay, I will buy you a beer, or a 1000-calorie energy drink, or whatever. -m

Actually, instead of a review, let me quote the opening testimonial from the inside-front cover.

Competing globally with dynamic capabilities is the top priority of multinational executives and managers everywhere. Rethinking strategy in a highly networked world is the big challenge. How can your company navigate successfully in this turbulent, highly networked and socially connected environment? …

If this does it for you, I couldn’t recommend this book more highly. -m

Someone overseas, whom I consider trustworthy but haven’t actually met, wants to buy one of my domains. Great!

There’s not huge sums involved, though it’s a hassle to move the money and coordinate the transfer. Readers: Any advice on how to complete the transaction? Escrow service? Anything lighter-weight? Comment below. -m

That’s my game idea. Unfortunately I won’t have time to develop the idea, so somebody else go for it–just mention my name in the credits ;)

My 7-year-old has just discovered Oregon Trail, or more accurately Westward Trail, a respectable online clone.

-m

Commentators, having long since run out of useful things to say about YHOO+MSFT, only bemoan how it continues to drag out. In reality, deals of this size do tend to take a while. Microsoft (and specifically Ballmer) aren’t walking. Why?

Because they need Yahoo. They need search share–the deal with Google only puts on more pressure. But they also need a non-schizophrenic brand under which to put all their audience attractors. In short, I’d say MSFT has been terrible at tactics (and non-intimidation-based negotiating), and YHOO has been mediocre at strategy and terrible at execution. Maybe they are meant for each other…

Prediction: by the end of the year 1) some kind of deal happens, and 2) Yang is out as CEO. $28.

Disclosure: I still hold long YHOO shares

Disclosure: The irony of this post is not lost on me

-m

I was on the right page at the right time and spotted this:

WIth zero votes for any candidate and zero precincts reporting, CNN calls it. Now, I’m no stats expert, but this one seems a bit extreme. Comments? -m

Reminder: Thursday evening at Yahoo! Sunnyvale headquarters is the launch party for the developer-facing side of SearchMonkey. In case you haven’t been paying attention, SearchMonkey is a new platform that lets developers craft their own awesomized search results. If you’re interested in SEO or general lowercase semantic web tools, you’ll love it. Meet me there. Upcoming link. Party starts at 5:30. -m

Update: The developer tool is live. Rasmus has a nice walkthrough.

Today happens to mark the 6th anniversary of my blog. To celebrate going into year seven I’m refocusing it, including a new name: Micahpedia.

Blogging is an important skill, a subset of the overall skill of managing your online persona, so it’s worth devoting some attention to. The ego-burst doesn’t hurt either. My concrete goal is to get in the top 10 search results for the query [Micah], though I face some stiff competition including the prophet.

From an SEO perspective, “Push Button Paradise” wasn’t the greatest choice of name. It suffers from the common SEO mistake of being excessively clever and/or cute reflection of what I happened to be working on at the moment, namely XForms. If you see the old name standalone, or in a blogroll, or in an RSS reader, you still don’t have much of an idea what it’s about or who’s behind it. True I get pretty good ranking on the exact phrase, but nobody searches for that…

I will continue SEO tweaks on this site as time goes on and welcome any advice from any of my 7 readers.

In short, Micahpedia is about what I’m reading, writing, thinking about, and working on. I have plenty to say about these things. :-) The best is yet to come. -m

“Rails is a lot of fun, and lets me do cool new things – but it’s hard to eat it.”

Simon St. Laurent

-m

I’m not involved in the the corporate wrangling about Microsoft and Yahoo! talks. Which leaves me relatively free to comment on it. [Disclosure: I am, not too surprisingly, a Yahoo! shareholder.]

Lots of things have been happening lately. A deadline of, well, today. Talks of Google adsense trials. And all kinds of merger speculation involving Rupert Murdoch in some fashion, or else AOL.

But I haven’t seen anyone point out this connection: Google owns 5% of AOL, having invested a billion bucks and taken over search there a couple of years ago. So if Yahoo! and AOL merged, there would already be a Google advertising connection in place. Running pre-trials now is just due dilligence on something that might happen anyway.

Having both an in-house advertising network and an outsourced one has some advantages too, namely in the form of “knobs” that can be adjusted to tune margins as conditions warrant. And maintaining the in-house system keeps Google honest and makes sure that relatively good deals can be negotiated in the future.

Lots of pundits talk about regulatory scrutiny, but honestly, it’s been years since any antitrust machinery in this country has been effective. And the recent spectrum auctions showcased Google’s skill at turning regulatory tables in their favor. If it came down to it, the smart people on both sides of the table shouldn’t have a problem crafting an agreement in a way that meets muster, even in the stricter EU.

Summary: based solely on public reports, it seems like the AOL connection might be a credible threat to Microsoft’s appetite. The ball is firmly in Steve’s court now. We’ll see what he does.

Harry Potter author J.K. Rowling, herself rowling in gazillions of dollars, is along with her publisher suing Steven Vander Ark, a poor librarian who produced a lexicon of the Harry Potter universe.

Rowling says it’s not about the money, it’s about control. Poppycock. If that was the case, she would have objected to the web site. Instead, Rowling is quoted as saying:

This is such a great site that I have been known to sneak into an Internet cafe while out writing and check a fact rather than go into a bookshop and buy a copy of Harry Potter (which is embarrassing).

A lexicon is a collection of existing (fictional) facts, not something that is going to wrest creative control of the franchise away from the author. This work makes the Harry Potter universe more valuable, not less. Even if legally this is a gray area, it’s a boneheaded move to sue one of your greatest fans for providing a valuable and useful reference.

What troubles the bean counters so much is that the printed lexicon costs, well, actual money, $24.95 to be exact. As an author it troubles me to see how out of touch copyright law is, and how badly the scent of a few dollars can make an otherwise reasonable person behave. -m

As spotted on TechCrunch, full article. This is a game-changer folks. Check out the comments attached to the article. -m

I discovered this by accident, but my life has been measurably better since.

You probably already know that you can switch apps quickly with Cmd+tab. But if you reach your pinky up a bit more and hit Cmd+~ you can rotate through the windows of the current app. This turns out to be most useful when, say, your email compose window gets behind the main email client window.

What is the equivalent keystroke on Linux? -m

Here’s the best news I’ve had all day: the creators of MST3K are reuniting under a new effort, called Cinematic Titanic, the firstfruits of which are due out this coming Monday.

I’ve been a long time fan of MST3K, watched most of the early episodes on UHF in Minnesota. And for the record, I like Joel better than Mike. :) -m

If you want to get anything done, give it to a busy person…

In my life, I’ve started four novels, completed my goals on three, gotten to “The End” on two, and completely flamed out on one.

The first was in 2001. I hadn’t written much since high school. Something clicked in my head that made me realize that writing wasn’t some kind of black art (as one particular teacher had drilled into his credulous students). It was doable. You take pencil and paper and write one word after another. Voilà. I was so taken with this simple idea that every single thing I ever learned about writing went out the window. I had Swifties, danglers, tell-vs-show, you name it. There’s enough material in there for several Bulwer-Lytton contests. By the time I had 70 hand-written pages, the thing collapsed under it’s own weight and the story reached an abrupt, borderline-surrealistic “ending” to abuse the term. I have evidence that I even typed it all in and pressed on for a 2nd draft.

By 2003 my non-fiction book was published–my writing career was under way! Part of the elaborate book proposal dance involved me writing some online articles, including one piece of fiction that was well-received in the tiny circle that was its intended audience. At this stage I adopted electronic writing, and ditched my crashy Windows laptop for a Mac, a vast improvement.

In 2005 I discovered NaNoWriMo, and though I thought it would be a lost cause, I signed up. No way it could be as bad as the previous attempt. I had a new job, and was able to skip a few lunches to write, not to mention intense evenings and weekends. The end goal is 50,000 words during the 30 days of November, that’s 1,666 and two-thirds words per day. All of the prior month I spent outlining, making maps, creating my universe. I used the simplest of tools, my text editor and one file per chapter. I learned that the command wc *.txt could easily give me a combined word count. To my surprise, it worked. I reemerged into daylight with a completed a full story arc loosely based on the earlier story, and ended up with just over 50,000 words. The text itself was very rough, but I read the whole thing out loud in a podcast to edit it. In terms of improvement, it was huge, but still far from publishable.

2006 and another NaNoWriMo rolled around, and I took off on a more ambitious storyline with far fewer notes going into it. The story itself involved the same general characters of the previous two episodes, but with a deeper, more mature feeling to it. In short, I finally wrote a piece of fiction to be proud about afterwards, though when I hit 50,000 words I felt really burned out; hit “save” and left the story arc unfinished.

The pull to dig in to an intensive 2nd draft of the story was immense, but just too many things were going on, including a new arrival in the family and a new set of job responsibilities. I never got more than a few dozen pages into the rewrite. When NaNoWriMo 2007 came upon me, I had a tough choice…do I write something fresh, or try to rework the previous novel? Fresh. A completely new story line, new characters, new setting, new everything. As of a few days ago, I finished the draft, compressing parts of the story as needed to meet both the 50 kiloword goal and the complete story arc. In preparation, I read a number of books, but as far as written outlines, maps, etc. go, almost nothing happened before November 1. I saved enough of the “fun stuff” that a second revision of this story will be a joy. Overall, another improvement year-over-year.

There’s only one kink to the “if you want to get something done…” idea: my slides for the XML Conference talk I have in a few days are still unfinished… -m

Amazon announced their ebook reader today, Kindle. Some of the earlier hype I’d read about it suggested that it would be not only a reading tool, but a writing tool as well.

Nope.

The obvious thing is the keyboard, an immediate non-starter for typing more than a few words. But if an external keyboard is possible, I could still live with it. For now, it seems like an OLPC would better serve my needs for an ultra-portable writing station. And for the same price, I get one and so does somebody else who need it. -m

Game On. Expect light posting this month. -m

I see from the system requirements that Leopard requires an “866″ Mhz processor. Is this a hard limit, or just an advisory? My first Mac–the one I wrote the book on–is a lowly 800 Mhz. box. Is it worth trying to upgrade it? -m