Archive for the 'stuff' Category

Sunday, June 30th, 2019

Xanadu in 2019

If you haven’t been following Ted Nelson on YouTube, you’re missing out. Recently, he’s been posting a series of informational videos on the Xanadu architecture and concepts. Even more recently, he hosted a live Q&A session, taking questions from Twitter.

Ted Nelson’s Channel

What’s Xanadu, you ask? It’s the original concept for a hypertext system, designed well before the web was a gleam in Tim BL’s eye. Nelson himself coined the term ‘hypertext’. It can be hard to wrap your head around a hypertext system not based on markup or HTML. For a brief technical overview, check out this article, now nearly a decade old.

One of the biggest points of contention between the “current web” and Xanadu is the de-facto security restrictions (such that they are) on browsers with the Same Origin policy. With several loopholes, once your browser loads a page from, other content, excluding images and other things, have to be loaded from the same site. [Yes, it’s possible, via preflights, to get around this, but the kind of per-site configuration needed flies against the notion of a global “docuverse”.] In contrast, the central construct of Xanadu is that documents consist of a central list of chunks of content and links to load.

Because we’ve lived in a web-wide-world for so long, it’s hard to think clearly about this alternate reality. What role would something like JavaScript play in a Xanadu-docuverse? I haven’t been able to find any detailed studies about how security design on a web-scale Xanadu deployment. What would be the nearest equivalents to cookies or webapps? To web forms (or dare I say, XForms)? To untrusted content surreptitiously leaking personal information? To CSRF attacks? To the OWASP security checklist?

It seems like Xanadu wants to consist of purely documents, and is less friendly toward other media types, and outright hostile to what we today call webapp technology. This makes for a clean and elegant design, but I can’t help wonder whether it’s also a factor in the takeover of one-ended “jumplinks” as seen in HTML, and the mountain of web architecture that came along with it.

My gut says that not all hope is lost. Large chunks of web technology, including HTTP, could be hugely useful in a global docuverse. One way or another, the browser ship has sailed, so it would have to work with modern web browsers. So what can we do about the Same-origin problem? How can we design a docuverse on top of today’s web? -m

P.S. I still have the same question from the earlier article:

“Our enfilade data structures and methods effectively refute Donald Knuth’s list of desirable features that he says you can’t have all at once (in his book Fundamental Algorithms: Sorting and Searching)”. I’m curious if anyone knows more about this, or if Knuth ever got to know enough details to verify that claim, or revise his.

Maybe Ted will comment about this on YouTube.

P.S.2. Xanadu is a registered trademark, here used for specific identifying purpose.

Saturday, January 2nd, 2016

Pixel editor

I did a thing.

I am experimenting with machine learning and neural networks. To do so, I need a real-world dataset to play with. For starters, I am using a 5×7 pixel array, as common in many DIY projects, representing digits 0-9. Please help me my drawing a picture of the randomly-selected digit below. All data will be released to the public domain.

Can you help me? Take a few seconds and draw a picture of a number by clicking on pixels to toggle their value.

Email me with any questions or suggestions.


Data collection by Google Custom Forms.

Machine learning datasets FTW! Stay tuned for code & results.

Thursday, February 5th, 2015

DIY fizzy yerba mate drink

Filed under need-to-try this: A homebrew version of a popular hacker drink called Club Mate.

I already have a carbonator cap and CO2 setup, as part of my beer brewing hardware.

One variation I would experiment with is cutting down on the sugar. Even though Club Mate isn’t very sweet, it still has a fair amount of sugar in it. Stevia could be a good alternative. -m

Friday, September 28th, 2012

Virgil Matheson: mentor

I’ve mentioned Virgil Matheson in these pages a few times, but never made a full accounting. When I had my O’Reilly book published, I submitted a simple dedication in the manuscript:

for Virgil

But for whatever reason, it didn’t make it into the printed edition. This post is a small step toward letting the world know about someone important to me.

We first met in 1985 or thereabouts. One day while riding my bike through a back-alley, I stopped to look at an equipment rack set outside a spare garage. Virgil came out to give a get-off-my-lawn kind of speech, and somehow we ended up talking about electronics.  This led to discussions about crystal radios, and in a subsequent visit, we built one, he explaining the principles of operation. Virgil, it turns out, was a retired teacher at the North Dakota State School of Science, where he taught AC theory and thermodynamics. I was going through some rough times, and Virgil ended up being a much-needed role model.

Around that time, I had ttempted to build a Heathkit radio set, but couldn’t quite get it working. I brought it to Virgil, and we traced through the schematic diagrams, eventually getting it working. Along the way, Virgil introduced me to all kinds of electronic test equipment, including oscilloscopes and galvanometers that he had hand-wound in his younger days.

The next year, I needed a science project, and I had become fixated on Tesla Coils. Virgil had worked at Westinghouse (but not in overlap with the good N. Tesla) and found this project right up his alley. We used his wood lathe to turn a base for the coil, and a standard lathe to wind a primary and two perfectly-spaced secondary coils on PVC pipe, after which we sprayed them down with insulating paint. We built a high-voltage power supply out of a car battery, ignition coil, and relay-type regulator from the junkyard. The thing would turn out serious spark on the primary side, and at one point, I accidentally made contact with it, knocking me clear off the metal bench I was sitting on. We used a spark gap and high-voltage capacitors from old equipment to make a resonator, and got the coil working. It could light a fluorescent tube from my full arm-span away. It was a smash hit at the science fair, too.

For one so knowledgable about the foundations of technology, he was awfully curmudgeonly about it. He bemoaned the day students started showing up in his class with hand-calculators instead of slide rules. He would never answer the phone (but would speak on it, if you could get his brother to pick up).

We kept meeting on and off, and we would have epic discussions/debates about technology, thermodynamics, perpetual motion machines, higher mathematics, theology, building test equipment, and logic puzzles. He taught me, in short, how to think.

A non-exhaustive list of things he taught me:

  • How to build a crystal radio set
  • How to troubleshoot a conventional radio (hint–check for signal at the volume control–that will narrow down the problem to either the front-end or back-end)
  • How to compute resonant LC circuits
  • How to use a slide rule
  • How to pick locks
  • How to compute power factor and plot phasor diagrams for AC circuits
  • The value of good tools and how to care for them
  • How to build a Tesla Coil
  • How to debate
  • Respect for high voltage
  • The joy of back-issues of Scientific American
  • The trouble with Pascal’s wager
  • How to debunk perpetual motion claims
  • How (and why) to use a planimeter

On a recent vacation, I went to see Virgil again–now in his 90s. He’s still vigorous and feisty, though his memory is starting to slip a little. It was difficult to come to terms with the possibility that, given the frequency with which I make it to that part of the country, it may be the last time I see him. Since this is posted online, he’ll probably never see it. But if he could speak to each one of you, I think he’d offer advice something like this:

Cherish the people in your life. Treat every meeting as if it might be the one that sets you on a new course–one that you’ll look back at years later in wonder. Don’t worry what others think of you, and never stop learning.

Thank you, Virgil, for all you’ve given me. -m

Monday, June 4th, 2012

Relax NG vs XML Schema: ten year anniversary

Today is the 10-year anniversary of this epic message from James Clark on the relative merits of Relax NG vs. XML Schema, and whether the latter should receive preferential treatment. Still relevant today–the discussion is still going, although an increasing number of human-readable web specifications have adopted RelaxNG in some form. -m

Sunday, January 15th, 2012

The ultimate breakfast smoothie

I’ve used this same recipe for three things: weight loss, after-exercise protein, and sore-teeth liquid diet. It’s great.

1 cup 2% milk

1 cup Dannon Fit & Light vanilla yogurt

1 scoop Syntha-6 protein powder (banana is great)


This yields 450 calories with a whopping 39g of protein, 48g of carb (but only 30g of that simple sugars), 11g of fat, and 5g of fiber.

You could live off 3 or 4 of these a day. (and I have)

Wednesday, January 5th, 2011

Why I am abandoning Yahoo! Mail (and why you should too)

This is a non-technical description of why Yahoo! Mail is unsafe to use in a public setting, and indeed at all. I will be pointing people at this page as I go through the long process of changing an address I’ve had for more than a decade.

What’s wrong with Yahoo Mail?

A lot of web addresses start with http://–that’s a signal that the “scheme” used to deliver the page to your browser is something called HTTP, which is a technical specification that turns out is a really good way to move around web pages. As the page flows to the browser, it’s susceptible to eavesdropping, particularly over a wi-fi connection, and much more so in public, including the usual hotspots like coffee shops, but also workplaces and many home environments. It’s the virtual equivalent of a postcard. When you’re reading the news or checking traffic, it’s not a big deal if someone can sneak a glance at your page.

Some addresses start with https://–notice the extra ‘s’ which stands for “secure”. This means two things 1) that the web page being sent over is encrypted, and thus unavailable to eavesdroppers, and 2) that the people running the site had to obtain a certificate, which is a form of proof of their identity as an organization (that they’re not, say, Ukrainian phishers). Many years ago, serving pages over https was considered quite expensive in that servers needed much beefier processors to run all that encryption. Today, while it still requires extra computation, it’s not as big of a deal. Most off-the-shelf servers have plenty of extra power. To be fair, for a truly ginormous application with millions of users like Yahoo Mail, it is not a trivial thing to roll out. But it’s critically important.

First, to dispel a point of confusion, these days nearly every site, including Yahoo Mail, uses https for the login screen. This is the most critical time when encryption is needed, because otherwise you’d be sending your password on a postcard for anyone with even modest technical skills to peek at. So that’s good, but it’s no longer enough. Because sites are written so that you don’t have to reenter your password on every single new page, they use a tiny bit of information called a “cookie” in your browser to stay logged in. Cookies themselves are neither good nor bad, but if an eavesdropper gets a hold of one, they can control most of your account–everything that doesn’t require re-entering a password. In Yahoo Mail this includes reading any of your messages, sending mail on your behalf, or even deleting messages. Are you comfortable allowing strangers to do this?

As I mentioned earlier, new, more powerful tools have been out for months that automate the process of taking over accounts this way. Zero technical prowess is needed, only the ability to install a browser plug-in. If there are any web companies dealing in personal information for which this wasn’t a all-hands-on-deck security wake-up, they are grossly negligent. Indeed, other sites like Gmail work with https all-the-time. But still, in 2011, Yahoo Mail doesn’t. I have a soft spot for Yahoo as a former employer, and I want to keep liking them. Too bad they make it so difficult.

The deeper issue at stake is that if this serious of an issue goes unfixed for months, how many lesser issues lurk in the site and have been around for months or years? The issue is trust, my friend, and Yahoo just overdrew their account. I’m leaving.


Q: So what do you want Yahoo to do about this?  A: Well, they should fix their site for their millions of remaining users.

Q: What if they fix it tomorrow? Will you delete this message?  A: No. Since I no longer trust the site, I am leaving, even though it takes time to notify all the people who still send me mail, and no matter what other developments unfold in the meantime. This page will explain my actions.

Q: Do you really want everyone else to leave Yahoo Mail?  A: No, only those who care about their privacy.

Q: What’s your new email address?  A: I have a couple, but <my first name> @ <this domain> is a good general-purpose one.

I will continue to update this page as more information becomes available. -m

Thursday, September 2nd, 2010

Is XForms really MVC?

This epic posting on MVC helped me better understand the pattern, and all the variants that have flowed outward from the original design. One interesting observation is that the earlier designs used Views primarily as output-only, and Controllers primarily as input-only, and as a consequence the Controller was the one true path for getting data into the Model.

But with browser forms, input and output are tightly intermingled. The View takes care of input and output. Something else has primary responsibility for mediating the data flow to and from the model–and that something has been called a Presenter. This yields the MVP pattern.

The terminology gets confusing quickly, but roughly

XForms Instance == MVP Model

XForms Model == MVP Presenter

XForms User Interface == MVP View

It’s not wrong to associate XForms with MVC–the term has become so blurry that it’s easy to lump variants like MVP into the same bucket. But to the extent that it makes sense to talk about more specific patterns, maybe we should be calling the XForms design pattern MVP instead of MVC. Comments? Criticism? Fire away below. -m

Monday, July 12th, 2010

Hard drive failing

My personal machine is ailing. It freezes up for 30 seconds at a time–even iTunes stops playing. FireFox crashes before it’s done launching. I’m scared to reboot for fear the machine won’t come back up.  SMARTReporter lists an operating age of 15k hours, a suspicious Power-Off_Retract_Count of over 25 billion–whatever this represents, it’s happened an impressive average of several hundred times per second for the life of the drive. Also a Load_Cycle_Count of over 866k (where typical lifespan is 300-600k). New HDD ordered.

Changing to a new hard drive is the digital equivalent of moving. Disruptive to your routine, and you’re living out of boxes for months afterward. Not much fun, but also an opportunity to reorganize a few things, to start fresh. -m

Saturday, June 19th, 2010

The primary virtue of the meadmaker is patience

I have a batch of chocolate mead that’s been brewing since 2007. Mead bulk ages well, but this is a new personal record. Today, I started siphoning it into the bottling bucket when I noticed that it wasn’t completely clear. I use a mineral called sparkalloid which causes any haze/protein/particulate to settle to the bottom, and there it was–a boundary layer between clear and sparkling.

So back into the carboy it goes, with a fresh dose of fining agents. What’s a few more months at this point? -m

Saturday, June 12th, 2010

Command Lines on the frontier of user interface

This came from a comment on the prior post, and it’s worth a shout of its own. Don Norman on the importance of command lines, including the ubiquitous search box, in modern UI. -m

Wednesday, June 9th, 2010

“Google syntax” for semantic queries?

Thought experiment: are there any commonly-expressed semantic queries–the kind of queries you’d run over a triple store, or perhaps a SearchMonkey-annotated web site–expressible in common type-in-a-searchbox query grammar?

As a refresher, here’s some things that Google and other search engines can handle. The square brackets represent the search box into which the queries are typed, not part of the queries themselves.


[term -butnotthis]

[term1 OR term2]

[“phrase term”]

[tem1 OR term2 -“but not this” filetype:html]

So what kind of semantic queries would be usefully expressed in a similar way, avoiding SPARQL and the like? For example, maybe [by:”Micah Dubinko”] could map to a document containing a triple like <this document> <dc:author> “Micah Dubinko”. What other kinds of graph queries are interesting, common, and simple to express like this? Comments welcome.


Friday, June 4th, 2010

The Swinger

I’m enjoying the results of this Python project from Music Hack Day way too much. It analyzes an audio clip to detect the beats, then uses time stretching and compression techniques (that don’t alter the pitch) to rearrange each measure into a “swung” groove. Fantastic. I wish they’d take more requests! -m

Try this one on for size: Just What I Needed by The Cars:

Just What I Needed (swing version) by davermont

Thursday, June 3rd, 2010

Reverse Engineering Corexit 9500

If you dig a bit, there’s all kinds of interesting background material about the terrible disaster ongoing in the Gulf of Mexico. For example, a map of the thousands of rigs and tens-of-thousands of miles of pipelines. Some of the best infographics are from BP itself. And for when you can no longer stand the overwhelming sense of disaster, a fake twitter feed.

But this really caught my eye, from Nalco, the manufacturer of the oil dispersant Corexit 9500 which is being used both in unprecedented quantities and depths in the Gulf. Here’s how they cleverly describe the ingredients of their product, an ingredient list they protect as a trade-secret:

  1. One ingredient is used as a wetting agent in dry gelatin, beverage mixtures, and fruit juice drinks.
  2. A second ingredient is used in a brand-name dry skin cream and also in a body shampoo.
  3. A third ingredient is found in a popular brand of baby bath liquid.
  4. A fourth ingredient is found extensively in cosmetics and is also used as a surface-active agent and emulsifier for agents used in food contact.
  5. A fifth ingredient is used by a major supplier of brand name household cleaning products for “soap scum” removal.
  6. A sixth ingredient is used in hand creams and lotions, odorless paints and stain blockers.

That is one impressive bit of verbal agility, my complements to their staff writer(s). It would be a fun exercise some day to see what kinds of toxic sludge could be described in similar terms. But let’s see if we can figure out the exact ingredient list: here’s the MSDS for the substance. According to it Propylene Glycol is clearly one of the ingredients, as are “Distillates, petroleum, hydrotreated light” and “Organic sulfonic acid salt”. “Wetting agent” and “surface-acting” are both code words for a surfactant. A little knowledge of chemistry along with household product label reading might go a long way… Got insight? Add a comment here to describe what you find.


6/10 Update: Nalco released the full ingredient list and cheat sheet:

CAS # Name Common Day-to-Day Use Examples
1338-43-8 Sorbitan, mono-(9Z)-9-octadecenoate Skin cream, body shampoo, emulsifier in juice
9005-65-6 Sorbitan, mono-(9Z)-9-octadecenoate, poly(oxy-1,2-ethanediyl) derivs. Baby bath, mouth wash, face lotion, emulsifier in food
9005-70-3 Sorbitan, tri-(9Z)-9-octadecenoate, poly(oxy-1,2-ethanediyl) derivs Body/Face lotion, tanning lotions
577-11-7 * Butanedioic acid, 2-sulfo-, 1,4-bis(2-ethylhexyl) ester, sodium salt (1:1) Wetting agent in cosmetic products, gelatin, beverages
29911-28-2 Propanol, 1-(2-butoxy-1-methylethoxy) Household cleaning products
64742-47-8 Distillates (petroleum), hydrotreated light Air freshener, cleaner
111-76-2 ** Ethanol, 2-butoxy Cleaners

The * footnote indicates, essentially, “contains propylene glycol”.

The ** footnote indicates that this chemical is found only in Corexit 9527, not the one most commonly used in the Deepwater Horizon cleanup.

Sunday, May 23rd, 2010

Martin Gardner will be missed

I first ran in to Martin‘s work in back-issues of Scientific American. He stopped writing his Mathematical Games column in 1981, but my mentor Virgil Matheson had all the older issues and had a free hand in lending them out, albeit one-at-a-time. From my mentor, I also got the best math book I’ve ever read, Calculus Made Easy by Silvanus P. Thompson. A newer edition of the book came out in 1998–and who came on board as the 2nd author to revise and modernize the text? Yep, that’s who. -m

Sunday, April 25th, 2010

Bananas Foster in a glass

The 60th anniversary of the creation of Bananas Foster is around the corner, and the project I started this weekend should be ready just in time. I’m keeping the recipe under wraps for now, but it involves ripe bananas, a particularly buttery variety of honey, brown sugar, homemade caramel, vanilla, and cinnamon. This should turn out to be a 3 gallon batch when all is said and done. It smells amazing in the primary. -m

Monday, April 19th, 2010

The Rick Wakeman clause?

Phrase seen in this article about whether video games are art, and Roger Ebert’s opinions thereon.

“Video games by their nature require player choices, which is the opposite of the strategy of serious film and literature…”

Hmm, Mr. Ebert doesn’t seem to be up on the concept of hypertext, which has manifold connections with cinema. See for instance the scholarly paper Cinematic Paradigms for Hypertext. In fact, making a hypertext or branching narrative requires even greater amounts of authorial skill.

But I’m still curious, what is the Rick Wakeman clause? From where did that term originate? -m

Tuesday, February 2nd, 2010

Larry Masinter on overspecification

Some thoughts worth considering on state of HTML development today. -m

Wednesday, January 6th, 2010


MZFinance.NoPasswordToken_message is the apparently Google-unique error message my iPhone gave me today whilst purchasing a free app. Always one happy to leave a new mark on the web, I’m recording it here. If you’ve seen it too (and you come here within the 30-day window) please post a comment on your experience. -m

Friday, December 25th, 2009

The Physics of Santa

Hands down, the stupidest Science Friday segment evar. I want my 11 minutes back. -m

Monday, December 21st, 2009

Failure as the secret to success

Excellent article in Wired, perhaps a good explanation of my career. :-)

Dunbar observed that the skeptical (and sometimes heated) questions asked during a group session frequently triggered breakthroughs, as the scientists were forced to reconsider data they’d previously ignored.

Which sounds like a fairly typical spec review at Mark Logic. Hint: we’re hiring–email me.


Friday, December 18th, 2009

Taste Your Beer

I just ordered a beer appreciation kit from I’m all for less swilling, more appreciating. This one includes little vials of 13 different kinds of hops to compare. Train your palate, but be warned: once you start down this road, forever will it dominate your soul. You’ll be picking out different flavors in everything you eat or drink, and some things you don’t (like toothpaste). -m

Sunday, November 22nd, 2009

How Xanadu Works: technical overview

One particular conversation I’ve overheard several times, often in the context of web and standards development, has always intrigued me. It goes something like this:

You know, Ted Nelson’s hypertext system from the 60’s had unbreakable, two-way links. It was elegant. But then came along Tim Berners-Lee and HTML, with its crappy, one-way, breakable links, and it took over the world.

The general moral of the story is usually about avoiding over-thinking problems and striving for simplicity. This has been rolling around in the back of my mind ever since the first time I heard the story. Is it an accurate assessment of reality? And how exactly did Nelson’s system, called Xanadu (R), manage the trick of unbreakable super-links? Even if the web ended up going in a different direction, there still might be lessons to learn for the current generation of people building things that run (and run on) the web.

Nelson’s book Literary Machines describes the system in some detail, but it’s hard to come by in the usual channels like Amazon, or even local bookstores. One place does have it, and for a reasonable price too: Eastgate Systems. [Disclosure: I bought mine from there for full price. I’m not getting anything for writing this post on my blog.] The book has a versioning notation, with 93.1 being the most recent, describing the “1993 design” of the software.

Pause for a moment and think about the history here. 1993 is 16 years ago as I write this, about the same span of time between Vannevar Bush’s groundbreaking 1945 article As We May Think (reprinted in full in Literary Machines) and Nelson’s initial work in 1960 on what would become the Xanadu project. As far as software projects go, this one has some serious history.

So how does it work? The basic concepts, in no particular order, are:

  • A heavier-weight publishing process: Other than inaccessible “privashed” (as opposed to “pub”lished) documents, once published, documents are forever, and can’t be deleted except in extraordinary circumstances and with some kind of waiting period.
  • All documents have a specific owner, are royalty-bearing, and work through a micropayment system. Anyone can quote, transclude, or modify any amount of anything, with the payments sorting themselves out accordingly.
  • Software called a “front end” (today we’d call it a “browser”) works on behalf of the user to navigate the network and render documents.
  • Published documents can be updated at will, in which case unchanged pieces can remain unchanged, with inserted and deleted sections in between. Thus, across the history of a document, there are implicit links forward and backward in time through all the various editions and alternatives.
  • In general, links can jump to a new location in the docuverse or transclude part of a remote document into another, and many more configurations, including multi-ended links, and are granular to the character level, as well as attached to particular characters.
  • Document and network addressing are accomplished through a clever numbering system (somewhat reminiscent of organic versioning, but in a way infinitely extensible on multiple axes). These address, called tumblers, represent a Node+User+Document+Subdocument, and a minor variant to the syntax can express ranges between two points therein.
  • The system uses its own protocol called FEBE (Front End Back End) which contains at several verbs including on page 4/61: RETRIEVEV (like HTTP GET), DELETEVSPAN, MAKELINK, FINDNUMOFLINKSTOTHREE, FINDLINKSFROMTOTHREE, and FINDDOCSCONTAINING [Note that “three” in this context is an unusual notation for a link type] Maybe 10 more verbs are defined in total.

A few common themes emerge. One is the grandiose scope: This really is intended as a system to encompass all of literature past, present, and future, and to thereby create a culture of intellect and reshape civilization. “We think that anyone who actually understands the problems will recognize ours approach as the unique solution.” (italics from original, 1993 preface)

Another theme is simple solutions to incredibly difficult problems. So the basic solution to unbreakable links is to never change documents.  Sometimes these solutions work brilliantly, sometimes they fall short, and many times they ends up somewhere in between. In terms of sheer vision, nobody else has come close to inspiring as many people working on the web. Descriptions of what today we’d call a browser would sound familiar, if a bit abstract, even to casual users of Firefox or IE.

Nothing like REST seems to have occurred to Nelson or his associates. It’s unclear how widely deployed Xanadu prototypes ever were, or how many nodes were ever online at any point. The set of verbs in the FEBE protocol reads like that a competent engineer would come up with. The benefits of REST, in particular of minimizing verbs and maximizing nouns, are non-obvious without a significant amount of web-scale experience.

Likewise Creative Commons seems like something the designers never contemplated.  “Ancient documents, no longer having a current owner, are considered to be owned by the system–or preferably by some high-minded literary body that oversees their royalties.” (page 2/29) While this sounds eerily like the Google Books settlement, this misses the implications of truly free-as-in-beer content, but equally misses the power of free-as-in-freedom documents. In terms of social impact there’s a huge difference between something that costs $0 and $0.000001.

In this system anyone can include any amount of any published document into their own without special permission. In a world where people writing Harry Potter Lexicons are getting sued by the copyright industry, it’s hard to imagine this coming to pass without kicking and screaming, but it is a nice world to think about. Anyway, in Xanadu per-byte royalties work themselves out according to the proportion of original vs. transcluded bytes.

Where is Google in this picture? “Two system directories, maintained by the system itself, are anticipated: author and title, no more” (page 2/49) For additional directories or search engines, it’s not clear how that would work: is a search results page a published or privashed document? Does every possible older version of every result page stick around in the system? (If not, links to/from might break) It’s part of a bigger question about how to represent and handle dynamic documents in the system.

On privacy: “The network will not, may not monitor what is written in private documents.” (page 2/59) A whole section in chapter 3 deals with these kinds of issues, as does Computer Lib, another of Nelson’s works.

He was early to recognize the framing problem: how in a tangle of interlinked documents, to make sense of what’s there, to discern between useful and extraneous chunks. Nelson admits to no general solution, but points at some promising directions, one of which is link typing–the more information there is on individual links, the more handles there are to make sense of the tangle. Some tentative link types include title, author, supersession, correction, comment, counterpart, translation, heading, paragraph, quote, footnote, jump-link, modal jump-link, suggested threading, expansion, citation, alternative version, comment, certification, and mail.

At several points, Nelson mentions algorithmic work that makes the system possible. Page 1/36 states “Our enfilade data structures and methods effectively refute Donald Knuth’s list of desirable features that he says you can’t have all at once (in his book Fundamental Algorithms: Sorting and Searching)”. I’m curious if anyone knows more about this, or if Knuth ever got to know enough details to verify that claim, or revise his.

So was the opening anecdote a valid description of reality? I have to say no, it’s not that simple. Nelson rightly calls the web a shallow imitation of his grand ideas, but those ideas are–in some ways literally–from a different world. It’s not a question of “if only things had unfolded a bit differently…”. To put it even more strongly, a system with that kind of scope cannot be designed all at once, in order to be embraced by the real world it has to be developed with a feedback loop to the real world. This in no way diminishes the value and influence of big ideas or the place that Roarkian stick-to-your-gunnedness has in our world, industry, and society. We may have gotten ourselves into a mess with the architecture of the present web, but even so, Nelson’s vision will keep us aspiring toward something better.

I intend to return to this posting and update it for accuracy as my understanding improves. Some additional topics to maybe address are: a more detailed linking example (page 2/45), comparing XLink to Xanadu, comparing URIs and tumblers, and mention the bizarre (and yet oddly familiar if you’ve ever been inside a FedEx Kinkos) notion of “SilverStands”.

For more on Nelson, there is the epic writeup in Wired. YouTube has some good stuff too.

Comments are welcome. -m

Xanadu is a registered trademark, here used for specific identifying purpose.

Wednesday, November 18th, 2009

MarkLogic and XSLT

MarkLogic fans should check out Norm Walsh’s posting about his talk at the NY User Group. If you follow the right Twitter feeds, this is probably not too much of a surprise, but now the cat is officially disjoint with the volume inside the bag. Disclaimer: be sure to read the disclaimer there. -m

Sunday, July 19th, 2009

Geek Thoughts: The integral is a lie

There are no limits, only quanta. There is no smooth, only lumpy.

Analog is a myth; the world is digital. We just haven’t found the extent of its mantissa. – SymbolismLost.

More collected Geek Thoughts at

Wednesday, June 10th, 2009

The Inmates are Running the Asylum: review and RFE

The central thesis of The Inmates are Running the Asylum by Alan Cooper is dead on: engineers get too wrapped up in their own worlds, and left entirely to their own whims can easily make a product incomprehensible to ordinary folks. For this reason alone, it’s worth reading.

But I do question parts of his thesis. He (with tongue in cheek) posits the existence of another species of human, called Homo Logicus. Stepping on to an airplane, Homo Logicus turns left into the cockpit with a million buttons but ultimate control over every aspect of the plane. Regular Homo Sapiens, on the other hand, turn right and tuck themselves into a chair–no control but at least they can relax.

But if there was only one “species” of Homo Logicus, members (like me) would never experience usability issues in software created by fellow Logicians. But ordinary fax machines give me fits. The touch-screen copier at work instills dread in my heart. And the software I need to use to file expense reports–written by enterprise software geeks probably very similar to me–is a usability nightmare. Words fail me in expressing my disdain for this steaming heap of fail.

The book is sub-titled “Why High-Tech Products Drive Us Crazy”, but one doesn’t have to look very far to find similar usability bugs in the low-tech world. Seth Godin, for example, likes to talk about different things in life that Just Don’t Work, along with reasons why. Some examples:

  • airport cab stand (75 cabs, 75 people, and it takes an hour)
  • “don’t operate heavy machinery” warning on dog’s prescription medicine
  • excessive fine print on liability agreements–intentionally hard to read and figure out
  • official “Vote for Pedro” shirts that look nothing like the ones in the movie
  • more examples on the web site

If anything, I think Cooper’s work doesn’t go far enough. It is relatively short on good examples, stretching out only four examples over four chapters. If properly-designed software is so hard to come up with examples of, then there are bigger problems in play (that would need to be dealt with by something more manifesto than book).

The book now 5 years old. Perhaps it’s time for an update. Particularly in the world of web software, lots has happend in 5 years. Flickr. Gmail. Yahoo Pipes. Google Docs. Even SearchMonkey. Instead of focusing on pointing at crappy software, I’d like to see more emphasis on properly-done interfaces. More delving into nuance, and common factors behind why both high-tech and low-tech products miss the mark.

But maybe that’s just me. -m

Saturday, April 4th, 2009

On YouTube’s bandwidth and Technologizer’s problem with basic estimation

This article states:

The analysts determined YouTube’s bandwidth costs by assuming that 375 million unique visitors would visit the site in 2009, with 20 percent of those users consuming 400 kilobits per second of video at any given time. That works out to 30 million megabits being served up per second. That’s a heck of a lot of bandwidth to devote to videos of sneezing pandas.

Do you honestly believe that YouTube is sending out 30 petabits per second (to put it another way, fully saturating over 200,000 OC3 connections)? That on average, every single user who counts as a unique visitor in 2009 spends 20% of 24hrs = 4.8 hours actually downloading video, every day of every week?

Gesundheit. -m

Update: the quoted article indeed gets it wrong, though it appears the original Credit Suisse analyst report was estimating peak usage, not a running average. Still doesn’t smell right. Updating the article and title to point the finger at the right people.

Wednesday, November 19th, 2008

Geek Thoughts: writing contest

To celebrate the unlamented demise of Valleywag, use as many of the ten insulting words you should know (along with any other appropriate words) as you can in a single short paragraph. Post in the comments below. This site is for geeks of all ages, so keep things PG. All right, PG-13. My favorite will be announced later.

More collected Geek Thoughts at

Tuesday, November 11th, 2008

No NaNoWriMo for me this year

I’ve successfully completed the National Novel Writing Month challenge–to write 50,000 words during the month of November–for three years running, and now I have three draft novels sitting around. At some point, racking up mere drafts gets to be pathetic, so this November I’m picking one to dig into with a heavy editing pass. I’m stocked up on red pens and ready to go, and maybe a third of the way through at this moment. Other writing efforts, like say this blog, get back-burnered for a few weeks.

The key to editing long works, by the way, is momentum. If you get stuck in one place, you quickly get burned out on it, like melting celluloid in an old-skool movie projector. Onward. -m

Monday, August 25th, 2008

Milorad Cavic is a cool guy

He won an olympic silver medal in the much-publicized race where the gold winner came in 0.01 second faster. And he blogs. He writes:

On winning a SILVER medal: I am completely happy, and still in complete disbelief that I was able to achieve this feat! I’m not joking… It’s a tough loss, but I’m on cloud nine. I congratulated Phelps and his coach Bob Bowman. I’m just glad the race was fun to watch for everyone. It was a pleasure for me, really.

What a fantastic attitude. I wish I had my head on that straight soemtimes. Milorad, if you are ever in the South Bay, I will buy you a beer, or a 1000-calorie energy drink, or whatever. -m