Archive for the 'IPR' Category

Sunday, November 22nd, 2009

How Xanadu Works: technical overview

One particular conversation I’ve overheard several times, often in the context of web and standards development, has always intrigued me. It goes something like this:

You know, Ted Nelson’s hypertext system from the 60’s had unbreakable, two-way links. It was elegant. But then came along Tim Berners-Lee and HTML, with its crappy, one-way, breakable links, and it took over the world.

The general moral of the story is usually about avoiding over-thinking problems and striving for simplicity. This has been rolling around in the back of my mind ever since the first time I heard the story. Is it an accurate assessment of reality? And how exactly did Nelson’s system, called Xanadu (R), manage the trick of unbreakable super-links? Even if the web ended up going in a different direction, there still might be lessons to learn for the current generation of people building things that run (and run on) the web.

Nelson’s book Literary Machines describes the system in some detail, but it’s hard to come by in the usual channels like Amazon, or even local bookstores. One place does have it, and for a reasonable price too: Eastgate Systems. [Disclosure: I bought mine from there for full price. I’m not getting anything for writing this post on my blog.] The book has a versioning notation, with 93.1 being the most recent, describing the “1993 design” of the software.

Pause for a moment and think about the history here. 1993 is 16 years ago as I write this, about the same span of time between Vannevar Bush’s groundbreaking 1945 article As We May Think (reprinted in full in Literary Machines) and Nelson’s initial work in 1960 on what would become the Xanadu project. As far as software projects go, this one has some serious history.

So how does it work? The basic concepts, in no particular order, are:

  • A heavier-weight publishing process: Other than inaccessible “privashed” (as opposed to “pub”lished) documents, once published, documents are forever, and can’t be deleted except in extraordinary circumstances and with some kind of waiting period.
  • All documents have a specific owner, are royalty-bearing, and work through a micropayment system. Anyone can quote, transclude, or modify any amount of anything, with the payments sorting themselves out accordingly.
  • Software called a “front end” (today we’d call it a “browser”) works on behalf of the user to navigate the network and render documents.
  • Published documents can be updated at will, in which case unchanged pieces can remain unchanged, with inserted and deleted sections in between. Thus, across the history of a document, there are implicit links forward and backward in time through all the various editions and alternatives.
  • In general, links can jump to a new location in the docuverse or transclude part of a remote document into another, and many more configurations, including multi-ended links, and are granular to the character level, as well as attached to particular characters.
  • Document and network addressing are accomplished through a clever numbering system (somewhat reminiscent of organic versioning, but in a way infinitely extensible on multiple axes). These address, called tumblers, represent a Node+User+Document+Subdocument, and a minor variant to the syntax can express ranges between two points therein.
  • The system uses its own protocol called FEBE (Front End Back End) which contains at several verbs including on page 4/61: RETRIEVEV (like HTTP GET), DELETEVSPAN, MAKELINK, FINDNUMOFLINKSTOTHREE, FINDLINKSFROMTOTHREE, and FINDDOCSCONTAINING [Note that “three” in this context is an unusual notation for a link type] Maybe 10 more verbs are defined in total.

A few common themes emerge. One is the grandiose scope: This really is intended as a system to encompass all of literature past, present, and future, and to thereby create a culture of intellect and reshape civilization. “We think that anyone who actually understands the problems will recognize ours approach as the unique solution.” (italics from original, 1993 preface)

Another theme is simple solutions to incredibly difficult problems. So the basic solution to unbreakable links is to never change documents.  Sometimes these solutions work brilliantly, sometimes they fall short, and many times they ends up somewhere in between. In terms of sheer vision, nobody else has come close to inspiring as many people working on the web. Descriptions of what today we’d call a browser would sound familiar, if a bit abstract, even to casual users of Firefox or IE.

Nothing like REST seems to have occurred to Nelson or his associates. It’s unclear how widely deployed Xanadu prototypes ever were, or how many nodes were ever online at any point. The set of verbs in the FEBE protocol reads like that a competent engineer would come up with. The benefits of REST, in particular of minimizing verbs and maximizing nouns, are non-obvious without a significant amount of web-scale experience.

Likewise Creative Commons seems like something the designers never contemplated.  “Ancient documents, no longer having a current owner, are considered to be owned by the system–or preferably by some high-minded literary body that oversees their royalties.” (page 2/29) While this sounds eerily like the Google Books settlement, this misses the implications of truly free-as-in-beer content, but equally misses the power of free-as-in-freedom documents. In terms of social impact there’s a huge difference between something that costs $0 and $0.000001.

In this system anyone can include any amount of any published document into their own without special permission. In a world where people writing Harry Potter Lexicons are getting sued by the copyright industry, it’s hard to imagine this coming to pass without kicking and screaming, but it is a nice world to think about. Anyway, in Xanadu per-byte royalties work themselves out according to the proportion of original vs. transcluded bytes.

Where is Google in this picture? “Two system directories, maintained by the system itself, are anticipated: author and title, no more” (page 2/49) For additional directories or search engines, it’s not clear how that would work: is a search results page a published or privashed document? Does every possible older version of every result page stick around in the system? (If not, links to/from might break) It’s part of a bigger question about how to represent and handle dynamic documents in the system.

On privacy: “The network will not, may not monitor what is written in private documents.” (page 2/59) A whole section in chapter 3 deals with these kinds of issues, as does Computer Lib, another of Nelson’s works.

He was early to recognize the framing problem: how in a tangle of interlinked documents, to make sense of what’s there, to discern between useful and extraneous chunks. Nelson admits to no general solution, but points at some promising directions, one of which is link typing–the more information there is on individual links, the more handles there are to make sense of the tangle. Some tentative link types include title, author, supersession, correction, comment, counterpart, translation, heading, paragraph, quote, footnote, jump-link, modal jump-link, suggested threading, expansion, citation, alternative version, comment, certification, and mail.

At several points, Nelson mentions algorithmic work that makes the system possible. Page 1/36 states “Our enfilade data structures and methods effectively refute Donald Knuth’s list of desirable features that he says you can’t have all at once (in his book Fundamental Algorithms: Sorting and Searching)”. I’m curious if anyone knows more about this, or if Knuth ever got to know enough details to verify that claim, or revise his.

So was the opening anecdote a valid description of reality? I have to say no, it’s not that simple. Nelson rightly calls the web a shallow imitation of his grand ideas, but those ideas are–in some ways literally–from a different world. It’s not a question of “if only things had unfolded a bit differently…”. To put it even more strongly, a system with that kind of scope cannot be designed all at once, in order to be embraced by the real world it has to be developed with a feedback loop to the real world. This in no way diminishes the value and influence of big ideas or the place that Roarkian stick-to-your-gunnedness has in our world, industry, and society. We may have gotten ourselves into a mess with the architecture of the present web, but even so, Nelson’s vision will keep us aspiring toward something better.

I intend to return to this posting and update it for accuracy as my understanding improves. Some additional topics to maybe address are: a more detailed linking example (page 2/45), comparing XLink to Xanadu, comparing URIs and tumblers, and mention the bizarre (and yet oddly familiar if you’ve ever been inside a FedEx Kinkos) notion of “SilverStands”.

For more on Nelson, there is the epic writeup in Wired. YouTube has some good stuff too.

Comments are welcome. -m

Xanadu is a registered trademark, here used for specific identifying purpose.

Wednesday, October 7th, 2009

US Federal Register in XML

Fed Thread is a front end for the newly XMLified Federal Register. Why is this a big deal? It’s a daily publication of the goings-on of the US government. It’s a primary source for all kinds of things that normally only get rehashed through news organizations. And it is bulky–nobody can read through it on a regular basis. A yearly subscription (printed) would cost nearly $1000 and fill over 80,000 pages.

Having it in XML enables all kinds of searching, syndication, and annotation via flexible front ends like this one. Yay for transparency. -m

Friday, May 22nd, 2009

More on the GOOG book settlement

From Brewster Kahle. Good read, so to speak. -m

Wednesday, April 29th, 2009

Pamela Samuelson on the Google Book Settlement

I found this explanation the most readable I’ve seen yet. She has slides too.  The settlement itself has been recently delayed, which seems like a good idea for something of this magnitude. -m

Saturday, April 11th, 2009

Subversive Google?

Google for RIAA, get this first result:

RIAARecording Industry Association of America – April 12, 2009

Trade group that claims to represent the US recording industry. Details on services, members, executives profiles, statistics, and contact information.

“Claims to” represent the US recording industry? The word “claims”, accurate as it may be, appears nowhere on their front page. :-)


Tuesday, April 7th, 2009

GPL’s Cloudy Future

I enjoyed this post, from Jeremy Allison as it turns out. It talks about how GPL software is “the new BSD” when it comes to cloud computing, since redistribuion of the software doesn’t happen, and thus doesn’t trigger the relevant clauses of the GPL. Any old company can use, re-use, and modify the software without sharing the code in the original spirit of the license. The community’s response–something I need to keep a closer eye on–is the AGPL, or Affero license. It works similarly to the GPL, but is triggered by remote use of the software, not just distribution, preserving the work’s copylefedness even in cloud computing situations. -m

Tuesday, March 3rd, 2009

Geek Thoughts: Amazon backtracks on text to speech

With apologies to a real news site. (02-27) 16:14 PST SEATTLE, (AP) Inc. changed course Friday and said it would allow copyright holders to decide whether they will permit their works to be read aloud by the latest laryngeal apparatus, a feature that has been under development for several thousand years.

The move comes nearly two weeks after a group representing authors expressed concern that the feature, which was intended to be able to read every book, blog, magazine and newspaper out loud, would undercut separate audiobook sales. The average American can use their larynx to read text in a somewhat stilted voice.

Amazon said in a statement that it, too, has a stake in the success of the audiobook market, and pointed to its Brilliance Audio and Audible subsidiaries, which publish and sell professionally recorded readings.

“Nevertheless, we strongly believe many rights holders will be more comfortable with the text-to-speech feature if they are in the driver’s seat,” the company said.

Amazon is working on the technical changes needed for authors and publishers to turn text-to-speech off for individual titles.

The Web retailer also said the text-to-speech feature is legal — and wouldn’t require Amazon to pay out additional royalties — because a book read aloud doesn’t constitute a copy, a derivative work or a performance.

More collected Geek Thoughts at

Tuesday, March 3rd, 2009

Boo to Amazon

Dear Amazon, Speaking as an author myself, you not only made a bad choice, you set a precedent in the wrong direction. The Author’s Guild doesn’t speak for me, nor do I want them to. TTS is only going to get better. The last thing we need is another backward industry fighting progress. -m

Monday, February 23rd, 2009

How Orbo works

I’m (just barely) enough of a writer that I can spend cycles on Steorn‘s claims without being branded a crackpot. After all, the novel I’m working on involves a similar device being invented 4,000 years ago. It’s all research.

Imagine if Earth’s gravitational field, instead of being a constant 1.0G, rocked back and forth between 0.99G and 1.01G at some fixed interval. That’d be perhaps not enough to feel, but enough to extract “free energy”. Arrange a heavy weight on a wheel, and time it so that it moves downward (doing work) during the heavier phase and returns to the top during the lighter phase. You’d have more than perpetual motion, you would be able to extract real work out of the device on a continuous basis.

Steorn’s claims are similar, but with permanent magnets instead of gravity.

Orbo is based upon time variant magnetic interactions, i.e. magnetic interactions whose efficiency varies as a function of transaction timeframes.

I get the feeling that they are being very, very careful about what they write. In particular, the word “efficiency” is very odd in this sentence. In my earlier example, it would sound unnatural to talk about the “efficiency of the gravitational interaction”. Unless one talks about the kinds of efficiency that go above 100%…. So let’s roll with it.

It is this variation of energy exchanged as a function of transaction time frame that lies at the heart of Orbo technology, and its ability to contravene the principle of the conservation of energy. Why? Conservation of energy requires that the total energy exchanged using interactions are invariant in time. This principle of time invariance is enshrined in Noether’s Theorem.

So some hitherto unknown process temporarily nudges a magnetic interaction in one direction, only for it to bounce back in the opposite direction, like in the gravity example. Get the timing right and presto, free energy. I don’t understand why they are so cavalier about “contravening” the principle of conservation of energy though. It seems to me that more observations would be in order. As in “the device produced 100 watts for 6 months straight, with no input power sources”–which could be true in various ways that don’t contravene conservation of energy. It’s almost as if they are deliberately being provocative in their statements. Go figure. -m

Tuesday, December 30th, 2008

RDFa parser in XQuery now open source

After a delay, the code to my RDFa parser in XQuery is now available under an Apache license. Go get it. This is some of the earliest XQuery code I ever wrote, so go easy on me. It follows the earlier work on a functional definition of RDFa. And feel free to send in patches. -m

Friday, August 22nd, 2008

The deal with Geek Thoughts

By now you’ve likely noticed the Geek Thoughts postings here. This is an experiment on a few different levels.

What makes comics special? To what extent are pictures, often little more than stick figures, a critical part of the web comic experience? Can a web comic still be funny and thought-provoking with only words?

Specifically with regard to yesterday’s posting, an homage to Garfield minus Garfield, and slightly-more-than-homage to xkcd: what does taking away the seeming-essential part reveal? Is it of the same or different nature as before? On the plus side, don’t worry about this blog becoming an xkcd transciption service–that’s not the point. Thinking (and maybe laughing) is.

It’s also an experiment in zero-overhead publishing. Setting up a dedicated blog, separate site, separate comment moderation, all that jazz…would be hard. Lower friction is the difference between a smooth running engine and a smoking heap of metal, and the same goes in life. If Geek Thoughts develops a huge following, maybe some day there will be all of that and T-Shirts too. But for now, it’s easy enough that I can actually do it, which is what matters in the beginning.

If this line of argument seems faintly familiar, it’s because I’ve used it before, with my (still sporadically updated) Patternalia series, inspired by Christopher Alexander’s works.

If you appreciate any of this, the best way to show it is with a link. Thanks! -m

Wednesday, July 16th, 2008

Starting to wrap my head around XQuery 1.1

Looks like a reasonably-sized revision. The first public working draft seems downright thin, in fact, relative to all the SHOULDs and MAYs in the requirements document. In particular, I’d like to see progress on 2.3.16 Higher order functions. (Then do we get a book XQuery: The Good Parts? …kidding..)


Tuesday, April 22nd, 2008

Shame on you, J.K. Rowling

Harry Potter author J.K. Rowling, herself rowling in gazillions of dollars, is along with her publisher suing Steven Vander Ark, a poor librarian who produced a lexicon of the Harry Potter universe.

Rowling says it’s not about the money, it’s about control. Poppycock. If that was the case, she would have objected to the web site. Instead, Rowling is quoted as saying:

This is such a great site that I have been known to sneak into an Internet cafe while out writing and check a fact rather than go into a bookshop and buy a copy of Harry Potter (which is embarrassing).

A lexicon is a collection of existing (fictional) facts, not something that is going to wrest creative control of the franchise away from the author. This work makes the Harry Potter universe more valuable, not less. Even if legally this is a gray area, it’s a boneheaded move to sue one of your greatest fans for providing a valuable and useful reference.

What troubles the bean counters so much is that the printed lexicon costs, well, actual money, $24.95 to be exact. As an author it troubles me to see how out of touch copyright law is, and how badly the scent of a few dollars can make an otherwise reasonable person behave. -m

Thursday, March 20th, 2008

Geeking out

I have here a pre-release copy of Cory Doctorow’s novel Little Brother.

With permission.

In plain text.

Being read with the UNIX command less.

On an XO laptop.

And so far it’s awesome. -m

Saturday, December 29th, 2007

SCO Group, long delisted from reality; Nasdaq follows suit

The new ticker symbol is SCOXQ.PK, as in “pink sheet”. From the soaring heights of the $20s, it’s now under a dime per share, as bankruptcy proceedings move forward and their attempt to charge fees for all Linux users continues to crumble. Serves ’em right. -m

Tuesday, November 27th, 2007

Amazon’s most valuable IP

Or, why the Kindle cost $399 at launch.

What is Amazon’s most valuable IP?

How about a list of registered users who are guaranteed as willing to pay a premium price for a nifty gadget (I mean “service”) along with the exclusive privilege of buying more things from Amazon? Somewhere in Amazon’s database land, alongside all the details and purchasing history they already have for each customer, there’s a single bit called something like owns_kindle. Those bearing this mark are the ur-early-adopters, the loyalists, the customers with a vary large net future value. The marketers dream. Opt-in isn’t even an issue–what Kindle owner won’t be interested in special offers and exclusive deals for their special device? Where else are they going to go?

That one bit alone is probably worth another $400, making it the most valuable IP in terms of dollars-per-byte that Amazon holds. Even if they do a drastic price cut soon (and such price cuts will at some point be inevitable to sustain the market), even if they refund half of the difference to the early adopters, they will come away with super-sized smiles. -m

P.S. s/Kindle/iPhone/ and s/Amazon/Apple/ and this entire post still holds.

P.P.S. There is a pretty good play Apple could make here around an ebook reader. Tie it to the same wireless service plan that the iPhone uses, make books available through the iTunes store (including tons of Gutenberg/public domain content/creative commons for free), and put it on a very slick designed piece of hardware. But even in this case, it will initially sell for a premium price for the reasons above. Game on!

Monday, November 19th, 2007

Kindle my disappointment

Where’s Project Gutenberg? One difficulty in launching an ebook platform is the lack of available titles. I keep hearing about 80,000+ titles, but expressed as a percentage of Amazon’s book catalog, it’s minuscule. There should be all kind of public domain titles ready to go on day one. And where’s the Creative Commons books?
There’s some public domain books to be found, but none are free. Take, for example, A Connecticut Yankee in King Arthur’s Court, a book (in paper form) sitting just out of arm’s reach as I write this, waiting to be read. If I had it on a device, particularly one with a good screen, I’d be more inclined to keep it, and dozens others, on hand in my backback and be ready to read at a moment’s notice. But no.

The problem is the the “we take care of the wireless delivery” part, called Whispernet(tm). It’s not really free, nor bundled in the service price. It’s bundled in to the cost of every media access. Is it fair to pay $9.99 for a New York Times bestseller? Sure. But it sucks to pay $1 for an A-list blog that’s free everywhere else, or to get literally nickeled and dimed for the privelege of “converting” and delivering your own content to your own device.

By the way, who gets the money paid for accessing, say, a CreativeCommons non-commercial licensed blog via the Kindle? Somebody should look into that.

I applaud Amazon for pushing to innovate in a space that badly needs it, but the financial model behind the wireless access encourages the wrong kind of things. Exceptions, like unlimited Wikipedia access (be still my heart!) still need to be hand approved by the gatekeeper. Information wants to be free, it doesn’t want to be a service, though that’s hard to see when the dollar signs get in your eyes.

Many folks are comparing this to the original iPod launch–remember, the huge klunky one with a tiny capacity, black and white screen, and a mechanical click-wheel? There’s some strong points of similarity, but stronger differences. For one, anyone with an iPod can easily rip their existing CDs, not to mention obtain MP3s from other methods (so I hear). There’s nothing like that yet for books.
Where’s the documentation for the new, proprietary ebook format? I don’t care about the DRM crap. I care about being able to create new content, or repackage existing content for which I have the rights, and for that, I’m having trouble coming up with a rationale for an entire new format. I would love to do some cool things with this platform. Perhaps I will some day, though my enthusiasm is somewhat lessened by the difficulties I would face getting anything cool onto the devices. -m

Thursday, August 16th, 2007

My favorite image for the week

Here. -m

Sunday, August 5th, 2007

The planetary air conditioner

If a free energy device (like this one) were really possible, simple economics would dictate that (as soon as patents expired, etc.) one or several would be found in every cell phone, iPod, notebook computer, desktop computer, appliance, automobile, airplane, house, building, and factory. That’s a lot of waste heat that would get dumped onto the planet’s surface, maybe even worse than burning-stuff-to-produce-greenhouse-gasses.

But if you have unlimited energy, it would be straightforward engineering to produce a large scale air cooling unit that would (beam|radiate|dissipate) heat into space. In effect, a planetary-scale air conditioner.

Remember when you were younger and you left a window open with the A/C on, and somebody said ‘what are you trying to do, cool off the whole outdoors?’ Finally an answer to that question. :)
November, National Novel Writing Month, is coming up. Could this be the kernel of a story? What other practical considerations would there be around free energy? Comment below.


Sunday, December 31st, 2006

Which Superhero am I?

You are Superman

Green Lantern
The Flash
Iron Man
Wonder Woman
You are mild-mannered, good,
strong and you love to help others.

Too cool to resist posting. Try the quiz yourself -m

Wednesday, December 20th, 2006

Evil record companies, circa 1900

This “click-through” license on an Edison-style phonograph cylinder has been making the rounds. But don’t miss the front side: a giant photo of Edison himself, and his name in the largest font possible. As the photo caption says:

They really weren’t concerned with artist promotion, I guess.

From day one, the record companies have been more concerned about their own well-being than artists, it seems. -m

Wednesday, October 4th, 2006

DRM backlash proceeds

Check out this site from Dreamhost: Files Forever. All files are DRM-free and can be re-downloaded at any time. Expect to see more of this from an unhealthy market that wants to break free from artificial constraints. Currenly in beta and open only to Dreamhost customers. -m

Tuesday, September 12th, 2006

Microsoft frees 35 standards

I got this link from Eve, and to think, I never even knew there was a The Microsoft Open Specifications Promise irrevocably lets any interested parties implement and use a list of technologies without fear of getting sued (at least sued by Microsoft). It is similar in tone and scope to earlier declarations about the Office XML formats, and the declaration from Sun about UBL. I’m not a lawyer, so if I’ve described this badly, get a real lawyer to explain it. :-P
This is a smart move; since obviously a great deal of work went into producing these standards, I’m sure Microsoft plans to benefit more by growing the “whole market” (in the language of _Crossing the Chasm_) then they would by nickle-and-dime asserting patent rights. They also come out far, far better in public opinion, especially among those most affected by these standards.
There’s another angle worth considering–the defensive. Giving away patent rights carte blanche might at first seem like a funny kind of defense, but here’s how it works: after today, what would happen if BigWebServicesCo started shaking down implementers of WS-Whatever? The attacker would be savagely torn apart in the court of public opinion, that’s what. Submarine patents are dirty business, so for a bigger target, creating an environment more hostile to such bad behavior is a powerful strategy.

Of course, smallish parasitic patent troll companies won’t be deterred much, but then again nothing seems to.

I’m optimistic that this is part of a positive trend. I’ll even refrain from further opinions on the WS-* technologies. :-) -m