One one hand, it shows the intense amount of careful thought Apple puts into the user experience. But on the other hand, it highlights the discovery problem. I know people who have been using iOS since before it was called iOS, and still didn’t know about these. How do you put these kinds of finishing touches into a product and make sure the target audience can find out about them? -m

A while back I recorded my thoughts on what I called MET, or the Model Endpoint Template organizational pattern, as used in MarkLogic Application Builder. One difference between 2009 and now, though, is that browsers have distanced themselves even farther from XML, which tends to undercut the eliminate-the-impedance-mismatch argument. In particular, the forms model in HTML5 continues to prefer flat data, which to me indicates that models still play an important role in XQuery web apps.

So I envision the app lifecycle like this:

1. The browser requests a particular page, say the one that lets you configure sorting options in the app you’re building
2. An HTML page loads.
3. Client-side script requests the project state from a designated endpoint, the server transforms the XML into a flat list, and delivers it as JSON (as an optimization, the server can package the initial data into the page delivered in the prior step)
4. Standard form interaction and client-side scripting happens, including manipulation of repeating structures mediated by JavaScript
5. A standard form submit happens (possibly via script), sending a flat list back to the client, which performs an update to the stored XML.

Another thing that’s changed is the emergence of XQuery plugin technology in MarkLogic. There’s a rapidly-growing library of reusable components, initially centered around Information Studio but soon to cover more ground. This is going to have a major impact on XQuery app designs as components of the app (think visualization widgets) can be seamlessly added to apps.

Endpoints still make a ton of sense for XQuery apps, and provide the additional advantage that you now have a testable, concern-separated data layer for your app. Other apps have a clean way to interop, and even command-line operaton is possible with off-the-shelf-tools like wget.

Lastly, Templates. Even if you use plugins for the functional core of your app, there’s still a lot of boilerplate stuff you’d not want to repeat. Something like Mustache.xq is a good fit for this.

Which is all good–but is it MVC? This organizational pattern (let’s call it MET 2.0) is a lot closer to it. Does MET need a controller? Probably. (MarkLogic now ships a pretty good one called rest:rewrite) Like MVC, MET separates the important essences of your application. XQuery will never be Ruby or Java, and its frameworks will never be Rails or Spring, but rather something uniquely poised to capture the expressive power of the language to build apps on top of unstructured and big data. -m

If you look through these logs, pay particular attention to the referer field. This tells you on which site you were browsing when the data set out on its voyage toward facebook.

What’s wrong with Yahoo Mail?

A lot of web addresses start with http://–that’s a signal that the “scheme” used to deliver the page to your browser is something called HTTP, which is a technical specification that turns out is a really good way to move around web pages. As the page flows to the browser, it’s susceptible to eavesdropping, particularly over a wi-fi connection, and much more so in public, including the usual hotspots like coffee shops, but also workplaces and many home environments. It’s the virtual equivalent of a postcard. When you’re reading the news or checking traffic, it’s not a big deal if someone can sneak a glance at your page.

Some addresses start with https://–notice the extra ‘s’ which stands for “secure”. This means two things 1) that the web page being sent over is encrypted, and thus unavailable to eavesdroppers, and 2) that the people running the site had to obtain a certificate, which is a form of proof of their identity as an organization (that they’re not, say, Ukrainian phishers). Many years ago, serving pages over https was considered quite expensive in that servers needed much beefier processors to run all that encryption. Today, while it still requires extra computation, it’s not as big of a deal. Most off-the-shelf servers have plenty of extra power. To be fair, for a truly ginormous application with millions of users like Yahoo Mail, it is not a trivial thing to roll out. But it’s critically important.

First, to dispel a point of confusion, these days nearly every site, including Yahoo Mail, uses https for the login screen. This is the most critical time when encryption is needed, because otherwise you’d be sending your password on a postcard for anyone with even modest technical skills to peek at. So that’s good, but it’s no longer enough. Because sites are written so that you don’t have to reenter your password on every single new page, they use a tiny bit of information called a “cookie” in your browser to stay logged in. Cookies themselves are neither good nor bad, but if an eavesdropper gets a hold of one, they can control most of your account–everything that doesn’t require re-entering a password. In Yahoo Mail this includes reading any of your messages, sending mail on your behalf, or even deleting messages. Are you comfortable allowing strangers to do this?

As I mentioned earlier, new, more powerful tools have been out for months that automate the process of taking over accounts this way. Zero technical prowess is needed, only the ability to install a browser plug-in. If there are any web companies dealing in personal information for which this wasn’t a all-hands-on-deck security wake-up, they are grossly negligent. Indeed, other sites like Gmail work with https all-the-time. But still, in 2011, Yahoo Mail doesn’t. I have a soft spot for Yahoo as a former employer, and I want to keep liking them. Too bad they make it so difficult.

The deeper issue at stake is that if this serious of an issue goes unfixed for months, how many lesser issues lurk in the site and have been around for months or years? The issue is trust, my friend, and Yahoo just overdrew their account. I’m leaving.

FAQ

Q: So what do you want Yahoo to do about this?  A: Well, they should fix their site for their millions of remaining users.

Q: What if they fix it tomorrow? Will you delete this message?  A: No. Since I no longer trust the site, I am leaving, even though it takes time to notify all the people who still send me mail, and no matter what other developments unfold in the meantime. This page will explain my actions.

Q: Do you really want everyone else to leave Yahoo Mail?  A: No, only those who care about their privacy.

Q: What’s your new email address?  A: I have a couple, but <my first name> @ <this domain> is a good general-purpose one.

I will continue to update this page as more information becomes available. -m

Let me grab a random shortened link from Twitter. Don’t go away, I’ll be right back.

http://bit.ly/b1fYi1

OK, that’s six characters in the domain, a slash, and six more characters. 50 years from now, if bit.ly is still in operation, the URLspace will be rather more crowded, and the part after the slash might be eight or nine characters. This is a significant cliff, since most people have trouble remembering more than 6 or 7 things in their head at a time. Thus, one could conclude that 50 years from now, newly minted bit.ly URLs will be less fashionable than those from newer link-shortening services, particularly if more short TLDs come online, which seems likely. In that scenario, fewer and fewer people will use bit.ly, and it will become a resource-pit as costs go up (for more database storage, among other things) while usage drops, an economic trend that has only one eventual outcome, leading to the breaking all the external links relying on this service.

I’ve been picking on bit.ly here, but the same principle applies to any shortener service. In fact, the more popular, the more quickly the URLspace will fill.

The moral: don’t use link shorteners for anything that needs to be more durable than something you’d scribble on a scrap of paper at your desk.

More collected Geek Thoughts at http://geekthoughts.info.

Ready to start: Full of adrenaline and excitement. Audacious goals seem readily reachable.

5-10% through: Whoa, this is difficult! And I’m only 1/10 or 1/20 of the way through? What was I thinking? It is important to ignore these thoughts.

One third point: Things seem to even out by this point. The hard slog presses on.

Halfway point: Wow, that’s halfway? Feels more like 90%!

Two-thirds point: Things are getting difficult. Should have treated this more like a marathon, less like a sprint.

90% point: There are two distinct kinds of endeavors from here. In what I call ‘type 1′ projects, the goalposts are strictly fixed, in which case a fresh burst of energy propels me through the glorious finish. But in a more sinister ‘type 2′ project, the finish line keeps receding away, as fast as or faster than I can approach. Depending on my level of stubbornness and anger, I will often finish anyway, just to spite the universe and the project masters, but at significant personal cost.

For anyone out there that has influence over large, ambitious projects, one of the most pivotal things you can do is make sure it is a type 1, not a type 2 project, as seen from the 90% line.

Finish.

More collected Geek Thoughts at http://geekthoughts.info.

Some interesting tidbits overheard, in no particular order…

“it is not necessarily clear that this approach would capture the difference between the ridiculous and the merely implausible.”

Complexity — what is the relationship betwen complexity and long-term data storage?

“Narratives with fancy words in them”

How do you store, say, a video in a format that will be readable in 100 years?

Order of magnitude scale changes produce discontinuities

“The Da Vinci Schema”

Dandelion DNA (Free license)

“Indispensible” — “I don’t think that means what you think it does”

“Keeping electrons alive is really difficult”

“I wondered…with my Topic Map brain damage…”

-m

• Easy start: blogger (299 posts in 24 months)
• Succumbing to the desire to roll your own (259 posts in 12 months)
• Realizing that rolling your own is too difficult: Pyblosxom (172 posts in 12 months)
• Moving to a mature platform you don’t need to worry about much: WordPress (500 posts in 42+ months)

-m

it’s time for the W3C to show some tough love and force the two (X)HTML Working Groups together.

On July 2, the W3C wrote:

Today the Director announces that when the XHTML 2 Working Group charter expires as scheduled at the end of 2009, the charter will not be renewed. By doing so, and by increasing resources in the Working Group, W3C hopes to accelerate the progress of HTML 5 and clarify W3C’s position regarding the future of HTML.

The real test is whether the single HTML Working Group can be held to the standard of other Working Groups, and be able to recruit some much-needed editorial help from some of the displaced XHTML 2 gang.  -m

The analysts determined YouTube’s bandwidth costs by assuming that 375 million unique visitors would visit the site in 2009, with 20 percent of those users consuming 400 kilobits per second of video at any given time. That works out to 30 million megabits being served up per second. That’s a heck of a lot of bandwidth to devote to videos of sneezing pandas.

Do you honestly believe that YouTube is sending out 30 petabits per second (to put it another way, fully saturating over 200,000 OC3 connections)? That on average, every single user who counts as a unique visitor in 2009 spends 20% of 24hrs = 4.8 hours actually downloading video, every day of every week?

Gesundheit. -m

Update: the quoted article indeed gets it wrong, though it appears the original Credit Suisse analyst report was estimating peak usage, not a running average. Still doesn’t smell right. Updating the article and title to point the finger at the right people.

But think of the downside if every phone, iPod, refrigerator, car, air conditioning unit, factory, etc. comes to contain a perpetual energy source. Total energy use would skyrocket, and all that energy still has to go somewhere, so it ends up as waste heat. Global warming on an unprecedented scale ensues.

There’s more. If overabundance of energy is the problem, it’s a mere engineering challenge to build planetary-scale air coolers, beaming waste heat out into space. Imagine an advanced civilization that’s already doing it. From a distance their planet might look nothing like what current exoplanet researchers are looking for.

There’s enough here for several novels. If you had unlimited energy resources, what would you like to see built?

More collected Geek Thoughts at http://geekthoughts.info.

Is anyone else seeing a reversal in recruiter cold-call strategies? How flooded is the tech job market at this point? -m