Philosophy in re RSS 2.0

Intro 

Over the next few days, as we discuss next steps in the RSS corner-turn, I want to have a place to write notes and ideas that don't make it into a spec or a howto or whatever comes of this work, but still should be on the record. These ideas may be useful in some other similar work sometime in the future, or maybe not. I'm going to write in chronologic order, and point to these, by section, from Scripting News.

Discontinuities 

In the early days of the Macintosh I learned about discontinuities from Apple. They were breaking developers all the time. Here's the scenario -- one or two people inside Apple Engineering thought they found a way to make something more elegant. They'd flip-flop, in one rev they'd change an interface to work a different way, and then two revs later, when a new guy was working on the code, switch it back, and then a couple of revs later they'd switch again. When the Apple developers made a change to the interface, they probably didn't visualize all the developers who had to change every time they did, and even worse, all their co-developers and users who had to change too. Two versions of the software would have to be deployed for every such change. Testers would have to test with two versions of the OS. Eventually it became overwhelmingly expensive and developers just gave in and stopped trying to support the old OS when the new one came out. This led to lots of broken software, dissatisfied users, and imho a general malaise about software reliability that still exists. There was a time when software worked much better than it does today, it could because the systems it ran on were so simple. If you want software to continue to work, you have to be commited to minimizing change, only doing it when there's a real benefit to users, and further, by not taking things out, only adding things.

Now in the 80s, Apple was at the center of the Mac universe. Whether it was smart or not, they could change the way the Mac OS works. Today, we, as would-be designers of RSS 2.0, are not at the center of the universe. Sam Ruby says it's just code, and Simon Fell says it's just data, but RSS isn't even code or data, it's just a spec. I could write in that spec that everyone has to stand on their head every Tuesday at noon, but it wouldn't happen unless people wanted to do it. We have these discussions on various XML mail lists all the time. It's very tempting to believe that you are at the center, where all decisions are made, where all goodness flows from; that if you convice these people that you're right, that's as good as having it done -- but emphatically this is not true. At best you can influence and document current practice.

I learned this another way. My first product, ThinkTank on the Apple II, was more capable than my second, ThinkTank 128 on the Macintosh. I thought "No will ever know, Apple II users won't talk to Macintosh users." Heh. Of course that was wrong. My first Mac users were my Apple II users. So they wanted to know where all the features went. We burned their braincells. I learned from this that you can't take features out. That's why you have to be so careful about putting them in.

So it's in this context that I ask why should we "deprecate" some elements of RSS 2.0. What do we gain by doing that? And what do we give up? Sjoerd Visscher says that the developers we would break are active and they can fix the bugs this would introduce. That sounds like an argument an Apple engineer would have made. (And it may not be true.) But given a choice, would a developer choose to spend his or her time keeping up with your changes, or spend time adding new features to delight their users? Of course it's much more fun and profitable to create new features, and the opportunities for breakage are much smaller when you do that. So what's the gain? A more elegant spec? We can have a more elegant spec, with no pain or opportunities for breakage. We can reorganize the spec, so that the most important stuff is up front, and the more esoteric stuff is at the end. It's much easier to change a spec than it is to change all the software that is compatible with the spec. That path has to be taken first, and only if it doesn't work should we consider a path that breaks developers and users.

Consider another example, shell commands in Unix. It's been quite a few years since I was a Unix user. I suspect that if I started using Mac OS X I'd see some shell commands that seem quite weird to me. Now suppose Steve Jobs and Avie Tevanian both retire, and by some miracle I'm made King of Apple. "I know how to make the Macintosh really clean and elegant," I say. "By edict, as King of Apple, I hearby declare all the shell commands I don't understand to be deprecated. Zealots go forth and harass all developers who are using these commands." What would we gain? What would happen? Over the next few years all the newly missing commands would reappear, probably with different names, taking different parameters. It's this exercise in futility that's so common in computerdom that inspired the motto "Still diggin." Dig a hole, fill it in, dig another one, fill it in, etc etc ad infinitum.

Now that said there are one or two elements that I, personally, ache to deprecate. Nothing could make me happier than to toss textInput into the bit bucket forever. Same with skipHours and skipDays. But their persistence serves a purpose, they say to developers that this spec is rock solid. No one is going to gratuitously change it. If we didn't delete these things that so obviously suck, we're not going to delete or change the things you depend on. And get this, for all we know, some developer does depend on the existence of textInput, skipHours, and skipDays. They've been in the spec for three years. In those three years it was kosher to build on them.

There's more to say about this for sure.

Ole Eichorn on Apache 2.0 

Eichorn is CTO at Aperio Technologies. He writes..

Fell behind a little, just read your excellent essay on discontinuities. You hit another nail on the head, as usual.

One of the more significant recent discontinuities occurred with the release of Apache 2.0. Although it has been under-reported, Apache 2.0 is significantly discontinuous (non-backward-compatible) with Apache 1.3. Many webmasters have decided not to upgrade for now, rather than have to recode their custom modules. And many of the custom modules out there are 3rd party, so the resources to make the changes are not readily available.

It is not clear to me why the discontinuity was required. There was no technical reason not to maintain backward compatibility. I think your essay gets it right, the people who made these decisions were not involved in the original development, and were not sufficiently aware of the impact their decisions would have on their developer community. Multi-threading processes, which inspired most of the discontinuity, primarily benefits Windows sites – a small proportion of Apache installations - and most Windows sites use IIS and aren’t going to change.

I bet in a few years we’ll be able to track Apache’s decline as the leading web server back to this point.

Jon Gale's question 

I love Scripting News and RSS, but I just have to wonder why you're moving from 0.94 to 2.0? Why not 1? I guess 2 sounds better but is there a real reason for it? No matter what it's called I'll be using it

It would make sense wouldn't it -- but there already was an attempt to do a 1.0 that was not compatible with 0.91, so rather than have more confusion, we chose a version number that had never been used. Dave

Monsters 

There's always been an unwelcome personal side to "debates" on technical mail lists, but we've reached a new low. One of UserLand's competitors, Kevin Hemenway, the author of Amphetadesk, explains on his weblog how he intends to kill me. Even he says it's too harsh; and it may be a joke, if so, it's not funny. I don't see the humor in my own death, esp at the hands of a person like Hemenway. (He also coined the term Jewgregator, and calls RSS 2.0 "Hitler" for some reason.)

If you're participating in the discussion about RSS, please understand why I choose not to communicate with him. He's not the only one who plays this rough. Bill Kearney has sent me private email about my deathbed, and what he hopes to teach me there, so I've chosen to filter his mail to a place where I never see it. I tried to come up with a word to describe how I feel about these people, this is what I came up with: monster.

Now if I thought Kearney and Hemenway were serious, I'd call the police, immediately. I don't think they are. But I think they use these kinds of emotions to attract young people who don't know better to their "cause". Older more responsible people look away. To a point, until a line is crossed. Hemenway has crossed that line. What happens next is stuff that will involve the police. I won't stand for these kinds of threats.

None of this means that RSS 2.0 will be delayed by even one moment. I thought competition in the software business in the 80s was rough, but this is so much worse. Competition used to require a certain collegiality and professionalism. It's not true today. Anyone who works with Hemenway or Kearney should be aware that these people are nothing less than monsters, who will stoop to any level to get their way. Their perversion may even be the reason they're involved.

Mr. Hemenway goes by the name Morbus Iff on his weblog, and writes for O'Reilly Associates, and for Ben Hammersley's syndication weblog. Mr. Hammersley is a reporter for the UK Guardian newspaper.

Postscript: Ben Hammersley threatened to sue me if I don't remove the previous paragraph. But every statement is true. Hemenway's participation in Hammersley's site is public, as is the fact that Hammersley writes for the UK Guardian. And that's material because he reviewed RSS aggregators for the Guardian last month. What a double standard. Imagine the shitstorm that would ensue if I tried to censor Hammersley. He'd have a field day with that. What a trip.

Post-postscript: The Guardian requests an apology. For what? They ran a tainted review. Hammersley is a participant in the debate over the future of syndication technology, yet he wrote a review for the Guardian where that was not disclosed. Now, either Hammersley didn't tell them, or they don't care, or British newspapers run ads without saying they're ads. Whatever it is, this whole thing stinks. How dare they bully me into silence.

Post-post-postcript: Now Bill Kearney is threatening to sue me too. If you know Bill, please ask him not to escalate. I'm not going to apologize. He abuses me and my reputation with splendor all over the place. There is ample record of this in the archives of the public mail lists. This must stop. Suing me is not the answer. Back down Bill, become a mensch, control your mouth (or your keyboard) and stick to technology. Don't make it worse by dragging it out. Thanks for listening.

More about Monsters 

Jeff Barr's post on Wednesday saying I lied begat a bunch of other posts saying even worse things. That's the trouble when someone who has a good reputation, as Jeff does, uses it to slam another person. People who read quickly come away with an impression that we're both dirty.

I used to think of Barr as a colleague and an equal, as recently as last week at Seybold. Now he's a problem. For what cause? What did he gain by calling me a liar? He has to earn a living, well so do I. I want to move on, but how do you pick up the pieces when people have done so much deliberate damage and aren't letting up?

Kearney did in fact talk about teaching me a lesson on my deathbed, and truly the only word I could come up with for such a person is "monster." At the time Kearney said this, I had requested repeatedly over months that he not communicate with me about personal matters, and requested that he not communicate with me privately. I was recovering from heart surgery and very weak. Deathbeds were much on my mind for reasons having nothing to do with Kearney. I was sick then, but he was too, in a cruel way. Read his public posts. It's totally consistent with this. And by the way, I'm still recovering from the surgery and suffer from the disease. Not out of the woods.

I've put up with too much abuse from Kearney. Enough is enough. If you want to defend him Jeff, and say I don't have a right to say what I think, then let's go to the next level. In the US we have a First Amendment that's going to be inconvenient for you. Or you could back down, and we can go on creating technology and evangelizing it. It's your choice.

Sean Gallagher: "I mean, here we are, a country on the edge of being dragged into a war by some whack-job Texan with the IQ of a Post-It Note (TM), and people are badgering each other over RDF?"

The SOAP of Syndication 

Emailing with Joe Gregorio, who said on the RSS-DEV list that RSS should not be considered a standard. I agreed. I cringe when people call it a standard. I know how it was developed. It's a format, not a standard. Then the conversation swung around to what I thought should happen. I spelled it out.

Joe, this is what I think is needed -- a shift in attention.

1. Think of RSS as you think of XML-RPC. A non-buzzword compliant application of XML.

2. Assemble a small group to create the SOAP of Syndication, designed to be totally in synch with the best practices of the W3C. I won't be part of that group, but I will support it. It must not use the name RSS for its format.

3. Go on from there.

RSS is what it is. The problem, since 2000, is that people are trying to change what it means. But it's already out there. Just as SOAP couldn't have changed what XML-RPC is, the compliant syndication format can't and shouldn't erase RSS.

I'm cc'ing this to a few other people who I think should hear this. You're free to forward or post it if you want.

Dave

Aggregator upgrade for RSS 2.0 

If you are the developer of a news aggregator such as NetNewsWire, Straw or Aggie, or the one built into Radio UserLand, you can upgrade your software to take advantage of a new feature in RSS 2.0, item-level globally unique identifiers, or guids. This section explains how to do that.

Every aggregator must have a strategy for deciding if it's seen an item before. In Radio, with formats prior to RSS 2.0, we mash together the title, link and description to form an identifier. Any change in any of the three items causes an item to reappear at the top of the News page in Radio. So even if a news source changed the spelling of one word in the item description, that would trigger the re-display of the item. This was less than optimal.

Now, for 2.0 feeds, we check if the item has a guid, and use that as the identifier, assuming it won't change even if some of the properties of the item change. This is true of the 2.0 feeds that Radio produces, and of Scripting News in RSS. We welcome other aggregator developers to use this new feature in RSS 2.0 feeds.

RSS 2.0 and weblog software 

I sent this email to Evan Williams, Ben Trott and Jake Savin on 9/18. I'm posting it here because there are other tool vendors whose support for RSS 2.0 would be very welcome. On Monday I posted equivalent evangelism for the other side, for aggregators.

Dear Ev, Ben and Jake:

I am writing today as the author of the RSS 2.0 spec [1] to ask for your support in Blogger, Movable Type and Manila.

Scripting News is already available [2] in this format, and I have the code for Radio, which will be released later today.

You could think of this as evangelism. ;->

There are a couple of important features for weblogs in 2.0.

1. <pubDate> at the <item> level. This means that you can associate a post time with an item. This is a piece of data that every blogging system keeps track of, and can be used by aggregators if the tools support it.

2. <guid> at the <item> level. It's an awkward name for sure, but it does something important, especially if you make its value a permalink to the item. Guid stands for globally unique identifier. One of the problems in aggregators that's hard to solve without guids is not showing an item again when its title, description or link has changed. It will also be nice to have guids be permalinks so that aggregators can display them as such and connect the RSS feed directly to the author's weblog.

3. Support for namespaces, which means you can add elements to your feeds that take advantage of special features in your tool. I implemented a module [3] in just one hour yesterday. It's easy and fun and of course quite powerful. As the developer of an aggregator I look forward to new innovations from tool developers such as yourselves.

There are other enhancements that are of less consequence for weblogs, but also useful. Of course the politics have raged, but I think they will settle down. It's a good format, with RSS frozen (modulo clarification, ie no new features) it's a stable place to develop, and with namespace support, there's plenty of room for innovation. Needless to say RSS would benefit enormously from the support of any or all of your products. (I have some influence over Jake of course.) So please let me know if you have any questions. I would be very happy to work with you on getting RSS 2.0 support into the leading weblog tools.

Dave Winer

[1] http://backend.userland.com/rss

[2] http://www.scripting.com/rss.xml

[3] http://backend.userland.com/blogChannelModule