Design by Committee

I'm proud to say that v1.5 of the WSO2 Mashup Server is now available for download!  This release has been a few months in the making, and I'm thrilled to see a major update to the Mashup Server finally out.  News release here. 

Thanks to the efforts of Keith, Tyrell, Channa, Yumani, and many others at WSO2, we've been able to put some snazzy new features in this release:

• Integrated award-winning Data Services: quickly and with zero-coding map queries on your databases, Excel spreadsheets, or CSV files into Web services.
• Google Gadgets: power a Google gadget with a mashup service - we generate a try-it, a code template for customizing the gadget, and even a build in Gadget dashboard based on Apache Shindig.
• Security, security, security: Limit access to your mashup service to only authorized users, encrypt and sign your messages, and access services which are secured.  I think our security interface is the easiest way to engage advanced WS-Security features that's out there!
• OpenID login support.

There's lots of little improvements too, and I hope to talk about those, as well as the major features above, in upcoming posts.

For now though, a hearty congratulations and a good night's sleep to the Mashup Server team!

Yumani has published a nice tutorial on Sending Instant Messages via Mashup Server showing how to use the instant messaging capabilities built into the WSO2 Mashup Server.  I think the ability to have instant alerts as part of your mashup is a cool feature, and as Yumani shows in this article, they are pretty trivial to do.

I'm looking forward to adding the next step in a future release - the ability to invoke a mashup using instant messaging.  Then we'll have a scriptable platform for building agents that can communicate via IM.  Imagine converting your favorite service to an IM conversational interface.  Or making a data source available through an IM conversation.  Or even imagine new games...  Endless possibilities!

Just posted a few snapshots from a Saturday day hike to Loch Leven Lakes, a wonderful and fairly accessible bit of the High Sierras.  Haven't been there in a while, despite it being a fantastic place for some bouldering with the kids.  It was good to get above California's persistent smoky haze for a few hours (though later in the afternoon even the high country took on a smoky hue.)

Wishing everyone a last minute happy 4th of July, and for those of you who missed fireworks this year, here's a couple of photos to hold you till next year.  Full Flickr set here.

Microsoft TechEd Online has posted a video taped in Orlando after our successful keynote appearance.  This video captures a panel discussion that I participated in - I'll just quote the blurb:

This Panel discussion centers around the use of standards to facilitate interoperability between different technologies. The participants have all implemented their own version of a reference application and speak about any challenges they faced in implementing the application. The main discussion points of this Panel include: interoperability through SOAP; WS-* standards and how they facilitate interoperability; common problems with interoperability; identity and security in a distributed application; and future directions/remaining challenges in interoperability. With Raghu Thiagarajan, Gerald Beuchelt, Gregory Leake, Jonathan Marsh, and Chris Haddad.

Watch the video here, or go here, search for "Industry Interoperability" and choose your playback format.

Related news:  The video of our demo within the keynote has been extracted from the entire keynote and made available on youtube.  Here's the link!

Microsoft, Google, and others have just launched the Information Card Foundation, to promote awareness and interoperability around InfoCards.  This is great news - I think InfoCards have the potential to solve some real problems with managing identities and securing them against theft.  Despite the client-side technology being available in Vista for a year plus, we're still slow to see installations emerge in the marketplace.  The only use I make of InfoCard (and then still pretty rarely) is in the WSO2 Mashup Server, which has both InfoCard and OpenId support (though you can use an old fashioned username/password).

Managing identity and security without complicating the user experience is still a challenge.  I'm hoping the Information Card Foundation can encourage broadly useable solutions as it gets rolling.  And I'm hoping some large properties (PayPal is a member) may actually start putting stronger identity management technologies in place on a broad scale.

It should be no surprise with our products supporting InfoCards and OpenId that WSO2 is now a member of both the Information Card Foundation and the OpenID Foundation.

Had a great though brief visit from Paul, during which we toodled around the north end of Tamales Bay, bordering the Point Reyes National Seashore, the site of one of our successful backpacking trip a couple of years ago.  My primary goal, the Drake Estero, was closed for Harbor Seal pupping season, but we had good fun anyway.  Flickr set here.

Also uploaded and organized some existing shots from plane windows.  Looking out the window is by far the best part of flying!  Flickr set here.

Yes, you heard right.  A few minutes ago in Orlando at TechEd IT 2008, Bob Muglia's keynote included a demo of StockTrader 2.0, an SOA sample application consisting of a client application, a business process service layer, and an order processing service in order to place sample stock trades.  Gregory Leake of Microsoft showed the application, with each of the three components built in .NET 3.5, and then I came on stage, representing WSO2, and we swapped out the WPF smart client for a PHP application based on the WSO2 Web Services Framework for PHP.  Then we swapped out the back end order processing service for a Java version hosted in the WSO2 Web Services Application Server.  After each swap we placed a successful trade.

Watch the keynote here.

The demo featured the cross-platform interoperability between .NET, Java-based solutions, and unmanaged code solutions such as the PHP application.  The Web Services used were completely secured with message-level security (WS-Security), and everything of course worked quite seamlessly.

You can download the WSO2 StockTrader 2.0 application as well, including PHP versions of the business service and the order processing service.

The good news in putting this demo together is that the wire-level interop worked pretty spectacularly out-of-the-box, just as the demo promotes.  The actual interop between the three major development platforms in use today (CLR-based languages, JVM-based languages, and unmanaged code based at some level in C) is impressive, and while there is more work to do to complete and verify interop deeply across Security, Reliability, Transactions, and Policies, it really seems like the goal of making this stuff both universal and "just plumbing" is approaching pretty rapidly.

On another note - yesterday we were speculating backstage whether a keynote at a major Microsoft event had ever featured an Open Source partner on stage.  None of us could think of any off the top of our heads.  Can you?  Were we the first?

Update: 10PM.  Here's the press release.  And a news article, with a nice (and accurate!) quote from me. ;-)

I was pretty excited to try out Microsoft's new "personal cloud" computing initiative - Live Mesh.  I signed up right away and it looks great and offers some very neat features.  Yet I have one problem with it, and that's an unfortunate tie with Vista that makes one or the other virtually unusable.

Live Mesh, for some reason unknown to me, requires that User Account Control to be turned on.  Vista users will know User Account Control (UAC) as an annoyance that comes turned on by default.  It prevents unauthorized and potentially dangerous changes from being made without an Administrator's approval, which I concede might be beneficial in some circumstances (a classroom?) but is incredibly annoying if you are used to being the Administrator yourself.  In that case all it does it pop up a regular stream of annoying modal warnings, to which you become accustomed to clicking through without reading in about 10 minutes, thereby rendering any protection useless.

But it's worse, these warnings actually prevent you from doing useful work when you really want to.  Like renewing your IP address.  Or adding or removing files in the Program Files folder - if you're editing them through an IDE, you don't even get the warnings, skipping directly to failure.  You might have to reorganize your file system a bit to work around UAC.  And some stuff simply doesn't work.  I could not under any circumstances get Adobe Flash 9 to install for me under UAC.

You can see I'm an anti-fan of this non-feature of Vista, which promises security but provides no real benefit and quite a number of headaches.

I don't really understand why UAC would be required to run Live Mesh, but how important could it be if I can run Live Mesh on Windows XP, which has no such concept?  Is this just a UAC marketing?

In any case the effect is that a serious Vista user (the kind who might be an early adopter of Mesh) is significantly disadvantaged.  For me, the benefits of Live Mesh aren't worth the pain inflicted by UAC, and I'll generally restrict my Mesh use to non-Vista platforms, or extraordinary circumstances where it's worth doing a reboot to access the Mesh.  What a shame.

Listen to this, and then check out this.  Wow.

One of my mashups fell prey to the dreaded scrape rot - a complete overhaul of the target site that invalidated all of my scraping rules.  The pages in question are from globalincidendmap.com, which previously powered my internationalincident mashup (see Sri Lanka Incident Mashup).  The change was catastrophic - queryable content from the site is no longer free, but requires a paid membership and a login process.  One option would be to pay for a membership, but besides the steep price I doubt that the license terms allow republishing of the data.  I could support the mashup only for paid users of the service, collecting credentials and forwarding them on, but that again is both questionably secure (a user would have to trust that I didn't abuse the credentials temporarily in my possession), and unrealistic since few if any of my audience would spring for the cost of membership.  In effect, my mashup has been totaled.

This illustrates one reason why scraping should be used only as a last resort, when no more stable forms of content are available - feeds or Web services.  When you mix the content and presentation, changes in the presentation are easily confused with changes in the content.  Although the scraping features of the WSO2 Mashup Server are popular, I like to think of them as a stop-gap while publishers find cost-effective ways of serving up presentation-free content, such as delivering simple services using the Mashup Server ;-).  Ideally, more and more publishers will recognize the value of raw content, and the need for Web scraping will diminish.  Gonna take a while though...

Even without scraping, there remains one of the deep problems with mashups and distributed programming, that of services that disappear, are altered, change usage terms, etc., breaking their dependent mashups in the process.  There has been lots written on this, which can be generally summed up as "this is a hard problem."

One thing we plan to do in the future to make sure that service changes don't harm downstream dependents is use more of the advanced functionality of the WSO2 Registry upon which the WSO2 Mashup Server is built - namely, versioning.  Today, each time a service changes, an old copy is retained in the database, but no longer is "alive" as a service.  Some future version will have a simple interface for continuing to keep the old versions online, and help users to lock into one of these previous versions.  Some cool dependency management features on the drawing board for the Registry will also help find and record dependencies and notify dependents of changes.

But would these help in the case of the internationalincident service?  This is a case where there is a deliberate change which prevents "unauthorized" access.  The solution in this case was to mark the service as obsolete, and go out and find a whole new source of data.  The new srilankanincident service is a result - though the data is slightly different, perhaps a result of focusing narrowly on Colombo, it was a fairly short task to reprogram it, and even improve it, once I had found a new source of data.  The speed of fixing catastrophic failures is my current best hope against scrape rot.

I've got another free Webinar coming up - again an Introduction to the WSO2 Mashup Server and to mooshup.com on 13 May from 9-10AM PST.  Join me if you:

Seems the WSO2 crew has been blogging about WSO2 appearing on the "cool 5" companies in a recent Gartner report (paid subscribers only).  What intrigued them about the WSO2 Mashup Server was support for the hitherto paradoxical "lightweight but enterprise-oriented" services.

And here I am a couple of days late.  I guess for breaking news and the real skinny on "cool" you would do well to add the feeds of Paul, Sanjiva, Daniel, Glen, and Keith to your blogroll.

Just noticed this interview I gave at Mashup Camp posted on YouTube.  (Does everyone hate hearing themselves talk as much as I do?)

I'm giving a free Webinar on 15 April 2008 9-10AM PDT about the approach we took to service composition in the WSO2 SOA Platform.  Instead of a declarative approach which my XSLT days showed can be powerful yet also limiting in many ways compared with a full programming language, the WSO2 Mashup Server uses a "scriptable Web Services" metaphor, and supports the ability to consume and produce Web Services using simple Javascript expressions.  Add to that the ability to script non-Web Service materials like Web pages and feeds, and you've got a powerful yet accessible platform for creating Web Service mashups.  Register now!