I'm proud to say that v1.5 of the WSO2 Mashup Server is now available for download!  This release has been a few months in the making, and I'm thrilled to see a major update to the Mashup Server finally out.  News release here. 

Thanks to the efforts of Keith, Tyrell, Channa, Yumani, and many others at WSO2, we've been able to put some snazzy new features in this release:

• Integrated award-winning Data Services: quickly and with zero-coding map queries on your databases, Excel spreadsheets, or CSV files into Web services.
• Google Gadgets: power a Google gadget with a mashup service - we generate a try-it, a code template for customizing the gadget, and even a build in Gadget dashboard based on Apache Shindig.
• Security, security, security: Limit access to your mashup service to only authorized users, encrypt and sign your messages, and access services which are secured.  I think our security interface is the easiest way to engage advanced WS-Security features that's out there!
• OpenID login support.

There's lots of little improvements too, and I hope to talk about those, as well as the major features above, in upcoming posts.

For now though, a hearty congratulations and a good night's sleep to the Mashup Server team!

Yumani has published a nice tutorial on Sending Instant Messages via Mashup Server showing how to use the instant messaging capabilities built into the WSO2 Mashup Server.  I think the ability to have instant alerts as part of your mashup is a cool feature, and as Yumani shows in this article, they are pretty trivial to do.

I'm looking forward to adding the next step in a future release - the ability to invoke a mashup using instant messaging.  Then we'll have a scriptable platform for building agents that can communicate via IM.  Imagine converting your favorite service to an IM conversational interface.  Or making a data source available through an IM conversation.  Or even imagine new games...  Endless possibilities!

Just posted a few snapshots from a Saturday day hike to Loch Leven Lakes, a wonderful and fairly accessible bit of the High Sierras.  Haven't been there in a while, despite it being a fantastic place for some bouldering with the kids.  It was good to get above California's persistent smoky haze for a few hours (though later in the afternoon even the high country took on a smoky hue.)

Wishing everyone a last minute happy 4th of July, and for those of you who missed fireworks this year, here's a couple of photos to hold you till next year.  Full Flickr set here.

Microsoft TechEd Online has posted a video taped in Orlando after our successful keynote appearance.  This video captures a panel discussion that I participated in - I'll just quote the blurb:

This Panel discussion centers around the use of standards to facilitate interoperability between different technologies. The participants have all implemented their own version of a reference application and speak about any challenges they faced in implementing the application. The main discussion points of this Panel include: interoperability through SOAP; WS-* standards and how they facilitate interoperability; common problems with interoperability; identity and security in a distributed application; and future directions/remaining challenges in interoperability. With Raghu Thiagarajan, Gerald Beuchelt, Gregory Leake, Jonathan Marsh, and Chris Haddad.

Watch the video here, or go here, search for "Industry Interoperability" and choose your playback format.

Related news:  The video of our demo within the keynote has been extracted from the entire keynote and made available on youtube.  Here's the link!

Microsoft, Google, and others have just launched the Information Card Foundation, to promote awareness and interoperability around InfoCards.  This is great news - I think InfoCards have the potential to solve some real problems with managing identities and securing them against theft.  Despite the client-side technology being available in Vista for a year plus, we're still slow to see installations emerge in the marketplace.  The only use I make of InfoCard (and then still pretty rarely) is in the WSO2 Mashup Server, which has both InfoCard and OpenId support (though you can use an old fashioned username/password).

Managing identity and security without complicating the user experience is still a challenge.  I'm hoping the Information Card Foundation can encourage broadly useable solutions as it gets rolling.  And I'm hoping some large properties (PayPal is a member) may actually start putting stronger identity management technologies in place on a broad scale.

It should be no surprise with our products supporting InfoCards and OpenId that WSO2 is now a member of both the Information Card Foundation and the OpenID Foundation.

Had a great though brief visit from Paul, during which we toodled around the north end of Tamales Bay, bordering the Point Reyes National Seashore, the site of one of our successful backpacking trip a couple of years ago.  My primary goal, the Drake Estero, was closed for Harbor Seal pupping season, but we had good fun anyway.  Flickr set here.

Also uploaded and organized some existing shots from plane windows.  Looking out the window is by far the best part of flying!  Flickr set here.

Yes, you heard right.  A few minutes ago in Orlando at TechEd IT 2008, Bob Muglia's keynote included a demo of StockTrader 2.0, an SOA sample application consisting of a client application, a business process service layer, and an order processing service in order to place sample stock trades.  Gregory Leake of Microsoft showed the application, with each of the three components built in .NET 3.5, and then I came on stage, representing WSO2, and we swapped out the WPF smart client for a PHP application based on the WSO2 Web Services Framework for PHP.  Then we swapped out the back end order processing service for a Java version hosted in the WSO2 Web Services Application Server.  After each swap we placed a successful trade.

Watch the keynote here.

The demo featured the cross-platform interoperability between .NET, Java-based solutions, and unmanaged code solutions such as the PHP application.  The Web Services used were completely secured with message-level security (WS-Security), and everything of course worked quite seamlessly.

You can download the WSO2 StockTrader 2.0 application as well, including PHP versions of the business service and the order processing service.

The good news in putting this demo together is that the wire-level interop worked pretty spectacularly out-of-the-box, just as the demo promotes.  The actual interop between the three major development platforms in use today (CLR-based languages, JVM-based languages, and unmanaged code based at some level in C) is impressive, and while there is more work to do to complete and verify interop deeply across Security, Reliability, Transactions, and Policies, it really seems like the goal of making this stuff both universal and "just plumbing" is approaching pretty rapidly.

On another note - yesterday we were speculating backstage whether a keynote at a major Microsoft event had ever featured an Open Source partner on stage.  None of us could think of any off the top of our heads.  Can you?  Were we the first?

Update: 10PM.  Here's the press release.  And a news article, with a nice (and accurate!) quote from me. ;-)

I was pretty excited to try out Microsoft's new "personal cloud" computing initiative - Live Mesh.  I signed up right away and it looks great and offers some very neat features.  Yet I have one problem with it, and that's an unfortunate tie with Vista that makes one or the other virtually unusable.

Live Mesh, for some reason unknown to me, requires that User Account Control to be turned on.  Vista users will know User Account Control (UAC) as an annoyance that comes turned on by default.  It prevents unauthorized and potentially dangerous changes from being made without an Administrator's approval, which I concede might be beneficial in some circumstances (a classroom?) but is incredibly annoying if you are used to being the Administrator yourself.  In that case all it does it pop up a regular stream of annoying modal warnings, to which you become accustomed to clicking through without reading in about 10 minutes, thereby rendering any protection useless.

But it's worse, these warnings actually prevent you from doing useful work when you really want to.  Like renewing your IP address.  Or adding or removing files in the Program Files folder - if you're editing them through an IDE, you don't even get the warnings, skipping directly to failure.  You might have to reorganize your file system a bit to work around UAC.  And some stuff simply doesn't work.  I could not under any circumstances get Adobe Flash 9 to install for me under UAC.

You can see I'm an anti-fan of this non-feature of Vista, which promises security but provides no real benefit and quite a number of headaches.

I don't really understand why UAC would be required to run Live Mesh, but how important could it be if I can run Live Mesh on Windows XP, which has no such concept?  Is this just a UAC marketing?

In any case the effect is that a serious Vista user (the kind who might be an early adopter of Mesh) is significantly disadvantaged.  For me, the benefits of Live Mesh aren't worth the pain inflicted by UAC, and I'll generally restrict my Mesh use to non-Vista platforms, or extraordinary circumstances where it's worth doing a reboot to access the Mesh.  What a shame.

Listen to this, and then check out this.  Wow.

One of my mashups fell prey to the dreaded scrape rot - a complete overhaul of the target site that invalidated all of my scraping rules.  The pages in question are from globalincidendmap.com, which previously powered my internationalincident mashup (see Sri Lanka Incident Mashup).  The change was catastrophic - queryable content from the site is no longer free, but requires a paid membership and a login process.  One option would be to pay for a membership, but besides the steep price I doubt that the license terms allow republishing of the data.  I could support the mashup only for paid users of the service, collecting credentials and forwarding them on, but that again is both questionably secure (a user would have to trust that I didn't abuse the credentials temporarily in my possession), and unrealistic since few if any of my audience would spring for the cost of membership.  In effect, my mashup has been totaled.

This illustrates one reason why scraping should be used only as a last resort, when no more stable forms of content are available - feeds or Web services.  When you mix the content and presentation, changes in the presentation are easily confused with changes in the content.  Although the scraping features of the WSO2 Mashup Server are popular, I like to think of them as a stop-gap while publishers find cost-effective ways of serving up presentation-free content, such as delivering simple services using the Mashup Server ;-).  Ideally, more and more publishers will recognize the value of raw content, and the need for Web scraping will diminish.  Gonna take a while though...

Even without scraping, there remains one of the deep problems with mashups and distributed programming, that of services that disappear, are altered, change usage terms, etc., breaking their dependent mashups in the process.  There has been lots written on this, which can be generally summed up as "this is a hard problem."

One thing we plan to do in the future to make sure that service changes don't harm downstream dependents is use more of the advanced functionality of the WSO2 Registry upon which the WSO2 Mashup Server is built - namely, versioning.  Today, each time a service changes, an old copy is retained in the database, but no longer is "alive" as a service.  Some future version will have a simple interface for continuing to keep the old versions online, and help users to lock into one of these previous versions.  Some cool dependency management features on the drawing board for the Registry will also help find and record dependencies and notify dependents of changes.

But would these help in the case of the internationalincident service?  This is a case where there is a deliberate change which prevents "unauthorized" access.  The solution in this case was to mark the service as obsolete, and go out and find a whole new source of data.  The new srilankanincident service is a result - though the data is slightly different, perhaps a result of focusing narrowly on Colombo, it was a fairly short task to reprogram it, and even improve it, once I had found a new source of data.  The speed of fixing catastrophic failures is my current best hope against scrape rot.

I've got another free Webinar coming up - again an Introduction to the WSO2 Mashup Server and to mooshup.com on 13 May from 9-10AM PST.  Join me if you:

Seems the WSO2 crew has been blogging about WSO2 appearing on the "cool 5" companies in a recent Gartner report (paid subscribers only).  What intrigued them about the WSO2 Mashup Server was support for the hitherto paradoxical "lightweight but enterprise-oriented" services.

And here I am a couple of days late.  I guess for breaking news and the real skinny on "cool" you would do well to add the feeds of Paul, Sanjiva, Daniel, Glen, and Keith to your blogroll.

Just noticed this interview I gave at Mashup Camp posted on YouTube.  (Does everyone hate hearing themselves talk as much as I do?)

I'm giving a free Webinar on 15 April 2008 9-10AM PDT about the approach we took to service composition in the WSO2 SOA Platform.  Instead of a declarative approach which my XSLT days showed can be powerful yet also limiting in many ways compared with a full programming language, the WSO2 Mashup Server uses a "scriptable Web Services" metaphor, and supports the ability to consume and produce Web Services using simple Javascript expressions.  Add to that the ability to script non-Web Service materials like Web pages and feeds, and you've got a powerful yet accessible platform for creating Web Service mashups.  Register now!

I just posted a finished version of a Mashup designed to help answer the question "is Sri Lanka getting safer or not?"  This is a question we on the global WSO2 team ask each time we arrange travel to that unfortunately troubled country.  Despite a spate of violence early this year, designed to coincide with the formal dissolution of the cease fire that has done little to prevent violence, it seemed to me things were getting a little better.  But I needed some facts to back that up.

The mashup plots bombings and other incidents as a bar chart, measuring the severity of the incident (how many killed & wounded) over the last 18 months.  The idea was to see if there was a clear overall trend in the violence or not, something not readily apparent from a Google map (see right).

The mashup service itself consists of several items, each one a simple task accomplished in half a page of javascript:

• Scrape the search results page at globalincidentmap.com to pull the essential data out of an HTML table for the country of interest, and put it into a simple XML structure.
• Cache the page if it has already been accessed within 24 hours (scraping is expensive, the first access in any 24 hour period will be pretty slow.)
• Parse the headline for patterns such as "eight killed" or "injures 7" and turn that into killed and wounded digits (this isn't perfect, but we can tolerate a few errors since we're trying to present an overall sense of the problem rather than perfectly accurate statistics.)  Also filter out as much as possible the killing of LTTE, as that's more a measure of war than of terrorism.
• Provide a helper method to look up details and get a link to a new story for any item.

Using this service (called internationalindicent) I created a custom HTML UI to format Sri Lanka-specific results into the bar chart and to make it interactive (click on a bar to see more info about the incident.)  Then I used "share this mashup" to upload the service to mooshup.com so others could try it out (or copy the code.)

The rough version took a couple of hours, mostly figuring out the details of scraping the page and coming up with the headline patterns to look for, but then I spent a couple more polishing the HTML UI so I wouldn't be embarrassed to share it.  In the process I demonstrated some of the powerful aspects of using mashup technologies in your development arsenal:

• Instead of investigating the current situation using Google News for 30-60 minutes each time we plan a trip for the latest information, I can browse this site in a few minutes, see the trend, and get details of any recent incidents I'm interested in.  This will pay for itself in terms of my own personal productivity before long.
• There is also a small user group (really, just the handful of WSO2 employees based outside Sri Lanka) who can also benefit from this micro-application, increasing their productivity as well.
• The WSO2 Mashup Server makes the data available as a service, so others can reuse it too, for alternate displays or to generate displays for other parts of the world.
• And, it's just fun!

So ... is Sri Lanka getting safer?  I'll have to let you be the judge of that.  Go to http://mooshup.com/services/jonathan/internationalincident/ to see for yourself!

[Update 5/27: This service is no longer available.  See http://auburnmarshes.spaces.live.com/blog/cns!F985A6952BC07C4!813.entry for more info.]

Just returned from some travel which included 2 days of Mashup University and a day of Mashup Camp.  A few thoughts:

• Maps are still well represented.  This surprised me a bit, but I recognize I've been well immersed in the WSO2 Mashup Server which supports a wide variety of User Interactions (html pages including maps, but also feeds, email, instant messaging.)  I predict non-map mashups will start to eat into the map-based mashup market share dominance over the next year.
• There's lots of interest in consuming various Web APIs.  Some vendors were promoting their APIs, others tools for consuming those APIs.  I think the WSO2 Mashup Server can tap into an underserved market here, since it's the easiest way I know of to deliver a comprehensive Web API on top of a bit of Javascript logic, which in turn can front information sources as diverse as databases and scraped web pages.  I think the Mashup Server can become a "design your own API" tool that can have great appeal to the mashup developer.
• A lot of Javascript was in evidence, further validating our choice to use Javascript for mashup logic in the WSO2 Mashup Server.
• For the first time I ran into a few people with serious interest in WADL.  Has its time finally come?  More on that in a subsequent post.
• The "unconference" style was quite interesting and successful, especially if you're like me and are more interesting in connecting with interesting individuals than in hearing yet another vendor pitch (mine excepted of course!) ;-)  One thing that is sorely lacking is any kind of organization to the conference Wiki, and a surprising lack of ability to record much of the activity there.  I couldn't even find who won the Best Mashup contest...

I can't wait till next year!

Paul Downey poses some interesting considerations for what a mashup consists of.  I think I'd list a pretty different set of criteria, but for now I'll just start by comparing a mashup hosted by the WSO2 Mashup Server against his list to see how we stack up:

• Zero touch: +1.  Each mashup is accessible, documented, try-able, and so forth right out of the box.  The admin console makes them discoverable, and if you have a user account (email-validated guests are supported too) you can tag, rate, comment a mashup to build a community around them to make any touch beyond "zero" positive and reciprocal.
• Safe: +.5.  The Mashup Server supports and encourages safe interactions, but it does not prevent users from hanging themselves if they so choose.  That is, I can have an operation exposed through GET, explicitly marked as safe by the author even, that has seriously detrimental side effects.
• Cool URIs: +1.  Each mashup has neat URIs for the endpoints, for the metadata and tooling associated with it, for the admin capabilities associated with it, and even for accessing the operations over HTTP.
• Open data formats: +1.  We're partial to XML and use Web Services under the hood, but also provide bridges to HTML, JSON, Feeds, etc.
• Eschew RIA: +1.  By default we don't provide any rich interface.
• HTML form access: +1.  By default we do provide an HTML try-it form, and stubs to quickly build your own HTML pages.
• Accessible URIs: +1.  All endpoints are available through both http and https by default.  Mashups can be easily migrated outside local contexts so their visibility is greater.  E.g. mooshup.com.
• SOAP/WSDL: -1.  Everything we do has rich metadata and SOAP 1.1 and SOAP 1.2 bindings along with the REST/POX binding, and we also make it easy to consume web services in these formats.  However, you don't have to know anything about these formats to write a successful mashup - they're just valuable parts of the plumbing.  So maybe a -1 is too stingy.
• Authentication: +0.  We support username/password and Infocard to access the Mashup admin site, but there isn't a drop-dead simple way to restrict access to the service based on these controls.  Not too hard to provision higher levels of security using WS-Security, but I'm not sure Paul would agree that's sufficient.
• Scratch your itch: +1.  Subjective, but the whole product is designed to serve the needs of individual developers to hack up services as easy as they can hack up a simple web page.
• Fun: +2!!  As long as you're willing to hack a bit of Javascript.

So I think the Mashup Server stacks up pretty well using Paul's criteria, which I think can be summed up as "has a nice web interface" and "fun and easy".  But some of Paul's criteria don't fit with that summary and those are unsurprisingly the ones I take some issue with:

RIA: I don't see any reason why a rich interface to a mashup is always bad.  Only when it locks up the data in a way that's impractical to reuse.  In our model, we support the separation of content and presentation in order to lift the limits on the kinds of presentation environments the user prefers.  A single mashup can (and ideally should) support as many presentation media as are appropriate, including simple web pages, RIAs, feeds, notifications and messages, portlets, widgets, whatever.  If an RIA "scratches my itch" then what's wrong with it?

For example, my iPod Touch comes with a YouTube app, which is an RIA for the YouTube site optimized to the screen limitations of the product.  I can also point Safari right at YouTube, but the optimized version actually is easier for simple access.  Is this bad?  It really depends on what the consumer wants.  I agree dead ending in an RIA may be inappropriate for some users of that service, but providing an RIA front end to a clean and well-documented interface that can be repurposed in other ways seems like a good user-centered feature that even Paul would support. I think he probably meant this item to mean "Does the site rely solely on so-called 'rich user experience' technologies in a way that precludes data from being accessed though a nice web interface?"

SOAP/WSDL: I understand why Paul would add this to the list.  Up until the Mashup Server Web Services were just too hard to consume to allow them to stand on a list with "scratch your itch" and "fun".  However I think we've turned the corner on that.  A Mashup Server author rarely has to even consider whether SOAP is involved in delivering a Web Service - we can take care of that for them.  Right now it's simply an alternative to the REST interface.

As for WSDL, in the Mashup Server it strongly supports the other goals of zero touch, cool URIs, and open data formats.  I'm pretty tired of doing one-off hand coding to access REST sites or sucking up their hand-crafted and varying-quality stubs.  WSDL has a big role to play in simplifying these interactions for the developer, in a way that I think Paul would appreciate.  It's a primary artifact in providing the "nice web interface" that we all agree is invaluable.

Anyway, thanks Paul for a provocative post!

Bro Jason and I again will improvise around a few sets at PachaMama's Organic Cafe (map) this Friday, February 15 7:00 – 9:00 PM.  Drop in for a listen!

I'm giving the first of a regular series of free Webinars to introduce people to the WSO2 Mashup Server and to mooshup.com on 12 Feb from 9-10AM PST.   Register now!

I'll also be talking about mashups at next Monday's Web Services on Wall Street conference, starting with the opening panel: "Enterprise Mashups For Wall Street – Leveraging SOA and Web 2.0".  If you're there, come say hi!