Design by Committee

Keith upgraded mooshup.com to the 1.5.1 release.  I used to find it pretty sluggish, but no longer!  Try it out...

Thanks for Tyrell for finally locating the source of the elusive Java heap space error, which I came across sporadically during my daily Mashup Server stress testing.  Tyrell describes the fix that you can add to your 1.5.1 installation here.

Now that I've had the opportunity to observe the Open Source industry from the inside for a couple of years, I find myself musing a fair bit about the economics - broadly speaking the exchange of value - involved.  Much of the customer appeal for open source solutions comes from the sticker price, which is generally zero.  But as the adage goes, if something sounds too good to be true, it generally is.  If open source software is to be high quality and broadly applicable, the customer demand for low-cost software needs to be matched with incentives for vendors and individuals to continue to produce high-quality software.  When the transaction isn't based on actual currency, what is the commodity that makes this a successful transaction for both sides?

For individual open source authors the incentives might include the joy of having a large and loyal user base.  It might include fame and the development of skills that lead to greater personal satisfaction and a more impressive and marketable resume.  But I'd like to focus here on the incentives for professional software development businesses to invest in producing more and better open source software.

The research, development and deployment of software is a complex and costly process.  How are open source vendors able to accomplish these goals without getting licensing revenue in return?  I believe there are many ways a vendor can receive value from a user besides an exchange of currency.  Although the commodities exchanged aren't tangible the exchange is rarely a zero-sum game, and can enrich the supplier without depriving or depleting the customer.  No matter how many smiles you give, you never run out.

Enforcement of non-tangible exchanges is impossible, and thus the exchange relies on the good will of the customer to give back.  My intention in this post is to enumerate some of the ways this non-monetary economy works, and I hope to encourage users to participate more fully and consciously in holding up their end of the transaction, and thus to perpetuate a virtuous cycle of open source software development.  If you use open source software, please consider one of the following ways you can remunerate the creator.

The laws of economics state that the more rewards there are for a product or service, the more of that product and service will be produced.  By increasing the rewards for vendors to create useful and high-quality open source software, you encourage more of that software in the future.  Isn't that worth an investment of a little time?  It doesn't even lighten your wallet!

As I posted back in 2005:

The US automakers don't seem tapped into [the fuel economy] trend at all.  They still seem to think circumventing mileage minimums by pumping out SUVs is the way to sustainable revenues.  Last week Ford and GM were put on notice that they were wrong.  At least the blue half of this country, and I suspect lots of export markets, are willing to invest their automobile acquisition budget in a choice that reduces pump costs, unsightly and unhealthy smog, and reduces our dependence on foreign oil, and maybe even get a bit of value appreciation while they're at it.  They're even more motivated to vote with their dollars since their election votes haven't provided much of a visible return.  Yet despite plenty of urging by the environmental community, Ford and GM seem to have ignored the inevitabilities of the long-term.  More and more of those purchasing dollars will head straight to Japan.  I suspect the next 15 years could be pretty rough as our automobile designers adapt.

High fuel prices have accelerated the timeline beyond what I had imagined, and Prius sales have accordingly boomed. 

I'm proud to at last announce I've joined those voting with their wallets for fuel efficiency and low emissions (and unfortunately against our domestic automakers) with my new purchase!

Let's hope American competitiveness is up to the challenge, and hope that they do a much better job of recognizing and capitalizing on long-term trends in the future.

Today I'm launching my new musical podcast of short piano or keyboard improvisations.  I call it Spontaneous Reflections, and am planning to update it approximately weekly.  It's available free in the iTunes store or you can subscribe directly here.

For those of you who don't know, I started my career in music school and playing keyboard in the showrooms of Reno, but found the vocational aspects much less exciting than I hoped, and switched over to a more intellectually and financially rewarding career in computer software.

After a long period of playing primarily pipe organ, a couple of years ago I returned to piano and keyboard and though I don't have my 20-year old chops (and pipe organ tends to ruin your sense of rhythm) I am exploring some new directions in my improvisation, really trying to dig into the instrument as a percussion instrument.

Live music is a funny thing though - each note goes out into the Ether and dissolves, gone forever except as ghosts in memory.  And I find that playing is quite a different experience than listening, heavily colored with the subjective.  To grow further I thought it valuable to be able to listen to some of my improvisations and see what works and what doesn't, and to help me practice more discipline about the overall shape of a piece.  This podcast thus is an audio journal, not polished or refined, but genuinely spontaneous.

I hope you enjoy it!

Can't help but repeat a few choice bits from a recent article Enterprise Mashups Part II: Why SOA Architects Should Care by Chris Warner and John Crupi.  Too bad I didn't see this before the Enterprise Capabilities of the WSO2 Mashup Server 1.5 webinar I gave this morning - I could have cribbed a few bits of wisdom! Like these:

Mashups bring a “user” into the SOA mix. ... By having the business build mashups upon the foundation of your service-oriented architecture, they essentially become SOA champions without knowing it.

This is a point we tried to hit strongly in the WSO2 Mashup Server, which can help bring individuals, teams, or enterprises together into a community around services, and strengthen the concept of a "user" and help build a diverse set of user experiences tied to services.  Until the SOA platform gets connected to the business users, the promises of SOA (business agility) won't be fully realized.

In addition to SOA-friendly formats, such as RSS and Atom plus REST and SOAP, mashup creators can publish mashups to spreadsheets, as WSRP-compliant portlets, wiki- and blog-friendly widgets, or even into a mobile phone as a micro-application. Mashups can become the vehicle through which services become part of the everyday tools of the enterprise business user.

Which is why we've added on to the RSS, Atom, REST, and SOAP support in 1.0 the ability to interact with spreadsheets and databases (through Data Services), and are starting our foray into the world of portlets, widgets, and gadgets with support for Google gadgets.

Mashups can go well beyond leveraging an SOA by becoming part of that SOA, allowing developers to create customized “service skins” from core services. ...

Because mashups can be exposed as REST-, WSDL- and JSON-based services, they look and feel like a real SOA-based service to developers who want to consume them. ... The mashup, created by the developer, becomes the tailored service which is directly aligned with their particular need. Major enhancements to core services can be accommodated with a reformulated or updated mashup by the mashup creators themselves.

Yes, we've found our users (and ourselves) using the WSO2 Mashup Server for more than just combining services into a new one, but for rapid development of new services, or for customizing services to a particular scenario.  From the outside, these don't just look like real services - they are real services - for example in the 1.5 release high levels of security can be applied to the services even though they are so easy to write.  A good example of a "skinned" service - the digit2image sample service that comes with the mashup server tailors a very general purpose and complicated service (Flickr) for a particular narrow purpose - to find a random image, with appropriate copyright, for a single numeric digit.  By providing a narrow API, this capability can be exposed more easily and efficiently (some caching is performed by the service to speed up returns).  And it allows innovation without touching the original Flickr service which we don't have control over.  For instance, I could search beyond Flickr for appropriate images without breaking the service contract or rewriting the Flickr service.

Toward the end, the authors write:

At this point any SOA architect worth his WSDL should see that mashups can greatly enhance an SOA and don’t necessarily ignore or break the principles that made your SOA great. But governance, granularity and scope for mashups do require subtle tweaks and adjustments to your enterprise toolset.

Indeed, governance is important as your SOA grows and chaos theory has a chance to get a toehold.  The WSO2 Mashup Server is taking the first steps into governance by relying on the WSO2 Registry to store mashups in a versioned repository, manage ownership, users and roles, and facilitate communication and the building of trust in an Enterprise SOA community.  But we still have to to a better job of working with the Registry's features for supporting multiple versions, product lifecycles, and dependency management.  For instance, while the Mashup Server stores all it's mashups, comments, tags, ratings, etc. in the Registry, when you deploy a new mashup, it doesn't automatically populate the registry with the WSDL.  We're looking at much better integration with the Registry as a primary feature of our next Mashup Server release.

Kudos to the authors for clearly presenting not just the case for service mashups in an SOA architecture, but why any SOA architecture that doesn't expand into the mashup space is missing out on huge opportunities.

As announced on the mooshup blog, the WSO2 Mashup Server team completed a new release, which includes 50 bug fixes.  Most of these are small, but a few are worthy of note:

• Issues with Email host object in v1.5 fixed.
• Private key management features.
• Fixes in the SMTP and JMS transports (Keith is working on an article about how to enable them.)
• WSRequest host object now supports a fire-and-forget message exchange pattern.
• Supporting http proxies more consistently (e.g. in the Feed object).
• Lots of little UI nits and polish.

I'm very pleased with this release, as it is often the case that some of the fit and finish work gets postponed in the last stages of a major release like 1.5.  Open source is about rapid and continual improvement, and this release fits that bill nicely!

Download it today!

As announced here:

Title: Enterprise Capabilities in the WSO2 Mashup Server 1.5
Date: August 12, 2008
Start Time: 9.00 am PST
Duration: 60 min
Presenter: Jonathan Marsh

The WSO2 Mashup Server provides a powerful platform for developing and deploying mashups. In this Webinar, we'll provide a brief introduction to the Mashup Server architecture and Javascript-based development model, as well as highlight the new features in the 1.5 release that support enterprise deployments. These include support for message level security (authentication, encryption, signing) on mashup services, and the ability to expose a data source such as a relational table, an Excel spreadsheet, or a simple CSV file as a zero-coding Data Service.

Attend this Webinar if you:

• You have services, web pages, or other information sources available but you want smarter ways to use those services.
• You want to expose or compose services in a simple yet powerful way, complete with enterprise-level security.
• You are an existing WSO2 Mashup Server user and would like an overview of the new features in the 1.5 release.

Your presenter, Jonathan March, is the Director of Architecture for Mashup Technologies, and led the design and development of the WSO2 Mashup Server. Jonathan, a product designer and devoted script hacker, has spent over ten years developing and standardizing, XML and Web service technologies, most recently at Microsoft where he contributed to the standardization of XML, XSLT, XPath, WSDL, and other technologies.

Register now!

1) Project Auburn, a celebration of civic pride and elbow grease, in addition to renovating the classic State Theatre, has renovated the rear garden of OLAS, making it a great site for sculpture, relaxing, and entertainment.  I helped put the fence up and did some of the repainting a few weeks ago when hundreds of volunteers turned out to make Auburn a better place.  Kudos to the Rotary Club and other civic organizations for making our town a better place!

2) The Auburn Art Walk, a recurring evening of open studios and galleries, occurs every Second Thursday throughout the summer.

3) OLAS is celebrating the new landscaping, in conjunction with the Art Walk, with a show entitled From Earth To Sky, August 14th from 6:00 to 9:00 PM.

4) I acquired a new keyboard, a Yamaha CP300, at long last giving me some mobility to my music.  (And making it easier to get quality recordings, more about which later.)

The outcome of all these fortuitous circumstances?

Jason and I will be playing our usual eclectic mix of improvisatory world/new age/jazz/undefinable music for the event.

If you're in the neighborhood, come on by!

What is XSS and why is it restricted?

[Note: also published on the Mooshup.com blog.]

Disclaimer: I'm not a security guru, so what follows is my opinion, observation, and experience.  Please feel free to comment and correct!

Modern browsers protect against release of private information to a third (possibly malicious) party by imposing “cross-site scripting” or “XSS” restrictions. The basic attack is that a browser is pointed to a web site that is trusted (to some degree) by the user. In using the web site the user may provide some confidential information to that web site, such as a password, bank account number, or some other information that could be exploited by a malicious entity. Naturally, the user would hesitate to provide this information to a site that he doesn’t trust (the user can be fooled through a set of techniques known as phishing, but that’s a different story.) Here we assume that the user has gone to a site that is authentic.

However, it is possible that even a trustworthy site, through poor construction or through compromised delivery mechanisms, could be “hacked” by a third party. For instance, accessing the site through an open (but malicious) wireless network may allow the page to be subtly changed during transmission. This change might be to insert a bit of script code that records the interactions the user has with the page, including information he enters such as a password, and also information that is provided by the website to the user. The inserted script could collect this private information, and then "phone home" to the attacker.  HTTPS can mitigate such attacks by securing the communication channel, but interactions with plain HTTP sites may still disclose user secrets of various levels of sensitivity.

Attacks can also come, and generally do come, over a trusted internet connection, even possibly through HTTPS. Anytime user-generated content appears in a page (e.g. comments on a blog, etc.) there is a possibility that third party, and thus untrustworthy, content is piggybacked on a trusted site.  Plain text third-party content is benign (what you see is what you get), but if the content can be submitted in html, it is possible that such html can include malicious scripts. For this reason, a trustworthy site that allows user-generated content must scrub any user-generated content provided to it, removing anything that could be executed as script.

From the perspective of browser vendors there are a lot of sites out there and not all of them consider the security implications of user-generated content adequately. To help protect the user, XSS counter-measures in the browser attempt to limit the ability of and scripts within the page to “phone home”. The is accomplished by preventing HTTP POST (the protocol used to submit forms and upload data) access to any web site domain called on by the page other than the one from which the main page originated. For instance, a page from http://wso2.org can access “safe” content from alternate domains like http://wso2.com, including images, stylesheets, even script libraries. The page however won’t be able to post a form containing user input to anywhere but http://wso2.org. The XMLHttp object provides a way to POST from script, and also prevents information from being posted to any domain other than the page domain.

So as a user, having a browser watch out for these types of attack and prevent them seems useful.  But let's consider situations where they get in the way of useful, trustworthy work.

Why a developer might want to access alternate domains

For a web developer (especially a mashup developer) XSS can be quite a pain, as it limits your ability to write a page that spans domains.  It limits your ability to host AJAX and Web Service interactions (powered by the XMLHttp object) anywhere other than your primary domain.  For instance, you can't host a Web service on mooshup.com and use it within your own application (at least directly from the browser).  Even though both sites may be trusted by you as the web site author, the browser enforces a blanket restriction on this access. (Each browser has mechanisms that may loosen this in some circumstances, but there isn’t anything with zero-config or cross-browser.)

This restriction limits applications such as gadget pages (e.g. iGoogle.com) that aggregate information from a large number of sources.  The Google Gadget framework, for instance, provides a way to GET information through a proxy on the trusted server, but currently disallows similar capabilities for POSTing.

The Loophole - Script Injection

Don’t start feeling too secure as a user, or too disappointed as a developer trying to do legitimate work – there are some loopholes that can be exploited.

As described above, an HTTP GET operation is assumed to be safe across domains, while HTTP POST is not.  If one could masquerade a POST as a GET, one could circumvent security restrictions.  In particular, script can be fetched regardless of domain.  This powers important functionality, such as third party libraries, an important feature supporting simplified development, analytics, and advertising.  Basically one needs to translate the body of the POST into url parameters on the GET (recognizing there are length and encoding issues to deal with), insert a <SCRIPT> tag dynamically into the page which uses the GET, and the server on the external domain can access the "posted" information.  It can even send a response back in the form of a block of script (essentially a callback).  Of course, you need to insert script into the page initially to get the ball rolling - which can be pretty difficult over a secure connection or for sites that properly sanitize user-generated content.  But if you're the owner of the original site, it's not terribly difficult once the technique has been mastered.

Naturally, a user can protect themselves reliably against these attacks by turning off Javascript in the browser.  Or cutting the internet wires.  Or burying the computer in the backyard and raising carrier pigeons.  All quite practical alternatives, don't you think?

WSO2 Mashup Server 1.5

The new WSRequest.js in the new WSO2 Mashup Server 1.5 release has facilities to “exploit” this loophole when the developer wants to access a mashup from a page within another domain.  If an "access denied" error is returned from the XMLHttp object, a script injection is attempted instead.  This allows you to use the Mashup Server's convenient stubs within a page or a gadget without encountering the XSS pain.  There are some restrictions in the fine print of course - only asynchronous calls are allowed, message size is limited, and the wire-level messages no longer are conformant to open standards for Web services, but those aren't unreasonable considering the alternatives.

Conclusion: time to drop XSS restrictions?

So, my question is, if XSS restrictions are so painful, yet circumvented with a modest bit of work (hey I'm no genius at this stuff and I did it) why are the XSS restrictions in place at all?  Instead of trading off convenience for security, you're imposing convenience without actually making a meaningful contribution to the user's security.  The additional level of security provided by making cross domain access simply obscure rather than truly prohibited doesn't seem worth it.  Is it time to dump XSS restrictions?  Or do we need to add a new (and further inconvenient) restriction against inserting <SCRIPT> tags into a page dynamically?  As long as there is any cross-domain access I don't think I'll be completely secure.  And that rules out advertisement insertions which I don't think is going to happen anytime soon!

Some of our Sri Lankan visitors wanted to see a bit more of the Silicon Valley than the inside of our conference rooms where we've been working all week.  Follows is a little driving tour I took them on to see some of the local sights, both large and small.

1. WSO2 Mountain View office:  Aspiring to someday be part of Silicon Valley lore, a fitting place to start when looking in the rear view mirror of history. http://wso2.com
2. Fairchild Semiconductor: After the invention of the IC at Shockley Laboratories, mass defections spurred the development of the integrated circuit by the eventual founders of Intel Corporation. http://en.wikipedia.org/wiki/Fairchild_Semiconductor
3. The GooglePlex: Formerly the site of Silicon Graphics campus, Google is the current symbol of Silicon Valley entrepreneurial success.  http://en.wikipedia.org/wiki/Google_Campus
4. The Computer History Museum:  Ironically located in the former Silicon Graphics Executive Offices building, vacated not long after occupation - a poster child for the valley's perpetual boom-bust cycle.  http://en.wikipedia.org/wiki/Computer_History_Museum
5. Apple Computer:  Famous for bringing computer technology to the general public through a continuing emphasis on usability.  http://en.wikipedia.org/wiki/Apple_computer
6. Stanford University: the intellectual force that anchored Silicon Valley here rather than elsewhere.  http://en.wikipedia.org/wiki/Stanford
7. Stanford Linear Accelerator: Not really central to Silicon Valley history, but it is big and impressive and highlights Stanford's role in attracting scientific talent to the area. http://en.wikipedia.org/wiki/SLAC
8. Sand Hill Road: the center of VC in the valley.  http://en.wikipedia.org/wiki/Sand_Hill_Road
9. Hewlett-Packard Garage: considered the "birthplace of Silicon Valley" - the prototype of the garage entrepreneurial culture.  http://en.wikipedia.org/wiki/HP_garage

The tour ends conveniently near to the best Ginger Gelato in the Whole Universe.

I'm proud to say that v1.5 of the WSO2 Mashup Server is now available for download!  This release has been a few months in the making, and I'm thrilled to see a major update to the Mashup Server finally out.  News release here. 

Thanks to the efforts of Keith, Tyrell, Channa, Yumani, and many others at WSO2, we've been able to put some snazzy new features in this release:

• Integrated award-winning Data Services: quickly and with zero-coding map queries on your databases, Excel spreadsheets, or CSV files into Web services.
• Google Gadgets: power a Google gadget with a mashup service - we generate a try-it, a code template for customizing the gadget, and even a build in Gadget dashboard based on Apache Shindig.
• Security, security, security: Limit access to your mashup service to only authorized users, encrypt and sign your messages, and access services which are secured.  I think our security interface is the easiest way to engage advanced WS-Security features that's out there!
• OpenID login support.

There's lots of little improvements too, and I hope to talk about those, as well as the major features above, in upcoming posts.

For now though, a hearty congratulations and a good night's sleep to the Mashup Server team!

Yumani has published a nice tutorial on Sending Instant Messages via Mashup Server showing how to use the instant messaging capabilities built into the WSO2 Mashup Server.  I think the ability to have instant alerts as part of your mashup is a cool feature, and as Yumani shows in this article, they are pretty trivial to do.

I'm looking forward to adding the next step in a future release - the ability to invoke a mashup using instant messaging.  Then we'll have a scriptable platform for building agents that can communicate via IM.  Imagine converting your favorite service to an IM conversational interface.  Or making a data source available through an IM conversation.  Or even imagine new games...  Endless possibilities!

Just posted a few snapshots from a Saturday day hike to Loch Leven Lakes, a wonderful and fairly accessible bit of the High Sierras.  Haven't been there in a while, despite it being a fantastic place for some bouldering with the kids.  It was good to get above California's persistent smoky haze for a few hours (though later in the afternoon even the high country took on a smoky hue.)

Wishing everyone a last minute happy 4th of July, and for those of you who missed fireworks this year, here's a couple of photos to hold you till next year.  Full Flickr set here.