I promised last week to touch base again today and I’m happy to report that the CloudKit gem is available for the brave amongst you to browse and use. I say this because there are several working pieces that are of value right now while the completed gem will take a few more days before being worthy of an “official” release and a push to RubyForge.

http://github.com/jcrosby/cloudkit/tree/master

Here’s what the code straight out of GitHub can do right now after a “rake local_deploy”:

• Generate a small, fast RESTful web app that supports OpenID and OAuth out of the box using Blake Mizerany’s killer Sinatra DSL (more on this later).
• Generate a GWT UI and REST client skeleton for those of you who dig that sort of thing. (A similar jQuery skeleton generator is in the works.)
• Generate RESTful resources with migrations plus ActiveRecord models that have their RESTful routing and authentication baked in.

Things that are extremely close and may be available by the time you read this:

• A rake task for running the app using Adobe AIR.
• JavaScript/ActiveRecord model synchronization stolen from my GWT on Rails project.

Other items in the works that won’t likely be ready when you read this but will appear rapidly, piece by piece, prior to the gem release:

• Desktop application updates.
• Automatic desktop schema migrations that match the server side counterparts.
• Automatic data/resource synchronization between desktop clients and server side storage.
• Transparent offline/online detection and management using the Application Context (a.k.a. MCP) mentioned in the previous post.

I’ll continue these frequent updates as the pieces roll out.

Just over two weeks after the release of GWT-REST, the next layer of the Actionatr stack is ready for release.

GWT on Rails

http://code.google.com/p/gwt-on-rails/

GWT on Rails integrates GWT client side compiled JavaScript with Rails RESTful web services by providing client and resource generators, migration support, asynchronous RESTful client support, and Rake automation. A summary follows.

GWT Client Generator

Until now, building a GWT interface for a Rails project required the use of either an IDE or command line tools to build a GWT “module” which then had to be tweaked for use with Rails instead of the default embedded Tomcat server. Using Rails conventions as a starting point, the GWT client generator allows one to generate a GWT interface (or any number of them) with a single command:

script/generate gwt_client dashboard /Users/jdoe/java_libs/gwt-mac-1.4.61

Rake support is included so that cleaning, compiling, and running your hybrid project are also one-liners:

rake dashboard:compile

rake dashboard:hosted

rake dashboard:clean

GWT Resource Generator

Any ActiveRecord model can be exposed on the client side by simply running the GWT resource generator:

script/generate gwt_resource Project dashboard

Migration Support

Once the resource generator has created a client side resource/model, your ActiveRecord migrations will transparently keep everything in sync. Any fields that are added, altered or removed will likewise be updated in JavaScript-land. By generating a base class that is recreated after each migration and a corresponding model that extends this base, custom code can remain intact while the basic model changes.

GWT-REST

The GWT-REST module comes bundled with this plugin, allowing easy CRUD operations with the built in RailsJsonResourceCollection class.

More details are available on the Google Code project site. Enjoy!

Today, I am pleased to announce the release of GWT-REST.

http://code.google.com/p/gwt-rest/

From the Google Code project site:

GWT-REST is an asynchronous RESTful client implementation for GWT.

GWT-REST can be used alongside GWT-RPC or as a replacement for it. The intent is to provide a drop-in solution for communicating with REST web services that may or may not be implemented in Java.

GWT-REST works out of the box with Rails 2.0 using JSON. I built it for Actionatr and extracted it to share with others who may be looking to do the same thing. Rails isn’t a requirement though. The code is built using interfaces that allow you to customize for other representations like XML and other server side styles like JSR-311/Jersey.

Not knowing if I cover enough technical subjects here on the blog, I thought I should throw in even more nerdery detailing the path from Actiontastic 0.9.3 to Actiontastic 1.0 with its web sidekick, Actionatr. This was inspired by Wolf Rentzsch as well as a post from Ivan Moscoso on the Pathfinder blog, where he asks about alternatives to GWT-RPC (which we’ll get to in a moment). This post could also be titled “Why I Should Blog More Frequently, In Smaller Chunks.” My apologies.

Background

Actiontastic has remained stubbornly at 0.9.3 for months now while I have refused to release a 1.0 that isn’t a 1.0. Sure, it could have been released anyway as a commercial app before so many others joined the Mac/Getting Things Done niche, riding the wave of cash to a beach front house on some island in the tropics. (A second scenario would be selling just enough copies to require keeping a full time job while relegating all evenings and weekends to email support, but let’s favor optimism when daydreaming, shall we?)

Instead, I have stuck with the original vision of building something I believe is a requirement for productivity applications — ubiquity. It’s not okay to not have my lists with me when I’m running errands. It’s also not okay to not have my lists with me when I’m away from the Mac and need to make some calls. Paper printouts don’t sync and thus don’t solve the problem. Web apps don’t cut it on the BART (yet). iCal/PDA syncing also doesn’t hit the 100% mark for two-dimensional lists that are central to canonical GTD. That’s where the hybrid desktop/mobile/web app comes into play.

“Just ship it” is good advice, but that very idea hinges on defining the minimum requirement(s) for “it,” a 1.0 release. Setting the bar at “ubiquitous” has increased the challenge for this project while focusing it in many other areas. For example, there won’t be lots of widgets and configuration options inside of dozens of popups, sheets, and panels. Instead, Actiontastic will remain focused on the Next Action idea with its filter bars catering to this sort of work flow.

Exploration

As I’ve mentioned before, there have been lots of technology prototypes assembled behind the scenes. These include:

1. A basic Java web app, started more than a year before Actiontastic was first released.
2. Hard wiring communication between blocks of code in Actiontastic to a web service built in Rails. (Not really a serious solution but good for exploration of the idea.)
3. Building an app that ran in the browser as well as the desktop using Flex and AIR (using Cairngorm which was pretty nice).
4. Building an app that ran in the browser as well as the destkop using JavaScript/CSS/HTML with AIR.
5. Building an app that ran in my own Mac app container (a variant of WebKit Apps) that used a WebKit plugin to allow local SQLite storage via JavaScript/Objective-C bindings. This option used only JavaScript/CSS/HTML for the interface so would have also worked online as well as on other platforms. ExtJS with Jester and a RESTful Rails back end were helpful with this approach.
6. Building method #5 above without the plugin, tying it more closely to the Mac platform with WebKit hooks.

All of the above failed. Here’s why, using the same numbering as above:

1. This was during the age of Struts, Spring, and Hibernate. XML configuration pain and mapping the same things in every layer of the app wore thin and led to picking up my first Mac programming book, Cocoa Programming for Mac OS X, if only for a change of pace. This also led to Rails.
2. As mentioned before, hard wiring interactions isn’t an elegant design and would have been a nightmare to maintain as the application progressed.
3. Although a Flex app can be themed to look like a native Mac app, I found that basic text support was not built into Flex in a way that current Actiontastic users would expect. This also required implementing my own widgets for things like sheets. Further, integrating with native services on the Mac required lots of hackery including an Actionscript-to-Java bridge to a Java-to-Objective-C bridge that has been deprecated. I’ll pass.
4. Scrollbars, plus some of the problems with option #3 above.
5. The plugin seemed like the wrong way to go as Google Gears, once it regains its Safari compatibility, will solve this same problem. More on that later.
6. Sadly, this is the one I hoped would work the most. I love the idea of using Web Standards to build interfaces that work on the desktop and in a browser. This one made it very far including a complete theming of ExtJS to mimic Leopard source lists, sortable tables, fluid layouts with dragable dividers, animation, and even HUD windows. This may still be the platform for Actiontastic 2.0, given enough compatibility progress in the browser world. In the end, juggling incompatibilities between browser versions, even the same browser (Safari 2 vs. Safari 3 Beta vs. Safari on Leopard before it shipped) made things grow ugly. If you see your app as “managing documents” or targeting a more web-native look and feel, this could still work, but when trying to mimic precise UI behavior from the desktop world, things are not so good. For the record, WebKit is my favorite JavaScript and CSS environment, but building a desktop-like web app that meets multi-browser expectations would require a team and a very robust architecture. This might include basic objects that could be extended for browser-specific quirks (rather than wedging in the ugly but common “if Gecko then…” kinds of hacks throughout the app). An architecture like this simply wouldn’t fit in the time line for 1.0. More on this browser-specific targeting in a moment, though.

This leads us to today, where the failures aren’t really failures. They have been excellent learning experiences and perfect excuses to explore some new technologies. Getting back to the title of the post, this is where GWT and REST come in.

Automagic REST

Months ago, I blogged about the possibility of a fast, compiled REST app that could be generated from a data model. Rails sort of hits in this direction minus the “fast, compiled” and “generated from a data model” part. This isn’t to be snarky either. The scaffold_resource generator in Rails (soon to be the default in 2.x) can do an impressive job of generating a REST service, but it lacks a few things out of the box that are offered more easily on the Java platform today.

For starters, Java-land seems to be heading in a better direction, thanks in part to competition from Rails. These days, if you want a REST service in a green field environment, you can just generate it from a data model. The anti-patterns encountered during failure #1 above are largely gone. This is due to a mix of technologies: Netbeans 6.0, Jersey/JSR-311, and JPA. Check out this tutorial, and there you have it, automagic REST that produces and consumes XML and/or JSON.

KloudKit

In the same blog post referenced above, I joked that REST was the new SQL. This is the tie-in to the Mac platform. If your Mac app uses Core Data, then it is following common ORM patterns. It retrieves, saves, updates, and deletes records, most likely from a SQLite database. These operations map exactly to those in a REST architecture — GET, PUT, POST, and DELETE. Naturally, this mapping means that a Mac app should be able to easily talk to a REST web service. This is where KloudKit comes in.

KloudKit is being assembled on top of OAuthConsumer so that it can do transparent authentication from a Mac app to its web companion. If you’ve ever published something from iMovie to YouTube or used a Flickr uploader, it’s sort of like that in the eyes of a user. With authentication out of the way, KloudKit can replicate actions taken on the Mac to the web. Right now, this implementation is forming as a framework that uses Core Data’s objectsDidChange notification, but I could see it also working as an extension to Wolf Rentzsch’s MOGenerator.

The MOGenerator option would have less invisible “magic” and could generate a rich data model to act as the seed for the generation of the REST app. All hooks would be exposed in the generated code, favoring a sort of “annotation over confusion” approach (with apologies to Michael Easter who seems to have been the first to coin this phrase).

Given a clear path for interface development on the Mac (Cocoa), storage (Core Data), authentication (OAuth), and communication with a RESTful web app (KloudKit), the browser interface to this RESTful hub is all that remains. This is where GWT shines.

The Google Web Toolkit brings these advantages to a browser interface:

1. Java is compiled down to JavaScript, reducing the amount of context switching needed during development. I’m not bad at switching from HTML to JavaScript, to CSS, to Ruby, to SQL, to Objective-C, to graphic design, but it could be easier. A mostly Java and Objective-C mix would go a long way to delivering desktop, mobile, and web apps for most operating systems.
2. Easy internationalization support.
3. Optimized browser-specific versions of the same app delivered according to each client’s needs.
4. Reduction of HTTP requests by merging all graphics into a single file. This file is delivered to the browser in one request. CSS tricks are used to display the pieces in the right spots.
5. Debugging of JavaScript using an IDE while running in “hosted mode.”
6. A large set of widgets, allowing appearance to be styled with CSS. There is even an add-on to integrate GWT with Ext.
7. Unit test support.
8. Google.

GWT’s native method of communicating with the server is called GWT-RPC and this is its weak link, in my opinion. Not that GWT-RPC isn’t fast or well integrated into the client-side JavaScript. It is. However, the current 1.4 version doesn’t handle Java 1.5+ code, meaning it doesn’t work with annotations, which rules out JPA. This can lead to EJB 2 flashbacks for those who have been there, as it requires an extra layer of Data Transfer Objects, mapping those DTOs to and from annotated objects using pseudo-annotations as GWT compiler hints, two interfaces for each service endpoint, etc.

Fortunately, GWT also includes a JSONParser and a RequestBuilder that can be crafted to work with a REST service. It may be even easier than this, however. GWT includes JSNI, a JavaScript Native Interface which allows it to interact with third party JavaScript libraries. Jester, mentioned above, supports asynchronous JSON to REST services and can be configured to use mappings outside of the default ActiveResource conventions. Tying these things together seems like a nice approach.

Conclusion and the 1.0 Architecture

To wrap this up, we have a desktop app (even mobile apps using JME, Android, or the iPhone SDK), transparently talking to a REST web service. A browser app, compiled down to a single page AJAX app talks to this same service. Given that GWT now supports Google Gears and Safari support is planned, it won’t be long before the pure HTML/CSS/JavaScript vision is possible on all platforms.

Last night, talented RedMonk Analyst, Michael Coté posted a write up covering Microsoft’s Astoria project including this tidbit:

“The whole point of Astoria for me is to be a thin, REST-y data-layer on-top of the cloud. The web is The Cloud, of course.”

And, this one:

“Think something like Amazon S3, but with semantics layered on-top.”

Yes! Yes! And, did I mention YES? This idea of a “REST-y data-layer on top of the cloud” is the foundation of Actionatr (although without any Microsoft in the stack). Months of experimentation and research have gone into what is now being solidified in code. Actionatr is a platform that just happens to have a web interface and a Mac interface, but this is really just the beginning.

REST is the new SQL. Sort of.

The Microsoft take on this is to build a tool that plugs into a regular SQL data store, but what I would like to get at are the possible business models that could be built on top of such a thing. In the course of building Actionatr, I was thinking about how we could bake a REST interface into MySQL, but given that authentication and some amount of other configuration would need to take place, more would be needed. I looked at using the newly 1.0 Restlet API to create a fast, configurable, auto-inspecting layer that could be dropped onto any good data model as an easily scalable data layer for RIAs, legacy web apps, desktop apps or a blend of these. Rather than over-engineering my way to permanent vaporware, I decided to just use Rails as a temporary REST-only back end (i.e no HTML interface) and leave the fast, fancy, compiled SQL-to-REST framework for later (as it would not have an immediate benefit for Actiontastic/Actionatr users).

Let’s step back for a moment to trace this evolution. After digging into Fielding’s dissertation that introduced the REST (REpresentational State Transfer) concept and thinking about ways to scale and optimize such an architecture, I came to see the HTTP transport mechanism as one big hash pipe plus four simple operations, mysteriously resembling CRUD and SQL. A hash contains key/value pairs. When a URI is the key and the data is the value (be it an XML representation, JSON, or anything else), the added HTTP request semantics of GET, PUT, POST, and DELETE give us enough information to build naive caching mechanisms at any level of a given architecture.

Getting Clean by Taking Off Your ;close

With this in mind, it is important to note that adding “extra” methods on top of these standards might be harmful if you want to scale. As one concrete example, I added a “;close” method to an existing set of methods for Action resources. Oops. The problem with such a thing is that it destroys the ability to insert these naive caching mechanisms into the middle of an architecture. The caches would have to incorporate new business logic to understand how data inside an Action resource would change after seeing a “;close” request come across the wire. Adding such a thing into what started as a clean hash lookup optimization problem would degrade performance and add complexity for all future model/resource changes.

Thus, solid RESTful design goes beyond mashups and hype. Clean RESTful design allows easier scaling by specifying the complete state for all PUT and POST operations. Subsequent GET operations no longer have to make it back to your core application; they can simply bounce off a lightning fast cache. For that matter, GETs could be partitioned, other operations could be queued and batched, etc., but that is a research area unto itself and beyond the scope of this post.

Domain-Specific RESTful Hosting

So, once you’ve got a clean REST concept going, it is only a small step forward to imagine SaaS (software as a service) models on top of this. What if, oh I don’t know, say Actionatr, pulled all of the standard elements together into a nice API, put it in the cloud, provided hosting for some dirt cheap price, and thus enabled ubiquitous checkboxes for its users regardless of OS or device? What if anyone could use this service, including developers of other productivity apps?

Beyond this, what would other standardized SaaS models look like? How about contact management? Or blogging? Or calendaring? Maybe these models would contain more elements than a good user interface should support (Actiontastic is an example of minimalism in this regard), but that wouldn’t mean that the larger master data models wouldn’t be useful. It might be more like a grab bag than an all-or-nothing approach.

Either way, one could imagine a business model based on domain-specific data hosting. The extra benefit that this would have over a more generic approach like Astoria is that a community could evolve around this domain-specific data in a way similar to the process for building microformats. So, the model could be versioned and improved over time based on collective feedback from many different application providers. In the end, this could be built on top of an Astoria-like framework but it could begin right now, before the abstraction of a framework has to occur. (Anyone want to build such a thing based on database metadata, Restlets and EC2 with me?)

Server Side Rendering is the New Legacy

Taking all of this one step further, consider the so-called “modern” web app. After creating these things for somewhere around a decade, the hackish nature of building interaction models on top of a “page viewing” legacy (bookmarks, back buttons, etc.) is starting to smell kind of funny. It’s not that HTML doesn’t serve its own purpose very well, buy why can’t we just use this HTTP-as-hash-pipe model to build ubiquitous data services and leave the application interfaces to rich clients like the soon-to-be-open-sourced Flex?