From Learn About the Ext JavaScript Library

Breaking News: According to the Ext Roadmap Full REST support is slated for 2.1 (Spring 2008) - great news!!

Contents 1 REST and RESTful Web Services 1.1 A Disclaimer of Sorts 1.2 HTTP Methods 1.3 HTTP Status Codes 1.4 Busting the Cache-buster 1.5 Representations of Resources, Content Types and the Accept: header 1.6 Data Stores, Custom HTTP Methods and in particular: Ext.data.JsonStore 1.7 HTTP Authentication 1.8 Cookies and Sessions 1.9 Cross-site Communication 1.10 Closing Remarks 1.11 Further Reading

REST and RESTful Web Services

REpresentational State Transfer (REST) is an architectural style. The term was coined by Roy Fielding (co-author of the HTTP standard) in Chapter 5 of his 2000 PhD dissertation which focuses on the rationale behind the design of the modern Web architecture and how it differs from other architectural styles. A crude explanation of REST goes as follows: you have Resources (conceptual objects such as entries in a database) which are exposed via a uniform interface (on the web this would be the HTTP Protocol and the five standard well-known HTTP verbs: GET, POST, PUT, DELETE and OPTIONS). Representations of the state of the resource are accessed and operated on via the uniform interface. For example, to view the user account you GET a representation of its state, to update it you PUT a new representation, to remove it you DELETE the resource, etc..

REST is not limited to HTTP (HTTP is the message-based protocol that drives the World Wide Inter-Web) and technically your uniform interface doesn't have to use any or all of the 5 well-known HTTP verbs listed above for it to be truely "RESTful", but at present most "RESTful" web-based services are built on top of the HTTP protocol using the 5 standard HTTP verbs, so that is what we will focus on. My treatment of REST is highly influenced by Leonard Richardson and Sam Ruby's 2007 definitive book on the subject, RESTful Web Services (O'Reilly). I highly encourage you to read the book if you want to learn more about REST. I don't want to get too distracted with talking about REST theory and why you might want to build your Web Services in a RESTful way (this is not the place for such a discussion anyway) so from here on in I will assume you have weighed up the pros and cons of REST and decided to join the revolution :)

A Disclaimer of Sorts

There's nothing magical about talking to RESTful Web Services from ExtJS or any other Ajax library, after all as far as we are concerned REST is just a set of best practices for communicating over the HTTP protocol. The good news is that the XmlHttpRequest object that makes Ajax possible fully supports all 5 standard HTTP verbs in all modern browsers (citation needed). The bad news is that web developers in the main have only recently begun thinking seriously about REST and hence most Ajax libraries/toolkits currently assume that you want to do everything via only GET and POST requests and hence you have to learn how to leverage the library at a deeper level if you want to make PUT/DELETE/OPTIONS requests. To this end, this article can also be seen as a tutorial on digging a little bit deeper down into the ExtJS API and bending it to your will (and hence might even be useful to the non-REST ExtJS community).

REST has only recently gained traction in the mainsteam (especially with its adoption by big-name players such as Amazon) so it is reasonable to expect that libraries such as ExtJS don't have it at the top of their feature list. The point of this article is to explain how you can "do REST" with Ext right now, you just need to jump off the beaten track slightly. The nature of this article means that we will be focusing on a few areas where ExtJS could be improved to make REST easier, but this is by no means intended as a general criticism of ExtJS. As mentioned above, you will currently find shortcomings in all of the major ajax toolkits when it comes to REST. The Ext API is always evolving and hopefully one day this guide will become obsolete. Until then, read on, and enjoy the power that Ext puts at your fingertips..

HTTP Methods

Specifying the HTTP Method to use is easy when you generate an ajax request with the low-level Ext.Ajax.request() method. This method isn't always at your fingertips when you're working through higher levels of the API (for example when configuring a Ext.data.Store to use for an Ext.grid.GridPanel) but it's good to play with the API at this level first to convince yourself that Ext can generate any sort of request you like (it's handy to be able to generate these requests on the fly when debugging your RESTful Web Service too, although you might find the command-line drive curl library more convenient).

Here are some examples (use Firebug console to run these examples live and watch the XHR requests in action)

GET a list from the /users resource:

Ext.Ajax.request({
 url: '/users',
 method: 'GET'
})

POST a new article to the /articles resource:

Ext.Ajax.request({
 url: '/articles',
 method: 'POST',
 params: {
  author: 'Patrick Donelan',
  subject: 'RESTful Web Services'
 }
})

PUT an updated representation of an article to the /articles/restful-web-services resource:

Ext.Ajax.request({
 url: '/articles/restful-web-services',
 method: 'PUT',
 params: {
  author: 'Patrick Donelan',
  subject: 'RESTful Web Services are easy with Ext!'
 }
})

DELETE the resource that lives at /articles/rpc-is-the-best-web-architecture

Ext.Ajax.request({
 url: '/articles/rpc-is-the-best-web-architecture',
 method: 'DELETE',
 success: function(){ alert('Begone!'); }
})

HTTP Status Codes

The HTTP specification defines a rich set of status codes which are supposed to be used to convey the status of an HTTP request. You will no doubt be familiar with the following status codes:

• '404 Not Found' - the resource that was requested could not be found
• '401 Unauthorized' - the request requires user authentication
• '200 OK' - the request was successful

and maybe even:

• '500 Internal Server Error' - something bad happened on the server
• '503 Service Unavailable' - the web server is probably overloaded

However if you did a survey of most webapps written in the last 5 years you'd be forgiven for thinking these were the only http status codes that existed. In fairness, for static websites built entirely out of html files these are the only status codes you're likely to encounter. But for dynamic websites generated by programming languages (such as Perl, PHP, Ruby, Java etc..) there are plenty of other status codes that you can use to properly convery the status of an http request.

Briefly: the HTTP status codes (all 3-digits long) are broken down into:

• 1xx - Informational (probably not relevant to you)
• 2xx - Success (for example '200 OK' and '201 Created')
• 3xx - Redirection
• 4xx - Client Error (for example the generic '400 Bad Request')
• 5xx - Server Error (for example '500 Internal Server Error')

The obvious question is, why bother? There are many reasons, and in order to not get off-track I will assume you're reading this article because you already know why (refer to the #Further Reading section at the end of this article if you want to learn more).

One good reason to use HTTP Status Codes (and one that is relevant to us as javascript developers) is so that whatever carries out the request doesn't need to understand the body of the response in order to process the response accordingly. For example, a script doesn't need to care what format your response body is in (or more importantly grok that particular format) to decide whether the request was bad (4xx, perhaps a validation error) or successful (2xx) or even whether the server choked on the request (5xx). You can build all such logic into your script in a clean and logical way, and you'll find yourself fitting in more natually with the rest of the HTTP ecosystem (for example if you script has a syntax error your web server will probably generate a 5xx status code for you). And equally importantly, you won't find the need to put status messages into the entity body itself, which is great both for you (one less non-standard thing to document) and great for anyone who uses your Web Service because they can rely on their existing knowledge of the universal interface rather that being forced to learn your special syntax.

One example of this, and one area of the Ext API that I'm prone to rant about (presented here in the spirit of constructive criticism - see #A Disclaimer of Sorts) is the way that Ext.form.BasicForm and Ext.form.Form actions expect to receive certain Ext-specific parameters in the server response in order to correctly handle automatic form validation and success/failure callback handling.

For example, a successful request is expected to return '200 OK' and the following entity body (in particular note the mandatory "success" property):

{
 "success": true,
 "data": {
  "field_id_1": "Field 1 Value",
  "field_id_2": "Field 2 Value"
 }
}

The entity body here is redundant because you already said it was successful with the 200 status code.

Furthermore, a request that fails validation is supposed to return '200 OK' (instead of an appropriate 4xx code to indicate that the client specified invalid data in the request), and the following entity body:

{
 "success": false,
 "errors": [
  { "id": "field_id_1", "msg": "Field 1 Error Message" },
  { "id": "field_id_2", "msg": "Field 2 Error Message" }
 ]
}

The biggest problem with all of this is if you only want to submit the form. If you forget to include the Ext-specific "success: true" property in the JSON/xml entity body and just return a success status code with an empty body such as:

{
}

you'll find that the form submission handler always calls the failure callback, even though your server is explicitly saying that the request was successful!

It's a gotcha that can leave programmers who aren't familiar ExtJS (or people like me who sometimes forget about the mandatory "success" property) scratching their head wondering why Ext is ignoring the status code.

Anyway, back to the land of pragmatic work-arounds. You can achieve Ajax form submission of Ext.form.BasicForm and Ext.form.Form without requiring the Ext-specific "success" property in the entity bodies by staying well away from the submit() method and instead passing the form as a parameter into a familiar Ext.Ajax.request() call which does the right thing in regards to http status codes and success/failure callbacks:

Ext.Ajax.request({
 url: '/some_resource',
 method: 'POST',
 form: 'my-form-id',
 success: function(){alert('Must have been 2xx http status code')},
 failure: function(){alert('Must have been 4xx or a 5xx http status code')},
});

The above approach means that you don't get automatic displaying of validation errors (which is a great feautre). You can easily isolate this piece of functionality from the Ext code base and call it directly after you've encountered a 4xx response code, something that I will post up here if I find the need to use it in one of my projects before Ext.form.BasicForm hopefully becomes HTTP Status Code aware!

Busting the Cache-buster

One of the benefits of using REST is that you're working with HTTP instead of against it (RPC anyone?!). For example, you're probably taking advantage of HTTP headers to control content expiry and caching. By default, Ext tries to give your GET requests a bit of a hand in avoiding unwanted caches by adding a 'cache-buster' query parameter to GET requests. You can get rid of this with the following one-liner somewhere near the start of your javascript code (which those fond of double-negatives will especially enjoy):

Ext.Ajax.disableCaching = false;

Representations of Resources, Content Types and the Accept: header

Let's say you have a resource exposed at http://server.com/resource that you want to GET a representation of. Some RESTful Web Service use the "Accept:" HTTP Header in your request to decide what serialization format to return the representation in. For example, if your request looks like this:

GET /resource HTTP/1.1
Host: server.com
Accept: application/json

Then the server might return a response (in JSON format) that looks like:

[ 
 {a:1, b:2},
 {a:3, b:4}
]

Whereas if your request looks like this:

GET /resource HTTP/1.1
Host: server.com
Accept: */*

Then the server might return data in its default format, which might be xml:

<opt>
 <data a="1" b="2" />
 <data a="3" b="4" />
</opt>

One common scenario in Ext is that you want to get all your representations in a particular format (probably JSON), in which case the easiest approach is to set a default header (the "Accept:" header) for all requests:

Ext.Ajax.defaultHeaders = {
 'Accept': 'application/json'
};

All subsequent requests will then have the "Accept:" header set to "application/json". For example, if we generate a simple GET request to /resource and fire up Firebug to check the headers sent:

Ext.Ajax.request({
 url: '/resource',
 params: {
  test: 1
 },
 method: 'GET'
})

Gives:

Host: server.com
Accept: application/json
X-Requested-With: XMLHttpRequest
..

Data Stores, Custom HTTP Methods and in particular: Ext.data.JsonStore

Ext has an incredibly well-designed object hierarchy. One example of this is Ext.data.Store which "encapsulates a client side cache of Record objects which provide input data for Components such as the GridPanel, the ComboBox, or the DataView". Continuing the trend of this article whereby we have a Web Service delivering representations of resources in JSON format, a time will probably come when you turn to the Ext.data.JsonStore class to dynamically populate a widget such as an Ext.form.ComboBox. You dutifully wire up the Ext.data.JsonStore into your Ext.form.ComboBox definition:

var cb = new Ext.form.ComboBox({
 store: new Ext.data.JsonStore({
  url: '/resource?query=something',
  fields: ['id']
  }),
 displayField: 'id',
 valueField: 'id',
 triggerAction: 'all',
 renderTo: document.body
})

only to discover that even with the query parameter in there (which is an almost certain give-away that a GET request was intended) Ext insists on using POST to get the data!

Now, if you're talking to a RESTful Web Service you'll probably be getting back an HTTP status code of '405 - Not Implemented' (assuming the resource only supports GET). And even if your Web Service is non-RESTful there's a chance you want to explicitly use GET too if your server-side language distinguishes between GET and POST params and you've done the right thing and told it to grab GET params for this request.

Ext.data.JsonStore is actually just a little shortcut definition for a full-blown Ext.data.Store that specifies an Ext.data.Proxy and an Ext.data.Reader with some hard-wired options. So the way forward is to bypass this shortcut and define your own Ext.data.Store instead - it only adds a few lines to your ComboBox definition and if you find yourself using it often you can define your own Ext.data.MorePowerfulJsonStore that lets you specify the exact options you want instead of hard-wiring things:

var cb = new Ext.form.ComboBox({
 store: new Ext.data.Store({
  proxy: new Ext.data.HttpProxy({
   url: '/resource',
   method: 'GET',
   params: {
 	query: 'something'
   }
  }),
  reader: new Ext.data.JsonReader({
   fields: ['id']
  })
 }),
 displayField: 'id',
 valueField: 'id',
 triggerAction: 'all',
 renderTo: document.body
})

As you can see, the definition of Ext.data.HttpProxy lets you specify any HTTP method you like. So even though you're probably going to always want to use GET to populate a data store, you now have the power to specify any HTTP method you like.

HTTP Authentication

It's common for RESTful Web Services to use an HTTP Authentication scheme that takes advantage of the well-defined (and extensible) authentication-related headers in HTTP. Some examples are HTTP Basic Authentication, HTTP Digest Authentication and Amazon's custom S3 authentication scheme (which takes advantage of public/private key concepts).

Here's an example of HTTP Basic Authentication - the scheme commonly used to password-protect directories/websites via an Apache .htaccess file. Let's say you have a password protected resource at http://mysite.com/protected_content:

GET /protected_content HTTP/1.1
Host: mysite.com

The web service sends back the following header:

HTTP/1.1 401 Authorization Required
WWW-Authenticate: Basic
..

Where the header WWW-Authenticate is set to Basic, indicating that the server wants you to use HTTP Basic Authentication WWW-Authenticate (this is an example of how HTTP Status Codes are used in RESTful Web Services to convey information about the the result of a request). Now, HTTP Basic Authentication is achieved by base 64 encoding the concatenation of the user's name and password with a ':' in the middle and prepending the string "Basic " to the front. For example, if your username is "Aladdin" and your pasword is "open sesame", you need to resubmit your request with the Authorization set as follows:

GET /protected_content HTTP/1.1
Host: mysite.com
Authorization: Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==

The server then receives this request, base 64 decodes the username and password (yes your password is essentially sent in plain-text in HTTP Basic Authentication which is why you probably want to run your web service over HTTPS) and then uses its Authorization rules to decide what content it will return to you (hopefully the protected content that we were after).

Now in theory, carrying out the same authenticated request via an Ajax call is just a matter of setting the Authorization header as listed above and then firing the request as usual, eg.:

Ext.Ajax.defaultHeaders.Authorization = "Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==";
Ext.Ajax.request({
 url: '/protected_content',
 method: 'GET'
})

Which does indeed work (and obviously you'll want to define a convenience function to compute the Authorization string for you - refer to this forum thread for the base 64 encoding algorithm). Except...

The problem is that web browsers try to help out when it comes to HTTP Authentication. If you try to request a resource that returns a 401 Authorization Required status code with the WWW-Authenticate header set to Basic, the web browser will actually prompt you with a login box. It will compute the base 64 encoded string for you, which is handy, but it will then store the authentication string in a local cache and use it to set the Authorization header on every request you make. This in itself sounds quite convenient, until you consider the following problems:

• You can't style the login box to suit the rest of your application
• You can't overwrite the Authorization header with a different value via the Ext.Ajax.defaultHeaders property as listed above (anything you set will be ignored) which means you effectively can't let the user sign in as someone else
• You can't remove the user's credentials from the browser's cache which means you effectively can't log the user out

Now, if you do your login via an ajax-enabled html form and the server never sends a 401 Authorization Required response, the browser won't intervene and you'll be fine. My current strategy for handling this in a RESTful way is to define a special resource called "/my_account" that the login handler GETs a representation of with the Authorization header set according to the HTTP Basic Authentication scheme using the username/password that the user entered into the form. If the respresentation is empty the login handler knows that the credentials are no good, whereas if the representation is that of a valid user the application can consider login successful and leave Ext.Ajax.defaultHeaders.Authorization set to the authentication string.

But what happens if the user tries to bypass your login form and access a protected resource directly? This is not such a problem for single-page ajax webapps, but a possible scenario for more traditional multi-page webapps where a user might bookmark a resource and try to jump to it directly. It's also possible in the single-page ajax webapps where you have a single ajax page calling a RESTful api (for example you javascript might internally get a list of all users to populate a combobox by doing an xhr request to /api/users and this resource would presumably be only available to authenticated users) - if the user were to try typing /api/users into their browser the browser would receive a 401 Authorization Required response, display the login box and then cache the result, meaning that you'd be stuck again.

Thankfully, the designers of XmlHttpRequest thought about this, and added two optional arguments to the xhr contructor that lets you specify a username and password to use for that request (again the base 64 encoding is done for you, which is nice). Unfortunately, as it currently stands ExtJS's ajax wrapper doesn't allow you to specify these 2 optional parameters. Until this situation changes (which I'm guessing is just a matter of time), you have some options:

• In some browsers you can put the username/password in the url via: http://username:password@mysite.com however this is officially unsupported in internet explorer (in all versions after IE6).
• Use Doug Hendricks' excellent ext-basex.js library, which among other cools things, modifies ext-base.js (Ext's native adapter) to allow you to pass in the aforementioned username/password parameters into the xhr constructor (and hence overwrite whatever the browser has in its cache).
• Be a bit non-standard and return a HTTP Status Code other than 401 Authorization Required so that the browser never tries to help. For example you could return 403 Forbidden. Technically this violates the HTTP/101 specification which says that 403 indicates that Authorization will not help and the request SHOULD NOT be repeated but you might want to be pragmatic and do it anyway (sadly you won't have this freedom if you're using someone else's web service).

Cookies and Sessions

Use of HTTP cookies to facilitate sessions is decidedly un-RESTful because you are turning a stateless protocol into a stateful one. This article will not spend time discussing why one might consider HTTP sessions to be a bad thing (if you are interested, check out the #Further Reading section at the end of this article). Cookies themselves are still useful, and don't violate RESTful principles if you use them for other purposes such as client-side local storage (if you don't minde the extra headers being attached to every request you make - HTML 5 local storage will hopefully alleviate the need to use cookies for this).

For example, you might want to trade security for convenience and add a "remember me" checkbox to your login form so that users can be automatically logged back in between browser sessions. In a multi-page ajax app you'll also need a feature like this to overcome the fact that client-side javascript state is lost on every page load.

To achieve this you could store the user's authentication details in a cookie:

Ext.state.Manager.setProvider(new Ext.state.CookieProvider({
 path: "/",
 expires: new Date(new Date().getTime()+(1000*60*60*24*14)) //remember for 2 weeks
}));
Ext.state.Manager.set('auth', "Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==");

And then test for the existence of this cookie at app (or page) load-time and if found, use it to transparently authenticate.

Obviously this is inherently insecure as anyone who can access the user's browser cookie cache can access their username and password. To mitigate against this you might want to consider using a more advanced HTTP Authentication scheme. It is also worth pointing out that the normal session-key-in-a-cookie approach used by countless websites is vulnerable to plenty of security holes of its own such as session hijacking, Cross-site request forgery, etc..

Cross-site Communication

Due to the Same Origin Policy that browsers impose on Javascript, the XmlHttpRequest object is prevented from communicating with external domains (it's an attempted security feature). There are known work-arounds, such as script tag injection (where you use the src attribute on html elements to trick the browser into communicating remotely) however this is limited to GET requests only so you're not going to be able to do much RESTful communication with it. (FYI this is implemented in Ext as the ScriptTagProxy class which is a drop-in for the HttpProxy class).

Your other options are to use Flash for cross-domain requests, or to wait for the W3C Access Control (which lets you make cross-site XmlHttpRequests) to be implemented by all the browsers.

Closing Remarks

This article covers the topics where I've found myself heading slightly off the beaten path when using Ext to talk to my own RESTful Web Services. If you find other areas that aren't covered, as I'm sure will happen as REST becomes more widespread, please drop me a line as I'd love to hear how your own experiences with REST and Ext go.

Further Reading

• RESTful Web Services, by Leonard Richardson and Sam Ruby, 2007 (O'Reilly) - provides a definitive guide to all things REST, as well as the Resource Oriented Architecture which you can use to implement highly functional, real-world RESTful Web Services.
• Roy Fielding's 2000 PhD Dissertation which first coined the term REST
• curl library
• HTTP Status Codes
• REST on Wikipedia (contains links to other REST resources, no pun intended)