Restfulie 0.5, atom feeds, content negotiation and default controllers

ATOM FEED

Restfulie 0.5.0 is out and its major new feature is its support to Atom feeds with variable media types.

A feed can be easily rendered by invoking the to_atom method:


@hotels = Hotel.all
render :content_type => 'application/atom+xml',
:text => @hotels.to_atom(:title=>'Hotels', :controller => self)


A collection might contain entries with different media types and each one will be rendered in its own way. The client code works as any other usual client would, note that there is still content negotiation taking place:


hotels = Restfulie.at('http://localhost:3000/hotels').get
puts hotels[0].name
puts hotels[1].name


And using hypermedia to drive our business, deleting an entry will send a DELETE request to that entry’s self URI:


Restfulie.at('http://localhost:3000/hotels').get.each do |h|
h.delete if h.city=="Sao Paulo"
end


In future releases we expect to support atom feed generation (and consuming) through Ratom.

CONTENT TYPE

Another easy to use feature is content type negotiation, which also happens when rendering a single resource:


@hotel = Hotel.find(params[:id])
render_resource @hotel


Users can now extend Restfulie in order to create custom formats (not based on json/xml/xhtml related media types)

CLIENT SIDE

Entry points have been extended and now they can be reached even if there is no class representing the received information in the client side:


support Restfulie.at('http://localhost:3000/cities').create(city)
Restfulie.at('http://localhost:3000/cities/5').get


One can also access the web response:

response = Restfulie.at('http://localhost:3000/cities/5').get.web_response


And play with content negotiation:


Restfulie.at(uri).accepts('media-type').get
Restfulie.at(uri).as('media-type').create(something)


DEFAULT CONTROLLERS

In the server side, a generic controller has been created which supports show, delete and create by default.

There is also a new method called render_created that can be used in order to answer a request with a 201 Created response and its resource location.

WEBSITE

Restfulie got its own website and all Ruby docs have been migrated.

Thanks to all the team and collaborators!

If you look carefully, you might find out next week’s upcoming news.

Continuous integrating: parallel tests the way they should be

We at Caelum have incorporated a lot of XP practices in our every day life for a while now. Still, we have faced a lot of issues with making the software one-click-deployable.

I will talk about the issues that arised in one of our continuous projects – that kind of project which has no end because the project is the heart of its company and as the company evolves, it should keep being developed.

The first big – and long lasting – problems were due to the build time taking much longer than the expected 10 minutes. Selenium end-to-end tests tend to be slow if run in a sequential way, therefore we adopted selenium-grid, which offered a tool to parallel run our tests. After a few months struggling with it, me and Lucas Cavalcanti submitted a patch to the tool where the user was able to control the grid-client machines in order to be able to cope with machine shut-down, and new agents config with a web front end. Unfortunately that patch was not accepted because the Selenium Grid project was going in a different direction from our necessity.

From that point on we noticed that our main problem was to parallelize the tasks, and that could be achieved with the Continuous Integration Server instead than with Selenium Grid. The most famous open source servers did not offer exactly what we expected, some of them offering just a way to distribute builds, not to parallelize them. This wouldnt make our build any faster, but only allow more (distinct) builds to run concurrently.

There is, of course, Cruise, which does exactly what we needed – parallelize the tests for us – but the client company did not want to spend money for that reason at that point, so an open source approach. Our decision was to create our own CI server, and face all the problems other teams have faced while doing that.

Right now, Integra – this new continuous integration server – has GIT and SVN support, all that email and tasks features and the most important thing at all for us.

Automatically Parallellizing Tests

Cruise goes a long way when it allows us to declaratively annotate our tests in groups and run each group of tests in a different agent at the same time. Integra has implemented the same idea but went a little bit further: instead of making you responsible for categorizing your tests in N different categories, and running it in N machines, Integra allows us to simply let it know the test set and the maximum time that we want our tests to be run.

Let’s say, I want my integration tests to be run in x=5 minutes, you can configure Integra’s test plugin with this number, it will – in its first run – just randomly run the build in a number of machines. After this first run, Integra knows the average running time of each test and can therefore create T(x) partitions in order to achieve a maximum running time of x. It’s clear that the number of partitions T(x) depends on the running time of each test and an eager algorithm was implemented that minimizes the difference between x and max(running_time_of_a_single_test).

Restrictions

If max(running_time_of_a_single_test) > x, we can’t achieve your goal. Another restriction might be the number of agents available being lesser than T(x).

Eager algorithm to partition the tests

The eager algorithm that we have implemented seems to give the best solution which minimizes what has been commented. Although I have convinced myself of it with a “during-the-shower proof” in my own mind, its probably time to put it down on paper to be sure about it.

Anyone willing to improve the algorithm?

Summing up, human tests should be automated to its maximum. Thats what we have been talking for a while now, and this test’s partitioning job – currently an human task – could have been automated, and now it is.

We intend to release Integra as soon as I get the html page design later next week because right now its as ugly as… anything else i can design.