Guilhermesilveira's Blog

as random as it gets

When I am unable to TDD…

with 12 comments

It has been a while since we started using unit tests (and other types of tests) in our projects. But test driven design has always been something that once in a while I feel unskilled enough to do from start!?

Many people (including myself), for many reasons, that TDD is the way to go… but what happens when I have no clue on what I am building?

In the last 3 open source projects that I have worked at, only one started with TDD since its conception. 2 of those projects are actually tools (TestSlicer and VRaptor) while the other one is an continuous integration server.

The first project is about running integration builds faster by running only the required tests. In other words, it should only run the tests affected by the change log.

The problem with creating this tool is that, while coding it for the first time, it is so unclear how it will work or what it will exactly do that it was impossible to test it prior to creation. The first attempt was to use TDD and some code was created. After a few days, it was clear that the way the tool was going to achieve its purpose was way unclear in order to create integration tests for it. Some days afterwards everything was even more clear:

  • unable to keep up coding it due to the lack of more advanced tests
  • it was possible to create such a tool

After the first version was used in production, the conclusion was that it was a great approach to use and drop and re-use TDD in this project, because the idea was way so unclear that it would require anyway a complete coding of the project from scratch – again. Due to the very early stage of the project and Its purposes and ideas evolving too fast in a short period, it felt/was counter-productive to keep tdd-ing.

VRaptor started from scratch with TDD and went just fine. We all knew its purpose and had a somehow clear vision of what we desired (a refactor friendly framework), not knowing exactly how to implement it – but in the end, achieving it. TDD win.

The third project suffered from the same problem that the first one had. We just had a short (unclear) glimpse of what we wanted: “run all our tests in parallel” instead of “running our builds in parallel“. But how?

Should it be the job of our agent machines to consume what our servers make available? Or should the servers manage the agents (as cruise) to do their job? Should it be implemented through low-level sockets or http based resources? Everything was so unclear and changed so fast in the first couple of days that it was impossible to test first, code afterwards at that time.

After the first trial on a private project, it was clear how to and even more clear what we wanted to achieve, so it was time to refactor and start TDD’ing.

This is the common feeling that I have found about TDD bugging people</a… whenever your project is a prototype to check that something is possible of doing, or you are just creating something completely new that you have no idea what it is, it sounds you should first create the prototype, throw it away and restart it with TDD.

Maybe typical web-based app’s won’t suffer of this problem because sprint plannings will help getting things clear in the developer’s mind. But developing a library or a tool for other developers it not the same type of task. At least during the first few moments…

Advertisements

Written by guilhermesilveira

August 24, 2009 at 10:00 am

Posted in agile, Uncategorized

Tagged with , , ,

12 Responses

Subscribe to comments with RSS.

  1. In such cases, I have been trying to write more coarse grained specs, generally in the acceptance level. From what I can say so far, it’s been working quite well. Even when I don’t have a decent picture of how to do what I want.

    In most spikes, I agree that we don’t know yet exactly *how* to build the thing, but you always know *what* you want. Define what means “to be done” writing a high level (coarse grained) spec which says your code is working and *what* it should do. When you pass these specs, the Spike is done.

    It also helps if you think in the BDD way rather than TDD. Think more about the behavior you are expecting instead of the methods you’d want to test (TDD).

    Just to give a concrete example, for the “tests in parallel” case, I’d start writing an executable spec like (in Java):

    public void shouldRunTestsInParallel() {
    List tests = stubOfTestsWithSize(9);
    Server[] servers = new Server[] {server1, server2, server3};
    ServerPool pool = new ServerPool(servers);
    ParallelRunner runner = new ParallelRunner(pool, tests);

    assertThat(server1, haveRun(3).tests())
    assertThat(server2, haveRun(3).tests())
    assertThat(server3, haveRun(3).tests())
    }

    It is there, only to drive your development (Behavior **Driven** Development). I don’t have yet any idea of **how** to do, but it only says **what** should be done and how it should work (program to interfaces/behavior: find’em first).

    Once you have this high level spec, it usually starts becoming clear on your mind what should be done. Then, you can start writing low level specs (fine grained) and start doing red-green-refactor cycles, until your high level spec pass.

    Real BDD, in practice! Nowadays I actually believe that BDD is the right way to do even spikes.

    Fabio Kung

    August 26, 2009 at 12:15 pm

  2. The general rule is: don’t write any code without a spec (test) forcing you to do so.

    Fabio Kung

    August 26, 2009 at 12:20 pm

  3. I get the idea now. The coarse grained spec would have helped me to drive my work towards what I wanted to achieve – it makes sense.

    What would happen in that case, as the test would involve a lot of infrastructure that was never used before in the project (starting up multiple servers and so on). I guess it would take me a few hours just to make that test pass if that was to be considered an integration test. Or should it be a more component-focused test?

    In another example right now, there is a tool I have been creating which I did not know what it was for. I just knew it was supposed to help evaluating some math systems in a parallel way. It started with erlang and moved – really fast – to ruby + java + loads of javascript (and erlang coming back soon, or maybe Scala?). Should I have created a XXX-language (which one?) from the start in order to test that the system was being solved in parallel?

    public void shouldSolveSystemInParallel() {
    List initialConditions = stubOfSystemWithDifferentStartingPoints(9);
    Server[] servers = new Server[] {server1, server2, server3};
    ServerPool pool = new ServerPool(servers);
    ParallelRunner runner = new ParallelRunner(pool, tests);

    assertThat(server1, haveRun(3).parts())
    assertThat(server2, haveRun(3).parts())
    assertThat(server3, haveRun(3).parts())
    }

    It does sound as an integration test (Server, Pool and Runner being really instatiated).. looks like I would go for this test and write the more specific ones (mocking each other component) before starting. Thats it?

    Anyway, which language should I have written the tests in? Right now I need to write code in every language for every part of the program

    If that was the case, I can see it evolving to what it is right now: solving it @ appengine with different clients (in different languages) consuming its output.

    guilhermesilveira

    August 26, 2009 at 12:47 pm

  4. I agree with @kung. The only exception to not doing BDD/TDD, IMHO, would be when using spike solutions. But these are very short (typically 1 day), just so I can experiment with some new concept I am unfamiliar with. And all code written will be thrown away afterwards.

    Like @kung said, for a project where I don’t know how I’m going to do something, BDD helps me to express what I want so I can keep focus and know when I’m done.

    Guilherme Cirne

    August 26, 2009 at 12:51 pm

  5. pt_BR:

    Fantástico post, Guilherme. (Aliás, seus posts tem sido muito bns.)

    “but what happens when I have no clue on what I am building?”

    Tempo atrás comecei uma discussão sobre criatividade e testes em uma lista de programadores que participo, e o Ronaldo Ferraz disse algo bem semelhante. Mais ou menos assim: “Primeiro você precisa saber o que quer fazer, senão, não tem como o design brotar dos testes” — já que TDD, em muitos aspectos, é sobre design, e não apenas testes por si só. O link da discussão é esse aqui:

    http://groups.google.com.br/group/programming-talk/browse_thread/thread/0d65270f8e70633f

    Abraço!

    Leandro Silva

    August 26, 2009 at 1:44 pm

  6. Good post Guilherme and excellent comment Kung!!!

    Leonardo

    August 26, 2009 at 1:56 pm

  7. I also agree with @Kung. It is useful to have some kind of higher level Specs to guide the implementation towards your goal, even if you are not very clear about how you’re going to achieve it.

    I usually find that those initial coarse grained specs even help to clarify my ideas about what I’m trying to do.

    Great post and discussion!

    Vitor Pellegrino

    August 26, 2009 at 2:22 pm

  8. pt_BR:

    “Maybe typical web-based app’s won’t suffer of this problem because sprint plannings will help getting things clear in the developer’s mind.”

    Conversando com o Rodrigo Yoshima durante a Rails Rumble, ele me disse que viu não sei a onde (porque não me lembro… hehehe) que o próprio Kent Beck disse que para start ups ele não aconselha começar com testes, por causa do time-to-market e, sobretudo, por ser essencialmente um protótipo [do que realmente virá a ser se tudo der certo e o dindin entrar na conta].

    Bem, como expressei na discussão que linkei no comentário acima, concordo totalmente com você. Outras pessoas na citada discussão também tem a mesma opinião.

    Acho que em caso de protótipo, teste de conceito, concepções de Professor Pardal, não é muito prático (não digo que não trará algum valor futuro) começar com TDD.

    Acho que TDD faz muito mais sentido quando você tem uma idéia clara na mente daquilo que você quer fazer e tem certeza de que é possível fazer. Só então você começa a fazer o design da solução e testá-lo pra ver se está bom. Depois, refactoring, mais design e mais testes. E assim vai.

    “Em minha humilde opinião”

    Leandro Silva

    August 26, 2009 at 2:37 pm

  9. pt_BR:

    “In most spikes, I agree that we don’t know yet exactly *how* to build the thing, but you always know *what* you want. Define what means “to be done” writing a high level (coarse grained) spec which says your code is working and *what* it should do. When you pass these specs, the Spike is done.”

    @Kung, concordo com você. No meu projeto Sparrow aconteceu mais ou menos isso: Escrevi specs de alto-nível, quase um vislumbre do que eu queria — o que realmente me ajudou a organizar as idéias — e fui correndo atrás…

    “It also helps if you think in the BDD way rather than TDD. Think more about the behavior you are expecting instead of the methods you’d want to test (TDD).”

    Exatamente. Eu não tinha testes, mas tinha algumas espectativas, alguns critérios de aceitação, mesmo não sabendo totalmente o que eu queria e como implementaria.

    Essa é a mágica do BDD, que vai além do TDD — mas que não o anula, e sim, o completa. É isso que muita gente não entende…

    – Imaginar ou esperar um comportamento é uma coisa;
    – Fazer o design desse comportamento é outra.

    Gosto de pensar que BDD me ajuda a descobri o que eu quero; e TDD me ajuda a melhorar o design da minha solução.

    Leandro Silva

    August 26, 2009 at 2:51 pm

  10. Leandro… Kent’s position on start-ups is here:

    http://www.infoq.com/news/2009/06/test-or-not

    “When I started [JUnit]Max I didn’t have any automated tests for the first month. I did all of my testing manually. After I got the first few subscribers I went back and wrote tests for the existing functionality.” Kent Beck

    I experimented 2-3 times the same thing last years. I started 2 experiments with no tests just to evaluate marketing value. Some moments you see that the value is not on tests.

    rodrigoyoshima

    August 26, 2009 at 4:59 pm

  11. I know that build something which, even you don’t know how it should work, is not an easy task, but I would do something like Kung suggested, and start writing a behavioral test (as he said, you know at least the expected behavior). I am not sure if you will create a very good test from the beginning, but you can always refact and adapt your test code (why not?). The important thing is that you will not write code without a test (as your brother says: all code is guilty until proven innocent – he says something like that, don’t he? :-).

    In your specific case, you were building a tool which the main functionality was to integrate with other tool. I have a particular case like that: a small tool which I built a few years ago, and should integrate with a very old tool. I started the development without using any kind of automated tests (because apparently it was too hard to implement – shame on me! (*)) and nowadays every time I need to update it, I freak out! So my advice is: write an unit test (yes, mock everything up). After that (when you already know how your tool and code are gonna work), write a integration test (automated if possible).

    The problem when you write code without a test is that you get used to it! 🙂 You will do it more and more, and then it will be hard to write tests for everything later!

    (*) In my defense, it was impossible to automate my test, because I needed an external hardware to get to work and exchange information to that tool. So, all my integration tests were manual! 😦

    Mauricio Aniche

    August 26, 2009 at 5:14 pm

  12. […] When I am unable to TDD… « Guilhermesilveira's Blog guilhermesilveira.wordpress.com/2009/ – view page – cached as random as it gets — From the page […]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: