Most common reasons why TDD fails!

I believe TDD is just a common sense. TDD gave me back the joy of programming.
I'm a mechanical engineer & I turned to programming because I love doing it. However soon after I started working in industry; my excitement turned into frustration.
I'm not alone with this experience; because most of us work on Legacy Code.


Doing TDD has made me more productive. The stress I used to experience in earlier way of working no more exists. Code no more comes into my dreams & spoil my sleep. I can get creative & aggressive; experiment different approaches. It's because of that great feeling :- My code is protected by Tests.... The tests that run fast & give me immediate feedback what I have done.


In spite of this; I wonder why TDD finds so hard to get accepted in programmers' community. When I give little thought over this; it seems obvious. Even when I started TDD; I failed; I failed many times. It's a awkward concept. Awkward because most of the programmers are not used to work this way. And changing the way of working always faces resistance.


In this presentation I'm sharing the hurdles that come in way to TDD. Overcoming these hurdles takes efforts; but its possible; and its worth for the benefits it gives. Hope you find this useful.


I'm writing a series of posts on this blog about my experience with TDD in not just with Programming but in many other things.

• My Experience with TDD
• TDD and Science
• TDD and Product Definition
• Want to get rid of Beurocracy; then you need TDD!

Best place to learn TDD (in my personal opinion) is the book "Test Driven Development by Example" authored by Kent Beck and the ObjectMentor Blog. In this blog I'll be mainly talking about different dimensions of TDD.

Your views are welcome.

You want to get rid of Beurocracy? Then you need TDD!

• Are you complaining that Beurocracy has got crept in your organization over the period of time?
• Do you think processes are overheads & they are killing your Creativity as well as Productivity?
• Are you annoyed by the "Process People" eating up your time & beating you with some nonsense metrics?

If yes; then it's just natural.
Its just like the code we write. We design "Process" as stringent as possible. As orgnaization grows old; people stop following them. Not only that; the needs change; hence the processes become redundant. We hardly bother to refactor them. Nothing different than the legacy code.

Try adopting a TDD approach.

• Process makers should not allowed to design any process unless they write a "failing test". This failing test should be approved by some one senior.
• Example: If you are making a security process then may be we have a stroy like following:
• Tom is programmer who doesn't bother to lock his machine; when he is not around. On a particular day at lunch time; some one opens his machine & use his account for malicious purpose. This should not happen.
• They design the process that is just enough to make the test pass.
• The design should not be more stringent than necessary ... just minimal.
• As they write & satisfy more tests; they should refactor their process design.

TDD and Product Definition

Are you in a role of designing products (not necessarily a software product) and your job is to write Functional Specifications?

If yes; have you came across an article called "Painless Functional Specifications" by Joel Spolsky?

If yes; then you are already doing TDD even though you may not be a programmer!


If you haven't read this article I will urge you to read it first. Its concise, highly useful & written in an interesting (Joel's) style. I'm sure you will thank me after reading it.


I think TDD is the most sensible way to write functional specifications.

• The first step in writing specifications is to write Stories. User Stories with names that highlight how user will be benefited.   
• Something like ... Stacy is a Project Lead in a software developer company. She uses an issue tracking tool called FastTrack (disclaimer: if a tool by such a name exists; then it's a mere coincidence).On the dashboard she can see all th issues assigned to her. However there are dozens of issues on her name; but at present she is interested only in seeing a handful issues which are work in process. The tool can provide a wealth of data; but that's overwhelming to her.
• This user story is like a "Test".
• To start with you haven't designed the product. So user is not able to do what he intended. So the test fails. A very important step in TDD: See the test fails!
• Then you design your user interfaces that is just enough (yes, just enough) to make the story (I mean the test) pass.
• Then you go on to next story. You may have to refactor you interfaces while satisfying more & more stories.

Looks simple; isn't it? But the advantages are great!

• If you are into the business of making products; you might have experienced there are just a lot of redundant features in the product. Many times we add them because...
• 
  
  We want to make the product rich with features. More features means there is more stuff for the sales people to talk about & impress.
  
• 
  
  We think that might be useful to the user.
  
• 
  
  or we add it just because we can add it; we have the technology available.
  

• Users don't really want features; they want to get their job done. Unwanted features only confuses them.
  
• 
  
  If our product is old; then maybe we are adding features for many years together. We just go on adding features. Hardly we bother we reorganize the interfaces; because of many reasons. This is similar to programming. We go on adding code; we don't bother to remove code or refactor it. Why? Because many times we don't understand it; its written by someone else & its working. Also the code is being called; we don't want to break any existing users.
  

TDD lets you to free from all these problems.

One just need to have following descipline:

• I will not write a single line of specification ( or user interface) unless I describe a failing User Story.

• I will design the user interface that is just enough to satisfy the above story!

Once I came across a quote that was placed in email signature of one of my colleague:

A Design is complete not when there is nothing to add more; it's when there is nothing left to take out from it!

TDD encourages us to do the same thing; right?

• TDD Homepage
• TDD and Science
  

My Experience with "Test Driven Development"

I first heard about the word "TDD" from my senior colleague Nitin Bhide a few years back. He also conducted a hands-on session for us on how to do TDD using cpp-unit.

The first reaction I had (though I didn't tell this to Nitin) was…

"Its interesting, but is suited for people who have a lot of time to spare and no delivery pressures. It's impractical & really awkward to do it".

However for some reasons I continued doing it (perhaps somewhere I liked it but was not able to convince myself). And … The more I did it; it made more & more sense doing it.

Today I feel "TDD" is one of the most scientific ways of developing software. In fact I think the TDD concept is not born in the software world at all. It is being used by Scientists, Product designers, people making policies & processes, history scholars (and many more…) since long time.

I must admit that I have not read any prescribed book on TDD. So what I say is not necessarily correct. I would greatly appreciate if someone corrects me if I go wrong.

My views are based on some inspirations, some online reading & numerous mistakes I made while practicing it over the years.

I owe a lot to following people who helped me in this journey:

• Nitin Bhide : from whom I first heard about TDD.
• Kent Beck : who taught me the rhythm of RED - GREEN - REFACTOR through his beautiful book "Test Driven Development by Example", which has greatly influenced my way of programming
• Robert C Martin : to whom I'm grateful for Design Principles, Design Patterns Unit testing, TDD and many other things I could learn.
• Michael Feathers : who taught me who to make it work even for legacy code. (through his book "Working Effectively with Legacy Code")

(It would be a great day for me if I could meet these two gentlemen.)

• Tathagata Chakraborty : my colleague who renewed my interest in TDD & gave me confidence that I'm thinking in right direction.
• Last & not the least "Dr. Goldratt" who gave me an entirely different line of thinking.

Keep reading...

• How TDD & Science are the same?
• Are you a Product Designer and want to write painless functional specifications? Then you must do TDD
• Are you frustrated by beurocracy in your organization & want to change it? Then you need TDD!

TDD and Science

"The Goal" has influenced my life, the way I think & act.
However I think what's even more valuable than the book itself is the preface written by the author; where he has expressed his views on Science and Education. It's just a two page preface; that gave me some interesting line of thinking.

 What is Science?
Is it about "solving the mystries of the universe" or "knowing the facts"?
No, science is about "postulate a minimum set of assumptions that through logical deduction explains the things happening around us".

This is a groundbreaking definition... it frees Science from being locked in fixed categories like Phisics, Chemistry, Biology. There is a science exists beyond it ... that explains how humans think, how organizations behave etc.


TDD (Test Driven Development) is just an application of above definition to the world of "Software Developement".
Are you wondering how? Let me take a real example.


There is something that has fascinated mankind from beginning of the life ... that's light.
Can we tell "What is the Light ?"
Do we "know" what light is? Perhaps the answer is "Not sure"... even today.
What we have is a theory which has got evolved over last many decades.


After all what is "Theory"?  Let's put Dr. Goldratt's definition explicitly.

A minimum set of assumptions; that through straight forward logical deductions; explains things happening in the universe.

So let's put down the observation or behaviour of light and try to make a theory according to above definition:
------------------------------------------------------------------------
BEHAVIOR #1 : When light falls on an opaque object; it gets reflected.
------------------------------------------------------------------------

Now let's see if our theory can explains this.
No it cannot. Because the theory is empty. We haven't written any code yet.


So let's put a minimum code that is just enough (yes just enough) to explain the observed behavior:


-------------------- Theory of Light --------------------------------
ASSUMPTION :

• Light is made up of tiny particles called "Corpuscles".
• These corpuscles are emitted from the source in all direction.

LOGICAL DEDUCTION:

• When any particle hit an opaque surface it gets rebounded according to some law ---- [Fact]

-------------------- Theory of Light --------------------------------


Wow! the behavior is explained! The test has passed!
(I'm sure you have started noting the parallels)


Can we prove this? Yea....h..no.; not sure.
Can we disprove this theory?  Yes quite easy; just write another test that makes this code fail.
------------------------------------------------------------------------ 
BEHAVIOR #2 : When light enters from one medium to another medium its path gets deflected.
Phenomenon of Refraction.
------------------------------------------------------------------------


Can our theory explain this?
        No. So, the test fails.
Let's write some code that is just enough to make this pass.
Is it ok we write any "crap code" as long as the test is passed?
         No, remember the definition. We have to make a minimum set of assumptions or sensible assumptions and then it should be left to logical deduction to explain the behavior.


-------------------- Theory of Light (Revisited)--------------------------------

 ASSUMPTION :

• Light is made up of tiny particles called "Corpuscles".
• These corpuscles are emitted from the source in all direction.
• Light being a particle is affected by gravity.

LOGICAL DEDUCTION:

• Denser medium has more material per unit volume. Hence it can exert more gravitational force ---- [Fact]
• When light particles enter a denser medium they are pulled by the gravity. Hence their path gets deflected.

-------------------- Theory of Light (Revisited)--------------------------------
Cheers! The test is passed.


But a word of caution: Our set of assumptions are increasing now. Like our code is getting into more if's and  then's. It's getting somewhat complex.


I'm skipping here.
Reason #1 : I need to go faster since I want to make some point.
Reason #2 : I'm not a physicist and I don't want to showcase my ignorance about these theories :-)


Let's assume that we go on explaining everything by adding more assumptions to our theory. Our code has got more and more complex. Also some assumptions we made were found to be incorrect.


-------------------- Theory of Light (Revisited)--------------------------------
ASSUMPTION:

• Light consists of Waves.

LOGICAL DEDUCTION:

• I'll leave it to you :-)

-------------------- Theory of Light (Revisited)--------------------------------


Did you realize what we have done?  Yes; its most important step in software development : Refactor the code!
When we are doing TDD; we don't fear to refactor. We are protected by Tests! We can ruthlessly throw away many lines of code that we wrote & cherished for a long.


Now let this theory evolve.
------------------------------------------------------------------------

 BEHAVIOR #3 : Light can travel through SPACE

------------------------------------------------------------------------


Now the Wave Theory fails. At that time "Electromagnetic waves" were not known and it was believed that wave requires a medium to propgate.


-------------------- Theory of Light (Revisited)--------------------------------

 ASSUMPTION:

• Light consists of Waves.
• Space is filled up with a medium called "luminiferrous ether".
• This ether is having infinitely small density.

LOGICAL DEDUCTION:

• It's obvious.

-------------------- Theory of Light (Revisited)--------------------------------

And so on...


I'll stop here. I'm talking on something on which I don't have authority to talk. I'll greatly appreciate if someone brings to my notice the technical mistakes (if any) I have made.


The point is ... TDD & Science are exactly the same. Not similar; it's the same.
TDD is one of the most scientific ways to develope software.


NOTE: I came across a book called "A Quest for The Theory of Everything" by "Stephen Hawkings"; where the author makes the same definition for Science.


There are many objections made against TDD:
-------------------------------------------------------------

• In TDD everything is all well; as long as the tests are passed.

One can also write some crap code; break design principles; abuse OO etc. This threatens extensibility of the software in long run.

• Tests overspecify the behavior.
• Tests are unmaintainable.

-------------------------------------------------------------


I hope above approach answer above objections well.


TDD is not replacement for Architecture. It's not replacement for Code Review. Uncle Bob has explained extremely well on one of his recent posts on blog: http://blog.objectmentor.com/articles/2009/10/08/tdd-triage


We still need all these things.
Note the definition of theory says "It should make a minimum set of assumptions, or make only sensible assumptions". We just can't write any code just to make tests pass!


Then what's the advantage the TDD gives?
In my opinion, the biggest advantage is; it lets the programmer to get creative.


When a programmer gets creative?
 ... when he gets aggressive.
When he can get aggressive?
... when he can focus on the job or algorithm on hand... rather than worrying about broken code or regressions.
If you start with constraints; then there are fair chances you miss a good solution.
When this can happen?
... when your code is protected by Tests ... The tests that give you immediate feedback of what you are doing.


Do you find this interesting? Pl let me know.


Also pl visit following links to read more where TDD makes sense even beyond the world of Software Development.

• TDD Homepage

The Reason for Technology

Some time back I came across a really interesting video titled "The Reason for Technology" by Dr.Goldratt. I thought I would make a presentation for the people who missed it.
The Powerpoint Presentation can be found at:
https://docs.google.com/present/edit?id=0ARMKCVUTFyrrZGR4ODR3bTVfM3J0ZmJtY2Nt&hl=en

The ppt contains some of my interpretations but except that most of the contents are what I scribbled down while listening to the video.

 Summary:

• For a Technology wants to be successful; it must diminish certain major limitation.
• Before the limitation was diminished; the world was living with the limitation. So people framed certain rules to accomodate that limitation.
• Organizations adopted the new technology; but the rules didn't change.
• Effectively they couldn't get benefits from the new technology.
• Not only that organizations have inertia to change anything; its difficult to realize that these rules are to be changed. They are so sunk within ourselves that they have become "a common practice".
• For a technology to be successful; it's ncecessary that we locate such rules that are to be changed; and institute new appropriate rules.

If by any chance you get to see this video; pl don't miss it. Its a

wonderful experience to listen to Dr. Goldratt! The video can also be
purchased from the TOC site :
http://www.toc-goldratt.com/index_TV.php?cont=647&t=9&p=3

If you think of or come across any such examples of technology implementations; request you to share the same.

Thank you.

Enjoy reading...