Unit testing

[asrael]

Hi.

I'm wondering if and how it would be possible to unit test code units.

Practically a separate main file file could serve for unit testing if the rest of the code is available structured in units as includes or such.
But does the IDE help there?
What's the proposed strategy?


Manfred

[Daedalus]

Using include files containing procedures would let you test most functions that purely do calculations by printing the results of known inputs. I don't think the IDE itself will help much here though, it would be up to you to create a skeleton program to run the procedures. Something like:

[ab3]INCLUDE "Calculations.ab3"

For i.l=1 To 100
Print "Result ", i, ": "
Nprint CalculateTemperature{i / 100}
Next i
NPrint "End of test"
End[/ab3]

For testing a routine in your "Calculations.ab3" file that converts a fraction between 0 and 1 to a temperature.

[asrael]

Yeah, or something like this:
[ab3]INCLUDE "Unit1.ab3"

Function TestDoSomethingOK{}
in$ = "foo"
result$ = DoSomething{in$}
NPrint result$

ret = 0
If result$ = "foobar"
ret = 1
EndIf

Function Return ret
End Statement

Function TestDoSomethingFail{}
in$ = "foo"
result$ = DoSomething{in$}
NPrint result$

ret = 0
If result$ = "foobuzz"
ret = 1
EndIf

Function Return ret
End Statement


If TestDoSomethingOK{} = 0
NPrint "ERROR in DoSomething"
EndIf
If TestDoSomethingFail{} = 0
NPrint "ERROR in DoSomething, wrong result!"
EndIf[/ab3]

Is there a function for exiting from program, like Exit()?
That could be used if a test fails, instead of checking a return val.

Manfred

[Daedalus]

Looks good.

Well, there is End d0, which returns the value in the D0 register as a normal AmigaDOS return value. This can be set with PutD0:
[ab3]If errorflag
PutD0 10 ; Error return code 10
End d0
End If
End ; No return code[/ab3]
However I believe this is a new feature of AmiBlitz 3 and so isn't possible under Blitz Basic 2. Still, might be useful for testing...

[asrael]

Yeah, ok.

Creating an include file with static Assert procedures which basically test one value against another and a comment shouldn't be that hard.

Manfred

[Der Wanderer]

You can do it like this:

Put each module into a separate include file, like I did for the AB3 includes.
You can write the unit tests directly into the include file guarding with "CNIF #__include = 0 ... CEND", so that the code will only be compiled if the include is the main code. If the include is actually included, the unit test code will be ignored.

[ab3]XINCLUDE "other_dependencies.include.ab3"

; ... my modules code ...
Function.s my_ModuleFunc{}
Function Return "foo"
End Function

CNIF #__include = 0
NPrint "Here comes the unit test for \\__THIS_INCLUDE..."
!TEST_EQ{my_ModuleFunc{}, "foo"}
!TEST_NE{my_ModuleFunc{}, "bar"}
NPrint "Unit test successfully passed."
End
CEND[/ab3]

You can write your own test macros like this
[ab3]Macro TEST_EQ ; { a = b }
If (`1) = (`2)
NPrint "\\__THIS_INCLUDE Unit test successful: \\22`1\\22 = \\22`2\\!"
Else
NPrint "\\__THIS_INCLUDE Unit test failed: \\22`1\\22 = \\22`2\\!"
PutD0 -1 ; exit with error code -1
End D0
End If
End Macro[/ab3]

It is also a good practice to add parameter tests to your functions. You can eliminate the overhead if you guard them with the #__debug constant, so they don't slow your release executable down:

[ab3]Macro _ASSERT ; {condition}
CNIF #__debug
If ((`1)=0) Then NPrint "\\__THIS_INCLUDE/\\__THIS_FUNCTION(\\__CODELINE): Assert failed: \\22`1\\22!" : End
CEND
End Macro

Function.l DrawObject(*myobj.object, x.l, y.l}
!_ASSERT{*myobj >< Null}
!_ASSERT{x >= 0}
!_ASSERT{y >= 0}
...
End Function[/ab3]

[asrael]

That's actually quite nice.

Would be great if those TEST/ASSERT macros could be put into a common include shipped with AB3.

I'm not so sure about running the test only when it's not included.
I could imagine having a test suite where multiple modules are included and run together.
Would it be possible to define a global var __IsTestSuite or similar where the test is still executed even if included.


Manfred

[Der Wanderer]

You don't want a global var for various reasons, mainly because the code will be created even if the variable suggests no test.

Here is an idea:

Assume our include does md5 computation. We call the include md5.include.ab3. Now, we create another file called md5.unittest.ab3.
For now this is just a regular ab3 program that does this:

[ab3]XINCLUDE "unittest.include.ab3" ; here come the Macros from...
XINCLUDE "md5.include.ab3" ; this is what we want to test

NPrint "Unit test for md5.include:"
!TEST_EQ{md5{"Hello World!"}, "1B42C4473A0E264F12FF"}
NPrint "Successfully passed."
End[/ab3]

Now we can add a menu entry "Run Unit Tests", next to "Create Executable".
Assume you are writing a program using md5 include. If you hit "Run Unit Tests" it would scan for the used includes and compile&run, if any, their unit tests and report failure/success, just like a compiling error.
This would be fairly easy to add to AIDE at least, since it scans the Includes anyway, and no modification needs to be done to the compiler.

*Note: Amiblitz3 Development is typically not on a professional level. Nobody really writes unit tests so far to safe time (I know this doesnt pay off in the long run).
If we introduce a new concept this might be difficult for AB3 users. Making this really simple and replacing the Includes Demo with a unit test might be sufficient.
What do people think?
Unit tests are also a good way to give usage examples, pretty much like the internal demos do already, just more comprehensive.

[asrael]

This looks pretty good to me.

Yeah, unit tests are good for a couple of reasons.
Also, given a more strict TDD approach, when you write the test first, you have to think more close about what you actually want to implement and that gives a new perspective.
Unit test covered code normally is cleaner, more modular with less side-effects.

Manfred