Testing styles in ScalaTest

Over time different libraries brought different testing styles, jUnit, xUnit, Rspec, ert. Nowadays since many people have different backgrounds and different taste when it comes to writing unit tests, most test libraries offer many testing styles and ScalaTest isn’t an exception. To me there are three groups of testing styles. One would the traditional jUnit, xUnit style where you define a test class and unit tests are methods in that test class, you also define setUp and tearDown methods that must be called before and after each unit test. Of course there are also setUp and tearDown versions that run once per test suit but I don’t want to talk more about it here. I must admit that this isn’t my favorite style and here is why. I like to unit test aggressively without skipping any business cases instead of just relying on code coverage and saying my other test already is hitting those lines. If I have a class with many methods I write many unit tests for each method, an example would be when a method takes a parameter and based on different values for those parameters it does different things, of course we want to have multiple unit tests for that method to cover all business cases for that method. So this alone groups unit tests in a test class by functions, another thing is that most likely unit tests for function A will share some setup logic. This testing style forces as to put setup code for all unit tests for function A in setup method, and when we have like 10 such functions, setup method will have the setup code for all those function which will make it long, hard to read, also it will be extremely hard to tell which setup code is for which unit test. Another option is to define setup code in each unit test with isn’t really a good idea since one we’ll have duplicated code and unit tests now will be longer than they need to be and hard to understand. I see some people like to solve this problem by moving this common code into private methods. Although this solves code duplication problem, also makes unit tests shorter but it introduced another big problem. When I read a unit test I want to know what the method is doing in 2 seconds, unit tests are documentations right. Now you come across this private method call, you have to find it’s implementation, read the ugly code there and jump back to continue reading what that unit test is trying to test. It’s a nightmare to me, I call that unit test anything but readable and I sure don’t want to be one maintain that code. Before I moved to next type of unit tests, I must mention what styles ScalaTest is offering here. FunSuite and FlatSpec, you can read more about those in ScalaTest documentation. Both are flat as mentioned, one is more like jUnit, other is more like xUnit.

Next group is property based testing or table based testing. I find this useful to test utility methods I write. Let’s say we’re writing absolute method but before doing it we want to have following tests:

absolute(0) = 0
absolute(1) = 1
absolute(-1) = 1

If we write this in flat of BDD style we have to write 3 unit test methods. What I like about table based testing is that we can define a table with all possible inputs and outputs and we write one unit test that just iterates over those. Less code is always a good thing right? Here is an example how this can be done in Scala test:

class AbsoluteTest extends PropSpec with TableDrivenPropertyChecks with Matchers { val examples = Table( ("input", "result"), (0, 0), (1, 1), (-1, 1)) property("math.abs should return the absolute value") { forAll(examples) { (input, result) => math.abs(input) should be(result) } } }

And third group is BDD style. What I like about this isn’t actually the BDD aspect, I like that you structure your tests hierarchically instead of flat. In my example about multiple unit tests per method, logical groups become nodes in that hierarchy and the root is the name of class we’re testing. Another nice thing about this is that each group can have it’s setup logic. And that hierarchy doesn’t have to be two levels deep, it can go as deep as you want. ScalaTest has following testing styles for BDD. FunSpec, WordSpec, FreeSpec, Spec and FeatureSpec. You can read about those in ScalaTest documentation, I will cover WordSpec here since it’s my personal choice. Let’s say we’re working on some code where we need to create user in db based on user’s facebook token. There is FacebookClient which has getUserInfo method that returns future of option user, it will have user when token is valid and it will return None if token isn’t valid for the user and we’re not able to get user info from facebook. Also we have Db trait with saveUser and findUserById methods. Both return future of option. If we’re not able to save user to db we return None, if we are, we return user with id generated from db. For find user we return user if we find it and we return None when user isn’t found. Those are the cases we want to test:

For createUser, it should call db to save user and return saved user when token is a valid facebook token
For createUser, it should NOT call db if token isn’t valid and return None
For createUser, it should return None if unable to save to db
for getUser, it should return user if user is found in db
for getUser, it should return None user isn’t found in db

Here are the tests in ScalaTest using WordSpec and the implementation:

case class User(id: Option[Int], name: String) trait FacebookClient { /** * Gets user info from facebook. * @param token The facebook token. * @return Returns User if token is valid, None otherwise. */ def getUserInfo(token: String): Future[Option[User]] } trait Db { /** * Save user. * @param user The user. * @return User if it's able to save to db, None otherwise. */ def saveUser(user: User): Future[Option[User]] /** * Finds user by user id. * @param id The user id. * @return User if user is found, None otherwise. */ def findUserById(id: Int): Future[Option[User]] } class UserManager(facebookClient: FacebookClient, db: Db) { def createUser(token: String): Future[Option[User]] = for { fu <- facebookClient.getUserInfo(token) dbu <- fu.map(db.saveUser).getOrElse(Future(None)) } yield dbu def getUser(id: Int): Future[Option[User]] = db.findUserById(id) } class UserManagerTest extends WordSpec with Matchers with ScalaFutures with MockFactory { val facebookClient = mock[FacebookClient] val db = mock[Db] val userManager = new UserManager(facebookClient, db) "The user manager" when { val userId = 1 val user = User(None, "some user") val dbUser = user.copy(id = Some(userId)) "createUser" should { val token = "token" "call db to save user and return saved user when token is a valid facebook token" in { // Arrange (facebookClient.getUserInfo _).expects(token) returning Future(Some(user)) (db.saveUser _).expects(user) returning Future(Some(dbUser)) // Act and Assert userManager.createUser(token).futureValue should be(Some(dbUser)) } "NOT call db if token isn't valid and return None" in { // Arrange (facebookClient.getUserInfo _).expects(token) returning Future(None) // Act and Assert userManager.createUser(token).futureValue should be(None) } "return None if unable to save to db" in { // Arrange (facebookClient.getUserInfo _).expects(token) returning Future(Some(user)) (db.saveUser _).expects(user) returning Future(None) // Act and Assert userManager.createUser(token).futureValue should be(None) } } "getUser" should { "return user if user is found in db" in { // Arrange (db.findUserById _).expects(userId) returning Future(Some(dbUser)) // Act and Assert userManager.getUser(userId).futureValue should be(Some(dbUser)) } "return None user isn't found in db" in { // Arrange (db.findUserById _).expects(userId) returning Future(None) // Act and Assert userManager.getUser(userId).futureValue should be(None) } } } }

As you can see tests are organized hierarchically, setup code for all the tests is in test class, setup code for each group is in it’s group, and setup for each test is in it’s test. This is nice and clean and very readable. Unfortunately this patter doesn’t always work, mainly when you need to do clean up for each test. One example would be unit testing actors. Let’s say instead of having this FacebookClient, Db, UserManager and future model we have actors: FacebookActor, DbActor and UserActor. To test UserActor we need to have test probes for FacebookActor and DbActor. We can’t put that code outside test functions because we need to create the actors and stop them in every test. What we can use here though is fixtures in ScalaTest.

case class User(id: Option[Int], name: String) object FacebookActor { case class GetUserInfo(token: String) case object TokenNotValid } object DbActor { case class CreateUser(user: User) } object UserActor { def props(facebookActor: ActorRef, dbActor: ActorRef): Props = Props(new UserActor(facebookActor, dbActor)) case class FindUserById(id: Int) case object UserNotFound case class CreateUser(token: String) case object CantCreateUser } class UserActor(facebookActor: ActorRef, dbActor: ActorRef) extends Actor { import akka.pattern.{ ask, pipe } import context.dispatcher import UserActor._ implicit private val timeout: Timeout = Duration(10, SECONDS) override def receive: Receive = { case CreateUser(token) ⇒ facebookActor.ask(FacebookActor.GetUserInfo(token)).flatMap { case user: User ⇒ dbActor.ask(DbActor.CreateUser(user)) case FacebookActor.TokenNotValid ⇒ Future(CantCreateUser) }.pipeTo(self)(sender()) case msg: FindUserById ⇒ dbActor forward msg case CantCreateUser ⇒ sender ! CantCreateUser case msg: User ⇒ sender ! msg } } class WordSpecDemo2Test extends TestKit(ActorSystem("test-system")) with ImplicitSender with WordSpecLike with Matchers with BeforeAndAfterAll { import UserActor._ import FacebookActor._ "The UserActor" when { val userId = 1 val user = User(None, "some user") val dbUser = user.copy(id = Some(userId)) "getting CreateUser message" should { val token = "token" "respond with CantCreateUser if token isn't valid" in withActors { (userActor, facebookProbe, dbProbe) ⇒ userActor ! CreateUser(token) facebookProbe.expectMsg(GetUserInfo(token)) facebookProbe.reply(TokenNotValid) dbProbe.expectNoMsg() expectMsg(CantCreateUser) } "respond with CantCreateUser if DbActor responds with CantCreateUser" in withActors { (userActor, facebookProbe, dbProbe) ⇒ userActor ! CreateUser(token) facebookProbe.expectMsg(GetUserInfo(token)) facebookProbe.reply(user) dbProbe.expectMsg(DbActor.CreateUser(user)) dbProbe.reply(CantCreateUser) expectMsg(CantCreateUser) } "respond with user after creating user in db" in withActors { (userActor, facebookProbe, dbProbe) ⇒ userActor ! CreateUser(token) facebookProbe.expectMsg(GetUserInfo(token)) facebookProbe.reply(user) dbProbe.expectMsg(DbActor.CreateUser(user)) dbProbe.reply(user) expectMsg(user) } } "getting FindUserById message" should { "respond with user if user is found" in withActors { (userActor, facebookProbe, dbProbe) ⇒ userActor ! FindUserById(userId) dbProbe.expectMsg(FindUserById(userId)) dbProbe.reply(dbUser) expectMsg(dbUser) } "respond with UserNotFound if user isn't found" in withActors { (userActor, facebookProbe, dbProbe) ⇒ userActor ! FindUserById(userId) dbProbe.expectMsg(FindUserById(userId)) dbProbe.reply(UserNotFound) expectMsg(UserNotFound) } } } override protected def afterAll(): Unit = system.shutdown() private def withActors(testCode: (ActorRef, TestProbe, TestProbe) ⇒ Any): Unit = { val facebookProbe = TestProbe() val dbProbe = TestProbe() val userActor = system.actorOf(UserActor.props(facebookProbe.ref, dbProbe.ref)) try { testCode(userActor, facebookProbe, dbProbe) } finally { system.stop(facebookProbe.ref) system.stop(dbProbe.ref) system.stop(userActor) } } }

As you can see fixtures makes it as simple as it was. All the setup logic is still where it needs to be and boilerplate to setup and stop the actor and test probes is centralized.