Sustainable C++ Testing

Riddle #3: Incomplete

We have a struct with data describing a book record. We focus on one operation - checking if the data provided is "complete".

#include <gtest/gtest.h>

struct Book {
  std::string title;
  std::string description;
  
  bool isComplete() const {
    if (title.empty() || description.empty()) {
      return false;
    }
    return true;
  }
};

TEST(BookTest, bookWithNoTitle_isNotComplete) {
  Book book;
  book.description = "My first book description";
  // title left empty
  EXPECT_FALSE(book.isComplete());
}

TEST(BookTest, bookWithNoDescription_isNotComplete) {
  Book book;
  book.title = "My first book";
  // description left empty
  EXPECT_FALSE(book.isComplete());
}

#include <gtest/gtest.h>

struct Book {
  std::string title;
  std::string description;
  std::string author;
  
  bool isComplete() const {
    if (title.empty() || description.empty() || author.empty()) {
      return false;
    }
    return true;
  }
};

TEST(BookTest, bookWithNoTitle_isNotComplete) {
  Book book;
  book.description = "My first book description";
  // title left empty
  EXPECT_FALSE(book.isComplete());
}

TEST(BookTest, bookWithNoDescription_isNotComplete) {
  Book book;
  book.title = "My first book";
  // description left empty
  EXPECT_FALSE(book.isComplete());
}

#include <gtest/gtest.h>

struct Book {
  std::string title;
  std::string description;
  
  bool isComplete() const {
    if (title.empty() || description.empty()) {
      return false;
    }
    return true;
  }
};

Book createCompleteTestBook() {
  Book book;
  book.title = "My book title";
  book.description = "My book description";
  EXPECT_TRUE(book.isComplete());
  return book;
}

TEST(BookTest, bookWithNoTitle_isNotComplete) {
  Book book = createCompleteTestBook();
  book.title = "";
  EXPECT_FALSE(book.isComplete());
}

TEST(BookTest, bookWithNoDescription_isNotComplete) {
  Book book = createCompleteTestBook();
  book.description = "";
  EXPECT_FALSE(book.isComplete());
}

#include <gtest/gtest.h>

struct Book {
  std::string title;
  std::string description;
  
  bool isComplete() const {
    if (title.empty() || description.empty()) {
      return false;
    }
    return true;
  }
};

TEST(BookTest, bookWithNoTitle_isNotComplete) {
  Book book;
  book.description = "My first book description";
  // title left empty
  EXPECT_FALSE(book.isComplete());
  
  // test sanity check
  book.title = "Title now present";
  EXPECT_FALSE(book.isComplete());
}

TEST(BookTest, bookWithNoDescription_isNotComplete) {
  Book book;
  book.title = "My first book";
  // description left empty
  EXPECT_FALSE(book.isComplete());
  
  // test sanity check
  book.description = "Description now present";
  EXPECT_FALSE(book.isComplete());
}

#include <gtest/gtest.h>

struct Book {
  std::string title;
  std::string description;
  std::string author;
  
  bool isComplete() const {
    if (/*title.empty() || description.empty() ||*/ author.empty()) {
      return false;
    }
    return true;
  }
};

TEST(BookTest, bookWithNoTitle_isNotComplete) {
  Book book;
  book.description = "My first book description";
  // title left empty <-- but also author empty
  EXPECT_FALSE(book.isComplete()); // passing because author is empty
}

TEST(BookTest, bookWithNoDescription_isNotComplete) {
  Book book;
  book.title = "My first book";
  // description left empty <-- but also author empty
  EXPECT_FALSE(book.isComplete()); // passing because author is empty
}

Can you spot a weakness in the tests?

Give me a hint

What if the book is to get an extra property?

Give me a hint

What if this extra property would participate in the completeness check? Show me

Reveal the answer

Diagnosis

In case Book was extended with an extra property (e.g. author) participating in completeness check, both tests would no longer be testing what they promise. They would be creating a Book without an author, and such Book is now considered incomplete. Consequently, we are free to break the completeness check by removing the title or description from it, and the tests would continue passing. Show me
Maybe there is a way to write the tests so that they are not sensitive to unrelated changes in the completeness check?

Reveal the remedy

Remedy

Verify test sanity. After perfoming the test itself, you can provide the field you intended to skip and verify the Book is considered complete. Such check is not really part of the test case, more of a test sanity check. Show me
In this solution, adding the extra property will make the tests affected by it fail automatically, prompting the maintainer to review them.
Reuse a Book instance template. Create a reusable template of a Book, which is meant to be complete. In individual tests, start by cloning that template and explicitly "destroying" only one concrete property the test is focusing on. Show me
In this solution, adding the extra property will trigger the expectation failure in createCompleteTestBook and that's the only chunk of test code the maintainer will have to adapt.
Especially for bigger structs, this approach has the added benefit of hiding details irrelevant for the test (populating of "all other fields") and reducing code duplication at the same time. Furthermore, we turn comment from the original code into a piece of code, and now we not only communicate our intent to the reader, but also employ the runtime to guard it.

Feedback on this riddle? 3c6120687265663d226d61696c746f3a666565646261636b407375737461696e61626c6563707074657374696e672e636f6d3f7375626a6563743d5375737461696e61626c6520432b2b2054657374696e672c20726964646c65233320666565646261636b223e446f2067657420696e20746f756368213c2f613e