Must I trust the contract?

A lot of my work is about testing the Mobile App with the back end. A feature on the Mobile App is communicating with the back-end to exchange information. Input from the user on the Mobile App is processed, send to the back-end and a response is given back.
These interactions are covered in the REST Service Contract Specification for a specific feature. It will not be a surprise to you that there are a lot of these specifications you need to test.
A test approach for this can be that you create a stub/driver to simulate your back-end. You can manipulate the back-end data to test your mobile app.
When the back-end is available, you can use the backend and put a proxy in between to intercept/manipulate the data exchanged between the mobile app and the backend.
For the context of this blog, it does not matter which approach you use for your tests. I just share my experiences about what I encountered testing a REST Service Contract Specification.
A business rule in a REST Service Contract Specification
“Messages are returned in reversed chronological order (latest first). If message has receivedDate (received messages only), then it is used for sorting, otherwise originationDate is used.”
Looks nice. So the back-end is taking care of the sorting. The Mobile App receives this list of messages. It only has to display the latest one to the user. The devs built the software for this feature such that list is not sorted, because it is already sorted. Can I trust this? No, but I have to trust this. The devs state that we must trust the contract. It is their (the back-end team) responsibility to send a correctly sorted list.
Looking further in the REST Service Contract Specification, if a variable which is part of a JSON response and is of type 32 bits integer (unsigned), for sure you can do some boundary tests on this. Again the devs state that the back-end is not sending an integer which is bigger than 32 bits (i.e. 2^32 = 4294967296). The backend must take care of that. But do they? I don’t know. Can I trust this? No, but I have to. This example is a bit easier to test. For this variable, I just force an integer into the JSON statement which is bigger than 32 bits. What I experienced is that the App did not crash. I observed something else. The Android version of the Mobile App acted differently than the iOS version of the Mobile App. This is the best thing I can do, check if the Mobile App crashes in extreme error situations like this. Or look for different behavior between the OS. For this specific variable, you can do more tests than only ‘sending’ a big integer. You can put a string in the variable , give no value to the variable or remove the variable from the JSON message.
As a tester, I will do these tests. Especially on variables for which I think they are critical and interpreted as a risk. In this situation, I tested a JSON message which contained a variable called ‘content’. This variable ‘content’ contains a message which must be displayed to the user during login. The test I did was that I removed the variable ‘content’ (with message) from the JSON message. The Mobile App accepted the JSON and try to display the message. It showed a screen with an error message coming from the Mobile App itself. When I clicked the error message ‘away’ I saw in the log file a request being send to the back-end to lower the ‘read message counter’ and filed the message as seen by the user (I will not be shown the next time the user logs in). The user did not read the message, because there was no message. A serious bug from legal point of view. The devs was glad I performed this test, but still saying that this is an exceptional case and will not occur, because of “the contract” etc ….
I can agree with the devs. In their context, the REST Service Contract Specification is an agreement where the back-end must comply to. Therefore the Mobile App does not have to build all kinds of safety guards which is time-consuming, cause a lot of overhead and is error prone. It is a risk assessment which make sense. They trust the contract. I’m a tester with a disruptive mind. I don’t trust it, but I can’t test everything. I also make an assessment to identify risk areas and deal with it with my test approach. To see if I can crash the App or expose differences between the iOS and Android operating systems. So, must I trust the contract?

One Comment Add yours

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s