Web Service Testing Automation using SoapUI Tool

International Journal of Computer Applications (0975 – 8887)

Volume 167 – No.2, June 2017

Meenakshi Garg

Department of MCA

VESIT

Mumbai, India

Niveta Lavhate

Department of MCA

VESIT

Mumbai, India

ABSTRACT

In today’s world, web services have become an integral part

of web. A web service is a service offered by one device to

another device electronically, for communicating with each

other over the World Wide Web. The web services can be

also act as an API which facilitates various services for

devices over network. Today majority of web applications

which keep running makes the use of web services and use

service-oriented architecture. A service-oriented architecture

(SOA) is a way of designing software where services are

provided to the other components by application components,

through a communication protocol over the network. The

testing of these web services should be considered as an

important factor now. Testing the web service manually

becomes complicated. The solution is test automation! Web

services can be tested by using test automation frameworks.

SOAPUI is such an open source tool which can be used for

testing of web services. SOAPUI provides a variety of testing

facilities. This tool provides the web service tester a modern

way through which he can save his time and perform testing

efficiently.

Keywords

SOAP, SOAPUI, REST, Automation testing, Web Service

Testing

1. INTRODUCTION

Test automation has become a trend in today’s world.

Automation tests are more efficient and exact when contrasted

with the manual testing. [1] There are various open source

tools available for web service testing. SOAPUI is one most

commonly used tool for web service testing. It can discover

the imperfections which can be easily missed out while

manual testing. It can be used for web service and API

testing. [2] It can provide instant and accurate results. Testing

is a basic necessity of any application since it gives chance to

measure the quality of programming. There is no other way to

check if the web service is serving its purpose or not. The web

service is not used only by a single system; it is used widely

by multiple systems. [3] It becomes necessary to test the web

services efficiently as they serve a large number of software

systems in order to guarantee the reliability. Web services do

not have any user interface which becomes another obstacle

in testing of web services. [4] Web service testing includes

functional testing, regression testing, load/stress testing and

monitoring. In any scenario, manual testing requires excessive

human efforts and it is prone to fail due to human mistakes.

[5], [6] whenever the programming code changes everything

needs to be retested. Manually testing it again and again

becomes tedious and unreasonable.

2. REVIEW OF SOAPUI

The SOAPUI appeared in 2005. The person behind its

existence is Ole Lensmar. [7] He was working in Eviware

software’s as an architect of a huge SOA project and he

realized that there is no test tool available for testing agile

development process and he decided to develop it. He worked

on SOAPUI in his free time and when Ole and his colleagues

realized what SOAPUI can do they made it available to other

people working with SOA and result was the open source

SOAPUI version got released. [7] Eviware Software was

acquired by SmartBear later in 2011.The open source

SOAPUI tool is licensed under European Union Public

License. This tool is downloaded 2,000,000 times since its

initial release. SoapUI NG Pro is a commercial version of

SAOPUI which focuses on additional functionalities and

enhanced productivity. The SOAPUI tool is completely

developed using Java platform which makes it cross-platform.

It makes use of Swings framework for user interface. IDEA,

Eclipse and Net Beans are also supported by SOAPUI at

today’s date. SOAP, REST web services, JMS, AMF can be

tested by using SOAPUI and it can also be used for making

HTTP(S) and JDBC calls.

3. CHOOSING THE SOAPUI TOOL

There will be a lot of reasons available for choosing SOAPUI

tool. SOAPUI is an open source tool which means it is freely

and easily available for use by anybody. This tool is also

available in a commercial version called SoapUI NG Pro with

extra functionalities for mission critical web services.

SOAPUI is considered as a standard for API testing in today’s

world. With this tool, functional testing, load testing,

regression testing, performance testing, interoperability

testing and many more things are possible. [8] It aims on

testing to be easy. For instance, we can just right click on

functional test and run it as a load test, it is that simple to use.

Web services can be simulated by using this tool. Test

recording and later reusing are most convenient features

available in SOAPUI. Code stubs from WSDL can be created

using it. Even the creation of the REST specifications

(WADL) from recorded communication is possible.

3.1 Platforms supported by SOAPUI

As the SOAPUI is java based tool, it can be easily run across

various operating systems, it has been tested with multiple

Windows versions, Mac and multiple Linux dialects. So, the

operating system will not be an issue for those who are

willing to make use of this tool. The basic requirements are

1.6+ versions of the JRE (Java Runtime Environment), at

least 1GB memory and 100MB of disk space which are again

not difficult to be available. [8]

4. SOAPUI TOOL IMPLEMENTATION

There is multiple ways for implementing the web services.

Two of the most common approaches are –

a) SOAP - (Simple Object Access Protocol)

b) REST-(Representational State Transfer architecture)

XML format is used by SOAP for sending and receiving

request and data is platform independent. Exchange of

messages between SOAP provider applications and receiving

applications happens in the form of SOAP envelopes. [9]

International Journal of Computer Applications (0975 – 8887)

Volume 167 – No.2, June 2017

SOAP messages do not get block by the firewall because it

makes use of simple http transport protocol. In case of REST,

its architecture runs over HTTP. REST uses simple URL in

some cases instead of using XML requests. REST requests are

faster and hence used in mobile and web applications. In

order to understand SOAPUI, it is necessary to have

knowledge about WSDL. WSDL is Web Service Description

Language. It is based on XML and used for describing the

services provided by the web service. All the operations

provided by a particular web service are described in a XML

format by WSDL. It also specifies that how the services can

be called i.e. what is the value for input and what response we

should get for each type of service.

So, in addition with functional testing SOAPUI also provides

non-functional testing such as performance and security

testing.

4.1 SOAPUI Features

Commonly used features of SOAPUI are listed below

4.1.1 Functional Testing

Testers are allowed to write functional API tests using

SOAPUI. It has drag and drop feature which makes the script

development faster. [10] SOAPUI allows test debugging. It

also provides support for data driven tests. It supports more

than one environment hence it is not difficult to switch

between QA, Production or Dev environment. It supports

advanced scripting which makes easy for testers to develop

their own custom code in necessary scenarios. Using SOAPUI

tester can use a large number of features. Some of them are

listed below

4.1.1.1 Drag and Drop

It is the best feature in perspective of new testers for building

test automation. [11] It speeds up the script development for

test automation.

4.1.1.2 Custom Scripts:

SOAPUI allows custom test script writing for test automation

scripts development which is an advance scripting option that

provides flexibility for writing the scripts for test automation.

4.1.1.3 Environment Switching

SOAPUI allows testing the web API by switching in

multiple environments such as UAT (User Acceptance

Testing), QA (Quality Assurance), SIT (System Integration)

etc.

4.1.1.4 Test Debugging

SOAPUI helps tester in debugging the test scripts and

allows user to write data driven tests. Tester can put

multiple check points to check that the test is getting executed

as expected or not.

4.1.1.5 Robust Tool

By considering all the above scenarios SOAPUI is a robust

testing tool for sure.

4.1.2 Compliance Testing

SOAPUI allows tester to validate the compliance tests and

automate those test scripts. Compliance testing consists of the

tests related to the source and the data authentication by using

license and certificates etc. It is also related to various service

level agreements.

4.1.3 Regression Testing

Regression testing is performed to analyze any

malfunctioning happening within the web services after any

new change is introduced. It is carried out to find out if new

changes are causing any problems in expected working of the

web service. [11] Using SOAPUI tool tester can execute the

test suits to check if the existing functionalities are working

fine.

4.1.4 Security Testing

SOAPUI can perform complete vulnerability scan set.

SOAPUI ensures authorization and authentication for request

and response web service model. Features that SOAPUI

supports under Security testing are listed below

4.1.4.1 Cross Site Scripting

It scans as well as detects any exposure related to service

parameters in structure message during cross site scripting.

4.1.4.2 Securing database

SOAPUI scans for any potential l SQL injection that could

cause possible harm to the database. Hence provides the

secure database.

4.1.4.3 Stack overflow

SOAPUI can scan and detect large documents and can find

out the stack overflow related defects.

4.1.4.4 Lifesaving scans

These scans are done to support unsteady behavior of some

APIs. SoapUI NG Provides fuzzy scans, boundary scans etc.

4.1.5 Load Testing

SOAPUI can be used to distribute the Load Tests across

multiple numbers of LoadUI Agents. It can simulate real

world and high volume load testing easily. To capture

performance parameters, it provides advanced custom

reporting options. End to end system monitoring is also

enabled by SOAPUI. The features provided by load testing

are:

4.1.5.1 LoadUI Agent

SOAPUI NG has something called LoadUI agents in large

number on which load can be distributed and performance

analysis, server monitoring and visualizations can be done.

4.1.5.2Monitoring Performance

SoapUI NG provides advance reporting system for

capturing various performance parameters in load testing.

Apart from this it also provides performance monitoring

for end-to-end system load testing.

4.2 Supported Protocols/Technologies

SOAPUI supports most comprehensive set of protocols which

is depicted in Fig 1.

Fig 1: SoapUI Supported Protocols

International Journal of Computer Applications (0975 – 8887)

Volume 167 – No.2, June 2017

SOAP –Simple Object Access Protocol

WSDL – Web Service Description Language

REST – Representational State Transfer

HTTP – Hyper Text Transfer Protocol

HTTPS – Hyper Text Transfer Protocol Secure

AMF – Action Message Format

JDBC – Java Database Connectivity

JMS – Java Messaging Service

4.3 SOAP-Integration

SOAP can be integrated with many tools like Maven, Hudson,

Junit, Apache Ant and many more. Few of them are described

as under -

4.3.1 Maven

Apache Maven is a tool for project management that can

manage the builds of project, documentation and repository

from a central location. SOAPUI tests can b e executed using

Maven with simple command.

4.3.2Hudson

Hudson is a java based continuous integration tool which

integrates with CVS, Subversion, Git, Perforce and RTC.

SOAPUI integrates with Hudson to find out the bugs that any

developer commits while developing the software.

4.3.3Junit

SOAPUI test flow can be controlled by Junit testing

framework which is used for testing modules.

4.3.4ApacheAnt

It is a command-line tool used for software building. Using

SOAPUI’s command line Ant automated tests can be

executed.

4.4 SOAPUI Overview

A brief overview of SOAPUI features is provided here which

includes [12] Menu Bar, Navigator, Interface Viewer, Request

and Response Panel, Test Suit, Properties Windows and

various types of Logs maintained by SOAPUI

4.4.1Menu B ar

The File menu which is depicted in Fig 2 has various options

for creating new projects for SOAP or REST requests. It also

has options to import existing project. Apart from this it

allows to open, close and save projects. [12] The file menu

provides us an option for setting global SOAPUI preferences.

All the menus in SOAPUI has tool tip text which tells what an

option does by hovering over it.

Fig 2: SOAPUI Menu Bar

The Project menu on the menu bar has various options

regarding projects. It allows to add WSDL and WADL (Web

Application Description Language) to an existing project. It

also provides an option for adding new rest service from URI.

This menu allows user to create the SOAP and REST Mock

Services. Apart from this it also allows to launch Test Runner,

LoadTestRunner, HTTP Monitor and Security Test Runner.

Suite, Case and Step menus are specific to a Test Suite, Test

case and Test Step respectively. They become active only on

selecting the Test suit, Test case or Test Step.

Tools menu allows doing required configurations for the

project. Finally, the Help menu provides address of helpful

resources privacy related terms of use and about the version

of software.

4.4.2File

As shown in Fig 3, SOAPUI tool has a visual and handy

structure of the File menu. Below the Menu bar there is

shortcut for File which allows single click solution for all

options present under the File menu. User can directly use

these options by clicking on it instead of going into File menu

again and again.

Fig 3: SOAPUI shortcut for File

4.4.3 Navigator

The Navigator filed on SOAP UI tool allows navigating

across various projects. Fig 4 shows a snapshot of SOAPUI

Navigator. Navigator allows switching between the projects

and services of the project.

Fig 4: SOAPUI Navigator

4.4.4 Interface Viewer

It provides the important information about SOAP/REST

service. It also provides details about WSDL, Service

endpoints and WSDL content.

Fig 5: Interface Viewer

International Journal of Computer Applications (0975 – 8887)

Volume 167 – No.2, June 2017

Interface Viewer is depicted in Fig 5. This can be opened by

double clicking on the SOAP/REST service.

The Request panel has various options. The first green run

button is used for hitting the request. The required input value

is given in the inputs fields which initially contains the

question mark(?) sign. As soon as the request is hit, the

response is shown in the response panel. The response then

can be validated. The request and response can be viewed in

two formats that are Raw and XML. Both tabs can be used to

see the different versions of same request. The bottom line

fields on these panels are Authorization, Headers,

Attachments, WS-Addressing and WS-Reliable messaging.

These fields are used for authentication and other security or

header related settings.[11]

Fig 6: Request and Response Panel

4.4.5 Test Suit

The Test Suit in SOAPUI can be created by right clicking on

the project. After creating Test Suit, Test cases are added into

it. Each test case can contain multiple test steps. The test step

can be a SOAP request, REST request, HTTP request, AMF

request or JDBC request as shown in the Fig 7.

Fig 7: Adding Test step in test case

The major role in test step is played by Assertions. SOAPUI

allows user to add assertions of various types. By double

clicking on the test step, test step is opened in new window

which has a green ‘+’ sign and it can be used for adding

assertions.

4.4.6 Assertions

Assertion means act of affirming or stating something[12] it

can be seen as a check point or a validation point. Window for

adding Assertion is shown in Fig 7. Assertions have many

types like Property Content, Compliance, Status and

Standards, Script, SLA, JMS and Security. Each assertion has

its description given in SOAPUI. So, the user can go and

select the appropriate assertion.[12]

Fig 7: SOAPUI Assertions

4.4.7 Properties

At the end of the Navigator there is a tab called Properties.

Properties tab is shown in Fig 8. This field provides the

details of Name, Description, file address, Resource Root,

Cache Definitions, Project password, Script Language and

Hermes Configuration.[13]

Fig 8: Properties

4.4.8 Logs

SOAPUI displays various logs like SOAPUI log, http log, and

memory log at the bottom of the tool interface. These logs are

depicted in Fig 9, Fig 10 and Fig 11 respectively.

4.4.8.1 SoapUI log

The SoapUI log is used for displaying the information of

response from the web server. This information is stored

inside the soapui.log file which is stored under the SoapUI

installation folder under the ‘bin’ directory.

International Journal of Computer Applications (0975 – 8887)

Volume 167 – No.2, June 2017

Fig 9: SoapUI Log

4.4.8.2 Http Log:

It is used for displaying the HTTP packet transfer. The

information is shown in RAW format in HTTP Log.

Fig 10: Http Log

4.4.8.3 Memory Log

The memory log monitors the memory consumption and

displays it in the form of the charts. When any memory

intensive operation is performed this type of log plays an

important role.

Fig 11: Memory Log

Apart from all these SoapUI also provides the operations of

jetty log, error log and wsrm log also.

4.5 SOAPUI NG Pro features

SoapUI NG Pro has additional features such as more number

of views like HTML, JSON, Outline and Overview including

basic XML and Raw view tabs. It also has more number of

assertions.[14] Its additional features are listed below -

 It provides the data driven testing to save time for

checking numerous real-world conditions.

 SoapUI NG Pro manages the multiple environment

related configuration at one central location which

makes it easy to witch between environments

without much overhead.

 It provides test coverage which can be used for

positive and negative tests and non-functional tests.

 SoapUI NG Pro provides the API refactoring, API

discovery and API monitoring.

 It provides test history, test debugging and result

comparison. It gives functionality of Point-and-

click testing, Code completion, Global events.

5. SOAPUI (OPEN-SOURCE)

DRAWBACKS

SOAPUI has got certain drawbacks also . Some of them are

listed below -

 SoapUI gives issues when working with JSON

response if it contains special characters. So, it is

not easy to add assertion on JSON response.

 Work flow testing and Load testing are difficult to

perform.

 SoapUI is easy for testing but difficult for

documentation.

 It is also observed that the SoapUI customer support

response is late.

6. CONCLUSION

SoapUI is an open source tool which is easily available for

everybody to use. It also has a commercial version with the

additional features. Continuous improvement of the SoapUI is

going on and making it stronger day by day. As it is an open

source tool various intelligent people are contributing towards

it and making it more efficient to use. Due to existence of

such tool it has become easy to test the web services and web

APIs. A lot of time and efforts required by manual testers are

saved due to this. SoapUI can be seen as a solution for

automation of API testing. Going forward more and more

people will start making use of it and it will become an

inseparable part of automation testing of web services. Rather

we can say that it has already become an integral part of the

web service and API testing.

7. REFERENCES

[1] Shariq Hussain, Paper on Web Service Testing Tools: A

Comparative Study

[2] Ravi Kumar, Research article on A Comparative Study

and Analysis of Web Service Testing Tools

[3] Evan Martin, Paper on Automated Robustness Testing of

Web Services

[4] Mustafa Bozkurt, Testing Web Services: A Survey

[5] Vinod Patil, Introduction to Testing Web services

[6] Tanuj Wala, A Comparative Study of Web Service

Testing Tools

[7] SmartBear Software Further Expands by Acquiring

World’s Most Widely Used Web Services Testing Tool

soapUI https://smartbear.com/news/press-releases/

smartbear-acquiring-web-services-testing-tool/

International Journal of Computer Applications (0975 – 8887)

Volume 167 – No.2, June 2017

[8] Getting Started With SoapUI

https://www.SoapUI.org/getting-started/introduction

.html/

[9] Understanding SOAP and REST Basics And Differences

https://blog.smartbear.com/apis/understanding-soap-and-

rest-basics/

[10] Why use SoapUI Pro for Functional Testing?

https://www.soapui.org/functional-testing/getting-

started.html

[11] API Testing Features https://www.soapui.org/open-

source/features.html

[12] Introduction for the SoapUI Newbie

https://www.soapui.org/getting-started/10-tips-for-the-

soapui-beginner.html

[13] Installation and Configuration of SoapUI

http://www.softwaretestingclass.com/installation-and-

configuration-of-soap-ui-complete-guide/

[14] Advanced SOAP and REST API Functional Testing with

SoapUI NG Pro https://www.soapui.org/professional/

soapui-ng-pro.html

IJCA

: www.ijcaonline.org