Commentaires 0

Retranscription du document

Shopping is often a process that requires more planning and frustration than it should. For example: inone household

a family member decides what should be bought, but another does the actual shopping.This can lead to items being bought that shouldn’t and vice versa. The Cellphone Shopper system seeksto make shopping easier by solving problems like this.

By allowing users to specify what they wishbought and where, and other users being able to view and buy

these items while on the move,it is hopedthat shopping willno longer be a source of frustration.

This report details the design of the CellphoneShopping system, as well as the implementation and testing of the backend component.

General Terms

Documentation, Performance, Design.

Keywords

Shopping, REST, Web Services, Web Server, Database.

iii

Table of Contents

1. INTRODUCTION

1.1 Problem Outline

1.2 Proposed Solution and Division of Work

1.3 Report Outline

2. BACKGROUND

2.1 Shopping Behavour

2.2 Related Systems

2.2.1 Similar Architecture

2.2.2 Similar Shopping Systems

2.3 Middleware

2.3.1 Java Remote Method Invocation

2.3.2 XML-RPC

2.3.3 Web Services

2.4 Communication Protocols

2.4.. SOAP

2.4.2. REST

2.4.3. XML-RPC

3.DESIGN AND IMPLEMENTATION

3.1 Design Motivation

3.2 Software Patterns

3.3 Software Model

3.4 User Requirements

3.4.1 User Interviews

3.4.2 Prototyping

3.5. Overview of Features

3.6. System Overview

3.6.1 Technologies Used

3.6.2 Database Layer

3.6.3 Logic Layer

3.6.4 Web-Services Layer

iv

4.TESTING AND EVALUATION

4.1. Methodology

4.2. Findings

5.CONCLUSIONS

6.FUTURE WORK

7. REFERENCES

5

List of Figures

Fig1. System Overview

Fig2. Soap Request Message

Fig3. Rest Request Message

Fig4.

XML-RPC Request Message

Fig5. Feature Driven Development Lifecycle

Fig6. ER Diagram

Fig7. Class Diagram

Fig8. Persistance Management

Fig9. User Attributes

Fig10. Example RESTful Web Service

Fig11.

JMeter Performance Test 1

Fig12.

JMeter Performance Test 2

6

1. Introduction

1.1 Problem Outline

Grocery shopping can be aburden,especially in situations where one persondecides what must bebought and another does the actual shopping. Some of the problems encountered while shopping are:



Difficulty in sharing the shopping list–

what is the list written on and where is it kept?



One person adding something to the list and another wondering who added it and why



Going shopping, only to realise that the shopping list has not been brought



The buyer not knowing which brand of item to buy



Co-ordination: who does the shopping and when?



When the buyer gets to the store, they do not know where items can be found and spend a longtime wandering around the store looking for them.

The key aim of thisproject is to make grocery shopping easier by using technology to aid the process.This does not mean that technology will be used to automate the shopping process entirely; rather, theaim is to use it to provide a tool for shoppers to use to make shopping simpler.

Two types of communication technologyare

used: cellular telephony and the Internet. The goal is toallow a household to share and manipulate a shopping list stored on a central server via a Web interfaceor a cellphone.The typical use case is that of one user creating the shopping list for the current day orweek, while another can then view that list when they go shopping. If any changes are made to the list,all the users can see them. This system will be referred to as theCellphone Shopping system.

1.2 Solution and Division of Work

TheCellphone Shopping system

is split into three components:

a Webinterface, a mobile application

and a server component managing the data for both of these. Marc Pelteret will be implementing theWeb interface, which allows

users of the system to manipulate and add shopping lists, as well as morecomplicated features that cannot be done on the mobile interface.Tshifhiwa Ramuhaheliimplemented

the mobile applicationwhichallows

users of the system to view and manipulate shopping lists while onthe move. FinallyGraham Hunter

with the overall system development, with specific attention paid to the backendcomponent.In the next sectionthe background literature relating to shopping behaviour will be explored

as well as similar systems

and

technologies that could be used. In section 3 the software methodologyused and the design

of the system will be outlined

as well as the proposed implementation. Section 4describes

the testing methodology used to evaluate the system and the results obtained. Section 5 willdiscuss the findings. Section 6 will summarize the project along with the lessons learnt. Finally Section 7will conclude the paper with possible future work.

8

2.Background

2.1Shopping Behaviour

The design of the system relies on how people behave in a shopping environment, what functionalitywould be used and where. Based on a user study several common shopping behaviours aredetermined

[15].

There exist twokinds of shoppers, those who make mental lists and those who make physical lists.Those who keep physical lists tend to carry it on their person. Some occasionally leave the list in theshopping cart. Many of those who keep physical lists mark off items as they are bought, showing theneed for interaction between the shopper and their list. Shoppers who keep mental lists rely on browsingbehaviour to remind them of what items should be bought.

Shopping typically involves three phases:



Pre-shopping phase which is where the shopper plans to go shopping and creates a list.



Shopping phase where the person is choosing the items.



Checkout phase where payment is made.

Shoppers often pick up items or otherwise use their hands while shopping.

Even though mobile phones are used more often than PDAs, the size of a PDA was preferred over thatof a mobile phone for use while shopping.

Shoppers tend to visit the same shop often and follow the same route.

2.2Related Systems

Systems that are related in some way to the proposed one are discussed below.

2.2.1 Similar Architecture

Listedbeloware example systems that deliver data to either a Web interface or a mobile interface usinga client/server approach.

2.2.1.1 Google Maps

Google Maps is a Web application that allows

one to view road maps through a Web interface [11], aswell as a mobile interface [12]. This application allows the user to view real-time traffic, get directionsand search for locations. Using SMS mobile clients are limited to searching maps by text, but using amobile application the client can view maps graphically, communicating with the server through the useof WAP. Much like theCellphone Shopping system, Google Maps interacts with both a Web andmobile interface to provide information to users. The difference is that the system does not deliverpersonalized information.

2.2.1.2

Google Calender

Google Calender is a similar application to Google Maps, but provides functionality for creating andmaintaining personal and public calendars [13]. Events

can be added by using natural language such as"Brunch with mom at Java Cafe 11am on Saturday" which is then parsed. Users can choose to receiveSMS or email event notifications on their mobile device. Calendarsalsocanbe shared between users,

9

allowing

multiple people to edit the calendar depending on the access rights. The application can beaccessed through a Web browser or a mobile device. Google Calendar is very similar to theCellphoneShopping system,

architecturally and functionally. It allows the management of a list through similarinterfaces, provides personalized information and lets users collaborate on a single list.

2.2.2 Similar Shopping Systems

Systems that deal with providing shoppers services to help them in the store arediscussed

below.

2.2.2.1 Easi-Order

The Easi-Order system worksby

using PalmPilots to create a shopping list, then sending that shoppinglist to a store and having the items pre-picked [14]. The system can pre-populate the shopping list usedbased on the user’s shopping history, as well as offer “impulse buy” suggestions. The store’s productsare electronically stored in a product index in categories and using a barcode scanner on the PalmPilotsproducts can be entered into the list by scanning their barcode.

This

system is similar in allowing shoppers to create and manage shopping lists. The difference is thateach user has a separate list, whereas theCellphone Shopping system

allows multiple users to have onelist.

2.2.2.2.

U-Scan Shopper

The U-Scan Shopper system involves a monitor being attached to a shopping cart handle which candisplay promotions when the shopper is near certain items [16]. Additionally a bar code scanner is on theshopping cart allowing the shopper to scan items for themselves and pay atan automated station. Finallythe system automatically generates a shopping list for each shopper based on their shopping history withthat store, viewable on the monitor.

The similarity of this system is that it allows the shopper to view their shoppinglist while shopping. Thedifference is that the U-Scan system automatically generates this list, whereas theCellphone Shoppingsystem

allows the user to create their own.

2.2.2.3.

Mobile Shopping Assistant

The mobile shopping assistant is a system developed to allow shoppers to access Web services that arepublished by the store by using their mobile devices

[2]. Some of the services available allow thecustomer to create a virtual shopping list, receive promotions and view a floor plan of the shop. Themobile devices used were normal mobile phones running a J2ME application. The mobile devicescommunicate with the Web services using SOAP messages, caching and compressing the messages forquicker data exchange. Some of the problems of mobile computing that this system tries to overcomeinclude:



Unstable network connection status.



Limited bandwidth of mobile communication channels.



No efficient XML parser.

10

Some of the problems encountered in development include local cache management on the mobiledevice, pre-fetching data from the Web services, data consistency and data conflicts.

In summary,

there are many systems that have a similar architecturaldesign in having a Web andcellphone interface connect to a server, and a few systems that deal with providing services to shoppersin stores.

2.3. Middleware

Middleware is a software component that sits between the network operating system and an application.It facilitates communication and management of distributed components in a network [10]. Thefollowing are examples of middleware.

2.3.1 Java Remote Method Invocation

Remote method invocation allows remote Java objects to be invoked from other Java virtual machines orremote hosts [6]. RMI has been in the Java SDK since version 1.1 and is used to develop distributedapplications in Java. The benefit of using Java RMI is that it requires no additional libraries ormodification of normal Java applications. One drawback is that the client needs an open port tocommunicate, which is not available if the server has a firewall. Compared to Web services, using RMIis significantly faster,

if there are open ports to communicate in terms of network and hardwareperformance.

2.3.2 XML-RPC

XML-RPC is a way to make RPC-style function calls to a machine over a network [10]. XML-RPCcommunicatesthrough

the HTTP protocolusing

XML messages. The benefits of using XML-RPC is tohave:



An easy to implement standard.



Discoverable services.



Messages that can go through firewalls.

XML-RPC is different from other RMI protocols as it does not need stubs generated beforehand [7].Instead primitives are used to construct requests and get responses. XML-RPC does not marshall

arguments as all data is sent as text.

2.3.3.

Web Services

Web services refer to an application component

that can be accessed over the Internet and used remotely.Web services use XML and HTTP as these are interoperable standards [1]. Any computer that cancommunicate via HTTP can use a Web service. The difference between Web services and itspredecessors is that it uses the SOAP protocol to communicate, has an interface defined using WSDLand is discoverable through the Universal Description, Discovery and Integration (UDDI) service. The

11

benefit of using Web services is that of interoperability betweendifferent platforms such as .NET andJava. A set of APIs for Java are available that can be used to implement a Web service,

called the JavaWeb Services Developer Pack. One of these is the Java API for XML based Remote Procedure Call(JAX-RPC), which makes implementing a Web service similar to creating an RMI remote object. Webservices differ from RMI in that they are executed in a Web container as servlets. The Web servicesclasses can be converted to servlets or a servlet handler can be used to forwardrequests. Compared toRMI some of the features that Web services do not have include:



Using object references.



Distributed garbage collection.



Dynamic class loading.



Remote object activation.

Web services have decreased network performance compared to RMI, due to the overhead of usingSOAP. Using SOAP messages, XML needs to be serialized and de-serialized, whereas RMI uses binaryserialization.

Each middleware provides different functionality and varying degrees of interoperability depending onhow each sends and receives messages. An overview of the message structure different middleware usesis discussed next.

2.4. Communication protocols

As bandwidth is limited when communicating with mobile clients, the messages each middleware uses

should be minimal

in size.

2.4.1 SOAP

SOAP requests consist of a header and a body. The body contains the XML request object, while theheader has information not required for the request [7].

Given below is an example.

<?xml version="1.0"?>

<soap:Envelope xmlns:soap="http://www.w3.org/2001/12/soap-envelope"

xmlns:m="http://www.uct.ac.za/shop">

<soap:Header>

<m:ID>1234</t>

</soap:Header>

<soap:Body>

<m:GetList>

<m:ListNamee>Example</m:ListName>

</m:GetList>

</soap:Body>

</soap:Envelope>

12

Fig 2:

SOAP Request message.

The SOAP protocol was not designed for wireless communication and does not consider the limitedresources that a mobile device has [8]. SOAP is generally used over HTTP and TCP, which has thefollowing limitations :



Congestion avoidance in TCP assumes packet losses are because of congestion-

in a mobilenetwork this could be due to disconnections instead.



Using TCP requires a connection to be established, resulting in increased overhead asacknowledgement packets are sent.



TheNetwork is not utilized well as a SOAP message that carries only a small amount of data willstill require a connection being initialized.

By using client side caching and compressing the SOAP messages the performance of mobile clients canbe significantly

improved by up to 800%.

2.4.2 REST

REST uses standard HTTP GET requests to make calls to aWeb resource[6].

The Web Servermanaging the resource will parse the URI and decide how to handle the request.

The parameter data isincluded asHTTP request parameters. The protocol is the simplest but relies on the HTTP protocol.

GET /List?ListName=Example HTTP/1.1

Host: www.uct.ac.za

Fig 3. REST request message.

2.4.3 XML-RPC

XML-RPC is verbose compared to non-XML

protocols like REST [9], but uses fewerbytes per messagethan SOAP. A request message that passes one integer is 168 bytes,

of which 41 bytes are used to markup the integer parameter. Compression can be used on these messages to reduce the size. Anotherdrawback of XML-RPC is that retrieving data using XPath can be complicated as the elements are notnamed semantically [6].

<?xml version="l.0" encoding=

"ISO-8859-i"?>

<methodCall>

<methodName>act. GetList</methodName>

<params>

<param>

<value><string>Example</string></value>

</param>

</params>

</methodCall>

Fig 4:

XML-RPC request message.

13

The protocols differ in how many bytes each message needs to be, with REST requiring the least, andSOAP the most. XML-RPC lies in the middle, but requires complex parsing.

2.5. Web Servers

The choice of whatWeb server software to use is important as it must support whatever middleware ischosen, as well as being easy to maintain and develop for.

2.5.1.

Jakarta-Tomcat

Jakarta-Tomcat is the Apache Software Foundation’s project for handling servlets and JSPs.

A servlet isa program that is executed in response to an HTTP request and returns an HTTP response [5]. Servletsare object classes and are run inside the Java Virtual Machine, which means servlets can run on manyoperating systems. As the software runsJava it allows the use of Java-RMI as well as Web services.

2.5.2 Microsoft Internet Information Services

Microsoft Internet Information Services (IIS) is a Web Server built into the Microsoft Windowsframework that can handle Web services and Web applications developed in ASP.NET [18]. As IIS doesnot support Java, Java RMI cannot be used. When developing Web services in ASP.NET many of thedetails such as creating WSDL documents, creating the SOAP messages and Web Discovery ServiceFile are done automatically by Visual Studio .NET [17].

Summary

of related work

Shopping behaviour has been explored in user studies before. Shoppers either keep a physical list or relyon the shop environment to remind them of what to buy. Meaning that a shopping system should offer alist management service and the ability to view a floor plan of the shop with items listed. Shopping canbe divided into three phases:

a phase wherethe shopper decides what to buy;

a phase where they choosethe items at the shop;

and a phasewhere they pay for the items. The shopping system should providedifferent services at each stage. Shopping involves the frequent use of both hands and constant attention,therefore the system should be fast to use as well as usable by one hand only.

Based on this theCellphone Shopping system

included the following in the design:



Allowing

the users to view and manage shopping lists on the Web interface before shopping andon the mobile interface during shopping.



Users can view a floor plan of the shop as well as design their own.



Items can be labeled as bought,

either while shopping or afterwards by both interfaces.



The cellphone interface was designed so that the most common action at a certain period, forexample buying an item, had the least amount of

button presses.

14

In terms of similar systems there are many that use both a mobile and a Web interface to show data. Forthe mobile interface multiple communication protocols can be used such as SMS for simple textresponse/request messages or a mobile application that uses GPRS to interface with the server. Systemsthat focus on providing mobile services while shopping have been implemented before. Someautomatically generate a shopping list based on previous purchases, while others allow the shopper tocreate and manage their own lists. One used a PalmPilot as a mobile device, another a custom deviceattached to the shopping cart and finally one used a mobile phone. Neither system had a Web interfaceproviding similar services that the mobile device had.

The problems highlighted in developing thesesystems involved those related to mobile computing. Mobile devices have unreliable networkconnections, limited bandwidth and a limited XML parser. Some systems use client-side caching of dataand compressionof the messages used to increase performance.

In the implemented system the mobileclient did use client-side caching of data, as well as sending updates in batches.

Two middleware choices were covered, Java RMI which allowed a normal application to communicateremotely with another, and XML based Web Services which requires a Web Server. Performance wiseJava RMI uses less network bandwidth due to the shorter messages and less CPU resources because ofnot needing to serialize and de-serialize XML messages. The drawback is that it is less interoperablethan Web Services as the clients need to be able to run Java code. The choice of middleware is importantdue to the limited resources the mobile client has. Using tomcat as a server allows the use of Java RMI,Web Services and normal Web Applications, whereas using IIS without Web Services drasticallyreduces interoperability.

What was chosen for the project was the use of a REST based Web Service, due to its simplicity andminimal message sizes. The Web Server used was Jakarta Tomcat as it allowed the use of Java whichthe developers were familiar with.

15

3.Design and Implementation

3.1. Design Motivation

Other Web based shopping systems have been implemented before but the design focuswas onproviding a system specific to a shop that bought the items for the user. Similar mobile-based systemshave been implemented overseas and been successful, highlighting a potential need for ashopping

system in South Africa.

This is supported by the background research done, which shows that shoppers do have a need for a

system that would aid in shopping. The ethnographic research done suggests that the focus for themobile component must be ease of use. The Web site should then allow the more time consuming

andharder functions to be done while the user is not shopping. The backend component is then necessary tointegrate the two.

The communication protocol used between the components should be minimalist dueto the mobile device’s low bandwidth.

3.2.Software Patterns

Software patterns provide proven and tested architecture designs that take into account issues not readilyapparent. Using applicable software patterns is then important to reducing errors in the project.Thesoftware pattern used for theCellphone Shopping system

is described below.

The Model-View-Controller framework divides an application into three parts, the model where the coreapplication is, the controller which directs the flow of the actions, and the view which handlespresentation[3]. In this system the Web and mobile device interfaces would be the View, the Controllerwould bethe Web API that links the

Web and mobile application inputs to actions that the componenthandling the data will perform. The model responds to input fromthe controller, returning data from thedatabase or changing the state of the view depending on the business model. The benefit of this model isthat individuals can develop each component separately.

3.3. Software Model

The software model used throughout

developmentwas

an agile method. Agile development methodsinvolve multiple iterations and the use of prototypes with the focus being on quick development. Thisaddresses the problem of requirements changing while the system is still in development. Agile

methodsalso emphasize the use of small teams and face to face communication, suitable for this system.

The specific agile method usedwasFeature Driven Development, which focuses on a model drivenshort iteration process. The first stage is to developan overall model of the system which defines itsscope and context. This model is comprised of domain area models relating to specific areas of thesystem.Once this is done a list of features is identified,

each taking no longer than two weeks toimplement. Once the features have been detailed development is planned around implementing the list.

16

Once a feature is implemented it is tested thoroughly.

The systemhadtwo iterations, with a featuregathering stage at the start of each.

Shown below is a representation of the method.

Fig 5:

Feature Driven Development Lifecycle

3.4. User Requirements

3.4.1.

User Interviews

To establish the features the user wanted, interviewswereconducted

using non-expert users with

differing shopping

behaviours. Users were askedabout

their shopping behaviour, what they would likefrom aCellphone Shopping system

and whether they would prefer to use a mobile device or a Webclient.

From this preliminarystudy,features wereextracted

which were used to develop a prototype.

3.4.2.

Prototyping

Based on the initial features a prototypebackendwas developed to test what technology should be usedin the system.

This was an evolutionary prototype with the final system building on it. The prototypeconsisted of a Tomcat Web Server providing a SOAP Web Service that interacted with a mySQLdatabase.

This helped to determine what features were possible and their implementation.

17

3.5.

Overview of Features

Based on the user studies donethe features that each component supported

in order to meet the systemrequirements can be divided into several categories.

3.5.1. List management

Features that fall under List management deal with creation or deletion of

alist as well as modifying andmaintaining a list’s settings. As such the system supports

the user adding multiple lists, deleting multiplelists and changing the settings of a list such as the list name, the access rights that a user has for a list andthe default shop associated withthe list.

3.5.2. List

Features that fall under this category have to do with adding or deleting items from a list, being able torepresent the list in some way, and able to support different types of lists. Specifically the usercanadd

an item to a list

or edit anexisting one. An itemhas

attributes specific to the list it is part of,

namely:quantity, the shop it should be bought from, an optional note attached it, whether it is a private item andonly viewable by the user who added it and whether it is

uncertain if the user wants it. Additionally theusercaneither delete an item, where it is removed from the list and added to deleted items, or checkoutan item, where it is removed and added to previous items.

The list allows

different ways of beingpresented,

either alphabetically, by a predefined shop layout, orby item category. Special types of listsare supported:

shops to have a layout defined for them, either generated by an administrator or by auser. Each user can only have one layout per shop. The useris

able to construct the layout using thewebsite and then sort the items on a list using it.

3.5.4. User preferences

The system stores

certain information about users. Each user will have a unique user name chosen bythem which is used to login via the Web site or cellphone along with their password. The passwordistransferred in an encrypted form such that the backend cannot reverse-engineer

it to plaintext.

Personal details

about each useris

stored:

their first name and surname, their email address and their cellphone number. Each useralso has

a list of trusted users that they have added who they can grant accessto their personal lists.

18

3.5.5.

Reminders

Usersareable to set reminders to pay items like utility bills that need to be paid. Reminders will have aname, a description and the date that it is due for as well as a notification period how long before thedate the user must be notified. The reminder can also be added as an item, appearing on every list of theuser except for favourite and previous item. The user will have the option of removing the reminder,which will also remove it from any lists it is part of.

3.5.6.

Notes to others

The system allows

users to send messages to other users using a gateway service, such as SMS or email.

loosely coupled in such a way that failure of one should not stop another fromfulfilling the requirements. Communication between each component willbe achieved through HTTP,with the responses from the backend in XML.

3.6.1

Technologies Used

The implementation of the backendhas

three layers. As the system relies on long term storage of data, adatabasewas

used. MySQL was decided upon as it is a free and stable database server which hasperformance comparable to other commercial products such as Microsoft SQL Server.

Asthebackend needs to support both a Web interface and a mobile client it was decided to provide aREST based API that would be accessible through HTTP GET and POST requests from either interface.Thiswas

achieved using Java servlets to manage calls. Responses are returned in XML, which can beparsed by both interfaces. Compared to SOAP based web services the interfaces can make calls far easierusing URL encoded data, fewer bytes are used in each message which impacts the mobile client asbandwidth is limited, as well as providing the greatest degree of interoperability with both interfacesaccessing the API in the same manner. To support this Tomcat was chosen as the web server as it hasgood support for Java servlets.

In order to implement the business logic of the system another layer between the database and the Webserverwas

used where POJOsheld

the data and format it as required by the interfaces. Persistance in thesystem will be handled manually, as a persistence manager would increase the system overhead andwasseen as unnecessary given the scope of the project.

Each user needed to be represented uniquely in the system, which this table is necessary for.UserName and UserPassword are used to authenticate the user to the system; UserEmail andUserCellphone are used to send messages between users. SessionIdent is used to determine the lasttime the user was logged in, in order to determine what notifications they should be sent. Usershasamany-to-many relationship with Lists, Shops and Trusted Users as well a one-to-many relationshipwith Items, Reminders and Layouts.

20

LinkUserToList:This table provides a many-to-many mapping between users to Lists, as one usercan have many lists, and a list can have many users each with different access rights.

Lists:The Lists table stores the attributes specific to a list such as the name of the list, and the date itwas added. ListUpdated is necessary to determine if a list was updated, with ListEditType specifying

how it was updated and ListUserChanged which user changed it. Listshas

a many-to-manyrelationship with items and users.

LinkListToItems:As a list can have many items, and an item can havemany lists this tableprovides a many-to-many mapping between the two. The item attributes that are specific to itsinclusion in a list are included here, such as quantity and the shop it should be bought from.

Items:This tabledescribes

the

item attributes that are not specific to a list, such as the name, thecategory it belongs to, and the user that added it. This tablehas

a many-to-many relationship withLists, a many-to-one relationship with Users and a one-to-one relationship with Reminders.

Shops:This table will store the shop name, location and when it was updated, as well as providing alink to a city and a suburb. Shops will have a many-to-many relationship with Users, a one-to-manyrelationship with Layouts and Lists as well as a

many-to-one relationship with Cities and Suburbs.

Layouts:Each layout is shop-and-user

specific, with one user onlyallowed one layout per shop.Layouts will have a many-to-one relationship with Users and Shops.

TrustedUsers:This table stores

the trusted users that a user has, with one user having many trustedusers and vice versa.

Reminders:Reminders stores

the attributes specific to a reminder, with one user having manyreminders and one reminder having one user. As a reminder can also be linked to an item, itemid isincluded as a foreign key.

Cities:The Cities table will not be edited by the user but by an administrator. Cities has a one-to-many relationship with Shops. Cities will have a one-to-many relationship with Shops.

Suburbs:The Suburbs table will not be edited by the user but by an administrator. Suburbs has aone-to-many relationship with Shops.

Categories:The Categories table will not be edited by the user but by an administrator. Categorieshas a one-to-many relationship with Items.

21

Cachingand BufferingofQueries

mySQL allows the use of caching databasequeries, which improves performance. The decision was tonot use caching

during system use

as tables were updated too often for this to be of use.

Buffering ofqueries, wherethe result

set of a query is stored in a temporary table to free up table locks while theresult is being returned, washandled automatically by the database optimizer.

Connection Pooling

Connection pooling refers to a technique whereby a thread requestsdatabase connections from a pool ofalready created connections.Each thread uses a connection exclusivly while it holds it.Once a threadhascompleted a transaction and the connection is idle, it is returned to the pool where other threads canutilize it. This increases performance as connections need not

be created everytime, while being threadsafe.

Connection pooling was used in the system, with a pool size of500.

Indexing

An index is a data structure that increases the speed of mostoperations. Indexes which have been createdfrom columns allowfaster random lookups, while requiring more disk space on the server. The primarykeys in each table wereautomaticallyindexed: UserId; ListId;ItemId;ShopId and ReminderId.Theforeign keys in each table were explicitly indexed.These were the variables used for select and updatequeries. The tradeoff of using indexes is that insert statements would be slower due to the index beingupdated, which is why ItemId was not used as an index due to the large amount

of inserts being done inthat table.

3.6.3.

Logic Layer

The logic component comprises

the

Java classes that perform the business logic of the system. Java waschosen as it is the language that all involved have the most experience with. The attributes of each classistaken from the corresponding database representation. The following diagram details the classesinvolved in the system and their relationships.

22

UserListItemShopLayoutStateReminder-contains items1-belongs to a list*-contains layouts1-belongs to a shop*-contains shops1-belongs to a user*1*-contains reminders1-belongs to a user*-contains users1*

Fig 7:

Class diagram

Persistance Management

When anobject is instantiated, its attributesare

initialized through the Load method that will load theattributes of the object from the database.

Thisis

done using JDBC to perform queries. The componentsof the object will also be instantiated and their Load method called when the container is initialized,resulting in a recursive initialization. This simplifies persistence in that as the User class is the superclass of the others, only one method call needs to be made by the Web API. When object attributes aremodified, saving the changes to the database will be done in a similar method by using the Save method,but this will not be recursive.

Finally each class implements

a Delete method that recursively removes

from the database the rows that refer

to it, and it’s containing classes.

Initialization is done up front,rather than lazily where it could be put off until the object was requested, as

in many use cases manyobjects are requested in a short period of time.The sequence diagram of this process is given below.

To reduce the amount of database connections made during system use, objects and their attributesare

cached. This increases

the amount of memory the system uses, but results

in a greater throughput

as adatabase connection need not be made for each request. As theUser object is the super class thatcontains the others, a vectorisused to storea soft reference to each Userinstance.

A soft reference wasused so that the object could be garbage collected if the system was running low on memory.

Theamount of users that are cached is limited to 100, with the older ones being replaced by the newer.

Each User

object

is

cached for thirty minutes

or when the user logs out, whichischecked when anotheruser logsinor out the system.

Thirty minutes was chosen as it was seen as average shopping time that auser needs.

When data needs to be accessed the appropriate user issearched for

and if available itis

instantiated, cached and returned. To achieve this,thevector of usersis

static, meaning all threads will

24

access the same object. To handle synchronicity a vector was chosen over a faster solution such as anarraylist or a linkedlist as it is synchronized.

To support synchronicitywhere shared classes are updated by different users, which will result ininconsistencies between cached objectsand the database,

each object checks

whether the correspondingdatabase table was updated before returning any details and if necessary will instantiate the object again.Thisis onlyapplicable for returning the object information as adding and deleting operations areidempotent and do not rely on synchronicity between objects.

Authentication

When a login request is made to the Web API, a user name and passwordis

needed for authentication.When the user object is instantiated these are compared to the database if there is a match the object willbe instantiated and its id returned by the Web API.

The system will assume from then on that any futurecalls using that id will be valid. This reduces the need of having to send the user name and passwordevery time a request is made to the Web API.

To protect the user’s password it will be

sent during thelogin request in an

encryptedformusing MD5,

which cannot be reversed. Thisis

done so that thebackend cannot know the user’s password protecting their privacy

Session Management

Each user of the system

corresponds

toa User object

with the last time that they were logged inupdatedwhenever the Login or Logout method is called. As users might not logout

the system when they aredone, sessionsare

closed after thirty minutes of inactivity.

Access Control

Access control in terms of the rights that a user for adding, editing and deleting items to a list,ismanaged in this layer as well as in the Web API layer. The rightsare

stored in a string with eachcharacter representing a right,with a value of 1 or 0. When a List object is initialized it loads

the rightsof the user that holds it, and when an operation is done on the list the rightsare

checked.

Notifications

The notifications that a user receives

are

updated based on the last time they logged in and the last timethe lists and shops they have addedhave been updated. If the difference between the reminder date andthe current date is less than the period then it will also be displayed.

Sorting

Sorting of items

in a list based on their name and categoryis

done using quicksort

as this is one of the

fastest algorithms for sorting with O(NlogN).

The decision to sort the items in the logic layer instead ofthe database layer was as a result of having to sort by shop layout

which required.This also has the

25

benefit in that multiple requests to sort a list by the same criteria means that the already sorted list can bereturned unless it was updated.

Sorting by shop layoutis

done by sorting the items into each aisle by the categories contained in them.The path that will be suggested is that of following

the aisle down to the end, then going back up thenext aisle. An assumption will be made that the shopper would come from the top left side of the shop; if

the entrance was to the top right the list would be read in reverse.

Items not in the layout are placed last.

XML Parsing

Each class

overrides

the toString method and parses

their attributes into XML form. This is makes itsimpler for the Web layer to return results of queries.This method will not recursively call the othertoString methods of other objects.An example of the result of this using the user class would be:

“<u id="3" n="UserName" f="Graham" s="Hunter"

e="hunter@test.com" c="083594"/>”

fig 9:

User Attributes

Where id is the unique identifier of the object, ‘n’ is the

user’s name, ‘f’ is the first name, ‘s’ thesurname, ‘e’ the email address and ‘c’ the cellphone number.

Limits

To ensure that a user of the system doesn’t overload it,limits are placed on the amount each containerclass can hold. Each list object cancontain up to 40 items

and

each User object can contain up to 20Lists, 20 Reminders, and 20 Trusted Users.

3.6.4. WebLayer

The Web

layerdeals

with communicating between the two interfaces and the backend to provide thefeatures of the system. It provides a REST-based API that will be accessible over HTTP. Web Serviceswereused rather than RPC as this made integration easier.

Web Server

Jakarta Tomcat was used as the Web server due to its support for Servlets, as well as being open sourceand freely

available.A Servlet is an object that receives a request and generates a response. A Servletcontainer, in this case Jakarta Tomcat,

manages the Servlets and maps the requests to them. A Servlet isonly initialized by the container once in their lifetime, where after it will service each request in a new

26

thread.Servlets are not destroyed after each request. Compared to CGI,Servlets are more efficient due tothis, as well as allowing the use of connection pooling. This is due to Servlets not being destroyed after arequest, so state can be kept.

The server was configured to have a maximum of 100 threads at one time, and the maximum idlethreads was set to 50. Tomcat provides connection pooling support called DBCP. This manages databaseconnections andwas configured to remove abandoned connections when they had been idle for too long.Additionally connections were verified that they could be used whenever they were requested from thepool.

Web Service

Data is requested through GET messages, and updated

using POST messages. In

the initial prototypecommunication was done using SOAP but a REST approach was preferred as the messages weresignificantly shorter and marshalling the parameters was easier for both interfaces. Only

GET and POSTmessageswere

usedas neither interfaces hadsupport for DELETE and PUT.

All methodsare

idempotent in that calling it multiple times will yield the same result except for addingan item, which will increase the

quantity of that item. Thismakes a REST approach more applicable as itis suitable for stateless services.

The Web Serviceswere

developed using the Sun Web Developer Pack which provides animplementation of JAX-RS. JAX-RS is the Java API for RESTful Web Services that hides the lower-level details of using Servlets to implement a REST Web Service, by providing a high-level API thatallows classes to use REST specific annotations to define resources and specify actions. An example ofthis is given below.

@HttpMethod(value = "GET")

@ProduceMime(value = "application/xml")

@UriTemplate("User_Details")

public String User_Details(@QueryParam("UserId") int UserId) {

Fig10:Example RESTful

Web service with annotations

This example responds to the URIhttp://example.com/User_Details?UserId=1. The annotations used inthis example include @HttpMethod which defines what httpmethod it should respond to,@ProduceMime which defines what result it will return, @UriTemplate which defines what URI it willrespond to and @QueryParam which specifies what parametersit expects. The URI templates will notfollow REST standards which recommend not using verbs or queryfields, as since the interfaces onlysupport GET and POST it can be ambiguous as to which method is being called.

27

The systemmakes

the assumption that the

user has permission to access any method in the API withoutauthorization, with the exception of the user login and delete methods which require a user name andpassword. This is as a result ofkeeping

to the stateless nature of Web communication.

Everymethod returns an XML responsewitheither the object information if it is a GET request, the idof the affected objects if it was a POST request and an error message if an error was encounteredbecause of incorrect parameters.

The Web serviceissplit into 4 classes relating to what resource they deal with. Namely UserResource,ListResource, ShopResource and ItemResource, with each one responding to a different URI. Each ofthese will provide methods for creating, deleting and managing their corresponding

resource.Acomprehensive listof available methodis given below.

User-related

Methods

Name

User_Login

Description

Authenticates the user to the System, updating the last time they were logged in

Type of Request

GET

Parameters

UserName, UserPassword

Result

Returns the details of the user. “f” being the first name, “s” the surname, “e” the email address, and“c” the cellphone number.

This section will cover the testing and evaluation of the system, specifically the backend component.Detailed below is the test plan that

will be used to evaluate how well the systemmeets its

requirements.Testing will involve both white box testing, where the test cases are based on internal structure and seekto test the different paths in a program, and black box testing where input isgiven and the output testedfor correctness.

4.1.

Methodology

4.1.1.

Objectives

The objective of this test plan is to provide a description of the tests that will be done to measure thesystems compliance with the original project specifications. These specifications include fulfilling thefeatures detailed in the design chapter, handling of multiple users at once as well as providing a stableand reliable system under extreme load.

4.1.2.

Scope

The tests covers

what was described above, with the focus being on testing the backend serverfunctionality and performance. The other components of the system will be tested by their respectiveprogrammers.

38

4.1.3Hardware and Software Requirements

The Server componentused in the test was:



Intel Pentium 4 3.0GHZ 512MB RAM

TheRequester machine

used in the test was:



Intel Pentium 4 3.0GHZ 512MB RAM

These are the following softwaretools installed on the test machine



Server

Windows XP with SP2 and running an Apache Tomcat Server.



Database:

MySQL

4.1.4.

Testing

The key areas of importance to the backend are tested.

Database integritywas checkedin terms ofallowing multiple reading

and writing at once. System testingdetermined

that the system doesimplement the required features. Business Cycle testing verifying

that operations related to the life cycleof the systemis

correct. Performance testingwas

done to test the response time of the system under anormal and abnormal load-

important for communicating with the Web and mobile interfaces as if theresponse time is too high the requests will time out. Load testingwas

done to ensure that the system canhandle the theoretical traffic it would receive in production. Finally security and access controlwas

tested to ensure correctness.

Data and Database Integrity Testing

Verify simultaneous record read accesses.

Verify simultaneous record updates.

System Testing

Verify a new user can be created-

System Use Case.

Verify a user can login-

System Use Case.

Verify a user cancreate a list-

System Use Case.

Verify a user can add items to a list-

System UseCase.

Verify a user can checkout an item-

System UseCase.

Verify a user can delete an item-

System Use Case.

Verify a user can create a reminder as an item–=卹s瑥t=啳攠䍡r攮=噥s楦y=愠us敲=捡n=d敬整e=愠l楳琠–=卹s瑥t=啳攠䍡r攮=噥s楦y=愠us敲=捡n=慤d=愠污lou琠瑯=愠shop=J=卹s瑥t=啳攠䍡r攮=

39

Verify a user can sort a list by name-

System UseCase.

Verify a user can sort a list by layout-

System UseCase.

Verify a user can sort a list by category-

SystemUseCase.

Verify a user canlogout the system-

System UseCase.

Business Cycle Testing

Verify that reminders are shown within theirnotificaton period.

Verify that updates to a list or shop are shown.

Verify that reminder are deleted when they expire.

Verify that users are logged out when the login

period expires.

Performance/Load

Testing

Verify response time for user creation.

Verify response time for login by Web interface.

Verify response time for login by Mobile interface.

Verifyresponse time to access items in a list.

Verify response time for sorting a list by item name.

Verify response time for sorting a list by shoplayout.

Verify response time for adding a reminders.

Verify response time when 10 users try to access thesame list at the same time.

Verify system response when loaded with 200logged on users.

Verify system response when 10 simultaneous userstry to sort the same list.

Verify Memory and CPU usage is no more than70% of the total resources available.

40

Security andAccess Control Testing

Verify Logon security through user name andpassword mechanisms.

Verify List Access control by trying to add anddelete an item without permission.

Function / DataSecurity: Verify that user can access only those functions/ data for which their user type is provided permissions.

System Security: Verify that only thoseusers with access to the systemare permitted to access it.

Technique:



Create a test user account

with limited access and attempt toperform functions which they have no rights to.

This includesadding items to a list they don’t have access to, deleting itemsthey don’t have access to, and deleting a list that they do notown.



Create a test user account with access to the same functions andattempt to perform them



Attempt to login with a test user with incorrect password

Completion Criteria:

For each known user type the appropriate function / data are availableand all transactions function as expected

Tools

The following toolswere

for testing the system:

Tool

Version

Functional Testing

Web interface

--

Performance Testing

JMeter

2.3

Test Coverage Monitor or Profiler

JMeter

2.3

DBMS tools

mySQL-Front

2.3

JMeter Test Plan

JMeter is a freely available desktop application that can simulate a heavy load on a Web server bymaking multiple requests. As JMeter is muli-threaded,allowingsimultaneous requests of the samemethod can be made. TheJMeter simulationused was derived from the abovestated performancetesting plan. A typical use case was constructed which involved: a user creating their account; loggingin; creating a list; adding twenty items to that list; adding a shop layout; adding a reminder; sorting the

43

list by name; sorting the

list by category and sorting it by the shop layout just added. A thread count of200 was used with a ramp up period of 10 seconds.

4.2.

Findings

Database testing showed that mySQL could perform with good response times and allowed simultaneousaccess.

This was to be expected as the queries performed by the system were only simple SELECT andUPDATE queries, not requiring transactional security.

System

testing of the system occurred throughout the system’s development due to the Feature drivenmodel used.A Feature was thorougly tested before development began on another.

During the finalintegration period most of the remaining errors were found and quickly fixed by using various use cases

simulating the user’s actions.

Errors encountered usually involved a mismatch between the cachedobjects and the persistent store.

Business Cycle testing showed that reminders and notifications operate as expected, as well as showingthat users would be logged off after 30 minutes.

The Performance testing was done using a JMeter test case.Two of the

results are given below.

Fig 11:

JMeter Performance Test 1

44

Fig 12:

JMeter Performance Test 2

The response times obtained in the resultsdo not take into account network latency, as the machinemaking the requests was calling the backend directly.As can be seen in the first figure, the averageresponse for a request is 334 milliseconds, well within acceptable limits.The 0 obtained for a minimumresponse time shows that the test machines were located near each other, not taking into account networklatency. The highest maximum obtained in both results was 15256 milliseconds

which was likely anoutlier given the high standard deviation in that method,while stillwithin the acceptable rate of 20seconds. The deviation of results was due to the server load being increased as the test went on.

Due tothe setup of the test where the number threads were slowly ramped up most of the results for throughputwere quite low, except for the adding

20 items loop, with a throughput of 19 requests per second.

As the number of threads reached 160, new database connections could not be made fast enough due tothe default JVM heap size of 64MB being reached, resulting in the errors in the second figure. Byincreasing the heap size that tomcat is allowed, the threads it can support, and the number of databaseconnections

performance of the system can be increased.

This would result in much more memory beingneeded. The recommended amount of RAM that the backend would needis estimated to be around 1GBto supporta load of 500 threads.

The amount of memory required by the system can be reduced byhaving more functionality in the database layer as opposed to the logic layer.

45

5.Conclusions

The

Cellphone Shopper system’s aim outlined in the introduction was to provide a means to makeshopping easier. To find what features the system would need to do this user studies were done invarious iterations of the project. Once these features were detailed, a feature driven development modelwas used to ensure that each one was implemented successfully.

The final architecture chosen for theimplementation was a REST Web service that allowed the Web and cellphone interfaces to call methodson

the server. The server used was Jakarta Tomcat with a mySQL database.

The systemwas implementedsuccessfully with all the features specified in the design. The backendcomponent fulfilled the initial requirements outlined in theprojectproposal

except

for performingstatistical analysis. To recap,

the requirements were that the backend should:



Send and receive messages to/from both the mobile interface and the Web interface in XML.



Retrieve information from the database on request from either interface.



Translate data from the database to XML and send it to either interface.

46



Perform statistical analysis on user data and present the results to either user interface.

As to why the statistical analysis was not done, this was due to time constraints and adifficulty inshowing the results in the Web interface. In the user interviews done this was the feature that was leastrequired.

In development of the system the followingproblems were encountered:



While RESTful Web services are seen as simpler to implement than SOAP based Web Services,difficulty was had in finding any documentation on standards and sample code to implement it.The JAX-RS API that was usedwas just out of

development and was poorly documented,meaning that trial and error was used to implement it.



Integration of the components was the hardest part of the project. The sequence of methodscalled that the Web API assumed would be used was different to the sequence that the interfacesused. This resulted in some methods being merged as well

as new ones being made.

For instancethe Web API assumed an item would be created first, then added to a list. This resulted inproblems for the mobile interface as sometimes the request to create an item would fail. Thefailure rate of the mobile clientin sending requests was much higher than expected,

which led tomethods being created to update the database in batches.



The attributes that the interfaces needed to be returned were not defined specifically meaning thatthe API had to be frequently modified as the interface was developed.



As REST was used there was no WSDL resource generated to show the methods and theirparameters. When changes to existing methods were made or new ones added this had to becommunicated to the other members of the team. Aresource was created midway through theintegration phase that listed the methods and their parameters which greatly helped the process.



The choice of not using a persistence manager proved to hamper implementation, as variousattributes were often added to classes frequently which had to be manually stored and loaded inthe database. This led tounnecessarybugs being created,

due to the mismatch between an objectsdata and the corresponding data in the database, which had to be accounted for.



Jakarta Tomcat needed a number of tweaks in order to run a reasonable amount of concurrentthreads. The default heap size proved insufficent, with an optimal size not being found.

In conclusion, the choice of the technologies for the system was largely correct. The

Tomcat server andthe MySQL database proved easy to integrate together, and the JAX-RS API was well suited for workingwith Tomcat. Both interfaces had no trouble in requesting resources as a normal URI was used. Initialproblems were encountered when trying to send form encoded data to update the system but these wereovercome.

The message sizes of each request and response was greatly reduced compared to a similarSOAP message which was a big advantage.

In future the implementation of RESTful Web services inJava will be easier due to the inclusion of JAX-RS in

future Java EE and SE releases and betterdocumentation being made available.

Communication between project members was vital in integrating the components, this done through acombination of face-to-face talks and online communication. Communicating online was extremely

47

beneficial as most of the team members worked remotely, but more face-to-face discussions would havebetter helped clear misunderstandings.

In hindsightthe following approachesshould have been used in implementing the system:



A

persistence manager should have been used, specifically Hibernate should have been usedinstead of MySQL. Hibernate manages the

use of persistence classes in Java while still allowingthe use of normal SQL queries.

This would have helped during development where rapid changeswhere made to classes. Enterprise Java

Beans could have

been used, but the overhead wouldhave meant thatthesystem requirements in terms of RAM needed would have been higher.



A session token should have been generated and given to the interfaces

upon login.This wouldbea required parameter for accessto the API, making

more secure.



Sorting should be done in the database layer, as optimization can be done there.



Certain objects

that

used more memory than most

should have been lazily loaded, such asprevious lists.

6. Future Work

If the system were to be developed further there are several feature that could be included.



Statistical analysis of shopping history that was not

implemented.



Integration of the system withretail shops where items can be bought online.



A shop route can be specified before hand, and the items sorted basedon the route and not thelayout.



Admin system can be implemented where the administrator can add shops, cites, suburbs, itemcategories and default shop layouts.



Allow the use of the cellphone camera to scan in barcodes of items:

instead of having to enterthe details in manually it can be automatically added to the system, or checked out by scanningthe item.



The choice of which shop to buy an item from can be recommended by the system, based onprevious users or prices.

Hanslo, W. and MacGregor, K. 2004. The efficiency of XML as an intermediate datarepresentation for wireless middleware communication. In Proceedings of the 2004 AnnualResearch Conference of the South African institute of Computer Scientists and informationTechnologistson IT Research in Developing Countries (Stellenbosch, Western Cape, SouthAfrica, October 04-