Saturday, April 25, 2009

Scalability is one of the biggest concerns in enterprise grade application architecture and it gains more importance for SaaS applications due to their very nature of business model.

Scalability in an application can be achieved in three dimensions.

1. Vertical Scalability: This is the easiest form of scalability. In this approach just keep on adding horse power of m/c on which application is deployed. Practically, here effort on the part of application architectural and engineering team is zero.2. Horizontal Scalability: This approach is quite common. This approach supports two flavors. Hardware approach and software approach. In layman’s terms if one can run two or multiple instances of application then software based scalability can be achieved by utilizing m/c’s resources optimally. The hardware approach requires little effort on architectural and engineering team. In this approach application should be able to on multiple m/c s to cater increasing business demands. The common technique used to achieve this type of scalability is clustering – software & hardware clustering.3. Multi Component Scalability: To achieve this form of scalability application to be developed very carefully. This approach requires significant effort from architectural and engineering team. In this case components of application should be independently deploy-able and should be scalable – horizontally and vertically. The famous example of this approach is EJB based solutions where EJBs can scale separately from web server.

Due to very nature of SaaS application customer class is very important. Customer may have various needs in terms of QoS and security. This challenge open one more dimension of scalability.

4. Customer Class Scalability: In this approach one can scale customer as per the required QoS and security needs.

Just to make picture clearer, all four dimensions of are applicable to binaries (code) and persistence layer (RDBMS; hierarchical data base – LDAP; File system – logs, file servers, multimedia repositories, etc).

Friday, April 24, 2009

Sunny Agarwal: no thanks boss.. I would like to have a discussiontushar Jain: suretushar Jain: any timeSunny Agarwal: in fact, just wrote the commentSunny Agarwal: but, forgot to ask one thingtushar Jain: whichSunny Agarwal: in which scope r u trying to explore provisioningSunny Agarwal: provisioning is a generic term.tushar Jain: agree provisioning is generic termstushar Jain: but in SOA parlance we always talk provisioning of servicestushar Jain: but most of people do not understand meaning of this termtushar Jain: and most of the information available on net talks provisioning in terms of Telecommunicationtushar Jain: and little bit of identity managementtushar Jain: Even Tivoli provisioning manager does not explain what is provisioningSunny Agarwal: u said u r writing in context of TelcosSunny Agarwal: if u wishes to explore provisioning in a larger scope then may be i can give u some pointstushar Jain: sureSunny Agarwal: Cloud computing and utility computing r emerging areasSunny Agarwal: where these provisioning concepts will cometushar Jain: I love to knowSunny Agarwal: Ok, so there are 3 pillars:Sunny Agarwal: 1. Grid ComputingSunny Agarwal: 2. Utility ComputingSunny Agarwal: 3. Cloud ComputingSunny Agarwal: where service provider is agnostic to service requesterSunny Agarwal: typical SOA arenatushar Jain: okSunny Agarwal: but at the same, time we need light governance in order to facilitate transactionstushar Jain: oktushar Jain: so here u have to provision a service for an entitySunny Agarwal: one interesting area is scalable provisioningSunny Agarwal: where u scale up systems dynamicallytushar Jain: what is thatSunny Agarwal: to provide dynamic service levelsSunny Agarwal: here virtual systems and virtualized hardware systems also come in pictureSunny Agarwal: again, I am just talking to u... what I have read across and seen...Sunny Agarwal: trying to know, how u want to move ahead on this topic nowSunny Agarwal: u wants to lay out a framework for delivery, solution, or a business process...Sunny Agarwal: or an Infrastructure setuptushar Jain: waittushar Jain: I see provisioning as subset of SOA because Cloud, Utility and Grid and SOA at different level and extension of SOAtushar Jain: the essential feature of all these three pillars is servicetushar Jain: so service should be provisioned in respective environmenttushar Jain: which need some sort of provisioning engineSunny Agarwal: okSunny Agarwal: so would it be something like a BPM engine, where this engine would maintain identity or service instance detailstushar Jain: I see provisioning engine will be an be an essential part of SOA stack just like ESB and UDDISunny Agarwal: ok, then how does u differentiate Service Registry products from provisioningSunny Agarwal: engineSunny Agarwal: something like Active Matrix Service gridSunny Agarwal: which provides all this?tushar Jain: I see service registry as mere listing of available service which may also indicate QoS. But not essentially as tool which can make service available to an entity - human & m/c.tushar Jain: This thought encompasses identity management systems into provisioning systemtushar Jain: provisioning engine and registry in conjunction will lay down the governance system.Sunny Agarwal: hmm.. Seems I need to read on this then. Identity management is a new word to metushar Jain: sureSunny Agarwal: so, this Provisioning system will come in between any service requester and service gridSunny Agarwal: as a receptionisttushar Jain: perfectSunny Agarwal: and yet, I have a flexibility to call the office boy or grid directlytushar Jain: once you are introduced to office boySunny Agarwal: yeah.. So that would be something likeSunny Agarwal: one enterprise service gridSunny Agarwal: internal business domains can call office boy directlySunny Agarwal: and external entities who wish to use these services better come thru receptionistSunny Agarwal: hmmmm..... Here we can have a relation of telco's..... N/W bulk providers and Virtual Service providers...tushar Jain: it should be true but trusted partners can call office boy directly, if system is desined in this way to enhance performance and reduce overheadSunny Agarwal: yepSunny Agarwal: so , what r the business drivers for this concepttushar Jain: Ant just be clear, I have tried to define provisioning g in dependent of domain - Telco or ISPSunny Agarwal: like for SOA, main driver is reuseSunny Agarwal: and inter oppSunny Agarwal: what is for provisioning?tushar Jain: I have used mention of these domain to just start the conversationSunny Agarwal: yeah... i got that... and it’s new to me too... something broadertushar Jain: main driver is governanceSunny Agarwal: broaderSunny Agarwal: I will talk abt it with my mentors in wipro alsotushar Jain: sure any timetushar Jain: and I am available on mail and messengerSunny Agarwal: hmm...interestingSunny Agarwal: i hope u r on my linked inSunny Agarwal: and ur blog is there now..tushar Jain: my bog is there in my profiletushar Jain: and I have started a group on linkedinSunny Agarwal: gr8 gr8tushar Jain: Now I need a favorSunny Agarwal: bolo sirtushar Jain: can I publish this conversation as blog entrytushar Jain: if you permitSunny Agarwal: why not.Sunny Agarwal: would only make me feel better.tushar Jain: thanksSunny Agarwal: my objective is to have authentic info and understandingtushar Jain: me tooSunny Agarwal: and especially in SOA world, it’s very deceptiveSunny Agarwal: terms r so misusedSunny Agarwal: OK, Tell me where does provisioning fits in OSS /BSS solutionsSunny Agarwal: at what leveltushar Jain: ok, first you tell me what do u understand by BSS and OSSSunny Agarwal: well, OSS is operational support system and BSS is business support systems..Sunny Agarwal: if u forgets about backend n/w infraSunny Agarwal: u r left with services and products plus all management around itSunny Agarwal: BSS would be more centric to business systems...tushar Jain: okSunny Agarwal: and OSS for operational.. Following the basic FAB stackSunny Agarwal: levels would be customer / internal / backend n/wSunny Agarwal: yes, there is a point.tushar Jain: whatSunny Agarwal: if u looking at unified OSS solution, then not much integration and services....Sunny Agarwal: if u follow best of breed COTS prods and then integrate them for OSSSunny Agarwal: then SOA would also come inSunny Agarwal: ab u punch intushar Jain: so what BSS and OSS are doingSunny Agarwal: nothing...tushar Jain: with integrated approach or as compartmentstushar Jain: they are just offering servicesSunny Agarwal: that’s ittushar Jain: Suuny I have to go in meetingtushar Jain: we will again chat on Monday

Wednesday, April 22, 2009

Currently, I am engaged in architecting a provisioning engine which will primarily cater to ISPs (Internet Service Providers) and TELCOs. While discussing architectural options and details, I realized lot of persons does not understand:

What is PROVISIONING?

Persons who do not understand provisioning include both technical and business.

To know what are experts are saying about my question, I googled but not able to get any satisfactory answer.

To understand provisioning, I have developed a small story.

Let’s assume Mr. X has applied for a job and reaches to an office for an interview. So now he is sitting on nice sofa and waiting for interview. Now he feels thirsty and requires a glass of water. He asks receptionist for water since there is no water cooler around. She asks Mr. X to relax and wait. She calls an office boy to serve water to Mr X. Office boy brings a glass of water and serves to Mr. X. Mr. X drinks water and thanks him.

In this story, Mr. X is a client who is asking for a service to be delivered to him. He is not talking to service provider but to provisioning engine (Receptionist) which in turn asks service provider (Office boy) to provide service (a glass of water) to client. Service provider provides service to client.

To further this example, assume Mr. X needs one more glass of water. Now will he ask again to receptionist or directly to office boy? There is no definite answer. If Mr. X wants water immediately after first glass, he can ask office boy for second one. But if Mr. X wants second glass after office boy has left the scene, he may ask receptionist again or may approach Office boy directly for glass of water.

So in real life systems, serving a service depends upon lot of factors:

1. Timings of first and subsequent service demands2. Infrastructure and system capabilities3. Conventions followed in particular domain/vertical

So what is provisioning:

Provisioning is a mechanism by which an intermediary system facilitates delivery of service to a client. So that client can maintain its ignorance about service provider in terms of how to register for delivery of service.

Wednesday, April 15, 2009

The problem stated in Design Challenge can be broken into three parts:

1. User Interface2. Object Model3. Data Model

Let us tackle each part separately and finally glue them together.

1. User Interface: User interface as depicted in IMAGE-1 requires that data to be populated in State list depend on selections in Country list. Similarly City list picks up cue from State list and finally PIN/ZIP list depends upon selection in State list. This type of pattern is also present in IMAGE-2, 3, 4 & 5. The traditional solution for this problem is to load page on each selection which affect subsequent list. But this results unnecessary heavy network traffic. To solve this problem AJAX is most efficient and contemporary answer.

2. Object Model: Assuming one to one correspondence in data model and object model, I can assume that following object model exists.

IMAGE-7

To categorize customers, one will build population criteria. This population criterion will be composed of AgeGroup, Status, Role, Gender, Country, State, City and PIN/ZIP Code. So there must be one class corresponding to Population Criteria. To hold customers as per population criterion there should be a class.

IMAGE -8

3. Data Model: To hold data for selection criterion and population there seems to be two tables required.

IMAGE-9

Notice that TB_PopulationCriterion has only three columns ID, Name & Description. There is no provision to store selection criterion. So, how to store it? There are few ways to store it.

Option 1: Create columns pertaining to AgeGroup, Status, Gender, Role, Country, State, City and PIN. Insert multiple rows pertaining to each criterion. This will result in multiple row insertion for a single population criterion in TB_POPULATIONCRITERION. This will also results in null columns. Updation of population criterion will be difficult, so recommended way is delete and then insert new rows. This option will require change in table structure in case if any new criterion is added over time period.

Option 2: Create columns pertaining to AgeGroup, Status, Gender, Role, Country, State, City and PIN. Insert only one row pertaining to one population criterion. This approach will require comma separated string to be made while inserting and then parsing it when taking data out. This approach is again afflicted by following issues:

a. Updation of population criterion will be difficult, so delete row and insert new.b. Need parsing code in object model.

IMAGE-10

Option 3: Store Java/C# object of Population Criterion and parse as needed. This approach has its own pitfalls:

a. Storing of Java/C# /etc. object will mandate that BI tool which will insert dynamic Population Criterion will also understand Java/C#/etc.b. Storing binary object may pose its own problems like mismatch in object versions.c. Stress on persistence layer.

IMAGE-11

Option 4: Store XML version of Population Criterion object. This approach seems to be very conceivable but raises following points:

a. Object Model has to support parsing logic of XML file.b. XSD to be developed.

IMAGE-12

Option 5: This approach is most dramatic but seems to be easiest. Store SQL queries pertaining to:

Sunday, April 12, 2009

Software of any size has quality attributes. These quality attributes decides the fate of that software from its idea/inception to its retirement. These quality attributes are independent of its size – enterprise, embedded, desktop, etc; architectural philosophies followed – SOA, Message Driven, Object Oriented, Procedure Oriented, etc; nature – real time, near real time, batch mode, synchronous, asynchronous, etc; usage – project & product; etc. For the ease of understanding these attributes may be classified as:

1. Business Quality Attributes: As name implies these attributes are interest of business persons and directly affect viability of software.a. Target Market Segmentb. Time to marketc. Roll out scheduled. Team composition – at each stage of SDLCe. Resource Composition – at each stage of SDLCf. Availability and cost of skill setg. Legacy system integrationh. Legal Compliance

4. Design Quality Attributes: These attributes are derivatives of System and Architectural quality attributes. But these are very important and lay foundation of successful implementation and support.a. Trace-abilityb. Information hiding and encapsulationc. Abstractiond. Loose Couplinge. Reusabilityf. Self documentationg. Separation of concernh. Increment-abilityi. Design for changej. Patch-abilityk. Usage of contemporary standardsl. Usage of contemporary best practicesm. Usage of appropriate design patterns

5. Implementation and Support Quality Attributes: These are the attributes which governs coding to retirement of software. Few of the quality attributes at this stage are resultant of System, Architecture and Design quality attributes and remaining can trace their origin in SDLC methodology followed, business decisions and environmental conditions.a. Outsource-abilityb. Testabilityc. Definition of SLAsd. Trace-abilitye. Modular-ability

Friday, April 10, 2009

Though this book was published in end of Dec 2008, I am able to read now. After reading this book I am not able to understand what the central theme of this book is. This book starts with idea of defining of SOA which it not successful. Then it suddenly jumps to web service and its implementation. Then with all of sudden book starts talking about Data service and finally it compares EAI and SOA.

It seems that book has been compiled as silo for each chapter. None of the chapter seems to be interrelated. There seems to be no connection among chapters.

The only good things about this book are that it compares web service implementation using JAX-WS 2, Axis 2, Spring WS & XFire/CXF and RPC & Document based web services.

Now basis of these attributes customer will be categorized (category & Sub-category) and then population criteria will be developed to populate those sub-categories.

There is only one level of categorization is allowed. It means a category can have only one level of sub categorization. So Cat1 has sub-categories SubCat11, SubCat12, SubCat13. Under SubCat11, SubCat12, SubCat13 only customers reside as per the population criteria. The development of population criteria is user driven. To develop a population criteria following user interface should be used.

IMAGE-1

The interpretation of selected items is (as depicted in IMAGE-1):

(Role: R1 OR R2) AND (Age Group: 0-25 OR 50-onwards) AND (Status: Active OR Inactive) AND (Gender: Male) AND (Country: India OR USA) AND (State: Karnataka OR Bihar OR California OR Texas) AND (City: Banglore OR San Jose OR Austin) AND (PIN/ZIP: 560030 OR 560070 OR 73344)

Apart from Population Criterion Creation screen there will be screens which will do the following tasks:

1. Update a population criterion2. Using population criterion, sub categorize customers.3. Update sub categorized customers as per the updated population criterion.

Update a Population Criterion screen will be similar to Population Criterion Creation screen. While appearing on computer terminal, Population Criterion Creation screen will show what are the different items are pre selected which user can change as per the requirement.

IMAGE-2

Populate a sub category screen will depict a selected population criterion and category/sub category. This screen will allow user to populate customers in a sub category as per the selected population criterion.

IMAGE-3

Re-populate a sub category screen will depict the updated population criterion and category/sub category. This screen will allow user to update population of customer under selected population criterion and category/sub category.

IMAGE-4

To support above screens there will screen which will allow formation (as well as updation) of association of category/sub category and population criterion. Call this screen as Association: Population Criterion & Category/Subcategory

IMAGE-5

The existing RDBMS schema is:

IMAGE-6

We can add tables/views as desired but do not change any existing tables/views.

Non-Functional Requirements

1. The selection items (Current: Role, Age group, Status, Gender, Address; Expected: Marital status, Family size, etc.) count is expected to increase as more static attributes will be captured over time period.2. To make population criterion using dynamic attributes (e.g. shopping behavior, browsing behavior, etc.) Business intelligence and/or data ware housing will be used. But even in these cases population and re-population should remain with in control of MyApplication.3. The population criterion creation and updation should be fast enough to qualify for Gold (confirmation of success or failure should be displayed to used with in 800 ms) service level.4. The population and repopulation of customers in a sub category should qualify for Bronze (confirmation of success or failure within 2000 ms or information of “working asynchronously” within 300 ms) service level.5. Association: Population Criterion & Category/Subcategory screen should qualify for Platinum (confirmation of success or failure within 200 ms) service level.6. SQL query should be SQL 92 compliant. Usage of Store Procedure, Trigger and Functions is prohibited.

So you got the problem, what are the possible solutions with their positives and negatives.