Author: admin

What is NServiceBus

NServiceBus is a Microsoft .Net based framework for implementing the Enterprise Service Bus pattern. Its a framework for implementing reliable messaging through various transports without having to write boilerplate code. NServiceBus also provides facilities for pub/sub, retries, format abstraction, encryption, workflow orchestration, and scalability. Below are few highlights of the system.

Let’s use .Net core 2.0 for building a simple bus. Everything in NServicebus can be configured using a simple class called “EndpointConfiguration”. An endpoint is a service which uses NServicebus framework for handling messages.

var endpointConfiguration = new EndpointConfiguration("UnicastBus");

NServiceBus uses persistence to store all the configuration and temporary data so that the data is not lost while there is a failure (fallacies of Network). We can configure NServiceBus persistence using the below code. We are using an In-memory persistence for the sample app. But in real applications, other reliable persistence like SQL, Raven, Azure Service fabric etc. will be used.

endpointConfiguration.UsePersistence<InMemoryPersistence>();

NserviceBus is a framework and is not a message queues on its own. However, it uses an Adapter pattern to fit any open source & commercial queuing systems. In our case, we are wiring it up with RabbitMQ (an open source message queuing system written in Erlang).

NServiceBus is one of the frameworks for doing reliable messaging for handling the fallacies of distributed computing. This is just a gentle introduction to NServiceBus. We shall see more advanced concepts in the future posts.

How many times have you encountered a situation where the software built on one platform doesn’t work on another platform? It is a very expensive affair to ensure that your software works on all the platforms (Mobile, Tablets, PC and so on). This is one of the crucial problems which Docker helps you to resolve but this is not it. There is much more to Docker which we will explore as we move in my upcoming blogs.

Docker is a container management service or a tool which is designed to help developers to deploy and run applications by using containers. It ensures that the code will behave the same regardless of environment/platform.

Key components of a Docker

Docker File – A dockerfile is a text document that contains series of all the commands a user a make use of to create a Docker Image. The Docker images are automatically built by reading the instructions from a dockerfile.

Docker Image – It’s a lightweight snapshot of a virtual machine. It is essentially an application with all its requirements and dependencies (read-only copy of OS libraries, application libraries etc.). When you run a Docker image, you get a Docker container. Once a Docker image is created, it guarantees to run on any Docker platform. These can be retrieved to create a container in any environment.

Docker Registry – A Docker image can also be stored at online repository called Docker hub. You can also store your image in this cloud repository (https://hub.docker.com/). You can also save these images to your version control system.

Docker Container – A Docker container is nothing but a Runtime instances of Docker images.

Benefits of a Docker

Build your application once and use it in multiple environments without the need of building or configuring once again. The application built on dev environment guaranteed to work in prod environment.

Centralized Docker repository makes it very easy to store and retrieve. Moreover, you don’t need to build the image from scratch. You can always leverage existing image and go from there. The sharing of an image becomes very simple as well.

Version Control – You can always create next image and version control them.

Isolation – Every application works inside its own container and never interfere with other containers. If you no longer need an application, you can delete its container. Every container has it’s own resources hence there is going to be no challenge.

Security – The isolation ensures that the applications that are running on containers are completely segregated. The container cannot look into or have a provision to control processes running on other containers. This greatly enhances security.

Multi-cloud platforms/Portability – The image built on Amazon EC2 would very well be ported to Microsoft Azure.

Productivity – This is an implicit benefit of using Docker. The speed of development is much faster as the main focus is writing code and business over worrying extensively about deployment/testing.

Choosing a correct database is very important in software development.

The relational database maintains ACID rules. Relational model requires relational database engine to manage writes in a special way. It locks the rows and table to maintain atomicity and consistency. Protecting referential integrity across the tables and rows increase locking time. Increase locking time means higher latency and slower application. Developers face some problems with the Relational database such as:

Object Relational Impedance Mismatch: When RDBMS is being served by an application program written in an object-oriented programming language, objects or class definitions need to be mapped to database tables defined by a relational schema. Misalignment layer of application objects to tables and rows is called Impedance mismatch.

As an example, below are the schema definitions:

Below is the application code:

Class Foo
{
int Id;
string[] colours;
}

In this example, object foo contains field Id and colours (which is a bunch of strings). To store this into the database we need to create 3 tables.

Main object

Colour information

Relation between color and main object

These forces the developer to write a mapping layer such as Entity Framework or ado.net to translate the objects in memory and what is saved in the database.

Many of the developers use object-oriented languages in development. Objects are not rows and tables. All objects are not uniform. Mapping those objects into rows can be painful.

Couch DB is schema-less. There is no relation between a collection of objects. A developer can store any type of document. The documents can be flat and simple or as complex as the application requires.

Couch DB document for “Foo”:

{

ID:1,

Colours:[“red”,”yellow”]

}

Scalability Issues: Scaling out and replicating data to other servers need to increase the lock. Relational database tries to be consistent and increase the locking time and as result application gets slower.

Replication is one of the features in couch DB. Replication takes advantage of clustering to achieve scalability. We just need to mention the source and destination database. Couch Db will handle to replicate the data into a destination. This can also be achieved through a REST call. It should be a POST request to the endpoint “_replicate” with the source and destination servers specified in the body of the request.

RDBMS With MS SQL SERVER

NO SQL WITH COUCHDB

Define table

No schema

Rows and Columns

Document

Dynamic Query

Predefined query

Join/Relations

Not required

T-SQL

Map Reduce

OLEDB/ODBC/EF/ADO.NET

REST API

Management Studio

Futon

Constraints, Triggers, SPS

Validations, show & List Functions

Couch Db stores JSON for documents, JAVASCRIPT as MAPREDUCE queries and HTTP for an API. Couch DB can be considered as a b-tree manager with an HTTP interface. It uses a data structure called a B- tree to index its documents and views. It maintains optimistic concurrency via MVCC (Multi-version concurrency control). Previous versions of a document are available till the database is compacted.

B-trees append data only to the database file that keeps the B-tree on disk and grows at the end. B-tree delivers fast searches, inserts, and deletes.

Couch DB features on the server-side document validation and on the fly document transformation. Although a document is stored in JSON, the document can be served as XML or CSV.

Architecture OF Couch DB:

The lowest tier is simple JSON based file store. The storage engine is responsible to accept the JSON documents and serialize them into the disc. Storage engine can access the JSON store. The query engine does fast access to the stored in JSON store. The query definitions are JAVASCRIPT functions stored in the database. B-tree structured index is built on every query and stored in a database. This helps the query engine to read the data fast. Replication engine provides capabilities in master-master bidirectional replication. Through Rest API we can access any of the three capabilities.

The Service Oriented Architecture (SOA) is not a technology instead it is a collection of Services. Precisely it is an architectural style for building business applications using loosely coupled services which communicates with each other.

SOA = Messages + Services

Services – Self-contained business functionality and communicate using messages.

Messages – It is a discrete unit of communication. It should be cross-platform, secure, asynchronous, reliable, follow industry standard and able to describe and discovery service.

The SOA shifts your thinking from classes and objects to messages.

The webserviceis nothing but a service available over the web (Or method of communication between two devices over a network) while API(Application Programming Interface)acts as an interface between two different applications so that they can communicate with each other. The API may or may not be web-based. Having said that all the all the web services are APIs but all APIs are not web services. The webservice always needs a network while for API it may/may not be needed.

SOAP is an XML based protocol which uses WSDL for communication between source and recipient. REST is an architecture style protocol which uses XML and JSON to communicate between consumer and provider.

SOAP uses RPC while REST directly uses URL.

The transfer is over HTTP, FTP, SMTP and other protocol for SOAP while for REST it is HTTP only.

SOAP is hard to implement over REST.

The performance is slower compared to REST.

SOAP is more secure since it does define its own security. The REST is less secure. The transport defines the security. (Smart consumer and provide while dumb pipe).

The SOAP message is large. The core element of SOAP message is a SOAP envelope. The SOAP envelope defines the start and end of the message. It has two parts 1. SOAP HEADER 2. SOAP BODY

WSDL – Web Service Description Language

The WSDL is an XML based interface that is used to describe the functionalities of the webservices .

In a typical agile team, the scrum master is the coordinator/facilitator who makes everything going in the team properly. However, when the team is highly self-organising, do we even need a facilitator? We have a detailed post on the role of a scrum master here.

We have asked this question to a group of experts in different social forums and we have received around 1200+ responses. Here is a quick stats around the same.

9% feels we don’t need scrum master at all.

14% believes that the SM is a full-time role.

Rest 77% strongly believes that full-time SM is effective in the start, or in an immature team.

This survey reflects what we have observed in our agile projects.

When we are building a team or team is immature(staffing changes etc), we need SM to help and coach the team by asking right questions. As the team grows up in maturity, the individual team members know how to organize themselves and hold each other accountable. This is one of the important reason why SM role becomes redundant after some time. And this is also the reason why organizations prefer to hire technical people who can play scrum master role. But in order to attain this maturity, the team members will have to be open for new learnings and continually improve.

Here we have picked some (not limited to) of the comments from the experts we did experiment with. The question being asked was

Do we really need a scrum master in Scrum project? If so, does it have to be full time? Isn’t self-organizing team killing the concept of SM?

Comments from experts

1. Yes….as much it seems like an oxymoron to have both concepts in concert with each other, you need to have the “check and balance system” embodied by having a separate SM from the team. The team is made up of humans, who are fallible and will attempt to take shortcuts over time. The SM is a check against the scrum team’s desire to “get things done” and circumvent the scrum process.

2. Ideally any team member can be called as a Scrummaster. I can only call the SM designated as a process expert contributor.
The key responsibilities certainly needs a defined role to drive it so it reduces the burden on either the Manger or team.

Also , with value and business ask in mind its challenging for the Manager to inspire, create value, and follow the princple 5 ” build projects across motivated indivduals and trust them that the job can be done” whilst if it can be done, it can be tried out.

3. Scrum is a smallish change to jow people work. 5 hours a week. That said…if you’re running scrum in a not friendly to agile work environment….scrum is essential. However, if you’ve got a long term agile team, whose environment lets them work, the role is less necessary.

Really…the scrum master role as practiced is NOT about scrum, but rather about making the team work effectively together. Once that’s good…?

The XP model creates agile team functionality differently, by shifting work patterns for 35 hours a week, rather than scrums 5 hour change. Honestly, that changes team culture far faster than scrum, and the tech practices shops dont find the role necessary.

Without a deep established culture or hardcore paired-tdd-ci practices, you want a scrum master.

4. Real professionals don’t need a SM. It will save them time.

5. Once the team is self reliant, scrum masters role becomes redundant.. Either scrum master has to be one of the tech leads or a contractual role to train the teams.

7. That depends on the nature of the project. If it is a fairly large sized project with multiple dependent upstream and downstream systems, there would be a fair amount of communication and coordination required. This should happen before, during and after sprint planning. Team members do not need to worry about all these administrative work. They can concentrate on actual tasks. Also, in my personal experience, however well planned you are, things do not go smooth as planned. In that case, there is a need to capture details around any delays by our own team or any other dependent systems and review and revise the integration tests, release plans, deploys to higher environments and finally present the cost of delay and value being delivered to business. This is all just a few tasks for an SM when the project is fairly large sized with up/downstream dependencies. Other typical tasks include, communications with end users whenever required, challenging and motivating teams, conducting team building activities which helps team not only build relationships but provides some relief from day to day and hour to hour work.

8. However mature teams are, if a candid discussion happens during grooming, planning team members need someone else to facilitate their conversations, disagreements and help them come to an agreement. SM can apply various techniques in these situations and it would be totally unreasonable to expect a participant team member to play that facilitator role and be neutral (just not practical)

9. I’ve had the great pleasure to work with many different teams across many organization types, development processes, and industries. Whenever a team loved Scrum Masters, it was because the Scrum Masters were properly performing their role and getting things done, removing road blocks for the team, facilitating effective meetings and conversations, and so much more.

All of the projects I’ve been on where there was no formal Scrum Master role were teams that had one at one point, but dropped them because they were not effective. They blamed the role instead of the individual.

Find effective Scrum Masters and hold them accountable just as you would any other team member. If they aren’t getting the job done for you, find one that will!

10. When a team is mature enough, this is true that need for a Scrum Master is over.
If the team is self-organizing, cross-functional and respects naturally Scrum ceremonies ; if communication with the PO is smooth, there is no misunderstandings, it means that a Scrum Master would be superfluous.
On top of that, the role of Scrum Master still exists: the difference is that it is endorsed by team members, with no prior assignation, but spontaneously, depending on the context.

11. The need of a SM depends on the tasks that he performs. So whether a SM will be required or not will be determined by the availability of resource to perform that role. So if there are skilled resources who is available to perform those tasks then I think the need for SM is already fulfilled.

12. Although SM role is critical but if thats the case all the times it means your team in not improving. If the team is improving, you need to assign other work to SM (Assign other projects, ask to contribute on technology, design etc). If that is not happening, you should plan to bring new SM.

13. Scrum master is ‘Servant Leader’ in practice. Which means he/she has to adapt to leadership style based on ‘Team’ is at what stage. Whether ‘Forming’, ‘Storming, ‘ Norming’ or ‘Performing’. Once the team reaches ‘Performing’, SM job becomes redundant. And at ‘Adjourning’ the last stage SM is no more needed team to do its function effectively, independent and consistent performance. At this SM would also reach its peak of leadership ‘Pinnacle’.

14. A team can gel despite all kinds of organizational impediments to agility. The SM is not obsolete until those impediments are identified and addressed effectively.

15. For a program with multiple streams and releases, a dedicated SM is more or less a necessity. With obvious(more) focus during the initial stages, the SM involvement can reduce as the team imbibes the mindset that is expected of a proper self organised team. He or she will ideally bring in the culture that sets the team on the progressive path

16. A dedicated Scrum Master is not needed in a Performing team. However, it is frequently the case that a team degrades over time after the Scrum Master leaves – so unless the team is able continuously keep focus on improving they might at least need regular check-ins from an agile coach (could be an actual agile coach or a leader in the organization or a Scrum Master from another team).

17. I don’t think need of SM can be completely eliminated, whether it is performing team or not as there are lot of things SM take care of such as improvement in processes, removing impediments, taking care of backlog, running sprint etc.

Someone from team should not be SM as this is independent role and it has to be a specialist for the job.

18. A team does not mature takes the time to mature. In a mature team, Scrum Master may not require, so a lot of other things e.g. sprint planning and refinement sessions merging, On demand retro instead of one retro per sprint, time boxing of sprint ( may be kanban), etc.

19. I don’t think the role of SM can be completely eliminated and not at least until the team is mature enough to identify impediments and able take actions to remove them or retrospect themselves. Yes, it would be a good idea to have a team member with willing to serve as SM as well, but that should happen at a later stage only as in SCRUM SM is a separate entity and can be supposed to serve multiple teams at the same time.

20. I think a SM is need at the start of the POD, but as stated above once the team starts moving and has proved they can produce and become self serving then the SM has be become obsolete. Now this can only happen if your product owner and B.A. are in tuned with the POD.

21. Instead of picking SM from outside, you can identify a good fit for this role within team and keep rotating. This would certainly increase your chances of getting better results and at the same time you are helping you team member to play interesting role who aspire to get into management.

Lots of teams that I work with generally follow the best practices of a Scrum stand up meeting. However, as time goes by team tend to fall into the slippery slope of just reporting what they have accomplished the previous or the current day and completely ignore the next actionable & impediments. This happens more when the team/scrum master is not very strict about the process. In this article, we will see the reasons why this is important and the benefits of the same.

Life without goals

Before we delve deep into practices of Scrum method, we will talk about few general principles how our subconscious works. As per the basic behavioral psychology, there are three parts of the self-concept.

a) Self-Ideal – Goals, aspirations, dreams etc.

b) Self Image – impression about self-based on previous experiences.

c) Self Esteem – Emotional component of how much we like ourselves.

When someone sets goals and achieves them, the self-image corrects and the self-esteem improves. And they start setting bigger goals and continues to go on an upward spiral.

How Psychology relates to Scrum

Whenever a team member talks about “What they will do ” before the next meeting, they are indirectly setting a time-bound goal for themselves. When they come and report the achievement of the same the next day, the self-image and the self-esteem goes up. This will bring more success as the time goes by.

The Sprint is systematically designed methodology using the basic psychology of commit and achieve cycle.

Let’s do an experiment. If you are part of an Agile team, state your goal for the day loudly to your team during the stand-up meeting. You would have that in your back of your mind entire day. At the end the of day, if you accomplish what you stated, you will have an amazing sense of accomplishment. Otherwise, you feel that you have not done enough. This comes to most members as long as they care about the company and their career. All you have to do is take that seriously and you would find yourself growing much faster than your peers.

I have personally asked this to many people and every time I had the same result. Isn’t it amazing? How simple it is to grow in an Agile team. Isn’t it?

So Next time you are in a stand-up meeting, would you talk about “what you are going to do “?

In business software applications, the domain objects (entities) are used to represent the business domain. As the application grows and adds more business logic, the service layer, mappers and other patterns gets applied. Often this leads to domain object becomes bloated and the related components become huge & un-maintainable.

CQRS solves the common problem of having a bloated Domain objects. The domain objects get bloated largely because of bounded context. The series of contexts which makes developers think that a single domain object is sufficient to handle all the related things. For example, a large Invoice object for handling Invoice, Shipment and handling change of address for customer . But in reality, these contexts (invoicing, shipment and change) need not be related to same Invoice entity.

What is Command, Query Responsibility segregation (CQRS)?

In order to simplify the Domain objects, CQRS proposes to have two types of domain entities.

With this separation, the complexity (number of fields, methods) of entities used becomes simplified. And hence the Data mapper layers & the service layers becomes more simplified.

Where can I use CQRS?

Largely complex system: Applying CQRS on a simple CRUD operation based system is a over kill. When there is a domain heavy system, like banking and financing systems, LOB applications where business logic, lots of boundary conditions are heavy. Where it makes DDD (Domain driven design) provides high value.

Situations where you will apply Microservices, Eventual consistency and Event Sourcing. When we have separation of concerns using CQRS, the microservices becomes much simpler to design and maintain. With Event sourcing we are focused on getting the data (query) from other related sources and is what CQRS propagates.

Final words

CQRS is a carefully thought out pattern for simplifying & solving large and complex systems.