Monthly Archives: December 2015

Post navigation

Document structure:

MongoDB will represent relationships between documents with:

Embedded data – The whole document can be retrieved in a single query like this. The drawback is that if the embedded document keeps on growing too much in size, it can impact the read/write performance. The document max size is 16MB.

Manual References – similar with FK in RDBMS. It is the approach of designing normalized relationship.

Database References – a document references documents from many collections.

Below some examples with each type of representation applied to different relations.

A boolean acknowledged as true if the operation ran with write concern or false if write concern was disabled.

If you want to add multiple documents at once, MongoDB provides insertMany with ordered (default – it will stop at first error) or unordered mode – even there are errors MongoDB will continue the insert with the next document.

C#

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

db.books.insertMany(

[

{

"_id":"001",

"title":"Book 1",

"year":1981

},

{

"_id":"002",

"title":"Book 2",

"year":1986

},

{

"_id":"001",

"title":"Book X",

"year":1983

},

{

"_id":"003",

"title":"Book 3",

"year":1983

}

],

{

"ordered":false

}

);

output:

[code language=”javascript”][/code]

Update with upsert = true

If set to true, creates a new document when no document matches the query criteria. The default value is false, which does not insert a new document when no match is found.

You can use save command to quick update/insert a document into collection:

JavaScript is a dynamic, object-oriented, and functional scripting language. One of the features that make it win over Java Applets in the browser scripting war decades ago, it was its lightness and non-blocking event loop.

Bocking means that when one line of code is executing the rest of it is locked waiting to finish. On the other hand, non-blocking gives to each line of code a shot and then through callbacks it can come back when an event happens. Programming languages that are blocking (Java, Ruby, Python, PHP, …) overcomes concurrency using multiple threads of execution while JavaScript handles it using non-blocking event loop in a single thread.

Some companies like Paypal moved from Java backend to NodeJS and reported a increased performance, lower average response times, and development speed gains. Similarly happens to Groupon that came from Java/Rails monoliths.

Overzeal­ous Services – don’t go straight into microservices, (fol­low­ing the hype per­haps?) rather than start­ing with the sim­plest thing possible. Instead, you should start by doing the sim­plest thing pos­si­ble, iden­tify the hotspots as they become appar­ent and then extract them into microservices. Start with monolith.

Schemas Every­where – avoid having a shared data­base between dif­fer­ent ser­vices. This cre­ates tight cou­pling between the ser­vices and means you can’t deploy the microservices inde­pen­dently if it breaks the shared schema. Any schema updates will force you to update other microservices too. Instead, every ser­vice should have its own data­base, and when you want data from another ser­vice you go through its API rather than reach­ing into its data­base and help yourself. Use semantic versioning.

Spiky Load between Services – You often get spiky traf­fic between ser­vices, and a com­mon solu­tion is to amor­tise the load by using queues between the services. Use rabbitmq.com, redis, etc.

Hard­coded IPs and Ports – One solu­tion would be to use a dis­cov­ery ser­vice such as con­sul or etcd, and there’s also Netflix’s eureka. Another solu­tion is to use a cen­tralised router. Both solu­tions require reg­is­tra­tion and dereg­is­tra­tion, and both require high avail­abil­ity and scalability.

Dog­piles – If one of your ser­vices is under load or mal­func­tion­ing, and all your other ser­vices keep retry­ing their failed calls, then the prob­lem would be com­pounded and mag­ni­fied by the addi­tional load from these retries. The solu­tion here is to have expo­nen­tial back­off and imple­ment the cir­cuit breaker pattern. Many libraries such as Netflix’s Hys­trix and Polly for .Net are available. You can even use the ser­vice dis­cov­ery layer to help prop­a­gate the mes­sage to all your ser­vices that a cir­cuit has been tripped (i.e. a ser­vice is struggling).

Debug­ging Hell – Debug­ging is always a huge issue in micro-service archi­tec­tures. E.g., a nested ser­vice call fails and you can’t cor­re­late it back to a request com­ing from the user. One solu­tion is to use cor­re­la­tion IDs. When a request comes in, you assign the request with a cor­re­la­tion ID and pass it on to other ser­vices in the HTTP request header. Every ser­vice would do the same and pass the cor­re­la­tion ID it receives in the incom­ing HTTP header in any out-going requests. When­ever you log a mes­sage, be sure to include this cor­re­la­tion ID in the log line.

Miss­ing Mock Servers – When you have a ser­vice that other teams depend on, each of these teams would have to mock and stub your ser­vice in order to test their own services. A good step in the right direc­tion is for you to own a mock ser­vice and pro­vide it to con­sumers. You can take it a step fur­ther, by build­ing the mock ser­vice into the client so you don’t even have to main­tain a mock ser­vice any­more. Use Any­point Plat­form or Swagger.

Fly­ing Blind – There are more than a hand­ful of tools avail­able in this space. There are com­mer­cial tools (some with free tiers) such as NewRelic and Stack­Driver (now inte­grated into Google AppEn­gin), AWS also offers Cloud­Watch as part of its ecosys­tem. In the open source space, Net­flix has been lead­ing the way with Hys­trix and some­thing even more excit­ing.