In brief, I will restructure and reconfigure my previous application I had created in my post mentioned in #2 above (Simple CRUD operation using MEAN Stack), then deploy the application on Microsoft Azure with continuous deployment configured with Github and finally I am going to connect all the different layers using the concept I have covered in NodeJS connectivity to MongoDB on Cloud. To start with this example I would recommend you to go through my previous posts mentioned above, this will give you a little background on what I am going to explain in this post. Step 1: Restructure the application, in my previous example “Simple CRUD operation using MEAN stack” I have provided an application build on MEAN stack. I am going to take the sample application and split this into 2 parts one is for Client having Angular JS as client side technology and another into Server build on NodeJS + Express technology. And finally this is going to connect to MongoDB which I have used mongolab as DBaaS (Database as a Service). So the overall architecture of the application will be as follows So my application structure will be as follows.

Step 2: Configure the Client and Server applications on Github, I have provided more details on application configuration on Github @ Continuous deployment of Node, Express application on Azure using Github. I am following similar steps here except the fact that instead of one consolidated application I am deploying two application i.e one for server and another for client. Step 3: Configure the Client application on Microsoft Azure having Github as source control with continuous deployment Step 4: Configure the Server application on Microsoft Azure having source control on Github with continuous deployment Step 5: Create a new app settings key in Microsoft Azure console, and follow the steps WEB APPS—>Your Application—>Configure—>Go to App Settings—>Add a new Key and Value as mentioned below. Step6: Configure your nodejs application with this key. Using this key my nodejs server application will pull the DB connection string from the environment variable configured on Azure.

Step7: Once your application is deployed on Azure, you can test your API by hitting the Url of NodeJS Express API, in my case it is : addressbookserver.azurewebsites.net/persons This gave me the JSON Response from the mongolab data store. In case if you don’t have any sample data on mongolab, you can login into your mongolab console and add few data with the schema similar to the one I have provided in the screen above. This is required only for testing purpose. From next time onwards my client application is going to GET/PUT/POST and DELETE these data. Step 8: So up till here DB Server, Express API and Angular Client everything deployed on Azure, so ideally when I will hit the client URL it should call the Express API and using the mondodb driver the Express server should fetch the data from MongoDB hosted on mongolab server. Let’s go ahead and test my application end to end, well even though I have data in MongoDB but I am not getting any output on my client, instead if you open the console you can see I am getting error from my Express Server.

This is due to the security feature which is provided by default by JavaScript, known as CORS (Cross-Origin Resource Sharing). This prevents JavaScript from making requests across domain boundaries, and has spawned various hacks for making cross-domain requests. CORS introduces a standard mechanism that can be used by all browsers for implementing cross-domain requests. The spec defines a set of headers that allow the browser and server to communicate about which requests are (and are not) allowed. Since in this example I have broken down my application into client and server and hosted both of them on 2 different domains, which in turn triggered this extra layer of validation. CORS is a very vast topic in itself, so instead of diverting from actual topic I would recommend you to go to the dedicated site for CORS and read about it, I believe this might give you a very good understanding of the internals of CORS. So coming back to the original article my next step will be to configure my application for CORS. Step 9: CORS (Cross-Origin Resource Sharing)

In the code above I am adding my client URL to Access-Control-Allow-Origin fo the server.js file in Express API. With this all the response server by the Express API will validate the requestor before it sends any further response back to the client. In this way the default security feature provided by Javascript will make sure that no one else other than my client domain: http://addressbookclient.azurewebsites.net will be able to access my API. You can refer the expressjs implementation of CORS @ http://enable-cors.org/server_expressjs.html Step 10: Now that we have everything in place let try refreshing the client URL: http://addressbookclient.azurewebsites.net So now I am able to get the data from mongodb through Express API running on Nodejs. And if I see the response, I can see the Access-Control-Allow-Origin is set for my client domain, which means if any other URL will try to access this they will get a similar error which I received in Step 8 above. But if you don’t want any constraints or restrictions then you can change this to Access-Control-Allow-Origin: ‘*’ here I have used wildcard asterisk to allow any domain to access my API. And this is pretty much I had to share on this topic, but to get complete picture on all the associated topics, don’t forget to visit my previous posts:

Monday, October 19, 2015

In this post I am going to cover different options where you can host your MongoDB database using different options such as DBaaS (Database as a Service), Paas (Platform as a Service) and IaaS (Infrastructure as a Service) using Microsoft Azure.

Option 1 : Using Infrastructure as a Service (IaaS)

In the first option I am using Microsoft Azure dedicated VM to configure MongoDB. This means that you can have your database in your favorite cloud in the same location as your application tier.

Step 1: Configure you mongo DB database on Windows Server 2008 on the VM hosted on cloud (Microsoft Azure). I found an excellent article which demonstrate how to configure your mongodb on Windows Server 2008 hosted on Azure. Instead of reiterating the same thing here does not make any sense, so I am providing you the link to the article. You can use this article to configure your mongodb database server.

This will take you to the mongodb console, you can use mongodb commands/queries to access the documents and collections.

Same will be reflected in the web console, or you have option where you can directly edit your databases, collections, users and documents.

Connecting to mongolab using nodejs

Step 1: install mongodb drivers on node using the command

$ npm install mongodb

Step 2: Create a server.js file and add the following code.

Step 3: Run the application to test connectivity to mongolab.

Now you are all set to start your development using MongoDB hosted on cloud platform. Since I am using these technologies for my learning purpose so personally I felt using the mongolab as most convenient option, as this will not have lots of configuration before you actually start your development and you don’t have to run those extra commands to make sure your server is running on the console.

Even though mongolab and Windows Azure VM is hosting mongodb in both the option, but the option have entirely different infrastructure and concepts. For learning purpose you may choose any of this option, but if you are seriously thinking to host your mongodb for your enterprise applications then you might have to do a detailed study on which one suits you better in terms of IaaS or PaaS.

Saturday, October 17, 2015

In this post I am going to cover a very basic CRUD operations using MEAN (MongoDB, Express, Angular and NodeJS) stack. You can use this example to kick start your project using MEAN stack.

Before I begin, I am assuming that the reader of this post has basic understanding of AngularJS, MongoDB, NodeJS, jQuery and other client side technologies like jQuery, HTML 5, CSS 3, etc.

Most of the commands, environment I have used is for Windows operating system, but there’s not much difference if you are using MacOS or Linux operating system, and moreover all the technologies used in this example are platform independent.

1. Setup the environment

Follow the URL in their respective sites to get detailed information on download and installation instructions. You can get the installers of all the OS which is supported by these framework and their installation instructions in these links.

2. Setup Solution

Solution Structure: In this example I am using a person database and the only function of this application would be to Create-Read-Update-Delete a person record. To achieve this I have created a simple solution structure, using the following structure.

Here I am using app folder for most of my application code, this folder contains angular controller, service and views. I am keeping my node modules very simple just to expose API’s which will be consumed by AngularJS services. I have covered most of the server side node modules in just one file called server.js I will try to cover specially more complex node module/architecture in my next post.

3. Downloading, Installation and configuration of Dependencies

Download and installation instructions: For this example my application is using following node-modules, to install these modules I am using node package manager using GitBash.

express: Express is a minimal and flexible Node.js web application framework that provides a robust set of features for web and mobile applications (Reference: http://expressjs.com/). I am using express in my middleware to create node.js server and API’s

4. Setup Middleware

I am using my node.js for middleware, to setup my middleware first I have created my server in node.js

Here in line 1 and 2 above I have imported express module and created a instance of express, and in line 3 I have used listen function of express to create new server on port 3000. With this example my server is listening on port 3000. When I run this in GitBash I will get the following message.

I still need to add few more modules to my server.js which will help the application to connect to my mongodb server, configure the application working directories. But before I do that let me first start creating GET/POST/PUT and DELETE API’s in server.js which can communicate with my Client module in AngularJS

4. Create GET/POST/PUT and DELETE server API’s

In the above code I have create a blueprint of my middleware in server.js

Node JS API Controller

Description

/persons

Get All Persons

/person/:id

Get single person by person Id

/addPerson

Add new person

/deletePerson/:id

Delete single person by id

/updatePerson

Update single person

in the first few lines of my code I have created the instances of express and configure my application to use my root directory of the application as a working folder. Now lets run the server and check the output in web browser.

In this example I have created stubs for Get by Id, GetAll, Update, and Delete person request. To test the code start the node server, though initially you can see the same output what I have provided earlier. But when you enter the URL’s in the browser with the URL’s on which I have created my API’s you can see the outputs in the console.

I have provided example of GET requests. I will write Angular modules to test POST/PUT and DELETE API’s

GET All (localhost/persons)

GET By ID (localhost/person/1)

5. Configure Angular JS Client

I am using simple MVC model in AngularJS Client, but you can fork or clone my Git repository to extend this solution. This example assume that you have basic understanding of Angular JS. To start with lets create a simple router which redirects my application to the home page of my person page.

Angular JS Routing

Here in the example above, I have mapped the paths /person and /person/:PersonId with the PersonCtrl and PersonAddressCtrl controller respectively and corresponding view as app/views/person.html and app/views/persondetail.html views respectively. Which means when I hit the URL:http://localhost:3000/person, this will go to the person.html and mapped to the PersonCtrl.js controller and when I hit a single person using URL:http://localhost:3000/person/1 out of the list using personId my routing engine will redirect the users to persondetail.html using PersonAddressCtrl.js

Angular Controllers – PersonCtrl

onError : This is going to be my common error module, this action method is used only for the purpose of displaying the error when something goes wrong during the whole communicate process wither at the server level or at the client level.

refresh : This will be my default get all person action method, which is used to load all the person data from mongodb database via nodejs. Here in this code I am calling the corresponding API ‘/persons’ which is created in nodejs and express to populate all the persons from mongodb. Once the data is returned from mongodb I have created a callback function onPersonGetCompleted event, which is used to populate the person data in the scope of the current module. refresh() is called as soon as person.html my default page is loaded in the application or every time we need to refresh my data from mongodb.

searchPerson : This action method will be called when I want to view a single person based on person id, for example during opening the update form for the selected person. This method is associated with the callback function onGetByIdCompleted, which populates the scope with a single person result from mongodb.

addPerson : As name suggest, this action method is used to add a new person, and the associated call back function to this is onAddPersonCompleted, which refresh the data once add is completed to reflect the uptodate information on the Person default page.

deletePerson : As the name suggest, only purpose of this action method is used to delete the selected person. This calls '/deletePerson/:id' API of nodejs and once the call returns it uses its callback function onPersonDeleteCompleted to refresh the update data in the HTML page person.html

updatePerson : This action updates the person information in mongodb. This calls the ‘/updatePerson’ API in nodejs. Once the update completed, it calls the callback function onUpdatePersonCompleted action in Angular to refresh the updated data in the Person page.

Let's put everything together to see my completed PersonCtrl.js, next I am going to show how this is going to integrate with the View and the final step will be to write the body of my API controller which will be used to transact with mongodb.

5. Completing the NodeJS express API Controller

Now lets revisit the API controller, the skleton of which I had created in the Step 4 above. Here I am going to write the body of my GET/POST/PUT and DELETE methods which will be used to transact with mongodb document database. In the chart below I have shown the NodeJS express API and corrosponding mongodb API against that which will be called to complete the transactions.

Now lets look into the completed API of my nodejs, Here in line #4, AddressBook is mu collection name which is equivalent to the tables in RDBMS and Person is the document. If you are still confused with the document and collection and other jargons of MongoDB then probably you can look into the link: http://docs.mongodb.org/manual/reference/glossary/, this covers all the mogodb glossary.

I have also introduced body-parser here, which is used to parse the request and response in json format.

This is the least code you can write to implement the CRUD operation using mongodb and node, but the possibilities are limitless you you want to extend this example. Some of the things which you can do with very little effort if you want to make this solution more structure and object oriented are:

2. Use ORM/ODM tools like mongoose. Mongoose provides a straight-forward, schema-based solution to model your application data. It includes built-in type casting, validation, query building, business logic hooks and more, out of the box.

This is just few of options which can be done with very little effort but the possibilities are unlimited. You can fork or clone my example to extend it up to your capacity for learning purpose.

Now lets get back to the pending items which will wind up my post by creating Views

6. Angular JS and HTML Views

I am going to create a Dashborad which will display all the persons I have in my mongodb database in AddressBook collection. I have provided below the screenshot to show you how the completed code will look like. Here I have used AngularJS bindings for model binding and CSS3 and HTML5 to make my page look little descent.On click of Edit I am going to edit the person using a Model popup with pre-populated person data and similarly I am going to have a model popup for Add Person with empty controls which will allow the user to add a new person record and delete is simply going to remove the record from the mongodb. My example is designed to take you to the persondetail.html page which will have more details of the person, like person address but for the simplicity sake I have not implemented that in this example.

Now to achieve this I will need only few pages, which are listed below

index.html : This page is going to have only the references to the controller, angularjs, bootstrap and other client side libraries. This page also have a

<div data-ng-view="data-ng-view"></div>

This acts as a place holder for displaying the views injected by angularjs routing engine.

person.html

This is the home page or default page in my application. Screenshots of this page is provided above. The purpose of this page is to list all the persons in the mongodb database, add, edit and delete the persons from mongodb.

I have used a very loosely defined model called person for my model and this is going to be saved in mongodb directly without any further transformation. My view is binded with persons model in the $scope through the PersonCtrl. Once the model is populated I have used ng-repeat to display all the persons in the table. In line #17, I have called the searchPerson with personId in PersonCtrl to get selected person to edit from mongodb. Here data-toggle and data-target is used to tell the Edit button to display the #personEditModal is clicked. And for delete in line #18 I have called deletePerson action in PersonCtrl, this in turn calls the NodeJs API for delete and remove the person from mongodb database.

And for Add and Edit modal popup, I have a very basic design, refer the screenshot above. I have provided below the example of edit person, only difference of edit and add person is the display captions/headings and on click of save button, Edit calls the updatePerson(person) action in PersonCtrl and in add person modal this calls the addPerson(person) action in PersonCtrl.

So with these code I have completed end to end data bindings, client side API using angular, server side API using nodejs and express and you might have noticed I have not done much like schema creation in mongodb data as mongodb is schema less document oriented database. So the same JSON which I have used to communicate between the different layers are stored directly into the mongodb database. But again this is just the start, the things I have not covered in mongodb is relation data example like person and person address documents and how they can be associated with each other using references, complex model bindings, etc. You can Clone, Form or download this example from my git repository @ https://github.com/bmdayal/MEANSample

Sunday, September 6, 2015

Normally we deploy the projects on Azure using Visual Studio IDE, but Azure is not just limited to all Microsoft technologies specially when the majority of developers are on open source or moving to open source.

In this post I am going to provide you a very step by step process to deploy your Nodejs project having a source control on Github to Microsoft Azure cloud.

The basic things which you need to setup before you can start your deployment are as follows:

Github repository account: First you need to setup a new account (if you don’t already have) in https://github.com/

Github for Desktop: I am using Github for windows to Commit and Push my changes to Github repository, but you can use other tools as well to push your files to Github. You can download from here: https://desktop.github.com/.

Microsoft Azure: If you don’t have Azure subscription you can signup for 1 month free trial. This will give you $200 credit to spend on all the Azure Services: You can visit this link to Signup for a month free service: https://azure.microsoft.com/en-us/pricing/free-trial/

You don’t need to have very advanced knowledge of Node, Github or Microsoft Azure to follow the steps I have covered here but I am assuming that you have at least basic knowledge of Nodejs, Github and Azure. Now Once you have all the required setup installed on your machine, lets get started with the basic steps which is required to deploy your Nodejs sample application on Azure using Github.

Step 1 Setup your environment: Create a working folder which you want to use for your application, and start Git Bash on your working directory. Alternately you can use windows command prompt. For this example I have created a folder in my local C drive as C:/Source/NodeOnAzureSample

If you already have your application which you want to deploy on Azure you can use that. For this example I am using express-generator to quickly create a sample application using Node, Express and Jade, I am using this just to save time and keep our focus on the actual topic.

Step 2 (Optional) Setup my demo project: This is a optional step if you already have your application ready. Here I am using the command “npm install express-generator -g” to generate my sample application template with a basic welcome page. You can find more detail of express-generator @ http://expressjs.com/starter/generator.html

This will create the sample files inside node modules globally in your C:/Users/YourUserName/AppData/Roaming/npm/express folder, or you can create the files in your local application folder by removing the –g from the command.

Now run “express NodeOnAzureSample” command, this is going to create all the required folders and files which you are going to start your development.

Step 3 (Optional) Restore the dependencies: Run “npm install”, this will restore all the dependencies like jade, express, etc which your application has dependencies.

Step 4 Test the application on localhost: After executing Step 1 to 3, we have our application ready to run on local machine, lets go ahead and test our sample application by switching to the application directory and run the command “npm start” in Git Bash (you can use powershell or windows command prompt) and enter “localhost:3000” in your browser. This should give you the following screen and your sample application is ready for hosting. Now in Next Steps I am going to configure my application with Github repository.

For simplicity sake and stick to the topic, I am not going to explain the folder structure, code, and other granular details of this sample application.

Step 5:Setup Github Repository @ Guthub.com Go to https://github.com/new and create a new repository, once you select all your desired options as shown in the image below. Click on the button “Create Repository”

You will be provided step by step information on the new repository which you have created,

Copy the URL, in your clipboard or notepad, you are going to need this URL during commit and push of your local application files to github repository at github.com online.

Step 6: Setup Github Repository on you Desktop: Open Git Gui and Select Create new Repository from the menu which will ask you to locate your application directory, select the folder which you want to deploy to Azure. In my case I have mapped my “NodeOnAzureSample” application to Git repository.

Once you select create, Git GUI will list all the files which are either new or modified. Next step is you need to Stage Changed files, Commit and Sign Off, to perform these first you need to setup a new repository in https://github.com which I have already explained in Step 5 above.

Selecting Push is going to final commit your files to online github repository, to do this you will need the URL of your GitHub repository at github.com, and this will also prompt you to authenticate yourself.

Once your files successfully pushed to online repository, you will get a confirmation message as below.

Now that your files are ready for deployment, lets switch to Azure to configure continuous deployment.

Step 7 Configure Azure website: to do this you need to go to https://manage.windowsazure.com dashboard. Select Web Apps—>And select New from the footer menu.

This is going to provide you a wizard to create a new WEB APPS. For this example I am going to use Quick Create option to create my web application and named this as NodeOnAzureSample. If the name is available you will get a green check mark next to the name. This will provide you with a URL as https://nodeonazuresample.azurewebsites.net. Once you are happy with the name click on “Create Web App” button.

This is going to provide you a welcome page of Azure Web site. In my nest steps I am going to replace the welcome page with actual application which we have created and pushed in Github.

Step 8 Configure deployment using Github: Select the Web App listed in the image above and Select Dashboard from the top menu.

From the dashboard, select Set up deployment from source control from right navigation menu.

You will be provided with a list of options, where you need to select GitHub, and select Next

You will need to authenticate yourself to GitHub where you have created your repository that needs to be deployed on Azure.

On successful authorization, you will be provided with the list of your applications which you have deployed on Github, for this example I have selected my “NodeOnAzureSample” application.

Select Done when you have made your selection, and this is pretty much what we need to do to link my Github repository to Azure website.

Deployment Stage 1

Deployment Stage 2: Notice you got a new menu called “Deployments” after Dashboard. Be patience, this might take few seconds to minutes depending upon the size of your application.

Deployment Stage 3: Deployment completed, you can see the initial comments which I had provided during my Commit to Github

Step 9 Testing your deployment on Azure website: Now that your application is deployed on Azure the next step is to test my application, yes no further configuration is required. Azure configures everything for you. Lets go to the WEB APP Dashboard and hit on the URL : nodeonazuresample.azurewebsites.net

Save the file, go to your Github Desktop—> Rescan—>Commit and Push the file with comment “Changed the welcome text from Express to Node, Express, Azure and Github”

Once you complete these steps, go back to your Azure dashboard. And probably by the time you switch back to your Azure your changes might have already been deployed to your website. Lets go ahead and verify.

And here we go, on my azure dashboard above I can see the latest changes are already deployed with the comment I have provided in my github commit.

Lets test this change in my browser by refreshing the page, and yes my changes are reflected below.

This is just a simple example of how the things are simplified in Microsoft Azure. This post will give a head start to integrate Node, Express, Github and Azure.

You can use any of your favorite open source like Angular, Knockout, Backbone, Jade, etc with your favorite source control like Visual Studio, GitHuib, Dropbox, Codeplex, etc and integrate with Microsoft Azure and above all you don’t need to be a Ninja on this technologies.