So, You’re building a cloud app on Windows Azure are you? Where are you going to store your data? This is a question that comes down to architecture, scale and options. Considering I don’t know your architectural decisions, or how scalable your particular application, I guess that leaves me with only one topic to talk about, What are your options for storing data on Windows Azure?

Windows Azure is an open cloud platform which enables you to quickly build and deploy your applications using your existing skillset. This is reflected in the data options which are available for use on Windows Azure which are also hosted on Windows Azure.

MySQL Sizing Options

MySQL on a Linux Virtual Machine

In addition to the ability to leverage a third-party service for using MySQL on Windows Azure, you can also leverage the new Virtual Machines feature. Follow the steps for setting up MySQL on an OpenSUSE Virtual Machine on Windows Azure.

SQL Database

Microsoft has created a cloud equivalent of SQL Server known as Windows Azure SQL Database (or SQL Database for short). SQL Database uses the same TDS Protocol as SQL Server which means all standard libraries or drivers [like the SQL Server Driver for PHP] work against SQL Database for Connectivity and Querying.

SQL Database Sizing Options

As you can see above, SQL Database offers sizing tiers which acts as a starting point for your database size, additional database sizing between the tiers is offered at a discounted database rate as described in the SQL Database section of the Windows Azure Pricing page.

You may have also noticed that the maximum database size is 150GB, once you reach a database of this size it becomes necessary to shard your data using a technical known as SQL Federation.

Unstructured Data (NoSQL) on Windows Azure

Unstructured Storage is becoming all the rage, which is understandable due to its ability to massively scale.

Windows Azure Table Storage

Windows Azure Table Storage is one of the three abstractions of Windows Azure Storage. Table Storage is an Entity-Attribute-Value model data store with a 100TB per account limit. Each subscription has 5 storage accounts by default, and more storage accounts can be added by calling customer support.

An entity in Windows Azure Table Storage is comprised of 3 required properties partition key, row key & timestamp and up to 252 additional properties. Properties can be one of the following data types Int32, Int64, String, Guid, DateTime, Double, Boolean or Byte Array.

MongoDB on Windows Azure

There are a few different ways that MongoDB is supported on Windows Azure which include in a Cloud Service, on a Virtual Machine or as a Service.

MongoDB in a Cloud Service

The creators of MongoDB, 10gen, have created a MongoDB installer which will install a MongoDB replica set in a Windows Azure Cloud Service. To do this they leverage a worker role in which their installer leverages a startup script to install the MongoDB Server. It also utilizes Local Storage as a cache before committing new transactions to Blob Storage.

To use the MongoDB in a Cloud Service configure and Deploy the MongoDB Installer.

MongoDB as a Service

Another option for MongoDB on Windows Azure is MongoDB as a Service provided by MongoLab. This service is [at time of writing] in preview and provides 500MB of storage for FREE. The MongoLab Preview is currently only available in the East US datacenter.

Preview Warning

We are pleased to be able to offer free databases running on Windows Azure. Be aware that both Windows Azure and MongoLab’s presence on Windows Azure are still in preview mode, and that these databases are not appropriate for production use. For example, we may need to take them down for maintenance from time to time without advance warning.

CouchDB on Windows Azure

CouchDB comes with a suite of features, such as on-the-fly document transformation and real-time change notifications, that makes web app development a breeze. It even comes with an easy to use web administration console. You guessed it, served up directly out of CouchDB! We care a lot about distributed scaling. CouchDB is highly available and partition tolerant, but is also eventually consistent. And we care a lot about your data. CouchDB has a fault-tolerant storage engine that puts the safety of your data first.

BigCouch by Cloudant

Cloudant has expanded the base implementation of CouchDB in a new incarnation they are calling BigCouch. BigCouch as a Service is offered in the West US datacenter in a cluster Cloudant refers to Lagoon.

In my last post I Announced the Windows Azure Powershell Extensions a new project that I will be iterating over when I run into tasks that are common in everyday Windows Azure Scenarios.

Being a Developer on the run, going from City to City or Coffee shop to Coffee shop, the absolute first Extension came to me naturally. Quickly create firewall rules for where ever you are by using Add-RoamingFirewallRule, when you’re ready to go remove the setting with Remove-RoamingfirewallRule.

Adding Functions to your Powershell Profile

Functions are only helpful in powershell if their accessible, if it takes you too long to set up a script there’s a good chance you will never use it. Let’s take a brief look at making sure your Powershell environment is up and running.

Open Powershell.

Create a Powershell profile

new-item -path $profile -itemtype file -force
Notepad $profile

Now that your profile has been created, and opened in notepad it’s time to copy the Add-RoamingFirewallRule and Remove-RoamingFirewallRule functions from the Windows Azure Powershell Extensions Project on GitHub and paste them into the open notepad document. After you’ve finished save and close the file. Restart Powershell.

Using Add-RoamingFirewallRule & Remove-RoamingFirewallRule

Let’s first take a look at the help files that are provided with the script.

Get-Help Add-RoamingFirewallRule

Get-Help Remove-RoamingFirewallRule

You’ll notice that both Add-RoamingFirewallRule and Remove-RoamingFirewallRule both require an EnvironmentsCsv parameter which is a path to a CSV file containing Subscription and Management Certification information.

Creating the Subscription CSV File

The CSV file ensures that you don’t need to login to the Windows Azure Portal anytime you’re looking for your SubscriptionIds [Just keep the CSV file up-to-date]. The file requires the header in order to map to the internal variables in the function. The first value is the SubscriptionId for your Windows Azure Account, the second value is the Thumbprint for a Certificate [installed in CurrentUser\My] which has been uploaded as a Management Certificate using the Windows Azure Portal. Save this CSV file in a memorable location.

That’s a Wrap

I hope that this helps speed up your day when working with SQL Azure at different locations or encourages you to get out of the office from time to time. If you have any requests or notice a bug please file an issue on github.

Recently, I was working on a project that required a site migration from a Shared Hosting server to Windows Azure. This application has been up and running for sometime and had acquired quite a substantially sized database.

Using Import/Export in SSMS to Migrate to SQL Azure

In addition to the SQL Azure Data Sync Tool, it is possible to use the existing Import/Export Wizard in SQL Server Management Studio to migrate data to SQL Azure. There are a number of things to keep in mind while using the Import/Export Tool:

SQL Server Native Client to .NET Data Provider for SqlServer

SQL Azure doesn’t fall under the typical SQL Server Native Client 10.0 Product SKU, this means that you’ll have to use the .NET Data Provider to migrate your data. The configuration screen for the provider is very intuitive, but there are two key settings that should be changed from their default values, Asynchronous Processing (set to true) and Connection Timeout (increase to 1500).

Without changing the Timeout value the data migration would error out after creating the fist few sets of rows. Making this an Asynchronous process was beneficial when exporting multiple tables at a time.

As Wayne explains in his post, there are a number of XML files which contain data mapping information used by the Import/Export Wizard in order to map the data from the source database to the proper data type in the destination database.

Database Seeded Identity Insert Issue

I’m not sure why this happened, but when using the Import/Export even with Identity Insert on, the ID [Identity] Column was not Inserting the correct values. To get around this I used the ROW_NUMBER to generate new Identities and rebuilt the foreign key tables.

There is a lot of chatter on the Forums and other blog posts that say that BCP with the –E switch is the most effective way to do exact copying (with Identity Columns).

Cost Effective Approach

A good thing to keep in mind while preparing your database for migration is that transactions as well as data transfer costs are applied to Queries to (and from) SQL Azure. With this in mind it would be best to set up a scenario where you would test your data migration to ensure the data migration would be performed in the least number of attempts as possible.

However, nothing feels more comfortable to a developer than something that familiar. After a little bit of investigating while preparing for my talk at Confoo on SQL Azure, I managed to find a post on the MSDN website that Explains what is needed in order to use the Generate Scripts Wizard in SQL Server Management Studio.

Create the Transact-SQL Script

In ObjectExplorer, right-click the database, point to Tasks, and select GenerateScripts.

In the ScriptWizard dialog box, click Next to get to the SelectDatabase step. Select School, select Script all objects in the selected database, and then click Next.

In ChooseScriptOptions, set the following options:

Convert UDDTs to Base Types = True

Script Extended Properties = False

Script Logins = False

Script USE DATABASE = False

Script Data = True

SQL Azure does not support user-defined data types, extended properties, Windows authentication, or the USE statement.

Click Next, click Next, and then click Finish. The Script Wizard generates the script. Click Close when the script is completed.

Each CREATE TABLE statement includes the "ON [PRIMARY]" clause. Delete all instances of that clause.

The reason you need to apply these changes to the script is that SQL Azure currently doesn’t support all the features of the currently release of SQL Server 2008. There are plans to start to incorporate some of the features that are in this outline, including the USE statement.

Hopefully this will make your life easier on your move to the cloud. Until then, Happy Coding!

I received an email a few days ago from Apress Books suggesting me the book to the left due to my previous purchase of Introducing Windows Azure. Both books cover the Windows Azure Storage Services API, and SQL Azure. The Introducing Windows Azure book covers Azure .NET Services, which was renamed to the AppFabric Services which is covered in Windows Azure Platform. This name change was announced during the first keynote [View Keynote] of PDC 2009. Along with Codename “Dallas” [Data as a Service], Microsoft Pinpoint Marketplace.

Codename Dallas is built on top of Windows Azure and SQL Azure. Dallas will supply developers with Data as a Service, as part of the Open Data Movement in the United States. If this doesn’t get you excited about Microsoft’s Cloud Computing Platform, I’m not sure what will. To ensure that you can develop on the cloud and use the SQL Azure Database Platform to your advantage read forward as I’ll be showing you how to Migrate a database to the cloud with SQL Azure Data Sync, which is an extension of the Microsoft Sync Framework.

Once you have your SQL Azure Account, you will need to provision a new database in the cloud and change the firewall settings so the local computer can connect to and modify the database. These steps are provided in Making Data Rain Down from the Cloud with SQL Azure. Now that we’ve got our SQL Azure firewall database set up we’re ready to sync a local database into the cloud.

When you first fire up SQL Azure Data Sync you will see this Introduction screen. [Hopefully future releases of this tool will allow you to check a box to skip this screen, like other SQL Server Tools.]

Hitting next on the Introduction screen will take you to the SQL Azure configuration screen where you will need to enter the details for your SQL Azure Account. For this step you will need to login to your SQL Azure account to figure out what server your database server is hosted on. Fill out the database information required, like you see below.

A good thing to note here is that this database should not be created in the cloud already, the data sync tool will be creating the database from the local instance you will be setting the details up for in the next step. So lets click next and enter the information for our local Database.

Once you have both databases configured [and tested] you’re ready to select the tables you wish to synchronize or create in the cloud database.

Due to the fact Relational Databases have Foreign Keys to link data from one table to the next you may need to synchronize the database in a particular order to maintain data integrity. If you have data in your tables and your tables are related to one another be sure to place the Primary Key table, before the Foreign Key table. Here is the screen that will allow you to order your tables in order they should be synchronized.

After you have reorganized the order in which your tables are to be synchronized, you will be shuffled off to a Summary page. This page outlines the Database Server, and asks if it is to create a 1GB Database, or a 10GB Database in the cloud.

After Reviewing the information and selecting the database size that should be created. Click on the Process button to Copy the local Microsoft SQL Server database to SQL Azure.

Once this process is complete the Microsoft SQL Server has been created on the specified SQL Azure Database server. The Synchronization is carried out by SQL Server Agent, be sure to copy down the Job name from the Finish Setup screen.

SQL Azure Data Sync is a great way to deploy your current databases into the cloud. You can use the cloud as a method of backing up your current infrastructure or you can start expanding into the cloud as a more economical solution of expanding your current data centers. As we count down to the New Year it seems fitting to think about moving into the future of Technology. Happy New Years!

Then you will Launch the Development Storage Initialization Tool [DSInit.exe]. You will need to set the SqlInstance [/sqlinstance:] that you wish to create the tables on, and you will need to force [/forceCreate] the tool to create the tables.

DSInit.exe /sqlinstance:<YourDatabaseName>/forceCreate

This will launch the Initialization Tool.

If you receive the error message below you it is most likely because the instance name was improperly set. Ensure to be use only the InstanceName do not include periods or slashes.

After your Development Storage is set up make sure you go into your Development Fabric and Start the Development Storage Service.