Search

Wednesday, November 05, 2008

When installing and using production web applications, backup is always a must. I have written about SQL Server Backup before and this post would expend on that. The main goal here is to create a solution that periodically backs up your database. To do this, we’ll use the backup script from the previous post, employ NAnt to execute the script and zip the created backup. Last, we’ll schedule the execution of the NAnt build script through the Windows Task Scheduler or the “at” command. This procedure can backup a local or a remote SQL Server (as long as the remote server is on the same network).

NAnt Backup Script

The script is somewhat complex. Here are the list of features:

Backup a local SQL Server instance or a SQL Server instance on the same network. The key is that the file system of the networked instance must be available through UNC shares to the machine executing the backup script.

Use a specified connection string or read the connection string from a .NET (or other type of XML)configuration file.

I will not dive into the script itself since it's properties and flow is documented fairly well.

Code

<project name="backupDatabase" default="run" xmlns="http://nant.sf.net/release/0.86-beta1/nant.xsd">
<!-- Set the name of the backup file that will be created -->
<property name="zipFileName" value="databaseBackup" overwrite="false" />
<!-- Set the path where the backup file will be finally stored -->
<property name="localBackupDirectory" value="C:\Temp" overwrite="false" />
<!-- Set the local path (relative to the SQL Server instance) where SQL Server will write the backup file -->
<!-- This has to always be a local path since SQL server can't write to network paths -->
<property name="sqlServerLocalBackupDirectory" value="C:\Temp" overwrite="false" />
<!-- Set the UNC path to the above "sqlServerLocalBackupDirectory" local path -->
<!-- This is only needed if you are backing up SQL server on the network -->
<property name="sqlServerUNCBackupDirectory" value="\\beehive\windows$\Temp" overwrite="false" />
<!-- For backing up a local SQL Server instance, commented the first "sqlServerUNCBackupDirectory" property -->
<!-- and uncommented the one below this line, or set the "sqlServerUNCBackupDirectory" property to "" -->
<!--<property name="sqlServerUNCBackupDirectory" value="" overwrite="false" />-->
<!-- Set the .NET connection string for connecting to the database -->
<!-- This setting always takes precedence over the config file below -->
<!-- To use a config file instead, set this property to "" (like so value="") -->
<property name="connectionString" value="Data Source=beehive\sql2005;Initial Catalog=ensembleVideo;User ID=test;Password=test;" overwrite="false" />
<!-- Alternatively set the path to where the build script should get the connection string from -->
<!-- This is usually a app.config or connectionString.config file -->
<property name="connectionStringConfigFilePath" value="C:\Inetpub\wwwroot\myApp\config\connectionStrings.config" overwrite="false" />
<!-- Set the XPath expression that will be used to grab the connection string from the config file -->
<property name="connectionStringXPath" value="/connectionStrings/add[@name = 'sqlServerConnection']/@connectionString" overwrite="false" />
<!-- Set the regular expression that's needed to get the database name from the connection string -->
<property name="getDatabaseNameFromConnecionStringRegEx" value="Initial Catalog=(?'databaseName'.*?);" overwrite="false" />
<property name="todaysDate" value="${string::substring(string::replace(datetime::to-string(datetime::now()), '/', '.'), 0, 10)}" />
<property name="todaysLocalBackupDirectory" value="${path::combine(localBackupDirectory, todaysDate)}" />
<target name="run">
<!-- If the SQL Server UNC directory was not specified and the local backup directory exists -->
<if test="${string::get-length(sqlServerUNCBackupDirectory) == 0 and directory::exists(localBackupDirectory)}">
<!-- This is a back of a local SQL Server instance -->
<mkdir dir="${todaysLocalBackupDirectory}" />
<property name="todaysSqlServerLocalBackupDirectory" value="${todaysLocalBackupDirectory}" />
<property name="todaysSqlServerUNCBackupDirectory" value="${todaysLocalBackupDirectory}" />
<property name="localSqlServer" value="true" />
</if>
<if test="${string::get-length(sqlServerUNCBackupDirectory) > 0 and directory::exists(sqlServerUNCBackupDirectory) }" >
<property name="todaysSqlServerLocalBackupDirectory" value="${path::combine(sqlServerLocalBackupDirectory, todaysDate)}" />
<property name="todaysSqlServerUNCBackupDirectory" value="${path::combine(sqlServerUNCBackupDirectory, todaysDate)}" />
<!-- This is a back of a networked SQL Server instance -->
<mkdir dir="${todaysSqlServerUNCBackupDirectory}" />
<property name="localSqlServer" value="false" />
</if>
<!-- If the connection string is empty,
the connection string file exists and the XPath to find the connection string is not empty -->
<if test="${string::get-length(connectionString) == 0 and file::exists(connectionStringConfigFilePath)
and string::get-length(connectionStringXPath) > 0}" >
<!-- Get the connection string to the database from the connection string config file -->
<xmlpeek
file="${connectionStringConfigFilePath}"
xpath="${connectionStringXPath}"
property="connectionString">
</xmlpeek>
</if>
<!-- If the connectionString property is not empty and the regular expression to get the database name is not empty -->
<if test="${string::get-length(connectionString) > 0 and string::get-length(getDatabaseNameFromConnecionStringRegEx) > 0}">
<!-- Get the database name from the connection string -->
<regex pattern="${getDatabaseNameFromConnecionStringRegEx}" input="${connectionString}" />
<!-- Execute the stored procedure to bckup the database -->
<sql connstring="Provider=SQLOLEDB;${connectionString}" transaction="false" delimiter=";" delimstyle="Normal">
exec dbo.backupDatabase '${databaseName}', '${todaysSqlServerLocalBackupDirectory}';
</sql>
<!-- Zip up the created databse backup file -->
<zip zipfile="${path::combine(todaysSqlServerUNCBackupDirectory, zipFileName + '-' + todaysDate + '.zip')}" ziplevel="9">
<fileset basedir="${todaysSqlServerUNCBackupDirectory}">
<exclude name="**/*.zip" />
<include name="*.*" />
</fileset>
</zip>
<!-- Delete all other files in the today's backup directory except for the created zip files -->
<delete>
<fileset basedir="${todaysSqlServerUNCBackupDirectory}">
<exclude name="**/*.zip" />
<include name="*.*" />
</fileset>
</delete>
<!-- Move the contents of today's backup directory to the local backup directory -->
<move todir="${localBackupDirectory}">
<fileset basedir="${todaysSqlServerUNCBackupDirectory}">
<include name="*.*" />
</fileset>
</move>
<!-- Delete the "todays" directories -->
<if test="${localSqlServer}">
<delete dir="${todaysLocalBackupDirectory}" />
</if>
<if test="${not localSqlServer}">
<delete dir="${todaysSqlServerUNCBackupDirectory}" />
</if>
</if>
</target>
</project>

Setting Up Backup Script and Scheduling

Review the build script and set following properties to match your setup:

zipFileName – The name of the zip file that will be created for the database backup

localBackupDirectory – The local directory where the backup will be stored

sqlServerLocalBackupDirectory - The local path (relative to the SQL Server instance) where SQL Server will write the backup file. This has to always be a local path since SQL server can't write to network paths.

sqlServerUNCBackupDirectory – The UNC path to the above "sqlServerLocalBackupDirectory" local path. This is only needed if you are backing up SQL server on the network. For backing up a local SQL Server instance, set this "”.

connectionString - The .NET connection string for connecting to the database. This setting always takes precedence over the “connectionStringConfigFilePath“ setting. To use a configuration file instead, set this property to "".

connectionStringConfigFilePath - Alternatively set the path to where the build script should get the connection string from. This is usually a app.config or connectionString.config file.

connectionStringXPath - The XPath expression that will be used to grab the connection string from the configuration file.

getDatabaseNameFromConnecionStringRegEx - The regular expression that's needed to get the database name from the connection string.

The next step is to create the schedule with either the Task Scheduler or with “at” command.

To use the Task Scheduler:

Create a .bat file with the following: “driveLetter:\path\to\nant.exe /f:pathToNAntBackupScript.build” and of course replace that with the actual path to nant.exe and to the NAnt build script you got here.

Go to Control Panel / Scheduled Tasks / Add Scheduled Task

Browse for the .bat file you created in #1

Configure the schedule run as often as you would like

To use the “at” command:

Do the same as #1 above.

Open a command prompt and execute the “at” command:
”at 23:00 /every:M,T,W,Th,F pathToBatFileFromStep1.bat”

Bonus

You don’t have to hard code the values in the NAnt build script. You can pass them from the command line. In that manner you can reuse the same script for different database. You simply need to call the script with –D:propertyName=”value" like so:

Monday, November 03, 2008

In the latest version of Ensemble Video I had a need to migrate the old mechanism of showing content to new way that involved a single page and some URL rewriting (more on that another time). The basics of the old mechanism were that for each publishing point used by the application, a separate directory with specific “index.aspx” was created. That was always a major pain when updates had to be applied since I always had to preserve the old files but still update existing content. Long story short, I got away from that but I still needed to figure out what publishing points were in use so I can set the database flag for the new mechanism. So, the NAnt fanatic that I am, I build a NAnt script to do that. In this example, you will learn:

How to parse XML for a certain value

How to traverse directories with NAnt

How to execute an “if” statement in NAnt

How to get file and directory names using the NAnt built-in functions

How to execute SQL script with SQL task from NAntContrib

The prerequisites are that 1). You have NAnt installed and 2). You have NAntContrib installed.

So let’s go one by one from the list above and you will see the final script at the end.

This parses the XML file under webApplicationDirectory\config\connectionStrings.config and uses an XPath expression to get the property of the connectionString with the name ensembleVideoConnection. The value is stored in the NAnt property "connectionString"

<-- Get only the directory name -->
path::get-file-name(orgDirectoryName)
<-- Get the directory name from the path -->
path::get-directory-name(orgDirectoryName)
<-- Create a full path based on the directory name and a file name -->
path::combine(webSiteDirectoryName, 'index.aspx')

Downloads

Friday, October 31, 2008

A friend wanted to export his UML diagrams created with Visio to XMI files so then we could run some kind of ColdFusion tool to generate code based on the created diagrams. It appeared however that Visio did not come out of the box a way to do that. Some googling reveled that there is a add-on Microsoft offers but you have to get it, compile it (in Visual C++ non the less) and install it. What a major hassle. So being the problem solver I tend to be, I installed Visual C++ and went at it. About almost an hour later, the procedure below did the trick.

Something to Note: The supported UML diagrams are: Static Structure, Component and Deployment diagrams, and State chart (directly from Microsoft’s documentation about the add-on).

Wednesday, July 23, 2008

It’s a pretty simple thing to do but I had to spend a few minutes today to figure out how to accomplish it. So here is a quick build file to rename all .sql files in a specified directory (relative to the script run directory) to .txt. Funny enough, this scripts gets called by my automation program before it uploads any SQL scripts to my blog. There is nothing to it and there are comments so here goes:

That is all good and swell, but if you are like me, you might hate to write that long SQL statement every time, so I thought I would shorten it a bit. Here is what I came up with for querying the information schema for procedures and functions. The same can be easily applied to views and tables.

The author of the blog post had a solution but it was not one I liked. It seemed like a hack and I hate hacks. So, I took a few minutes and here is the result. The key is using the new SubSonic.Select query available in SubSonic v2.1. The code offers 3 different ways to accomplish the task.
C# Code:

Thursday, March 27, 2008

Windows Management Instrumentation, or WMI for short, is a very powerful and often under used technology. I am simply fascinated by how much you can accomplish using WMI. It is mostly intended for computer management tasks but it can be used for much more. Some examples include getting all kinds of computer information, managing system services and application. This article will cover a possible solution for listing all available printers on a particular computer and show you to get this information through ColdFusion. WMI can be used through VBScript, C++ and .NET. However since part of my current work is done in .NET, the solution relies on .NET and IIS. Further more, here are the requirements that the we will satisfy:

List all available printers including shared printers

Allow for filtering the list of printers by the printer name

Be language independent

Accomplishing the first two tasks is fairly easy with WMI. So lets get started.

List All Available Printers

This is pretty straight forward using C# provided that you reference the System.Management namespace and add the appropriate "using" directive:

What is left to do is just loop through the management objects found and display/aggregate the printer properties:

if (moc != null)
{
// Loop through the found printers and collect the information
foreach (ManagementObject mo in moc)
{
// Do something with the printer name
// mo.Properties["Name"].Value.ToString();
}
}

Filtering By Printer Name

There is two ways that this can be accomplished. Let's explore both.

Using WMI query clause.
WMI on Windows XP or newer supports the "like" operator so filtering the printer list by the printer name or a partial printer name is very trivial:

// Create an object query to get the printer list
oq = new ObjectQuery("select * from Win32_Printer where Name like \"%myPrinterName%\"");

Where "myPrinterName" is the full name or part of the name of the printer. Notice to use of "%" as those work as a wildcard.

Using simple string compare.
Since there are plenty of servers still using Windows 2000 the string compare method is more compatible:

if (moc != null)
{
// Loop through the found printers and collect the information
foreach (ManagementObject mo in moc)
{
if (mo.Properties["Name"].Value.ToString().ToLower().Contains("myPrinterName"))
{
// Do something with the printer name
// mo.Properties["Name"].Value.ToString();
}
}
}

Language Independence

A simple way to implement languages independence is through a web service. So we take the code above and wrap it in a web service. The two addition are the separation of the code in two functions (getPrinters() and getPrintersWithNameFilter()) and the addition of a small helper function called saveProperties() . Here is the final result:

This way we end up with a serializable C# class which can be populated at runtime and still be serialized as our example XML. I called mine systemData and used it above as:

// Create new instance of the systeData class
systemData systemDataInstance = new systemData();
// Create a new instance of the systemDataPrinter class
systemDataPrinter printer = null;
// Create an array of printer objects to store the printer data
systemDataInstance.printer = new systemDataPrinter[moc.Count];

where moc.Count is the number of WMI ManagementObject that was returned for the WMI query.

The last point worth mentioning is inside the saveProperties() function. The function takes an array list of properties, loops over them and for each one sets the according property in the systemDataPrinter class. It does so by using reflextion:

// Get the printer property corresponding to the current property name
pi = printer.GetType().GetProperty(prop.Name, BindingFlags.Public | BindingFlags.Instance | BindingFlags.IgnoreCase);
if (pi != null)
{
// Set the value of the current property through reflextion
pi.SetValue(printer, value, null);
}

Gotchas

To get network printers listed you have to have the printer installed on the machine under a local account and configure the web service to use the account when performing the query. You use the "impersonate" tag and specify the user and the password:

Downloads

Monday, March 24, 2008

Logging can be a very useful tool when developing any application. Furthermore, logging is a must when deploying applications to a production environment. Keeping track of errors and informational messages can help you greatly reduce the time it takes to debug and fix a problem. As in any language, there is more than one way to enable logging in .NET. This article will cover getting started with logging using the free log4net framework.

Downloads

Thursday, March 20, 2008

Turn off automatic reloading of your application

This will greatly reduce the time the application needs to process all request. This is the way I develop Model-Glue applications. In your Model-Glue configuration (the "modelGlueConfiguration" bean in ColdSpring.xml), look for the "reload" property and set it to false:

<property name="reload"> <value>false</value></property>

Keep in mind that changes made to anything else but your views will require an application reload. For my approach to that, read the next tip.

Use two browsers

The basics of this point has to do with using one browser for your development and one for reloading the application. I suggest this since if your application keep session state, reloading the application from the same browser will reset the session while reloading the application from a different browser will not. If you turn the Model-Glue automatic reload, your application will run faster but you will not be able to see any changes made in your controllers or model. Once you made a change you will need to manually reload the application by invoking the "reloadKey" with the "reloadPassword" as defined in the ColdSpring.xml file. By default the "reloadKey" is set to "init" and the "reloadPassword" is set to "true". So you would reload by invoking "index.cfm?init=true". To take this further I add the following code to my onRequestStart inside Application.cfc to reload the application if you simply go to "/yourApplication/?init":

Turn on debugging

Debugging is enabled by default but to turn it on or off you change the "true/false" value inside the "modelGlueConfiguration" bean in your ColdSpring.xml" file:

<property name="debug"> <value>true</value></property>

Use a custom ColdSpring bean for your configuration settings

To store your own configuration values to be used throughout the application you can take advantage of Model-Glues " SimpleConfig bean. This bean configuration will go in your ColdSpring.xml file and look something like:

<!--- Get the path to the label templates from the configuration ---><cfset var networkPathToLabelTemplates = variables.appConfig.getConfigSetting("networkPathToLabelTemplates") />

Take advantage of an ORM framework such as Reactor

Initial versions Model-Glue used to come with Reactor but do no longer. I found it is a little difficult for a novice to start using Reactor with Model-Glue. To take advantage of Reactor, you have to download it, unzip it and create a ColdFusion mapping called "reactor" that points to your Reactor directory. Next comes the Model-Glue configuration. You need to add the following bean definitions to your ColdSpring.xml:

Now you can take advantage of Model-Glue's built in scaffolding and generic messages (genericList, genericRead, genericCommit and genericDelete).

Use event beans

Event bean is a predefined CFC that will store values form your form. It works by creating a one to one relationship between your form values and getters/setters in your CFC. Once you have your event bean created you can populate it from everything in the form scope by using the makeEventBean function:

Creating event beans has to do with defining getters/setters and local variables for each one of your form fields. It can be a tedious process but thankfully there is the Rooibos Generation to do it for you by simply taking the names of your form fields and generating all the code for your. You can learn more about this at Dan Wilson's blog article about So you want to create a ModelGlue:Unity application? ( Part 3 ).

To use your newly generated bean, you will have to add the bean definition to your ColdSpring.xml file:

Keep in mind that if your form changes, you have to regenerate the bean. If you do not care about reusing your beans through external consumers, such as Flex, I have written about an alternative approach in "Get All Form Fields in ModelGlue the Elegant Way".

Use tracing to debug the value of your variables

Once you have debugging enabled, you can add values to your debug output by using the event.trace() method:

<cfset arguments.event.trace("myVariableValue", myVariable) />

Split your configuration in multiple files

You can tell ModelGlue to include different configuration files by using the "include template" directive. I use this approach to separate my events based on the action they are related to:

Monday, March 17, 2008

While looking for a solution of a different problem, I realized that SQL Server has a feature called extended stored procedures. Extended stored procedures, at least in SQL Server 2000, have to be written in C++ and compiled to a DLL. A good example of such a procedure is my previous article on Using Regular Expression in SQL Server. I am not going to get into how to write extended stored procedures in this article since my C++ skills are rusty at best. Instead, I will talk about using an existing stored procedure to query the file system and how to move from using ColdFusion to using SQL Server.

Using ColdFusion

Getting a list of files is pretty trivial in ColdFusion. All you have to do is use the "<cfdirectory>" tag like so:

Tuesday, March 11, 2008

Object oriented programming has been around for quite some time and was made popular by C++ back in the day. Nowadays, even web scripting languages support the paradigm. JavaScript does not have true OO but allows you to use the design pattern in your code. Today, we will dive into using object oriented programming with the popular JavaScript framework Prototype.

Prototype makes it easy to declare your own objects by using the "Class.create()" method:

var sampleObject = Class.create();

Once you do that you can start writing your class specific methods and properties inside the "prototype" object:

sampleObject.prototype = {
}

To declare private properties you simply specify the name of the property followed by a colon and the value:

The important thing to remember is that each property needs to be separated from the next by a comma. As shown above, property values can be a array, a number, a string but cannot be empty value. While:

currentLinkID: 'myLink',

is valid while

currentLinkID:,

is not.

Declaring your own functions is not any different. Before we look into functions however, there is one function that requires special attention: the "initialize" function:

initialize: function() {
}

The initialize function is what JavaScript calls automatically when you create an instance of your object. If you are familiar with OO, this function is the constructor of your object. Any setup and initial requirements for using your object should be done in here.

So to get back to regular functions, they are declared in the format functionName: function() {} as in:

Big deal right, including that in your html page and/or a separate JavaScript file does not do anything for you. The next step in making it of any use is to actually create an instance of the object like so:

var sampleObjectInstance = new sampleObject();

Creating an instance of the object automatically calls all your code inside the "initialize" function. Here, a good practice is to wrap the creation of your object inside the windows load or dom:loaded (Prototype v1.6) event like:

To expand on using properties inside your object, whenever you want to access a property such as "currentLinkID", you have to prefix it with "this" as in:

this.statusMessage = 'Who Am I?';

That is because inside your object's function, without "this", the code does not know about the property. The same applies to using function so you cannot simply called the "doSomething" function with:

doSomething();

but instead if you have to use:

this.doSomething();

This can get a little more complicated when it comes to using event listeners inside your code. Let me elaborate. To tie an event observer that will call the "doSomething" function when a users clicks the link with ID 'myLink', you would usually do:

$(this.currentLinkID).observe('click', this.doSomething);

While that will work, if you try to access any class properties (such as "statusMessage") inside the "doSomething" function, you will get "undefined" for their values. That is because, again as pointed out above, the function is not aware that it belongs to an object so it does not know that the object has properties. The remedy is simple, simply append .bind(this) to the function when it is tied to the event as in:

$(this.currentLinkID).observe('click', this.doSomething.bind(this));

A similar approach needs to be applied when using the prototype built-in Ajax object. If you want to tie your custom functions to the "onFailure", "onComplete" or "onSuccess" functions, you need to use the "bindAsEventListener" function:

Thursday, March 06, 2008

While working on a video content management system, I was in need of capturing frames from video files so they can be used as a preview for video. The system already had some code in place but it only worked for video encoding in Windows Media format (.wmv extension). That would not do, I thought, not in this day and age when we have so many file formats and video codecs. So I need to able to:

Capture video frames from within .NET code

Capture images from all kinds of different video formats

Not reinvent the wheel while satisfying #1 and #2

Enter MPlayer

Stolen directly from the Information page of MPlayer, "MPlayer is a movie player which runs on many systems (see the documentation). It plays most MPEG/VOB, AVI, Ogg/OGM, VIVO, ASF/WMA/WMV, QT/MOV/MP4, RealMedia, Matroska, NUT, NuppelVideo, FLI, YUV4MPEG, FILM, RoQ, PVA files, supported by many native, XAnim, and Win32 DLL codecs. You can watch VideoCD, SVCD, DVD, 3ivx, DivX 3/4/5 and even WMV movies..". To add my own description, MPlayer is an open source command line video player.

So What

Metallica has an old song called "So What", great tune for the metal heads in all of us. The answer is simple, MPlayer can do a lot more than play video. It can capture images, stream video over http and even transcode video from one format to another (but the last feature is grounds for another article).

' Specify the path to the video file
dim videoFilePath As String = "drive letter:\path\to\myVideo.mpg"
' Declare the mplayer instance with the mplayer executable residing in the
' same directory as your executable
dim mPlayerInstance As new mPlayerWrapper()
' Capture frames
mPlayerInstance.captureFrames(videoFilePath)

This will capture 12 frames with 5 second interval between each frame and put them in the same directory as your video file. The filename of each frame will be "myVideo_thumb01.jpg" to "myVideo_thumb12.jpg". Each frame will be scaled to 270x200.

Capture arguments

mPlayerPath - sets the path where the mPlayer executable (mplayer.exe) is located. The default is in the same directory as your code is executing.

currentFilePath - sets the file path to the video file you are using

cleanOutputDirectory - deletes all the "jpg" images in the capture output directory before capturing

captureInterval - sets the interval at which frames will be captured. Only applicable if using a time interval capture method (as outlined below)

numberOfFramesToCapture - the number of frames to be captured

captureExactNumberOfFrames - tells the wrapper to attempt to capture the exact number of frames as specified by the "numberOfFramesToCapture" property. This can be used if a file has too few frames but you still want to capture an exact number

useTimeSeekToCapture - used to set the wrapper method of capture to seeking through the file instead of capturing a frame at an interval

thumbnailPrefix - the prefix to be used when creating the filenames for captured frames. The default is the name of the video with "_thumb" append to it as in "myVideo_thumb01.jpg"

capturedFrameWidthHeight - the width:height that each frame will be scaled to. The default is 270:200

scaleCapturedFrames - used in conjunction with the "capturedFrameWidthHeight" property to scale down the captured frames. Set to true by default