Shafqat's Recent Online Activity

Twitter

May 20, 2014

There was this old cpmputer in office that no one was using and I wanted to install Ubuntu and test out some Microsoft alternative tech in there. The monitor was quite old. It was a Samsung 923nw. There was no available driver for Ubuntu or linux. The monitor supported 1440x900 pixels but the display properties was showing Unknown monitor and a highest resolution of 1024x728.

You can simply create custom resloution in linux and set that to a monitor. Here is how ...

1. First figure out the monitor ID in the system. Type the following command. (Command is in red, output is in blue)

April 09, 2012

How much fun would it be if you could run Ubuntu on you Galaxy Tab 10.1 device without any trouble. Nice, right? But how would you feel if you did not have to reboot but run Ubuntu from Linux easily while you are playing Angry Bird or reading news on feedly app? Cool, eh?! You might be thinking we might run Ubuntu on VM ( Virtual Machine ) and that would take a lot of resources and time to boot. Nopes! It would not run from a VM or take up resources. How is that possible? Well it can be done because of a process called CHROOT. Here is a primer.

CHROOT

Since I am a windows user this is quite a big deal, but most Linux users know about CHROOT. Android is a linux based operating system, and Ubuntu is also linux. Here is the wikipedia definition “A chroot on Unixoperating systems is an operation that changes the apparent root directory for the current running process and its children. A program that is run in such a modified environment cannot name (and therefore normally not access) files outside the designated directory tree.”

So once android have booted up, we are already in linux but if we spawn a new process that has the working directory set to Ubuntu distribution then it would run in everything from that installation while the operating system services are already loaded by android. Now if we start the vnc server ( as remote desktop service) and then connect from an android client then we should be able to see Ubuntu screen on android.

Articles on How to

Now there is already a tutorial on the site galaxy tab hacks at this location. (Article 1)

However I am going to merge it down to my experience by putting in correct steps by merging these two.

What we need …

I am assuming that you know what adb is and how to connect via adb. If you do not know please refer to Article 1. I am running ICS on my galaxy tab ( pretty fast ) which is already rooted and have busybox installed. If you don’t have a root just download super one click root and root your tab, and then download the busybox installer from market ( now play) and install.

3. Create a folder in your tab called Ubuntu ( you can create it anywhere, we will create it in /sdcard/ubuntu folder

4. Transfer the contents of the zip to the folder. It has the Ubuntu image a few shell script files that will help us install.

Steps to install CHROOT Ubuntu

5. Connect to the tab via adb by typing “adb shell”

6. Go to the folder by typing “cd /sdcard/ubuntu”

7. Install by typing “sh ubuntu.sh”. This will install the shell. Don;t worry about error messages.

8. Now become the root super user by typing “su”. You might need to do this before step 7, but I don’t remember. If you face issues execute it before step 7.

9. Boot into linux by typing “bootubuntu”. And instantly you would be in command shell for ubuntu.

10. We want to install vnc server, but for installing we need to update the install catalog. The catalog for update is in a wrong location. Lets update that. The catalog resides in /etc/apt/sources.list Type the following command to delete that list file

“rm /etc/apt/source.list”

Then create the list file with correct location.

“cat > /etc/apt/source.list”

Then put the correct location

“deb http://old-releases.ubuntu.com/ubuntu/ karmic main universe”

Now to save file, press control+d twice and then press enter. This will save the above text in the list file.

11. Now update software catalog by typing “apt-get update”. Make sure the tab is connected to the internet.

12. Install open ssh ( optional step, but I did it anyway) “apt-get install openssh-server”

14. We are almost done, but there is a bug, the keys of the keyboard will be all wrong. So lets fix that. Type “cat > $HOME/.vnc/xstartup” . This will start editing the file. Lets type the following as it is …

However if you want to connect to dhcp ( I use static ip on my wireless), type the following

“nmap -p 22 --open -sV 10.0.0.0/24”

Connecting from your PC

If we want to connect from your PC, we need to tell it to bind to the static ip. Use this to start the vnc server “vncserver –geometry 1280x750 –interface x.x.x.x” – replace x.x.x.x with the ip in the wireless. Then connect via any vnc client from to the ip on 5901 port.

December 02, 2010

In my mind two great inventions Microsoft came in last ten years were Windows Presentation Foundations (also Slilverlight) and Powershell. We in .NET used to jealous of Linux Shell Programming as something that powerful was not possible in Windows. Powershell changed all that, at present it is the most powerful scripting language and shell available ( btw: Its 10 times better than linux shells even).

I use PS (PowerShell) in everyday repetitive tasks. We use it to download source code from source code control, build code, release code, create release notes, email to sqa, mark task as fixed in task management system, run automated test scripts and in many more tasks. From now on I am going to share a few PS recipes that make my life easier on a regular basis. I use it as much as I use C#.

When doing automated build, its best to clear the old binaries, or even its good to clear the binary files when running from VS because old binaries can sometimes create unpredictable results.

Now we want to delete the dll and pdb files. We will create a function called Delete-Binaries. Get file function will return the list of files of a certain type. Then we can delete the files. See below

December 01, 2010

Windows NT is a dynamic filesystems. It has certain cool capabilities such as file and directory linking, spurse files, alternate streams and many more. Today we are going to see what Alternate Streams can do for us.

Alternate Streams

We can programmatically open files in NTFS as file streams. NTFS supports alternate file streams for any file. We can open multiple file streams on a single file. To a normal user only the default file stream would be visible. This technique can be used to hide sensitive data in plain sight, or hide the encrypted data in hidden stream. Some viruses and rootkits use this technique to hide. Even two years ago many of the famous antiviruses did not use to scan for alternate file streams. Before we go into programming details lets have fun exercise with alternate file streams.

Step 1. Open up command prompt in windows Step 2. Type the following notepad myfile.txt Step 3. Type "Default text" in notepad then save the file Step 4. again in command prompt type notepad myfile.txt:hidden_data Step 5. Type "My gmail password : yoho" in notepad then save the file Step 6. Open the myfile.txt. What do you see?

You can see the "Default text" in the file. But the data you saved in the alternate stream named "hidden_data" is not visible. If you look in the explorer you will not find the file named myfile.txt:hidden_data, rather you will only see myfile.txt. Now if we type in the command again "notepad myfile.txt:hidden_data" then we will see that the text "My gmail password : yoho" is on notepad.

So, myfile.txt have an alternate file stream called "hidden_data". We can open and keep multiple such streams for the file. This is a cool feature, isn’t it?

.NET Does not support NTFS alternate strems

NTFS streams have been around for more than 10 years and we have .NET version 4.0 released. But unfortuntely .NET Framework yet do not have any support for alternative file streams. This is quite unexpected and sad.

I am quite disappointed in Microsoft for not providing such a basic file system manipulation has been in NTFS for over 10 years.

So? What is our option

You can write Alternative file stream class yourself by using the basic windows APIs. But it would be a lot easier to use a prebuilt library. Greg Duncan has already created a library. His code can be found here.

January 19, 2010

Update : I am no longer using e72. I do not see symbian as preferred platform for developing smartphone applications. I have moved on to android which I personally find almost magical.

A little Background

A few days ago I got a Nokia E72 phone as a gift and I really loved the phone because of its 600 Mhz processor. I needed a phone with a full keyboard so that I can check and send emails easily from anywhere. I like windows mobile platform but did want to try raw processing power. As usual I tried writing .NET in Symbian ( usual Nokia phone OS ) and had fun. But I needed more functionality that is native to the OS and decided to go with python. Although Nokia has a cool gadget API, I wanted take advantage of the hardware on the phone. There is a python compiler for the S60 platform, I installed it and wrote my first tiny python code.

Video Demo

This is what the application does

A python primer for a c# (or any) developer

Unlike perl, which looks horrible (sorry if any perl developer is reading this, I mean you no harm, it is just my opinion), python looks pretty structured somewhat like Delphi/Pascal. My objective was to write a few programs for my phone and play around so I wanted to learn python fast in an hour. Before we proceed further we should know a few facts about python. Firstly Python is a dynamic scripting language that is interpreted. Then …

See the indentation that you have to make for the function? Yep that makes most python code look nice at the same time

No variable Declaration

There is no variable declaration required to use a variable, just make up a variable and use it. See example

myname = “Shafqat Ahmed”
print myname

Importing namespaces

Like .NET you can import namespaces ( called modules ) in python.

import math
import appuifw

A primer on python for Series 60

Since my objective was to write code on python for Symbian, I am going to share few little tidbits.

For Python for S60 make sure you use Unicode everywhere

The Python S60 SDK for Symbian (I think other phones also have this same requirement) requires you to use unicode strings in most places. See below

myname = u“Shafqat Ahmed”
print myname

See that “u” in front of the string declaration? That is how you make a unicode string.

Important modules

For a PyS60 app usually the first required module is the Application UI Framework for short appuifw. The e32 module contains some system function, the contacts module lets us use the contacts on the phone, the messaging lets us use sms messaging, the telephone library lets us dial or hang up a phone number, the audio module lets us play music, or record voice note or record a phone conversation.

Using the Symbian Native UI

There are simple functions that lets the user use the native Symbian UI widgets. For example if we want to read values as input. We could write something like

import appuifw
name = appuifw.query(u”What is your nam?e”, u”text”)

Code

Reading the Accelerometer data on PyS60 on Nokia E72

In order to access the accelerometer data we need to use the sensor module. Then we will setup a callback where the accelerometer data will be sent. Here is how the code looks like ( I created a class to organize code, so the actual code look a little different)

Then I can access the data in my callback function each time by accessing accelerometer.x for X axis data, accelerometer.y for Y axis data and accelerometer.z for Z axis data.

Managing Sensitivity

The E72 has a -60 to +60 range value for each axis. We could use a constant value to move the graphic object, then that would move evenly. But we want to the app to be more responsive human like. So if we take the value of the axis and devide it by a value, say 10 then I would get 1 for a 10 value from the accelerometer, a 3 for 30 on accelerometer, a 6 for 60 (max value) on accelerometer. Similarly I would get a -1 for -10. A value of +/-10 mins the device is a little tilted, +/-60 means highly tilted. So can move the object faster if the device is more tilted.

Also I have added a menu for controlling sensitivity. If I use 5 instead of 10 then the division would result higher values thus making the movement faster. The application code looks like this.

January 01, 2010

J2ME has always been the standard in cross phone applications. Most java app are truly cross phone. The only phone we could write code in .NET was the Windows mobile or Pocket PC. A company called AppForge came up with a solution to run .NET code. But in 2007 the company was acquired by Oracle and shut down.

Now Red Five Labs have come up with a .NET based solution called Net60 which runs windows mobile based .NET code. Currently .NET 2.0 framework is supported and they plan to add support for compact framework 3.5. Now what made me exited is their licensing policy. With AppForge it was not possible to develop when the trial expired. With Net60 that is not not case. You can get a IMEI bound license for your non commercial application. Now, isn’t that wonderful? That means I can write my own apps to run on symbian and use them. That also paves way for .NET based open source apps. Lets face it, Nokia holds 41% market share for smartphone applications, which is a very large number (source).

I recently got myself a Nokia E63 phone, which is actually an economic version of Nokia E71 (only diff is 2Mpx cam instead of 5 Mpx cam and no GPS and plastic body instead of steel). It has the full qwerty keyboard. I spend a lot of time reading and writing mails as communication has become a large part of my role, so the full keyboard is really helpful. I was looking for a good way to write C# code and run it on my phone ( E71- Symbian OS S60 3rd edition feature pack 1). After a few tries it worked very nicely. So here is what you need to do in order run .NET code on your Symbian phone.

Aquiring the SDK

1. Get the Net60 SDK from here. It is about 13MB. You would have to provide your phone IMEI no and email. That is the phone where the application would run. If you want a version where it will run on different phones, then you would have to get a commercial license. 2. Check your email to get the free non commercial express license from Red Five Labs. ( I’d strongly suggest you see the video tutorial at their site, which is the next set of steps about)

Writing the Application

1. Create a compact framework 2.0 application (not CF 3.5) 2. After you are done with the application add a Genesis Setup Project to the solution 3. Point to the executable, fill up the necessary fields 4. Add Net60.sisx as dependency, also add the license that you received from RF Labs. 5. Compile, install the output on your cell phone and run

Debug version vs Release Version

The debug version comes with logger and a runner tool. If you are developing, I would suggest that you use the debug version. Making a build and deploying each time you change the code can be pretty annoying. There is a NET60 Launcher tool that can run a .NET executable. So instead of building installation package and deploying each time you can just use that tool to run the exe. Make sure you copy the executable under c:\DATA\RedFiveLabs\Apps\ folder so that it shows on you NET60 Launcher.

Additional API

There are some additional API provided by RF Labs to control various features in Symbian OS. These are distributed over a few assemblies. These assemblies are

August 23, 2009

A few days ago we wanted to how would our application behave under high stress scenarios and wanted to take as little time as possible. We thought we would be able to use Microsoft Web Stress tool. But it turned out that the tool location is not available anymore on the Microsoft web site. We tried an older version, but it seemed that it was not a proper tool to run load tests. We have a custom Sharepoint application running in front of an application server. All of the core business logic goes through the application server. The web application calls the application server via a few web service calls. This is how the application is supposed to be deployed.

Since all the core requests are coming from web to the app via web services, and our goal was to to stress test the application server, we could simulate stress by calling the web service in the application server.

Issues

The first and basic issue is, that we had very limited time and in that time it was not possible write real life test scenarios for most of the use cases.

The second issue was, the test should be as realistic as possible. Writing tests may not be best way.

Solution

The solution was to record a real use case, and then play it again many times with a little varied parameters.

Implementation

So we injected a little code in the web service (we could do this with context handlers, but we did not have the time). Serialized each of the web requests sequentially, In the start of a use case we would clear the folder where we were serializing the web requests. After we have finished the use case then we would de-serialize the requests and inject some randomizing items to a few parameters that changes in real life and then run in multiple thread to simulate real stress. The diagram below shows the whole process. The process if repeated in multiple threads to simulate multiple clients and it worked.

So if any of you want to simulate real life scenario without writing too much code, give this a try. Will update this blog with sample code a few days later.

May 26, 2009

For teams located in different locations conference calling is a must. Most of these calling facilities have toll free 1-800 dial in numbers. Skype lets you dial in to the toll free 1-800 numbers without any cost. One annoying thing with these conference calls is that you have to dial in a pin code to enter. If you are dialing in to one conference only then you memorize it. But if you have to dial in several different conferences calls in a day it becomes a pain. You would probably have to lookup the outlook calendar entry and type it in, which is quite annoying and distracting. Recently I have been trying to do everything with Powershell. As you have guessed … … now we can dial into a Skype meeting with one powershell command.

I just wrote a function ( just bare minimum, no error handling ) that calls a Skype contact or a phone number waits for certain time then enters the pin as DTMF codes. I have the different meeting and number to different functions which calls this function. And now I can get into any meeting without hassle.

The function is called Call-SkypeContactSendDTMF. The first parameter can be a skype contact name or a saved phone only contact or a number to dial. The second parameter is the number of seconds to wait after the phone has been picked up by the conference bot. The third parameter is the DTMF codes. You can add pause by putting in “@” to pause for a second. Here is the calling syntax

May 23, 2009

Recently I have been trying to do almost everything in Powershell. With the release of Powershell V2 RC and the awesome visual debugger that comes with Powershell called Powershell ISE, I have a Powershell window open at all times. I am getting my google calendar events form command line, doing recursive ftp uploads and download as well. Now I have started to even do outlook automation with PS.

The Problem

Any slightly efficient developer would write functions and reuse them later. So I tended to write functions add kept on adding them to my Powershell profile file. After a while that turned a little messy and hard to manage. So now I thought why don’t write different script files put my functions in there and use them when needed. So I created functions for Google Data API, Outlook functions. Now when I call them from the prompt, after execution of the script the functions are no longer there. It is because the scriptblock runs in a different execution context and after execution the context is removed. So the functions I have put the file no longer exist when I call them.

For example I have written a function in myfunc.ps1 to convert string into base64 string which looks like this

Now after running the script .\myfunc.ps1 when I call the function, it no longer works.

Solution

After running around the internet for hours, looking for a solution to run scripts in the same executions context and trying out several different approaches, I was about to give up. Then I found the worlds easiest solution in Donn Felkers blog. All you need to do is to add a simple dot ( “.” ) at the start of the line to make it run the in the same context.

So if you were running the script like this,

.\myfunc.ps1
or
c:\somefolder\myfunc.ps1

You should run it like this

. .\myfunc.ps1
or
. ‘c:\somefolder\myfunc.ps1’

See the added dot? That does the trick. I wish this was advertized more, then I wouldn’t have had to search for hours to find a solution.

April 14, 2009

Whenever we try to use a .NET component from VB6 or VC++ applications we tend to expose the .NET classes as COM components and register them and from the VB6 or VC applications so that we can instantiate and use the COM components written in .NET. One downside to doing interOp this way is the use of Registry to store the COM information as COM uses registry to lookup the prog id and clsid. This hassle can be avoided by using xcopy deployment with Registry Free COM.

Isolated Applications and Side by Side Assemblies

.NET is not the sole platform to come up with xcopy deployment and freedom from dll hell. Cpp and Delphie had this for long time. Isolated Applications and Side by side Assemblies have been invented as a solution to the versioning problem.

According to MSDN, Isolated Applications are self-describing applications installed with manifests. The benefits of Isolated Applications are that these are more stable and reliable since they are not affected by installation or change in other software, they will always run with the component version they were originally intended for, they can take advantage of Side by Side Assemblies feature, can be installed with xcopy deployment without impacting the registry. A side by side assembly can be a dll, windows class, COM server, type library or interface defined by manifests.

Private and Shared Assembly

There are two types of Side By Side Assemblies. Private Assemblies are those which are for consumption of one single application, usually placed in the application folder or sub folder. But a Shared Assembly can be used by multiple applications. A fully Isolated application uses only side by side assemblies. If you want to find the Shared Assemblies installed in your computer look in \Windows\WinSxS folder. The shared assemblies are installed here.

Manifests

Manifests are xml files that accompany assemblies with extra information like dependency, binding, activation. A manifest can be an external file or can be embedded as a resource within the assembly. In older operating systems like Windows XP, manifests need to be embedded inside the side by side (denoted SxS) assemblies to work properly. Assembly Manifests describe side by side assemblies and thus needs to be embedded to work properly in operating systems like Windows XP. Application Manifests describe isolated applications and do not need to be embedded inside the application.

Normal Registry Based COM Component Load Procedure

If we look up a COM self registering dll we will find that it usually* has four stdcall calling convention functions. Look at the dependency walker image below of a standard self registering COM server dll.

When we use regsvr32.exe to register a COM dll, the register server application calls the DllRegisterServer function of the dll and that function call inside the dll takes care of putting information in the windows registry so that it can be used by any user application. Now lets have a look at the registry where all these information is recorded. Let us take a standard windows COM component and see its registry structure. The COM component that we will look up today is FileSystemObject from Micorsoft scripting runtime. The progid of the object is Scripting.FileSystemObject. Lets say we are using script code like this

Set fso = CreateObject( “Scripting.FileSystemObject” )

So lets try to have an understanding how this object is created from the code above. If we go and try looking up the string “Scripting.FileSystemObject” under the HKEY_CLASSES_ROOT key then we would see this.

As you can see there is a node called CLSID and it contains the Class ID ( GUID of the class) inside the node. Now lets go and visit that Class ID node under the HKEY_CLASSES_ROOT\CLSID registry node. See below

As we can see that this is how the dll containing the class object is found, COM infrastructure then calls the DllGetClassObject function to retrieve a pointer to the object we are looking for. As like all COM interfaces the object implements IUnknown interface and all scriptable COM objects implement IDispatch interface. However how COM identifies and uses the object beyond the scope of this article. The point that I wanted to make is that the windows registry has a very important function here and to have any COM component usable by other applications we had to register it in windows registry.

Registry Free COM Access in an Isolated Application

In an Isolated Application we can skip entering all the information in the registry and rather enter them in the manifest. We need to create an application manifest which will have the filename in the format of application.exe.manifest. Then we need to create an assembly manifest and embed that assembly manifest inside the dll. Then we would have created an Isolated Application and would not need to register to windows registry to access the functions of the COM dll.

The side by side assemblies are searched in the following order

1. In the executable folder
2. Subfolder of the executable folder, subfolder must have the same name as the assembly. For example see inside your Windows\WinSxS folder
3. In language specific or culture specific subfolder of the executable

Activation Contexts are data structures that allows the OS to redirect loading. For example it can redirect the loader to load a specific version of a dll, or redirect COM object loading process. We will use the COM object loading redirection to avoid using windows registry. For more information on Activation Contexts read this.

Doing it by example

I have setup a sample application with source that will show how we can create a registry free COM component. Please note that there is already a practical example in the following article in MSDN, Registration-Free Activation of .NET-Based Components: A Walkthrough by Steve White and Leslie Muller. You might want to read that article if you find my example not clear enough.

1. Create C# based class that serves funny quotes and expose that as a COM component and register it.

2. Create a VB6 executable that uses the COM dll to show quotes

3. Unregister the COM dll to show that without COM registration in windows registry the application is no longer working

4. Create an application manifest, then create an assembly manifest and recompile the C# application with the embedded assembly manifest to create our isolated application.

5. Move the isolated application to a different folder and prove that the application is using COM without registry.

1. C# COM Dll

We are going to create a dll called QuoteSource and a class that serves Quotes called QuoteProvider and it is going to implement the IQuoteProvider interface. The methods in the interface will be exposed.

Here is how the interface looks like.

Here is the class that serves quotes and implements the interface above

‘

The assembly info contains the assembly guid

We need to make sure that a tlb file is generated and the assembly we created is registered. We will select “Register for COM interop” option from the build properties screen. See below …

2. VB6 Client Executable that uses the C# Library

We have created a simple VB6 application that refers and calls the C# assembly dll. See below:

Let compile the executable and it becomes VB6Client.exe

3. Unregister the C# COM dll to make sure that we a re not using the reference

We will use the following command line in the folder where the dll resides

regasm /u QuoteSource.dll

Lets run our VB6Client.exe and press the Get Quote button… walla! It does not work anymore. The executable exits with automation error since we unregistered it from windows registry.

First we are going to create an application manifest. The application manifest has to have the name in the format of exetutable full name + .manifest. Since our executable is called VB6Client.exe our application manifest will be called VB6Client.exe.manifest. Here are the contents of the manifest.

Now lets create the assembly manifest and embed it inside out QuoteSource.dll. First we will create an xml file called QuoteSource.manifest. Please observe that we have put in the progid and clsid of the COM exposed class.

Now we need to convert this assembly into a resource file. We will use rc.exe that comes with windows SDK, or VC++ or VB6 installation.

We will create a text file called QuoteSource.rc will put in the following line here.

1 24 QuoteSource.manifest

Here 1 is the manifest resource id and 24 means that it is a type of manifest resource. We will now invoke the Windows SDK resouce compiler from the command line. Check where in you computer you have the resource compiler installed. Here is the command line

rc.exe QuoteSource.rc

After invoking this it will create a file called QuoteSource.res. This is a windows resource file.

Now we will use a custom build command to create the dll. Observe the command line below

See the win32res switch, it tells the compiler that it is a win32 resource.

Now copy the dll to the same folder as the exe, also copy the application manifest. There is no need to copy the assembly manifest as we have already embedded it inside the dll.

Run the the application and press the Get Quote button, it works!

5. XCopy Deployment

Copy or move the folder elsewhere, even to another computer. It will still work without any need to register the .NET COM dll. This is because we have created the application and assembly manifest and windows is treating our application as a Isolated application.

Last Words

This way you can create XCopy deployment for even COM based applications and avoid storing information in registry. Even a multi dll VB6 or VC++ application can be made Isolated Application.