Code & More

Npgsql is the most popular Postgres data provider for .net. None of the integration examples involving entity framework I found online used the db first paradigm. After a lot of testing I posted a working example of EF code first with Npgsql below.

Note: this example requires the database to already exist. EF automatic creation/migrations will not work.

Db configuration class, this ensures the correct connection factory is used. My testing found this was the only way to get it set, various connection string formats were ignored/overridden at runtime.

Next define a context class decorated with the custom db config attribute. Ensure the default schema is set to “public” (or the relevant schema name).

Key Points:

Ensure context decorated with Npgsql config class

Ensure correct schema name is set in OnMOdelCreating method

Don’t use an initializer

Apply any case/naming conversions needed

Some sort of case conversion will likely need to made in the OnModelCreating method, in my case I made all column names lowercase and did the necessary column name transformation there. For table names I used decorator attributes on the entity classes.

I recently started developing an Android Wear app and needed to pair a watch running in an emulator to a physical phone(s4). The current documentation on the net is somewhat stale so the following is an up to date procedure:

Power on Android Watch AVD

Connect the physical phone via usb and ensure usb debugging is turned on in system options.

Turn on blue tooth debugging on the watch

swipe to get to the settings menu

scroll down to the about menu

tap 7 times on the build number to enable developer mode

navigate back to the main settings menu and select developer options

tap enable avd debugging

tap enable bluetooth debugging (disabled until avd is enabled)

on the home screen a notification should be displayed stating bluetooth debugging is enabled

In an SDK console type “avd devices” both the physical phone and watch emulator should be listed

Type “adb -d forward tcp:5601 tcp:5601” to map the emulator ports

On the phone download the Android Wear app from the play store.

Open the wear app and choose pair device.

On the pair device screen press the physical menu/settings button on the phone. An option to “connect to emulator” will appear, tap it.

The phone should find and connect to the emulator and display the connected options in the wear app. You can scroll to the bottom and tap one of test options to send a test notification to the paired watch.

Bitwise Ops

//num is odd or evenfuncisEven(nint)bool{returnn&1==0}//test nth bit is setfuncisSet(xint,nuint)bool{return(x&(1<<n))>0}funcsetBit(xint,nuint)int{returnx|(1<<n)}funcunSetBit(xint,nuint)int{// negate = all 0 except 1 positionreturnx&^(1<<n)}functoggleBit(xint,nuint)int{// ^ = xor = both same = 0 else 1// only of or is true//xor twice returns original valuereturnx^(1<<n)}//turn off rightmost 1-bitfuncturnOffRightMost(xint)int{returnx&(x-1)}//turns off all other bits except rightmost "on" bitfuncisolateRightmostBit(xint)int{returnx&(-x)}//produces all 1's if x = 0funcrightPropogateRightmostBit(xint)int{returnx|(x-1)}//flip all bits in integerfuncflipBits(xint)uint{return^uint(x)}

funcmerge(vals[]int,helper[]int,lowint,middleint,highint){//copy both halves into helper slicefori:=low;i<=high;i++{helper[i]=vals[i]}helperLeft:=lowhelperRight:=middle+1current:=low//iterate through helper array. Compare left and right// half, copying the smaller of the 2 into originalforhelperLeft<=middle&&helperRight<=high{ifhelper[helperLeft]<=helper[helperRight]{vals[current]=helper[helperLeft]helperLeft++}else{vals[current]=helper[helperRight]helperRight++}current++}// Copy the rest of the left side of the array into the// target array.remaining:=middle-helperLeftfori:=0;i<=remaining;i++{vals[current+i]=helper[helperLeft+i]}}funcmergesort(vals[]int,helper[]int,lowint,highint){iflow<high{middle:=(low+high)/2mergesort(vals,helper,low,middle)mergesort(vals,helper,middle+1,high)merge(vals,helper,low,middle,high)}}funcMergeSort(vals[]int){helper:=make([]int,len(vals))mergesort(vals,helper,0,len(vals)-1)}

Data Structures

typeHeapstruct{data[]int//maxSize int//itemsInArray int}funcNewHeap(sizeint)*Heap{h:=new(Heap)h.data=make([]int,0,size)returnh}//moves element up to its proper position//respecting binary tree propertyfunc(h*Heap)up(idxint){for{parent:=(idx-1)/2ifidx==parent||h.data[idx]<h.data[parent]{//at end of array or idx is in proper positionbreak}//Swap with parenth.data[parent],h.data[idx]=h.data[idx],h.data[parent]idx=parent}}func(h*Heap)down(idxint){end:=len(h.data)for{//start with left childchild:=2*idx+1ifchild>end{break}ifchild+1<end&&h.data[child]<h.data[child+1]{//use right childchild++}ifh.data[child]<h.data[idx]{//proper positionbreak}//swap parent with childh.data[child],h.data[idx]=h.data[idx],h.data[child]//move to grandchildrenidx=child}}func(h*Heap)Pop()int{result:=h.data[0]h.data[0]=h.data[len(h.data)-1]h.down(0)h.data=h.data[:len(h.data)-2]//shrink when excess capacity hits 3/4iflen(h.data)<=cap(h.data)/4{h.resize(len(h.data)/4)}returnresult}func(h*Heap)resize(sizeint){len:=len(h.data)temp:=make([]int,len,size)foridx,val:=rangetemp{temp[idx]=val}h.data=temp}func(h*Heap)Push(vint){h.data=append(h.data,v)h.up(len(h.data)-1)//grow when cap hitiflen(h.data)==cap(h.data){h.resize(len(h.data)*2)}}

Nupic is Numenta’s current open source implementation of Jeff Hawkin’s hierarchical temporal memory(HTM) model. It currently consists of the CLA (cortical learning algorithm) which is a single stage/layer of the HTM implemented in a mix of python and C++.

In an effort to better understand the difference between the current implementation and the whitepaper I decided to try and port the spatial and temporal poolers to Go. Porting line by line gave me the opportunity to understand the design better as well as it’s dependencies: python, numpy, etc…

One of the more difficult parts of this project was interpreting the numpy expressions and translating them into a statically typed language. A few nested numpy expressions can easily end up being 10 of lines of Go.

The result is 2 simple APIs for the spatial and temporal pooler which are go gettable.

packagemainimport("fmt""github.com/zacg/htm""github.com/zacg/htm/utils")funcmain(){tps:=htm.NewTemporalPoolerParams()tps.Verbosity=0tps.NumberOfCols=50tps.CellsPerColumn=2tps.ActivationThreshold=8tps.MinThreshold=10tps.InitialPerm=0.5tps.ConnectedPerm=0.5tps.NewSynapseCount=10tps.PermanenceDec=0.0tps.PermanenceInc=0.1tps.GlobalDecay=0tps.BurnIn=1tps.PamLength=10tps.CollectStats=truetp:=htm.NewTemporalPooler(*tps)//Mock encoding of ABCDEinputs:=make([][]bool,5)inputs[0]=boolRange(0,9,50)//bits 0-9 are "on"inputs[1]=boolRange(10,19,50)//bits 10-19 are "on"inputs[2]=boolRange(20,29,50)//bits 20-29 are "on"inputs[3]=boolRange(30,39,50)//bits 30-39 are "on"inputs[4]=boolRange(40,49,50)//bits 40-49 are "on"//Learning and prediction can be done at the same time//Learn 5 sequences abovefori:=0;i<10;i++{forp:=0;p<5;p++{tp.Compute(inputs[p],true,false)}tp.Reset()//not required}//Predict sequencesfori:=0;i<4;i++{tp.Compute(inputs[i],false,true)p:=tp.DynamicState.InfPredictedStatefmt.Printf("Predicted: %v From input: %v \n",p.NonZeroRows(),utils.OnIndices(inputs[i]))}}//helper method for creating boolean sequencesfuncboolRange(startint,endint,lengthint)[]bool{result:=make([]bool,length)fori:=start;i<=end;i++{result[i]=true}returnresult}

CLA and Hierarchical Temporal Memory are finally starting to grow in popularity and an open source community is starting to grow around the open source Nupic framework. Currently Cerebro is the best tool (and only) for prototyping CLA models it allows you to visually dissect your work.

Because Cerebro is a webapp built on mongodb and python it requires a bit of work get setup. To make it easier for newcomers to get up and running quickly I created a dockerfile containing all the dependencies required to run Cerebro. Once the docker image is built Cerebro can be run with one command. (Eventually users will be able to pull an official built image from dockers index)

Trying to configure a custom header/title layout with the newer Android Holo Theme can be painful, often producing cryptic error messages. According to google the issue is with the new actionbar added in the halo theme conflicting with the older title configuration.

Here is what worked for me: after creating a layout named “CustomHeader.xml” I added the following lines to my activity OnCreate method:

Recently when deploying an existing codebase to Azure I required the ability to configure log4net to write to azure storage services.

After some quick googling I found a NuGet package that seemed to be fairly well maintained and had a modest amount of users according to the NuGet download stats. NuGet Package here: http://www.nuget.org/packages/log4net.Appender.Azure/ or type the following in package manager console:

1

Install-Packagelog4net.Appender.Azure

However I immediately ran into a few issues, first problems building it with my project as it was built with dependencies on older frameworks such as log4net. Additionally when running the table appender in a real application with multiple concurrent loggers I began to get errors related to the way it submitted batch inserts to the storage service.

I was able to fork the NuGet project on github, update it’s dependencies, and fix the batch processing issue, you can find my branch here: https://github.com/zacg/log4net.Azure I have submitted a pull request for these changes, hopefully they will appear in the next NuGet package release.

Once referenced just choose your preferred storage method and set your connection string like so:

If you ever have to debug log4net appenders here are a few snippets that may help you. The first is a function to flush the log buffers. The second turns on log4net debug logging, it will log to system.diagnostic.trace, just place the app setting in your config file. And the last snippet is just a setting to write your trace logs to file, again place in your config file.

And finally a list of the errors encountered and fixed to help out random googlers.

Could not load file or assembly ‘log4net, Version=1.2.12.0, Culture=neutral, PublicKeyToken=669e0ddf0bb1aa2a’ or one of its dependencies.

WRN: Comparing the assembly name resulted in the mismatch: Build Number

All entities in a single batch operation must have the same partition key

I ran into a few roadblocks along the way but ended up with a working log4net appender that can write logs to Azure storage services with a few simple configuration settings. If you have any comments feel free to share below.

When defining custom form templates in backbone forms you may want to conditionally include content based on whether or not the form is nested. The following code will allow you to check using the template markup:

12345

<%if(this.options.fieldTemplate!="nestedField"){%>//form is nested inside another<%}else{%>//form is not nested<%}%>

This check will enable you to write fewer, more reusable form templates. The following changes to the default bootstrap template included with backbone forms allows you to include form submit and cancel buttons by setting a template data flag.

While writing some basic CRUD code for a recent project, I decided to create a base struct containing the redundant fields for all of my data entities (id,date_created, etc…). I then embedded this struct in all data entities and tagged it as inline so json/bson marshalers would treat it accordingly. A problem arose when I wanted to pass newly created data entities from clients into a json webservice. Normally when instantiating a data entity struct I would use the associated creation method (NewSomeEntity()) which would set the appropriate id/created date etc… however the json marshaler is not smart enough to do this as it builds the object graph. If the object graph is only one level deep, you can just run an init function on the new object returned from the marshaler, when the object contains n-levels (n-many relationships) it becomes a problem.

I had two options: I could implement custom marshal interfaces for every data entity struct, or I could write a function that reflects over the object graph after it has been built by the json marshaler and run my initialization function against any new/unintitalized base entity structs. I decided to go with the later option.

There are a few key functions needed to achieve the method described above, mainly: reflecting over an object to get a list of it’s fields, checking the type of the reflected field against your base struct, checking if the reflected field value is uninitialized or in the case of a pointer nil, and finally setting the value of an empty field to an initialized struct.

Recently while generating Go bindings for a C++ library with Swig I begain noticing memory issues. Specifically when memory pressure was placed on the application I noticed output paramaters of standard types like std::vector were being randomly deallocated. After triple checking my Swig template files for the correct wrapping signatures and reviewing the generated Go and C/C++ code I was at a loss.

After reviewing the Swig project commit log and noticing some fixes for Go related issues, I remembered I had installed Swig from the debian aptitude manager. Sure enough after running “swig -version” my swig version was at 2.0.7 (the current version at this time of writing was 2.0.10). 2.0.7 lacks many bug fixes related to Go. Downloading and installing 2.0.10 from the Swig website solved my memory issues.

Solution:

Check Installed Swig version

12

swig -version

If version is < latest: unistall packaged version, then download and install current release from the website: http://swig.org

When developing mobile cross platform apps with Xamarin I like to keep as much common code as possible in a shared portable class library (PCL) for easy reuse. This includes any webservice calls and the associated boilerplate code. I have been using a fork of the Restsharp library which has been modified to be PCL compliant. This strategy was working fine until recently when I needed to start uploading files through webservices. The Restsharp library has robust file upload support but because it is often tied to OS specific file operations it was left out of the PCL port. As a work around I simply imported the necessary file upload code from the original library using raw bytes and streams as interfaces rather than filenames. Leaving it up to the caller to implement the file loading in their native OS code.

Recently while working on an Xamarin based Android project I came up with a handy helper class for displaying basic list views. It saves alot of boilerplate code by not forcing a new adapter implementation for every domain object you want to use in a list view.

Recently I had a requirement to install a shared library as a build/deployment step in a QtCreator project, which of course required sudo permissions. Not surprisingly Qt Creator does not simply let you prepend “sudo” to a custom build step, here is the workaround I found:

First I moved all the commands that needed to run with sudo into a single make file like the following, mine was called InstallLib.make:

Next in Qt Creator with your project open, goto the project section, add a new “custom process” build step. In the command field type: “ssh-askpass” this program will popup a widget to enter the sudo password when executed. In the build step arguments field enter: “Sudo Password | sudo -S make -f InstallLib.make”. This will make the ssh-askpass program execute the “InstalledLib.make” make file when a correct sudo password is provided.

If you prefer not to use the GUI, you could also edit your projects .user file and add some xml similar to the following:

While writing test coverage for a recent javascript project I was required to test every possible call to a function with a large number of binary parameters. Here is the function I came up with to generate the combinations:

It’s pretty straight forward, pass in the length (n) of binary digits and it will return an array of all possible combinations as boolean values.

The base GO libraries provide a handy function for marshaling structs into JSON. I recently came across an issue when writing webservices in GO for an existing javascript client. The problem was the client expected the JSON data to have field names starting with lowercase letters. GO’s naming convention is obviously going to make all struct fields uppercase by default as they need to be exported. I ended up copying the JSON marshaler code from GO’s library and modifiying it with a new paramater that will lowercase JSON field names when set. Because GO isn’t on github and I am strapped for time I just copied the code into a new util namespace and made the modifications as a couple of other gophers in #go-nuts were interested in using it.

Usage is simple, when the 2nd paramater is set to true, all fieldnames will start with a lowercase letter(other capitalization remains unchanged):

Recently while working on a Go based project I needed to use some functionality from another large C++ library. The library’s size and complexity made re-writing it in Go unfeasible. After some research I decided to use the popular SWIG (Simplified Wrapper and Interface Generator) framework to enable interop between my two projects.

The following is a brief tutorial to guide you through wrapping a C++ class with a Go package which can be called from any Go program.

At first glance qt creator seems to be a feature rich ide however it still lacks many basic features such as being able to add sub folders for code to projects through the project explorer window. The following tutorial demonstrates how to get around the ide limitation by adding the folders manually.

The goal is to achive the below folder strucutre, project being the root, and “ModuleA” being the folder we need to add.

1234567

--Project
--/ModuleA/
----a.cpp
----a.h
----b.cpp
----b.h
--main.cpp

Start by navigating to the projects root directory, and create a new folder:

1

mkdir ModuleA

Create a new file inside named “modulea.pri”. If you are going to copy existing files into this new folder you need to add them to the pri file. (Adding new files can be done through the qt creator gui once the project is configured properly)

Now we need to include the new .pri file for the folder in the project configuration. Open up the .pro file located in the project root directory and add the following line:

title:Project.pro

1

include(ModuleA/ModuleA.pri)

Reload the project, the new ModuleA folder should now be visible as a subfolder in your main project. Right clicking on it will give you the option to add new files. Simply repeat the above process for each new sub folder you wish to add.

I am a frequent user of the popular jQuery plugin jqGrid. It comes with a large feature set for viewing and manipulating tabluar data in the browser. When I am working on an asp.net mvc projects I work with it via Robin van der Knaap’s lightweight html helper library: https://github.com/robinvanderknaap/MvcJqGrid . It has strongly typed html helpers and a handy model binder for handling async grid functions.

After primarily working in C# for the last 5 years I recently switched back to c++ for a large project. I am going to use this article to post notes/gotcha’s/tips as a I come across them. I already have a few to add, and will update as I go along. Hopefully they help some c# programmers out there. They are listed in no particular order.

C++ does not support template type constraints/guards on template definitions

Gotcha: C++ does not support template type constraints/guards on template definitions. I ended up using static asserts (requires c++ 11), the example shown below uses helper methods from the popular boost library.

Member access specified at group level

C++ supports multiple inheiritance

Unlike C#, C++ supports multiple inheiritance, while this may seem like a benifit it should be used very very rarely. If you find your object model requires multiple inheiritance it is best to reevaluate your design or you will end up facing issues like the diamond of death.

Rule of 3

Remember the rule of 3.
If you require any of the following then you should explicitly define all three.

copy constructor

copy assingment operator

destructor

Temproraries with Parameterless Constructors

Don’t include parathesis on paramaterless constructors the compiler will favour resolving it as a function declaration and you will end up with an error message like:

STL containers copy their values.

When adding items to STL containters remember the container does a copy and keeps track of it’s own copy of the item being added.

Follow the Virtual Constructor Idiom

When creating abstract base classes follow the virtual constructor idiom
and create virtual clone and create methods.
This allows you to create collections of base types while still being able to copy them without knowing their concrete type.

Dynamic Linking Libraries

Here are some tips and tools for working with dynamically linked libraries and shared objects.

Shared library files to be registered must start with “lib”

Use ldconfig -v to see registered libraries

Use ldconfig to reload linked libraries

Export ldpath to temporarily point executable at your lib, handy for scripting.

Use ldd command to view an executables dependencies.

Use nm –demangle to view shared objects exported symbol list.

Use c++filt to lookup runtime symbol lookup errors.

Can’t call virtual methods from constructor

Virtual methods can’t be called from within constructors. This usually means you have to use a two stage approach to initialization of derived classes.

Avoid importing namespaces

It is bad practice to use “using namespace ;”. When including third party libraries there can be naming conflicts which become a pain to track down in a large project.

By default qt creator will add some message pump related code to your main.cpp file, this can be commented out if you are just writing a console application (leaving it unmodified may prevent you from seeing output in the xterm console window)

If you are using ubuntu you will probably have to configure the x-term environment settings.

When running your first opencv project with CDE you may experience the “cannot connect to x server” error when calling code that requires the opencv imageview or UI elements.

To fix simply set DISPLAY in your projects environment variables, make sure a file in your project is selected then goto run-> run configurations -> environment and add a display variable like the following:

The blogging engine I am using dosen’t have a “following” function so I am using this post to list the blogs I follow, I recommend them when people ask me where I learn various development related topics:

The following is a brief tutorial on getting the opencv library setup with eclipse CDE on Ubuntu 12. The same eclipse project settings detailed in this tutorial can be re-used to build applicaions on top of opencv.

Now we need to build a makefile with cmake if you are just messing around and aren’t sure which modules to install you can run the following command which will include the most common ones including python bindings. Otherwise you can skip to 4a to select which options you want.

We will use the cmake gui to select which components to include, start by installing it:

12

sudo apt-get cmake-qt-gui
sudo cmake-gui

A user interface dialog should appear. Select the directory you extraced the opencv source files to for the source directory. Create a new build direcory and select that in the gui as the build destination. Then click the configure button to the lower left, select “unix make files” for a generator, and “use native compilers” option.

The configuration process should populate the gui dialog with the available components, tick off desired ones, hover over the right column with the mouse cursor to see a more detailed description.

Click generate, cmake should populate the build directory with the necessary make files. Check the output window to ensure there were no errors.

Navigate to your build directory in a terminal and make. Then install.

123

cd release
make
sudo make install

To configure the dynamic linker we need to add a line to the end of ld.so.conf.d. The following command will open the file in a text editor (the file may be blank, that is fine).

If you don’t already have eclipse with cde installed do that first with the following command

1

sudo apt-get install eclipse eclipse-cdt g++

Open eclipse and create a new empty c++ project using the linux GCC toolchain

Next we need to add some code to the project so we have something to build/run. I am goint to take the bag of words
example out of the /samples/cpp/ directory of the opencv project. Simple copy/paste the file into your new project.

Next we need to tell eclipse what libraries to include with the project and where to find them. Goto project -> properties on the file menu or just right click on the project in the projects pane and click on properties. A dialog should appear, click on C/C++ Build -> settings then click on includes.
Add the following to the include paths list Noted: if you need to compile projects with the older c++ api, you would replace opencv2 with opencv

1

/usr/local/include/opencv2

Next goto the GC++ Linker tab and add the following to the library search paths list:

for other projects you can remove uneceoctopress spell checkingssary libs or add dependent other dependant ones.

These project settings should now allow you to compile projects referencing opencv. Goto project -> build the project should build. Then run it, if you are using the bag of words example from the samples folder you should see console output similar to the following: