Category Archives: Software

I can’t help but believe the Microsoft CEO showed his true colors before the backtrack. I just don’t think enough guys care to listen. The “trust the system” comment he made is the worst cause it’s the system that’s letting women down.

Something big is amiss and the Microsoft CEO’s original statement betrays it. I mean, I’ve been doing this work for over 10 years and I’m crazy good at software engineering. Today, I want out of the field more than ever because of the treatment I’ve been taught to expect in the workplace. But! Student loans…

Sure, there are a some good advocates for women in a tech workplace. But there are way more than enough of those who prefer to tell us how to feel and prefer to tell us how to “fix” our lives rather than engage with us to see what we’re talking about.

To begin, I just couldn’t resist using an a Atari harkening font in this one. 🙂

This particular project evolved my GLSL skills and started to teach my brain how to work on a GPU better which, naturally, means I have a very nicely decorated whiteboard to my right at the moment. And I’ll be sharing it too, in a moment…

I’ve been thinking about doing a Computer Vision project for a little while now. I was inclined to do a desktop app and use OpenCV but on doing some research on what goes in to computer vision I realized, “Hey, I know the tools to make this work in browser.” So, I did.

My first task to get Computer Vision working was take the RGB feed from my WebCam and convert it to HSV. I’ve done chroma keys before just using the raw RGB values but with the recommendation that HSV might work better, I figured why not give it a try? This is where the GLSL came in and what I had to write was an implementation of the following:

So my final result for Hue is actually $$H”=H+H’$$ This was because in the HSV colorspace, $$H$$ is represented as a circle going round from \(R \rightarrow G \rightarrow B \rightarrow R\) So, in order to get select the right hue, an offset will need to be applied to get the value round to the right place.

$$S=\frac{\Delta}{max(R, G, B) + Epsilon}$$

And finally

$$V=max(R, G, B)$$

I found some other documentation about this calculation online and found it messy. That might be normal when discussing math like, I don’t really know, this but my engineering brain likes things broken down into their individual components. Hopefully, the above will help someone else out! 🙂 Now to calculate that on the GPU making it time to live up to my promise of bringing the whiteboard in…

I ended up being heavy on mix() and step() in order to reduce the branching I had to write. Branching, as I understand, is getting to be less of a concern in GPU programs but the article saying that branching implementation was improving was from 2011 and that’s too recent to assume all GPU’s will behave nicely. Below those function notes, there’s a tracking of how my values were going to flow through in the vectors so I could end up with a the maximum in the first position, the two values I needed to subtract to get my $$H$$ and finally the offset $$H’$$ that went with the selected $$H$$ so I could calculate $$H”$$ properly. For the curious, the shuffling that’s done with mix() and step() was effectively a vectorized ternary operator.

The conversation of RGB to HSV and writing the GLSL shader to do that was most of the effort. But the shader wasn’t quite done yet. The final step was passing Min and Max values for each value: H, S, V. When all three where within the defined min and max, the shader needed to return white, when any of them were out of bounds, the shader had to return black. This way, the user can select an object in the real world by its color that will function as a paddle. Now I did look for and find shader implementations of RGB to HSV online, however, I opted to roll my own to make sure I understood what was going on as this was pretty much the beating heart of this project.

On the JavaScript side of things I did run into a problem. I wanted to use sliders so users could hone in on the color they wanted to select. However, HTML5 only allocates one thumb per slider. I know I could have used jQueryUI for to get two thumbs but I opted to stay away and handle the problem myself. It was also tempting to use three.js to handle the WebGL side of things but I decided to not bring it in either. The changes weren’t too significant from my ANSI.WebGL project so I saw no need to take on another dependency.

The JavaScript I did write is pretty straight forward. There’s a ball (a div that’s been styled to look like aball) that flys around and when it hits a paddle it flies back the other way. Hitting here has been defined as the left or right side of the ball hits a white paddle pixel. When that happens, the column of pixels is counted for connected white pixels to figure out the length of the paddle. If it’s above a threshold (provided to filter out artifacts) the calculation figures out where the ball was hit in relation to the paddle then the ball will bounce away at an angle. The only “gotcha” I ran into with this part was when I read the pixels from the buffer, I found them inverted which broke my hit detection! But it was a simple fix to right them thankfully.

For much larger computer vision problems I probably would turn to something like OpenCV. Still getting to understand something like sometimes, we convert to the HSV color space, is I think very important! Understanding how the tools you are using work I hold to be key to actually using them well. It also allows you to answer a question I’ve mentioned before as being part of my process. 😉

You will need either a recent version of Firefox or Chrome in order to run this project due to the needs of getUserMedia() and WebGL.

The problem I set out for myself of rendering ANSI art in a web browser turned out to have more ins and outs than I anticipated and quickly became a case where I needed to iterate and refine my idea in order to carry it out. In the beginning, I knew doing something like rendering a few thousand <span>s, one for each character so each character could be styled, was not going to be a viable solution because it would not scale to work well on all browsers. So I attempted to go with SVG as my solution and use a mask in order to get all the colors I needed, kinda like using an image brush, as this would afford me the possibility of limiting the number of elements I needed to create. After I got the right @font-face setup, the solution worked great in Opera (Blink), but not so hot in Firefox.

At that point I considered dropping the whole project as I found myself being pushed to a HTML5 canvas based renderer for speed and I didn’t really want to reinvent the wheel as I found an implementation online that solved the problem that way already. Then, I lost my job. 🙁

But! Ok! 🙂 New reasons to putter: Keep my skills fresh, solve an interesting problem, learn something new and have a new project to show off to would be employers!

On project resumption, I opted to try and go the WebGL route. One thing I like about this solution is I only needed one sprite of all the characters to render any color combination I wanted as I’m able to draw each character with a fragment shader. When it reads black for the current texture pixel, put the background color, white put the foreground. Simple rule with a lot of flexibility.

The project was also a great opportunity to play with newer features to JavaScript that I haven’t been able to use due to the need to support legacy browsers. In addition, it features my ability organize which represents one of my two greatest strengths as a software engineer. The other being I do this job primarily because I have a passion for it. 🙂

Titling this post was hard. I wanted something attention getting but yet not hyperbolic. Hopefully what I’ve selected works. I do think it’s a pretty no nonsense way to sum up what I’m about to go into here because Computer Security boils down to answering the question of, “Secure against whom?” Well, I think I should have been driven to a point where I gave up and started to work on something else instead of being here now, able to write this post with the findings I have.

What got this started was when I took Introduction to Computer Security in Grad School my professor’s stated goal was to make the students in the class think like an attacker. Today, I think the “attacker” mentality is a good one for all software engineers to have. It gets us thinking off the happy path, in the weeds and muck, perhaps finding a path to somewhere that you as the author didn’t even know existed.

In this post, that path metaphor may be too literal.

After I was done with the class I remember thinking to myself, “Gee, I know how to do some Internet Explorer automation, I wonder if I can use IE to get my physical location on Earth?” So, I spent some time figuring out how to do just that and those results results are here. I was going to make this post a retelling of that find but I got to thinking, “You know Paige? Over the years a lot of people have picked on Internet Explorer. Don’t cha think there’s enough fodder out there for that?” Well, I happened to agree with me so I thought, what else could I go after? Well, what about the service that makes HTML5’s Geolocation work itself?

Using the HTML5 Geolocation service via the familiar way is done in browser, through JavaScript, with code that has these basic parts:

This is pretty much what my Internet Explorer solution relied on. Back then, I found that my IE instance was “hung” and that turned out to be because there was a, “Do you want to allow [X] website access to your location?” prompt up. I didn’t want to look at a flashing IE window as automation ran so I added some registry keys to grant permanent permission to IE for it to be able to use Geolocaiton on a website I browsed to, automatically. When I was done I realized I’d in effect created something that could get my location on Earth without my permission.

Now, back to the task at hand.

What happens at the lower levels when you call getCurrentPosition in JavaScript? Well, in my case where the only wireless access I have is WiFi, all the wireless access points are scanned for their MAC addresses and signal strength because HTML5 Geolocation works by triangulating your device with the Access Points that it can see.

A lot of code is about to follow but all this C++/CLI code is doing is collecting the aforementioned information.

Regarding the elided bits, I’m not going to reveal what I did to fill them in cause that gets into information I’d rather you discover on your own. A little bit of critical thinking is all that’s needed to find it just like only a little bit of critical thinking was needed to carry this out end to end. Anyway and as for WiFiAccessPoint and Request, they are simply:

When I ran my project, my position was returned to me with an error of 33 feet. I was given no prompt and if I hid my console window I would have had no indicator that my process was running unless I was staring at process monitor.

The only thing that prevents an attacker from developing software that gets your device location is a requirement from the W3C (The group that dictates how HTML5 Geolocation shall work) that mandates apps ask for permission. If it would always have to ask instead of leaving it to merely trusting developers, or if it took someone with more knowledge than I to pull this off I wouldn’t be bothered. As much. I like Geolocation for navigation, but I also like being asked my permission to opt apps in to using the Geolocation services on my devices. Google Maps, ok. Others, it depends.

You know, something that’ll make you appreciate .NET really fast is programming COM by hand in C++. I suspect what follows can be converted into C# but I didn’t go that route because while working to get to this point, I didn’t want to worry about doing COM Interop in .NET on top of the problem I was trying to solve. Therefore, I ended up in C++ and COM.

For the SVG to bitmap problem I was originally trying to solve, I started out by going using the canvg JavaScript library. Definitely a recommended way to go if you’re trying to solve this problem and the output works for you. Personally, the Graphic Designer side of me looked at my nice pretty anti-aliased browser handled SVG renderings, then at my jaggy HTML Canvas rendering and went, “Aw.” I also didn’t like being limited because SVG has those spiffy diffy filters and I wanted to be able to use them too. So I figured I was going to have to do the rendering server side.

On writing that last, now seems like a good time to remind folks, sanitize/be mindful of your inputs! 🙂

I knew I could get another library to handle the rendering but I didn’t see why I needed to. I have a perfectly good SVG renderer on my computer after all. It’s just embedded in Internet Explorer and much of IE is COM. That means I can get at its objects too, if one knows how.

I’m going to assume that most of the people who read this are more comfortable with C#. I’m also going to assume that because you’ve made it this far after I’ve talked about C++ and COM, odds are you’re my kind of developer! 🙂

Finally, before the good stuff a note that as is my motif I’m not going to be giving ya’ll a final working solution here but I will give you the knowledge to make it work for you. I’m a teach a developer to code kind of gal after all!

I like to start with the beginning so, here’s your includes because it drives me batty when people leave those out of their code walk throughs!

I’ll assume knowledge of the basic #include, the #import directive is going to pull in type description information for the COM interfaces I’ll be using. The #pragma comment() is a way to add a reference to a type library in code instead of adding the reference to the project. Great way to avoid, “Gagh! I setup my libraries for Debug/Release mode only!” Finally, HIMETRIC_INCH is a constant needed to get the output to render at the right size. We’ll see that again later.

First thing to note is the GdiplusStartup call. That’s getting our application prepared to do the final rendering using GDI+. You wanna use GDI+ in your application, you gotta do this. As cleanup, when you’re done you need to call GdiplusShutdown as shown above.

Next is the actual initialization of COM with CoInitalizeEx. This is responsible for setting up COM for use by the calling thread. I’m also calling it with COINIT_APARTMENTTHREADED so I don’t have to worry about concurrency.

I have some constants here (width, height) that just happen to match my SVG output. Bad, icky, terrible thing to do but I’m here to illustrate as simply as possible.

The most interesting thing happening here is the call to CoCreateInstance. Here I’m creating an HTML Document and note, I am bypassing IE entirely. Also note the CLSCTX_INPROC_SERVER parameter. That’s telling COM To create the document in the space of the running process. This is one of the very extremely important things for our goal here.

The way that I’m checking if my HRESULTS succeed with SUCCEEDED(hr) is a result of my absolute abhorrence to using goto statements and desire to avoid arrow code. Also, I’m using smart COM pointers here. Usually I’d have to be aware of:

IUnknown->AddRef()

IUnknown->Release()

In order to avoid a memory leak. If you don’t know what I’m talking about here, that’s okay for the moment just if you find yourself working with raw COM you will need to know about it, else you’re going to have a bad time. By that I mean you’ll have a memory leak.

Next, we have to set the size of our document which SetDocumentViewPort handles.

In completing the task at hand, this was one of the trickiest things to figure out starting with what interface to use and setting it to size in what units? The MulDiv lines are handing the unit conversion from pixels to “device units”. Then, via a IOleObject, SetExtent is called which tells the object, in our case our HTMLDocument its size.

Next for you JavaScript developers we’re actually going to get into a little familiar territory.

Need to do a little manual memory management here around the SafeArrays. But another one of those very extremely important things to note is the X-UA-Compatible meta tag. This’ll be familiar and I’ve got mine set to IE=11 for no good reason beyond it’s not IE 7. It turns out, when you embed IE objects like this it will default to IE 7 for rendering. Therefore, to be able get the SVG rendering goodness we’re after we need to tell IE to please be a current version of the browser.

The body tag of the HTML is elided but it’ll also necessary to set your CSS margins and padding on your body element to be 0px.

Next see those calls to write() and close()? Those are in fact the same write() and close() you know of from JavaScript! They end up being a handy way to create the document we want to render.

From there I’ve got a check on the current documentMode to verify we’re not using IE 7. The documentMode property however, is not implemented on the IHTMLDocument2Ptr I’ve got currently so I needed to get an IHTMLDocument6Ptr. From there I pump Windows messages to get the document into the complete state. Sleeping here will get you nowhere in the COM Context that we’re using.

And we’ve got our image! What this does is gets an IViewObject which handles rendering to a Device Context Handle, in this case our GDI+ graphics object context. Note, here is why we told COM we wanted to create objects in the current processes’ memory space too. Had we not done that, we wouldn’t have an image coming. But we did and do so we’ll render out with the Save() call.

We could actually render out a whole lot more than just SVG here, I’m talking full webpages with no changes to the code. This has also been only one demonstration of what you can do with IE automation. I’m thinking of revisiting another soon. But for now, a demo project for ya is available here. Enjoy!

A colleague of mine told me about currying sometime ago and I’ve been intrigued by the idea ever since. After playing around a little bit with F# and coming to understand the concept better, I thought I’d take a crack at making it work in C# because it seemed like a fun technical challenge to take on.

To begin, currying is taking a function with multiple arguments like:

string Foo(string str, object obj, int num)

And transforming it so the function call be called as a chain of function where the single parameter to that function correlates to a parameter in the original function call.

Foo(str)(obj)(num)

Currying is apparently a common thing in functional programming languages, like F# is.

So I got to thinking that this is interesting, but I wanted to be able to use the concept on an arbitrary function in C#. I didn’t want to create a bunch of Func<> templates by hand either. I just wanted it to work with any method that takes multiple parameters.

Now, I’ve chosen to start with focus here and so I didn’t bother with instance methods yet. Not going to be hard to add, I just wanted as clean a path to solving the problem at hand: the transformation itself.

First, I started to think about the individual method signatures. In the case of Foo, they would be:

What I needed to do then, was generate these types, define a way to capture the arguments passed in by the caller, and generate a function. The class I’ve come up with thus far to do that is as follows:

It takes a MethodInfo object so the user can choose whatever (currently static) method they want to use. Will need to be static and public right now. So what’s happening here is Make() gets all the parameters from the MethodInfo passed in then creates the Func<>s in the way discussed above. In addition, it creates the parameter name expressions that will allow the final Lambda to capture and use the parameter values the user passes in. Note, the parameter names equal the supplied MethodInfo’s for clarity. Then finally, starting from the Method call, it builds out the curried method structure before returning a lambda.

The first way that I came up to use this with was via a dynamic method call as such.

Which achieved my goals, but I’m not a fan of dynamic. I’m in a strongly typed language here so I’d like to take advantage of that. The best solution I’ve come up with to achieve that goal would be to build some kind of utility that integrates into Visual Studio to build the Curried Calls I want. Building the plugin is outside the scope of this document, but, I can render the C#:

public static string CSharp(MethodInfo methodInfo, string name)
{
var lambda = Make(methodInfo);
var builder = new StringBuilder();
builder.AppendFormat("public static class {0}\r\n", name);
builder.Append("{\r\n");
//PrettyPrint is an extension method to render the type in C#
builder.AppendFormat("\tpublic static {0} Exec()\r\n",
lambda.Type.PrettyPrint());
builder.Append("\t{\r\n");
//Actually, the lamda.ToString() needs to be fixed in order to deal with
//namespaces properly but I left that out as it's not germane to the
//theory I'm trying to work with.
builder.AppendFormat("\t\treturn {0};\r\n", lambda.ToString());
builder.Append("\t}\r\n");
builder.Append("}\r\n");
return builder.ToString();
}

Which I put in my Curry static class and will generate a static class like this for me…

The following a is a pretend re-enactment of how the following webpage, I think, came to exist as is.

“Users are asking for the ability to have different addresses on their account. One to actually send credit cards to, one for statements and one for general correspondence. We have to enable that facility.” says Boss Man.

“Ok, we’ll put three address fields on the contact information section part of the webpage.” says Software Engineer.

“No, not enough people are asking for it that I think we should show this to everyone. We’ll only put one box up and give users a link to another page in order to manage the all three.” says Boss Man. “The contact information page will continue to show just one address. Say, the one for statements.”

“But what if there is a mistake? What if the addresses get incorrectly updated by a bug or human error?” Software Engineer protests, “How will the users find out?”

“They’ll be able to tell when they click the link. Like I said, all three addresses will appear then.” Boss Man says. “We’ll also change the title of the surrounding box from ‘Mailing Address’ to ‘Statement Mailing Address’ to let them know that there is more than one address. They’ll be fine!”

“But!” Software Engineer continues to protest.

“They’ll be fine.” Boss Man reassures.

And scene!

That, is what I thought of when I realized why my new credit card has been MIA for the last month. I didn’t get mad at the company, I didn’t get mad at the very, very nice and sweet customer service representative I ended up talking to. I didn’t even get mad at Boss Man. I felt for the Software Engineer that had to implement the website that let me go unawares that I had incorrect addresses in the system. Had I not been told an earlier attempt to send me my card had been thwarted by an “address unknown” error on the part of the Post Office, I don’t know that I would have noticed the link indicating that there were other address possibilities. I never entered multiple addresses before. I made just one update when I moved last. However, after I made said update, I had no visibility regarding the current system status and the newly mismatched addresses. Thus, no new card for me has arrived thus far.

Scenarios like these are why I care so very much about usability. Beyond Jakob Nielsen’s usability heuristics, I’m mindful of the “Pit of Success” as Jeff Atwood Defines it:

The Pit of Success: in stark contrast to a summit, a peak, or a journey across a desert to find victory through many trials and surprises, we want our customers to simply fall into winning practices by using our platform and frameworks. To the extent that we make it easy to get into trouble we fail.

Only modified to apply to usability, instead of development.

I used to be the type of designer that thought my interfaces were “Da Bomb Yo!” and users would just get them. If they didn’t that wasn’t my problem, my UI was great! Then I watched someone very close to me, who I take to be computer savvy, struggle with a UI I designed. That was the moment my ego regarding design got put out to pasture and I’m better for it. Now I listen to user feedback as valuable input into the design process. 🙂