My thoughts on tech, stuff, and nonsense.

Author Archives: joemonti

It has begun… The FAA will now be requiring registration of UAV (Unmanned Aerial Vehicles) such as quadcopters starting December 22nd. It will cost $5, and be valid for 3 years. And it will apply to hobby vehicles weighing between 250 (0.5lbs) grams and 25kg (55lbs). You will be able to register online or by paper.

In the world of cloud computing, Agile development, and the race to do more with less, the role and use of DevOps has grown exponentially. At this point it is practically a buzz word; a point of bragging rights that you don’t need developer-only developers or operations-only operations people. The article How DevOps is Killing the Developer by Jeff Knupp does a great job detailing the background, use, and drawbacks of DevOps. This post is an extension to that.

In a constrained workplace, especially in that of a startup or any small team with big goals, DevOps can be a necessity. In that environment, Developers aren’t the only ones taking on additional roles either. Everyone in the company has to step up to fill additional roles.

The more important part, and the reason for my post, is that DevOps shouldn’t be over-used when the resources are available. Developers don’t work at their best, most productive, and most innovative when they are also worrying about QAing and deploying code, maintaining servers and dealing with production issues.

As anyone who knows how I work (I would hope), I’m not afraid to ssh into production systems and update IPTables or restart Tomcat. And I enjoy being able to do that. It’s a good change of pace and it is good to experience how your code is functioning and is being used in the real world. Staying too detached from production can insulate you from issues or inefficiencies in your code. There is no better way to get a bug fixed than to put a developer on the Nagios alarms for it.

As a Developer, having a wide range of skills along the development stack is a tremendous asset. But it is an asset that should be used sparingly, as it takes developers away from their most skilled and enjoyed roles. Used effectively, however, the DevOps role can be greatly beneficial to a company, whether it is out of necessity or to mix things up.

There is an additional potential use case for a full-time DevOps role when there are also developer-only developers and operations-only operations people. In this case I think it makes the most sense as an individual or team that is part of the Operations team but focused on development of tools and systems for use by or in support of the Operations team. This could be monitoring, maintenance, or deployment management tools. It is likely not software specifically for primary business objectives, but rather in support of operations and management of the platform.

Drogon’s not dead! While it’s been over a year since my last post on the topic, I have been working away. Although I went most of the year without touching it, the pace has picked back up over the last few months and I’ve worked through a number of changes and improvements.

The only hardware change was swapping out the frame for a new Q450 V3 Fiberglass Quadcopter Frame (450mm) from Hobby King. It’s a bit smaller but lighter. So far I really like it. All the electronics are the same.

First, I had attempted to ditch the transmitter/receiver a bit too early in development, relying completely on wifi and the Raspberry Pi. This required a lot of support software that took away from the more important, and fun, flight control software. It ended up being too much of a distraction, so back on the receiver went. I re-integrated the receiver code and back on track I was. On the plus side I now have streaming video to an Android app from the Raspberry Pi’s camera. I also swapped out the Java code on the Pi to a new Python app (It’s all in my Github repos). All it’s really doing now is capturing logs from the Arduino and providing video streaming.

Next, I developed a self-tuning addition to the PID algorithm. This should hopefully be saving time and producing a better tuned PI. It still requires manual tuning to get it started, which I’m working through now, but progress is being made. I also re-calibrated the accelerometer-gyroscope relationship and how the data is pulled into a single position, so I am getting much better and cleaner data. I also incorporated rotational correction. This uses the Gyroscopes Z-axis by increasing opposite motors and decreasing the other opposite motors to control and take advantage of the motor/propeller torque.

The last major area was in test rigs. I’ve gone through a few iterations of test rigs. The first test rig I used, which is in my first videos, was tethering to the floor. This had several issues in not giving the quadcopter enough freedom of motion and significantly impacted flight. The second test rig I used was hanging from the ceiling. This was much better at providing more range of motion, but still had issues in giving it too much freedom while still having too much effect on flight. My current test rig is now a balance beam. It is basically two 2x4s perpendicular to each other, with one raised slightly. This works well with my current frame. When I get the PID tuning working well, I may graduate to the hanging rig (hanging, with pulleys to small weight to pull slack).

I came across an interesting post on Deep Learning / Deep Neural Networks which was related to my previous post. The interesting point about that post, which I had in the back of my mind while writing my previous post about neural networks, was that Deep Neural Networks aren’t designed for replicating the full extent of Human intelligence. What they are getting really good at is replicating only a part of what the Human brain does, but a part which is important and incredibly useful and effective for specific engineering problems such as classification.

One specific thing that Deep Neural Networks do well is the task of classification at very large scale. The Human brain is pretty slow and even though it packs in an enormous amount of neurons that work in parallel, way more than any computer or cluster of computers is able to model, it needs a lot of neurons to classify one image, and can only classify a small number of images at once. Deep Neural Networks can sift through millions or more images per day without getting tired or having to sleep and the process can be cheaply replicated across tens or more thousands of computers. They do need a steady diet of electricity, but that is in relative abundance (though limited and has it’s own set of issues).

Deep Neural Networks aren’t going to give us Strong AI anytime soon, but they are beginning to be able to perform certain tasks faster and more accurately than humans.

I came across an article about a paper which details the discovery of an interesting property of feature encoding in a Neural Network and an inherent flaw in the discontinuity of feature learning. The flaw has to do with the finding that specific new classification examples are universally incorrectly classified by (classical and deep) Neural Networks. The classification errors follow a specific blind spot in how individual neurons or set of neurons are able to be trained. As an example, certain images that a human would instantly correctly positively classify, a Neural Network would incorrectly negatively classify. And certain images that a human would correctly negatively classify, a Neural Network would incorrectly positively classify. This flaw permeated many different types, configurations, and differently trained Neural Networks.

The flaw is not necessarily a flaw in how Neural Networks (in software) are implemented, and it is not necessarily the case that the human brain does not suffer the same flaw. But rather, it is a limitation in modelling a specific operation of the neurons of the human brain and how that singular operation is abstracted and taken advantage of in machine learning problems. Since there is no reason yet to believe that the same flaw does not exist in how neurons in the human brain are able to model and classify data, it must be presumed that the flaw of software Neural Networks is that they are limited to a small subset of how the human brain uses neurons.

There are two primary areas in which (classical and deep) Neural Networks ignore critical human brain functions which cause them to be limited in their ability to be as flexible and robust at performing intelligent tasks, even that as mundane as classification. These areas are time and prediction. Yes, a critical function of the brain is to both learn and classify information. But it does so in the context of time and with the aid of prediction; without either of which the brain cannot function. You do not experience life in frames as a computer vision system typically does. Even when you stare as still as possible at an image, you cannot identify it without, nearly imperceptibly to you, your eyes moving across the image (called a saccade). Granted, your eye can only physically see in clear detail a small area in the center of your vision, but the movement and time dimensions are important none the less. Just like how you move your fingers over a surface to identify it through touch, your eyes function in much the same way, by creating a constantly changing temporal pattern. Computer vision systems do not (typically) do this.

Prediction too is critical at filling in the messy and incomplete stream of information you experience. You may not realize it, but you have a blind spot in each eye where your optic nerve exits the eye. It is possible to detect it, in a way similar to how we can detect a black hole, but for your normal behavior and experience it’s imperceptible. Through the use of time-changing and predictive input processing, however, you experience a continuous and consistent reality. Prediction fills in the gaps from the missing input of your eyes’ blind spots and other missing or variant features. Markov models can form predictions, but most Neural Networks ignore or don’t fully address or take advantage of prediction.

Time and prediction, I believe, allows us to completely overcome adversarial examples or neural blind spots like those described in the above mentioned paper. By adding a time dimension, it not only allows a time component to be a constraint in a model, thus treating the same order of a pattern differently when important distinctions are made with respect to the time between various parts of the pattern. But the time dimension also spreads the data over a wider swath, and not necessarily that which resembles what is captured by a single computer image. By manipulating and absorbing a still image we are able to overcome major deficits in the quality or other deficiencies in an image. Computer vision systems are starting to take the queues of how the human eye and its saccades process images, but it’s just beginning to become common practice. Prediction too lets us fill in gaps, correct for small errors or discrepancies, and know what to expect and do next, particularly critical in real-time systems.

The human brain is a highly complex, highly structured, dynamical system. Neural Networks do a good job at emulating the modeling characteristic of the human brain to perform classification tasks. But thanks to books like On Intelligence by Jeff Hawkins and How to Create a Mind by Ray Kurzweil, among others, I have come to realize, at a high level, the depth and breadth of how the brain implements intelligence. The trivial wiring of standard Neural Networks barely begins to explore its capability. Deep Neural Networks take it a step further, but there are so many more ways in which it can be taken further.

There is a flaw in LinkedIn‘s skill endorsement system, at least in so far as how mine has evolved. The result of which is that my endorsements do not match my actual relative levels of expertise. The key issue with my LinkedIn profile is the disproportionately high endorsements for MySQL, more than twice that of my next highest skill, PostgreSQL. The reality is I know much more about PostgreSQL than MySQL and anyone I have worked with I hope would know that. For some reason this has bothered me, so here is my explanation as to why I think it is happening and how I think it could be fixed.

My sample size of endorsements is really small so it’s not very statistically significant on its own, but I see a pattern emerging that is likely the result of flawed logic in the way LinkedIn promotes making endorsements. When you view someones profile on LinkedIn it may show you a box at the top of that user’s profile with a few of their skills, asking you to endorse them. How does LinkedIn come up with that short list from the possibly large list of available skills?

I assume as a user gains endorsements, those for which they have been endorsed will be more likely to be presented to new users to endorse. But what happens when a user has no or few endorsements? I can’t say for sure but I can bet they are using potentially many indicators from the ecosystem of LinkedIn’s available data such as number of users having that skill or total endorsements for that skill. They may also be using factors such as that skill’s relevance (i.e. occurrences) in job listings or job searches. The idea being that when having no user-specific indicators for which skills are important for that user, skills considered more important or relevant across the community at large or those most likely to lead to targetable skill sets for job searches should be favored for promotion for endorsements. The problem with that assumption, however, is that what the crowd is good at or what most recruiters are looking for isn’t necessarily what you are good at. If endorsements are meant as a way for recruiters or others to gauge your level of expertise or proficiency in a skill, then LinkedIn’s logic for promoting endorsements is flawed.

What I think would be more valuable is for users to first rate their own expertise in their skills, then let other users endorse those skills. When promoting other users to endorse them, the short list of skills to present should be based on a combination of your own rating, the endorsements from others, as well as indicators from the LinkedIn ecosystem as to what skills are important to the community. If others organically endorse skills in a disproportionate way to how they were rated by the user, that is ok and potentially interesting information. At least a user has some influence on how that process evolves.

Came across a project, the LIDAR-Lite by PulsedLight on the crowdfunding site Dragon Innovation (not to be confused with Drogon, the quadcopter). It is an optical distance measurement device for accurate high performance sensing with many advantages over IR or sonar range sensors. Looks pretty impressive and perfect for quadcopters among many other applications.

I decided to create a Wiki for the Drogon Quadcopter project to have something that is better suited for that type of content. Blog posts on the topic will stay at joemonti.org, but all information, documentation, etc will be at drogonquad.com. Just in case you forgot:

I started working on a new little robot called Gremlin. It is based on a Parallax Boe-Bot base with an Erector set and plexiglass frame. For electronics it has a Raspberry Pi w/ WiFi adapter, 16-channel I²C Servo controller and the Raspberry Pi Camera Module. Power comes from 10 AA batteries (4 for servo controller and 6 for Raspberry Pi), but I will likely upgrade to LiPos. Here’s a short video showing the Android control app (sorry for the crummy production quality):

It’s currently missing the 6 AA batteries for the Raspberry Pi (I’m waiting on a few parts), so the tether is just there to power the Pi.

The Android app is a little something I wrote which connects to the robot over the network (WiFi). It has a live video stream and virtual joystick controls. Once I get the whole mobile power assembly hooked up I’ll be able to use it for telepresence.

With my primary robotics project, the Drogon Quadcopter, grounded for the winter, I’ve started Gremlin to keep some of the work going The goal is really to have a smaller, easier to work with mobile robot for building a general purpose robotics software platform for Drogon and any other robotic projects I pursue. Also I’d like to use it as a test-bed for building learning algorithms and working with the camera, also applicable to Drogon.