Browsed byTag: gbrewster

Well, as promised from my previous post, I am here to unveil the new module that will CHANGE THE WAY WE INSTALL FIBER CUSTOMERS (not really, but I like to think so).

So naturally, after a fiber project is completely built out and operational, the time and effort needed to build a system is finally found and put to good use. During our initial build out, we hired contractors to assist in the deployment of ONT’s for our new fiber customers. Being contractors, they would show up late for work, leave early, put in poor effort and often just caused a lot of issues. This issue resolved itself once we discontinued our relationship with the contracting company, and took some of their employees (the good ones). So given that we no longer have an issue with getting ONT’s installed in people’s houses, this module has lost some of it luster. But hey, I am a college student getting paid to write code right now, so I gotta find something productive to do!

In reality, this module will actually solve some other issues that we currently are facing, but able to manage.

Purpose

The field tech console is web interface that allows technicians, installers and contractors, manage and track their appointments. In the past, we have used Google Calendar to push out appointments to contractors and installers. While this has worked, see my calendar post for information as to why I chose to leave Google and move my calendar system in house.

So just looking at the screenshot, it is easy to see now what this page does. Appointments are stacked, and the appointment details are provided to techs.

So, nothing really fancy, Google calendar does the same thing, only looks fancier. Not true!

The ability for calendar events to tie into customer records is helpful, since it can allow for appointments to be assigned to customers or tax lots for later reference. For example, I can look at a customer record and see who has visited them in the past.

Not only that, but these events have statuses. Depending on how the repair/install/pizza run went, it is recorded if it was successful, incomplete, or the customer failed to show up.

Location tracking

Now this feature is probably the most ideal for managers, and least ideal for really bad contractors that lie on their time cards. There are stages built within an event that tracks the progress.

Allowing for updated statues during an event utilizes the device and it GPS. Coordinates of the device are recorded under the record, along with the time stamp. From the last post, the result is a map on the final report for the event.

So now we can know when and where a status was updated, and how much time it took between each task. Now, one issue we thought of was that the numbers would just be fudged. While true, if a manager were closely watching each event, it would be apparent when times did not match up, or locations looked off. Apart from that, time stamps on when ONT’s arrived and departed are recorded, which can be compared to the time that it was provisioned.

ONT Provisioning

One of the coolest features that the field tech console allow for, is real time provisioning of ONT’s Not to get too much into detail, but if you read my previous post on the Calix NBI, you know that I used to provision units using the FSAN on each ONT. That FSAN is assigned to the customer and used for numerous things. One big issue that we have just dealt with in the past, was that ONT’s are assigned to customers now, instead of using a registration id. This means the ONT’s are provisioned in the morning, and handed to the installers. This resulted in the wrong ONT getting assigned to the wrong customer sometimes. We no longer really have this issue, but we can use this new feature to our advantage.

Usually on the day of an installation, a customer record is created, and a configuration for the ONT is made. This prevents techs from providing the wrong service package to the customer, or messing up other options.

ONT’s are assigned from inventory to installers. Those installers can then take batches of ONT’s to their vehicles and not requiring them to check into the office every morning. Since the FMS tracks ONT’s, it presents a list of assigned ONT’s to an installer.

Then, once an appointment has been set to the proper status, the ONT can be provisioned!

Obviously this is where the happy green success message would show up, if the ONT wasn’t already provisioned elsewhere. But you can see how an installer has more autonomy to do their job now, and we can actually collect more information than before! This module is going live this Friday, and we will begin to see first hand how well it works. Of course there is more to this module, but I have shown you a good portion of it. There is room in the future for expansion and addition of new features.

For a while now, I have been debating moving the calendar integration within our FMS. Currently we tie into the Google Calendar API via the Auth 3.0 Javascript API. While this works, we miss some of the features that Google refuses to add. I began the search for an in house solution to our problems.

I had a couple of requirements and features that were ideal within a calendar solution. It has to have a non sucky API, be fast, deliver in multiple formats (XML,JSON, HTML), and had to be open source. Currently my employer uses Google Apps, which I have utilized heavily during the initial development of the FMS. While unlikely to ever die soon, Google has been known for killing off programs quickly, leaving users frantically looking for a new solution (*cough* reader). Now, given my current age, I never had to experience lotus notes, exchange or other forms of email. Once I got into the game, gmail was making waves, and I hopped on the train and never looked back. This left me severely limited on knowledge of calendar systems, what other possibilities existed out there. I began to look for a complete overhauled solution.

Attempt 1

My first attempt included a Zimbra based system, which has a dead simple API. Simply add a username and password with proper permissions to an account within an http header, and it returns HTML, JSON, XML or iCAL formats. To push an event or import, is done via iCAL format. I began to entertain this option, and built a front-end to create and view events. Once I had a quick and dirty example, I presented it to my boss. It didn’t pass, and I had to begin brainstorming other ideas.

Attempt 2

After scratching my head and putting the project on the back burner for a couple of weeks, I finally had to sit down with my boss and discuss options. It was obvious that there was no perfect solution. By giving up Google calendar, we would loose its nice interface, and tie-ins to accounts and mobile devices. On the other hand, doing bi-directional updates between our systems to google calendar was not ideal. With the same requirements as before, I proposed a PHP/MYSQL driven solution. This would separate our installers/contractors from needing Google accounts, and allow for many custom options, such as tax lot association and customer account links. This option took a bit of convincing, but finally got approved, and I began development.

I will attempt to not bore you with too many technical flows, but I utilized Serhioromano’s Bootstrap Calendar to fit within our FMS. I had a few minor tweaks, and bugs, but I soon got a decent enough system working. Here is an example of the script used to pull calendar events from the database, and format it to work with the JS calendar.

There were also three main types of events, which used a common tag to represent what kind of event it was.

fiber-install – Obviously a fiber install. The reason this one is different will be explained later

install – Different from an install. Could be a WiFi or wireless one

Later this week, we will be going live with this calendar. Some other features include editing an event, up until the event has been closed by a technician, and recording of additional data that can be used for metrics.

This is the default view. The events stack better than when I had it tied into Google Calendar, and it is much, much faster at loading.

This is the main edit event page. As you can see, the event is closed, and only reports can be generated now.

Here is a completed event that I captured while on campus. Don’t mind the duplicate entry for “Cleaning up”. I fixed that bug :).

So that is all fine and dandy, but why stop there? By looking at the report, you’ve got to be asking yourself, wait, how are there geolocated time stamps?!? Ah, let me explain. The reason we were moving to this new calendar system was also in part to a new module, called the Field Tech Console. This console will allow technicians to receive jobs, and update their event as time goes on. This brings on numerous events, which I will explain in the next post. For now, assume it is GOING TO CHANGE WE INSTALL FIBER. Not really, but I like to think so…

So there you have it. My new calendar module. By moving our calendar in house, it will allow us to tie in our planned operations with records and pretty much everything else. It may not be as fancy as Google Calendar, or even as portable, but the trade off is some prettiness for a much more efficient record managing system.

Before I conclude this post, I want to showoff the add event page. There are actually three different types of event add pages. One for Pending Requests (a module for people requesting service), current customers (repairs and stuff) and general events (main line conduit fix or pick up pizza).

When designing the fiber management system (FMS), one feature that is heavily needed is a way to provision, change, delete and query customer hardware. Being a Calix shop, we have our access hardware, and a CMS (Calix Management System) which aggregates all of our access systems together. The result is a one stop shop where ONT’s can be managed across one interface. Upon our first deployment, we used the CMS client to provision and manage all of our ONTs for customers. We quickly hated using the client for most of our operations. Now, I am in no way trying to bash on Calix’s software, I believe CMS is a great tool. The client on the other hand… I cannot say the same.

CMS features a northbound interface, which uses an XML http request to exchange information between CMS and 3rd party applications. The CMS API is by far my most favorite/hated feature of CMS. The ability to use any language to make and retrieve requests is awesome, but can be difficult in some languages. Let me provide a bit more information. My FMS system uses PHP for a majority of its server side rendering. I know how bad PHP is, but in the case of making a system web accessible from any device drastically limited my options. Further details on why I used PHP will be made available in later posts. The access diagram for the FMS is below.

Since PHP is a server side translator, it allows our information to processed within the same network as our CMS server, ultimately allowing clients to process information in the management VLAN where the CMS and Calix equipment resides. With the ability to serve clients with data via web, any mobile or web accessible device can use the FMS.

Tying it all in

Given my current situation, I cannot explain in detail what I have fully done with this API, but what I can explain are some of the modules we tied the northbound interface in with. Some modules that reside within the FMS are: customer records and basic inventory. Which means that we can record our ONT inventory and tie it in with customers, allowing me to knew exactly what customer has what piece of equipment. Further with the northbound interface module, I can use the inventory data to generate a configuration linked with a customer and have one click customer provisioning. Behold! I have made our jobs so much easier! Upon creating a customer, I can provision an ONT in under 15 seconds, record where it is going, where it was and I don’t even have to unpack the unit. I can hand it to an installer, whom installs it at a customers house, plugs it in, turns it on and it provisions instantly. Now, while a lot of this does not involve the northbound interface, you can see how I have taken three tasks and combined them in to a fluid cycle. (Note, this cycle does not work for everyone, or every deployment, which is why I am currently redesigning the process). Inventory is linked to a customer, and linked to CMS, and the installer does not have to carry a buttset around and punch in registration ID’s. I wish I could explain more, because with the addition of other modules, and more detail, A LOT more cool stuff can be automated.

Building the module, take one

When I first designed this module, I honestly knew almost zero about http requests, Python, XML, SOAP, PHP, JS and Love. Refrain from laughing as much as possible when you see my code and keep in mind that I was learning. Also keep in mind that once I got this working, it became the bomb-diggity for our company. I decided to use Python, because WHY THE HECK NOT?!? Mostly I was aware of how bad PHP was, and I wanted to make the service available even in non web based environments (Yes, I am aware that CLI PHP is thing, but still I wanted to refrain from using it as much as possible). So I got to work writing python scripts that would take in FSAN numbers, ONT ID’s, bandwidth profiles and everything else as arguments. I could then process it and and spit out the result. The code was messy, some outputs were XML, others were strings, and some just returned a 1 or 0. If you want to see how bad it is, here is the link to the github repo. A lot has changed and is not reflected there. It is now a lot more structured and allow for provisioning of SIP services, RG interfaces, and other stuff.

Provisioning was a simple three click process, auto assigned the ONT to a customer, and made an note in the inventory on the whereabouts of an ONT. It even allowed for error checking allowing us to make sure things went smoothly during the provisioning process.

CMS integration was a module added in the early stages of the FMS development. After the module was created, it didn’t really change mostly because it just worked. Only until recently, have I had to make drastic changes to the FMS, which pretty much required a redesign of the whole module.

The process effectively went from this

to something like this

🙁 I am sorry that I cannot show you the inner workings of my new process, but I have been instructed that I cannot display the secret sauce. As you can see, this new flow is a lot more structured and has a lot more going on. What I can explain is what makes the system a heck of a lot better than before.

Dynamic, TO THE MAX

One benefit this new module brings is the added dynamic that previously did not exist. The old code relied on a config file that stored all of the variables for connecting to CMS.

This limitation allowed you to only query one system from one CMS server. The new module allows for dynamic addition and deletion of systems, meaning that multiple CMS hosts with multiple access systems can be queried.

One feature, or more rather byproduct (but works as a feature) is that the result is transferred to the client as an http request. This allow AJAX queries from server to client to occur instantly and be encoded in JSON. I effectively created and API for an API :P. I feel that there is potential to allow access from 3rd party devices/systems in the future.

I must wrap up this post, but before I do, I want to show you a screenshot of what kind of useful data is possible through the new Calix-NIAPI module.

It is certainly cool what can be done when you tie services into one another! If you have questions regarding this system, processes involved please contact me at gbrewster@agoasite.com