Ryan McIntyrehttp://blog.randomdust.com
Currently focusing on Azure and open source tech, with some .NET, SQL Server, and other random topics thrown in.Thu, 14 Feb 2019 18:28:08 +0000en-UShourly1https://wordpress.org/?v=5.0.318715799Azure Service Bus Monitorhttp://blog.randomdust.com/2019/02/azure-service-bus-monitor/
http://blog.randomdust.com/2019/02/azure-service-bus-monitor/#respondThu, 14 Feb 2019 18:19:10 +0000http://blog.randomdust.com/?p=2021Read More]]>A key component to many architectures these days, especially microservice architectures, is a queue to separate services and provide a reliable communication mechanism. Azure has a few solutions that fit different needs for queue’s, but one often used for enterprise level reliable message queuing is Service Bus.

Another key component to software applications is monitoring. When building cloud native solutions (whether on Azure, AWS, GCP, whatever), this need is amplified due to the multiple services and their respective SLA’s (or lack thereof, in some cases) which in aggregate make up the solution and its SLA. It’s also important to monitor traffic on each service to determine where and when you need to scale, thus leveraging one of the great benefits cloud solutions gives you. In Azure, we have Azure Monitor built-in that we can use to pump metrics to a single location and do nice things like fire alerts, build reports, automate scale, or track information for regulatory needs.

With the above in mind, getting some detailed metrics from Service Bus into Monitor would be very beneficial when it comes to insights at the queue or topic level. The integration between Service Bus and Monitor currently provides metrics such as incoming, outgoing, size, connections, errors, and others. However, when looking at a topic that may have multiple subscriptions set up, it would be beneficial to have some metrics at that level, too, so we can monitor Service Bus and how incoming messages are being distributed to various topics and picked up by any listeners. Or not, in which case they become dead-lettered.

With Service Bus having a nice mature API and Monitor now providing the ability to send in custom metrics, building a solution to monitor topics and subscriptions is now within reach! To show how this can be done, I built a Python app that uses the Azure Python SDK to query Service Bus and send counts for both active messages and dead-letter messages for subscriptions within a topic (that’s a mouthful) to Monitor. There is a Dockerfile in the repo that packages it up so it can be run easily as a cron job in Kubernetes or any other platform that is container friendly.

I feel the code (hosted on GitHub) is self-explanatory (famous last words), so I’m not going into the technical detail in this post. Please comment here or on GitHub with any questions/comments/PR’s.

]]>http://blog.randomdust.com/2019/02/azure-service-bus-monitor/feed/02021Lightroom has encountered problems reading this photohttp://blog.randomdust.com/2018/12/lightroom-has-encountered-problems-reading-this-photo/
http://blog.randomdust.com/2018/12/lightroom-has-encountered-problems-reading-this-photo/#respondFri, 28 Dec 2018 18:43:43 +0000http://blog.randomdust.com/?p=2013Read More]]>I had this error after a rebuild of my office computer where I do my photo editing in Lightroom 6: “Lightroom has encountered problems reading this photo”. It wasn’t on all pictures, just some of my most recent shoots. After some searching and troubleshooting using my two favorite searchengines, I was still drawing a blank as nothing was working. Cutting to the chase, I needed to grab the latest Lightroom 6 update (6.14, at the time.)

When I rebuilt my computer, I used the Lightroom 6 installer I had lying around. Since it was an older version, it didn’t know how to read the photos that had been edited with the newer version of LR6. Seems obvious in retrospect, but I wanted to post the solution here in case it helps anyone else as the error message isn’t all that helpful in troubleshooting.

]]>http://blog.randomdust.com/2018/12/lightroom-has-encountered-problems-reading-this-photo/feed/02013Container for sfctlhttp://blog.randomdust.com/2018/03/container-for-sfctl/
http://blog.randomdust.com/2018/03/container-for-sfctl/#respondWed, 14 Mar 2018 04:55:02 +0000http://blog.randomdust.com/?p=2008Read More]]>The most recent version of the Service Fabric CLI (4.0.0) has a dependency conflict with the Azure CLI (2.0.27) with the former needing knack v0.1.1 and the latter needing knack v0.3.1. While this may be fixed soon, and using a virtualenv in your environment would resolve it, I thought it would be a good idea to do something a bit more permanent.

I created a Docker image based on the Azure CLI image that also has sfctl installed in a virtualenv, with the added bonus of tmux auto-configured to fire up one pane with Azure CLI and another with Service Fabric CLI. Now they will run just fine regardless of any current/future Python dependency conflicts, and can be run on any host that has Docker installed with a simple “docker run -it ryanmcintyre/sfctl”.

az sfctl side by side

I didn’t include any custom tmux conf or anything that wasn’t already in the base image or mentioned previously so it’s fairly vanilla.

]]>http://blog.randomdust.com/2018/03/container-for-sfctl/feed/02008Scripts for Service Fabric Cluster Certificateshttp://blog.randomdust.com/2018/01/scripts-for-service-fabric-cluster-certificates/
http://blog.randomdust.com/2018/01/scripts-for-service-fabric-cluster-certificates/#respondMon, 08 Jan 2018 18:53:45 +0000http://blog.randomdust.com/?p=1997Read More]]>If you need to create a Linux Service Fabric cluster in Azure, there are a couple great templates to get you started, one is a five node/1 node type and another for a seven node/3 node types cluster. Both repo’s assume you have a certificate in Key Vault already, but if you don’t there is a PowerShell script you can use to generate a self-signed cert and upload to Key Vault. Very handy if you have/know PowerShell, but if you aren’t you’re stuck. I created some scripts to help with that.

These scripts mimic the same behavior as the PowerShell script, but can be run in Bash. The first, create-key-vault.sh, creates a Key Vault if you don’t already have one. The second, create-service-fabric-cluster-certificate.sh, is the Bash equivalent of New-ServiceFabricClusterCertificate.ps1. Except I use Key Vault to create the cert instead of creating one locally and then uploading it. This means you need to download the cert afterwards if you want to use it to connect to your cluster after creation. I should probably add some steps on how to do that, but another day. One of the parameters expects a JSON file with the policy info. I have a sample file that can be used, you’ll just need to update it with your specific domain CName.

Finally had a productive day! I made it through about 2.5 courses on Pluralsight and am starting to get a good feel for the basics of Python. I still have a ways to go, but the foundation is there. A few notes I took:

Python code doesn’t have to live in a class; self is similar to this; No formal construct for override; Everything is public, no protected/private

A couple editor notes of VS Code vs PhCharm:

PyCharm has a nice “override” hint in the editor that Code doesn’t show

When there are unresolved references, PyCharm tells you. Code doesn’t.

In one of the classes I took, the instructor showed a code structure like this:
When I ran it set up like that, I got an error that the app couldn’t find the student module (which was imported in app.py). I had to move student.py into the webapp folder for things to work, like this:Maybe a Python version thing, or something for me to learn about setting up a project structure.

I used virtual environments before so I was looking forward to learning more. One class I took used virtualenv and another used venv. Not sure which one is preferred or what the difference is.

Traditional code formatting uses four spaces for indentation. I’m not sure if VS Code defaults to four spaces per tab but I’ll have to look into that.

I was happy to make some good progress and not deal with any non-training issues and I look forward to taking more advanced courses and start working on a real project. I have a pet project in mind but I may also find something out there I can contribute to and get my feet wet (I’m looking at you, Tommy!) That will wrap this series of blog posts, although I may jump back in and post things as I learn more.

Day 3 of my foray into learning Python was a bit more productive than day 2, but still had some hiccups. Other than trying to figure out why my networking drops out randomly (although rarely), I had two noteworthy items:

In VS Code when using “Run Python file in terminal” it runs python, not python3. Not sure yet if there’s a setting to change this but I’m guessing there is. (edit: Well that was easy :))

When I run a script in Code that results in an error, I get the error message and line number in the terminal, which is great. But I noticed PyCharm includes a link in the terminal that takes you straight to the line in the code file that generated the error. Would be great if Code did this, too.

]]>http://blog.randomdust.com/2017/12/python-101-day-3/feed/01983Python 101 – Day 2http://blog.randomdust.com/2017/12/python-101-day-2/
http://blog.randomdust.com/2017/12/python-101-day-2/#respondWed, 20 Dec 2017 22:24:51 +0000http://blog.randomdust.com/?p=1979Read More]]>Following my first day, day 2 could alternatively be titled “Fedora 101 – Day 1”. While I’ve used my Surface Pro 3 with Linux installed (currently running Fedora) when traveling for various dev tasks, I’ve never used it as my main machine at home at my desk. So day 2 was actually spent doing nothing at all with Python and instead working on how to get my machine to recognize my two external monitors and scale properly. Once I found this post referring to a setting in experimental-features for scale-monitor-framebuffer, and rebooting, I was able to scale each monitor separately.

The rest of the day, what was left, was spent figuring out why the Pluralsight videos wouldn’t play in Chromium or Firefox. I finally got Firefox to work looking here, and that was a good way to end the day I thought!

]]>http://blog.randomdust.com/2017/12/python-101-day-2/feed/01979Python 101 – Day 1http://blog.randomdust.com/2017/12/python-101-day-1/
http://blog.randomdust.com/2017/12/python-101-day-1/#respondTue, 19 Dec 2017 17:25:29 +0000http://blog.randomdust.com/?p=1974Read More]]>I’m working on a bit of a pet project this week and without going into an explanation of what I’m working on (mainly because I’m kind of figuring it out as I go :)), I’m going through some training to formally learn Python. I’ve dabbled with it before but I wanted to do something a bit more structured to cover all the bases so I’m taking some Pluralsight courses. I thought it would be good to post up a few notes each day with items worth sharing and/or saving for later when I forget. Like tomorrow. So day 1…

Beyond basic helloworld, data types, loops, string manipulation, and other basic programming tasks, day 1 had some development environment learnings I wanted to note. While the course I’m taking is mainly using PyCharm, I wanted to use VS Code. There’s a Python extension for Code so all is good there. I also prefer to use WSL, which added some wrinkles.

WSL is currently running Ubuntu 16.04, which means to get Python 3.6 installed you either need to build it locally or pull it down from a PPA. Using a promising post I found, I first tried to build locally and then quickly “reminded” myself that I don’t have a c compiler installed. So I punted and went the PPA route. Now my WSL environment has Python 2 (2.7.12) and 3 (3.5.2 and 3.6.3) installed. Golden.

Keep in mind I didn’t install Python in Windows, just WSL. This means (as I found out) a few things won’t work as desired:

Intellisense in Code doesn’t work as I think it picks up hints from the Windows Python installation

That last one is exacerbated a bit due to me running WSL as my terminal in Code. So I can manually execute python3 and run my code in the IDE, but none of the built in tooling works to make it seemless. I tried to figure out if I could trick Code into using my WSL version of python through some settings, but that didn’t help. Below is as close as I got, but it’s still using a Windows path instead of Linux path format.

So until we can run VS Code directly within a WSL environment, I think my ideal dev environment isn’t possible. Bummer, but I’ve been meaning to spend more time on my Fedora machine anyway, so looks like the rest of this week will be on that OS.

]]>http://blog.randomdust.com/2017/12/python-101-day-1/feed/01974Networking fix for Fedora on SP3http://blog.randomdust.com/2017/12/networking-fix-for-fedora-on-sp3/
http://blog.randomdust.com/2017/12/networking-fix-for-fedora-on-sp3/#respondMon, 18 Dec 2017 20:33:22 +0000http://blog.randomdust.com/?p=1971Read More]]>When I switched from Manjaro to Fedora 26 on my Surface Pro 3 a couple months ago I was having network issues in that my wireless adapter would lose its connection and then be “frozen” until I rebooted. After some investigation and trial-and-error, the following two posts helped stabilize things:

<edit>
Things still aren’t completely stable. Wifi drops once or twice a day, and when I’m hardwired I lose that connection occassionally, too. I tried this suggestion also and it seemed to help a bit but not 100% stable yet.

]]>http://blog.randomdust.com/2017/12/networking-fix-for-fedora-on-sp3/feed/01971Bash on Windows Productivity Talkhttp://blog.randomdust.com/2017/06/bash-on-windows-productivity-talk/
http://blog.randomdust.com/2017/06/bash-on-windows-productivity-talk/#respondTue, 27 Jun 2017 00:03:42 +0000http://blog.randomdust.com/?p=1968Read More]]>The bi-annual Denver Dev Day was last week and I had the opportunity to present a topic titled “Using Bash on Windows to Increase your Productivity” to an awesome room of fellow techies. The idea for the session came from my increasing use of Bash and Linux, specifically Windows Subsystem for Linux (WSL), and I thought this talk might not only help others learn a few new tools or tricks but also help me learn what others are doing. I was right on both accounts! If you were there, thanks for coming and I hope it was worth your time investment. If you weren’t there, the session abstract and slides are below. Although, the majority of the time was in Bash showing different scenarios and trying different things folks threw out at me, which was fun! Here’s a sampling of what I showed:

Edit Windows files, with mnt and alias

Built in VS Code support

Launch project from bash

Integrated shell

Run any Win exe

Echo $PATH to show what was included and my modifications

Launching Visual Studio 2017

Docker tools

K8s / minikube

Running minikube start requires window to have administrator rights, so we discussed differences between Windows and Linux users/permissions

Run bash from CMD

dir | bash “grep Desk“

bash -c “ls -lh” | findstr Desk

Azure

Multi-window/multi-account (I use a separate Linux user for each Azure subscription)

Session Abstract:

Did you know Windows 10 can run Bash on Linux?? While it may seem weird seeing those words together, that’s no reason to shy away and not consider how this new capability can be leveraged to increase your day to day productivity. Think about all the Linux features, code samples, tutorials and tools that are out in the world. Now think about all the Windows counterparts. Bash on Windows gives us the option to use all of it on a single operating system and I’ll show you how!

This session will show you how to get up and running and then we’ll spend some time looking at specific development scenarios and why you would want to use Bash. If development isn’t your focus, we’ll also look at some DevOps scenarios targeting Azure. Finally, I’ll show you some of my favorite tools, tips and tricks along the way that can help you leave the room with knowledge you can immediately put to good use.