Menu

Author: Admin

Integrate is once again upon us and I am happy to announce that I will once again be speaking at it. The event occurs from June 26 – 28th at Kings Place, London. This is my 4th trip to London to speak at the event and I am very excited to be heading back to Europe. Thank-you BizTalk360 for the opportunity to speak at this conference again.

Early bird pricing is still in effect until the end of May and seats are going fast. Please visit the event website for more details.

This is the top Microsoft Integration conference and as a result the bar is pretty high for content. As a result, I have a confession. I have been holding back on #MiddlewareFriday.

Every week I try to come up with something interesting to talk about, but I wanted to ensure I had an especially interesting topic to share in London. So behind closed doors, I have been building my presentation and demos over the past few months. The result is a talk called Give your Bots connectivity, with Azure Logic Apps.

At my employer, I have been thinking a lot of what the future of the workforce looks like and the impact that Artificial Intelligence (AI) will have on organizations. One of the areas that I have been exploring within AI is BOTS, or conversational apps. I think these software applications are very interesting and have the opportunity to improve end user experiences and flush out manual work from organizations.

However, a BOT is pretty much useless without connectivity. Azure Logic Apps makes a great companion for BOTS as we execute tasks across a variety of cloud and on-premises systems.

We will also explore some Azure Cognitive Services that will give our BOT more human-like intelligence in order to reduce friction between our BOT and user.

On a previous episode of #MiddlewareFriday I talked about a global integration event occurring on Saturday, March 25th 2017. Happy to announce that I will be speaking at the New York meetup which is being held at the Microsoft Technology Center near Times Square. This is my second opportunity to speak at the MTC and am very much looking forward to it.

The FREE event will take place from 8:30 am to 5 pm and registration is open.

VNB Consulting is hosting the event and I would like to thank Amit and Howard for inviting me to speak. My topic will be on protecting Azure Logic Apps with Azure API Management. I will be using the new Azure API Management Design Surface, so even if you are very familiar with Azure API Management, you will learn something new.

The event is also taking place in 9 other places simultaneously worldwide including Australia, Belgium, Finland, India, New Zealand, The Netherlands, Norway, Portugal, Sweden. You can find more information at the bootcamp’s website.

The purpose of this post is to talk about a side-project that I have going on with Saravana Kumar and BizTalk360. The purpose of #MiddlewareFriday is to create a video blog of new and interesting developments going on in the industry. Each week we will publish a short video that has some new content. The content may feature news, demos and will also highlight other activities going on in the community. From time to time we will also bring on some guests to keep the content fresh and get some different perspectives.

For both Saravana and myself there is no direct commercial incentive in doing the show. It really comes down to participating in a community, learning by doing, improving communication skills and having some fun along the way.

I am going to keep this post updated to keep a running list of the shows – in part to aid in search engine discovery.

It is that time of year where I like to reflect back on what the previous year has brought and also set my bearings for the road ahead. If you are interested in reading my 2015 recap, you can find it here.

Personal

2016 was a milestone birthdate for myself and my twin brother. In order to celebrate, and try to deny getting old for another year, we decided to run a marathon in New York City. The NYC Marathon is one of the 6 major marathons in the world so it acted as a fantastic backdrop for our celebration. Never one to turn down an adventure, my good friend Steef-Jan Wiggers joined us for this event. As you may recall, Steef-Jan and I ran the Berlin Marathon (another major) back in 2013.

The course was pretty tough. The long arching bridges created some challenges for me, but I fought through it and completed the race. We all finished within about 10 minutes of each and had a great experience touring the city.

Kurt, Kent and Steef-Jan in the TCS tent before the race

At the finish line with the hardware.

Celebrating our victory at Tavern on the Green in Central Park.

Speaking

Traveling and speaking is something I really like to do and the MVP program has given me many opportunities to scratch this itch. I also need to thank my previous boss and mentor Nipa Chakravarti for all of the support that she has provided which made all of this possible.

In Q2, I once again had a chance to head to Europe to speak at BizTalk360’s Integrate Event with the Microsoft Product Group. My topic was on Industrial IoT and some of the project work that we had been working on. You can find a recording of this talk here.

On stage….

I really like this photo as it reminds me of the conversation I was having with Sandro. He was trying to sell me a copy of his book, and I was trying to convince him that if he gave me a free copy, that I could help him sell more. Sandro has to be one of the hardest working MVPs I know who is recognized as one of the top Microsoft Integration Gurus. If you have ever having a problem in BizTalk, there is a good chance he has already solved it. You can find his book here in both digital and physical versions.

BizTalk360 continues to be an integral part of the Microsoft Integration community. Their 2016 event had record attendance from more than 20 countries. Thank-you BizTalk360 for another great event and for building a great product. We use BizTalk360 everyday to monitor our BizTalk and Azure services.

On a bit of a different note, this past year we had a new set of auditors come in for SOX compliance. For the first time, that I have experienced, the auditors were really interested in how we were monitoring our interfaces and what our governance model was. We passed the audit with flying colours, but that was really related to having BizTalk360. Without it, our results would not have been what they were.

Q3

Things really started to heat up in Q3. My first, of many trips, was out to Toronto to speak at Microsoft Canada’s Annual General meeting. I shared the stage with Microsoft Canada VP Chris Barry as we chatted about Digital Transformation and discuss our experiences with moving workloads to the cloud.

Next up was heading to the south east United States to participate in the BizTalk Bootcamp. This is my third time presenting at the event. I really enjoy speaking at this event as it is very well run and is in a very intimate setting. I have had the chance to meet some really passionate integration folks at this meetup so it was great to catch up once again. Thank-you Mandi Ohlingerand the Microsoft Pro Integration team for having me out in Charlotte once again.

At the Bootcamp talking about Azure Stream Analytics Windowing.

The following week, I was off to Atlanta to speak at Microsoft Ignite. Speaking at a Microsoft premier conference like Ignite (formerly TechEd) has been a bucket list item so this was a really great opportunity for me. At Ignite, I was lucky enough to participate in two sessions. The first session that I was involved in was a customer segment as part of the PowerApps session with Frank Weigel and Kees Hertogh. During this session I had the opportunity to show off one of the apps my team has built using PowerApps. This app was also featured as part of a case study here.

On stage with PowerApps team.

Next up, was a presentation with John Taubensee of the Azure Messaging team. Once again my presentation focused on some Cloud Messaging work that we had completed earlier in the year. Working with the Service Bus team has been fantastic this year. The team has been very open to our feedback and has helped validate different use cases that we have. In addition to this presentation, I also had the opportunity to work on a customer case study with them. You can find that document here. Thanks Dan Rosanova, John Taubensee, Clemens Vasters and Joe Sherman for all the support over the past year.

Lastly, at the MVP Summit in November, I had the opportunity to record a segment in the Channel 9 studio. Having watched countless videos on Channel 9, this is always a neat experience. The segment is not public yet, but I will be sure to post when it is. Once again, I had the opportunity to hang out with Sandro Pereira before our recordings.

In the booth, recording.

Prepping in the Channel 9 studio

Writing

I continue to write for InfoQ on Richard Seroter’s Cloud Editorial team. It has been a great experience writing as part of this team. Not only do I get exposed to some really smart people, I get exposed to a lot of interesting topics that only fuels my career growth. In total, I wrote 46 articles but here are my top 5 that I either really enjoyed writing or learned a tremendous amount about.

Azure Functions Reach General Availability – Something that I like to do, when possible, is to get a few sound bytes from people involved in the piece of news that I am covering. I met Chris Anderson at the Integrate event earlier in the year, so it was great to get more of his perspective when writing this article.

Netflix Cloud Migration Complete – Everyone in the industry knows that Netflix is a very innovative company and has disrupted and captured markets from large incumbents. I found it interesting to get more insight into how they have accomplished this. Many people probably thought the journey was very short, but what I found was that it wasn’t the case. It was a very methodical approach that actually took around 8 years to complete.

Another article that I enjoyed writing was for the Microsoft MVP blog called Technical Tuesday. My topic focused on Extending Azure Logic Apps using Azure Functions. The article was well received and I will have another Technical Tuesday article published early in the new year.

Back to School

Blockchain

I left this topic off of the top 5 deliberately as I will talk about it here, but it absolutely belongs up there. Back in June, I covered a topic for InfoQ called Microsoft Introduces Project Bletchley: A Modular Blockchain Fabric. I really picked up this topic out of our Cloud queue as my boss at the time had asked me about Blockchain and I didn’t really have a good answer. After researching and writing about the topic, I had the opportunity to attend a Microsoft presentation in Toronto for Financial organizations looking to understand Blockchain. At the Microsoft event (you can find similar talk here), Alex Tapscott gave a presentation about Blockchain and where he saw it heading. ConsenSys, a Microsoft partner and Blockchain thought leader was also there talking about the Brooklyn Microgrid. I remember walking out the venue that day thinking everything was about to change. And it did. I needed to better understand blockchain.

For those that are not familiar with blockchain, simply put, it is a paradigm that focuses on using a distributed ledger for recording transactions and providing the ability to execute smart contracts against these transactions. An underlying principle of blockchain is to address the transfer of trust amongst different parties. Historically, this has been achieved through intermediaries that act as a “middleman” between trading partners. In return, the intermediary takes a cut on the transaction, but doesn’t really add a lot of value beyond collecting and dispersing funds. Trading parties are then left to deal with the terms that the intermediary sets. Using this model typically does not provide incentives for innovation, in fact it typically does the opposite and stifles it due to complacency and entitlement by large incumbent organizations.

What you will quickly discover with blockchain is that it is more about business than technology. While technology plays a very significant role in blockchain, if your conversation starts off with technology, you are headed in the wrong direction. With this in mind, I read Blockchain Revolution by Alex and Don Tapscott which really focuses on the art of the possible and identifying some modern-day scenarios that can benefit from blockchain. While some of the content is very aspirational, it does set the tone for what blockchain could become.

Having completed the book, I decided to continue down the learning path. I wanted to now focus on the technical path. I am a firm believer that in order for me to truly understand something, I need to touch it. By taking the Blockchain Developer course from B9Lab I was able to get some hands on experience with the technology. As a person that spends a lot of time in the Microsoft ecosystem, this was a good learning opportunity to get back into Linux and more of the open source community as blockchain tools and platforms are pretty much all open source. Another technical course that I took was the following course on Udemy. The price point for this course is much lower, so it may be a good place to start without making a more significant financial investment in a longer course.

Next, I wanted to be able to apply some of my learnings. I found the Future Commerce certificate course from MIT. It was a three month course, all delivered online. There were about 1000 students, worldwide, in the course and it was very structured and based upon a lot of group work. I had a great group that I worked with on an Energy-based blockchain startup. We had to come up with a business plan, pitch deck, solution architecture and go to market strategy, Having never been involved in a start-up at this level (I did work for MuleSoft, but they were at more than 300 people at the time), it was a great experience to work through this under the tutelage of MIT instructors.

If you are interested in the future of finance, aka FinTech, I highly recommend this course. There is a great mix of Finance, Technology, Entrepreneurs, Risk and Legal folks in this class you will learn a lot.

While some people feel that Twitter is losing its relevancy, I still get tremendous value out of the platform. The following is just an example. Someone I follow on Twitter is Dona Sarkar, from Microsoft, I had the opportunity to see her speak at the Microsoft World Partner Conference and quickly became a fan. Back in October, she put out the following tweet, which required further investigation on my part.

Dona’s talks, from the ones that I have seen, are very engaging and also entertaining at the same time. If she is talking about “Gary Vee” in this manner, I am thinking there is something here. So I start to digest some of his content. I was very quickly impressed. What I like about Gary is he has a bias for action. Unfortunately, I don’t see this too often in Enterprise IT shops; we try to boil the ocean and watch initiatives fail because people have added so much baggage that the solution is unachievable or people have become disenfranchised. I have also seen people being rewarded for building “strategies” without a clue how to actual implement them. I find this really prevalent in Enterprise Architecture where some take pride in not getting into the details. While you may not need to stay in the details for long, without understanding the mechanics, a strategy is just a document. And a strategy that has not/cannot be executed is useless.

If you have not spent time listening to Gary, here are some of his quotes that really resonated with me.

Bet on your strengths and don’t give a f&%# about what you are not good at.

Educate…then Execute

You didn’t grow up driving, but somehow you figured it out.

Results are results are results

I am just not built, to have it dictate my one at-bat at life.

Document, Don’t Create.

We will have people who are romantic and hold onto the old world who die and we will have people that execute and story tell on the new platform who emerge as leaders in the new world.

I am built to get punched in the mouth, I am going spit my front tooth out and look right back at you and be like now what bitch.

If this sounds interesting to you, check out a few of his content clips that I have really enjoyed:

I find it is harder and harder to do this. The world is changing so fast, why would anyone want to tie themselves down to an arbitrary list? Looking back on my recap from last year, you won’t find blockchain or bots anywhere in that post, yet those are two of the transformative topics that really interested me in 2016. But, there are some constants that I don’t see changing. I will continue to be involved in the Microsoft Integration community, developing content, really focused on iPaaS and API Management. IoT continues to be really important for us at work so I am sure I will continue to watch that space closely. In fact, I will be speaking about IoT at the next Azure Meetup in Calgary on January 10th. More details here.

I will also be focusing on blockchain and bots/artificial intelligence as I see a lot of potential in these spaces. One thing you can bet on is that I will be watching the markets closely and looking for opportunities where I see a technology disrupting or transforming incumbent business models.

Also, it looks like I will be running a marathon again in 2017. My training has begun and am just awaiting confirmation into the race.

With everything now confirmed, I am happy to report that I will be speaking at Microsoft’s largest conference, Ignite, in September in Atlanta, Georgia.

I will be sharing the stage with John Taubensee from the Azure Service Bus team. We will be talking about some real world messaging scenarios. Service Bus and Event Hubs are important pillars in our Integration strategy at TransAlta so I am looking forward to discussing how we use these technologies to provide mission critical messaging for our business.

Denodo is a Data Virtualization Platform. But what does that mean? Well I consider it to be a Data Integration tool. On the surface, you may think does this tool compete with other Integration Brokers/ESBs. If used correctly, I don’t think so . If overused/abused, then sure it can compete – much like many other tools.

Instead, Denodo competes more with ETL tools such as SSIS or SAP Data Services. What it aims to do is to eliminate, or reduce, the amount of ETL (aka data copying) from data sources only to consolidate those data sources upstream. As soon as ETL is in the mix, you have batch processing and risk running into stale data or missed opportunities.

Instead, you introduce a “data abstraction” layer that sits in front of physical data sources and can project virtual database/table views. These views can span tables and even multiple data sources. Data sources are not limited to databases either. Denodo can virtualize files and scrape web pages. Scraping a web page seems like a very tedious thing to do – and it can be but not if you have tools that are flexible enough that when you want to make a change you can easily do so.

One of the other benefits of Data Virtualization platforms, such as Denodo, is that they can take a regular ODBC/JDBC data source and project it as something else such as REST service. In my scenario, this is part of the reason why I am using Denodo to begin with.

In the app that I am building in the previous blog post, I need to bring in multiple data sets. One data set is on premises and has an Oracle backend, another is a csv file that gets published on a regulator’s website every hour and the last is a Cloud data source.

All of these data sources have been projected as RESTful services using HTTP and JSON. But now I have an issue. How do I connect a Logic App with these on-premise REST services? Yes, Logic Apps has an HTTP connector, but can it see on-premises endpoints? Denodo sits in our Corporate Intranet and I am not interested in opening up inbound firewalls. Even if I could access Denodo from Logic Apps, Denodo does not have any metadata exposed, such as Swagger, so then I would have to deal with message shapes in Logic Apps.

One way to solve this problem, is to use an Azure API App. By doing so we can wrap our Denodo RESTful services in our Web API, decorate it with Swagger and then publish to Azure. By doing this we can now enable a Hybrid Connection since an API App is really running in the context of a Web Site. We also can plug into Logic Apps easily since Swagger is a first class citizen in Logic Apps.

By adding an Azure API App, my architecture now looks like this:

In my previous post, I walked through deploying my API App so I am not going to get into more details here on that process. At this point, I am assuming the API App has been deployed.

The Hybrid connection will take a couple minutes to provision. Once provisioned, we will see that it has a Not Connected status. This is normal since we have not set up our On-premises agent yet.

Next,we need to download and install the agent. Click on the name of your Hybrid connection which in my case is OnPremDenodo.

Next, click on Listener Setup and then Install and configure now.

An executable will show up in your browser. Open it.

Click Run

Midway through the installation you will be prompted to provide a Hybrid Connection String. You can get this from the Azure Portal in the blade that you downloaded the Hybrid Connection agent from.

Once your connection string has been entered, you installation should be complete.

In the Azure Portal, your Hybrid Connection should now have a status of Connected

Notes:

From within your API App, the URL of the REST service that you are trying to hit will be your local REST service URL. It will not be your Hybrid Connection.

In fact, you actually don’t need to reference the Hybrid Connection URL anywhere. When your API App gets called and in turn you call your On-Premise URL, your Hybrid Connection will kick-in and you will access that resource through that tunnel.

Logic Apps will access your API App using the public Azure endpoint, hence the need to lock it down using Azure AD as I discussed in my previous post.

Conclusion

In this post we reviewed how it is possible for a Logic App to access an On-Premise REST service by using Hybrid Connectivity through a custom API App. We could have also used a site to site VPN connection, but that would have involved more moving parts, approvals and people.

In addition to my day job of being an Enterprise Architect at an Energy company in Calgary, I also write for InfoQ. Writing for InfoQ allows me to explore many areas of Cloud and engage with many thought leaders in the business. Recently, I had the opportunity to host a Virtual Panel on Integration Platform as a Service (iPaaS).

Participating in this panel was:

Dan Diephouse – Director of Product Management at MuleSoft where he is part of the team that launched MuleSoft’s iPaaS offering: CloudHub.

Darren Cunningham – Vice President of Marketing at SnapLogic where he focuses on product management and outbound product marketing.

Jim Harrer – Principal Group Program Manager in the Cloud & Enterprise division, at Microsoft, where his team is responsible for Program Management for BizTalk Server and Microsoft’s iPaaS offering: Azure Logic Apps.

Overall, I was very happy with the outcome of the article. I think the panelists offered some great insight into the current state of iPaaS and where this paradigm is headed.

You can read the entire article here and feel free to add comments in the article’s comment section.