Websites: Publish WebJobs from Console Projects

In January, we added a feature called WebJobs that makes it really easy to take background code (programs or scripts) and run it within a Website. Until today, publishing a WebJob required you to either use the portal or Azure PowerShell. With today’s release, you can alternatively publish your WebJobs from right within Visual Studio.

To get started, open an existing Console project or create new one:

Implement your Main method. While not required, you can use the Microsoft Azure WebJobs SDK. The SDK, available as a NuGet package, makes it especially easy to integrate Azure Storage and trigger a function in your program when items are added to Queues, Blobs or Tables.

When you are ready to publish your project to Azure, select your project, right click and choose the Publish as Azure WebJob command.

You will then see a screen for configuring the WebJob and choosing how you want to run it: Run Continuously, Run on a Schedule, or Run on Demand.

If you choose the continuous option, make sure your code is written to run in an endless loop. Once you select OK, you will see the Publish Web dialog where you choose which Website to publish to. You can deploy your WebJob as part of an existing Website, or you can create a new Website.

Finally, open the Azure portal and navigate to the Website that contains your WebJob. Select the WebJobs tab, and you will see helpful information about your WebJob. This includes a link to a dashboard for viewing logs and monitoring status.

Websites: Publish WebJobs from Web Projects

Instead of publishing your WebJobs one at a time, we also make it easy to publish your WebJobs as part of publishing your Azure Website. This way, it is just one step to publish your Website and all related WebJobs. Right-click your Web Project, and choose Add->New Azure WebJob Project or Add->Existing Project as Azure WebJob to associate new or existing Console projects.

Now when you select Publish to publish your project to an Azure Website, the associated WebJobs will be deployed as well.

Mobile Services: Push Notification Wizard for .NET Mobile Services

With Visual Studio 2013 Update 3, we’ve added support for .NET Mobile Services in the Push Notification Wizard, making it easier for you to add push notifications to your Windows Phone and Windows Store applications. The Wizard will help register your client for push notifications and then provide you with sample backend logic to send push notifications. To invoke the wizard, select your project, right click and choose the Add -> Push Notification command.

Mobile Services: Provision Mobile Services during Project Creation

Now as you create your project, you can also create a Mobile Service to accompany it that can be used as a Dev/Test environment. As you develop your project, you can quickly publish it to the accompanying mobile service to test your code against the live service. To provision a new mobile service simply select the checkbox to Host in the cloud in the New ASP.NET Project dialog.

Notification Hubs: View and Manage Device Registrations

Notifications Hubs already has support for sending test notifications to your devices (Windows Phone, Windows Store, iOS, Android, or Kindle Fire). In Visual Studio Update 3, we’ve added more tooling to enable you to view your device registrations, edit tags, and delete registrations. In addition, we also provide a registration count by platform to provide a quick summary view. To access the notifications hubs tooling, find the Notifications Hubs node in Server Explorer, right click, and choose Diagnose.

Virtual Machines: Configuration

In the 2.4 release of the Azure SDK, we continue to improve our support for using IaaS virtual machines from Visual Studio. Whether you’re building services that run on Azure virtual machines or using virtual machines to host your development or test environments, you’ll find the tools in Visual Studio make your job easier.

In this release when using Server Explorer you’ll now notice a Configure option on the context menu for Virtual Machines. Click Configure to open a virtual machine configuration window where you can view and edit properties of the VM as well as the endpoint, extension and disk configuration for that VM. You can also perform operations such as Start, Shutdown, Connect and Capture (more on that later) from the VM Configuration window.

The VM Configuration window also allows you to open multiple VMs at once so that you can easily compare one configuration to another.

To add an endpoint, choose the type of endpoint from the list of available endpoints then click Add. Once the endpoint is added, you can change the name and/or ports to suite your needs. Click Update to apply the changes to the VM.

Similarly you can add or configure extensions on a virtual machine from the configuration window. Virtual Machine extensions are software components that extend the VM functionality and simplify various VM management operations like getting the VM into a desired state or running the latest anti-malware on the VM. See the blog post on VM Extensions for more info. To add an extension just choose the extension from the list of available extensions and click Add. If you add an extension by mistake you can remove it if you have not yet clicked Update.

Depending on the type of extension being added you may also need to provide configuration, you can do this by selecting the extension and clicking Configure. The format and schema of the extension configuration is defined by the extension publisher. You can find more information on configuring extensions by clicking the link at the bottom of the configuration dialog.

From the extension configuration dialog you can supply both the public and the private configuration. The public configuration can be read and updated as necessary. The private configuration is write-only. Once you apply the private configuration, it can only be read by the extension. It cannot be retrieved from Visual Studio, the portal or any other tools. The extension publisher may choose to have sensitive information such as password stored in private configuration.

Virtual Machines: Snapshots

Snapshots provide an easy way to capture a VM image that can be used to create replicas of the same VM. Imagine you have a virtual machine that is configured exactly the way you like; you may have installed new services, configured the firewall, and attached data disks to suite your needs. That’s a good time to create a VM image so that you can use the customized VM to create replicas in the future.

To create a VM Image click on the Capture Image button in the VM configuration window. This will bring up a dialog that allows you to provide an image label and description for your VM image. For a Windows VM you must run Sysprep on the VM to create a generalized VM image. Sysprep will remove any system specific information from Windows so that it can be easily reused as a template for new VMs. If you have run Sysprep you can select the checkbox for I have run Sysprep on the virtual machine and click the Capture button to capture the VM image.

Once you have captured a VM image you can create new virtual machines based on that image snapshot. Select the Create Virtual Machine option from the context menu on the Virtual Machines node in Server Explorer to bring up the Create New Virtual Machine wizard. On the Select Image page of the wizard set the Image Type dropdown to Private Images – VM Images to display your captured VM images.

You can continue with the next pages of the wizard to provide additional information about the new VM like Machine Settings, Cloud Service and Endpoints. Once finished configuring the endpoints you can click Create to create a new virtual machine based on the VM Image you captured.

Virtual Machines: Remote Debugging for 32-bit Virtual Machines

Visual Studio now supports remote debugging of 32-bit applications running on Azure Virtual Machines. You can enable debugging on a specific instance of a virtual machine from the context menu in Server Explorer.

Read-Access Geo-Redundant Storage (RA-GRS) allows you to have higher read availability for your storage account by providing “read only” access to the data replicated to the secondary location. You can now select RA-GRS as a redundancy option for your storage when creating a storage account from Visual Studio.

Storage: View Storage Activity Logs

Azure Storage logging provides a trace of executed requests against your storage account which can be used for diagnostics. If logging is enabled for blob containers, tables, or queues, the logs will appear in the $logs blob container. Similarly, the $metrics tables contain information on capacity for blob containers, and on transactions for blobs containers, queues, and tables. You now have the ability to view the $logs blob container and $metrics tables directly from Server Explorer in Visual Studio.

We launched Emulator Express to address the longstanding customer ask to be able to test multi-role Cloud Services locally without requiring administrator privileges. This has since become the primary Compute emulator used by our customers for new projects. To focus our future investments on a single code base, we are deprecating the Full Emulator in favor of Emulator Express.

New Cloud Services projects in SDK 2.4 will use Emulator Express by default. Do note, however, that Emulator Express is limited to one instance per role.

Cloud Services: New Networking Capabilities in the Service Model

The following capabilities have been available in IaaS via PowerShell. With today’s SDK 2.4 release they are now available for Cloud Services (Web and worker roles), via the service model.

Internal Load Balancing (ILB)

ILB enables users to run highly available services that are accessible only within a trust boundary, in this case the Cloud Service. The internal load balancer settings are defined at a deployment level in the service model.

The .cscfg has a deployment-wide NetworkConfiguration section that has been extended to contain load balancer information:

An internal load balancer is then referenced in the endpoint definition in the .csdef. Referencing an ILB is only supported for InputEndpoints. Referencing ILBs is supported for all role types (worker role and Web role).

This syntax is designed to support different types (IPv4, IPv6) and multiple public IP addresses in the future, but for now it is restricted to exactly one public IP address. The type of the IP address is inferred to be IPv4.

Configurable Idle Timeout for Azure Load Balancer

The Azure Load Balancer supports a configurable TCP idle timeout. You can now set it for a duration between 4 and 30 minutes for inbound connections, only. This configurable idle timeout period allows your application to maintain TCP connections for up to 30 minutes even if there is no activity on the connection.

Endpoint settings for Cloud Services are made in the .csdef. So, in order to update the TCP timeout for a Cloud Services deployment, a deployment upgrade is required. An exception is if the TCP timeout is only specified for a Public IP. Public IP settings are in the .cscfg, and they can be updated through deployment update and upgrade.

If you don’t already have an Azure account, you can sign-up for a free trial and start using all of the above features today. Then visit the Azure Developer Center to learn more about how to build apps with it.

Credits

This blog article is a collaboration of multiple people. Big thanks to Saurabh Bhatia, and the Azure Cloud Tools team for their contributions.

Does this enable a scenario where you can deploy a .Net Console WebJob from command line for CI deployment?

Brady Gaster

I’ll summarize the details of how you can do this in an upcoming blog post, but want to provide the most concise answer I can offer to that question.

We built the WebJobs tooling’s actual publish functionality into a NuGet package so that we could update the publishing routine as it evolves. The entire process, for instance, of creating a Scheduled WebJob involves numerous Azure API calls, so we built it as a NuGet to enable accommodation if the API routine changes over time. This enables a quick update – a NuGet version revision – rather than a full VS update. Currently, this NuGet relies on VS to authenticate to Azure.

Our tests needed to do something you’re probably trying to do – automate the publish of a WebJob. So we needed to write the publishing function in a way that’d be pluggable. So we’ve already implemented a “headless,” command-line enabled method of executing the WebJob publish functionality. Now our testing team is testing the execution of this functionality from a command line in order to verify the 1.0.1 update to the WebJobs publishing NuGet package. Once that happens there’ll be a blog post announcing the update and documenting how you could do it all yourself in a command-line build.

That said, you can use the Azure PowerShell Cmdlets (like my buddy Guang pointed out earlier) to do WebJob-related functionality today, so if you need a method to do this from the command line you’ve definitely got options now. Watch this blog and the ASP.NET & Web Tools team blog for updates on this very soon.

RichiCoder

Awesome! Thank you for the thorough response. I’ll dive into the Cmdlets now to see what I can do, and will look forward future posts on WebJobs.

Can you specify a reserved IP address when using the Configure VM tooling.. or is that still PowerShell only?

Saurabh Bhatia

Reserving IP addresses and creating VMs using that IP address is still only possible through PowerShell. Once the VM with the reserved IP is created it will show up in Server Explorer and you can configure it just like any other VM but you can’t set it to use a reserved IP at that point.
Thanks

Dave Forster

ok, thanks for the update. Are there any plans to support this in the near future ?

Saurabh Bhatia

We are discussing it. Curious to understand what your end to end scenarios is?

Dave Forster

I (and a lot of other people) just think it’s got to be easier to assign a static (reserved) IP address to a VM, especially where it’s role is a Web Server.

I spent hours last week researching how to do this, and it was only actually possible at all due to a recent Azure PS release.. but still a lot of hoops to jump through to achieve something that should (imo) be a simple task. Not everyone is comfortable in a PowerShell world.. this is why you put all that effort into management portals and IDE tooling.. right?

A lot of DNS providers do not provide support to assigning a CNAME to root domains or offer forwarding services. Therefore a static IP is fundamental for a lot of people. This is effectively the scenario we found ourselves in

Saurabh Bhatia

Thanks for the explanation I will take it back to the team.

Guang Yang

@RichiCoder:disqus You can use New-AzureWebsiteJob to do that.

David Peden

“To focus our future investments on a single code base, we are deprecating the Full Emulator in favor of Emulator Express. … Do note, however, that Emulator Express is limited to one instance per role.”

Will this shortcoming be addressed in a future update to Emulator Express before Full Emulator is fully decommissioned?

Varun Maker

Hi David, we currently have no plans of supporting multi-instance on the Emulator Express, since that would have a negative perf impact on the emulator express; the feedback we have received from customers indicates that to debug multi-instance scenarios, most users test directly in the cloud.

Do you have any debugging scenarios that specifically use the multi instance feature of the Full Emulator?

Denys Kholod

>Do you have any debugging scenarios ..

Yes, of course!!!

Let’s say an instance should take a lease on some resource, on its start, free on shut-down. Having at least 2 instances in emulator is a must have!!! There are a lot more scenarios!!!
Deploy to cloud and have things running is about 10 minutes.
How do you imagine this?
Or do you have better suggestions?

Varun Maker

That makes sense. Thanks for clarifying, Denys. Last question — I am assuming that multi-instance feature on the Full Emulator works as expected for your scenarios today, yes? Just wanted to ask since we have received feedback from users that the multi-instance feature is not very useful since the scenario doesn’t quite work as expected due to concurrency issues on the same box.

I will convey your feedback to our engineering team.

Denys Kholod

1) Yes, multi-instance feature currently working as expected, we’ve not faced with any concurrency issues so far;
2) We had no need to simulate 3 or more instances;
3) Simulator helps us to shoot about 80% of issues;
4) To save time in cloud we usually do RDP to one of instances and ‘hot-replacing’ web-site output dll’s.

David Peden

Testing is hard. Testing in the cloud is harder. Testing in Production is an excuse by most not to figure out how to properly wire up integration tests. With a deployment time hovering around 15 minutes, having to test in the cloud means extremely expensive test cycles. If you get something wrong, it’s another 15 minutes to try again. And you have to stand up an entire setup of new instances which is completely unnecessary when you can run locally for free. The Full Emulator works as designed. If people are experiencing concurrency issues, they need to learn how to write better code.

Please don’t use hobbyist feedback to make decisions for enterprise customers.

Thank you for your consideration.

Phil Lupercio

These are my sentiments exactly. Thanks David, you really hit the nail on the head!

mellamokb

Any updates on this?

Multi-instance emulation is absolutely necessary. We have a scalable queue processing system that involves complex mutex sharing to avoid duplication of effort. I use the emulator multiple times a week in testing because of out-of-the-box debugging support and rapid development (<30 seconds per session). Regarding concurrency issues, these days everyone has (or should have) 4-core or 6-core machine, which is only going to increase over time.

Just two weeks ago we were testing a new job-sharing system and I tested with a 4-role setup with 1,1,1,6 instances (9 total). It worked beautifully and I was able to rapidly work out all the bugs in one afternoon. Spin-up and tear-down literally takes less than a minute for the entire service in the emulator.

I shudder to think how well that would work with having to test in the live Azure environment (and nevermind how the dreaded "role instances recycled too many times" message means that a so-called 10-minute round-trip testing cycle suddenly balloons into 1 hour…, and nevermind how the default 5 update domains means it actually takes 10*5=50+ minutes to update all the domain groups when you have more than a few instances)

David Peden hit the nail on the head IMO: "The Full Emulator works as designed. If people are experiencing concurrency issues, they need to learn how to write better code.

Please don't use hobbyist feedback to make decisions for enterprise customers."

Hear hear!

Phil Lupercio

I really agree with David here. This is a pretty big showstopper for one of my largest projects. The one cloud service spans anywhere from 20-100 cores depending on the time of day + traffic.

I have a handful of worker roles that poll queues for tasks to perform and have some very intricate logic which divides a large task into smaller sub-tasks that are requeued that other roles then perform, then sending their results back to the worker which started the process. Not unlike Hadoop in a sense. Modelling this type of acrobatic interaction in development is absolutely crucial – being limited to one role instance defeats the “scale-out” concept.

There’s 4-5 developers as well, so testing directly on Azure would be quite costly (hundreds of Azure cores aren’t cheap) and pose a lot of maintenance overhead. Not to mention time wasted on redeploying for each code change.

Tobias Jamin

I am really afraid of the news that the full emulator is deprecated. We are developing a multi-tenant SaaS solution by using cloud services and on startup our WebRole installs SSL certificates and configures HTTPS-Bindings for SSL (-> SNI) for every tenant to support multi-domain SSL. How should we emulate and test that without full IIS? It is not possible. I hope that the full emulator will not be removed in the near future. However, if you are going to remove the full emulator please tell how I am going to emulate my cloud service with the described behavior!

Varun Maker

Tobias, Emulator Express supports full IIS today…just not be default. You would need to manually change the settings for emulator express to get it to use full IIS. Does that answer your question?

Tobias Jamin

But the phrase “Full Emulator is deprecated” gives the impression that full emulator is only tolerated in the new SDK/Tools and will be decommissioned later (first step => change the default, second step => decommission full emulator). And with David Peden at least one person has the same apprehension as I have.

Any chance someone can shed some more light on the Internal Load Balancing for Web/Worker roles (PaaS)? I’ve tried various combinations of .cscfg files and can’t get it to work. One error I get when publishing is: “Error: Specifying a static virtual network IP address for PaaS deployments is not supported.”

+1 on “Error: Specifying a static virtual network IP address for PaaS deployments is not supported.”. I have also tried to get this working on a ‘simple’ multi tier application (UI-server, Webservice-server) with no luck.

mohit_srivastava

Hi guys. Thanks for reporting. I have followed up on the networking team. They are working in a sample. I will update as soon as it is available.

mohit_srivastava

The ILB team has found there is a bug in the implementation. During packaging, the ILB configuration is stripped. Will keep you posted on progress.

Narayan

Yes, there is a packaging issue which stripes away the ILB reference in the configuration. We will provide an update on the fix

Jarle Nygård

18 days and no updates… This is a blocking issue for us.

Narayan

with the latest SDK 2.4 release, ILB is now supported. However, there is still no support for ‘StiaticVNetIpAddress’. ILB can be used by just specifying a subnet. The ILB IP will be assigned from the subnet.

Dmitry

The SDK 2.5 are available. I’m still receiving “Error: Specifying a static virtual network IP address for PaaS deployments is not supported.” when using StiaticVNetIpAddress option. Is this issue still exist?

Narayan

Yes, it will release in a couple of weeks.

Thanks,
Narayan.

Anders Østhus

Any updates on this bug and the updated samples ?

mohit_srivastava

This is a hotfix for this coming out today. Will post details as soon as it is available.

I am still getting the same confusing error message, “Specifying a static virtual network IP address for PaaS deployments is not supported”. I have tried various settings in .csdef and .cscfg but can’t make it work.

mohit_srivastava

Sorry you are still having difficulty. I have pinged Narayan (he replied on this thread earlier) to help you.
Also, are you building locally, or are you deploying and building via TFS/VS Online?

Konstantin Shapkin

I am building locally. I tried deploying via Visual Studio, Powershell and by uploading the package/config to the management portal. The same error.

mohit_srivastava

Narayan just posted below (in response to Jarle). The ‘StaticVNetIpAddress’ feature of ILB requires a server side change that will be deployed in early October. We will keep you posted. Thank you for reporting the issue and your patience.

Farida

Has the change been deployed? If not, can the blog be updated so that it does not contain the ‘StaticVNetIpAddress’?

Stijn Planckaert

Deployed without the ‘StaticVNetIpAddress’ resulted in a succesfull creation of the ILB, but when I want to check the assigned VIP by using the ‘Get-AzureInternalLoadBalancer’ command, the IPAddress field is empty … Anyone got this working?

mohit_srivastava

I have pinged Narayan from the networking team to update on status.

Narayan

Please create an endpoint that used the ILB. The VIP is allocated only when an endpoint is created which references the ILB.
The fix for StaticVNetIPAddress is under development, we should be able to release it in a few weeks.
Thanks.

Stijn Planckaert

Thx Narayan, that was indeed the problem. Now we are waiting for the hosted build controller to be updated to the latest azure sdk, as it doesn’t seem to be able to handle the ilb configuration.

Patrik Gustavsson

Is F# not supported? I can’t see any Azure F# templates. There are 4 templates under Cloud in Visual C# but the entire Cloud folder is missing in F#.