Speaking at NSBCon 2014 in London

I am thrilled to announce that I will be speaking at the very first NSBCon, which will be organised in London later this year in June, and has an impressive lineup of speakers. The topic that I’ll be covering will, obviously, be the relationship between the Microsoft Azure platform and NServiceBus.

NSBCon2014

NSBCon2014

For years now, I’m in the unique position to be both a contributor to NServiceBus and an early adopter of the Microsoft Azure platform. Furthermore I’m using this combination for my own startup project (www.messagehandler.net), so I have learned a few things about using NServiceBus on the Microsoft Azure platform.

In this session I will try to transfer some of my experiences to you, not only give you an introduction to the capabilities that the Microsoft Azure platform can bring to your NServiceBus solution, but also show what the differences are with traditional NServiceBus development and what you need to keep in mind.

To be more specific, we will dive deeper into the following topics:

  • What does it mean to run an nservicebus application in such a gigantic network as the azure platform? What are the benefits? What are the issues? And what can we do about those?
  • Which transports are available, and which ones are recommended, in this environment.
  • How about storage and hosting options?
  • And to wrap up, I’ll share some of my lessons learned for developing against azure ( pro tip: stop using the emulator! )

I’m also looking forward to (finally) meet many of you in real life that are actively using, or are interested in using, NServiceBus on Azure. I’ll definitly be hanging around in the hallways in between sessions talking to folks, so go register if you haven’t done so already and do not hesitate to grab me for a chat at the event!

See you there!

 

Releasing NServiceBus.Azure 5.0.0

Hi all,

I have published a new release of the Windows Azure support earlier today. This release is labeled 5.0.0 as it includes a few breaking changes, related to decoupling our binaries from the windows azure binaries that are distributed through VS (and not through nuget)

The first and probably most breaking change is the removal of the dependency to ServiceRuntime.dll, we will now lookup your version of the serviceruntime at runtime instead of at compile time. But because we had an inheritance dependency on one of it’s types in the past (through our RoleEnvironment class in NServiceBus.Hosting.Azure), we had to change the way this dependency works. So now we do composition over inheritance for the roleentrypoint as shown in the samples https://github.com/Particular/NServiceBus.Azure.Samples/blob/5.0.0/VideoStore.AzureServiceBus.Cloud/VideoStore.Sales/WorkerRole.cs

The second breaking change is that we removed our dependency on the WindowsAzure.Diagnostics.dll as well (as this is also distributed by the VS tooling). This forced us to remove the AzureDiagnostics configuration method, which configured diagnostics for you in the past. But as VS now also comes with a nice editing environment for the diagnostics settings, this functionality was not that usefull anymore so we removed it. Instead you should now configure diagnostics in VS itself and use .TraceLogger() instead.

The upside of both changes is that NServiceBus will now work irrespective of the Visual Studio tools your using (Either VS2012, VS2013, SDK 2.1, SDK 2.2 and future versions), so for a large part I hope we got rid of dll hell (Except for windows azure storage client, but that is another story)

Additionally, we also splitted the nuget packages for supporting azure services as transports and as persisters. Now there are 2 more packages that contain the transports

NServiceBus.Azure.Transports.WindowsAzureStorageQueues (Storage SDK 2.1.0.0)
NServiceBus.Azure.Transports.WindowsAzureServiceBus (ASB 2.2.0.0)

Which can be used indepently from the persisters, which are still in

NServiceBus.Azure (Storage SDK 2.1.0.0)

This change will make it a lot easier for hybrid scenarios where people only want to use windows azure servicebus for example with the regular on premises nservicebus.host.exe

I hope the upgrade will go smoothly for you, most changes should be in the azure cloudservice roleentrypoint, people using the transports on premises should not be impacted to much, besides a change in assembly and package references.

I have gone through the upgrade of the sample projects already, so when in doubt you can refer to https://github.com/particular/nservicebus.azure.samples/tree/5.0.0 for an example.

If you have further questions, do not hesitate to contact me.

MessageHandler – Early adopter program

Those of you who have been following me lately, know I’ve been working on a ‘Message processing as a service’ offering called MessageHandler and as MessageHandler is taking shape quickly, we decided to open it up for a few more early adopters.

As part of the Early Adopter Program, you’ll play a pivotal role in the validation of our service and it’s technology components. You’ll have access to the latest bits far before everyone else, giving you a strategic advantage over your competition. Furthermore you will also have the opportunity to provide suggestions to improve or even extend the capabilities before they’re released to the general public, giving you a say in how the final solution turns out.

So hurry up and subscribe for the early adopter program, seats are limited! You can sign up on http://www.messagehandler.net

Access Control Service – Why not to rely on the nameidentifier claim

Over the past week, while preparing http://www.goeleven.com/ for a migration to our production account on windows azure, I learned an important lesson that I would like to share with you, so that you don’t have to make the same mistake as I did.

The web front end, outsources authentication to various identity providers using the windows azure access control service. All of these identity providers provide a common claim that can be used to authenticate the user, being the nameidentifier claim. The value of the name identifier is unique for every user and each application. But in the scenario where there is a man in the middle, such as the access control service, this means that the value of the nameidentifier is actually unique per user per access control instance (as that is considered the application by the identity provider).

This prevents you from doing a certain number of operations with your access control service namespace as the value of the nameidentifier changes when you switch access control service instance, aka you loose your customers.

Things you can no longer provide are:

  • Migration of your namespace
  • Disaster recovery
  • High availability
  • Geographic proximity for travelling users

Therefore it’s better to correlate the user’s information with an identity on another claim, email address for example, which remains stable across different access control service instances.

The live id provider, does however not provide any additional information besides the nameidentifier, so I’m sad to report that I will have to stop supporting it!

And to make matters worse, I did not save any of the other claims. so now I have to go beg all of my users to help me upgrade their account 😦

So if you are a user of http://www.goeleven.com/ , please help me update your account:

  • For LiveId users: Login with your account, navigate to profile > identities and associate any of the other providers.
  • For Non LiveId users: Just login with your account, this will automatically fix your identity.

Thanks in advance…

Launching Goeleven.com

First of all, let me wish you all a happy 2012!!!

Yes it’s new year’s day again, the day where everyone traditionally makes new and personal commitments for the next year. Almost for everyone these commitments are focussed on improving their personal life in one way or another.

I’ve also chosen this day to announce my latest project,  ‘Goe Leven’, as the intent of this project is very much the same as our personal commitments: Improve  personal life. You may not know this yet, but ‘Goe Leven’ is actually a Flemish sentence that means ‘living a good life’. And that’s just what I want you to do.

My mission is simple: Through an online platform I want to help you simplify the hectic life that you are leading today so that you have more time to enjoy it.

Over time, I will be releasing multiple services, apps and maybe even devices that are all intended for personal use for you and your family, at home, in the garden or on the go. And all of them will have the same goal, make day to day life simpler and give you some peace of mind.

Today, the first of these services is launching in preview. It’s an action management service that allows you to organize your daily chores and projects at home. I’ve  named, and based, the service on a methodology  called, ‘Getting Things Done’,  by David Allen. The method rests on the principle that a person needs to move tasks out of the mind by recording them externally, into a trusted system like ‘Goe Leven’. That way, the mind is freed from the job of remembering everything that needs to be done, and can concentrate on actually performing those tasks and gives you more time to be simply relaxed.

The platform is available as of now from http://www.goeleven.com and, if you’re interested, access to the private preview of the ‘Getting Things Done’ service can be requested in the store. I will onboard more people in small groups every week so that I can take the proper time to help you with any questions or issues that you may have.

Also, if you have an idea for additional services for personal use, or if you have a major personal topic on your mind that you think software could solve, let me know via the wishlist at http://goeleven.uservoice.com.

I hope that all of you may have a ‘Goe Leven’ as from now on 🙂

Video: Simplifying distributed application development with NServiceBus and the Windows Azure Platform

Yesterday I delivered a presentation to my fellow mvp’s on how NServiceBus and the Windows Azure Platform can be combined to simplify distributed application development in the cloud. The session has been recorded and is now available for everyone who is interested.

Enjoy (and sorry for my crappy presentation skills)

Hosting options for NServiceBus on Azure – Web roles & web workers

Understanding webroles

Up until now I’ve only discussed worker roles, deliberatly, as they are relatively straight forward to use in combination with NServiceBus.

Webroles however are a bit different when it comes to NServiceBus hosting, not that it is per se more difficult to do so, but you need to understand how webroles work to avoid weird effects when running NServiceBus inside them.

There are 2 flavors of webroles, Hosted Web Core and Full IIS, and it makes a difference to host NServiceBus depending on which one you use as their process model differs quite a lot.

Web roles models

If you’re webrole uses HWC there will be only 1 process hosting your assemblies, representing the web role. In this scenario there is no difference between a web and worker and you can simply host an NServiceBus endpoint inside of it the same way.

But in Full IIS mode, which is the default, you need to know that 1 webrole project will result in 2 processes which share the same assemblies. One of the processes is the role host, the other is an IIS website which will actually serve requests. But as they share assemblies, combined with the automagical nature of the NServiceBus hosting model, this would result in 2 processes with identical messagehandlers and configuration, not what you want.

Therefore we only support the hosting model for the role hosts (represented by webrole.cs in a webrole project). For the IIS website you have to configure the bus manually, using the Configure class and it’s extension methods.

Configuring a role manually has also been shown a lot in previous posts, but for completion sake I’ll show it again.

Configure.WithWeb() // With allows you to control the assemblies to load
    .Log4Net()
    .DefaultBuilder()
    .AzureConfigurationSource()
    .AzureMessageQueue()
            .JsonSerializer()
    .AzureSubcriptionStorage()
    .UnicastBus()
            .LoadMessageHandlers()
            .IsTransactional(true)
    .CreateBus()
    .Start();

Web workers

Now that you know that you can control the behavior of NServiceBus in both processes, you can create what we call ‘web workers’ in azure lingo. This are Windows Azure instances that host both webroles and worker roles. These are very interesting from a cost saving perspective, you can in fact host multiple website and workers when combining webworkers with NServiceBus’ dynamic azure host.

To set this up, you need to add a webrole.cs file to your webrole project in which you inherit from NServiceBus’ RoleEntryPoint and configure the endpoint AsA_Host. This will setup the role host to spin off child processes that act as workers, see my previous post on the details of how to do this. The host itself will not run a bus to avoid loading the website’s messagehandlers in it’s process space as these processes do share assemblies. (Remember! Selecting another role, like AsA_Worker, will load the messagehandlers)

In global.asax you can then configure the bus manually to be hosted in the website using the Configure class as stated above. All message handlers referenced from the webrole will be hosted in the IIS process and, as manual configuration ignores any IConfigureThisEndpoint implementation, you also avoid running a second dynamic host.

IIS 7.5

One small caveat when running in IIS version 7.5, which is installed on the latest azure OS families based on Windows Server 2008 R2, is that you cannot access either RoleEnvironment nor HttpContext early in the request lifecycle. Trying to configure NServiceBus at Application_Start will result in an exception as it does use these constructs internally.

An easy way to resolve this is to use .Net’s Lazy<T> to postpone initialization of the bus to the first Application_BeginRequest occurance in a thread safe manner:

private static readonly Lazy StartBus = new Lazy(ConfigureNServiceBus);

private static IBus ConfigureNServiceBus()
{
   var bus = Configure.WithWeb()
                  ...;

   return bus;
}

protected void Application_BeginRequest()
{
   Bus = StartBus.Value;
}

Want to get started with your own webworker?

Then check out the AzureHost sample in the NServiceBus trunk and build from there.

Happy coding!

Hosting options for NServiceBus on Azure – Shared Hosting

Yesterday, I discussed the dedicated hosting model for NServiceBus on Windows Azure. Today I would like to introduce to you the second model, which allows you to host multiple processes on the same role.

In order to setup shared hosting you start by creating a dedicated azure role which represents the host’s controller.

public class Host : RoleEntryPoint{}

The nservicebus role that you need to specify is a special role, called AsA_Host.

public class EndpointConfiguration : IConfigureThisEndpoint, AsA_Host { }

This role will not start a UnicastBus, but instead it will load other roles from azure blob storage and spin off child processes in order to host them.

The only profile that makes sense to specify in this case is the Production or Development profile, which controls where the logging output is sent to. Other behaviors belong strictly to the UnicastBus and it’s parts, so they cannot be used in this context. Besides this profile, you can also set a few additional configuration settings that control the behavior of the host in more detail.

  • DynamicHostControllerConfig.ConnectionString – specifies the connection string to a storage account containing the roles to load, it defaults to development storage.
  • DynamicHostControllerConfig.Container – specifies the name of the container that holds the assemblies of the roles to load, as .zip files, it defaults to ‘endpoints’.
  • DynamicHostControllerConfig.LocalResource – specifies the name of the local resource folder on the windows azure instance that will be used to drop the assemblies of the hosted roles, it defaults to a folder called ‘endpoints’
  • DynamicHostControllerConfig.RecycleRoleOnError – specifies how the host should behave in case there is an error in one of the child processes – by default it will not recycle when an error occurs.
  • DynamicHostControllerConfig.AutoUpdate – specifies whether the host should poll the storage account for changes to it’s childprocess, it defaults to false.
  • DynamicHostControllerConfig.UpdateInterval – specifies how often the host should check the storage account for updates expressed in milliseconds, it defaults to every 10 minutes.
  • DynamicHostControllerConfig.TimeToWaitUntilProcessIsKilled – if there are updates, the host will first kill the original process before provisioning the new assemblies and run the updated process. I noticed that it might take a while before the process dies, which could be troublesome when trying to provision the new assemblies. This setting allows you to specify how long the host is prepared to wait for the process to terminate before it requests a recycle.

Changes to the hosted roles

Any dedicated role can be used as a child process given a few minor changes.

The first change is that all configuration has to be specified in the app.config file. This is intentional, I have removed the ability for the childprocesses to read from the service configuration file. Why? Well, otherwise every childprocess would have been configured the same way as they would share configuration settings, same storage account, same queue, same everything… that’s not what you want. Hence the only option is to limit yourself to using the role’s app.config. Note that the RoleEnvironment is available from the child processes so if you want to read the configuration settings from it, just specify the AzureConfigurationSource yourself, but I advise against it.

The second change is that you need to add a reference to NServiceBus.Hosting.Azure.HostProcess.exe to the worker’s project. This process will be started by the host and you’re role will run inside it’s process space. Why does NServiceBus force you to specify the host process, can’t it do it itself? Well it could, but that could get you into trouble in the long run. If it would decide on the host processes for you, you would be tied to a specific vresion of NServiceBus and future upgrades might become challenging. When you specify it yourself you can just continue to run any version, or customize it if you like, only the name of the process matters to the host.

The provided host process is a .net 4.0 process, but it still uses some older assemblies, so you need to add an NServiceBus.Hosting.Azure.HostProcess.exe.config file to your project, which specifies the useLegacyV2RuntimeActivationPolicy attribute

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
    <startup useLegacyV2RuntimeActivationPolicy="true">
      <supportedRuntime version="v4.0"/>
      <requiredRuntime version="v4.0.20506"/>
    </startup>
</configuration>

The final step is to put the output of the role into a zip file and upload the zip file to the container which is being monitored by the host.

If the host process is running and it’s autoupdate property is set to true, then it will automatically pick up your zip file, extract it and run it in a child process. If the property was set to false, you will have to recycle the instances yourself.

Up until now I’ve only discussed worker roles and how to host message handlers inside them, next time I’ll take a closer look at hosting in a webrole and show you how you can create a so called webworker, where both a website and worker role are hosted in the same windows azure instance.

See you next time.

Hosting options for NServiceBus on Azure – Dedicated Hosting

One of the main concerns people have regarding the windows azure platform is that of costs, especially hosting costs as that makes up the majority of the equation.

This is mostly due to a descrepancy between the model imposed by visual studio, where individual projects map to roles, and the way a lot of people develop their solutions, using many small projects. Especially nservicebus developers follow the guidance to build their solutions suing autonomous components, individual and standalone processes per message type. Can you imagine having a role, with a minimum of 2 instances, per message type?

I can… for certain types of apps. I deliberatily use the short term ‘app’ to represent a system that offers a limited number of features to a large amount of users. In this case it does make sense to use a dedicated role for individual message types.

But for traditional applications, like most of us create on-premises today, it doesn’t make sense. Often these applications offer a lot of features to a limited set of (enterprise) users. For this type of application visual studio for azure imposes a very costly model onto you.

The good news is, NServiceBus gives you a solution for both models… and it allows you to switch between them when required. Very interesting if you want to save a few euros when starting up a new online service, but still want to be able to spread out when traffic picks up.

Dedicated hosting

The first model that I would like to discuss is the dedicated host, which I have used in the past samples quite a lot already, so your probably familiar with it. This one is most suited for ‘apps’ that already picked up some traffic. Every messagehandler, or small group of, is hosted in a dedicated role.

First thing to do is inherit from NServiceBus’s RoleEntrpoint.

public class Host : RoleEntryPoint{}

And add a class that specifies the role that this endpoint needs to perform.

public class EndpointConfiguration : IConfigureThisEndpoint, AsA_Worker { }

Furthermore you need to specify, through profiles specified in the config, how the role should behave.

NServiceBus.Production NServiceBus.OnAzureTableStorage NServiceBus.WithAzureStorageQueues

Depending on this behaviour you may need to specify some additional configuration settings as well.

Roles & Profiles

The dedicated host can perform multiple roles, but it can only perform one of them. The roles that are supported today are:

  • AsA_Listener – This role is the most basic of them all, it will accept messages and could potentially reply to them, but that’s it.
  • AsA_Worker – This role is probably the most used role, it will accept messges, support saga’s and publish events if needed.
  • AsA_TimeoutManager – A specialised role that handles timeout messages sent by saga’s
  • AsA_Host – A specialised role that allows hosting multiple nservicebus processes, we’ll discuss this one in more depth next time.

Using profiles you can control what internal implementations is used by the roles to do their magic.

  • Development – Specifies where nservicebus should log to, in this case the console.
  • Production – Specifies where nservicebus should log to, in this case windows azure tablestorage.
  • OnAzureTableStorage – Specifies where nservicebus should store subscriptions and saga state, in this case azure table storage
  • OnSqlAzure – Specifies where nservicebus should store subscriptions and saga state, in this case sql azure.
  • WithAzureStorageQueues – Specifies what communication mechanism to use, in this case azure storage queues.
  • WithAppFabricQueues – Specifies what communication mechanism to use, in this case appfabric queues.

Next time we’ll take a closer look at the second hosting model, shared hosting…

Stay tuned

AppFabric queue support for NServiceBus

Last week at the //BUILD conference, Microsoft announced the public availability of  the AppFabric Queues, Topics and Subscriptions. A release that I have been looking forward to for quite some time now, as especially the queues are a very valuable addition to the NServiceBus toolset.

These new queues have several advantages over azure storage queues:

  • Maximum message size is up to 256K  in contrast to 8K
  • Throughput is a lot higher as these queues are not throttled on number of messages/second
  • Supports TCP for lower latency
  • You can enable exactly once delivery semantics
  • For the time being they are free!

For a complete comparison between appfabric queues and azure storage queues, this blog post seems to be a very comprehensive and complete overview: http://preps2.wordpress.com/2011/09/17/comparison-of-windows-azure-storage-queues-and-service-bus-queues/

How to configure NServiceBus to use AppFabric queues

As always, we try to make it as simple as possible. If you manually want to initialize the bus using the Configure class, you can just call the following extension method.

.AppFabricMessageQueue()

If you use the generic role entrypoint, you can enable appfabric queue support by specifying the following profile in the service configuration file

NServiceBus.WithAppFabricQueues

Besides one of these enablers you also need to specify a configuration for both your service namespace and your issuer key. These can be specified in the AppFabricQueueConfig section and are required as they cannot be defaulted by nservicebus because there is no local development equivalent to default to.

<Setting name="AppFabricQueueConfig.IssuerKey" value="yourkey" />
<Setting name="AppFabricQueueConfig.ServiceNamespace" value="yournamespace" />

That’s it, your good to go.. this is all you need to do to make it work.

additional settings

If you want to further control the behavior of the queue there are a couple more settings:

  • AppFabricQueueConfig.IssuerName – specifies the name of the issuer, defaults to owner
  • AppFabricQueueConfig.LockDuration – specifies the duration of the message lock in milliseconds, defaults to 30 seconds
  • AppFabricQueueConfig.MaxSizeInMegabytes – specifies the size, defaults to 1024 (1GB), allowed values are 1, 2, 3, 4, 5 GB
  • AppFabricQueueConfig.RequiresDuplicateDetection – specifies whether exactly once delivery is enabled, defaults to false
  • AppFabricQueueConfig.RequiresSession – specifies whether sessions are required, defaults to false (not sure if sessions makes sense in any NServiceBus use case either)
  • AppFabricQueueConfig.DefaultMessageTimeToLive – specifies the time that a message can stay on the queue, defaults to int64.MaxValue which is roughly 10.000 days
  • AppFabricQueueConfig.EnableDeadLetteringOnMessageExpiration – specifies whether messages should be moved to a dead letter queue upon expiration
  • AppFabricQueueConfig.DuplicateDetectionHistoryTimeWindow – specifies the amount of time in milliseconds that the queue should perform duplicate detection, defaults to 1 minute
  • AppFabricQueueConfig.MaxDeliveryCount – specifies the number of times a message can be delivered before being put on the dead letter queue, defaults to 5
  • AppFabricQueueConfig.EnableBatchedOperations – specifies whether batching is enabled, defaults to no (this may change)
  • AppFabricQueueConfig.QueuePerInstance – specifies whether NServiceBus should create a queue per instance instead of a queue per role, defaults to false

Now it’s up to you

If you want to give NServiceBus on AppFabric queues a try, you can start by running either the fullduplex or pubsub sample and experiment from there.

Any feedback is welcome of course!