Queued WCF services

Wcf contains a great feature called queued services. This is basically a Wcf binding, netMsmqBinding, that uses MSMQ queues for message transport. It’s very easy to use (once set up) and allows for asynchronous and decoupled communication. It’s useful in several scenarios:

  • Handling load peaks
  • Robustness, if the service is down the messages are queued up in the MSMQ queue and are consumed by the service again when it’s available.

Requirements

The following is required for queued services to work

  • MSMQ must be installed
  • WAS must be enabled and running
  • All service methods must be one-way

How it works

The client adds a service reference using a normal HTTP meta data endpoint and gets a reference to endpoints that are exposed with the netMsmqBinding. The client endpoint’s address is the MSMQ queue to post to in the following Uri format:

<endpoint address="net.msmq://localhost/private/myservice/myservice.svc" binding="netMsmqBinding"  ... />

The client calls the client proxy just like for any other Wcf service:

var client = new MyService.MyServiceClient();
client.MyMethod(...);

The service is hosted in IIS and WAS (Windows Process Activation Service) is used for listening for messages in the queue. When a message is detected, WAS starts the IIS application containing the service that should receive the message. The service then consumes the message using the netMsmqBinding and processes it.

As already mentioned, the exposed service methods must be one-way:

[ServiceContract]
public interface IMyService
{
    [OperationContract(IsOneWay = true)]
    void MyMethod(...);
}

Throttling

If there are many messages in the queue then our service will process many of them in parallel. I think the default number of simultaneous requests is 12 and this may be too much for our service if the reason we’re doing queued services is to handle peak loads. Luckily this is really easy to configure:

<behaviors>
  <serviceBehaviors>
    <behavior name="MsmqWithMex">
      <!-- Expose metadata via HTTP GET. -->
      <serviceMetadata httpGetEnabled="true"/>
  
      <!-- At most 3 concurrent requests are allowed -->
      <serviceThrottling maxConcurrentCalls="3" />
    </behavior>
  </serviceBehaviors>
</behaviors>

Note: Throttling is available in all Wcf bindings, not just the netMsmqBinding.

Setting it all up

So far we have only discussed the implementation of the service, but we also have to do some configuration in the operating system and in IIS:

  • Install Messaging – if this is not already done, then install it
  • Create queue – Wcf relies on that the necessary queues exist. A good pattern is to check for their existence on application startup and create then if they’re missing.
  • Install/configure WAS for MSMQ in IIS – This is probable the hardest step so I’ll describe in detail:
    1. Enable the net.msmq protocol for the site containing the service:
      %windir%\system32\inetsrv\appcmd.exe set site "Default Web Site" -+bindings.[protocol='net.msmq',bindingInformation='localhost']
      

      The result should be that “net.msmq” is included in the list of protocols:

    2. We can also enable the same protocol in the single application containing the Wcf service:
      %windir%\system32\inetsrv\appcmd.exe set app "Default Web Site/DeltagarloggService" /enabledProtocols:http,net.msmq
      

      Make sure that the protocol is added:

  • Make sure that Net Msmq Listener Adapter service is running:

Troubleshooting
If something goes wrong, here are a few tips:

  • Are the Message Queueing and Net Msmq Listener Adapter services running?
  • net.msmq protocol enable on IIS site and application?
  • Messages still in queue? If empty: check dead-letter queues and outgoing queues.
  • Enable msmq journaling
  • Check event log
  • Restart app pool for the service
  • Restart web site
  • If queue is transactional, check the DTC
  • Browse to the service’s svc file. If that consumes the messages it’s a WAS problem.
  • If the service was hosted in Windows Server AppFabric, then see if it has any logged errors.

For more tips, Tom Hollander have written great blog posts about queued service here.

Happy queueing!

/Emil

Authentication for IIS-hosted WCF services

WCF is very powerful and very, very complicated to configure in many cases. Seemingly simple requirements can get really difficult to get right and security definitely falls in that category. I recently had the following need:

  • WCF-services hosted in IIS 7.5
  • Windows authentication should be used for the services
  • The client must not be required to be in the same domain as the service application

Simple enough, right? Not for me it wasn’t…

My first idea was to set the authentication settings in IIS to Windows Authentication and configure the client to use credentials from a config file, this was the method I’m used to when calling ASMX services, but it didn’t work. It was really frustrating as well as it’s so difficult to understand what’s going on.

No matter what changes I did in the configuration files, I kept getting errors such as

The HTTP request is unauthorized with client authentication scheme 'Anonymous'.
The authentication header received from the server was 'Negotiate,NTLM'.

I was finally able to come up with a solution which I thought I’d share with you. I don’t know if it’s optimal, but if you think it’s not then please leave a comment.

What I did was this:

  • Disable IIS authentication by setting it to Anonymous in the IIS Manager
  • In the binding configuration for the services, set security mode to Message and to use Windows Authentication:
    <security mode="Message">
     <message clientCredentialType="Windows" />
    </security>
    
  • On the client side we now have the option to pass on the currently logged in user’s credentials (requires no extra handling, just call the service) or to explicitly give the credentials to use. The latter is done like this for a ClientBase<T> proxy:

    client.ClientCredentials.Windows.ClientCredential.Domain = "xxx";
    client.ClientCredentials.Windows.ClientCredential.UserName = "yyy";
    client.ClientCredentials.Windows.ClientCredential.Password = "zzz";
    client.MyMethod();
    
  • This solved the problem for me. I think the major difficulty I had was that IIS and WCF security settings collided, so I believe the lesson learned is to do security on only one place and that place should probably be WCF.

    On a related note, I have also had problems with service discovery, WSDL, etc if the web application is not set to Anonymous authentication in IIS, so it seems to be a good idea to do it like that for that reason as well.

    Also, I’m not the only one to have problems with WCF security, here are a few others:
    Post 1, Post 2.

    /Emil

Distributed transactions with WCF and NHibernate

I recently started working on a new project in which we wanted to use WCF services that utilized NHibernate for database work. We also wanted those services to support distributed transactions so that several calls to one or more service would be done within the same client transaction. This is possible thanks to functionality in the System.Transactions namespace and in WCF which supports transaction flowing and of course the Distributed Transaction Coordinator in the operating system (see MSDN for more info on the DTC).

Note: The code below has been tested on NHibernate 2.1.1, Windows XP and .Net 4 Beta 2. Older versions of the .Net Framework should also work, but not necessarily older versions of NHibernate. I believe distributed transaction support was introduced in 2.1.0, but it may or may not work in similar ways to what is described here in older versions since the ADO.Net supports the System.Transactions namespace.

The goal is to write code like this on the WCF client side:

// TransactionScope is from the System.Transactions namespace
using (TransactionScope tx = new TransactionScope())
{
    service1.MyMethod();
    service2.MyMethod();
    tx.Complete();
}

If all goes well, the results of both service calls are comitted to the database. If the call to service2 fails and we get an exception so that tx.Complete() is never executed, then all database updates are rolled back are rolled back and nothing is persisted, even if service1 is hosted in another process or on another machine.

Note also that we’re not limited to database updates, any resource that supports transactions and knows about System.Transactions will be able to roll back updates.

For the above to work, we have to do several things:

  • Configure the service binding to support transaction flow (on both the client and service side). Example:
    <system .serviceModel>
      <services>
        <service ... >
          <endpoint ... bindingConfiguration="TransactionalWsHttp">
             ...
          </endpoint>
          ...
        </service>
      </services>
      <bindings>
        <wshttpbinding>
          <binding name="TransactionalWsHttp" transactionFlow="true"></binding>
        </wshttpbinding>
      </bindings>
    </system>
    
  • Mark the methods that should be enlisted in the distributed transaction with the TransactionFlow attribute. Do this on the service contract interface:
    [TransactionFlow(TransactionFlowOption.Allowed)]
    void SaveMyObject(Foobar object);
    

    That setting dictates that the client’s transaction will be used, if there is one. If there isn’t, a new one will be created automatically for the method call and it will be auto-committed when the method returns.

    You can also use TransactionFlowOption.Mandatory to require the client to supply the transaction. If it doesn’t then WCF will throw an exception.

  • Mark the method implementation with the OperationBehavior attribute like this:
    [OperationBehavior(TransactionScopeRequired=true)]
    public void SaveMyObject(Foobar object)
    {
       ...
    }
    

    This will supply the method with the transaction (the other settings are not enough, they simply indicate that WCF should be configured to be prepared supply the transaction and this setting says that the method really wants it).

This is actually all that is required! NHibernate will now detect if there is a so called ambient transaction (to do this yourself, look at the System.Transactions.Transaction.Current static property, if it’s non-null there there is a transaction) and will enlist its session in it. When the transaction completes, then the saved data will be comitted to the database. If there is an exception so that transaction is never completed then all data will be rolled back.

Important notes:

  • Don’t close the NHibernate ISession explicitly when using the above pattern as NHibernate will do that itself when the distributed transaction completes or is rolled back. If you do, then you will get an exception later when the transaction tries to do it.
  • Don’t create transactions explicitly, let System.Transactions do that for you.
  • If you get this error:
    Network access for Distributed Transaction Manager (MSDTC) has been disabled. Please enable DTC for
    network access in the security configuration for MSDTC using the Component Services Administrative tool.
    

    then you have to do as it says and enable network access in the DTC on the server where the WCF service is hosted (and also on the database server I would assume, but I haven’t actually checked). Here’s an instruction on how to do that:
    Network access for Distributed Transaction Manager (MSDTC) has been disabled

I think this is really cool stuff. Not only does it simplify transaction management in NHibernate, it also allows us to write much more robust distributed service-oriented application with very little effort. You also get support in the operating system, for example för statistics:

DTC statistics

I haven’t tried with other databases than SQL Server but as NHibernate seems to support System.Transactions it is possible that it works with other DB systems as well. If you have any experience with that, please leave a comment 🙂

I will continue to update this post if I do more findings on this subject. When I google about this there wasn’t much information on this subject so hopefully this post will help others with the same needs.

/Emil