Thursday, April 14, 2011

Adventures with BizTalk: HTTP "GET" Part 5: WCF-Custom Adapter

[Note: This post is based upon an old blog post that I'm migrating for reference purposes, so some of the content might be a bit out of date. Still, hopefully it might help someone sometime...]

[Note 2: Since I originally wrote this blog post, the following Microsoft article provides a very good description of some of the techniques applied below:]

OK, so we've essentially ruled out the BizTalk HTTP Adapter for retrieving files from a dynamically-obtained URL, so where to next?

Well, one of the options that searching the net suggested was the use of the WCF-Custom Adapter. Most of these were in the context of getting BizTalk to talk to REST-based services via HTTP, but that's half the battle: Communication with REST-based services relies on being able to control the HTTP verb, as that's an important part of REST.

So, the first thing we needed was to replace the Adapter on the BizTalk Send Port for retrieving the remote file with the WCF-Custom Adapter. Once we've done that, we need to be able to control the HTTP verb used by the Adapter...


This is where WCF extensibility really shines. We can create a custom WCF Behavior and associate it with the Endpoint within the WCF-Custom Adapter configuration. This WCF Behavior, which we'll refer to as the HttpVerbBehavior, implements a WCF Message Inspector to enable specifying the HTTP verb through the WCF endpoint configuration. The implementation is beyond the scope of this post, but is well described in the following articles:
The key piece of code looks like:

public class VerbMessageInspector : IClientMessageInspector
  // ...
  public object BeforeSendRequest(ref System.ServiceModel.Channels.Message request, IClientChannel channel)
  HttpRequestMessageProperty mp = null;
  if (request.Properties.ContainsKey(HttpRequestMessageProperty.Name))
    mp = (HttpRequestMessageProperty)request.Properties[HttpRequestMessageProperty.Name];
    mp = new HttpRequestMessageProperty();
    request.Properties.Add(HttpRequestMessageProperty.Name, mp);
  mp.Method = this._verb;

  if (mp.Method == "GET")
    mp.SuppressEntityBody = true;

    Message msg = Message.CreateMessage(MessageVersion.None, "*");

    msg.Properties.Add(HttpRequestMessageProperty.Name, mp);
    request = msg;
  return null;


The key parts are highlighted, namely that we set the Method to the HTTP verb specified through configuration, and if the verb specified was "GET", we suppress the message body and ensure that the MessageVersion is set to None.

Once the HttpVerbBehavior is compiled and added to the GAC, it also needs to be added to the section of the machine.config file. You also need to restart the BizTalk host instances, and re-open the BizTalk Admin console before it will show up. Once all that's done, you can go to the Behavior tab of the Send Port's WCF-Custom Adapter configuration, and add and configure the httpVerbBehaviour.


If we were dealing with XML-based content (such as in the REST scenario), we'd be very close to done. However, in our case we're not, we're dealing with binary content being returned... If you try to send a message out through the Send Port as it's currently configured, the message will successfully reach the remote URL via HTTP GET, and the file will be returned: but it won't make it into BizTalk. Actually, it won't even make it into the client-side WCF endpoint, which needs to process the response before BizTalk can take over.

The reason for this is kind of involved, and it's to do with the Message Encoder that is being used by the WCF-Custom adapter. WCF is (largely) Message-based, and WCF Messages are (largely) assumed to be XML-based. So when the client-side WCF endpoint receives a response that is not XML-based, it doesn't know what to do, and bails out. The layer this happens at is the WCF Message Encoder, and we can get around it by creating our own. The idea behind our WrappedTextMessageEncoder is that it will receive the binary contents of the file, Base-64 encode them, and wrap them in a configurable "wrapper" XML element, before sending the message on through the WCF infrastructure. It's a pain, but something that seems to be required to get any further in our scenario.

The following articles provide more information on creating custom WCF Message Encoders:
The guts of our WrappedTextMessageEncoder though is really the following excerpt:

public class WrappedTextMessageEncoder : MessageEncoder

  public override Message ReadMessage(Stream stream, int maxSizeOfHeaders, string contentType)
    XmlReader reader;

    if (!string.IsNullOrEmpty(this.factory.InboundWrapElementName))
      byte[] contents = new byte[stream.Length];
      long bytesRead = stream.Read(contents, 0, (int)stream.Length);

      string encodedContents = System.Convert.ToBase64String(contents);
      string wrappedContents = String.Format("<{0}>{1}", this.factory.InboundWrapElementName, encodedContents);

      StringReader sr = new StringReader(wrappedContents);

      reader = XmlReader.Create(sr);
      reader = XmlReader.Create(stream);
    return Message.CreateMessage(reader, maxSizeOfHeaders, this.MessageVersion);

The exciting bits are highlighted, namely that we allow the configuration of an "InboundWrapElementName" property, and if it's specified, we Base-64 encode the contents of the incoming Stream and then wrap them in an XML element named according to the "InboundWrapElementName". It's basic, but works...

Again, we need to deploy the WrappedTextMessageEncoder to the GAC, configure it in the section of the machine.config file, and refresh BizTalk. Then we can configure it through the Binding tab of the Send Port's WCF-Custom Adapter configuration (or via the Import/Export tab if it doesn't show up on the Binding tab).

So what we'll have now is a response message that makes it through the WCF client, and can potentially be consumed by the BizTalk WCF-Custom Adapter back into the Send Port. Unfortunately though it's now Base-64 encoded and wrapped in an XML element, which we need to strip out to get it back to a usable form!

Send Port Configuration

To recap on our Send Port configuration, at this stage we should have:
  • Transport Type: WCF-Custom
    • Endpoint Address: Hard-coded for now, but will eventually be dynamically specified within orchestration
    • Binding: wrappedTextMessageEncoding + httpTransport
    • Behavior: httpVerbBehavior
  • Send Pipeline: PassThruTransmit
  • Receive Pipeline: PassThruReceive
One thing I haven't mentioned before now is using the PassThru pipelines for the Send/Receive... as we're not actually sending any message body out, and as we're not anticipating doing anything useful in the pipeline with the message body in, we don't need anything more than these... the XML pipelines would just add overhead that doesn't add any value...

To get the message back to its original state (ie, just the binary contents, not wrapped, not Base-64 encoded), I originally thought (and implemented) that I'd need to strip this out in the orchestration, or at best in a custom pipeline component. Fortunately though, we can do even better!

In addition to the configuration above, we also want to apply the following:
  • Transport Type: WCF-Custom
    • Messages:
      • Inbound BizTalk Message Body:
        • Path:
          • Body path expression: /*[1]
          • Node encoding: Base64
This does our job for us! How cool is that? We tell BizTalk that the message body is located in the first child node of the incoming message, and that the message body is Base-64 encoded. BizTalk then extracts the message body, and decodes it for us!!!

Send / Receive inside the BizTalk Orchestration

So, now we've got the response message back into the BizTalk MessageBox through our WCF-Custom Send Port, how do we send the request message and receive the response message inside the orchestration?

Well, this bit is actually pretty straight-forward:
  • Define a new Port Type, eg GetFilePortType, with a single Operation, eg GetFile, and with a Request and Response both of type System.Xml.XmlDocument.
  • Define a Port, eg GetFilePort, of type GetFilePortType.
  • Define two Message Variables, eg getFileOutRequest and getFileOutResponse, both of type System.Xml.XmlDocument.
  • Use a Construct Message and Message Assignment shape to construct the request message, getFileOutRequest. The Message Assignment shape should include the following code:
getFileOutRequest = new System.Xml.XmlDocument();
  • This initialises the getFileOutRequest message variable, and gives it some content. As this content will be suppressed by the HttpVerbBehavior, it could be anything, but it needs to be something (ie, not empty).
  • Use a Send shape to send getFileOutRequest out through the Request operation message of the GetFilePort's GetFile operation.
  • Use a Receive shape to receive the Response operation message from the GetFilePort's GetFile operation into the getFileOutResponse message variable.
  • The getFileOutResponse message variable will now contain the bytes of the retrieved file. You can now do what you like with it, which may include sending it out to the filesystem via a Send Port using the FILE Adapter and a PassThruTransmit pipeline...

So, there you have it, retrieving a remote file using the WCF-Custom Adapter. What do you think? My thoughts are:

  • We make use of an existing BizTalk Adapter, and hence can benefit from tranditional BizTalk Adapter scalability and configuration (eg, setting a proxy, backup transports etc).
  • We can, if required, extend and customize the behaviour through WCF extensibility points such as custom behaviors.
  • There are lots of moving pieces to develop, test, deploy, and support and maintain. Just by the length of this post, you should get a feel for how involved this solution is...
All in all, when it came down to it, this option just seemed like too much hard work for something that should be a simple requirement. Don't get me wrong, I really like taking advantage of the BizTalk Adapters where possible, for the reasons listed above. But at the end of the day, this solution really is pretty complicated to comprehend, let alone build and maintain - for such a "simple" requirement.

So, what else can we try?

No comments:

Post a Comment