Sunday, April 17, 2011

Fun with the BizTalk 2010 Oracle E-Business Suite Adapter

I recently had an opportunity to use the BizTalk 2010 Oracle E-Business Suite WCF adapter, and thought I'd share some useful tips for getting it up and running and connected to an EBS instance.

We'd already installed the WCF adapters that come as part of the "Install BizTalk Adapters" (aka Adapter Pack) in the BizTalk installation media.

Next thing was that we needed the Oracle database client installed on the BizTalk dev machine and to be able to establish connections to Oracle databases. We were using Windows Server 2008 R2 x64 for our development OS, but had actually had quite a bit of trouble first locating and then getting the x64 Oracle client to work with BizTalk. It was fine from Visual Studio as it's a 32-bit application and uses the 32-bit Oracle client, but as soon as we attempted to interact with Oracle from the BizTalk runtime via a 64-bit host, no dice.

In the end we created a 32-bit BizTalk host and configured the Oracle adapters (Database and EBS) to use the 32-bit host for their receive & send handlers. This saved us wasting a lot more time attempting to get x64 Oracle client to work... unfortunately, there doesn't seem to be a heap of good documentation on this scenario anywhere.

Once we'd overcome that hurdle, the next was in establishing a connection to E-Business Suite itself. The documentation for the EBS adapter is VERY good, but there are a few things that can be a bit confusing when you're starting out.

First of all, when you're reading the documentation that comes with the adapter, make sure you know the "model" that the section you're reading relates to: the adapter can be used as a pure WCF adapter (ie, straight from .NET) or from BizTalk... there are some great tutorials in the documentation, but if you're trying to work with the adapter from BizTalk, the tutorials on getting it to work from .NET probably are going to be slightly less relevant and possibly even more confusing.

The next point of confusion is configuring the adapter binding itself. There are a host of properties to configure, and I'll list the key properties we used below. In particular though, note the fact that you need to specify 2 different sets of credentials when configuring the adapter: a set of credentials to connect to the Oracle database underlying E-Business Suite, and then a set of credentials for connecting to the E-Business Suite application itself.

The key properties we configured were:

Binding:
  • Application Short Name - For example, "SQLGL". You can identify the Application Short Name from the Action property of the binding that is generated within Visual Studio - something that looks like eg "InterfaceTables/Select/SQLGL/GL/GL_INTERFACE".
  • Client Credential Type - Identifies the type of credentials you're going to be using & where you'll specify them.
  • Oracle E-Business Suite Organization ID - Identifies the Organization you're interacting with in E-Business Suite. This will be a number, something like "83".
  • Oracle E-Business Suite Responsibility Key - Identifies the responsibility the E-Business Suite user has to interact with the application identified by Application Short Name for the Oracle E-Business Suite Organization ID. Will be something like "1_GL_SUPER_XXX".
  • Oracle Password - Password for the credential type NOT specified by the binding Client Credential Type property.
  • Oracle Username - Username for the credential type NOT specified by the binding Client Credential Type property.
  • Use Ambient Transaction - In our case set to False, as we didn't want our interactions with E-Business Suite to be wrapped in an encompassing transaction.
  • Enable BizTalk Compatibility Mode - Set to True in our case because we were using the adapter from BizTalk.
Credentials:
  • Username - Username for the credential type specified by the binding Client Credential Type property.
  • Password - Password for the credential type specified by the binding Client Credential Type property.
Your E-Business Suite administrator should be able to assist you with locating the correct values for each of these properties, but the 2 most important things to note are:
  • You need to enter credentials into the binding Oracle Password / Oracle Username properties and also into the credentials Username / Password properties. If you set the binding Client Credential Type property to "Database", then the Oracle Password / Oracle Username binding properties should be set to the E-Business Suite application-level credentials, and the credentials Username / Password properties should be set to the Oracle database-level credentials; if you set this property to "EBusiness", the Oracle Password / Oracle Username binding properties should be set to the Oracle database-level credentials, and the credentials Username / Password properties should be set to the E-Business Suite application-level credentials.
  • The specific credentials you choose depend on the values you specify for the Application Short Name, Oracle E-Business Suite Organization ID and Oracle E-Business Suite Responsibility Key properties: the E-Business Suite user you choose must have the responsibility identified by the Oracle E-Business Suite Responsibility Key property in the organization identified by the the Oracle E-Business Suite Organization ID property for the application identified by Application Short Name property.
Once we'd actually configured the adapter correctly, using it was very straight-forward, and the adapter documentation provides some great examples of each scenario.

HTH!

Adventures with BizTalk: HTTP "GET" Part 6: Custom .NET Code

[Note: This post is based upon an old blog post that I'm migrating for reference purposes, so some of the content might be a bit out of date. Still, hopefully it might help someone sometime...]

So, we've seen that we can retrieve files from a dynamically-obtained URL inside BizTalk using the WCF-Custom Adapter, but it's not exactly straight-forward.

In the absence of any other bright ideas, next step is resorting to custom .NET code, so let's look at how we might achieve the desired result using this approach.

So, the basic design of the utility we'll build is that we'll pass in the URL of the file we want retrieved, and the utility will take care of retrieving the file and populating a BizTalk message instance with the results. That way, similar to the previous WCF-Custom Adapter approach, we can then do whatever we want with the message back in our orchestration.

.NET Component

Without further ado, here's the code for the .NET component that will do our job for us:

public class WebUtility
{
  public static void Get(string sourceUrl, XLANGMessage targetMessage)
  {
    if (targetMessage.Count == 0)
    {
      throw new ArgumentException("Parameter 'targetMessage' must have at least one message part.", "targetMessage");
    }

    long bytesProcessed = 0;

    // Assign values to these objects here so that they can be referenced in the finally block.
    Stream retrievedStream = null;
    WebResponse response = null;
    MemoryStream memStream = null;

    // Use a try/catch/finally block as both the WebRequest and Stream classes throw exceptions upon error.
    try
    {
      // Create a request for the specified web-based resource.
      WebRequest request = WebRequest.Create(sourceUrl);
      if (request != null)
      {
        // Send the request to the server and retrieve the WebResponse object.
        response = request.GetResponse();
        if (response == null)
        {
          throw new System.Exception("GetResponse returned null.");
        }

        // Once the WebResponse object has been retrieved,
        // get the stream object associated with the response's data.
        retrievedStream = response.GetResponseStream();

        memStream = new MemoryStream();

        // Allocate a 1k buffer.
        byte[] buffer = new byte[1024];
        int bytesRead;

        // Simple do/while loop to read from stream until no bytes are returned.
        do
        {
          // Read data (up to 1k) from the stream
          bytesRead = retrievedStream.Read(buffer, 0, buffer.Length);

          // Write the data to the local file
          memStream.Write(buffer, 0, bytesRead);

          // Increment total bytes processed
          bytesProcessed += bytesRead;
        } while (bytesRead > 0);

        memStream.Seek(0, SeekOrigin.Begin);

        //Load the Binary representation into the first message part:
        targetMessage[0].LoadFrom(memStream);

        //Set properties of the message being returned.
        targetMessage.SetPropertyValue(typeof(FILE.ReceivedFileName), response.ResponseUri.Segments[response.ResponseUri.Segments.GetUpperBound(0)]);
        targetMessage.SetPropertyValue(typeof(HTTP.ContentType), response.ContentType);
      }
    }
    finally
    {
      // Close the response and stream objects here to make sure they're closed even if an exception
      // is thrown at some point.
      if (response != null) response.Close();
      if (retrievedStream != null) retrievedStream.Close();
      if (memStream != null) memStream.Close();
    }
  }
}

The key parts here are:
  • The Get method accepts two parameters, the url to retrieve the file from, and the target XLANGMessage instance that we want to populate with the retrieved file. It's important to note that we force this to be passed in from the calling orchestration rather than being constructed in our component. This forces the orchestration to control the lifetime of the message instance, as opposed to the custom .NET component, and is (apparently) a recommended practice.
  • We use the .NET WebRequest and WebResponse to retrieve the file from the specified URL.
  • We then force the entire response to be loaded into a MemoryStream that can be better consumed by BizTalk.
  • We use the LoadFrom method of the XLANGMessage Part to load the first Part of the provided target XLANGMessage instance from the MemoryStream.
  • We set a couple of Message Context properties for good measure that might be of use to the calling orchestration, FILE.ReceivedFileName and HTTP.ContentType.
Usage

To use this component from an orchestration, we need to:
  • Declare a message variable of type System.Xml.XmlDocument, eg "myMessage".
  • Use a Construct Message and Message Assigment shape with the following code:
myMessage = new System.Xml.XmlDocument();
WebUtility.Get("http://localhost/vdir/some-file.txt", myMessage);
  • Replacing the hard-coded URL with our dynamically-obtained URL.
  • After this Construct Message shape, we'll then have the dynamically-retrieved file in our "myMessage" message variable, ready to do what we'd like with it.
Introspection...

Pros:
  • Much, much simpler to implement than the WCF-Custom approach.
Cons:
  • We lose the "power" of using a BizTalk Adapter. If we wanted to extend this to allow for a proxy server, we'd need to extend our code accordingly, etc...
Overall though, the simplicity of this approach probably wins out over the WCF-Custom solution. I still don't particularly like moving away from using BizTalk's Adapters (after all, isn't that what BizTalk's meant to be good at?), but the simplicity of this approach makes it much more straight-forward to get up and running with.

Thursday, April 14, 2011

Forcing a .NET Application to Execute as 32-bit

I recently had a requirement to force a .NET application (EXE) to execute as a 32-bit application on a 64-bit operating system. To cut a long and painful story short, I needed the application to execute as a 32-bit application because only the 32-bit version of the Oracle database client was installed, and when the application executed in its default form, it attempted (and failed) to use the 64-bit Oracle database client. Anyway, that's a story for another day.

During the course of solving this problem, I came across a .NET utility I've not used previously: CORFLAGS.EXE. This utility essentially lets you force a .NET assembly (EXE or DLL) to execute as 32-bit or 64-bit.

In my case, all I needed to do was execute the following command on the target .NET EXE file:

CORFLAGS.EXE <target-exe> /32BIT+

where <target-exe> is the path to the .NET EXE to set the 32-bit flag on.

There are a number of other options the utility accepts.

You can find CORFLAGS.EXE in the Windows SDK, typically installed to eg %programfiles%\Microsoft SDKs\Windows\v7.0A\bin.

Adventures with BizTalk: HTTP "GET" Part 5: WCF-Custom Adapter

[Note: This post is based upon an old blog post that I'm migrating for reference purposes, so some of the content might be a bit out of date. Still, hopefully it might help someone sometime...]

[Note 2: Since I originally wrote this blog post, the following Microsoft article provides a very good description of some of the techniques applied below: http://social.technet.microsoft.com/wiki/contents/articles/invoking-restful-web-services-with-biztalk-server-2010.aspx]

OK, so we've essentially ruled out the BizTalk HTTP Adapter for retrieving files from a dynamically-obtained URL, so where to next?

Well, one of the options that searching the net suggested was the use of the WCF-Custom Adapter. Most of these were in the context of getting BizTalk to talk to REST-based services via HTTP, but that's half the battle: Communication with REST-based services relies on being able to control the HTTP verb, as that's an important part of REST.

So, the first thing we needed was to replace the Adapter on the BizTalk Send Port for retrieving the remote file with the WCF-Custom Adapter. Once we've done that, we need to be able to control the HTTP verb used by the Adapter...

HttpVerbBehavior

This is where WCF extensibility really shines. We can create a custom WCF Behavior and associate it with the Endpoint within the WCF-Custom Adapter configuration. This WCF Behavior, which we'll refer to as the HttpVerbBehavior, implements a WCF Message Inspector to enable specifying the HTTP verb through the WCF endpoint configuration. The implementation is beyond the scope of this post, but is well described in the following articles:
The key piece of code looks like:

public class VerbMessageInspector : IClientMessageInspector
{
  // ...
  public object BeforeSendRequest(ref System.ServiceModel.Channels.Message request, IClientChannel channel)
  {
  HttpRequestMessageProperty mp = null;
  if (request.Properties.ContainsKey(HttpRequestMessageProperty.Name))
  {
    mp = (HttpRequestMessageProperty)request.Properties[HttpRequestMessageProperty.Name];
  }
  else
  {
    mp = new HttpRequestMessageProperty();
    request.Properties.Add(HttpRequestMessageProperty.Name, mp);
  }
  mp.Method = this._verb;

  if (mp.Method == "GET")
  {
    mp.SuppressEntityBody = true;

    Message msg = Message.CreateMessage(MessageVersion.None, "*");

    msg.Properties.Add(HttpRequestMessageProperty.Name, mp);
               
    request = msg;
  }
  return null;
  }
  //...

}

The key parts are highlighted, namely that we set the Method to the HTTP verb specified through configuration, and if the verb specified was "GET", we suppress the message body and ensure that the MessageVersion is set to None.

Once the HttpVerbBehavior is compiled and added to the GAC, it also needs to be added to the section of the machine.config file. You also need to restart the BizTalk host instances, and re-open the BizTalk Admin console before it will show up. Once all that's done, you can go to the Behavior tab of the Send Port's WCF-Custom Adapter configuration, and add and configure the httpVerbBehaviour.

WrappedTextMessageEncoder

If we were dealing with XML-based content (such as in the REST scenario), we'd be very close to done. However, in our case we're not, we're dealing with binary content being returned... If you try to send a message out through the Send Port as it's currently configured, the message will successfully reach the remote URL via HTTP GET, and the file will be returned: but it won't make it into BizTalk. Actually, it won't even make it into the client-side WCF endpoint, which needs to process the response before BizTalk can take over.

The reason for this is kind of involved, and it's to do with the Message Encoder that is being used by the WCF-Custom adapter. WCF is (largely) Message-based, and WCF Messages are (largely) assumed to be XML-based. So when the client-side WCF endpoint receives a response that is not XML-based, it doesn't know what to do, and bails out. The layer this happens at is the WCF Message Encoder, and we can get around it by creating our own. The idea behind our WrappedTextMessageEncoder is that it will receive the binary contents of the file, Base-64 encode them, and wrap them in a configurable "wrapper" XML element, before sending the message on through the WCF infrastructure. It's a pain, but something that seems to be required to get any further in our scenario.

The following articles provide more information on creating custom WCF Message Encoders:
The guts of our WrappedTextMessageEncoder though is really the following excerpt:

public class WrappedTextMessageEncoder : MessageEncoder
{
  //...


  public override Message ReadMessage(Stream stream, int maxSizeOfHeaders, string contentType)
  {
    XmlReader reader;

    if (!string.IsNullOrEmpty(this.factory.InboundWrapElementName))
    {
      byte[] contents = new byte[stream.Length];
      long bytesRead = stream.Read(contents, 0, (int)stream.Length);

      string encodedContents = System.Convert.ToBase64String(contents);
      string wrappedContents = String.Format("<{0}>{1}", this.factory.InboundWrapElementName, encodedContents);

      StringReader sr = new StringReader(wrappedContents);

      reader = XmlReader.Create(sr);
    }
    else
    {
      reader = XmlReader.Create(stream);
    }
    return Message.CreateMessage(reader, maxSizeOfHeaders, this.MessageVersion);
  }
  //...
}

The exciting bits are highlighted, namely that we allow the configuration of an "InboundWrapElementName" property, and if it's specified, we Base-64 encode the contents of the incoming Stream and then wrap them in an XML element named according to the "InboundWrapElementName". It's basic, but works...

Again, we need to deploy the WrappedTextMessageEncoder to the GAC, configure it in the section of the machine.config file, and refresh BizTalk. Then we can configure it through the Binding tab of the Send Port's WCF-Custom Adapter configuration (or via the Import/Export tab if it doesn't show up on the Binding tab).

So what we'll have now is a response message that makes it through the WCF client, and can potentially be consumed by the BizTalk WCF-Custom Adapter back into the Send Port. Unfortunately though it's now Base-64 encoded and wrapped in an XML element, which we need to strip out to get it back to a usable form!

Send Port Configuration

To recap on our Send Port configuration, at this stage we should have:
  • Transport Type: WCF-Custom
    • Endpoint Address: Hard-coded for now, but will eventually be dynamically specified within orchestration
    • Binding: wrappedTextMessageEncoding + httpTransport
    • Behavior: httpVerbBehavior
  • Send Pipeline: PassThruTransmit
  • Receive Pipeline: PassThruReceive
One thing I haven't mentioned before now is using the PassThru pipelines for the Send/Receive... as we're not actually sending any message body out, and as we're not anticipating doing anything useful in the pipeline with the message body in, we don't need anything more than these... the XML pipelines would just add overhead that doesn't add any value...

To get the message back to its original state (ie, just the binary contents, not wrapped, not Base-64 encoded), I originally thought (and implemented) that I'd need to strip this out in the orchestration, or at best in a custom pipeline component. Fortunately though, we can do even better!

In addition to the configuration above, we also want to apply the following:
  • Transport Type: WCF-Custom
    • Messages:
      • Inbound BizTalk Message Body:
        • Path:
          • Body path expression: /*[1]
          • Node encoding: Base64
This does our job for us! How cool is that? We tell BizTalk that the message body is located in the first child node of the incoming message, and that the message body is Base-64 encoded. BizTalk then extracts the message body, and decodes it for us!!!

Send / Receive inside the BizTalk Orchestration

So, now we've got the response message back into the BizTalk MessageBox through our WCF-Custom Send Port, how do we send the request message and receive the response message inside the orchestration?

Well, this bit is actually pretty straight-forward:
  • Define a new Port Type, eg GetFilePortType, with a single Operation, eg GetFile, and with a Request and Response both of type System.Xml.XmlDocument.
  • Define a Port, eg GetFilePort, of type GetFilePortType.
  • Define two Message Variables, eg getFileOutRequest and getFileOutResponse, both of type System.Xml.XmlDocument.
  • Use a Construct Message and Message Assignment shape to construct the request message, getFileOutRequest. The Message Assignment shape should include the following code:
getFileOutRequest = new System.Xml.XmlDocument();
getFileOutRequest.LoadXml("");
  • This initialises the getFileOutRequest message variable, and gives it some content. As this content will be suppressed by the HttpVerbBehavior, it could be anything, but it needs to be something (ie, not empty).
  • Use a Send shape to send getFileOutRequest out through the Request operation message of the GetFilePort's GetFile operation.
  • Use a Receive shape to receive the Response operation message from the GetFilePort's GetFile operation into the getFileOutResponse message variable.
  • The getFileOutResponse message variable will now contain the bytes of the retrieved file. You can now do what you like with it, which may include sending it out to the filesystem via a Send Port using the FILE Adapter and a PassThruTransmit pipeline...
Introspection...

So, there you have it, retrieving a remote file using the WCF-Custom Adapter. What do you think? My thoughts are:

Pros:
  • We make use of an existing BizTalk Adapter, and hence can benefit from tranditional BizTalk Adapter scalability and configuration (eg, setting a proxy, backup transports etc).
  • We can, if required, extend and customize the behaviour through WCF extensibility points such as custom behaviors.
Cons:
  • There are lots of moving pieces to develop, test, deploy, and support and maintain. Just by the length of this post, you should get a feel for how involved this solution is...
All in all, when it came down to it, this option just seemed like too much hard work for something that should be a simple requirement. Don't get me wrong, I really like taking advantage of the BizTalk Adapters where possible, for the reasons listed above. But at the end of the day, this solution really is pretty complicated to comprehend, let alone build and maintain - for such a "simple" requirement.

So, what else can we try?

Tuesday, April 12, 2011

Back Online

So yea, you, my imaginary followers, may have noticed I've been somewhat absent...

I've been rather busy...

The new company I've joined, Chamonix IT Consulting, has been going great guns and we've had more work than we know what to do with! A very good situation to be in, but not overly conducive to keeping my blog up to date.

Anyway, I'm back now, and hoping to maintain momentum for a while.

So what's been keeping me busy for the last few months?

Well, I've been doing some really interesting work for a number of clients, including a proof of concept into replacing WebMethods with BizTalk 2010 as a middleware platform - which was so successful it's now become a full BizTalk implementation and migration / development exercise; and another prototype replacing an established paper-based forms process with an automated workflow process based on SharePoint 2010, InfoPath and SharePoint Designer workflow - which has also been very successful to the extent that we have other customers within the business queuing up to assist with their processes!

It's been great to be involved in a couple of such successful projects - and although we've been working with some of the latest technologies in the Microsoft stack, that really is the best part of the success: being able to use these technologies to make our customers' lives better! I know it's an oft-cited goal of IT, but in my experience a lot of the time, for one reason or another, the benefits that were part of the original vision for an initiative are never realised.

I find it interesting that both these successes have started as proof of concepts or prototypes and have progressed in a fairly agile manner... is there something in that?