Pages

Contact

Category: M3 Web Services (MWS)

I forgot how to troubleshoot Infor M3 Web Services (MWS) and wasted time researching again. This time I share what I rediscovered so I remember next time.

Scenario

I created a simple web service of type M3 Display Program (MDP) that creates a Facility in CRS008. The steps work correctly in M3:

MWS Designer error

I created the equivalent web service in MWS Designer (MWSD), but it throws ‘Unable to execute MPD!’:

MWS Designer log

The MWSD log does not provide more information:

We can increase the MWSD log level from WARN to DEBUG/TRACE, it shows some HTTP/SOAP information (remember to revert when done as it fills the disk space):

Fiddler proxy

We can use an HTTP proxy such as Fiddler to see the HTTP/SOAP request/response:

SoapUI

We can use another SOAP client such as SoapUI to test the web service:

Generated Java code

MWS Server generates Java code for each web service. I do not see any error with the generated code. The code uses Apache Axis2/CXF for SOAP, and the old Intentia Movex MPD for the interactive session:

MWS Server log

The MWS Server log provides some more information, it mentions m3.program which hints me to believe my web service is correct and the problem is in M3:

MWS Server DEBUG/TRACE

I increased the Grid’s log levels of Subsystem and MWS to DEBUG and TRACE (remember to revert when done as it fills the disk space):

I ran the test case again, and opened the merged log:

This time the log shows more information. It shows the <MPD> XML, and it shows the error Abnormal termination of mvx.app.pgm […] Dump log created:

M3 DUMPLOG

The M3 dumplog shows even more information, it shows there was a NullPointerException on READ_DSP ICFCMD:

I do not know how to analyze an M3 dumplog, so I will forward it to an M3 developer and ask for help. The problem points not to MWS but to the M3 program. That is as far as I can go in my troubleshooting.

Conclusion

That was how to troubleshoot M3 Web Services and digg more information about an error. I dug as far as I could, all the way to M3, and asked help to an M3 developer. Furthermore, there are plenty other ways to troubleshoot and dig beyond.

X.509 Token Policy in MWS has been available for at least 7 years. For the setup, we create a public-private key and digital certificate for the SOAP client, and the SOAP client and SOAP server exchange certificates to authenticate each other.

Documentation

For an overview of WS-Security (WSS) in the context of M3, see my previous post.

The MWS Designer (MWSD) User Guide has two modest chapters dedicated to WS-Security and X.509 Token policy, and snippets of source code for a Java client:

For more information about the implementation of WS-Security in MWS, read the documentation of Apache CXF and Apache WSS4J (Merlin), and explore the MWS server source code in the lws-server and lws-common JARs:

Enable X.509 Token policy

First, create a web service in MWSD, of any type (API, MDP, SQL), deploy it, and test it to ensure it works correctly, e.g. CRS610MI.GetBasicData:

PROBLEM 1: Do not use the weak crypto recommended by the MWSD User Guide in the parameters keyalg and sigalg. Instead, let the keytool use the default values which today for RSA is 2048 bit key size and SHA256withRSA hashing [2].

PROBLEM 3: The MWSD User Guide uses -alias myalias. But when we upload the certificate to the server, the server changes the alias to “trustedClient_” + hostName regardless. So I use that alias too.

Note: Keep the private key private! If you need to send it somewhere, do so only over a secure channel (regular email is not a secure channel). On the other hand, the public key and certificate are public, so you can shout them in the street no problem.

Here is the client certificate with stronger crypto:

Now, upload the client certificate to the MWS server, so that the server can authenticate the client (peer authentication):

Note: Upload over HTTPS (secure) and not HTTP (clear text); otherwise verify in a side channel the key fingerprint the server received.

Test with SoapUI

Now, test using any SOAP client that supports WS-Security, such as SoapUI.

PROBLEM 1: The Timestamp must be non-zero, e.g. 10; if I set it to zero the server throws “Security processing failed (actions mismatch) […] An error was discovered processing the <wsse:Security> header” and I do not know why.

Add an Signature action (it is for the client to sign the message with its private key, and for the server to use the client’s public key and verify the integrity of the message received). Select the client Keystore, select the Alias of the client, set the keystore Password, in Parts add Name Body with the Namespace of <soapenv:Body> which is http://schemas.xmlsoap.org/soap/envelope/ and Encode Content, and leave the rest default:

Add an Encryption action (it is for the client to encrypt the message with the server’s public key, and for the server to decrypt it with its private key). Select the client Keystore, select the Alias of the server, set the keystore Password, in Parts add the same as above, and leave the rest default:

PROBLEM 2: SoapUI is buggy and does not always seem to immediately pick my changes in the configurations, so I had to close and re-open it to pick my changes.

Now for decryption, go to Incoming WS-Security Configurations, add a configuration, e.g. incoming (the server will encrypt the SOAP response with the client’s public key, so this is for the client to decrypt that using its private key; and the server will sign the SOAP response with its private key, and the client will verify the signature of the message received using the server’s public key). Select the Decrypt Keystore, the Signature Keystore, and set the keystore Password:

PROBLEM 3: There is a bug with decryption in SoapUI 5.2.1, and I solved it by replacing the version of wss4j.jar as explained in this post:

Now, create the sample SOAP request (SoapUI already created a sample request), remove the <soapenv:Header> which we do not need, set your input parameters (e.g. CustomerNumber), add Basic authentication with the M3 Username and Password, select to Authenticate pre-emptively (optional), select the Outgoing WSS and Incoming WSS, and click Submit:

The client will encrypt and sign the Body, the server will decrypt it and verify the signature, the server will execute the web service (e.g. CRS610MI.GetBasicData), it will encrypt and sign the response, and return it to the client.

Result

We now have the decrypted and verified SOAP response (e.g. CustomerName, CustomerAddress):

PROBLEM: Plain HTTP and authentication

My understanding of WS-Security is that by design it is an option to transport the message over plain HTTP. That scenario will occur when the message passes the TLS termination point and into proxies and gateways over plain HTTP. For that, we could securely set the M3 user/password in the SOAP header at <cred:lws> and add them to the Encryption and Signature actions. However, I tried it, and I removed the user/password from the HTTP Basic authentication, but MWS throws “401 Unauthorized […] WWW-Authenticate: Basic […] fault […] missing_credentials”:

I found some old documentation from 2009 that sheds more light; maybe I have to use the Username Token instead; for future work:

Grid best practice

As a general best practice for the Grid, ensure the Configuration Manager > Routers > Default Router, has WWW Authentication Methods disabled for plain HTTP, and enabled for HTTPS only, to prevent sending user/password over plain HTTP:

Troubleshooting

Here are some tips for troubleshooting.

Use SoapUI’s six tabs of logs:

Set the MWS logs to DEBUG level:

Set the MWS Debug Settings to create dump files of all the encrypted and signed SOAP requests (_IN.xml) and responses (_OUT.xml) in the MWS\dumps folder:

Set your SOAP client to use a proxy like Fiddler:

Conclusion

That was my result of testing X.509 Token policy for M3 Web Services with SoapUI. It requires quite a good understanding of the public-key cryptography concepts (public-private keys, certificates, keystores, the dance between client and server, encryption, digital signatures), and it opened more questions than it answered.

Here is my high-level understanding of SOAP Web Services Security (WS-Security, or WSS), at least the WSS X.509 Certificate Token Profile, and how to apply it in the context of Infor M3.

WS-Security

WS-Security is a standard by the OASIS consortium to provide message encryption and digital signature, the usual security properties to prevent eavesdropping and tampering of a message. It uses asymmetric cryptography with public-private keys and digital certificates. There is an additional property which is central to WSS: the security happens at the message level, not at the transport level, i.e. the security follows the message even across proxies and deep packet inspection gateways, for end-to-end security. WSS is common for example in financial institutions that need to inspect and route a message through several nodes that can read the non-secure part of the SOAP envelope yet not reveal the secret in the message, until it reaches the appropriate destination. If a node on the path gets compromised, the security of the message is not compromised. Despite its continued use, WSS has only had few major updates in 10 years, is not considered secure [1][2], the Internet agrees it is complicated and design-by-committee, and there is no industry momentum behind it.

SSL/TLS, on the other hand, provides similar security properties with encryption and digital signature, using public key cryptography as well, but the security happens at the transport level, i.e. before the message level, for point-to-point security only. Thus, intermediate proxies and deep packet inspection gateways are unable to reveal the message to inspect it and route it, unless they have a copy of the destination’s private key. The workaround is to setup a chain of TLS segments, but the compromise of a node on the path, would compromise the message. TLS has additional security properties such as cipher suite negotiation, forward secrecy, and certificate revocation. TLS is constantly being updated with the latest security properties, and is widely supported and documented.

I have seen WSS interfaces used at banks and credit card companies that still have ancient mainframes and old middleware, and WSS is always used in combination with TLS, with peer authentication, thus four sets of public/private keys and digital certificates.

Infor Grid

Several applications of the Infor Grid expose SOAP web services, but I could not find how to setup WS-Security at the Grid level, so I assume it is not supported at the Grid level, only at the application level; that’s OK as SOAP over HTTPS is sufficient for the Grid’s needs.

M3 Web Services (MWS)

The MWS application does have settings to configure WS-Security (X.509 Token policy); that would be useful for an external partner to call M3 with message-level security (otherwise there is always SOAP over HTTPS); I will explore this in a future post:

M3 Enterprise Collaborator (MEC)

The MEC application on the Grid does not have built-in SOAP web services. But in MEC Partner Admin the MEC developer can setup SOAP. The SOAP clientSend Web Service process does not support WS-Security; this is most unfortunate as here is precisely where I want to setup the secure interfaces with the banks and credit card companies, bummer, I will have to develop my own WS-Security client in Java. On the other hand, the SOAP server WebServiceSyncIn does support WS-Security; I will explore this in a future post:

Future work

In future posts, I will:

Explore WS-Security in MWS

Explore WS-Security in MEC SOAP client

Explore WS-Security in MEC SOAP server

Conclusion

That was my high level understanding of WS-Security, at least the WSS X.509 Certificate Token Profile, and how to apply it in the context of M3. I am no expert in WS-Security, but this understanding is sufficient for my needs with M3.

There are a few tools that can be used to communicate with M3 outside of smart office including report writers like DB2 or MySQL for reading, M3 Enterprise Collaborator (MEC) for running transactions and of course my favorite the M3 API toolkit. Out of all these options there are drawbacks to each. The report writing is limited to reading data unless you are living life dangerously. The MEC tool can be complicated and time-consuming to set up and pretty much can’t be done without training or a consultant. The M3 API is not all that user-friendly and can be time-consuming especially with long transactions (like adding new items) and deployment can be a bit of a nightmare.

As mentioned above the M3 API toolkit is by far my favorite way of interacting with M3 outside of smart office typically with some added functionality of table lookups which is a much better way to get info out rather than an API call. The reason for choosing the API is simple. The documentation is excellent and the possibilities are endless! That being said there are still some drawbacks.

While the API toolkit supports many different languages if you want to use more than one platform transactions will have to be completely rewritten.

Deployment can be difficult. The toolkit needs to be installed on every computer or device that wants to communicate with M3.

If database access is desired drivers are required and permissions will need to be granted for every client.

Some transactions are long and time-consuming to set up.

There is good news though. Hosting your own custom web service using WCF that uses the M3 API toolkit eliminates all of these drawbacks. If your web service is well thought out expanding your functionality and streamlining day-to-day business activities becomes easy.

So let’s get started. Out of all of the transactions in M3 one of the simplest transactions is confirming a pick list because it only requires two inputs. For the sake of getting your feet wet with this new setup without overwhelming you we will start with this transaction. As we run through this example realize that while this transaction is simple the true power of the web service becomes more obvious with more complicated transactions.

Step 1 Start a new project

In Microsoft Visual Studio start a new project using the template WCF Service Application. I’ve named my project M3Ideas. (creative right?)

Once the project opens you will see two important files in the solution explorer on the right. One will be called Service1 and the other will be called IService1. Service1 is a class where all of the code for actually running transactions using the API will take place and IService1 is an example of what our client applications will see and be able to use. Notice that there is both Service Contracts with Operation Contracts which are the functions that our tablets or computer programs will call and there are Data Contracts with Data Members which is how data will be presented to our software. This is what makes the Web Service powerful, we get the ability to create our own objects and essentially make a wrapper class for the M3 API Toolkit that can be used by any program we want that needs to interact with M3.

So lets start renaming the items to suit our needs. Since our goal is to report Pick Lists I’ll chose to rename the IService1 interface to MWS420 after the M3 program for reporting pick lists. Do this in the solution explorer on the right and Visual Studio will rename it everywhere. I’ll also make just one Operation Contract for now called ConfirmPickList which takes two integers, the delivery number and the suffix. Right now I’ll go ahead and delete the CompositeType class below but don’t forget how to make Data Contracts this interface won’t be using them but with longer transactions they are pretty much the greatest thing on earth. At this point my interface looks something like this.

Remember this is just a prototype for what the client applications will get to use. You might be wondering why I named the interface after only one program. What if you want to use more than one program in you web service? The reason I did this is simply for organization and clarity when making the client applications. When I go to run transactions in other programs I will make new interfaces which will look just like this one only with their own name. This will make it so that the client has to not only specify which transaction to run but which interface the transaction comes from. This enables me to use similar function names for more than one program and still know exactly which program the transaction goes with. A good example of this is if I wanted to confirm manufacturing operations in PMS070 I can use similar function names and the client application will easily know which program each transaction belongs to even if the name isn’t as descriptive as it probably should be. It will become more clear what this will look like in future posts where we connect to the web service from our various clients.

Step 2 Set up the transactions

Ok lets look at the Service1.svc file now which is where the code for this transaction will be placed. Go ahead and rename this file to M3.svc and rename the class M3 as well. This is where all the code for the transactions will go. The single most important thing in this file is the interface implementation right after the class name. In an effort to be organized we will use several partial classes rather than one class. Each partial class will implement one of the interfaces we set up for our program. The code will look like this.

Notice that each partial class has a colon before the interface name that it implements. Since I’ve used partial classes each one implements just one of my interfaces. If you really wanted you could use just one regular class that implements all of the interfaces. All you would have to do is list them off and separate them by a comma. I think doing it this way will be a bit more straight forward though.

So now lets get to the fun part and set up the M3 APIs and show the program how to connect to M3 and make the transactions come to life. The first thing we need to do is add a reference to the M3 API. In this example we will use the 64 bit library although you can use whichever one you want. It is interesting to know that the target platform that this service will run on is completely unrelated to the programs that will connect to it. This is another huge advantage to using the web service instead of each client using the API toolkit directly.

To add the reference go ahead and right click on references in the solution explorer and select add reference. On the left select browse and again browse at the bottom and locate the file MVXSockNx64.dll. The file should be located in C:\MvxAPI. Once the file is added you should see the file in the list of references.

Once the reference has been added you can start using the library to communicate with M3. All you need to do is add the using statement at the top of the file and you can start using the library to run transactions. Don’t forget there is a help file that is well documented that will show you how to set up the transactions. Although running these transactions isn’t that elegant the documentation will tell you how to get it done.

To run transactions you will need the port number that the API uses to connect to M3 (there is one of these for each environment), a username and password that is set up in M3 and has permissions to use the APIs that you want to use, as well as the host name. When we are done our transaction will look like this. Note my port numbers might be the same as yours but they don’t have to be. Yours could be different.

I went ahead and put some of the constant Information in a static class called Info. This will make it so I don’t have to type in the data each time and I can use it in all of my partial classes. I’ve also set up the transaction which is exactly how the documentation says to do it. This includes padding the spaces so that each input is in the correct position of the string.

Step 3 Publish

Now that we have our first transaction set up lets publish it and test it. Once it’s been tested we can change to the port to production. To host the web service you will need a computer or virtual machine that is running IIS. You might need to enable the feature in windows. If you are unsure how to enable the feature a simple google search will walk you through it.

Ok to publish the web service right click on project in the solutions explorer and select publish. Set up a publish configuration to publish to the file system in a folder of your choosing. We’ll copy these files to the computer that will host the service. You will also need to locate the file MvxSockx64.dll file and copy that to folder as well. Go ahead and put it in the bin subfolder with the other libraries that got published. Next copy that folder to the C drive of the computer that will host the service and open IIS. On the left side of the screen expand the tree and right click Default Web Site and select add application. Then show IIS what folder your files will be and name your service.

To verify that your service is up and running expand the tree on the left more and select the application you just added. Then on the far right select browse and it should open a browser. Select the SVC file and it should bring you to a screen with directions how to use it. In the next post I’ll run through some samples on how to use the service to streamline reporting pick lists.

Here is the screen that you should be able to get to. If something happened to go wrong it will be displayed on this screen.

If you have any questions on what the web service can be used for please feel free to ask in the comments. Also if you run into any problems please let me know.

MWS authentication

MWS works with SOAP over HTTP over SSL/TLS with the digital certificate of the Infor Grid.

The Infor Grid router for MWS must have Basic authentication enabled over HTTPS (secure) and have all authentication disabled over HTTP (insecure); you can check in the Infor Grid > Configuration Manager > Routers > Default Router:

MWS from IPA

In the IPA Configuration > Web Service Connection, we set the Basic authentication with the M3 user and password:

In Infor Process Designer (IPD), we use the SOAP Web Service activity node to the HTTPS URL of MWS:

Tip: un-hard-code the scheme://host:port and replace it by a variable <!_configuration.main.MWS> to define.

Here is an illustrated example of how to use the Java Debugger in Eclipse to debug an existing Infor Grid application at runtime for which we don’t have the source code; for example I will debug the M3 Web Services (MWS) server, line by line, while I make a SOAP request.

For that, I will use the technique I learned in my previous post, Hacking Infor Grid application development (part 6), where I learned how to use the Java Debugger for my own Grid application for which I had the source code, and I will extend that technique to an existing Infor Grid application for which I don’t have the source code but for which I have the JAR files.

I had to learn this technique in order to troubleshoot a timeout issue we are currently having in MWS when making a SOAP request. This helped me pinpoint the modules, packages, classes, methods and lines of code that were at cause and report the results to Infor Support so they can report them to Infor Product Development for help. To that effect, please join the campaign and sign the petition asking Infor Product Development to make their source code available so we can better troubleshoot issues and learn. Also, this week is Open Access Week and it’s a great opportunity to ask Infor to make more of their documentation available.

Logging and DEBUG level

In order to narrow down the search space for the issue I was troubleshooting, I increased the log level of the application’s node to DEBUG level. In my case, I selected ALL modules: SYSTEM and MWS. I had also selected TRACE but it didn’t produce any log.

I then opened the log file and found interesting information about the bottleneck issue, that helps me narrow down which modules, packages, and classes to debug. In my case they are: MWS com.lawson.webservices.m3.program.M3Session, and SYSTEM DistributedLock:

Remember to revert the log level changes after you’re done to avoid clogging the disk space with increasing log.

Setup the Java Project in Eclipse

Let’s setup a new Java Project in Eclipse with the JAR files of the Infor Grid application (e.g. MWS in my case). For that:

Create a new Java Project in Eclipse, give it a name, and select Add External JARs:

Add the Java decompiler

JD-Eclipse will change the file association in Eclipse to open class files that don’t have source, decompile them, and attach the decompiled source back to the class:

Also, JD-Eclipse will realign the line numbers of the recovered source code so they match with the debug information of the class, otherwise the debugger will show the wrong lines of source code and it will be very confusing:

Now open a class file (double-click) and JD-Eclipse will show the recovered source code with line numbers realigned:

Now debug again with the breakpoints. This time the debugger will show the source code. This is the ideal scenario for debugging line by line:

Problem

I realized that as I’m debugging, JD-Eclipse will not decompile classes which it hasn’t previously decompiled. So if the debugger steps through a class that hasn’t been previously decompiled, the debugger will show the usual “Source not found”.

The workaround is to open all the classes we need the source code for, prior to debugging, one by one, so JD-Eclipse can decompile them, and attach the source code back to the class, and then we can try debugging again.

UPDATE 2014-10-29: That is not true anymore, and I found the solution: we have to change the default file association for the *.class files in order to use Java Decompiler’s Class File Editor, and we must do so every time we start Eclipse because Eclipse will revert to the default Class File Viewer when we exit Eclipse:

Future work: batch decompilation with realignment

I tried the plugin Realignment for JD-Eclipse by Alex Kosinsky that does batch decompilation, but it doesn’t realign the line numbers, so that doesn’t work. I also tried the plugin JDEclipse-Realign by Martin “Mchr3k” Robertson, it does decompile, and it does realign the line numbers, but I couldn’t get the source code to show while debugging, even after having previously decompiled the class, and I don’t know if it does batch decompilation.

UPDATE 2014-10-29: Also, I tried Java Decompiler’s > Save All Sources, but it doesn’t realign the line numbers, even though JD-Core’s change log for version 0.7.0 says “Added an algorithm to realign the decompiled source code to the original line numbers”:

That’s future work to solve.

Summary

That was an illustrative guide of how to do Java debugging on an Infor Grid application, at runtime, without having the source code of the application, having just the JAR files from LifeCycle Manager. This technique helped my troubleshoot an issue with MWS. I hope it will help you too.

That’s it! If you learned something, please click Like below, click Follow to receive notifications about new blog posts, share around you, sign the petition, and write the next blog post with me, you are great. Thank you.

Once in a while I receive this question of how to call an M3 Web Service (MWS) with SQL adapter in a Mashup for Infor Smart Office. As a reminder, MWS has three adapters: M3 API, M3 Display Program (MDP), and SQL (JDBC). Each will return a different SOAP response, thus each will need a slightly specific XAML. The trick with the SQL response is the new0Collection.

Here is my sample SQL:

SELECT OKCUNO, OKCUNM
FROM MVXJDTA.OCUSMA
WHERE OKCONO=? AND OKSTAT=?

Here is the web service in MWS Designer (note the new0 result set in the output):

Here is the Web service test tool in Smart Office (note the new0Collection):

Here is the resulting Mashup:

And here is the final XAML source code with the new0Collection binding highlighted: