My personal stuff and my interest in Business Processes Management, Collaboration Methodologies and Technologies, Data Security and User Experience.
Technologies that I currently focus on are Dojo, iPhora Foundation/Touch, Linux, NOSQL, and NodeJS.

Thursday, January 26, 2017

Creating Beauty with UglifyJS

For the past five years, we have been using a Notes-client base tool called iPhora Application Designer, that we created to build our iPhora applications written for our Dojo-based iPhora-MVC platform. This tool is written entirely in LotusScript and @Formula language.

With this tool we compile the XML/JS view code into pure JavaScript onto a targeted application server all without opening Domino Designer. Rarely do we venture into using Domino Designer. We only use Domino Designer to enhance our tool or fix a bug. Since this tool is Notes-client based, a key advantage of this tool is the ability to replicate every day our code to a backup server and when we are at home we can replicate to our computer at home.

Currently, our JavaScript MVC platform has over a 1,000,000 lines of Dojo/Javascript code and it grows every day as we add new widgets and modules. Majority of the code is automatically generated by our iPhora Application Designer. Can not imagine generating the code manually.

As part of the compilation process, we created a code HTML/JS minimifier in LotusScript to compress the code. However, it never really worked the way that we wanted it to and it could not obfuscate the code. So we decide to look elsewhere for help.

There are many JS minimifier out there, but we wanted the ability to integrate it with our development process and also our iPhora Application Designer. So we looked into integrating tools that we can run using the LotusScript Shell command to process the JavaScript.

After researching different alternatives, we decided to tried a node-based solution called UglifyJS which has a command line interface which we can invoke using the LotusScript Shell command.

First, you need to download and install the latest version of Node. Make sure that you include the NPM manager during the installation process.

https://nodejs.org/en/download/

After installing node and npm, open a command line window and install uglifyJS from the NPM depository using the following command line.npm install uglify-js -gNow that you have installed UglifyJS, you need to confirm that it is working correctly. Create a simple JavaScript file called test.js and run UglifyJS using the following command.uglifyjs test.js -c -m -o test.min.js-c : compress the code-m : mangle the code-o : declare that there is output file with the following nameIf the resulting test.min.js is a compressed mangled version of test.js then it is working. We could manually create files and then uglify them one at a time and then uploaded them to the server, but that is time consuming and laborious. Automation is key to reducing the time that you need to spend coding. So we needed to figure out a way to automate the process.Our XML/JS code are stored in Notes Rich-text fields. When the code is compiled, the generated compiled code is pushed into the targeted Domino server using good old DXL.

In order to minimify the code using UglifyJS we needed to change our process. Rather than compressing the code using the LotusScript and then pushing it up to the server, we needed a way to get it into the format that the Node-based uglifyJS can use, compress and mangle the code, read the code and push it into the server all in one single operation. Show below is the process and code that we created to achieve our goal.Read JavaScript Code from Rich TextTo get the code from the rich-text field using "GetUnformattedText()" Lotusscript rich text item method.Set ritem = doc.GetFirstItem(fieldName)If Not(ritem Is Nothing) Thencode = ritem.GetUnformattedText()else code = ""end iffileName = tempGetNotesDirectory() & |\| & |input.js|Dim fn%fn = FreeFileOpen fileName For Output As fnPrint #fn,codeClose fnCreate Batch File to Process the JavaScript

Run the batch using the LotusScript Shell command to create the file "output.js"batchInstance = Shell("UglifyJS.bat",0);The problem that you have running a batch file using Shell is that there is no way to know if the process has been completed using LotusScript. However, an easy way to check if the process is done is determine if the file output.js has been created. If not, wait and check again. The following code allows you to query the file system for completion. The uglifying process is very fast, but a large JavaScript file may take longer. The code waits for 1 second and checks the file system. It tried this 10 times and if it can't find it the entire process is killed off.Function runBatch() as BooleanDim tempdir As StringDim i As Integeri = 0On Error GoTo ProcessErrorbatchInstance = Shell(tempdir &"\node_batch.bat", 6)Do While i < 10Sleep(1)If Dir$("output.js",0) <> "" ThenExit DoEnd Ifi = i + 1LoopIf i > 10 ThenrunBatch = false ' process timeoutElserunBatch = true ' process ran successfulEnd if

End FunctionAfter the process completes we need to read the output.js file so that we can push it up to the Domino server.Function readJSFile() As StringDim tempdir As StringDim fileName As StringDim uglifiedCode As StringOn Error GoTo ProcessErrortempdir = tempGetNotesDirectory()fileName = tempdir & |\output.js|Dim fn%fn = FreeFileOpen fileName For Input As fnuglifiedCode = Input(LOF(fn), fn)Close fnreadJSFile = uglifiedCodeEnd FunctionUsing our DXL-based methods we stream the compressed and mangled file up to the Domino server.There is one more step that is required. We need to delete all the files that we created in the temporary directory so that we can process another file. If not, the process can not be repeated since there will always be a output.js file.

Function removeTempFiles() as Booleantempdir = tempGetNotesDirectory()kill tempdir & |\uglifyjs.bat|kill tempdir & |\input.js|kill tempdir & |\output.js|End Function* tempGetNotesDirectory(), function to find the Notes temporary directoryThe inclusion of UglifyJS to our iPhora Application Designer has been so successful that we have added other Node-based developer tools to it using the same method to connect our Notes-based tool to new and modern development tools. So far we have added CSSLint and Minifier to cleanup and compress our CSS file. We are looking for other Node modules that we can include to help us automatic the development and testing process. There are thousands of npm modules that run on Node. There are many to choose from.If you are using the Domino Designer, you should look at this Node-based tools to add new capabilities in your developer's toolbox. How that can be done is beyond my skills.

1 comment:

You might go a step further. Have a look at webpack. Besides minifying it also can combine all JS and CSS into a single file, so you need less https calls. When your JS is ess6 compatible you might even use tree shaking to minimise it further

Always be Happy

About Me

I have been working with Notes and Domino since Notes 3. I have started three different companies after working as a consultant in a large firm including Phora Group, Taishan Works and ReCor. I promote the use of collaboration and business processing technologies.
I am the Past President of the GRANITE Lotus User Group here in Chicago and co-organizer of the Midwest Lotus User Group Conferences.
I focus on Business Process Management, UI Design, and Data Security. IBM Champion 2015, 2016, 2017