Backing up and Restoring SAP BusinessObjects

For most BusinessObjects administrators the process of backing up BusinessObjects is a hands-off task. At most organizations, the server administration team is ultimately responsible for the management of the processes, leaving the BusinessObjects administrator with little day to day involvement. The BOBJ Administrator generally provides the requirements for the backup and simply trusts that the processes are managed by someone else. In general terms the backup processes require that the following components of BusinessObjects and the environment are backed up daily and at approximately the same time:

The Input File Repository Directory

The Output File Repository Directory

The CMS System Database and Audit Database

The Operating System’s registry or other configuration files (Most admins opt for a full server backup)

The SAP BusinessObjects Installation directory (Most admins opt for a full server backup)

Any Java Web Application Server (Tomcat) custom settings

Assuming that all the above components are backed-up on a recurring basis and at the same time, Administrators can rest assured that they have the components required to restore their BusinessObjects environment. However, if you have ever been involved in the restore process you might not think it is all that simple. Depending on the type of restore, the process can be quite daunting. Performing a full system restore, where all of the above components are restored, is the most basic type. A full restore assumes that the entire server, database or file system was lost and only a full restore (to the original equipment) will bring it back to life. However there are two main questions that need to be addressed before a full restore is commenced. First, should I use this process for restoring an individual corrupt or lost info object? Inevitably a power user will “ accidentally” delete a report or a critical Universe will be lost or removed from the repository. When this happens most administrators do not want to fully restore their system to the last viable backup state. This brings us to the second question. What happens to the content that was created or modified after the last full system backup? Executing a full “in-place” restore will likely result in the loss of items modified or created after the backup occurred. Depending on the number of days or hours between the last good backup and the proposed restore point, multiple new items can be lost. If viable copy of the content did not exist in another location or environment, the restore process has a bleak outlook. In the distant past, the only viable option to this problem was a less than desirable process. Administrators would have to setup a temporary BusinessObjects environment and restore the CMS Database and Input / output FRS directories to the temp environment. The process was anything but straightforward and typically required hacking the CMS database (to remove the server object’s records) and then re-creating the SIA to start the temporary system. The most difficult part of the temporary restore processes was that it was seldom tested and prone to issues. However, assuming they were able to give the temporary system life, a simple Import Wizard migration could then be used to restore the lost item.

Starting with BusinessObjects XI R2 administrators had a new option where they could utilize the Import Wizard to create a Business Intelligence Archive (BIAR) file that could be used to store a portion of the CMS System Database and Input / output FRS in a portable file. Administrators could now access their BusinessObjects environment on a daily basis, generate an export of their critical BOBJ content and save it for a rainy day. While this seemed like the answer to a long awaited question, there were a few problems with the solution. First, most BOBJ administrators did not have the discipline to routinely backup their system. Second, large repositories (2 GB+) were just “too large” for this to be a practical solution.

With the release of SAP BusinessObjects 3.x, a new biarengine.jar library was introduced. This library could be executed at the Java command line and utilized to generate .BIAR files based off a CMS SDK pseudo SQL statement. Because it was driven at the command line, it could be scheduled or utilized in custom applications to generate .BIAR files. Depending on the requirements and size of the repository, this command line could be utilized to schedule the export of objects on a recurring basis. In the event that a single object needed to be restored, Administrators could utilize the Import Wizard and the .BIAR file to restore any missing objects. It could also be utilized to restore multiple objects in the event of a complete repository loss as well. With the combination of the complete system backup (Managed by the server Administrators) and scheduled .BIAR file extract, Administrators now had a viable process to recover from any type of failure.

There are, however, a few limitations to utilizing the biarengine.jar or .BIAR files in general. If you search the SAP notes database you will find note 1219208 which indicates a 2GB limit for the size of .BIAR files. This is not exactly 100% true today. Both the biarengine.jar and Import Wizard in later versions of 3.1 and 4.0 will automatically split the .BIAR file into 512 MB files. I have generated an export of over 9000 info objects (reports and Universes) that resulted in 3.7 GB of .BIAR files (8 total files). I had no issues with files of this size when restoring from the Import Wizard. However, SAP tends to state that .BIAR files are not intended to perform backups therefore use caution with this processes if you have a large repository. If you find that you are having issues with large exports, you can look at altering the command line options and run multiple passes with smaller sets of objects in each batch. In short, use the where clause of the CMS pseudo query to limit the size of each pass. I will also state that the biarengine.jar runs under java. If you utilize the 64-bit version of the JRE to execute the biarengine.jar, the process Java heap can go well beyond the 2GB barrier of a 32-bit process. For large repositories, the export process can take several minutes or hours, but I seldom have any issues.

With the initial release of SAP BusinessObjects 4.0 the biarengine.jar method was considerably less useful as a backup tool. The biarengine.jar still exists in 4.0 but until SP4 only the biarengine.jar command line could be utilized to import the .biar files that it generated. The import processes of the biarengine.jar required that everything be re-imported. There was no GUI to allow Administrators to restore just one deleted report. Much like the full restore scenario, this was not a viable solution assuming content had been modified following the .BIAR file export. Starting with SP4, the LCM could also be utilized to import biarengine.jar .BIAR files, but again it required that every object in the .BIAR file be re-imported. It appears that SAP’s overall strategy relies on forcing Administrators to the LCM to promote objects. I personally think this is a huge mistake because the LCM is a really poorly implemented product. In addition, they unwittingly broke the .BIAR file restore processes by forcing all objects to be restored when re-importing. I would be completely content using the LCM if SAP would simply allow Administrators to restore a single object from a BIAR file. However I suspect that the LCM object dependency algorithms are the leading reason that everything in a BIAR must be imported.

In 4.0 SAP also created the LCM_CLI.BAT file (Command line) as an alternative to the biarengine.jar. It is a similar command line utility but it generates the next generation .BIAR file called a .LCMBIAR file. Much like the biarengine.jar command line it too will force Administrators to import all objects in the .LCMBIAR when using the LCM to restore its contents. You can review the SAP BusinessObjects 4.0 SP5 Administrator guide for more details. I personally don’t like the LCM_CLI because it generates an automatically named job in the promotion management folder each time it is executed. This will lead to a proliferation of LCM jobs that will require Administrators to routinely delete them. Jobs are generated as objects and not instances; therefore the standard instance limit options do not apply.

Until recently I thought all hope was lost in finding an out-of-the box simple and incremental BusinessObjects 4.0 restore solution. However, I recently found a posting on SAP Idea Place (Concerning the LCM) where someone listed a hidden command line option for the Upgrade Management Tool (UMT). The UMT was the version 4.0 replacement for the long-lived Import Wizard. The UMT is locked down and will only allow Administrators to connect to a source repository running XI R2 or XI 3.1 and a target repository running BOE 4.0. In short, it could only be utilized to migrate (upgrade) from a prior version of BusinessObjects to the new version 4.0. Administrators could also use the UMT to import .BIAR files generated in XI R2 or XI 3.1 versions of the Import Wizard or the 3.1 biarengine.jar Command Line Interface (CLI). As I found in the posting above, there was apparently no technical reason that SAP prevented the UMT from being utilized to migrate content from one 4.0 system to another. By simply adding an extension to the UMT startup command line, Administrators can now disable the version checking option within the UMT. Below is an example of the option that can be added to the startup.

-jar upgrademanagementtool.jar -internal_use_only_noversioncheck

The fact that the name of the extension includes the words “internal use only” makes me think that it is an extension that is untested or unsupported. I also wonder if SAP was simply playing nanny and forcing Administrators to use the LCM for 4.0 to 4.0 migrations. Assuming that there are known issues, A strong technical explanation on why the UMT should not be used to promote content would be very welcome. SAP Note 1797144 states that this is not a supported means to promote content from one 4.0 system to another 4.0 system (live to live). However there are no mentions of this being an issue when importing .BIAR files generated with the 4.0 BIAREngine.jar CLI.

Assuming that there are no side effects in using the “no version check” extension, it is now possible to use the UMT to incrementally import objects within .BIAR files generated with the 4.0 biarengine.jar CLI. Based on testing, this process works without any apparent side-effects. Administrators can now develop a batch or shell script to automate the backup of their repository on a recurring basis. By adding the above extension to the UMT startup, Administrators can leverage the UMT to import one or more objects from a version 4.0 .BIAR file.

As I have stated in the article there are potentially a few limitations to using the biarengine.jar file as a backup tool. Attempts to export large repositories in a single execution of the script might fail due to their size. In addition, they might require hours to execute. However, an experienced Java developer should be able to leverage the BusinessObjects SDK and the biarengine.jar to create smaller and smarter extracts of the content for large repositories. For smaller repositories the biarengine.jar has proven to work as a viable backup tool. However, it should not be the only tool used in your backup strategy. The key items I listed at the beginning of the article should be backed up to disk or tape frequently. Regardless of the size of your repository it is still best practice to back up the system using your IT department’s approved tools and methodologies. However adding the biarengine.jar export script to your strategy will give you more flexibility when choosing how to restore lost or corrupt content.

If you would like to see an example Windows .BAT file program I created, please contact me via Twitter@jdh2n or on Linked In . I also have it available on the BOB Board Download Section. The example scripts I provide is only setup to backup < 500 MB of content. If you have more than 500 MB of content, you will need to make changes to the script.

Post navigation

54 comments

Jonathan, Have you tried this with a variety of content? I tried something similar by editing the version number in the exported 4.0 .biar file to be 3.1 instead of 4.0. I found it worked for simple content like reports but other content would error out because the upgrade management tool would attempt to upgrade it.

I have used it to re-import WebI reports, UNV and UNX universes, Crystal Reports, and Program Objects. I was also testing on BOE 4.0 SP5. I’m not sure if that makes a difference. I can see your point though. Given that the content is registered as the 4.0 version in the BIAR (.xml file), I wonder if the UMT skips the upgrade process? Until SAP releases an official statement on this internal extension, there is no way to know. For know I am just taking the risk (based on my own testing) that there are no noticeable issues.

Great summary. It confirms my own experience. You just forgot all the bugs (my favorite: missing BusinessObjects.xml in .biar…).

You could add the version management in LCM. It would be great for restoring lost files with 1) a periodic automatic commit and 2) a recursive commit. Without it it’s much too slow and anti-ergonomic to use regularly. Unless there are hidden tools somewhere…

So SAP/BO has written 4 different tools to import / export (wizard, biarengine.jar,Upgrade Manager, LCM) and not a single one is officially able to restore a lost file if the user makes a mistake. This is totally crazy.

Thanks to the comments. I debated adding the VMS to the conversation but in the end decided to skip it. As you mentioned there are two issues with using VMS. First, it currently requires a manual processes to check-in or add objects to VMS. Second, mass object check-in is extremely slow and prone to time-out. In the end it was no different than using the XIR2 Import Wizard to generate .BIAR files. However, I can see two ways that SAP could enhance VMS to work as a backup solution. First, automatically check-in objects each time the objects is modified. This will institute a more incremental or as needed approach to the process. There will also need to be limits on how many versions of an object are auto-saved. Something similar to the current LIMIT options in BOBJ. Secondly, a background process that can run via scheduling to check-in objects based on a CMS query. Something similar to the BIARENGINE.JAR library. If you couple this with VMS check-in limits, either solution could serve as a viable backup solution.

Very good point. The APOS Storage Center solution will be great for those that need a software package that is consistently supported across multiple versions and has an easy to use interface. Using the BIARENGINE.JAR CLI, LCM CLI, Import Wizard or Upgrade Management Tools might require in house developed code and they will likely carry some risk. SAP could decide at any time to stop supporting any of the methods mentioned in this article. For example, until the recently discovered UMT hack was discovered, there were no incremental restore options available in BOE 4.0. Thanks for the information and links.

as usual your article was thorough and enlightening. Since all of the BOBJ environments I am currently using at my client are Virtual, they are simply creating a snapshot of the VM environment nightly and backing all of them up. Since we are forced to restore all objects via LCM or the command line (and we use LCM here), it would seem to be as practical as creating a BIAR file and with no size constraints. What are your thoughts?

In general the VM backup option will restore the server to a state as of the last backup. If there were any changes to the system, after the last valid backup, those changes would be lost upon restore. I think that every BOBJ Administrator needs to devise an asynchronous backup method (in addition to the full system backup) to contend with the incremental restores. In 4.0, the BIARENGINE.JAR CLI mixed with the UMT hack can facilitate this. In 3.1, the BIARENGINE.JAR and Import Wizard will work as well. There are also 3rd party applications from APOS and 360Suite that are more polished alternatives.

I found today that Explorer Information View Sets can not be promoted with the UMT tool. However, the Information Space, Universe and Connection were able to promote without issue using UMT. After reviewing the IDT Universe I found that none of its changes were overridden either. The LCM was able to promote the Explorer Information View Sets and IDT Universe without issue. In short, it looks like there are some items that are not promoting correctly. As always, use caution with unsupported features.

Hi,
We just tried to restore a BIAR with the Update Manager on our brand new BI4.0 SP6, and the -internal_us_nocheckversion option: this seems impossible to restore a single object. The “Select object” panel is greyed. I would have bet that it works.
Did it change with SP6? Are we missing an option when generating the BIAR with biarengine?
Thanks for any idea.

Thanks for this great post.
I was running some initial tests and when trying to test a restore using the UMT I see the “couldn’t find entry businessobjects.xml” that some users have seen? Can someone shed some light on how to get around this error?

@lastot : a very common bug with XIR2, XI 3.1 and maybe BI4 is the missing businessobjects.xml in the biar. I always check that it is present in all my backups (I rename the .biar as .zip and open it).

The first export I tried was over 1 GB split over two files so I tried a much smaller extract that resulted in only a 2 MB extract. The SQL query I used was “SELECT TOP 100 * FROM CI_INFOOBJECTS WHERE SI_NAME LIKE ‘A0001%’ AND SI_INSTANCE = 0”.

When I run the export I can see as a result of the biarengine.jar command – “CE SDK Exception Occurred : Not a valid query (FWB 00025) (FWM 04002). The query runs fine in Query Builder.

I can also confirm that there is no businessobjects.xml in the zip file.

I have been using this backup method for a while and have also had to restore a few items. The nightly .biar files are about 1.9 GB. I’m not seeing any issues like the one you are describing. Can you try and backup just a single item using the SI_CUID = ‘xxxx’ in your query string?

I’m not sure how, but it sounds like there is a problem in the properties file that feeds the biarengine.jar the CMS SQL query. In that file there is a parameter that specifies the number of queries. Did you specify the correct number in that parameter and (in addition) specify the same number of queries?

Have you tried a query that does not have a string in single quotes within the WHERE clause? I looked at the queries I use with the .biarengine.jar and they all use the SI_ID or SI_OBTYPE in the filter.

Update: I just copied over the batch file and properties file from production (where I have the issue) to development. I entered a valid SI_ID and made the necessary batch/properties changes and ran it. I checked the .zip file and it correctly generated a businessobjects.xml file.

At this point it seems to be related to a problem with the production environment?

I was trying to create a BIAR file from BI 4.0 environment (using version check disabling). I imported all the folders, objects and universes and it has splitted my biar files into 9 biar files (total 4.28 GB) each 512 MB as you mentioned above.

My question is:

While importing specific contents from the biar back into the destination system how I can check which biar contains which specific report or universe? So in case I want to restore a specific report I can directly import from that specific biar part!

The short answer is to make sure your group of.BIAR files all have their original names. Each .BIAR creates a chain to the next BIAR when restoring files. Only one file has the .xml catalog file. Therefore you need all of the files to restore. You can then import the first file and the UMT should find the remaining files as needed. In my example script the files are named TEMP_EXPORT.BIAR . When there are 3 512GB files (for example), you would have TEMP_EXPORT.BIAR, TEMP_EXPORT1.BIAR and TEMP_EXPORT2.BIAR. Using my script, this is the original name of the files. Before importing them with UMT, you have to make sure they have their original names. I have a different script that I run for larger environments. In that script I rename the TEMP_EXPORT*.BIAR files as needed to prevent subsequent backup jobs from overriding the files. However, before I restore content using the UMT, I have to rename them back to their original names.

Hi,
I’ve been using the method for various weeks now. Very nice! I did notice that not all reports are exported, although I expected them to be in the biar file. When I execute the same query in the Query Builder, the reports are shown.
E.g. I want all folders and reports of a specific folder in the public folder root; reports of any kind, although we basiscally only use webi.

exportQuery1=SELECT * FROM CI_INFOOBJECTS WHERE si_id=23
exportQuery2=SELECT * FROM CI_INFOOBJECTS WHERE si_id=11842
exportQuery3=SELECT * FROM CI_INFOOBJECTS WHERE si_ancestor=11842 AND SI_INSTANCE=’0′
exportQueriesTotal=3

Excecuting this query in the Query Builder delivers me all reports; but the Biar enigine is not picking up all reports. I don’t have a clue what the reason is. Any idea?

This looks promising, but I can’t quite get it to work on our XI 3.1 sp5 system. Running on the server, I keep getting “memory allocation error”, “invalid CMS query”, or a BIAR file missing a BusinessObjects.xml file (possibly due to a universe issue). Setting “includeSecurity=false” helped (lots of users). Running on a workstation, I don’t get the above errors – I get a BIAR file that looks good, but when opening with the Import Wizard, it will be empty except for *dependencies* (groups, CALs, …), no folder objects. The CMS query returns a good list of reports… Ideas?

If you exclude dependencies you then have to craft additional queries to get all of the mandatory dependencies. This would include items such as the individual parent folders, security, etc… You .biar file will not open in UMT because it is missing the basic hierarchy of parent objects for the given reports. Not including dependencies makes this process much harder.

Try adding a query the includes all folders, users and groups as a starting point. You might have to add other infoobjects but I would need to do a lot of research to figure out the exact items that are needed.

Just a comment about my own experience.
Backup by biarengine and selective restore by Import Wizard work well in BI 3.1 no problem with that. I migrate my BI platform to 4.2 SP2 in June and I was always able to backup with biarengine and restore with UMT (by changing before the biar version in the meta.properties file in the biar). But with BI 4.2 SP3, grrrrr, It does not work, the biar opens in UMT but no content is listed and restore is not possible. So, I struggle now with lcm_cli.bat to backup as lcmbiar (it works) and Promotion Management to restore all or a part of the lcmbiar in my BI platform (it does not work : timeout when opening the lcmbiar).

it’s incredible that there is no easy backup solution in a tool like BI of an editor like SAP.

Correct. Starting with 4.2 you want to look at the LCM BIAR Command Line Interface (CLI) and selective object import using Promotion Manager. Not a great alternative because large lcmbiar file take forever to process but outside of purchasing 3rd party tools, this is the best option you have.