XER File Slow To Import to P6? Try Cleaning POBS Data

The other day one of our Plan Academy members called our support line with an unusual P6 problem. I’m always happy to help out our members solve P6 problems, even when it’s 4:30pm on the Friday afternoon before I’m off on a full week of vacation….

OK. Full disclosure here.

I had already powered down my laptop for the weekend and was just about to grab something ice-cold and carbonated when the phone rang. I was hoping it was something we could solve quick or that I had already written a solution for. I was tired, and it was hot in the office. The patio was calling. This wasn’t how I was expecting to wrap up my week. Oh well.

The call was was from Doug (not his real name) from Alberta, a Plan Academy member who works for a large national construction company…and he was having problems with an XER file he was sending to a client. Doug’s client was trying to import Doug’s XER File, but it was taking hours to import Not an exaggeration – Doug said it took 6 hours for the client to import his P6 XER file successfully. Something was clearly wrong.

Sometimes you get those problems that just can’t stay away from because it’s something you haven’t seen before and curiousity just takes over. That’s what happened here for me. I was thinking “6 hours to import? How can that possibly be?”

So I made a bargain with myself. If I could help Doug in the next 30 minutes (by 5pm), then I would reward myself with not one, but TWO ice-cold carbonated bevvies. I would be on vacation after all and sometimes you have to motivate yourself!

Now Doug described his project schedule to me. It was a few thousand lines – not uncommon – and it was both resource and cost-loaded. He and his client were using Primavera P6 Professional v8.4.

I did some quick research online about slow importing of XER Files, but there wasn’t any obvious leads there. Here’s what I suggested to Doug.

XER File Slow To Import Into Primavera P6? POBS XER File Data May Be the Culprit

Doug’s XER File was clearly taking too long to import. In this case, I can see 2 things that might need addressing:

The XER File is is having issues.

The client’s machine or database is having issues.

My experience is that most machines and databases don’t run into slow performance issues like they used to 10+ years ago…back in the day when you had too many background processes running on your PC and those programs would grind the processor to a halt. So I was thinking there was likely something going on with the XER file.

Here’s what I recommended to Doug:

A) Rather than use the XER File Format, try the XML format that P6 supports instead.

The XML format functions much like the XER File format. There are some differences, but the project will come across regardless. If the XML file imports quickly, then there’s a problem with the XER file.

B) Remove the POBS Data from the XER File.

This solution was suggested recently on PlanningPlanet where this POBS issue came to my attention (and thanks to the smart P6 experts who suggested it there). I’ll outline the steps to easily removing POBS data soon, including a tip that will be a major timesaver, but first, what did Doug do?

Doug opted to try suggestion B). By removing the POBS data from his XER File, we found that he was able to reduce his file’s size from 25Mb down to 2Mb! Wow! That’s a lot of POBS data!

For Doug, removing the P6 POBS data also reduced the import time from 6 hours to >2 minutes! That’s an exponential improvement!

Yes, we got Doug fixed and I made it to the patio and kicked off a great vacation week. But I’m not finished with this POBS issue. I’m not completely sure why it’s cropping up right now, but I’ve seen enough online grumblings about it to want to tackle it in detail.

So let’s dig in and get down to the bottom of POBS data and XER files in Primavera P6.

What is POBS Data?

Since this is a fairly new issue with P6, I’ve done some homework around the POBS data issue.

We do not utilize the POBS table yet we export/import the data from this
table when completing XER Export/Import. The XER export/import should be
written to exclude this data with XER export/import operations of P6
Professional.

I have a hunch that POBS data is related to Primavera Cost Manager, a now-defunct cost management solution that was once part of the Primavera suite years ago. Primavera Cost Manager integrated tightly with P6 Professional and looking at documentation that still exists, there was a concept of a Performing Organizational Breakdown Structure in Cost Manager. But alas, only Oracle can confirm this.

How Can I Clean POBS Data From My XER File?

[NOTE: a simple alternate solution is provided at the bottom of this post]

Since XER files are actually text files, you can edit them with a text editor like notepad. Here’s what one looks like when you open it up:

We’re going to delete a section of data from the file. To denote a new section, look for a “%T”. To find the POBS section, look for “%T POBS”. That indicates the start of the POBS data in your file. You should see it somewhere near the top of the file in first 100 lines. Above, you can see it on line 29 of the file.

Deleting POBS Data

Basically, the process is to delete all lines starting from (and including) the “%T POBS” line, until the next “%T” in the file. Do not delete the next “%T” line.

So if you’re using notepad, you can simply highlight this huge swath of text and hit delete.

In my example above, there’s less than 200 lines to highlight so it’s pretty easy.

But you’ll see that the amount of POBS data isn’t consistent. Here’s a short 30-activity project schedule as viewed in a text viewer.

The image above might be more typical. The POBS data spans from line 29 to line 4412. Most of this file is POBS data!

So if you’ve got a huge file and a huge amount of POBS to clear out of your XER file, let me know you the quick way to clean it.

2) Find the “%T POBS” line in your file – somewhere in the first 50 lines. Note what line number this is, you’ll need it later.

3) Now, let’s find the next “%T” line. Place your cursor at the end of the “%T POBS” line. Hit CTRL+F (or use the Search menu) and search for the next “%T” occurrence in the file.

4) Make a note of the line number you’ve been taken to. Mine was line 4415.

5) Now we’re going to use some tools to highlight the POBS data since we know the line numbers. Go back to the “%T POBS” line. Place your cursor at the beginning of the line. Now Goto Edit -> Begin/End Select. This essentially tells Notepad++ to start highlighting here.

6) Now hit CTRL+G (or goto Search -> Goto). Make sure to choose Line and type in the line number from step 4) – for me 4415.

7) Now that you’re on the next %T line, tell Notepad++ to end the highlighting here. Goto the Edit menu and uncheck Begin/End Highlighting.

8) Now all of your POBS data is highlighted. Simply tap Delete or Backspace to remove it from the file. Make sure you delete the %T POBS line as well, but not any other lines.

9) Save the file and import.

For the xer file I used in this writeup, removing the POBS data took it from 4489 lines down to 102 lines. And it reduced the file from 564KB down to a mere 21KB.

Why Are Primavera P6 Users Having Issues With POBS Now?

That’s a tough question.

Reports from Oracle say that once Primavera P6 POBS data is in your database, it will be exported along with an XER File. The receiving database will then “be infected” with this data. The data seems to build up in the database (something I’ll cover fixing in my next post) and continues to grow as files go back and forth.

But why is this suddenly an issue now?

One theory I have is that a recent release, possibly Primavera P6 v8.4, had POBS data in the sample database that is often setup at install time. I’m choosing v8.4 as it was referenced in the Bug I mentioned earlier in this post. Users often use the sample database as a production database and the data could be propagating from there.

Whatever my theories are, these POBS issues seem to be appearing more and more.

Conclusion

If you’re having difficulties with performance importing or exporting Primavera P6 XER files, try cleaning the POBS data from them. Yes, you can use windows Notepad, but if the file is large, you’ll be scrolling for an hour. Try using the techniques I’ve shown above along with the Notepad++ text editor to save you time.

I fully recognize that what I’m offering here isn’t truly a permanent solution. Constantly editing XER files to have a reasonable import time isn’t viable and it is a product issue that needs to be addressed by Oracle. Here’s hoping this gets you by in the meantime.

Update – 2016

It turns out that using notepad is really too tedious, although some commenters on this post had some great suggestions for making that process easier. But let me make life even easier by recommending a simple utility created by Plan Academy reader and senior planning engineer Ahmet Kochan. Ahmet’s POBS Cleaner is dead simple to use and free! POBS Cleaner can clean 1 or more XER files at the same time. It creates a newly cleaned version of your XER file with the extension __cleaned.xer in case you ever need the original file.

There are some other tools on the market that clean POBS from XER files, but Ahmet’s tool is free and so simple to use that I highly recommend it.

About Michael Lepage

Michael is an avid project controls blogger and is the Chief Learning Officer here at Plan Academy. Michael has taught 1000s professionals how to use project controls software like Primavera P6 over the past 10 years through his online courses and tutorials. Michael is a member of AACE, the Guild of Project Controls and holds his PMP certification from PMI.

You Might Also Like

Comments

Michael
I’m using POBS Cleaner from Emerald Associates. I was using your quick method, but this is even quicker. It’s a free download and scrubs a 25mb file clean in less time than it takes to open the .xer in Notepad. There is also a POBS cleaner utility in xertoolkit

I was kind of curious on tour post; I am a regular reader of this posts, but I can share with all of you a courious experience from the past: there was 2003 or 2004 maybe, and I has exactly the very same situation, but there were not hours, the hours became a weekend trying to import a file into a powerful laptop. In a nutshell, the issue was not the number of lines of the schedule, but the type of resource availability assigned per activity (I mean, many resources per activity). What I discovered, per a desperate mistake or casualty was this: if I were able to export first of all from the source all the resources in an .XER file and then import them in the new database, it was more or less quick (by that time, it took one minute…) then I exported the project and imported it into the new database, and it took only five minutes the import who had lasted days!!!!!.
Well, this happy coincidences happens. Hope this can become a hint for future cases, in particular working with old Primavera Setups, migrations and that type of cases.
Regards everybody. Marcos.

Marcos, thanks for sharing your experience and solution with us. We’ll have to see if we can confirm that the issues you mention from 2003/04 have been fixed in newer versions of Primavera P6 Professional.
Thanks again!
Michael

I created a tool to clean POBS data from XER files. Multiple XER files can be selected and cleaned at once. Original files will be kept and new clean copies will be created with a “_Clean” suffix. It is a totally free tool to help community to prevent long import times and waste of database space. Here is the link for download : https://www.dropbox.com/s/qrvxchtwsvs5778/POBS%20Cleaner.rar?dl=0

I used to import a project in about 45 minutes whose file size is about 31 MB. Then I cleaned the file using your method and I shaved off 53,200 lines from a total of 207,000 lines, which reduced the file size to around 23 MB.

Now, I am importing in about 20 minutes.

Is there anyway for me to reduce the file size further and hence reducing the importing time?

I tested Ahmet’s tool and found a problem with it. I wrote my own little tool in C, and it simply copies byte for byte the file, skipping the offending information.

Ahmet’s tool apparently reads characters, and those characters are interpreted according to local language, or character sets, or whatever. You can see the problem in the line corresponding to currencies, in which the pound sign is changed and altered. It also mangles characters with accents, and imagine many characters not in the standard english character set.

So a little revision of the tool has to be done.

Nice interface! Mine does not have one it is just a command line C program, but it is so fast that the DOS windows hardly gets noticed. Once is is used, it gets added to the windows explorer context menu, so using it is a breeze for individual files.

Instead of scrolling the normal way with the mouse, once you’ve found the end of the POBS data click with the mouse before the next %T and use the side scroll bar to navigate back to the top of the data in less than a second.
I use notepad to routinely check and clean XERs and this technique takes me less than 10 seconds even on 25MB files!
Thanks for the instructions, this has now become the basis of a work instruction on POBS for our planners. I include a link back to this page. 🙂