Using the File Manager from MP Tasks

by by Matthew Xavier Mora
Edited by Peter N Lewis

How to get data in and out of your MP Task

One of the most common complaints I received while supporting MP Library in Developer Technical Support was that you could not call the toolbox from an MP Task. Multiple preemptive tasks are not much use if you cannot get data into and out of them efficiently. This article shows one way to get data into and out of an MP Task using the file manager, however the techniques used here can be modified for other I/O operations (like audio, video or networking). "But, I thought you couldn't call the file manager from MP Tasks?" Well, you thought wrong. :-) Read on...

Background

In the early version of the MP library there was no easy way to call the toolbox because the MP Library was designed to be compatible with Copland's kernel tasking model. Since the Mac OS toolbox wasn't going to be available from Copland's kernel tasks, the same was done for the Mac OS version of the MP Library. After the Copland project was canceled it was decided to publish a few previously undocumented routines that let you work with the Mac OS toolbox from a task. One of the routines published is MPRPC. MPRPC is a remote procedure mechanism that lets you specify a routine to execute at a time when it is safe to make toolbox calls. It does this by suspending the task and then executing the supplied routine during SystemTask() time. The task is suspended until MPYield is called or until any toolbox routine calls SystemTask(). MPRPC is used internally in the MP Library to implement calls such as MPAllocate and MPAllocateSys (which is why these are blocking calls).

The code in this article is based on the MP File Library that I wrote before the MPRPC call was published. The MP File Library used MPQueues to communicate with the main task and have it execute toolbox commands.

Review

Let's review some of the MP programming guidelines and how adding blocking calls can change some of these guidelines.

Your tasks should do a considerable amount of work. If not, the benefits of using MP will be lost in the overhead of the scheduler and task switching. Adding blocking calls to your tasks adds additional overhead. The main benefit here is that by being able to call the toolbox from an MP Task your task can run autonomously from the main application thread. This results in a better user interface response from the application since the application can off load a time consuming task and call the main event loop more often giving the blocking calls more time to execute the toolbox calls.

You should allocate no more than (MPProcessors() - 1) number of tasks. While it is important to keep the number of tasks low so that task switching does not impact performance, adding blocking calls to a task will also hurt performance if nothing calls MPYield(). "Wait, I thought MP Tasks were preemptive?" Yes they are but if the task is blocked waiting on a resource, the resource can't be released until the main thread calls WaitNextEvent() or another task calls MPYield(). That being the case, if you use MPRPC calls it is a good idea to bend the n-1 rule and create an extra task that can help unblock any waiting tasks.

You should use MPQueues or MPSemaphores when communicating with MP Tasks. This does not change if you are using MPRPC so you should heed this warning.

Get On With It

OK, so how do I call the File Manager? For this simple example I will implement five MP calls that duplicate FSpOpenDF, FSClose, FSRead, FSWrite and SetFPos. Those are all the calls we need for a simple demo. We'll start with a FSRead type call.

First lets define a structure to pass to the MPRPC callback routines that will hold the values that we need to handle all the File Manager calls.

typedef struct FSParamRec{
short refNum; // file ref num
long count; // for read
Ptr buffPtr; // for read
FSSpecPtr spec; // for open
short permission; // for open
short posMode; // for setfpos
long posOff; // for setfpos
OSErr result; // error result
} FSParamRec,*FSParamRecPtr;

Now lets implement the callback routine that gets called at main application time. This routine will be executing at SystemTask time which means you can call any toolbox routine except for any routines that might call SystemTask() again.

First we check to make sure the parameter that was passed in is not nil then we simply call the File manager's FSRead call. When FSRead returns, we put the result into the result field and then return the pointer to the struct that was passed in.

First we allocate a FSReadRec on the stack that gets passed to MPRPC. We fill out the fields in the struct with what was passed into us, call MPRPC and wait for the result. Then return the result to the caller.

That's it. You can now call FSRead from an MP Task. Using the same basic techniques you can implement all the file manager calls you need to get data in and out of your tasks. Now lets see how the task calls the new routines.

The MP Task itself is pretty straight forward as a result of the blocking I/O calls since there are no flags or spin loops to worry about.

In our task we immediately block (as every task should) on MPWaitOnQueue waiting for the FSSpecPtr from the application. When MPWaitOnQueue returns, we check the file spec pointer to make sure it is not nil and precedes to open the file. We set the file position to the beginning of the file and start the read operation. Notice that for either the async or non async case the code is still the same. The only difference is to the application since the task is blocked until the read completes. After the read completes, this is where you would do some serious processing on the data. It is very important that you do a lot of processing to minimize the overhead of the blocking I/O calls. The demo doesn't do any processing so the next thing to do is to set the gCount variable indicating we got the data making sure the write get synchronized with the other processors. We close the file and return. Returning from the task kills the task. You might want the task to hang around and be ready to process another file. In that case set up a while loop on MPWaitOnQueue. You can set the exit termination condition to be a nil FSSpecPtr.

Adding More Features

The FSRead technique is good at getting data in and out of your task but you basically block the entire application while it waits for the FSRead to complete. We can improve this by using asynchronous file manager calls to keep from blocking the main application task while executing a Read call.

We need a different structure to do an async read. I wrap the new struct around a ParamBlockRec to contain the flag needed to signal the completion of the read call.

It just sets the callPending flag, signals the processors to sync up and returns. We can't set a MPQueue or a MPSemapore in here (which would be the better way to do it) because MP Library calls can't be called at interrupt time.

Handling asynchronous routines gets a little more complicated but it saves having to make sure other tasks are running just to call MPYield(). Now you might be thinking why are we using a flag when you could just spin on ioResult? Read on to see why this is not good idea...

Gotchas

When working with multiple processors some conventional Mac programming wisdom goes out the window. A good case in point is when ioResult is set. Normally ioResult is set to 1 to indicate a call is pending. The last thing the file manager does before calling the ioCompletion routine is to set ioResult to the error result from the parameter block call. None of this really changes when multiple processors are involved but the non-main processors are not bound by the 68k enable/disable interrupt tricks. So if your MP Task spins on ioResult waiting to see when the read is complete (ioResult != 1) your task starts to execute before the file manager is done with the parameter block. After the file manager sets the ioResult field, it gets the ioCompletion routine's address from the parameter block and jumps to it.

In our case the parameter block in on the stack and when the task unblocks, the stack is released and your task crunches merrily along where a parameter block used to be (and is still in use by the file manager). The second processor could be a 200 MHz CPU and in the time the file manager has set ioResult and jumps to the completion routine, your task could be millions of instructions away using the memory where the parameter block used to be.

The same is true for many of parts of the Mac OS Toolbox. The critical region technique of disabling interrupts does not work well when multiple processors are involved. So be careful and always use MPQueues, MPSemaphores and MPCriticalRegions to coordinate your various tasks.

Another gotcha may be in your thought process. You might be thinking that it would be cool to use the same techniques mentioned in the article to make every Toolbox call available from MP tasks. While this is possible, and would make your task code a lot easier to write, it is not a good idea. The benefits of multiprocessing only come from careful algorithm design, implementation, and profiling. Guideline #1 mentioned above says that your task should do a considerable amount of work to gain any performance improvements. Having your task block, waiting on a bunch of toolbox calls is not going to improve performance. On the other hand having to load all the data you need into memory before your task can start running may not be feasible either. This is where a careful balance of having main processor moving data in and out of your task while processors n+1 crunch along can really pay off.

More MP Information

Hopefully, this article piqued your interest in Multiprocessing. If you want more information there are a number of documents and resources to help you get the most out of MP. An introduction to MP systems was printed in MacTech March '96, TechNote 1071 on Multiprocessing is on the web http://www.apple.com/developer/ and I have set up a MP mailing list where developers can ask questions on MP programming issues. The list includes folks like the senior engineer who wrote the MP Library as well as Chris Cooksey and myself. For subscription information you can go to my web site http://www.best.com/~mxmora/mxm.html. Also, don't forget Apple Developer Technical Support is there for information about MP's past, present and future.

Summary

I hope this article shows how easy it is to get data into and out of your MP tasks. Use this information wisely and you should see some real improvements in your applications performance. You can use these techniques to work with other I/O technologies like networking, graphics and sound. I have created a MP File Library that you may want to use based on some of the techniques used in this article. It uses a slightly more complicated model for better performance. You can download a copy of my MP File Library from my web site at http://www.best.com/~mxmora/software.html. Good luck, and happy multiprocessing.

Matthew Xavier Mora was the engineer responsible for answering questions on Multitasking support in Apple's Developer Technical Support. As a self proclaimed evangelist for the Multi-processing API library he was instrumental in convincing both third-party developers and Apple engineers to implement MP support in their software. If you were ever thinking about moving into the Silicon Valley, consider that this article was written while Matt was sitting all night outside a school building waiting to register his son for pre-school. When Matt is not out doing crazy things like that you can reach him at mxmora@best.com.

Community Search:

MacTech Search:

Software Updates via MacUpdate

Alfred 3.4.1 - Quick launcher for apps a...

Alfred is an award-winning productivity application for OS X. Alfred saves you time when you search for files online or on your Mac. Be more productive with hotkeys, keywords, and file actions at... Read more

Maintenance 2.2.7 - System maintenance u...

Maintenance is a system maintenance and cleaning utility. It allows you to run miscellaneous tasks of system maintenance:
Check the status of the hard disk
Repair permissions
Run periodic scripts... Read more

NeoOffice 2017 - Mac-tailored, OpenOffic...

NeoOffice is a complete office suite for OS X. With NeoOffice, users can view, edit, and save OpenOffice documents, PDF files, and most Microsoft Word, Excel, and PowerPoint documents.
NeoOffice 3.x... Read more

SyncTwoFolders 2.2.4 - Syncs two user-sp...

SyncTwoFolders simply synchronizes two folders. It supports synchronization across mounted network drives and it is a possibility to run a simulation showing in a log what will be done.
Please visit... Read more

Hopper Disassembler 4.2.13- - Binary dis...

Hopper Disassembler is a binary disassembler, decompiler, and debugger for 32- and 64-bit executables. It will let you disassemble any binary you want, and provide you all the information about its... Read more

iFFmpeg 6.4.3 - Convert multimedia files...

iFFmpeg is a comprehensive media tool to convert movie, audio and media files between formats. The FFmpeg command line instructions can be very hard to master/understand, so iFFmpeg does all the hard... Read more

Firefox 55.0.2 - Fast, safe Web browser.

Firefox offers a fast, safe Web browsing experience. Browse quickly, securely, and effortlessly. With its industry-leading features, Firefox is the choice of Web development professionals and casual... Read more

FileZilla 3.27.1 - Fast and reliable FTP...

FileZilla (ported from Windows) is a fast and reliable FTP client and server with lots of useful features and an intuitive interface.
Version 3.27.1:
Fixed Vulnerabilities:
Change client... Read more

Merlin Project 4.2.7 - $289.00

Merlin Project is the leading professional project management software for OS X.
If you plan complex projects on your Mac, you won’t get far with a simple list of tasks. Good planning raises... Read more

Dashlane 4.8.4 - Password manager and se...

Dashlane is an award-winning service that revolutionizes the online experience by replacing the drudgery of everyday transactional processes with convenient, automated simplicity - in other words,... Read more

Latest Forum Discussions

War Wings is the newest project from well-established game maker Miniclip. It's a World War II aerial dogfighting game with loads of different airplane models to unlock and battle.
The game offers plenty of single player and multiplayer action. We... | Read more »

How to win every 2v2 battle in Clash Roy...

2v2 is coming back to Clash Royale in a big way. Although it's only been available for temporary periods of time, 2v2 has seen a hugely positive fan response, with players clamoring for more team-based gameplay.
Soon we'll get yet another taste of... | Read more »

Roll to Win with Game of Dice’s new upda...

Joycity’s hit Game of Dice gets a big new update this week, introducing new maps, mechanics, and even costumes.
The update sets players loose on an exciting new map, The Cursed Tower, that allows folks to use special Runes mid-match. If you feel... | Read more »

Bottom of the 9th (Games)

Bottom of the 9th 1.0.1
Device: iOS iPhone
Category: Games
Price: $4.99, Version: 1.0.1 (iTunes)
Description:
Play the most exciting moment of baseball in this fast-paced dice and card game!
| Read more »

The best apps for viewing the solar ecli...

If you somehow missed the news, many parts of the United States will be witness to a total solar eclipse on August 21 for the first time in over 90 years. It'll be possible to see the eclipse in at least some capacity throughout the continental U... | Read more »

The 5 best mobile survival games

Games like ARK: Survival Evolved and Conan Exiles have taken the world of gaming by storm. The market is now flooded with hardcore survival games that send players off into the game's world with nothing but maybe the clothes on their back.
Never... | Read more »

Supercell has been teasing fans with 2v2 windows that allow players to team up for limited periods of time. The Summer of 2v2 was just this past July, but players are already clamoring for more of that sweet, sweet team-based action.
The fans have... | Read more »

The best deals on the App Store this wee...

It seems like the week's only just started, and yet here we are with a huge pile of discounted games to sort through. There are some real doozies on sale this week. We're talking some truly stellar titles. Let's take a look at four of the best... | Read more »

The iOS community is in late summer holding mode awaiting the September arrival of the iPhone 8 and iOS 11. iOS 11 public betas have been available for months — number six was released this week —... Read more

Samsung Electronics Launches New Portable SSD...

Samsung Electronics America, Inc. has announced the launch of Samsung Portable SSD T5 – its newest portable solid state drive (PSSD) that raises the bar for the performance of external memory... Read more

TrendForce Reports YoY Gain of 3.6% for 2Q17...

Market research firm TrendForce reports that the global notebook shipments for this second quarter registered a sequential quarterly increase of 5.7% and a year-on-year increase of 3.6%, totaling 39.... Read more

Amazon has new 2017 13″ 2.3GHz/128GB Silver MacBook Pro on sale today for $100 off MSRP including free shipping. Their price is the lowest available for this model from any reseller:
– 13″ 2.3GHz/... Read more

WaterField Unveils Collaboratively-Designed,...

In collaboration with customers and seasoned travelers, San Francisco maker WaterField Designs set out to create the preeminent carry-on system to improve the experience of frequent fliers. The... Read more

Miya Notes Mac-Client for Google Keep (Launch...

MacPlus Software has announced te launch of Miya Notes for Google Keep 1.0, a powerful Mac-client for Google Keep. Millions of people use Google Keep on their phones and online, but a convenient Mac... Read more

Apple refurbished iMacs available starting at...

Apple has previous-generation Certified Refurbished 2015 21″ & 27″ iMacs available starting at $849. Apple’s one-year warranty is standard, and shipping is free. The following models are... Read more

MacTech is a registered trademark of Xplain Corporation. Xplain, "The journal of Apple technology", Apple Expo, Explain It, MacDev, MacDev-1, THINK Reference, NetProfessional, Apple Expo, MacTech Central, MacTech Domains, MacNews, MacForge, and the MacTutorMan are trademarks or service marks of Xplain Corporation. Sprocket is a registered trademark of eSprocket Corporation. Other trademarks and copyrights appearing in this printing or software remain the property of their respective holders. Not responsible for typographical errors.

All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.