Which means BigClass's declaration requires all these other includes to be loaded before the declaration can appear. Even by users of BigClass who otherwise have no need to care about all those subcomponents.

What I'm aiming to do, is somehow divide up BigClass so that users of the class only need to include BigClass's ".h" file. And only the implementation of BigClass must include all the components. Which means BigClass's declaration in the user-visible ".h" file can't mention those subcomponents. I've seen these kind of issues addressed on various language forums and books. And usually the answer is to forward-declare the component classes and use pointers instead of whole objects.

But there's a twist to it: I'd like to do it using as few dynamic allocations as possible. Ideally zero. The various components are all relatively small, and there's quite a few of them. The pointer to some of them (on a 64-bit arch) would be larger than the object itself! It is wasteful to dynamically allocate them just so they can be pointed to, and to invoke allocation so often when the whole thing could be automatically allocated on the caller's stack, were it not for the many ".h" files necessary to fully define its layout.

The direction I'm thinking about -- but I'm not sure if this is the best way to get there -- is to define BigClass to have all its usual methods, but in the private area, put only void *internal_state[N];. Make N big enough to reserve enough space to hold the real data without having to specify what's in it. (I have some ideas for working out what N should be but in the interest of brevity let's assume the build system can figure out N.)

The implementation file defines BigClassImpl, which contains the actual component classes and all the associated includes. It also defines the methods for BigClass to consist of a cast of the internal_state array to BigClassImpl * and invoking the corresponding method in BigClassImpl.

Does this make any sense so far? The odd requirements come from embedded-systems-aware design goals, and the fact that so far dynamic allocation hasn't been used and am reluctant to introduce it merely to save some #includes.

Make sense? Is there a cleaner, less crufty way of doing this that won't cause native C++ coders to tear out their eyeballs? What about using a common base class, would that help in any way here? Thanks!

I'm not so much a fan of your idea; by introducing build scripts and implementations you are making the solution more complex than it needs to be.

What your idea looks like in the first place is forward declarations. A forward declaration basically declares a type before defining it; so, when you use a forward declaration the size of the type is not known yet. Therefore, in your same code, you introduce pointers such that the size of the classes do not matter; but that isn't enough and isn't a solution here, because you made an additional restriction stating that it should not use dynamic allocations and that it should not care about the subcomponents, therefore you can not forward declare them.

Quote:

Which means BigClass's declaration requires all these other includes to be loaded before the declaration can appear. Even by users of BigClass who otherwise have no need to care about all those subcomponents.

It sounds to me like you either want to use polymorphism (keep clicking the next arrow near the bottom to read all the pages, I'm not referring to the linked page alone) or template classes here depending on what you want your classes to do; either way, it will help you separate your user's implementation from the original eclass without trying to implement these concepts manually again.

I can't give you more information as I'm not sure what your classes are meant to do; in other words, I'm staring at an XY probem, where X is your goal and Y is your implementation, and I am only aware of Y. Without referring to too much programming terms, what are you trying to do here? Could you give an example in layman terms?

You probably want a uint8_t internal_state[] rather than a void *, so that you can control the size more precisely. The general idea looks like it will work, but it seems like a lot of work to avoid those includes. Are they so expensive this is worth all the complexity? Are pre-compiled headers an option to reduce the cost? Be aware that, with the code you showed, none of the sub-objects will execute destructors.

One other way you could do this that would look a bit nicer would be to conditionally define the class, like so:

The destructor must be out of line and implemented in a class file which defines IMPLEMENT_BIG_CLASS. If you let it be inline, it will be inlined in files which think the only member to destroy is uint8_t internal_state[]. I think even my proposal is ugly and should be avoided where possible, but if some of those helper includes are expensive enough, I can believe this could be justified.

One other way you could do this that would look a bit nicer would be to conditionally define the class. ... I think even my proposal is ugly and should be avoided where possible, but if some of those helper includes are expensive enough, I can believe this could be justified.

Once another part of the application defines IMPLEMENT_BIG_CLASS, you suddenly have much bigger objects and much more includes everywhere even when you intend not to have them; or even worse, you could end up with clashing definitions of BigClass depending on where you define it; therefore this is a bad design.

The way you describe the problem could be implemented with polymorphism; you basically have one file NormalClass.h which defines NormalClass without the helper classes and then you have another file BigClass.h which defines BigClass that derives from NormalClass with adds the helper classes. That way, your consumer can choose which header file to inherit; where NormalClass is expected, a BigClass can be passed but not the other way around. The benefit here is that the BigClass provides the functionality that NormalClass already provides and that BigClass can choose to add and / or override functionality.

It's way too much to explain the whole polymorphism concept here, but the link I gave above explains it fairly well; if not, consider grasping a well known C++ book and reading it in there.

Here's a definition from DuckDuckGo that summarizes it fairly well: "Polymorphism in object-oriented programming, the ability of an interface to be realized in multiple ways."

One other way you could do this that would look a bit nicer would be to conditionally define the class. ... I think even my proposal is ugly and should be avoided where possible, but if some of those helper includes are expensive enough, I can believe this could be justified.

Once another part of the application defines IMPLEMENT_BIG_CLASS, you suddenly have much bigger objects and much more includes everywhere even when you intend not to have them; or even worse, you could end up with clashing definitions of BigClass depending on where you define it; therefore this is a bad design.

The layout I proposed guards against BigClass being too small, and could easily be adapted to require that it be precisely the right size. Each masked class like this would need its own preprocessor symbol, and you would need a guarantee that the build system sets the size parameter correctly. If you are concerned, you could encode the size directly into the name of the class to force a linker mismatch in the unlikely case someone builds different modules with different sizes. If BigClass is forced to be the proper size in all cases, and all access to its data is handled through accessors in a file which defines the implementation macro, then how can a caller misuse it?

TomWij wrote:

The way you describe the problem could be implemented with polymorphism; you basically have one file NormalClass.h which defines NormalClass without the helper classes and then you have another file BigClass.h which defines BigClass that derives from NormalClass with adds the helper classes. That way, your consumer can choose which header file to inherit; where NormalClass is expected, a BigClass can be passed but not the other way around. The benefit here is that the BigClass provides the functionality that NormalClass already provides and that BigClass can choose to add and / or override functionality.

As I understand the problem posted, polymorphism is not appropriate here. Akkara wants to define BigClass on the stack in arbitrary modules, some of which do not know the true composition of the underlying class. He knows he specifically wants BigClass, with all its helper objects and extended methods. The extended methods in turn require that the helper objects exist in the class on which they are called.

The layout I proposed guards against BigClass being too small, and could easily be adapted to require that it be precisely the right size. Each masked class like this would need its own preprocessor symbol, and you would need a guarantee that the build system sets the size parameter correctly. If you are concerned, you could encode the size directly into the name of the class to force a linker mismatch in the unlikely case someone builds different modules with different sizes. If BigClass is forced to be the proper size in all cases, and all access to its data is handled through accessors in a file which defines the implementation macro, then how can a caller misuse it?

This feels like the C way of doing things; when you're using keywords like "guards", "preprocessor symbol", "encode size into name of class" and "implementation macro" I'm not sure where the C++ concepts have gone too.

Hu wrote:

As I understand the problem posted, polymorphism is not appropriate here. Akkara wants to define BigClass on the stack in arbitrary modules, some of which do not know the true composition of the underlying class.

This part sounds more like the need for templates.

Hu wrote:

He knows he specifically wants BigClass, with all its helper objects and extended methods. The extended methods in turn require that the helper objects exist in the class on which they are called.

This is what polymorphism can do.

There are some conflicting thoughts here, as well as some missing information; I think we should wait for more clarifying information before we pick a particular implementation. My example of polymorphism was based on your example, not on Akkara's; I don't know what Akkara is really after.

Thank you all for the replies. It'll take a bit of time to digest it all. In the meantime here's some more context of what I'm trying to do.

The specific thing I'm trying to do is write a "flac" decoder library in C++.

The goals are mostly educational: to improve my C++ by practicing with a problem that is meaty enough to require thought but no so elaborate that I'll never finish.

Another goal is to understand the language from the viewpoint of a embedded systems programmer. How well does it work when there's these extra constraints? Find out, "is this a tool I will want to use in what I normally do?".

With that context, the "BigClass" is the top-level FlacDecoder class itself.

This class uses as components: a bitstream reader/parser class; a metadata parser class; a frame-header class; a subframe class; various codec classes; and probably one or two others I'm not remembering right now. Each of those classes in turn have instances of the error-handler class (as self-contained members, no multiple inheritance (yet)).

The component classes are smallish in a sizeof() sense. But they are quite large in the verbosity sense. There's lots of enums full of magic numbers for the various frame types, for the various metadata types, for the various state-transitions in the parser, for the error codes, and so on. There's also a lot of code to gracefully handle all the things that might go wrong (or at least try to) - missing headers, early EOF, mismatches between what the header claims and what's in the file, and so on.

None of these component classes (with the possible exception of the bitstream reader) are particularly useful on their own. They are there only for readability and dividing a big problem into smaller easier-to-understand pieces. Each instance of the flac decoder uses exactly the prescribed set of component classes in exactly the same way, so dynamic allocation doesn't buy anything features-wise. Dynamic allocation could allow for easier use of forward-declarations. But with all the memory leak and pointer problems out in the wild, I take a view that if I don't need to allocate dynamically, I very much prefer not to, even when not doing embedded code. It saves a lot of headaches from having to make sure the destructor is written properly to making sure the assignment methods work right.

But for a user of this library, it would be nice if only need to #include "flac", not have to precede that with a half-dozen #include "flac-header"; #include flac-bitstream"; and so on.

I could put all those subclasses in the main flac.h file. That makes it a rathar large include file that contains a lot of details that are only used in one or two implementation files and nowhere else.

Hence, this thread. I was wondering what's the best way to achieve clarity and best-practice separation of data while also meeting common embedded-systems goals. Perhaps it can't be done to 100% satisfaction on both counts. I'm here to find out.

Admittedly, my c++ isn't as sharp as it could be, particularly when dealing with libraries, But if you want to use "BigClass" in any bit of code, isn't it sufficient to simply use #include "BigClass.h" where BigClass.h has all the includes for its parts? When you use it, the preprocessor should read the specified header and everything it includes, so those files just need to exist. And since the library should be pre-compiled there should be no need of anyone to know where "BigClass.cpp" or any other source files are.

EDIT: For example, in the Header file for CERN root's TGraphErrors.h, it includes TGraph.h, however I know from experience that I don't need to include TGraph.h in code using an error plot._________________First things first, but not necessarily in that order.

The Doctor is correct. C++ headers can, and usually should, include the prerequisites so that a source file that includes just the header and nothing else compiles cleanly. If you remove the requirement that users of FlacDecoder not know the types of the internals, this becomes a fairly common programming problem.

Regarding memory management: use the standard library and augment it with Boost where necessary. Standard containers give you dynamically sized arrays, doubly linked lists, and smart pointers. Avoid use of std::auto_ptr in favor of std::unique_ptr (C++11) or boost::scoped_ptr. A pointer entrusted to one of those will be freed when the smart pointer destructor executes. If the smart pointer is a member, then it will automatically have its destructor execute when the destructor of the containing class executes.

In general, if you find that you need to use delete p; in your program logic, you are either doing something fancy or you are not using the tools well.

The usual way to go in my experience (which isn't representative by any means) is to move all your private data classes into a BigClassPrivate helper class, and include that in your BigClass via pointer (pimpl idiom). So you'll end up with one dynamic allocation in BigClass independent of the number of helper class in BigClassPrivate

With that context, the "BigClass" is the top-level FlacDecoder class itself.

This class uses as components: a bitstream reader/parser class; a metadata parser class; a frame-header class; a subframe class; various codec classes; and probably one or two others I'm not remembering right now. Each of those classes in turn have instances of the error-handler class (as self-contained members, no multiple inheritance (yet)).

Genone wrote:

The usual way to go in my experience (which isn't representative by any means) is to move all your private data classes into a BigClassPrivate helper class, and include that in your BigClass via pointer (pimpl idiom). So you'll end up with one dynamic allocation in BigClass independent of the number of helper class in BigClassPrivate

This in not so much more about C++ but rather about design. You are dealing with the The Blob here depending on what those components are doing; so, you will not want to create the class in its current way and decide how to refactor it such that you get an alternative approach. You probably want to at least have multiple classes, and not just one; because one appears to be complex as was shown before.

Akkara wrote:

The component classes are smallish in a sizeof() sense. But they are quite large in the verbosity sense. There's lots of enums full of magic numbers for the various frame types, for the various metadata types, for the various state-transitions in the parser, for the error codes, and so on.

This describes the data classes you read about in The Blob.

Akkara wrote:

There's also a lot of code to gracefully handle all the things that might go wrong (or at least try to) - missing headers, early EOF, mismatches between what the header claims and what's in the file, and so on.

It would probably benefit to separate this into validator classes.

Akkara wrote:

None of these component classes (with the possible exception of the bitstream reader) are particularly useful on their own.

This describes the data classes you read about in The Blob.

Akkara wrote:

They are there only for readability and dividing a big problem into smaller easier-to-understand pieces.

They would otherwise be in The Blob; so, you already did one step to make The Blob smaller but it is still big as it is. You made the data part smaller, more readable and easier to understand.

Akkara wrote:

Each instance of the flac decoder uses exactly the prescribed set of component classes in exactly the same way, so dynamic allocation doesn't buy anything features-wise.

Agreed; though, what if you want to replace a class at runtime when you want to load a different class? Not sure if FLAC changes; but, different versions of its standard might be something you need to deal with.

Akkara wrote:

Dynamic allocation could allow for easier use of forward-declarations.

Agreed.

Akkara wrote:

But with all the memory leak and pointer problems out in the wild, I take a view that if I don't need to allocate dynamically

This should never be the reason to view it as such; if you have fear for memory leak and pointer problems, you're doing something wrong in terms of how you use the language and its libraries.

In the first place, avoid pointers when you can; see RAII for an example, although you can interpret the RAII concept for pointers as well where you simply explicitly write the delete call as well at the end of the same function (or in the destructor if it was in the constructor). There are smart pointers and shared pointers for all those kind of situations where you have to pass pointers to other sections of the application; so, if you use them well you should not stumble upon memory leak and pointer problems. And if you do because of bad programming; don't worry, you can always use a tool like valgrind to find where you have made such errors.

Akkara wrote:

I very much prefer not to, even when not doing embedded code. It saves a lot of headaches from having to make sure the destructor is written properly to making sure the assignment methods work right.

This is writing wrong code on purpose; as a result, you are making things more complex or impossible than they need to be causing its own headaches (eg. this thread) as well.

You should lay the focus on writing them right; because, that's essentially the thing that is expected. Having the RAII concept in mind helps with that...

Akkara wrote:

But for a user of this library, it would be nice if only need to #include "flac", not have to precede that with a half-dozen #include "flac-header"; #include flac-bitstream"; and so on.

That has nothing to do with the design; it's quite easy, you could include all those other header files from flac.hpp file without even declaring or defining anything in flac.hpp.

Though, consider that, some users of your library will consider this a bad design; why should they include a lot of code that they do not end up using? It slows down compilation for no good.

Please note, that once your library is compiled as a library; you will not need to declare every function and class in the header file you provide but solely those that the user will use. Some people explicitly write different header files flac.hpp and flacImpl.hpp where the former is to be distributed towards the users and the latter contains more details like private function calls and what not that the user does not need to be aware about.

Akkara wrote:

I could put all those subclasses in the main flac.h file. That makes it a rathar large include file that contains a lot of details that are only used in one or two implementation files and nowhere else.

There are includes for this purpose; there is no need to make such big file, it gains nothing. And if you intend to do that for documentation, consider to rather use a documentation generation tool like Doxygen.

Akkara wrote:

Hence, this thread. I was wondering what's the best way to achieve clarity and best-practice separation of data while also meeting common embedded-systems goals. Perhaps it can't be done to 100% satisfaction on both counts. I'm here to find out.

For a start, you should look into the Blackboard design pattern; where you start with "How can the task of compiling or constructing a dataset be divided up into smaller tasks?" which you could easily interpret as "How can the task of decoding a FLAC file be divided up into smaller tasks?".

What it will do is have your main class that you present to users use function classes instead of defining the logic itself; and all those functional classes will talk with just one data class. This in a way that your main class no longer uses data classes but instead functional classes that make it a much better design. (Perhaps they also have some of their own local data classes; but that's not part of the bigger picture.)

The direction I'm thinking about -- but I'm not sure if this is the best way to get there -- is to define BigClass to have all its usual methods, but in the private area, put only void *internal_state[N];. Make N big enough to reserve enough space to hold the real data without having to specify what's in it. (I have some ideas for working out what N should be but in the interest of brevity let's assume the build system can figure out N.)

The implementation file defines BigClassImpl, which contains the actual component classes and all the associated includes. It also defines the methods for BigClass to consist of a cast of the internal_state array to BigClassImpl * and invoking the corresponding method in BigClassImpl.

Recently I was writing an agent based simulation and for performance reasons (move semantics) I wanted to use the pimpl idiom. But the agents were short-lived and the dynamic allocation was hurting performance. So instead of this:

Code:

class agent {

struct representation {
type1 data1;
type2 data2;
...
typeN dataN;
};

unique_ptr<representation> rep;
};

I did this:

Code:

class agent {

struct representation {
type1 data1;
type2 data2;
...
typeN dataN;

static stack<unique_ptr<void>> cache;

static void* operator new(size_t size);

static void operator delete(void* pointer);
};

unique_ptr<representation> rep;
};

The new and delete operators just pop and push from the stack, recycling existing allocated memory and resulting in an order of magnitude speed up. (I also provided a way to flush the cache, not that it mattered in my application.) There are more sophisticated approaches involving specialized allocators, but this simple technique might help you reduce the number of dynamic allocations._________________Your argument is invalid.

Last edited by dmitchell on Wed Aug 14, 2013 8:26 pm; edited 1 time in total

With that setup, all users of myclass.h only see the interface and not the implementation details, which can be hidden in a library. You can even get rid of the shared_ptr<> and do the livetime stuff by yourself, but i'm so used to it, i would not miss it

Take care of copy/assign operators, because the default gerated ones copy the poiter. Regardless if smart or not, both instances would use the same data, if you dont care about these operators (and make a deep copy).

The direction I'm thinking about -- but I'm not sure if this is the best way to get there -- is to define BigClass to have all its usual methods, but in the private area, put only void *internal_state[N];. Make N big enough to reserve enough space to hold the real data without having to specify what's in it. (I have some ideas for working out what N should be but in the interest of brevity let's assume the build system can figure out N.)

The implementation file defines BigClassImpl, which contains the actual component classes and all the associated includes. It also defines the methods for BigClass to consist of a cast of the internal_state array to BigClassImpl * and invoking the corresponding method in BigClassImpl.

I completely agree: Don't do this. You make your code dependent on undocumented compiler behavior. For example, the compiler might want to (and on some CPUs maybe even has to) do certain alignments depending on the members which he cannot do (or at least cannot optimize) if he does not know the members.

If you end up using more space "just to be sure", first it still is not sure that your code is compatible and second, your code would probably need less space (and thus perhaps also run faster) if you use pointers, instead.

But on the other hand: Why do you want to avoid these include files in the first place? Just include the files which you need in the header and be with it: The produced code is not changed by it. If your aim is to minimize compilation time, C++ is the wrong language, anyway.

If your aim is to minimize compilation time, C++ is the wrong language, anyway.

It is something to not deal with unless going for a really large project; there are quite some techniques to deal with it, after which it is mostly no longer a problem to be bothered about.

The library itself can be build faster with things like (in no particular order) forward declarations / pimpl / guard conditions / interdependency (making sure that only necessary things are included and so on) / parallelism / lower optimization flags / shared libs / incremental building / not using unnecessary templates / ... , but the time the library takes to build doesn't matter that much as it only needs to be build once; what is of much more importance, is the time that it takes people to compile against the library. And that's a particular field where you can spare out a lot of compile time; how, by only giving them headers with the information that they need. Private functions? Delete. Not to be published helper classes? Delete. And so on... On top of that, the user of the library can again even spare out more with the whole set of forward declarations / ... I gave above .

But really, for most projects, this shouldn't be a necessary thing to do...

This makes me wonder, as I am just curious, what you would consider the right language for optimizing compilation time? Assuming you don't mean an interpreted language.

Idea: Implement the decoder as a stream buffer that can be used with iosteams.

That's a very interesting idea, one I had also thought about.

It sure would get a lot of "nifty" points.

But I'm not sure how practical it might be nor how I might implement some of the operations. The ".flac" file needs to be decoded into _something_. Usually that is whatever representation the follow-on processing requires, whether it's float for an audio processing application of some sort, or integer if it's mostly for listening or format conversion. Making it look like an iostream requires a intermediate format to be picked. This would probably be 16- or 24-bit integer byte-packed and channel-interleaved, same as one might read from a "raw" audio file. But it seems silly to decode and re-interleave and re-pack a sample stream in this format when most likely the reader of this will immediately unpack it and un-interleave it. There's also the question of how to query properties of the flac file to avoid lossy situations such as reading a 24-bit flac file and presenting a 16-bit packed stream out of it.

Also: how would errors in the flac file be communicated to the iostream's users? As well as error options (abort, replace-with-silence, resync-and-restart, etc.)

And how would the positioning type of calls be implemented? Flac isn't easily seekable (unless it has "seek tags" embedded, but those are optional). I suppose it is possible to rewind to the beginning and re-parse up to the point being asked for. But that wouldn't be particularly efficient.

Which reminds me of another thought that pops to mind from time to time:

I read somewhere that C++ methods define "promises" offered to users of the class. That one calls method X, and it "promises" to do whatever's been documented to the instance of its class.

But there doesn't seem to be a way of annotating "cost", other than scrupulously commenting and testing.

Seeking around in a file is an example of an operation that's usually thought of as fast. But it is not in the case of the decoded output of a flac file. I suppose one could decode the whole thing and then seek on that. Or maybe use a caching scheme of some sort. But some of these files can run into the gigabytes, and all that does is trade one high cost (of time) for another high cost (of memory).

Anyway, I want to thank you all once again for the many suggestions and best-practices advice. And I envy how quickly some of you can write long detailed posts! I feel I've got a lot to say but it gets all bottled up stuck trying to come out because it takes me a longish time to put it into words. So thanks again!

Summarizing what I've understood regarding my original question, of hiding unnecessary details of a class, to a user of that class:

First, to clarify the original goal, it wasn't to speed compilation of the library. It was only to present a simpler ".h" to the library's users, so those compilations don't get needlessly encumbered with a myriad of trivial details whose only purpose (from the point of view of the users) is to define the size of the class.

The library would get compiled seeing the classes in all their glory. No questions or crazy hacks there.

It seems that the most recommended solution, suggested by several people here (thanks!), is the pimpl idiom. The library internals knows about and uses the full class in all its glory. The library only exposes a class containing a single pointer to the user, which has methods calling the internal methods on the pointer. (is that right?)

TomWij wrote:

Please note, that once your library is compiled as a library; you will not need to declare every function and class in the header file you provide but solely those that the user will use. Some people explicitly write different header files flac.hpp and flacImpl.hpp where the former is to be distributed towards the users and the latter contains more details like private function calls and what not that the user does not need to be aware about.

That's an interesting observation, which lead to this idea: suppose, for whatever reason, I insist on the class being statically allocatable. Then a better way to do that ("better" compared with my original approach), is for the build scripts to export a customized "flac.h". This "for-public-use" flac.h would have the same method-forwarding thing going from BigClass to BigClassImpl, and for the opaque "char internal_state[N];", instead of trying to manually fill in the correct number for that N, the build script itself could simply "printf" sizeof(BigClassImpl) and stuff it in there. That'll guarantee it is correctly figured. This "flac.h" would be the "export" version of the include file. It wouldn't be used by the library except possibly in a short test program to make sure it was built and works correctly.

Quote:

... RAII ...

I had seen that term before in various places.

I don't fully understand how to work in all the various ways something might go wrong within that idea-set. Taking the flac-file reader example: Opening the file might fail outright. Or it might succeed but the header information isn't workable. Or the info looks like it could be OK but the CRC (error check) embedded in it doesn't match. Or the decode fails part of the way thru, after several frames have already been delivered. Or the decode finishes, but the final overall file checksum doesn't match.

What's the best way to communicate these various possibilities to the caller, in such a way that the caller can take action based on what happened and what is desired. Examples: skip failed decode blocks vs error out; log but otherwise ignore/try to decode thru checksum errors; substitute alternate values for header variables and try again; and so on. In C I'd return error codes and test for them, able to dynamically alter operating parameters with each loop iteration if need be.

Obviously the same approach works in C++. But is that the best way? And how do you error-out of a constructor? I read that throwing an exception is the only way to "fail" a constructor. I've also read that "incomplete construction" - setting an object in a constructed but "not ready" state - isn't usually a good idea. But I'm not so sure what is a good idea

It seems that the most recommended solution, suggested by several people here (thanks!), is the pimpl idiom. The library internals knows about and uses the full class in all its glory. The library only exposes a class containing a single pointer to the user, which has methods calling the internal methods on the pointer. (is that right?)

Now that you made clear that you are speaking about the public interface: Yes, this is the usual way.

But I'm not sure how practical it might be nor how I might implement some of the operations. The ".flac" file needs to be decoded into _something_. Usually that is whatever representation the follow-on processing requires, whether it's float for a audio processing application of some sor, or integer if it's mostly for listening or format conversion. Making it look like an iostream requires a intermediate format to be picked. This would probably be 16- or 24-bit integer byte-packed and channel-interleaved, same as one might read from a "raw" audio file. But it seems silly to decode and re-interleave and re-pack a sample stream in this format when most likely the reader of this will immediately unpack it and un-interleave it. There's also the question of how to query properties of the flac file to avoid lossy situations such as reading a 24-bit flac file and presenting a 16-bit packed stream out of it.

Also: how would errors in the flac file be communicated to the iostream's users? As well as error options (abort, replace-with-silence, resync-and-restart, etc.)

And how would the positioning type of calls be implemented? Flac isn't easily seekable (unless it has "seek tags" embedded, but those are optional). I suppose it is possible to rewind to the beginning and re-parse up to the point being asked for. But that wouldn't be particularly efficient.

Sounds like it might be impractical. I don't know enough about flac to offer specific guidance. I have had success with boost.iostreams gzip filter and thought it might be possible to do something similar.

Quote:

Which reminds me of another thought that pops to mind fro time to time:

I read somewhere that C++ methods define "promises" offered to users of the class. That one calls method X, and it "promises" to do whatever's been documented to the instance of its class.

But there doesn't seem to be a way of annotating "cost", other than scrupulously commenting and testing.

Seeking around in a file is an example of an operation that's usually thought of as fast. But it is not in the case of the decoded output of a flac file. I suppose one could decode the whole thing and then seek on that. Or maybe use a caching scheme of some sort. But some of these files can run into the gigabytes, and all that does is trade one high cost (of time) for another high cost (of memory).

These promises sound like Eiffel's Design by Contract. I think there are C++ libraries for approximating Design by Contract but I don't think there is any built-in support.

Quote:

Anyway, I want to thank you all once again for the many suggestions and best-practices advice. And I envy how quickly some of you can write long detailed posts! I feel I've got a lot to say but it gets all bottled up stuck trying to come out because it takes me a longish time to put it into words. So thanks again!

Just like in programming, shorter is better. :P_________________Your argument is invalid.

I don't fully understand how to work in all the various ways something might go wrong within that idea-set. Taking the flac-file reader example: Opening the file might fail outright. Or it might succeed but the header information isn't workable. Or the info looks like it could be OK but the CRC (error check) embedded in it doesn't match. Or the decode fails part of the way thru, after several frames have already been delivered. Or the decode finishes, but the final overall file checksum doesn't match.

These sound like good candidates for exceptions.

Quote:

What's the best way to communicate these various possibilities to the caller, in such a way that the caller can take action based on what happened and what is desired. Examples: skip failed decode blocks vs error out; log but otherwise ignore/try to decode thru checksum errors; substitute alternate values for header variables and try again; and so on. In C I'd return error codes and test for them, able to dynamically alter operating parameters with each loop iteration if need be.

These sound like good candidates for constructor parameters or a policy-based design (though I find policy-based design better in theory than in practice)._________________Your argument is invalid.