New Objects (Attribute Categories)

So I'd like to create a new object class and bind attributes to these new objects. I found the documentation, but I'm still confused. I also tried looking at the Core Commerce add-on for help and examples, but it doesn't seem to follow the documentation.

I don't want to create new pages. I want to create a new object class.

I could use the page class, but I don't need to add blocks to these objects, nor do I want them to be in page lists. So the page class is excessive of what I need.

Say I'm making a car enthusiast web site. Users could add cars to their profile. They could choose what type of car they have and then add attributes to the car to describe it. 'Car' would be the new class that needs to be created.

Calendar events... I know there are add-ons in the market place that use page/block objects as event objects. But ideally, I'd imagine these should be their own kind of object.

Houses or Property... for a real estate site.

Again, I know the page class would work for these ideas, but using a page object is unnecessarily. They should be their own kind. Because they aren't pages, and shouldn't be treated like pages. And in most cases would be ideal if they weren't treated like pages.

I certainly understand the "no need to re-invent the wheel" ideology. But c5 is a powerful tool, and the web isn't the only place it can be used. Thus a Page doesn't always need to be the device used for landing the information. :)

I've been thinking outside the box lately, and thought it would be cool if a non-web application accessed the c5 API to get data.

Basically use c5 as the web CMS, as well as a Database Management System. Making c5 a sort of information hub. This brings the ease of extendability and integration that c5 already offers to any project you could be working on... be it a web app or not.

Notice how I didn't pass a $pkg variable through to the third argument, because we're not in a package. But I _would_ pass a $pkg variable through to the third argument if I were in a package's install() method. BTW - I'd suggest you look again because the attribute key categories in eCommerce _are_ installed in the install() method of the package, after running $pkg = parent::install();

Bahh, I feel stupid... I was looking at the 'on_start' function as if it was the 'install' function. That's why I was super confused.

I need to get my eyes checked! >.<

Regardless, so far I can add attribute keys associated to the new object. Now I'm just getting the dashboard part worked out. Scavenging code from the users, collection and file classes. I'll scan through the eCom package as well for some help.

A 'New Object' object is created during install, but I have yet to build an Add New Object page. However, the dashboard has a New Object List page (that also servers as a View New Object page) and an Attribute page (basically scrapped from the user's Attribute page).

I'm hoping to finish this up so it can be reviewed and tossed into the doc pages.

What happens if you add a $this->debug(); to any of the lines of your list function? It should print out the full SQL query of your search right before it runs it, and we can see what it looks like and where it might be failing

The filtering problem was caused by how the NewObjectList constructor was setting the query. It has been removed entirely in favor of the User/Page/FileList method "createQuery" which is called in the "get" and "getTotal" functions.

I'm still having issues with Select ATKs when searching, but I haven't investigated the problem yet.

The JS issue was simply a missing "/" between REL_DIR_FILES_TOOLS and the rest of the path.

The final part of this package is to get NewObjectTypes involved. But I need to learn more about "Attribute Sets".

I've been developping with C5 for 3 weeks now for a site that sometimes required "objects" that don't fit well as "pages". Reading this thread is pure enlightenment to me, as you guys make a great summary of what I was thinking, with pros and cons of pages vs custom objects. Thanks a lot for this thread.

ijessup, I'm about to create an "Event" package. I need something similar to the calendar that we can find on the marketplace, but having them stored as pages would be an handicap in my system. I need "objects" stores in custom tables, with custom attributes associated with them (with new attribute category).

So, I will start working on my new package, and the files you provided here will be a great kick-start for me. I will make sure I post back my results here and on the marketplace (for free) when i'm done.

I had a similar idea about events as objects instead of pages. I'm looking forward to seeing what you come up with!

I have attached the latest version of the New Object Package, however, it also installs a lot more than just the New Object framework.

It also includes Tony's Multiple File Attribute and my newly created Color Picker Attribute... as well as an install method to help you add Attribute Keys and display those Keys in the search result's header. As well as a few other fun things to help you get on your feet with c5 if you aren't already.

Certainly feel free to rip-out/rewrite what ever, but hopefully the bloatware will help you in the long run. :D

I too have been thinking about this issue recently (coincidentally), but I came to a different conclusion than Isaac (more along the lines of Chad): I feel that the "page as the central object" concept is the essence of concrete5, and instead of creating non-page objects, I was wondering how someone could extend the page model itself more fully into a database (where every record happens to have a corresponding page in the CMS). The idea I came up with was to create a spreadsheet-like bulk editing interface in the dashboard for page attributes (and also file attributes). Kind of like when you edit table properties in phpMyAdmin -- each row is a page (or file), and each column is one of its attributes. In this way, you could edit lots of attributes for each page much more efficiently than clicking on each page in the site map and going to "properties" (or every file in the file manager).

I totally see your point about having data that isn't represented by a web page, but I guess my thought process is that if you need something like that, C5 probably isn't the best tool for the job. Like if you're bypassing the editable web page thing, then what benefit is there to using C5 at all since all of its tools revolve around the page anyway? Also, how do you show search results for an object that isn't represented by a page?

I understand, and agree, that if the data will be displayed as a page, then by all means use the page object.

However, often I run into the situation where data needs to be accessed through XML or JSON feeds. So a new type of object is necessary unless we want to weed out "Pages" that have no place in the feeds.

Your idea of extending a page is actually more like crippling them. Because each page object is not a simple DB row, we can keep things far more dynamic. Example being page attributes and blocks.

Arguably, attributes could be columns, but this reduces our ability to remain dynamic (using the same system for different scenarios).

However, I think my solution captures the best of both worlds. A "table" of objects.

Granted, its not really one table in the database, but the end result essentially is and everything remains dynamic.

I think you are a bit closed minded when it comes to how powerful and how much potential c5 really has. The benefit of using c5 is that a website can be directly integrated with a database of any sort.

You could use c5 as an administration tool to run an entire online TV channel distribution network. Create your own Hulu of sorts, honestly, with little effort.

That being said, any sort of value networking can be accomplished with c5. From asset tracking to relationship management and beyond.

Let's use the TV analogy as a quick reference. Sure, a channel might be a page, but would the shows and advertisements be pages?

Lets looks at asset tracking... lets say we have a shipping company that want to track their cargo across the globe where an item could be on any one of hundreds of different shipping vehicles from trucks to cargo ships. In this case, which one of these "objects" would be a page?

Everything is moving to the cloud. There is no reason why a company's internet site can't be their intranet site at the same time.

c5 is plenty capable of catering to these clients with some light modifications. And heck, I've pretty much given you the mods you need.

The best part is that it all flows together. So if you wanted to search for an object based on an attribute, you'd uses the the same ItemList class to do so as you would if you wanted to search for pages.

The PageList class is an extention of the ItemList class just like the NewObjectList. So all of the functionality that you get in with the PageList you get with the NewObjectList except that it is not so specialized for Page objects.

No I didn't make a NewObject_List block that you can add to a page, but you can create, filter and sort a list programatically. Which could be turned into a block if needed.

Yeah, that all makes perfect sense to me. I guess where we differ in opinion is that I don't think concrete5 is the right tool for the job for those use cases you bring up.

I don't see the benefit of bending C5's page- and content-oriented nature to do something that would be better achieved by a more low-level framework such as Rails/Kohana/CodeIgniter/etc.

But that is just my opinion, and I definitely think what you've done is really great and appreciate you sharing it with the community (and who knows, maybe I'll have some hybrid "gotta have a cms but also gotta have a database" job soon, in which case I would definitely give this a shot).

I certainly don't discredit your thoughts - more over, I invite them. I wouldn't dare say I know best. Frankly, I haven't even heard of Kohana, and without people like you mentioning such things I could argue my skills wouldn't be what they are today.

I certainly agree that separate frameworks could be more efficient, however, in the case studies I've worked on, a tightly integrated system is most often sought after.

A couple reasons being:
From an end user's perspective, it's more comfortable to work in a uniform environment. Which, arguably, you could form fit an other framework with and within c5's UI.

From a developer's perspective... As much as I do try to stay on top of emerging concepts, I'm human. I know its not possible to know everything, and I certainly don't expect that out of others.

So restraining the application to a single "engine" has proved most effective, for me. Training employees and getting them up to speed on a project is far quicker when they have been working with variants of the same system.

Not to mention: design, development and deployment are far quicker when all you have to do is make a few adjustments from a dashboard page.

Yes, I see what you're saying -- if you already have an established C5 site and want to add to it, or are building a new site that will have a decent amount of content pages on it IN ADDITION TO the object list / database / whatever, then absolutely this makes a lot of sense. I would *much* rather work with an object model inside of C5 (even if it's not really leveraging C5's strengths) than rebuild a CMS in another framework.

I guess I was thinking of scenarios where the only thing the site does is this non-page object stuff. In my mind, a TV show distribution channel and a cargo shipment tracker are prime examples of applications that should NOT be built on a CMS, because they don't actually leverage any CMS functionality. The C5 dashboard is a convenient place to put config pages for advanced packages, but would make for a poor user experience if you had to build an entire application in there.

If, on the other hand, this were the marketing site for a TV station and it had a lot of content pages, but then also needed some kind of large database for the tv listing schedule, then yeah this would make more sense to me.

-Jordan

PS - Kohana is great -- definitely my favorite PHP framework. The only problem with it is lack of extensive documentation -- you kind of need to understand how MVC frameworks work in order to make sense of it. Rails, on the other hand, has the most excellent learning materials around (IMO). I highly recommend learning Rails if for no other reason than it will teach you many things that you can bring back to PHP (that's what happened to me).

Strictly as an addition to c5 does my idea provide a solution worth using. Or as a starting point for an advanced block of some sort. But as a stand-alone system I would suggest other solutions.

An application devoted purely to value networking I'd suggest something like VI|Agents' VI SixD. It allows for externally triggered events and a ton of cool stuff, but you gotta be willing to fork out 6 digits or more.

Speaking of Rails... I saw a book called Rails for PHP, I think I might pick it up.

Hi everyone, I don't have too much knowledge about OOP, however I am trying to learn both php5 & C5 by following tutorials and discussions here.

I would like to thank you for the great discussion that you make here, it is very helpful for me to realize how exactly C5 works.

C5 is such a great CMS that even I created 2-3 websites with it, I've just realized here on your discussion that it is built on page objects, and that information is what I was lacking and keep having difficulty figuring out how to pull data from mysql to an area on the page. The data I mentioned here consists of company names, addresses, phone #, etc. however, there are more than 10000 rows and 10 columns.

So, in order to accomplish that I should create pages for each of those companies listed on that particular table, and I should add a page-list block on the parent page of those pages to show them in a list.

I used to have no clue how to accomplish that, however it is still so hard for me to create that number of pages and putting that information to each of them.

If I understood correctly, Isaac's approach fits my need better. Therefore, I would like to use this add-on. However, before I use it, I want to make sure if I am doing right.

Here is what I understood, please correct me if I am wrong
1. When I create a new object on my dashboard, basically I create a row in the new object table on my database, and
2. When I create a new object attribute, I create a column in that table, then
3. I can go to my dashboard and make any changes on that table and show them by adding a block to a block area of a page.

Are those assumptions right? (Probably, the second part of #3 is wrong because I couldn't find any related block type when I tried to add a block.)
Would you please explain how we can show that objects on a page?

Also, what I realized is there is a resources folder under single pages folder in the package, does that mean the package is going to create a single-page called "resources"? and if yes, what if I already have resources page on the same level of my website. Are they going to conflict?

lastly, while I was trying the add-on, I created a new object and on the dashboard/new_objects/search page, I clicked on a picture link under pictures column and it pointed me to this link:
website.com/home/user/website.com/files/3613/0290/3639/My-picture.png where I got a "Not Found" error.

I've been working on a super beefed up version over the past couple months. It's not finished, or in a releasable state, yet, though.

To answer you question...
1. Sort of.
2. Sort of.
3. Sort of.

Lol... sorry let me explain.

1. If I remember correctly (and I say that because the version I'm using now has evolved tremendously) the tables, per se, exist only virtually... as in they exist in the MySQL database, but not in a traditional sense.
2. Again, in a virtual sense yes, but not traditionally... technically there is the SearchIndex table if you need data from low level perspective.
3. Did I include a block with it? I can't remember.

These objects are for very low-level architecture stuff. You are going to need to create your own front-end (and editing interface) for everything. If you don't have a lot of development experience, it is going to be very challenging (a good challenge if you're up for learning, but just be aware that it won't be straightforward or quick). I would actually recommend in your case not worrying about these objects and instead just make a few single_pages that query the database directly. Basically how you'd build this functionality in any php program, not having anything to do with the CMS specifically. The only difference is your pages will be wrapped inside the theme's design so they will look the same as other pages on your site.

This is about as low level as you'd need to get. c5 uses a frame work (ADODB) that sets up the communication pathways for you. What my package does is takes advantage of these pathways and introduces a simplistic user interface so you don't have to know any code to build a database.

As I stated before, what you have is no where near as sophisticated as what I have developed it into, to date. I'll be releasing the new version on the marketplace soon (probably for free), but I need to make sure its in a state that won't require constant support requests due to simple bugs.

I'll be your first guinea pig. (-: I need to feed low level information into Chad's ProEvents package so that the data is associated/displayed with a dated event. I was going to work on your old version when I started to conceive my need for new objects that are not page-based. I would like to take a look, maybe try a few things. No great expectations here, and no desire to bang on you for "updates". Just happy to get a head start. Can you share it?

As of right now you can request lists formatted in XML, JSON, or an HTML table(for Excel).

The requests require an API code and a Key code that use the (though can be changed) MCRYPT_DES encryption algorithm and in MCRYPT_MODE_ECB mode with an algorithm key that you can set (planning on adding this option to the dashboard).

You can also bypass the the authentication process, but your results we be restricted to only what guests are allowed to access... unless you specify otherwise in the rest.php file.

I should have the ability to submit data tomorrow... considering I have a Flex app that will require this feature, you can pretty count on that estimation.

Check your email in a few min. There is some sensitive code that needs to get pulled out. I have a job that pulls data from an MSSQL database and dumps it into this system. It's very hack at best, but I'm still not ready to release that at the moment.

This sounds really cool. In light of the original discussion from a few months back, would you mind sharing your thoughts on what benefits you've seen from using the object system (as opposed to just coding up your own data models with SQL queries behind data retrieval functions, etc.)? Genuinely curious because you've obviously buit a serious infrastructure here and I'm fascinated by what the various pros and cons might be.

I just re-read through some of the older stuff here, and may have answered my own question. So maybe the advantage (other than presenting a unified programmer API, which does make sense although I wish C5's internal API was more consistent :)...

* Objects are tied into site search? (But do the search results actually link to anything?)

* This is a generic library that can be used to easily build out new schemas?

* This has some kind of built-in dashboard interface for CRUD operations (adding/editing/deleting) on those generic objects once the schema is defined by the programmer?

* Somehow these objects can be tied to pages (optionally)? Maybe?

Am I on the right track here or totally misunderstanding your use cases?

You've pretty much got it. It's basically a "heavy" (probably not the best marketing term) database. Don't expect blistering fast query speeds. However, if you need speed there is a SearchIndex table in the "real" database. I use that for the dashboard's search page to handle large queries.

It's a dynamic extension of the AttributeKeyCategory class, which I'm calling Virtual Tables. They mimic any "real" database table, but they are easier to manage (thanks to c5's AttributeTypes), integrated into c5's API, accessible externally to c5, and have permissions down to each individual "row" (which I call Virtual Items).

You can create as many Virtual Tables as you want, on the fly, without having to upload (or create programatically) files that would normally be needed. The whole package uses 6 "real" tables and never needs more. This includes the included block's table and an Archive table (archive system working but not easily accessible at the moment).

You will eventually be able to easily import and export data to/from the system. You can sort of export with the REST (it's more like logic URL really) tool, but I haven't added any sort of import functionality, yet.

This is primarily targeted towards developers, but it does bring database management into the hands of non-SQL engineers. So "site owners" should feel at home with it as well.

So the tables become what is now known as the Attribute Category, upon which you can bind attributes with their type. The information is now read/write accessible externally because it's merely a table - not a an "actual" C5 Attribute Category. This enables the use of the c5 core on tables that are not "hindered" by being bound to either collections, files or users.

Very interesting. I'm not really sure how to use it yet.. But you got me thinking about it.

EDIT: In my present case, I've got a complex number of interactions that can become much easier to manage because I can create separate tables to handle their logical separation. Then perform the logic on them that's needed before feeding those results into a collection DB (say, a ProEvents table) that would then be parsed into a page when and where I need it. I can also export that database so that the "logical data map" of what the site is doing is more readily understandable to the naked eye.

It looks rather amazing to me. I'm not sure why it would be slow. You mean "slow" in relation to a custom built sql database engine?

Yes, a low level SQL query is faster than my system even with the SearchIndex. We are talking pico-seconds per item, but that can add up when you have thousands upon thousands of entries. Thanks go to the PHP overhead.

If you set the VirtualTableItemList's get() function's 3 parameter to true, you will then get an array of VirtualTableItem objects. This is by-far horrifically slow in large numbers, but it does make it easier to manipulate those items. So in small queries it makes things pretty easy. However, even in queries limited to 500 results, it takes about 3 seconds for PHP to form the array in memory. There are some tweaks I am planning on making, but that's not high priority at the moment.

Ya know, now that I think about it, I kind of depreciated that portion of the code... or rather just ignored it because of how slow it was. But what it's supposed to do is give you a PHP array of objects so that something like this would work:

I have to look at the code deeper. Does Virtual Tables cause a loss in access to its data using c5 class functions that would otherwise be available if they were in their own Attribute Category (like the eCommerce addon)??

I don't have a large database to test, but you might try setting the table widths (and heights) to a fixed number. That way the PHP doesn't have to first build the table and then calculate the display, it can just lay it out first time through. If it helps the speed then you can maybe provide that option within the block's setup. (?)

When you create a VirtualTableItem you get an extended Object. At the moment, there is no way to grab data from other tables outside of the VirtualTables package. So you wouldn't be able to grab Pages, User, etc... or at least that's not programmed in. The VirtualTablesList class generates the list based on the VirtualTableItems table in the "real" database. I could extend this so that if the VirtualTable doesn't exist it check for the "real" table. However, your code would end up looking something like this:

Other than being able to export data using logic URLs, I don't see this as beneficial.

It is important to note, though they will show up on the ~/index.php/dashboard/settings/manage_attribute_types/ page and appear to be AttributeKeyCategories, in practice VirtualTables CANNOT be accessed via the AttributeKeyCategory class. However, they are a direct extension, so all functions from the AttributeKeyCategory class have been inherited.

I noticed the interesting link that can be made from one table to another on a given page (block setup > advanced tab). I haven't played with this much yet, but can that be expanded so as to drag and drop a row from one table to another?

Again, nice work Isaac. I mean it; not just flattering you. You did a lot of work in here and, to me, it seems like a lot of potential can be gained from it. It's got me considering a whole new way to manage data. Can you answer my question up top? (I'm wondering whether I should jump.) (-:

Yeah, I use the block as a sort of dynamic relational databasing tool.

Yes and no. No in the sense that it would be very difficult to add drag'n'drop in it's current state, but by changing the tab's mechanics I can add that feature.

I wouldn't expect that on the priority list. Still fighting the REST uploading feature. I have it parsing JSON and XML files into stdClasses (storage Object). Now I just need to do something with the data in a logical fashion and include support for user permissions.

You can send either an XML/JSON file or a string. It looks at $_POST['data'] and file_get_contents($_FILES['data']['tmp_name']) for the data.

#4: Agreed. I'll add this to my TODO list. I'm thinking a couple radio circles that asks if you want to highlight the related items or remove non related items.

#5: Not a bug. This feature actually works as intended. However, you can't just change to Object mode anywhere. In this case (when you changed it), the view.php file is trying run a foreach loop on a VirtualTableItem object. For obvious reasons, this will not work. The way I have it written you access an item's attributes like this:

The proposed attributes for the Table and the Table Items (in your PM) is so as to perform logic on their content to provide more control of the results / output(?).

My understanding is that your external API is to feed in/out (JSON, XML, text strings) info so as to provide a relational databasing tool. Without pushing you for my cause (Virtual Tables is providing me a significant head start) but proposing a use-case scenario that might be valuable to others: My goal is to use the tables to build a graphical interface as a database "management" within a c5 website. I have a condition where about 10 choices have to be made for any given calendar event, and each of those choices requires the understanding of some history to each decision. I could use the Virtual Table blocks to layout that history per area of "decision." I could then drag/drop those "choices" to another Virtual Table and "build" that calendar event, which could then be saved into the appropriate ProEvents db for display. That means I would need to "import/export" to local db's within the c5 database. That's not your present (XML, etc) focus but I'm wondering if it could be included into the interface you will need to build. I understand it can be hard-coded into existence by me. Although its not likely that others will be faced with this kind of multi-decision/history scenario(?), but Virtual Tables could offer a clearly unique ability beyond that of the traditional method of a dynamically loaded select/option (etc) interface - and one that could provide more information about that selection choice to the end user.

Also, the above mentioned Virtual Table import/export of local database tables can provide a significant layer of abstraction on information retrieval. A database of 10k rows can be filtered down to 100 rows before importing into a Virtual Table for use in the above mentioned "management page", speeding up the ability to interact with the information/decisions at hand before saving it back to the local non-Virtual Table database.

Just some thoughts. I greatly appreciate your work as is. Thanks.

EDIT: I just made a quick check and it seems I might be wrong on the Bugs 7 & 9. Not totally sure yet, might be something else - or just a bad day.

#6: Added a conditional statement to check if "columns" exist. If not there won't be a header row on the rendered table.

#7: displayName is a variable not a function. Line 55 is the code that renders the tables header cells. Has nothing to do with the naming convention, so I'm not sure why thew an error. Considering your EDIT I'm going to call that a fluke, but keep you eye on it.

#8: As expected. Unfinished front end administration tools.

#9: No schema changes in the "real" database. Unless you uninstalled the block then reinstalled rather than just overwriting the old files, I'm unsure what could have caused that. Perhaps a cache error? I may have to look into this further.

The additional AttributeTypes are an attempt to leave the system open ended. I can't think of too many use-cases, but it may be a life saver to somebody.

Keeping the system as open-ended and extendible as possible is a primary focus. The original idea for the JSON/XML feeds actually came from an MMOShooter game I used to play where they hosted match statistics accessible via logic URL requests. People made some cool sites using a community developed PHP API. So I thought something similar would be worth adding. That being said, there are a lot of things this package can be used for that I haven't even dreamt of yet. I'm just trying to think of every possible way to get/put data as easily as possible.

To give VT the ability to assemble and display internal information - most likely that of additional attribute categories (i.e. NOT Collections or Files) - circles back to one of your original comments in this post (http://www.concrete5.org/index.php?cID=66258#78507... ). It's the ability of VT to harness the c5 core AND the CMS ability to display the cross-compiled content that takes this to another level. I've been involved in very complex product developments where the ability to cross-examine one department to the other would have been very helpful. Could it all be dumped into one category and the same searches be done on the whole lot? Sure, but now were talking human failure to track the web development of such an effort (in my view).

With that in mind: A VT Table Attribute could be used to direct the internal query to the appropriate Attribute Category(ies). Table Item Attributes could be used along with your existing filters to reduce the initial VT Table build, thereby freeing up the subsequent queries done before exporting the results into one of the formats you have created - or just displaying the info and dumping it at close, or using another Table Attribute to save that "virtual collection."

Another example: In manufacturing, there is a constant struggle to keep the knowledge of "parts needed" on the floor with that of "parts ordered" by the purchasing department. (Reducing on-hand inventory is a major cost savings incentive.) Software developers always try to sell these mammoth packages that "do it all" by linking the accounting department and everybody else (order entry, etc). Inevitably, it's merely theoretical because the assimilation of that amount of data into one structure causes a lot of problems (too numerous to list) and is only an option for those large enough to expense it. And none of those packages I've ever seen integrate into a "nice" Internet display - whereas c5 would "just be getting started" as a provider in that area. VT can provide a "momentary connection" of that information. You actually break apart the burdensome connections between two behemoth entities in software and yet still allow the ability for one side to see what the other is doing. It also enables the two entities to be developed at separate times, reducing the upfront capital costs and risks associated with trying to build this type of information hub within a company. The possibilities I can see on this subject alone are significant. I mean, think of the amazing configurations of data display each department can "create" on their own. Amazing. In a sense, you're giving each department the ability to "code" their own interface specific to what they need.

Essentially, this is the benefit of separation between page-based collections and "other" information that this thread originally touched upon.

See, this goes back to what I was saying about me not know what VT "could" be used for, and that it's essential to leave it open-ended so that my ideas don't inhibit your final product's potential.

What you're talking about sounds like some sort of AutoQuery block... sounds to me like it would be worth a time investment. Can you write up an expected work flow, or draw up a diagram of how such a block would be created/used?

To me, "AutoQuery" is a time-based initiation to query a db - like a mail program retrieval system. But I think you used the term to describe an interface (block) that would query certain databases of choice for the purpose of display(?). I think your VT blocks already perform the display aspect; in fact, the diversity of what VT blocks can be added to a page and how their display can be manipulated and how they can be inter-connected is the unique and powerful aspect of Virtual Tables. The question then becomes "what are they displaying?" External xml/json feeds to compare statistics (such as your gaming example)? Cool. Comparing internal Collection attributes within C5? Not so cool; as c5 collections are not specific to any one "department" or "entity" that would be of interest to correlate with (in my view). The same feeling (might) be applicable to the c5 File attribute category. But....

When you think in terms of newly added c5 attribute categories being developed for specific "divisions" within a company (Customer Service, Order Entry, Payables, Receivables, Purchasing, Marketing, etc), or even just an attribute category specific to just one cause (as in my first example:http://www.concrete5.org/index.php?cID=66258#163178... ) then it becomes a little clearer as to what Virtual Tables can provide. What I am describing here is the "foundation" from which you can harness the c5 engine to create as many separate "entities" - as separate as the File Attribute Category is to the Collections Attribute Category within c5 - in order to build those pages specific to each category within each of those specific departments. Virtual Tables is the connection point between them all. And how those connection points look; meaning, what is available for view (permissions), what connects to what (existing VT setup dialogue box choices) and the interactive "JS methods" that help to sort/view correlating information is all customisable. Example: You are a product manager and you are trying to make sure it's all coming together? Make a c5 page that pulls one VT from the purchasing department, show "these" columns in it and make it associate with a second table you add that is from the Receiving department and make the Receiving department table correlate with the inventory department. Now, concerned that you've got enough Product xyz on hand for a new order you just got? Click on the product in the table row and watch the other tables update to reflect the status for that one product in those various departments. (Nice! But that's just one minor example.) Typically, the only way a company can get that type of information is to purchase mammoth software packages that force entire departments to re-learn their method of doing things - and it is either an all win or all lose proposition, as they are all inter-dependant on each other. In reality, many companies have ONE department that is suffering and needs attention (which forces them into the marketplace for a solution). With this "C5 For Business" package (or method of solving this problem using Virtual Tables) a company can address this one area and then move forward into a fully integrated system as money and time permits.

If that makes sense, then the biggest thing you've got to do is something you've (mostly) already done. Enable your Virtual Tables package to create additional c5 attribute categories on install. Then tap into those user specified form-filled values on the VT block when added to a page. That's how a VT block can be directed to load up "Receivables" (in my example) and so on. I think it's unreasonable to assume you could build all of the "departments" within this "C5 For Business" package (which could be an extension of the Virtual Tables package?), but other marketplace addons can do that and they would rely on your package as the one that creates "the walls, hallways and doors" of this infrastructure.

AutoQuery may have been a poor choice in words, but you pretty much got the idea. Basically it would be a block that makes it easy to set up multiple blocks (table views) and the relationships between them. I.e: making sure all the onClick events are set in-place correctly.

If I'm following you correctly, yes, you can automatically create VirtualTables and assign AttributeKeys to them on install. You would just need to append a little bit of code to the package's install function right before the cache flush, like so:

//add new Virtual Table and add new AttributeKeys$attributeArray=array(array('type'=>'number','handle'=>'increment_id','name'=>'Auto Incrementing ID','searchable'=>0,'searchableIndexed'=>0,'editable'=>0,'header'=>1),array('type'=>'text','handle'=>'name','name'=>'Name',

This also shows a few magic attributes... an AttributeKey with the handle 'increment_id' will automatically be given the value of 1+ the highest increment_id number. 'date_created' will automatically be valued at the date/time the item was created. And 'date_modified' automatically is given the current date/time when the item was last updated. These items are hidden from view in the dashboard when adding and editing an item, but can still be used to filter item lists.

Here is an example of a Job that I created that will check a MSSQL database and update VT to reflect any changes.

The run() function is what brings everything together, but there are some other tricks in there that make it easy to pick the right tables and how to convert the data, as well as some conditionals to fix any potential issues.

So maybe an "AutoQuery" block might be better labelled a "TableQuery" block(?).

I would like to take one more stab at articulating a point I do not feel is being recognised. I am not sure if you hear it but are advising me in a different direction, or that I am not getting the idea across in my description. Your last response pointed out the ability (and how) to create additional VT tables and binding attributes to those tables within the VT package controller's install method. I understand that would work and that you could use that approach to solve the scenarios I have used to describe potential uses of Virtual Tables. But that approach is not extensible from a community perspective - extensible in the manner that outside "packages" cannot be "used" by the "Virtual Tables" package.

Consider the eCommerce addon package. It creates it's own attribute categories. It cannot be "used" by the Virtual Tables addon. Virtual Tables cannot "dynamically load" information from the eCommerce addon because it is not "designed" to look for C5 caterories as a source from which to load any of it's virtual tables. Why would you even want to do this? Well, change the name of "Virtual Tables" (in your mind) to that of "C5 For Business" and step back one more level. Again, I'm going to use the business analogy but the benefits go beyond just "business." "eCommerce" is just one of a multitude of "divisions" within a company. Meaning, the products you have on the shelf, the shopping cart and inventory are just SOME aspects of running a business. The eCommerce "inventory" is simply what is "on the shelf" and ready to be purchased. The eCommerce "Order Entry" is merely a sales ticket from which a monetary transaction can be formed and the shipping of that product can be created. These are all good things but herein lies two major points. Firstly, NONE of the information generated within the C5 "eCommerce" addon is available to the Virtual Tables addon. That means NONE of the sales and shipping information that has accrued "on the front-end of doing business" can be taken advantage of within the Virtual Tables addon. Secondly, consider this: What if someone built a C5 addon that tracked "Purchases" for your business? NONE of the information that could be used to track those expenditures would be available within the Virtual Tables (or, C5 For Business) addon. Same with a Customer Service addon. Same with a Service Call Request addon. Same with a Production Schedule addon. Etc, etc. Virtual Tables becomes isolated as a standalone package that is capable of doing some interesting things - but it's functionality is limited to the specific tables (and values) that the developer using VT implements. Even then, for a business to "use" the Virtual Tables ability it would need to "double input" information; for example, enter orders once in the eCommerce package and again inside the Virtual Tables package in order to be used for some database query of information to show some associative relationships for management decisions. (Yes, this is common in smaller businesses.)

My suggestion is this: What if Virtual Tables looks to the C5 attribute Categories and offers you a select/option from which to choose (i.e. <option>core_commerce_product</option> <option>core_commerce_order </option>) which can be used in the

That means that anything built in the marketplace that uses attribute categories can be taken advantage of within the Virtual Tables addon (aka: C5 For Business). As it is now, most packages simply use the collection category to implement their package. I think that happens because there is no over-riding need to implement a package's own attribute category unless the package itself is of such magnitude as to warrant it (such as the eCommerce addon). Also, it is due to the fact that most addons need to become a page in the end anyway (as Chad pointed out); but now they can be integrated with the "C5 For Business" addon package that "communicates" with all of them via the Virtual Tables functionality.

Would somebody design a "Marketing" package with it's own category so as to be an additional package that when plugged into the system becomes integrated into the "C5 For Business" core package? I think so. They certainly would NOT do it if it's functionality was limited only to what it could do all by itself. What would a "Marketing" package entail? It could show the cost of building the advertising campaign for a particular company product, the advertising dollar spent to air that campaign, the timeline on when that campaign is expected to begin and end, etc. All of which is information that can be pumped into "Virtual Tables" so that it can be assimilated with other "divisions" (read: C5 addons) in order to generate extensive information (relational database analysis) for management in all areas. "All areas" because Virtual Tables can be configured to show, sort, display and link to whatever various tables it wants per page.

It's the ability to "integrate" with other C5 packages that gives your Virtual Tables the real power. I think your addon would enable a whole new round of packages to be built that would be geared towards this integration (via the C5 attribute category). You don't have to build those packages, you just have to accommodate them. And in the meantime, your Virtual Tables package provides power "as is" by a developer using the ability to create it's own tables and attributes - which you just explained in the previous post.

EDIT: BTW, thank you for your examples and tutorials on how to most effectively use Virtual Tables. That's a big help to me. I re-articulated my point because I think there is a nice opportunity for you and to expand the uses of Concrete5.

There is one hurdle I see with bringing in the other AttributeKeyCategories (without actually copying the data): "standards". A Page objects is (though extended from the same parent) completely different (in terms of logic) than a File object and a User object. VT Objects are standard, offering exactly the same functions across the board.

What you ask is possible. What I could do is extend the current VirtualTableKeyCategory class to act as a wrapper, or just ignore VT and pass through the intended class. There is an advantage and disadvantage to both sides.

If I go the extension route, we maintain VT's standards. However, we'll lose the original class's custom functions (version control, etc). The opposite is true if I just pass through the original class.

Actually, before the package was as dynamic as it is now, it actually just created AttributeKeyCategories. The problem I ran into though was that files HAD to be created in order them to function at all. This caused a problem if the server the site was hosted on did not allow for this. I don't regret transitioning to the new way to handle AttribueKeyCategories, but I should have keep in mind this sort of backwards compatibility.

BTW, I'm not so sure copying the files isn't better. It would provide two benefits (that I see):

1). You could change / alter the data without affecting the original copy, then provide a method to save the results when you're ready (and if it has been allowed).
2). Copying the original files based on the initial search filter could greatly reduce the size of the virtual table and speed up interactions with it.

Also fixed a major issue where items are pulled from the DB. For the sake of speed, I was using the SearchIndex to populate the array. It now uses the Attribute's getSanitizedDisplayValue() function. This does increase process overhead, but within an acceptable amount.

The problem with the previous method was, "what if an attribute didn't take advantage of the SearchIndex?", you get a NULL return every time even if the attribute value isn't NULL.

This also provides a formatted output of the value based on the AttributeType's controller conditions. If there is no getSanitizedDisplayValue() function specified, it will fail back to the standard getValue() function.

Yes and no. This is very similar to Ricalsin's scenario. Yes, in the sense that the whole point of this package is to make it super easy to build and manage a database. No, in that (at the moment) it is not cross package compatible.

Regardless, I would seriously hesitate deploying this package in a production environment in its current state. There are a lot of know issues at the moment.

If you really wanted to use VT, you could use the a Number or Text attribute for the UPC code for now, but ideally I would find a way to have it point directly at the item in core commerce's inventory.

Also, the only way (at the moment) to input data is via the Dashboard or logic URL calls (rudimentary REST) using an XML or JSON feed/file. I have yet to build a front-end input system (form builder).

Sorry, I should have been more specific. He doesn't want this to interact with coreCommerce at all. He's counting this stuff on paper with not a whole lot of transactions. A commerce POS would be very cool though :)

I have no problem creating a stand-alone form (Ajax) to post to the table and return to the 'Bought' field for another scan. All he'll want back out of it is a list and possible export to Excel, and c5's user/permissions ACL.

The front desk logs-in as normal, and leaves this form running all day. He can log in to his account from home and check the numbers.

I would love to see a form builder, but it's not necessary for this scenario. I can see quite a few use cases where this would be an excellent "extension" for c5.

In either case, I would love to play with your code on a dev machine and I believe I left you a PM with my email address?

I think your needs are fairly basic and Isaac's solution here would be overkill for your particular situation. I'd just create my own database table to store this and create a dashboard form to manage it. See this how-to for a good overall explanation of how to do this kind of thing:http://www.concrete5.org/documentation/how-tos/developers/build-a-s...

There is a logic URL tool included. You might want to comment out the username and password encryption if you don't know anything about DES. Technically you can specify your encryption method, but by default it is simple DES. There is also a bypass mechanism that I think is uncommented right now.

I'll working on making it actually REST standard later, but you can push and pull results with it.

I'm using it with an AIR application right now to save time card information.

This sounds exactly what my first thoughts were after toying around with C5 a bit. I love the block approach, but not all data belongs in a block or directly tied to a page. That seems like an issue with separation of data and presentation/context to me.

The packages is almost complete. I've been working on farther separating VirtualTables from AttributeKeyCategories to help with potential legacy conflicts. This is mostly complete, but it broke the block that comes with the package.

Unfortunately, I haven't had time to finalise everything due to a move and other dead lines.

"I've been working on further separating VirtualTables from AttributeKeyCategories"

Nice! Because that was the biggest concern I had but I guess I never conceived a way of doing it apart from the attribute categories.

Using C5 categories (in developing your own API) enables you to "come up off the floor" (if you consider the AttributeKeys and AttributeValues the floor) and lighten the load on db inquires; though newly assigned attributes store and retrieve a C5 "unique" avID and akID from those base tables, subsequent db access stays up off the floor and keeps the system light(er). So "separating" VT from that important C5 structure (categories) and still provide the Virtual Tables functionality of data-basing information outside of the often-used "collections" category would mean - to me - that VT is now playing with the "higher up" deck of information (categories). That means even faster access times AND the ability to interact with each custom category completely independently. i.e. Nothing has to be "built" using VT.

Future addons will not require VT to work, but developers will be more motivated to get their structure off the collections category so that it "can be used" by VT and therefore make it's information available to any other API that wants to tie into it.

Is that accurate in your view, Isaac? ("No, don't put the couch down. Get up the stairs first, then have a beer and THEN answer the question." hehe.)

I'd like VT to be able to function with legacy apps as well. The VirtualTableKeyCategory object can get the list object of any VirtualTable or Category, so long as VT has the correct relative path to the list model.

All list models use the DatabaseItemList class, so they all play nicely together for the most part. however, each has their own "special" details (ie Meta Data). This is the hard(slow) part. Each non-VirtualTableItem would need to be parsed and this takes up valuable processing power.

VirtualTables on the other hand can work both ways: As a low level(very fast) database query in the form of an array or stdObject, or as an object list (not so fast, at least on my developmental VM).

VT also has it's own API that piggy-backs the existing c5 APIs and attempts to keep with c5 function naming conventions.

"VT also has it's own API that piggy-backs the existing c5 APIs and attempts to keep with c5 function naming conventions."

Yes, but that's only C5 methods and conventions - not custom addon methods, because they are unknown(?). Custom coded methods are lost in their use within Virtual Tables(?). (Which is probably not that big of a loss, right?)

The original goal was "relational database analysis" so associating the data and even using some core C5 methods on the objects by interpreting their class names via the directory naming nomenclature is great news.

I'm just having a theoretical moment of thought regarding VT tapping into custom methods. Could serializing an object into a string and then unserializing it under a new "vt" class name be any kind of a solution? For example, wouldn't it be nice if the eCommerce method of getOrderDisplayTotal() located in the ecommerce/models/order/model.php file be readily available to someone inside VT? Or, would re-creating a package's models directory by unserializing it (and renaming the class) within Virtual Tables give you easy access to the package's attributes meta data?

Maybe I should first ask whether you've seen any kind of significant benefit that could be gained if in fact VT was able to tap into those custom functions?

Just want to point out that technically the "category" shouldn't give any benefit to database performance -- the attribute keys themselves are indexed (by virtue of being primary/foreign keys) and unless you're dealing with a monstrously large database (millions of rows) they will be found quickly -- this is the whole reason SQL databases exist.
That being said, it's still a good idea to isolate the categories for easier management and to make the code more understandable from a conceptual point of view.

I thought the "searching" for the keys was made much quicker in the categories, and that once found the "accessing" of them (in the lower tables) was virtually insignificant.

I'll take your advice as correct. Thanks.

EDIT: But, Jordan, I think it would help to get either a confirmation or restatement on the benefits of C5 categorization. In my view, the benefits are not merely organizational. Granted, accessing the base tables (attributekeys and attributeValues) is insignificant once you have the keys. But getting those keys requires a (example) "select where" query to be performed on tables and if those tables can be cut down in size then that can provide significant benefits in access time. Can you comment on that for me, please?

Heya, just wanted to check in on this. I've been inclined to create a framework to standardize creating object collections in C5, but don't want to reinvent your wheel. This is the main limitation of C5 that really bugs me at this point, and I'd like to get this kind of feature into the core to promote C5's spreading.

Thanks! I figured it would be encapsulated in a package until it was time to integrate into the core. Is there a reason you didn't do that? Kinda difficult to jump in when the relevant files are so scattered.

Originally it was, but I ended up having to override tons of core files. I talked with Andrew about it, and he became interested in making it a core feature. So my focuse as of late has been just that, making it a core feature.

The result it much cleaner code and a more integrated solution so that all "objects" work the same way.

No matter what the Class you can always getObjectByID(#) and getObjectList() where the returning result is the appropriate object type.

Excellent. That's exactly what I was hoping for, and I'm very glad to hear that Andrew is on board. I was going to just give the code a glance and let it roll around while I turn out my current project; it's my first C5 project, so I need to let some of the fresh dust settle before I start hacking at the core. I will look for updates on this and contribute when/if I can.

I wonder if many people (not Andrew) actually understand it or what it can be used for(?). I'm sure Andrew is spending most of his time putting out fires.

"I'm working on a way to build add-ons (c5 packages) that take advantage of VT." :::: By studying existing c5 packages it's clear that most people DO NOT know (or care) to build attribute categories - probably because when you build an app that is not expected to communicate with other apps then there's really no need; essentially, you're at Chad Strat's thinking that says "I need to be a page eventually." But this brings up your original argument that - thinking outside the box - c5 has an incredible architecture to harness for use in a greater way. I'm curious to know your approach to the quoted statement (above).

I was thinking there should be an app that "creates" an attribute category so as to get people OFF existing c5 core categories - sort of a developers tool. You don't really care what the name or even how many categories exist - you just need an app's information to not be buried inside one of the core c5 categories. Either that or create (essentially) another core attribute category that could eventually become the fourth core category on which most new apps are bound. I mean, "File / User / Collections" are just the start. There needs to be something like an "Integration" category; where all apps go to be unhindered by either user, page-collections or file clutter. Once there, they can speak amongst themselves (VT), pull (per usual) from either user and file categories - then go to the collections category when they are ready to display their content.

Please feel free to send another update when you want me to see it.

Rick

-------------------- Response --------------------

My thoughts on the quoted text should be self evident once you see the single_page included in the VT add-on "Time Logger".

Yes the data eventually gets rendered on a page, but none of the data would actually need to be a page of its own. See attached screenshots.

The "Time Logger" package adds the following tables on install:
Companies
Projects
Tasks
Expenses
Invoices - To be used later

Companies and Projects, at the moment, only have a single attribute (Name). You could use a Select AttributeType, but the idea was to showcase relationship building between Virtual Tables and their items. What's cool about making them Virtual Tables, and not Select Attributes, is that you can not only keep consistency between other AttributeKeyCategories, but also extend the amount of information the item holds. For example, you could also include the company's address, point of contact, etc.

Tasks and Expenses are the heart of the add-on. They both use the included AttributeType, VirtualTableItems. Each 'Tasks' item forms a relationship with a 'Companies' item and a 'Projects' item. Sure, you could use a Page as the data container, but why weigh down such a simple piece of data? And where would you put it in your sitemap?

The other argument is, why not just use a MySQL table? You absolutely could, but forming these kinds of logical relationships is painfully difficult without some sort of framework... and you're already using c5.

I think people don't use AttributeKeyCategories, because they aren't easily accessible. Hopefully this package changes that.

Although this post has been quiet, Issac (ijessup) has been working with C5 on it's development. In my view, the latest work adds a lot to the C5 CMS core and opens a good-size door for expanding c5 offerings of web-based solutions to the business world.

I believe it is up for a review before too long. I am hoping it goes well. It would be interesting to see what c5 developers could do with this new capability. (There are some "sic" programmers around this place.) :)

Honestly speaking, I can't event begin to see where you would automate this.

Call me a lack of visionary, but the major issue that sucks a TREMENDOUS amount of time is the search/filter advanced filtering dashboard pages. This is EXTREMELY js heavy. specifically speaking the per-item menus. It's nasty!

Setting up this aspect alone far a custom object type takes a minimum of 3 hours. It sucks.

So yea...anytime I CAN use a page-object. I will.

Hell, even with the page-object approach...I had a need to customize the advanced search page per-item menu's...that sucked terribly.

Especially now that the ccm-app.js minimized is loading when you don't actually WANT it to. So you end up having to custom code all your JS to get your desired results.

Contrary to the assessment that I default to system objects, I actually have developed several custom object types /attribute categories. (I was very inspired by ljessup's original model. although no longer valid with 5.5)

I now have two working models of custom 5.5 apps, both page based and custom object types with advanced ajax filtering. Both were difficult and time consuming.

I agree that it's a big idea. It's really something to keep your eye on as a community - something to eventually work towards. RESTfull architecture has a future...

Your contributions to C5 are obvious, Chad. I've looked at how you accomplished your addons and it's good, clean thinking - certainly more economically viable than the theories presented in this thread, which was originally wrapped around accommodating a RESTfull design and therefore binding to pages was not ideal.

The C5 architecture has the potential to harness the above concepts. C5 has some difficult structure to wrap your head around, mainly because it keeps everything broken up in order to be accessible dynamically. In my mind, that makes it capable of being the first CMS that can be ridden to the extremes of this thread.

I can't say I agree with your complaints of C5's advanced search design and how it uses the ccm.app.js to return results. Once you figure it out it becomes very easy to manipulate; granted, it's tough to first understanding it.