vvvv-Message

about

The vvvv-Message pack has been in open development for four years. In that time it grew alongside free and commercial projects involving the likes such as Projection Mappings, Device Bindings, UI Widget Handling, On-Site Data Plumbing, Web Parsing, Boygroup Replacement, and not least, Bus Driven Patch Architecture.

The pack is exposing the SharpMessage functionality to vvvv in a fully spreadable way.

Install

You can use this VPM link to directly install the latest fully supported version with all dependencies

Alternatively, the latest releases and prereleases will always be at the GitHub Release page, just create a directory *packs/vvvv-Message* and unpack to there.

Getting Started

Use F1. Look at the help patches, if you are in doubt, because there is a lot of information in them. The shortest route to try out Message, however, is the 101 girlpower.

Anyway, first two nodes advised to check out are ConfigKeep and Split.

If you feel comfortable with splitting and maybe even editing, understand that a Message can become stateful (i.e. across time, not just a single vvvv frame) when kept inside a Keep.

It helps if you can make yourself unlearn AvoidNil, Framedelay, S+H, Change and their family. There should always be a solution without duplicating some of them; the Message nodes can be smart enough themselves. If you still "need" them in bulk, chances are you haven't found the best combination of Defaults and Keeps yet. Oh, and abolish S and its evil twin R, that helps too.

Mission

The original idea of this pack sounds convoluted: make plugins with easily typeable io pins and use that to retain order in bigger patches and saving lots and lots of links by simply packing all kinds of attributes in a single meaningful object and handling that object in a very easy and data flowing manner. But it is plain: teach vvvv how to write layouted letters and fill them into marked envelopes, meant to be given from hand to hand.

Goes without saying, but every hand in this metaphor is a of course a node. And there are no letters or envelopes of any paper or other material kind, but only smart C# Messages, trademarked with a proper Topic and automatic localized timestamping.

Use nodes to create, read and write, reshape, analyze, search and sift Messages, and, probably most useful, keep them and use them as stateful objects that can be manipulated anywhere else in your patches' folds.

Message is meant to be open, it strives to connect with other relevant protocols.

Features

42 vvvv nodes total: Don't let the number fool you, most of the time you will not need more than a dozen of them, but this is a vetted pro pack. So you get the full suite to freely evaluate and even use it in non-commercial work, as seems fair.

At first glance, it might sound the most boring of all, that you can tightly define the structures of your Messages, both per-node (thankfully in a custom vvvv window) and for many nodes simultaneously across entire applications. This feature is called Formular (yay, more Kafka to the vvvv)!

Once at this meta level of patching, you don't have to replicate structure for your data anymore, but simply select your predefined Formular and get the benefits of named and typed pins - anywhere in your patch.

Industry standard serialisation with both Json.NET and binary MsgPack are available to help with persisting, streaming or even distributing data among vvvv instances - local and networked. Additionally it helps to establish api bindings with good ole' OSC.

It comes with girlpower examples showing how to connect to TouchOSC, Ableton, Reaktor, XTouch midi interface and Duration, just to showcase its broad versatility and making all these things easily accessible through Message manipulation.

Extending traditional data-flow, the pack allows down-stream edits (i.e. Message is conciously mutable). This is big, because it helps any project, where you need read-write access to your data across patches without having to resort to framedelays. The pack's change management is just the icing to the fact.

Oh, you can also put DX11 resources into a Message. Or a spread of Messages in a Message. However deep you'd like to go. But that is totally up to you.

License

This is released with Creative Commons BY-NC-SA 4.0, so no commercial use!

If you need a license other than that contact us at license@intolight.de
Consider that intolight rates start at €42 per vvvv license, and compare that to the productivity you gained.

pretty handy contribution, using it a lot.
But when using, i was thinking about the necessity of the connections at all.

Why not declare Message to the new S+R Nodes...!?

Message(Join) as a generic Datatype, that can handle everything and Message(Split) just as an reciever. So instead of the "Message-Input" of Message(Split), there could be just the Adress of some Message.

And with Message-Type there is a way of sorting adresses.
For Example when Message(Join) is type Event and Message(Split) also, Message split would only show up Messages of Type Event.

Would be The End of all the S+R hassle and patching would be much more <3!

i'd rather not add it to the common addonpack. tbh i'd like to develop a solid pack out of it, because i consider it most useful only for advanced application development. most work is done in that direction, as you can see here

thanks for the feedback. and yes i make good use of the generic nodes from the sdk. they are truely awesome, can't wait till nuget can be used from within our toolkit.

As you can see from my short-witted comments in the source i decided against keeping ringbuffer, buffer and cons. if i were to elaborate I'd say Zip does everything Cons does. I did replace it when cons did not behave as expected some alphas and betas ago. Even though these problems are probably fixed, I did not feel the need for a Cons since.

As for Buffer and Ringbuffer - I could not find use for them, but you can implement them yourself with 2 lines of code. I did not omit them because of laziness, but by choice of simplicity. I have not had a use case yet that couldn't be done with Store (Message).

If you have a use case, add a reference to %vvvv\lib\core\VVVV.Nodes.Generic.dll in the project explorer and a using VVVV.Nodes.Generic

which version did you try, x64 or x86? the one that is for download on the top of the page?

if you used the latest version from github you will have to compile in a different ide than vvvv, because as of yet vvvv does not support native nuget management.

anyway, thanks for hint with the csproj. if the pack is properly installed in /packs, it will automatically replace the nodes by their compiled counterpart in VVVV.Nodes.Messaging.dll, but will not reflect this autoreplace after saving the v4p, this needs to be done manually.

One question about performance:
My message would have the complexity of let's say 5 spreads of 8 floats each and 5 spreads of 4 bools each.
If a MIDI message changes a single value, in practise one of the spreads has to be sent and parsed for each MIDI message coming in, right? Unless i split the message into having 60 individual members.
I can't imagine that latency will stay down either way?
In comparison to Eksposing, isn't this a lot more string parsing behind the curtain?

Disclaimer: I'd be using about six different instances of such a message receiver with individual addresses.

For one, there is no string parsing going on behind the curtain. Remember, you are directly manipulating .net types, not some obscure comma-separated string representation of them.

Your case seems like a walk in the park for Message.

However, performance is a big topic for Message, so it would be awesome to get more feedback if you hit certain limits. I admit I have postponed most possible optimisation for a time when usage tells me what to optimize.

Right now I have a application with some 70+ widgets with a whole range of attributes like size, pos, color, Command when hit, type, source, etc. As long as they are not edited, there is no perf hit at all, because Message plugins will only do work when change has occurred (from upstream, and for Keep plugins, even downstream, but deferred by one frame).

But if change occurs, all the Message plugins will evaluate which might have a negative impact on framerate.

Note to self: Especially the deferred change detection from the Keeps seem to be too massive to be on by default.

edit: for your case, you should start with defining a Formular for your type, feed some Defaults into a Hold, and then use Edit nodes downstream. Feel free to post a patch for more help.

This new version is so fresh, some critical features are only working with the current alpha.

It now comes with a quick "Formular" definition to type your stuff across your vvvv application, with an advanced concept to hold Messages that I coined "Keep" and some nifty pin-management to detect actual changes and only then get active.

seems to be packed wrong, as it seems to have folders recursed such as nodes/core and nodes/nodes, and a different version of VVVV.Packs.Messaging.dll in the two core folders. I got it to work by taking the bottom-most nodes and using those, but not sure I got the right version of VVVV.Packs.Messaging.dll.

Then the girlpower examples, such as Durations gives red nodes for Message and AsMessage; looks like those have been renamed

It's been over a year since the last official update here on this site.

As some of you might know, there has been a lot of development on Message, usually freely accessible on github, but never here. To give everybody a chance to do some proper alpha testing (and general tinkering) without the need to compile yourself, I want to leave the release candidate for the next official release.

Download from our owncloud, and unpack both zips to the /packs folder of a current alpha vvvv install.
You will need to install dx11-vvvv as well.

Yes, I would like that, too. Even though I still don't know how to properly explain the pack, because its purpose is pretty high-level. And exotic "unfrozen object oriented data streams" are hard to picture, much harder than, say, imagepack or dx11.

One thing I learned from the last workshop is, that this pack offers solutions to problems, that users will only encounter after a few years of vvvv practice. So this time it should definitely cater to advanced users from the start, where I define Advanced as "has abandoned at least one promising project of her/his own making out of disgust about the unmaintainable patching".

Last two times I simply glided along the girlpowers, but that's not saying too much about the conceptual side of using vvvv-Message, like where to start and what to consider.
So requests for live-patching relatable use-cases are very welcome!

you probably made message to solve your own vvvv patching problems. it would be interesting to hear about them and how message solved them. i wonder about the ratio between "elegant solution" and additional node noise caused by message.

If this were for my own problems only, I could have cut down on dev time. I might even have sticked to poco, just because I can.
But I believe, that this object oriented flow approach can help greatly with having some actual architecture in your projects. It can enhance productivity with vvvv at the top end, when stuff gets really complicated for a patching environment.

That said, of course a lot of intolight project problems, or rather their solutions found their way into the pack, one way or the other. But that's natural, I guess.

In a nutshell, every serious patcher will eventually confront the problem of having a more "complex state", that needs to be made available to multiple modules, maybe even across multiple instances and machines. The standard way is to calculate that state somewhere singular, and send it (usually through multiple links, or even multiple S+R, or infamous hand-crafted serialisation modules) there, and put back together the pieces you need.

With message, you can put your state data into a single object (or multiple, however you like), and noodle that thing around, through one link. You can send it over network and serialize it to disk in a standardized manner.
You can partition the state-calculation into more than one module, that can act independent of, or interlinked with each other.

The elegance is plain: Everything is tangible in patch, and everything is geared for fast patching. The revolution is, that changes to a Message can affect Keeps that are connected only upstream, without having to use any FrameDelay (which I am sure gregsn would snarl at, hehe).

There is an absolute minimum of stateful nodes (only 4.5 of them right now). The rest is stateless in regards to flow data, which makes the kit amazingly straightforward to use, as long as you keep in mind, that any change to a Message will be so across your entire patch.
The nodes themselves are usually a tad more high-level than you would expect, so you can do much more with less.
The nodes of vvvv-Message know about the structure of the Message data (unlike a Zip and Unzip), and this knowledge is heavily utilized, so you don't have to constantly worry about it.

Here you see, how using Message will clear out half a dozen nodes, and make your patch more readable, while actually gaining possibilities (like making all data available to other modules, that can split out only the Fields of interest, or even edit it).

I guess you'll just have to try it yourself, and find some patterns (usually around the choice of a Keep), that suits your needs. That said, I am sure, this pack will provide you with moments of "Shit, how can this be so easy. This was soo hard all these years!".

Yes, totally elegant solution.
But when not using networking/osc and I only needed some custom datatypes to save myself from wiring loads of links through many levels of patches, woei's Struct seemed much simpler with less node overhead.

I guess Message offers way more convenience and features that may be needed at times, but I sadly just didn't have the patience to wrap my mind around of it works, as it seemed to need way more nodes for a simple setup (Keep volatile Messages vs. permanent Structs).

So, looking forward to learn more about this, pretty sure it solves other problems I have at times.

In the past, vvvv-Message lacked documentation, and I can understand anyone not investing the time with all these nodes, all doing very strange things here.
Because for basic scenarios, you might have to first learn them all? And to add to the bizarr: Just to read this, you'd have to scroll through endless lists of exceptions, that these nodes are producing?

But again, everything you know is wrong.

Start with a ConfigKeep node, and go from there with a Split, an Edit maybe. Those three nodes alone can make a tremendous difference in your patching experience. Read help patches, and learn at your own pace.

This is a major release here, and it's got half a year extra to be polished, thanks to the late release of beta35. So the suite is tested, prod ready and documented- also in code and patch.

I have for the last few weeks been getting to grips with your message pack and really loving it. I am using it for an elaborate preset loading and saving mechanism of about 140 different parameters, each with about 10 attributes.

To be able to load older presets even if I add new things, I need a way to edit the existing messages and overwrite the attributes of all the messages with existing topics. Edit(Message) does the trick, but I cannot specify topics there, so it just edits the values in the order they came in. I need something like Edit, but being able to specify which topics to change.

Say I set up using ConfigKeep 3 messages:

topic:Speed,
property:Speed,
value:1

topic:Lightness,
property:Lightness,
value:0.5

topic:Saturation,
property:Saturation,
value:0.1

Now I have a preset I want to load, which contains those messages as well. I can just read the preset (from .json file), split and use edit with Force to inject the new values.

But how can I edit those 3 messages if the preset I am loading only contains 2 of those and in a different order? I assumed I could just use ConfigKeep again but Force it, but that doesn't seem to work as expected - it only changes the messages coming out of ConfigKeep, but not the messages of the same topic and formular globally!? To be able to get to the topic I actually have another field called property, which uses the same string as the topic.

I need something like EditByTopic, where I can input a bunch of messages like Edit, but then have an input for Topic and any of the fields I choose like ConfigKeep and it sets the values for the messages with the existing topics.

Sift is your friend, if you want to filter by Topic (which often happens!). This would be between your ConfigKeep and the (forced) Edit.

Alternatively you can Split or Read all your Messages, and check the binsize of the attributes, to see if a field is not initialized, and use that to Select only the incomplete Messages to fix them with Edit. Or you use it to spread the Update pin of the Edit, to make sure you only update incomplete Messages.

There are really a few ways to do what you want, even without coding a new Edit node. The most brutal is Inject, but even ContainsField might be worth a look.

On a sidenode: currently there is no such thing as a "Field Order", so it does not matter, in what order they are in the json.

Hope that helps :)

Edit: @seltzdesign chatted me up directly, and explained his problem in depth. The easiest solution turned out to combine an Inject with a PruneBut.
If you want to know why, leave a like here and participate in Node17

Cannot confirm on a fresh beta35.5_x64 with vvvv-Message, installed with vpm. Unfortunately the log you provided does not show any hint of the reason this happens to you.
Can you please start a new thread (or a github issue, as you wish) with a link to a full startup log? Please also leave details, like your pc specs, and do you use x64 too? Which dx11-vvvv (or any other packs) do you use?

i think i had the same problem: when I downloaded the pack from github (as zip), for some reasons the dll's in the folder >nodes>plugins are not included, thus the nodes were not there at all.
It worked with vpm in the end.

@mfo can you share the patch? The Formular node turns red, if it cannot parse your definition and will reject the update bang. That can admittedly be confusing, but I never had a case, where this "didn't work" for a valid definition.

Also, the vpm script downloads and unzips the very same zips from the github releases, so missing dlls seem unlikely (maybe try with 7zip.exe?!)

Dunno how to upload files here. https://we.tl/FDRHatGcKS
There isn't much in the file anyways. No red nodes, no exceptions thrown. A simple fresh Formular, just a String Foo. After updating neither the Create nor the Split node show a pin for Foo.

Again and again, messages comes to the rescue. After quite a learning curve I think I got the basics now. Storing and loading presets in json format with around 250 different settings and 10 parameters each works nicely. Also use it in situations where you would need multiple send nodes and its working really well.