Freeframe plugin from Processing Sketch


Processing for Resolume is a walkthru, from download to install to finished project, of how to build a Freeframe Plugin from a Processing sketch with custom Processing controls spawning from inside the Resolume environment itself!

About Processing

from Processing.org:
Processing is an open source programming language and environment for people who want to create images, animations, and interactions. Initially developed to serve as a software sketchbook and to teach fundamentals of computer programming within a visual context, Processing also has evolved into a tool for generating finished professional work. Today, there are tens of thousands of students, artists, designers, researchers, and hobbyists who use Processing for learning, prototyping, and production.

About Spout

Spout is a realtime video sharing framework for Windows. Senders and receivers include FreeframeGL plugins, a Java interface for Processing, Jitter externals for Max/Msp, VIZZable, modules for Ableton Live, and a Virtual Webcam as a universal receiver. There is also example code for creating your own applications with openFrameworks and Cinder. Now these applications running on Windows can share video with each other in a similar way to Syphon for OSX.

The team behind Spout does not charge for their work, and there’s a lot of time, effort, and licensing that goes into keeping the project running. If you want to support them here’s your chance!


About this Processing for Resolume tutorial

Use this tutorial however you want, feel free to add to it, be sure to forward it to whomever and give it away for free. If you charge some lazy bastard for this tutorial, you suck. If you use it for a performance and you make money– you rock! If you have some questions, advice, or praise, contact me at:
info(at)ericmedine(dot)com

What you will need:

– Computer (duh), PC running at least win7, ATI Radeon HD 5000 series or better. NVIDIA GeForce 210 or better. 256MB of VRAM, 2GB RAM (these are the minimum system requirements for Resolume)
– Processing environment (64 bit– important!!) and compiler (download here: https://processing.org/download/?processing
– Spout for Processing Library (64 bit– important!!). This is a library for Processing that lets you run Spout. You can add it to your Processing library by starting Processing, then going to Sketch > Import Library > Add Library and then search for Spout (or download here: https://github.com/leadedge/SpoutProcessing/wiki
– Spout Controller executable and demo (download the executable here: http://spout.zeal.co/download-spoutcontrols/

NOTE: YOU SHOULD MAKE SURE THE DEMO SKETCH WORKS ON YOUR CONFIGURATION BEFORE GOING THROUGH ALL THE TROUBLE OF MAKING YOUR OWN!
It’s easy… here’s a 4-step guide on how to get the sketch running in Resolume.

Other resources:

Processing experiments:
http://www.openprocessing.org/
Spout:
http://spout.zeal.co/
Spout Controller
My website:
http://www.ericmedine.com

NOTES ON PROGRAMMING ENVIRONMENTS
You may prefer to use the Eclipse environment to work on your Processing files. I personally prefer it due to its code hinting, build management, and ease of use in integrating other JAVA libraries. The basic functions in this tutorial are designed to work in the standard Processing environment though.

You shouldn’t need to install the JAVA runtime in order to get Processing to work– it has its own runtime. However, if you’re getting weird compiler errors when running sketches with a lot of 3rd party libraries you may have to do just that.

Spout Control for Resolume
Step 1: Download and install Processing from Processing.org, the SpoutController from GitHub, and download the demo sketch(es).
Step 2: Build a simple Processing sketch and copy the spout support files into your sketch folder
Step 3: Compile the sketch and drag the folder into Resolume/plugins/vfx
Step 4: Launch everything and watch the magic unfold!

STEP 1

spout_installer_windowFirst, download Processing and Spout. You’ll notice that there’s a couple different versions of Processing– file this knowledge away for future notice. When this tutorial was written the current version of Processing was 3.1, and although it seems pretty stable I tend to stay a version behind. However, we want 64 bit for this and Processing 2.1 doesn’t always export 64 bit correctly it seems…. I dunno. Windows gives me a headache. There’s a bunch of external libraries and plug-ins for Processing that unfortunately tend to be version specific, and I’ve found that sometimes I can’t get something to compile unless I switch to an older (sometimes newer) version of Processing. Try everything with Processing 3 and see how it goes I guess? Processing for Resolume has been written with the most current technology in mind.

Next, download and install the Spout Controller installer from GitHub at https://github.com/leadedge/SpoutControls. It seems to be quite stable to me, but any bug reports or suggestions I’m sure would be appreciated.

Finally, download the demo sketches. The reason you want this is that they have critical spout support files that you’ll need in order to make the sketch run– SpoutController.dll, JSpout.java, spout.pde, as well as the “code” folder. These will ALL have to go in the folder of your new Processing sketch.


STEP 2
Let’s get down to it, boppers! This is going to be super simple sketch initially, and later on we’ll add some more complicated aspects, as well as the support files that Spout needs to talk to Resolume. For now, let’s just make a basic mouse following circle. We’ll setup our colorspace as RGB, smooth the animation, set up our size as 800×600, and render a 25×25 pixel ellipse on mouse position every time the “draw” function is called.

[sourcecode language=”actionscript3″]

void setup(){
colorMode(RGB);
size(800,600, P3D);
background(0);
smooth();
}

void draw(){
background(0);
fill(255);
ellipse(mouseX, mouseY, 25,25);
}
[/sourcecode]

It’s pretty much the most basic “mouse-follower” sketch you’re going to run across.

Hit the “play” button in Processing and make sure it compiles. You should probably save your sketch out as well. NOTE! It’s really important that your sketch name is the EXACT SAME NAME as the folder it’s in, or else it won’t run. Also, did you remember to put those crucial support files from the demo in your folder?



Ok, now that you’re feeling good with that, let’s add the Spout controls that hook into Resolume. We’ll put the initial declaration outside of our “setup” function and “draw” loop so they’ll be accessible to the main class. This also makes sure that the spout class is loaded at runtime.

NOTE: The support files that allow this sketch to run correctly do one other very important thing– they export a configuration .txt file that allows Resolume to “see” the sketch hooks. This file is included in the demo sketch (“CirclesLines.txt”) but if you’re rolling your own from scratch, it may not exist. HOWEVER, if you run your sketch once it will be created in the folder.

It MUST have the same name as your sketch and the folder it’s in. If this file is not created when you run your sketch for the first time, you may not have the correct support files included with your sketch.

[sourcecode language=”actionscript3″]

// DECLARE A SPOUT OBJECT HERE
Spout spout;

//// spout control vars
String sendername;

String[] controlName;
int[] controlType;
float[] controlValue;
String[] controlText;
String UserText = "";

void setup(){
colorMode(RGB);
size(800,600, P2D);
background(0);
smooth();
}

void draw(){
background(0);
fill(255);
ellipse(mouseX, mouseY, 25,25);
}
[/sourcecode]

So– we’ve declared our Spout object that initializes the Spout class in “spout.pde” (you did copy those files into your folder, right? You might have to quit and restart Processing so it will recognize them). We’ve also set up the arrays that will hold our control data. This data has 3 characteristics– the type it is (controlType), the value it’s passing (controlValue), and its name (controlName). There’s also a var that will hold any text we decide to pass in, but we won’t really need this for now.

Next we have to make sure that we initialize our control parameters. SpoutController and Resolume really only handle 3 types of inputs: Boolean (true/false), Strings (text) and Floats (numbers– this includes all numbers including negative numbers and decimals).

So whatever we build with our sketch, any controls we use will be restricted to those types of variables. We’ll do this inside our “setup” function:

— First, create our new Spout object that we declared earlier
— Second, initialize our sender name (this is important, I’ll talk about it later)
— Third, initialize the arrays that will hold ALL our controls (the ones we declared earlier).

[sourcecode language=”actionscript3″]

void setup(){

colorMode(RGB);
size(800, 600);
background(0);
smooth();

// CREATE A NEW SPOUT OBJECT HERE
spout = new Spout();

// INITIALIZE A SPOUT SENDER HERE
sendername = "MySenderName";
spout.initSender(sendername, width, height);

// Create control arrays
controlName = new String[20];
controlType = new int[20];
controlValue = new float[20];
controlText = new String[20];

// Create controls
spout.CreateSpoutControl("User Text", "text");
spout.CreateSpoutControl("Show Lines", "bool", 1);
spout.CreateSpoutControl("Number Circles", "float", 0, 2, 1);

// Open the controls for the controller using the sender name
spout.OpenSpoutControls(sendername);

}

[/sourcecode]

So you can see that we have examples of all three of our control types– text, true/false, and number. These three control types will tell Resolume to create those controls in the Resolume environment and listen to their inputs.

NOTE! Be sure to verify that our “sendername” is correct. Right now it’s called “MySenderName”, and in the demo sketch it’s called (totes obvs) “CirclesLinesDemo”. This name HAS TO MATCH your main sketch name in your folder, otherwise the JSpout.dll will not be able to hook your Resolume output to your Processing sketch. It doesn’t matter what the name is… it can be “dognose”, as long as your main sketch is named “dognose” and the folder it’s in is also called “dognose”. Processing for Resolume requires that these naming conventions are consistent and clear.

Easy peasy! Of course, our Processing sketch is not listening to the controls yet… let’s add that in!

Let’s add a “for” loop in the “draw” function that checks through all the control arrays and checks to see if any data is coming in that matches their names (that we declared up in the “setup” function). If there is data, then do something!

This “for” loop is wrapped in an “if” statement just to make sure the sketch doesn’t have a panic attack if something gets initialized out of order (unlikely), if there’s a no controls at all, or there’s an error declaring the vars (more likely).

[sourcecode language=”java”]

void draw() {

background(0);

//////////// CHECK THE INPUTS
// SPOUTCONTROLS – Check for updated controls
int nControls = spout.CheckSpoutControls(controlName, controlType, controlValue, controlText);
// print("nControls = " + nControls + "n");
if (nControls > 0) {
for (int i = 0; i < nControls; i++) {
// print("(" + i + ") : [" + controlName[i] + "] : Type [" + controlType[i] + "] : Value [" + controlValue[i] + "] : Text [" + controlText[i] + "]" + "n");
if (controlName[i].equals("User text")) {
if (controlText[i] != null && !controlText[i].isEmpty())
UserText = controlText[i];
}
if (controlName[i].equals("Number Circles")) {
//// do something here
}
if (controlName[i].equals("Show Lines")) {
// do something here
}
}
}
// this function draws the circles
drawCircles();

// SEND A SHARED TEXTURE HERE
spout.sendTexture();
}

[/sourcecode]

One thing that’s important to notice is the final line of code: spout.sendTexture();

This bad boy is what sends the image information from inside Processing to resolume to display– remember we’re not just sending control data back and forth, we’re sending image data as well!

You’ll also notice the “drawCircles” function. Don’t stress– we haven’t built that yet.

Now that we have input coming in from Resolume, input being checked by Processing, control data being passed from Resolume to our sketch, let’s have our circle-drawing object function react to that data.

Right now we have a circle following a mouse, which is not so exciting. Instead, let’s have a bunch of circles moving around randomly and have them connected with lines, doing the science-y motion graphics thing that people love to use Processing for.

To do this, instead of drawing just one circle in the “draw” function, we’ll have our “draw” function handle a bunch of circles that are controlled independently. First we’ll make a “BasicCircle” class by adding a new “tab” in processing.

 

[sourcecode language=”actionscript3″]

class BasicCircle{

float tSize;
PVector tPos = new PVector();

BasicCircle(float x, float y, float size){

tPos.x = x;
tPos.y = y;
tSize = size;
}

void update(){

fill(color(255,255,255,125));
ellipse(tPos.x, tPos.y, tSize, tSize);
}

//// end class
}

[/sourcecode]

The circle needs a start position (notice that we’re using PVector, a Processing-specific var that stores an ‘x’ and ‘y’) and size that we pass it on initialize, and an “update” function that we can call to control its position from our main class (or anywhere, but main class for now). Now we can make a circle-holding array list, spawn a bunch of circles and add them into the list, then use a “for” loop to update everything. The amount of circles in our array will be controlled by (surprise!) Resolume.

Since we will be adding and removing objects from the array we’ll need to use an array list instead of a regular array, which can’t change once it’s initialized. Info about array lists is here: https://processing.org/reference/ArrayList.html.

So go back up into the top of our sketch and add an array to hold our circles and set a variable to hold the number of circles that we start with, the size of the circles, and a variable that defines the maximum amount of circles we can have on screen at any time. Remember– we are going to be able to set the amount of circles, so we need to set that range. How about anywhere from “0” to… “200”. Let’s do 200.

Also, we will want to set up some temporary circle objects that we can initialize and garbage collect later on, when we start doing some more complicated animations. I’ll explain that later on.

[sourcecode language=”actionscript3″]

int numCircles = 15;
int maxCircles = 200;
int circSize = 15;
/// circle objects
BasicCircle tCirc;
BasicCircle nextCirc;
BasicCircle nextRCirc;
ArrayList <BasicCircle> CircleArray = new ArrayList();

[/sourcecode]

In our “setup” function we’ll build a “for” loop to populate our array with the current number of circles and initialize a random position. Put it anywhere in the “setup” function.

[sourcecode language=”actionscript3″]

/// create all circles
for (int i=0; i< numCircles; i++) {

float zx = random(0, 800);
float zy = random(0, 600);
/// initialize a new circle from our
/// BasicCircle class and add it to our ArrayList
BasicCircle tCirc = new BasicCircle(zx, zy, circSize);
CircleArray.add(tCirc);
}

[/sourcecode]

Remember how we had that “drawCircles” call in our “draw” loop? Let’s build that now! It uses a “for” loop to draw each circle.

[sourcecode language=”actionscript3″]

void drawCircles(){
for (int i = 0; i<CircleArray.size(); i++) {
tCirc = CircleArray.get(i);
tCirc.update();
}
}
[/sourcecode]

Run the sketch and make sure you don’t have any compiler errors. You should see 15 circles, randomly positioned, not doing much of anything. Now let’s hook up our “update” functions to our Spout controls. We’ll go to our “Number Circles” data check in our “draw” loop and pass it the number we get from Resolume (newNum) from “controlValue” to a function called “updateNumberCircles()” that we haven’t written yet. This function will add (or subtract) BasicCircle objects from our CircleArray, depending on if the number is greater or less than the current size of our CircleArray.

[sourcecode language=”actionscript3″]

if (controlName[i].equals("Number Circles")) {
int newNum = int(map(controlValue[i], 0, 1, 0, maxCircles));
updateNumberCircles(newNum);

}

[/sourcecode]

Notice that we’re using the “map” function, which is specific to Processing (more info here: https://processing.org/reference/map_.html). We use this to convert the value that we get from Resolume (a range from 0 to 1) to the new number of our circles (a range from 0 to “maxCircles”, which is the upper limit of circles onscreen that we defined earlier on).

Now let’s create our “updateNumberCircles” function:

[sourcecode language=”actionscript3″]

//////// UPDATE NUMBER OF CIRCLES ///////////////////////
void updateNumberCircles(int tNum){

// if the new number is bigger
// than the current number of
// circles in our array
// then add them
if(tNum > CircleArray.size()){
for(int i=0; i<tNum; i++){
float zx = random(0, tWidth);
float zy = random(0, tHeight);
BasicCircle tCirc = new BasicCircle(zx, zy, tSiz);
CircleArray.add(tCirc);
}
}
// if not then remove the last
// item in the array
if(tNum < CircleArray.size()){
for(int i=CircleArray.size(); i>tNum; i–){
CircleArray.remove(CircleArray.size() – 1);

}
}
}

[/sourcecode]

One thing to point out is that when we remove circles we want to count backwards so we’re not going to get an “array out of bounds” error. Try compiling your sketch now and using Resolume’s controls– you should see some boring looking circles, not moving… but you can add and subtract them from inside Resolume!

This is pretty exciting in a “we did something technical” way, but kind of dull in a “this is hella boring looking” way. Let’s add some movement, then add some lines to connect the circles!

Ok, so… movement can be any damn thing, but I found a simple “wander” function that seems to do what I want. Try searching for “random drift Processing” and someone will have a ‘tute for you. I added it to the BasicCircle class. Rather than break it down, I’m gonna copypasta since I don’t really know or care how it works. Something about “sin” and “cosin”.

Also notice the “checkBoundaries” function– this makes sure that the circles move off one side of the screen they’ll respawn on the other.

[sourcecode language=”actionscript3″]
class BasicCircle {

float tSize;
PVector tPos = new PVector();

float wander_theta;
float wander_radius;

// bigger = more edgier, hectic
float max_wander_offset = 0.1;
// bigger = faster turns
float max_wander_radius = 2.5; //// seems to do nothing

BasicCircle(float x, float y, float size) {

tPos.x = x;
tPos.y = y;
tSize = size;

wander_theta = random(TWO_PI);
wander_radius = random(max_wander_radius);
}

void update() {
float wander_offset = random(-max_wander_offset, max_wander_offset);
wander_theta += wander_offset;
checkBoundaries();
tPos.x += cos(wander_theta);
tPos.y += sin(wander_theta);

fill(color(255, 255, 255, 125));
ellipse(tPos.x, tPos.y, tSize, tSize);
}

void checkBoundaries() {
if (tPos.x > 800) {
tPos.x = 0;
}
if (tPos.y > 600) {
tPos.y = 0;
}
if (tPos.x < 0) {
tPos.x = 800;
}
if (tPos.y < 0) {
tPos.y = 600;
}
}

//// end class
}

[/sourcecode]

So, you should have a Processing sketch that has a bunch of circles drifting around randomly. Nicer, but not a lot going on still. Compile it and make sure it works.

Does it work? If so, let’s add some lines and connect them, and for extra fun add in a toggle so we can switch them off from inside Resolume!

First we’ll have to build a function to draw lines. It’s simple enough– do a “for” loop somewhere that gets the position of each circle from our CircleArray and sets the start point of our line to that… and then set the END point of our line every other circle in the array. Nested “for” loops are perfect for this.

Once again, notice that we’re accessing the “x” and “y” values of our PVector positioning.

[sourcecode language=”actionscript3″]

/// draw lines
void drawLines(){

for (int i = 0; i<CircleArray.size(); i++) {
/// get a circle
tCirc = CircleArray.get(i);
int nxtId = 0;
// get every other circle
for(int j = 0; j< CircleArray.size(); j++){

nextCirc = CircleArray.get(j);
stroke(255);
strokeWeight(1);
/// draw a line between tCirc and nextCirc
line(tCirc.tPos.x, tCirc.tPos.y, nextCirc.tPos.x, nextCirc.tPos.y);

}

}

}

[/sourcecode]

And now, in our “draw” loop, add a function call for it– “drawLines()” right below “drawCircles” function that already exists.

[sourcecode language=”actionscript3″]

// this function draws the circles
drawCircles();
drawLines();

[/sourcecode]

Not super exciting tho. How about we toggle the lines on and off using Resolume? Simply make a new variable (showLine) at the top of your main sketch, and listen for the Boolean value being passed by Resolume to switch the “drawLines” function on and off.

Remember the boolean listener we have in the “draw” loop? Add its value to the “showLines” variable like so:

[sourcecode language=”actionscript3″]

if (controlName[i].equals("Show Lines")) {
showLine = (boolean)(controlValue[i] == 1);
}

[/sourcecode]

and then switch the “drawLines” function to only fire if the variable is true, like so:

[sourcecode language=”actionscript3″]
// this function draws the circles
drawCircles();
if(showLine){
drawLines();
}

[/sourcecode]

And this is what it should look like!

Huzzah! You have a moving, vaguely geometric shape that drifts around and you can control its number of points with Processing, and hide and show the connecting lines.

For added coolness, try having a line-length checker that will only spawn lines if circles are a certain distance from each other… or add in another function that goes through every three adjacent circles and draws triangles between them, rather than lines between everything.

Right now it “wraps” when the circles move off screen. Throw some directional flags into BasicCircles so they bounce, rather than wrap!

Once you start doing stuff like that you might try building polygons using the circle co-ordinates you already have, maybe start adding a Z-index to your circles and lines.

The sky is the limit!

What you will need to run Processing in Resolume:

– Computer (duh), PC running at least win7, ATI Radeon HD 5000 series or better. NVIDIA GeForce 210 or better. 256MB of VRAM, 2GB RAM (these are the minimum system requirements for Resolume)
– Processing environment (64 bit– important!!) and compiler (download here:  https://processing.org/download/?processing
– Spout for Processing Library (64 bit– important!!). This is a library for Processing that lets you run Spout. You can add it to your Processing library by starting Processing, then going to Sketch > Import Library > Add Library and then search for Spout (or download here: https://github.com/leadedge/SpoutProcessing/wiki
– Spout Controller executable and demo (download the executable here:  http://spout.zeal.co/download-spoutcontrols/

Instructions for running the demo sketch in Resolume

1) Unzip and drag the folder with your sketch (should be called CirclesAndLines) into Resolume/plugins/vfx folder and run it once (open it in Processing and click the “Play” button).

2) Find the FREEFRAMEGL>x64 folder inside your SpoutControls folder (you’ve installed SpoutControls, right?) and drag/copy the “SpoutController.dll” file into the Resolume/plugins/vfx folder. IF YOU DON’T DO THIS YOUR CONTROLS WON’T SHOW UP IN RESOLUME!

3) Start Resolume. The correctly named sketch title (“CirclesAndLines”) should appear in Sources/FFGL Sources

4) Drag the title (“CirclesAndLines”) from Sources/FFGL Sources into a Resolume cell just like you would any other video source.

Processing should launch behind the scenes (it may ask if you are sure you want to launch the .exe, click ‘yes!). You have installed Processing 64bit on your machine, right? Right?

5) Click on the cell and the sketch should play, it should have controls like “Number Circles”, “Show Lines”, etc.

UPDATE: I’ve had to turn comments off since I’ve been getting mad spam on this post in the last couple days. Sorry for the hassle. If you have questions you can email me directly.