ProcessingJS for HTML5

Processing is the go-to visual programming language used most often by artists, VJs, data visualization types, and anyone who wants to make engaging programmatic visuals. The only downside of that is that it is basically JAVA with a wrapper that obscures most of the finicky or novice-unfriendly aspects of the language, and as such really doesn’t play nice with web browsers.

However! Processing.js is a javascript wrapper for Processing that makes your data visualizations, digital art, interactive animations, educational graphs, video games, etc. work using web standards and without any plug-ins. It’s HTML5 compliant, exposed to other javascript and even jquery functionality, and will compile to iOS using the Cordova libraries for XCODE– you can write a Processing app and sell it in the app store.

For realsies– here’s one I did: http://emaqdesign.com/apps.html

This tutorial shows how to build a standard, standalone ProcessingJS for HTML5 application, port it to an HTML page, and from there you can compile it to run directly on an iPad or even iPhone, using a combination of Processing for JS, XCODE, and HTML5 to iOS compilers. Programming environment is Mac OSX and XCODE (barf) and maybe some Eclipse if I feel like it.

About Processing:
Processing is an open source programming language and environment for people who want to create images, animations, and interactions. Initially developed to serve as a software sketchbook and to teach fundamentals of computer programming within a visual context, Processing also has evolved into a tool for generating finished professional work. Today, there are tens of thousands of students, artists, designers, researchers, and hobbyists who use Processing for learning, prototyping, and production.

About ProcessingJS:
Processing.js is the sister project of the popular Processing visual programming language, designed for the web. Processing.js makes your data visualizations, digital art, interactive animations, educational graphs, video games, etc. work using web standards and without any plug-ins. You write code using the Processing language, include it in your web page, and Processing.js does the rest. It’s not magic, but almost.

Resources: 
Processing home: www.processing.org
ProcessingJS home: http://processingjs.org/

STEP 1
First, download Processing and ProcessingJS. You’ll notice that there’s a couple different versions of Processing– file this knowledge away for future notice. There’s a bunch of external libraries and plug-ins for Processing that unfortunately tend to be version specific, and I’ve found that sometimes I can’t get something to compile unless I switch to an older (sometimes newer) version of processing.

STEP 2
Open up Processing and make a new sketch. This tutorial assumes a certain amount of familiarity with the Processing IDE and object oriented programming. If you’re new to these concepts, either go to the Processing website and familiarize yourself, or say “fuck it” and figure it out as you go!

So: Processing>File>New to make a new sketch. This is going to be super simple, and later on we’ll add some more complicated aspects. For now, let’s just make a basic mouse following circle. We’ll setup our colorspace as RGB, smooth the animation, set up our size as 800×600, and render a 25×25 pixel ellipse on mouse position every time the “draw” function is called.

[sourcecode language=”css”]
int tWidth = 800;
int tHeight = 600;

void setup(){
colorMode(RGB);
size(tWidth,tHeight);
background(0);
smooth();
}

void draw(){
fill(0,0,0,15);
rect(0, 0, tWidth,tHeight);
fill(255);
ellipse(mouseX, mouseY, 25,25);
}
[/sourcecode]

There’s a couple fancy things I put in there just so it’s not the boringest sketch in the world, but it’s pretty much the most basic “mouse-follower” sketch you’re going to run across. The fancy thing is the “rect(X,Y,Width,Height)” function that I call– this draws a rectangle the size of the sketch. The color is provided by the “fill” I invoke immediately before. Since we’re using RGB space, the “fill” of the rectangle is black (R,G,B = 0,0,0)with a 15% opacity. That’s how we get the nice fade.

You’ll notice that I’ve declared some variables outside the “setup” scope– we want certain variables to be outside the scope of a particular function, and exposed in our Processing sketch.

Hit the “play” button in Processing and make sure it compiles. Don’t worry about where you’re going to save it out since this code is going to be copied and pasted right into an HTML page.

Step 3
So, now that we have a sketch set up, let’s go ahead and run it in a browser! Now, Processing will allow you to compile as a java applet that we can run in our browser-of-choice, but as we all know JAVA is crazy insecure in a browser environment, and in fact anyone with half a brain has JAVA turned off by default. So, to get this Processing/JAVA sketch to run, we’re going to use ProcessingJS as a wrapper, which will then execute it in a <canvas> tag! Let’s go ahead and build an HTML page and tweak it so it will run our sketch!

[sourcecode language=”html”]</pre>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<title>Processing Demo</title>
</head>

<body>
</body>
</html>
<pre>
[/sourcecode]

Pretty basic stuff, right? This is the HTML stub that will contain our Javascript code that will enable the HTML page to interpret our Processing. Now we can add the Javascript code! You’ll have already downloaded the ProcessingJS libraries, so make a new folder called “js” in the directory that’s holding your HTML page, and put the “processing-1.4.1.js” file in there.

Feel free to take the version numbers off the .js file if you like, but I like leaving it in there so if I have some bugs, problems, or updates, I won’t go nuts looking up what version number I’m using.

So now that we have our working directory done, we’ll add the javascript import to the head of the document like so:

[sourcecode language=”html”]
<script type="text/javascript" src="js/processing-1.4.1.js"></script>
[/sourcecode]

And we’ll add the canvas tag to the body of the document– the ProcessingJS libraries will look for anything that is identified as “Processing” and execute it in the “canvas” tag.

[sourcecode language=”html”]
<canvas id="sketch"></canvas>
[/sourcecode]

Where does the Processing sketch code go, you ask? Well, there’s multiple ways to do it– I’m going to demo the easiest. Supposedly you can embed a .pde file right into the HTML, but I constantly get cross domain and cross-origin errors, so I’m going to stick with what seems to work best for me, which is to add the Processing code between two “script” tags that I include right before the “canvas”tag. You can also put the processing code in the head of the document and reference it from the body, but for the purposes of this demo we’re putting the javascript wrapper in the body of the document.

[sourcecode language=”html”]
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<title>Processing Demo</title>
<script type="text/javascript" src="js/processing-1.4.1.js"></script>
</head>
<body>
<script type="text/processing">
int tWidth = 800;
int tHeight = 600;
void setup(){
colorMode(RGB);
size(tWidth,tHeight);
background(0);
smooth();
}
void draw(){
fill(0,0,0,15);
rect(0, 0, tWidth,tHeight);
fill(255);
ellipse(mouseX, mouseY, 25,25);
}

</script>
<canvas id="sketch" style="border: 1px solid black;"></canvas>
</body>
</html>
[/sourcecode]

You’ll notice that all we did was put our Processing sketch info between the “script” tags. How does the javascript know to run processing? “script type=”text/processing” defines it as ProcessingJS enabled. No fussing about or “unescaping” anything, and although most IDEs will flag some of the Processing syntax as an error, the code will run without any hiccups. The “script” tags that hold your Processing sketch MUST be right before the “canvas” tag, or else it won’t run. No line breaks, extra code embeds, or anything else. You’ll also notice that we did not define much of anything in the canvas tag itself– the width, height and all that is inherited from the Processing sketch, we don’t have to use CSS to style the canvas tag. The border is there just so I can make sure the canvas data is being rendered, even if the Processing sketch itself is broken. I haven’t experimented with having CSS background images in the canvas tag and a backgroundless Processing sketch on top for some cool layer effects– is that even possible? Try it and let me know!

Step 4
Now, this is all well and good, but let’s keep our eye on the prize, which is making a generative visuals application that has interactivity. First off, it’s nice that “mouseX” and “mouseY” translate into touch events, that way we can get some basic functionality. However, things get complicated in terms of interface elements such as buttons and input text boxes, we can rapidly run into issues. For instance, Processing has a lot of nice 3rd party libraries for buttons, text input boxes, input dials and sliders, etc (ControlP5 I’m looking at you!) but they all use imported JAR files in order to run. My experience is, if you want to extend the functionality of your ProcessingJS sketch beyond the most basic Processing functions, you’ll have to use Javascript or JQuery.

This may be a security issue, and it may have to do with how the ProcessingJS wrapper works… I dunno. Either way, we have to roll our own interface elements such as buttons, sliders, toggle switches and whatnot… or we can use HTML inputs! Since “canvas” is an HTML5 tag, we can use all kinds of native HTML5 input types in our processing sketch. Sliders, search bars, toggles, buttons, text boxes and other inputs are exposed to Processing with a little trickery. This seems like a nit-picky concern… but when I program, I don’t want to spend a million years re-inventing a draggable slider.

The following may be a little confusing for beginners, since we’re going to be coding in HTML, Javascript, and Processing all in the same document, like having a bunch of linguini mixed with rotini and a couple fusilis… it’s all pasta, but there’s some minor differences. Wherever possible I’ve indicated what kind of code it is and where it should go in our HTML page. I’ve also included the code for entire working file at the end of this ‘tute.

So, let’s get down to it boppers!

We’ll first do something simple, like have a slider that changes color. Then we can expand it to have any other functions that we want. HTML5 has a nice slider that we can use, so let’s go ahead and make a “div” tag that will contain our interface elements. We’ll give it an ID called “interface” so we can do stuff with it later on if we want. Then we’ll add a basic slider input element and have that tell our processing sketch to change color. Put the following HTML code right after your canvas tag.

[sourcecode language=”html”]
<div id="interface" style="width: 800px; height: 100px;">
<input type="range" class="colorslider" min="0" max="255" value="10" step="1" onChange="updateColor(this.value)"/>
</div>
[/sourcecode]

The slider input has a bunch of different elements to it– min value, max value, current value, its step value, and, of course, “updateColor”, an “onChange” action that calls to a function (that we haven’t built yet) called “updateColor”. That function is then going to talk to our Processing code by calling to ‘sketch’, the ID of the canvas tag that displays our Processing sketch.

Let’s make the “updateColor” Javascript function. We’ll put this in the head of our document, right after our ProcessingJS import function. Notice that we make a variable called “processingInstance” that we instantiate by telling it to get our previously mentioned canvas ID. This variable is what we call whenever we want to talk to our processing sketch– when you start building more complicated interactivity you’ll want to make this a global variable that you instantiate once on page (or device!) load.

[sourcecode language=”javascript”]
<script type="text/javascript">
var processingInstance;
function updateColor(newValue){
console.log("SLIDER: " + newValue);
processingInstance = Processing.getInstanceById(‘sketch’);
processingInstance.changeBGColor(newValue);
}
</script>
[/sourcecode]

When you run this in your browser you should see your processing sketch, a slider, and when you move the slider your output window (firebug? chrome developer console? whatever) will show the value. It will also tell your Processing sketch in the canvas tags to call the function “changeBGColor”… but we haven’t built this function in our sketch yet, so the window will also throw an error.

Let’s build our “changeBGColor” function in our Processing sketch. First we’ll add some variables that will hold our color values. Since our Processing sketch is in RGB, we’ll need three different variables. By passing the slider value to the variables, it will change the background color. For instance, I’m going to add a variable “tRed” which will stand in for the red value, and add it to the rectangle I draw that makes the background. Then I’ll add the “changeColor” function that we pass our value into, and PRESTO! Slider control of our red value.

[sourcecode language=”javascript”]
int tWidth = 800;
int tHeight = 600;
int tRed = 255;

void setup(){
colorMode(RGB);
size(tWidth,tHeight);
background(0);
smooth();
}

void draw(){
fill(tRed,0,0,15);
rect(0, 0, tWidth,tHeight);
fill(255);
ellipse(mouseX, mouseY, 25,25);
}

void changeBGColor(int tValue){
tRed = tValue;
}
[/sourcecode]

Now you should be all good! We have an HTML slider that controls the background color of our Processing sketch by accessing a Javascript function in the head of our document. If you’re a big fan of inline Javascript you might be tempted to put this call right in the code for the input tag, but I like breaking out all the functions into their own area so I can debug more easily. Here’s the full code for the page– it’s not too exciting looking, but the possibilities are pretty cool!

[sourcecode language=”html”]

<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<title>Processing Demo</title>
<script src="js/processing-1.4.1.js" type="text/javascript"></script>

</head>

<script>
var processingInstance;

function updateColor(newValue){
console.log("SLIDER: " + newValue);
processingInstance = Processing.getInstanceById(‘sketch’);
processingInstance.changeBGColor(newValue);
}

</script>

<body>
<script type="text/processing">
int tWidth = 800;
int tHeight = 600;
int tRed = 255;

void setup(){
colorMode(RGB);
size(tWidth,tHeight);
background(0);
smooth();
}

void draw(){
fill(tRed,0,0,15);
rect(0, 0, tWidth,tHeight);
fill(255);
ellipse(mouseX, mouseY, 25,25);
}

void changeBGColor(int tValue){
tRed = tValue;
}
</script>

<canvas id=’sketch’ style="border: 1px solid black;"></canvas>
<div id="interface" style="width: 800px; height:100px;">
<input type="range" class="colorslider" min="0" max="255" value="10" step="1" onChange="updateColor(this.value)"/>
</div>
</body>
</html>
[/sourcecode]

Sweet! Now for the next step— take this HTML page and make it into an iOS app that you can run on an iPad or iPhone!