BenjaminBenBen

Offline QR Codes

2025-08-19T00:00:00Z

Using QR Codes to send data from offline devices

👉 From a sensor

If a sensor has a display, it can show a qr code with the data encoded in a url. An example is this temperature sensor sharing recent values in a collector page.

🛜 Everything works offline. The microcontroller has no network code and the collector page can work offline.

🔋 Power efficient. E-ink displays don't require continous power so controller could go into deep sleep between updates.

🔨 Robust. You could take a polaroid photo of the sensor and mail it to your friend for them to scan it if you want!

💾 Collection. Storing historical data is possible by forwarding to server when the collector device is back online.

👉 From a web page

You don't need a microcontroller to collect data. Browsers are a rich source of content. When you draw in the box below it'll encode your art in a qr code (source).

You might notice as you draw more lines the QR code becomes more dense. I implemented some rudimentary path simplification, but there's limits to the amount of content you can share this way.

The key requirement is that target page state should be derived from the url, and once you've got that you can save and share content in any number of ways, not just a qr code.

👉 Between web pages

Browsers can also scan QR Codes. This allows us to create a bidirectional socket between two offline devices using their front-facing cameras.

I implemented this as a joke, but after a while it started working oddly well.

To manage the connection I used a basic protocol; a client bootstraps the connection by displaying the url of the current page (useful for setting up the second device). When a matching url is detected the devices go into "data mode", where a string of [RX, TX, Data] is encoded in the qr code.

RX - last message id seen by the device
TX - message id being transmitted
Data - payload (string)

Messages are chunked to a fairly arbitrary string length. This could definitely be improved to target the capacity of a particular code size. An interesting extension could be switching between different resolutions to pick the highest capacity for a camera+screen combination.

Side note: QR Codes support structured append for spreading messages over multiple codes. Pretty cool, but I didn't use it.

The coding interface feels a bit like a WebSocket:

const qs = new QRSocket();

qs.on("message", (message) => console.log(message));
qs.send("Hello World!");

One area for improvement is disposing of the socket; currently you have to close the page otherwise the camera will still be active. Having some method for pausing the socket might make this feel more natural.

🕹️ There's some demos online at remotehack.space/QR-TX and source on github.

Send - Basic example that sends text of different lengths.
Chat - Send multiple messages between devices. Panda and I built this at remote hack, it was a lot of fun and slightly mind-bending when we were lining up our phones remotely over a discord room.
Signalling - Negotiates a peer-to-peer webrtc video link between devices.

[Chatting](https://remotehack.space/QR-TX/?demo=chat) between my desktop & phone

As before, these demos have a Service Worker which caches the page. I've enjoyed giving demos to people before showing them that my phone is in aeroplane mode 🤯. It feels like there's interesting use cases of being able to sync with someone when either of you are completely offline.

I shared a quick demo at Future of Coding [slides] last year. And a longer talk about QR Codes and other stuff at MKGN [video].

More things

That I've built:

QR Code Scanner - This scanner auto-redirects to any detected urls (even if that url is 'javascript:alert("hello")' 😬).
QR Undistort - This looks for a QR Code from a camera feed and transforms the video element so that it's central. Like this!
QR Text - An text-based version of qdraw.

That I like:

"I built a QR code with my bare hands to see how it works" - great video exploring how qr codes work. This made reed solomon encoding finally click for me.
Snake in a QR Code - there's something I love about "chmod +x" on something you captured from your webcam.
qrs & txr - cool approach for one-way streaming using fountain codes.
meshtastic - not QR codes, but it's pretty cool to use WebBluetooth to communicate with devices over a different network.

Video Space

2025-07-16T00:00:00Z

Here's a video that's played back in the space it was captured

👀 It's interactive, drag the viewer to adjust the viewpoint

← More videos

Locating frames

I was hoping to use the telemetry data from my drone; it produces a text file with it's location as it captures video. However this doesn't include orientation or camera gimble info so I wasn't able to map it into a pose.

So I decided to use COLMAP, a Structure-from-Motion tool which allows you to take a series of images to build a scene. COLMAP stores the position from which each image was captured which I was able to use for aligning the video frames. As a bonus, this works for other video sources, not just drone footage.

I wrote some slightly scrappy code to extract and serialise the poses and points into a ply file that I could load into a webgl component. You can read some of the process (and see some gaussian splats) on this bluesky thread.

Implementation

I’m pretty happy with how this is structured, it’s a web component that wraps a <video /> element with links to the colmap data:

<pose-tracker poses="poses.ply" points="points.ply">
  <video src="motocamp.mp4"></video>
</pose-tracker>

Internally the video element is hidden but still drives the playback of the component, which is some html controls & a webgl canvas element.

The canvas is rendered by threejs. The key trick is using a single 2d texture array to stash the historical video frames, with an instanced mesh that allows everything to be drawn together. My original approach for pushing the frames was using a 2dCanvas to write the pixels into a array buffer, but I found WebGLArrayRenderTarget which lets you populate texture arrays directly!

I didn't want/need every frame of the video, so I sampled it (from 60 → 2-5 Hz) and interpolate to find the position at a set timestamp. Orientation is quite straightforward in threejs, but for translation I was really happy when I found curve-interpolator.

Data sources & formats

Using structure from motion is cool, but you can get potentially richer data from sensors on the capture device.

For drones, the UZH-FPV Drone Racing Dataset is a great example of the sort of data that's available.

For capturing from a mobile device WebXR Raw Camera Access could be an option for capturing pose-aligned video.

GoPro cameras have a telemetry format which looks like it captures a bunch of metadata.

And for output formats, I enjoyed using ply because it's so lightweight/flexible (it can be just a text file!). But if I was doing this properly I'd probably use something like mcap to link everything together.

I've got a fairly shonky pipeline for processing videos now, so if there's something that you think would be interesting give me a shout and I'll run it through!

Stacks Camera

2025-06-29T00:00:00Z

I’ve been working on a side project - stacks.camera

It’s a tool that lets you capture a series of images by overlaying the most recent capture when you’re taking a new photo.

You can see my public stacks at u/ben.

Here's some that I've enjoyed creating:

Window - I started taking pictures out of my window as a test, but I’ve found it pretty satisfying to keep adding to.
Circles - stacks can be collaborative, this is one that I created to experiment with that.
Pipes - it’s been fun to walk around objects and create animations.
Poolbeg Chimneys - Sarah and I captured this on a walk along Dublin Bay, it felt quite fun!

Why I've been working on this

I’ve been playing around this topic for years (gif-stack, video-stills, scrub, overlay), I wanted to make something more than a hack/proof-of-concept, and see if resonates with anyone.

It was also a chance to experiment & learn with some stuff. I’ve been learning a lot about Cloudflare Workers/Images/Workflows, Passkeys, image loading/rendering approaches, and also seeing where AI tooling can fit into my development.

It’s rough around the edges, but I’ve enjoyed the decisions that have made it that way! Please do sign up (it’s very minimal), and I’d love to know what you think.

👉 stacks.camera

Dublin London Dublin

2025-06-12T00:00:00Z

I've not posted here for a while 5 years 😱

This is some stuff that I've been doing since then:

Working at Oxa I had super fun times working on Map Tech and various web things across the company.
Moved to Dublin with Sarah I packed for a couple of weeks, which covid turned into a couple of years.
Got a job at Wayve and moved to London started out in Experiment Orchestration 🙅🫶 then created the Robot Web team to focus on in-vehicle and remote access tools.
Sarah moved to London after a year of us travelling between the UK & Ireland, Sarah came to join me in London.
Sarah and I got married it was a lovely day with lovely people 🥰.
Left Wayve and moved back to Dublin earlier this year we left London and returned to Dublin.

I've managed to travel a lot - Namibia, Whistler, France, Chile and a few trips to California for work. I've taken some motorbiking trips, been snowboarding a bunch of times, and travelled by bicycle on a few trips.

During lockdowns we started Remote Hack which was such a lovely way to connect and explore some fun projects. And in person I loved catching up with folks at places like MKGN, OGN & emfcamp.

Next up, I'm taking some time off to relax, recharge and work on some side projects before I start looking for a new roles later in the summer.

PS. This is where I'd usually curse myself by saying I'll post more often, but I think I'll just leave it at that for now!

Hacks

2020-05-26T00:00:00Z

I built some things a (long) while ago – I found it really satisfying but never bothered to blog about it.

Cardboctober

Way back in 2016, Max, Pete & I tried to make a Google Cardboard hack for every day of October.

It was an awesome experience. There's something very liberating about the daily time constraint; it's stops you striving for perfection, and lets you move on to something new without feeling regret.

I also loved sharing the experience with Pete and Max. We all took different directions with our hacks. I was trying to build my own render engine, Max was exploring UX ideas and Tetris, and Pete went off playing with cats, fish, the moon, archery, lasers and beer cans.

Webgl hacks

Into 2017 (don't worry, I'll be blogging about newer stuff soon), I decided to learn WebGL from the ground up. Before then, I'd used it a bit through libraries or frameworks.

I spent a whole day on my first hack, and ended up with this:

I'm still proud of that square.

Over a few weeks, I explored a few more ideas and learned a lot. It's kinda cool to think back to this, as I'm doing quite a lot of WebGL as part of my job these days.

Simple Things

I gave a one-off talk at a hack day.

It was about applying bounds to your creative output (much like Cardboctober & WebGL Hacks).

I described the properties that I'd like from a "Thing":

Easily described & understood
A self-contained object
Open to be reused

Then I talked about cassette tapes a lot because I think they're a really great example of a unit of creativity.

Making a lot of things

This sounds like a similar talk, but it was entirely different.

I'd followed "what is a unit of creativity" through to JavaScript, and I wrote an experimental code editor to play with some of the ideas of constraint and exploration.

Although I'm really proud of the ideas and effort that I put into this talk, I'm unhappy with how I gave it. I was the closing talk at my favourite conference, and instead of getting confidence from that, I ended up trying to prove that I was worthy of being there by throwing down a bunch of technical and portfolio stuff.

Having said that, I just found a really awesome post about it by Mathieu. Also, Joe & Calum wrote some kind words about the talk.

Microhacks

I ended that talk with the idea of "Microhacks":

Plan the goal, not the approach – as developers, we can spend a lot of time working out a perfect technical solution and not actually doing anything.
Schedule as little time as possible – it's easier to fit in "smashing something out in 20 minutes".
Explore – if it doesn't look like you thought it would; great! You've got a different thing.
Define your outlet – have a goal of where you'll publish it, so you'll know when it's done.
Publish first, tinker second – we often go into a "clean up" phase. For hacks, you can do this later (or not at all).
Do it again - Make a fresh copy, and do something even cooler this time.

2018 ➡️ 2020

2020-05-25T00:00:00Z

I've been up to a load of things over the last couple of years.

A few highlights:

🇳🇿 Cycled around New Zealand with Sarah
🤖 Started working at Oxbotica
🏍 Bought a motorbike

For a more visual guide – I drew some emoji over the top of a calendar (see twitter for more details).

It might not be the most efficient diary of events, though it was pretty fun to put together 💕.

Blog Post

2020-05-25T00:00:00Z

My friend Oliver logged this issue over on GitHub.

Thanks dude.

I hope this blog post, and the words within it, helps resolve the issue for you?

GIF

2016-03-03T00:00:00Z

Animated gifs are brilliant…

…But they’re also kind of rubbish. We can’t pause a gif, skip to a particular point, or grab individual frames from the image element.

One way around these problems is to load the file content with JavaScript and decode it manually.

There are a bunch of different libraries that can decode gif (check out omggif, gify & gif-stream). I’m a fan of the decoder in jsgif; it’s awkward to use, pretty inefficient, a bit dated, and feels nice.

parse(file, {
  hdr: (data) => {
    // handle header data
  }
})

You provide an object with a set of callbacks for the different parts of the gif. Callbacks are fired as soon as that part of the gif is parsed and we can choose what we do with the data.

For example, we can listen out for the hdr (header) block, and pull out the global colour table for the gif.

Global colour table for example.gif

With the actual image data; each frame of the animated gif starts with a Graphics Control Extension (GCE) block which contains information about the section being drawn, followed by the actual (LZW encoded) pixel data.

I've skimmed over a lot of really interesting stuff here - for a really in-depth look at gif encoding, check out this blog post.

Once we've got those decoded pixel values back, we can map them through our colour tables to get the RGB values for each pixel, and now we've got something that can be drawn to a canvas.

Exported frame data

Each frame has an overwrite rule provided in its GCE block, which defines how the frame data is drawn to the existing graphic - either being appended to, or completely replacing it. This, combined with the transparency capabilities of the gif allow frames to update only the changed pixels of the image.

By implementing overwrite rules, frame delays, and probably some other stuff, it's possible to play/pause/scrub through a gif file (see libgif-js).

Though, back to the frame data - we don't have to settle on canvas .drawImage. If we load our data into a WebGL context, we can stack up the frames to get an overall feel for the structure of the gif:

Other gifs

A spinning shape - full re-draw each frame
>256 colours - a palette limitation workaround
Iso cubes by Bees&Bombs - looks awesome
Hexagon Friends also, Bees&Bombs - looks pretty
Trefoil yup, Bees&Bombs - nice angles
Adventure time - a more traditional animation
Kitten - the majority of gifs

Recommendation

Here's what the original gif spec has to say about animation:

Animation : The Graphics Interchange Format is not intended as a platform for animation, even though it can be done in a limited way.

…I kind of feel for the person who wrote this; it must be hard to look around the web and see millions of people ignoring your advice. Though I hope they see the positive side - animated gifs might be a terrible hack; but they're totally brilliant.

ps. check out Bees & Bombs

In with the old

2015-09-09T00:00:00Z

After more than a year of no posts - I'm still here

It’s been a really busy year; a lot of my spare time has been spent hacking on, or preparing for, talks. In the last year I’ve spoken at:

MK Geek Night, FFconf, Breaking Borders, All Your Base, Oxford Geek Night, jQueryUK, MK Geek Night, talk web design, Upfront, Full Stack Fest and Reasons to be creative.

photo: All Your Base, by Garrettc

It’s been pretty exhausting, though I’ve got a lot from it. I’ve been able to explore ideas with combining capabilities of lots of devices, how we can use IoT to inspire the way that we build applications, and recently I’ve been looking into our relationship with the data that we produce.

I've also been helping out with JSOxford quite a lot; it's so awesome to see the community grow. This year we had our second Summer of Hacks, which had FIVE events (totally Ryans' fault).

Next up

I've got a few blog posts ideas logged as github issues; so I'm going to try and start working through those. A bit of a warning; I'm going to focus on getting stuff written than making it super-great quality. If you see typos, mistakes, or bad grammar; that's totally why, ok. (also, it would be ace if you could correct them.)

(most things I'm working on are in github issues: talks, projects and other stuff)

And also

Although I’ve not been posting much online; other people have been putting stuff online with my face on it:

Max built bensmindpalace.co.uk; it’s got my face, some code, and a soundtrack.
Michael made this page, which transitions from lorem-ipsum to lorem-bensum, then into a picture of my face.

Thanks, guys.

Open inbox

2014-07-09T00:00:00Z

I'm pretty rubbish at responding to emails - I've tried inbox zero a few times but never stuck at it. I thought being public about the state of my inbox might encourage me to keep on top of things.

Home

Work

(key: orange - unread, yellow - read)

It's updated every 30 minutes by a short python script that uses the new gmail api to fetch the inbox counts, then post them at a heroku app. The heroku app stores the data in Redis. The server is slightly hacked together, though the source is on github.

I guess you could say that it's a slight invasion of my privacy - though I think the benefits of having it open far outweigh that. It forces to be honest about my inbox, helps other people see if I'm making progress, or spot patterns and help me.

Hyperlinks

If you are doing some more heavy duty time series monitoring, I'd check out influxDB - seems like a really powerful tool.

I couldn't use it for this data, but redis-timeseries is a nice node library for storing event counts in redis (like website hits or other events).

LastFM to CSV

2014-05-29T00:00:00Z

I made lastfm to csv - a page for downloading lastfm listening data as a csv file. API requests are made directly from the browser, avoiding the need for any server-side code.

give it a shot now (my username is benjaminf)

My friend Andy wanted some lastfm listening data for a visualisation project so I wrote him a ruby script that uses the user.getrecenttracks method to get the track history for a user. This worked fine though if someone else wanted to use it - they would need to install ruby, and have a familiarity of the command line.

I figured it might be easier for people to do this in the browser - so I put together lastfm-to-csv, which lets you enter a username and download a csv file containing all the tracks for a user.

Using the browser

Generating a csv of all the tracks requires many requests to the api, these need to be processed and combined together. I decided to do this all from the browser, by using xhr to make requests to the api and storing the processed csv data in a javascript Blob.

A few libraries that I found useful:

reqwest - for making xhr calls, nice and lightweight.
async - for organising the calls to the api and processing the results.
Filesaver.js - lets you download the resulting Blob objects as a file.

A few things I noticed by doing it this way:

people have access to their data instantly - no waiting around for a file to be generated on the server before downloading it.
no servers to keep running - all code runs in the browser
harder to avoid api limiting - it someone gathers data in several windows, they could get rate limited.
potential access issues. Last fm gives you access to a nice cors endpoint, though that's not always the case with other apis.

Andy blogged about his awesome last fm visualisations, you should check them out - turns out that I'm quite obsessive.

Unknown Pleasures

2013-10-21T00:00:00Z

The artwork for Joy Division's Unknown Pleasures album is based on a graph of radio waves from the first identified pulsar - this is a graph of radio waves from the same pulsar, but recorded in 2012.

Origins of the Joy Division artwork

Stephen Morris (the Joy Division drummer) found this image in the Cambridge Encyclopedia of Astronomy. It's a graph showing the regularity of the pulses of CP 1919 (PSR B1919+21) - the first radio pulsar discovered (1967).

Peter Saville took this image and inverted it for the for the cover of the Unknown Pleasures, there's more information in the video below.

This blog post gives some information about when the graph appeared.

Data

Disclaimer: I know almost nothing about pulsars

The data for the graph above comes from Pulsar Group CSIRO Astronomy and Space Science.

Recording of the first-discovered pulsar CP1919 (PSR B1919+21) made at the Parkes radio telescope in April 2012. The observing frequency was 732 MHz and bandwidth 64 MHz. The audio is the detected and dedispersed signal modulating white noise.

Credit: R. N. Manchester, G. Hobbs and J. Khoo, CSIRO Astronomy and Space Science

You can listen to it here - you're listening to a star which is pretty mental.

I took that wav file and had a look at the data with sox (I eventually pulled out numbers with canvas_waveform).

There was a lot of blank space in the data. I'm not sure if this is because of the way that I extracted it, or the way that it had been pre-processed, or maybe that equiptment has changed over the last 40 years.

When I managed to plot it, it's no-where as pretty as the original one. I've just been talking to @olorton about why this might be the case - we think it might be because I'm plotting amplitude rather than frequency, he's going to have a hack tonight to see if he can get something better. If you have any ideas, please do give me a shout.

Visualisation

Looking at the original plot, I thought that the peaks obscuring the preceeding ones above meant a loss of information. Though when I plotted it with a transparency it became a lot harder to read - you don't know whether lines were going up or down and it looks a bit messy too. Though I could see that some lines were totally flat, which is cool and I don't know why.

Another thing that I realised when I came to plot the graph was that there is a period of silence between each of the peaks (the pulsar has a period of 1.33730s and a 0.04s pulse width) - I've added a toggle so that you can see pluses along with the silence.

Both of these points are good cases for the original visualisation which is to show how periodic the pulsar is (I think).

Also, this shows 80 cycles of the pulsar - so it covers about 1m47s (I was slightly sad that none of the songs on the album are this length).

Other things

If you liked listening to the stars, check out this list of unexplained sounds on Wikipedia, they are mostly things in the sea and pretty awesome. I like how the 52-hertz whale is described as being "just higher than the lowest note on a tuba".

Context require

2013-10-01T00:00:00Z

This is how I organise JavaScript assets on this site.

view code on github

Motivation

When I started this blog, I knew that I wanted to include specific and varied scripts on each post.

I didn't want to serve all my js files together in one blob with each page load:

The built js file would get bigger with each post I add
I wanted the flexibility to use any new library that I came across (didn't want to think "I've already used X, so I'll just use that")

Approach

I use require.js to modularise my code. If I want to make part of the document fancy - then I define it in a file called fancy.js like this:

define(['jquery'], function($){
  return function(element){
    $(element).doFancyStuff()
  }
})

… the module defines a function that can be applied with a particular dom node.

Then, in the markup - I specify which module I want to apply to a particular piece of markup:

<div class="cr" data-cr="fancy">
	<h1>RAINBOWS</h1>
	<p>UNICORNS</p>
</div>

I then stitch this together with another require.js module which looks through the page, loads any modules and applies them appropriately. It looks something like this (I've used jQuery here for succinctness):

$('.cr').each(function(){
  var self = this, requirement = $(self).data('cr');

  require(requirement, function(module){
    module(self);
  })
})

Profit

Now I can load only the necessary scripts to display a page; which vary across pages on my blog:

lllocal - only loads jQuery and a plugin to thumb between images
tweet-globe - will load in a datafile and a vector manipulation library
wtcss - won't load any extra libraries

Dogfooding

I've enjoyed using this approach - it's made it really easy to add new posts. I've felt like I've been writing code rather than tweaking and maintaining it.

It also feels like a good distinction of concerns - by starting with the html/dom I've focussed on what I'm trying to enhance with javascript.

Only loading scripts for on-screen elements

This approach kind of sucked for my homepage though - I've got all my posts in full, so every single script would be loaded.

So I rewrote my script to defer the loading of a module until the related element is on-screen. It looks something like this (again, jQuery here for brevity):

// using jquery.inview
$('.cr-defer').one('inview', function(){
  var self = this, requirement = $(self).data('cr');

  require(requirement, function(module){
    module(self);
  })
})

I've written a way to display the modules as they are loaded, which you can turn on with the button below (if the module loaded okay!)

This should reload the page with a panel to the left which will display:

cr - the script with loads in the modules for the page
cr-debug - the module that displays the panel on the left
ko - knockout, which is used to update the panel

as you scroll down the page, you should see more modules loading in as you go past the posts.

Limitations / solutions

I can use this approach because I've got independent bits of content. Creating larger scale interconnected sites requires a lot more thought and planning. Addy Osami did a great talk on building large scale JS applications at last years jQueryUK. He also has an online book - Learning JavaScript Design Patterns which is worth a read.

The other limitation of this approach is performance. Require.js has a great build tool which lets you compile your components into a single file - though this would defeat the purpose of what I was trying to do in the first place.

The issue isn't with the size of download, but the waterfall effect that happens when each dependency is loaded (as a module must be loaded before the dependencies can be found). This (and a solution to this) is described brilliantly in a presentation at last years JSConfEU - A novel, efficient approach to JavaScript loading

Also, if you're interested in this kind of stuff - have a read of Alex Sexton's blog post about deploying javascript applications.

I have a feeling that this is the kind of problem that people have dealt with or have had ideas about before. I'd really love to hear what you think - ping me on twitter or comment on hacker news.

The other side of responsive

2013-09-20T00:00:00Z

Yesterday I gave a talk "The other side of responsive" which was about how responsive web development gives us a great platform for creating interfaces that combine multiple devices. This post explains some of the tech/approaches that I used for it.

I'm writing this in a car, with the limit of a half charged laptop, so apologies for any mistakes or over-wordiness. Also - for context - my mother (who is driving) is playing Mozart Clarinet Concerto in A really loud, which is awesome.

The setup

My laptop has a node.js server which does two things:

Serve the static content of the presentation (written with reveal.js)
Host a binary.js server which publishes anything sent to it to any other connected browsers

My phone has a 3g connection, and acts as a hotspot for my laptop (I would have used the wifi, but it was a bit shakey on my phone - this worked a lot better)

There's a page hosted at benjaminbenben.com/archive/party which has the markup for each of the phone slides, and some JavaScript to link it up to my talk. I took some effort to make this as performant as possible (from accessing the web server with your phone, the "hello" is able to display within the first network roundtrip!); so I was glad to hear Drew talk about web performance, it's such an important aspect of working with the web.

I used PubNub to communicate with the devices in the room. I had two channels, one to give the status of the slides and another for devices to publish information about themselves and forward touch events when we "went collaborative". The publish / subscribe style worked brilliantly for this - all devices would publish and the slide deck would be the only subscriber, and the other way round for the slide states. PubNub has a few features which were really useful for this:

multiplexing - this meant that your device only needed one connection for both of the channels.
windowing - this option let my slides recieve messages in 500ms batches, which fixes the number of requests that my laptop would make, regardless of how many people connected.
backfill - if you were to refresh a device, all the 'hot' messages would be sent down, so the browser would be able to replay them all and catch up with all the other devices; this also allows people to join in half way through.

The talk

My first slide was the short url for the benjaminbenben.com/party with a counter below it. When someone loads the page, there is a script that:

Generates and locks down a uuid for the user, so that reloading the page won't create more devices
Uses modernizr to find the capabilities of the device
Subscribes to the slide deck messages
Publishes a 'hello' message

The hello message looks something like this:

message = {
  uuid: 'some-long-random-id',
  type: 'hello',
  features: 'appcache webgl webrtc ...',
  pixels: 1234567,
  innerC: 'red', // random colours
  outerC: 'blue' // for the circles
}

The counter on the slide deck increments when it gets one of these messages, the features and colours are stored - so from this point I know that I can display the capabilities chart (which is nice).

I then continue to the title slide and wave my hands about a bit. I've got the slides open on my phone too, when I go to the next slide I use binary.js to broadcast a message to all other browsers, which proceed to that slide.

The next slide is a file input field, which looks like this:

<input type="file" id="photo" accept="image/*" capture="camera"/>

The capture attribute means that it fires up the camera on my phone rather than asking where I want to get my file from.

When I take and accept the picture of the geek night, it is streamed with binary.js to all other connected slide decks - it's based on this binary.js example.

Once the image is in the slide deck, this sequence of things happens:

It's displayed on the presentation
It starts uploading to s3
The s3 url is published to all devices
It's base64 encoded and sent to twitter (using codebird, which gives you a proxy to the twitter api for client side apps)
The twitter embed html requested
The twitter embed html is appended to the presentation is published to all devices
The twitter widget script is added to render the tweet (this also happens on the devices)

So, at this point - the the last slide is rendered (on the devices as well). Also, the devices are displaying the picture on screen (I forgot to say that).

The next slide is the interactive slide of circles representing each device, this is an svg generated with d3. There is a basic animation loop which applies and dampens the speed of a circle and repels it from any nearby circles, this was running for the last couple of slides - so they've kind of organised themselves into a nice pattern.

D3 is fantastic, the enter/exit/transition approach is so intuitive for dynamic data - if someone joined at this point, a circle would pop onto the page and everything would just carry on.

By this point a coinciding circle was displayed on each device, when anyone pressed their circle, the x/y coordinates were published with PubNub. When received by the presentation, the x/y speed on the underlying data is incremented accordingly - d3 does the rest.

Moving to the table of capabilities is just stopping the animation loop, then transitioning the elements to new positions and appending new elements for each capability. I <3 D3.

@benjaminbenben blowing minds with real time analysis of the audience's devices @mkgeeknight #mkgn pic.twitter.com/e8bohM3iaL
— Al Power (@alpower) September 19, 2013

I then used 3 of Brad Frosts slides from his blog post "this is the web". Each slide sends a message out to all the devices which keep in sync. The middle slide (This is the web) just displayed the word "web" on the devices rather than the image, to show that your device is part of that.

The last slide was a quote by Igor Stravinsky about the freedom of constraints. The slide only showed part of the quote (highlighted below) - the full quote was displayed on each of the devices.

My freedom thus consists in my moving about within the narrow frame that I have assigned to myself for each one of my undertakings. I shall go even further: my freedom will be so much the greater and more meaningful the more narrowly I limit my field of action and the more I surround myself with obstacles. Whatever diminishes constraint diminishes strength. The more constraints one imposes, the more one frees oneself of the claims that shackle the spirit.

I like that quote.

Graphing links

2013-08-08T00:00:00Z

This is an example of displaying content pulled from a PhantomJS webservice

I used this example when I talked about "serving websites to websites with PhantomJS" at this months Oxford Geek Nights

Link Areas

With phantomjs you are able to access more than just the HTML/DOM of a page - how the page is eventually rendered in a browser. In this example - we can pull out all the links of a page and find out what area (in pixels) they consume.

Using this script we can get a map of the links to the element areas, which looks something like this:

{
  "http://benjaminbenben.com/":6534,
  "http://bit.ly/Pesy75":10640,
  "http://lanyrd.com/cqfdw":18012,
  "http://mynameismartin.com/":8646,
  "http://oxford.geeknights.net/":84000,
  "http://oxford.geeknights.net/ogn29":52052,
  "http://oxford.geeknights.net/ogn30":52052,
  "http://oxford.geeknights.net/ogn31":52052,
  "http://oxford.geeknights.net/volunteer.html":29520,
  "http://torchbox.com/":4760,
  "http://twitter.com/oxfordgeeks":20112,
  "http://www.marianamota.com/":33128,
  "http://www.torchbox.com/":3360,
  "https://github.com/LuRsT":8580
}

Graphing the data

We can now pull in this data with AJAX and render it on the page using d3 (the force directed graph layout)

initially populated with this blog post, double click to check one of the urls.

examples

PhantomJS WebServer

2013-07-28T00:00:00Z

The PhantomJS WebServer module lets you create self contained web applications that are easy to deploy to heroku using the PhantomJS build pack.

I'll be talking about this at Oxford geek nights on the 7th of August - come along if you're in the area.

(tl;dr - deployed example here & more involved app here)

Let's start with a base PhantomJS script - this loads the Oxfordshire lanyrd page and outputs the names of any upcoming events:

var page = new WebPage();

page.open("http://lanyrd.com/places/oxfordshire/", function(){
  var events = page.evaluate(function(){
    return $('.vevent .summary').map(function(e){
      return '* ' + this.innerText
    }).toArray().join('\n');
  });

  console.log('Upcoming Events in Oxfordshire:');
  console.log(events);

  phantom.exit();
});

This script can be run with phantomjs example.js and it will print the names of all upcoming events in the terminal - something like this:

Upcoming Events in Oxfordshire:
* Oxford Geek Night 32
* WitneyMeets
* XML Summer School 2013
* Sterling Geo Intergraph ERDAS UK UGM
* All Your Base Conference 2013
* jQuery UK 2014
* World Humanist Congress 2014

…super cool. Have a look at the quick start guide on the PhantomJS wiki to find out how this works and what other things are possible.

Using the webserver module

To expose this script with the webserver module, you have to add a few things:

// import the webserver module, and create a server
var server = require('webserver').create();

// start a server on port 8080 and register a request listener
server.listen(8080, function(request, response) {

  var page = new WebPage();

  page.open("http://lanyrd.com/places/oxfordshire/", function(){
    var events = page.evaluate(function(){
      return $('.vevent .summary').map(function(e){
        return '* ' + this.innerText
      }).toArray().join('\n');
    });

    // Rather than console logging, write the data back as a
    // response to the user
    //
    // console.log('Upcoming Events in Oxfordshire:');
    // console.log(events);

    response.statusCode = 200;
    response.write('Upcoming Events in Oxfordshire:\n');
    response.write(events);
    response.close();

    // We want to keep phantom open for more requests, so we
    // don't exit the process. Instead we close the page to
    // free the associated memory heap
    //
    // phantom.exit();

    page.close();

  });
});

This can be run in the same way as the previous script - phantomjs example.js - then when you visit localhost:8080, you should see the list of events in your browser.

With phantomjs, you're not limited to sending plain text back to the client - you can render images of the webpage and send that back (either by reading the file back with the File System Module, or using base 64 to send back an embeddable data-uri).

Deploying

There is a PhantomJS Buildpack for heroku which makes deploying lovely.

To get your app ready for deployment you have to do a few things:

Set the port based on environment variable `PORT`

var port = require('system').env.PORT || 8080; // default back to 8080
server.listen(port, function(request, response) {

Add a file named `Procfile` containing the command to spin it up:

web: phantomjs example.js

Commit your files to git then create a heroku app with the build pack

heroku create --stack cedar --buildpack http://github.com/stomita/heroku-buildpack-phantomjs.git
Creating quiet-lowlands-5118... done, stack is cedar
BUILDPACK_URL=http://github.com/stomita/heroku-buildpack-phantomjs.git
http://quiet-lowlands-5118.herokuapp.com/ | git@heroku.com:quiet-lowlands-5118.git
Git remote heroku added

Push your code up to heroku

git push heroku master
Counting objects: 10, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (7/7), done.
Writing objects: 100% (10/10), 1.34 KiB, done.
Total 10 (delta 2), reused 0 (delta 0)

-----> Fetching custom git buildpack... done
-----> PhantomJS app detected
-----> Fetching PhantomJS 1.9.0 binaries at http://stomita-buildpack-phantomjs.s3.amazonaws.com/buildpack-phantomjs-1.9.0.tar.gz
-----> Extracting PhantomJS 1.9.0 binaries to /tmp/build_2idj4c8tadrpx/vendor/phantomjs
-----> Discovering process types
       Procfile declares types     -> web
       Default types for PhantomJS -> console

-----> Compiled slug size: 15.5MB
-----> Launching... done, v5
       http://quiet-lowlands-5118.herokuapp.com deployed to Heroku

To git@heroku.com:quiet-lowlands-5118.git
 * [new branch]      master -> master

The example app should now be available on the reported url (in this case: http://quiet-lowlands-5118.herokuapp.com).

View Deployed Example Site GitHub Source

A more involved example

I've put together a more complex version of this style of app - it allows you to specify any webpage, renders a screenshot and returns some information about the page (a list of links).

It also serves a static page with a form to submit the requests to the app. It's deployed to phantomjs-webserver-example.herokuapp.com and the source code is on github.

I've tried to make it an easy project to modify for your own use - so fork away and have a hack!

View Demo GitHub Source

A few issues / gotchas

Mongoose (the embedded server) doesn't parse the POST parameters with the default jQuery contentType header 'application/x-www-form-urlencoded; charset=UTF-8'; I had to drop the charset and it seemed to work okay.
Firefox has trouble parsing large data-uri strings in json objects, so I've split the image and json on separate lines and decode them when the request comes back (unfortunately Firefox fails to add the xhr header that fixes the mongoose error)
GET parameters aren't parsed. I'd much sooner use a GET request for this app, as there's not any state change and it would allow the responses to be cached. In wtcss I fudged this parsing.
Sometimes the page render returns a blank image, especially when on heroku and under stress. This is a known issue - a work around is to wrap the .render in a setTimeout.

Shaker.io

2013-07-01T00:00:00Z

For a couple of releases now - it's been possible to build a customised version of jQuery. I felt that one of the barriers for using your own version was finding which modules your site actually uses - so I started working on shaker.io

Shaker.io is a tool for finding which modules of jQuery you're using - it does this by providing an instrumented version of the library which tracks which functions you use.

Transport

Once the script has tracked which modules have been used, the data is shared with the main page where the list of your dependencies can be updated. Eventually we'll add a means of sending events between devices (so you can test on a phone, and see the results on a development machine).

your custom domain is able to share data with shaker.io by embedding a hidden iframe which can share messages with the main window using storage events. See my blog post for more on those.

Early days

There's a lot I want to do with this tool. I started with the goal of reducing the number of jQuery Mobile plugins I was using on a page (jQuery Mobile has a great download builder), though as I've been hacking about with the idea, it feels like it could do a lot more.

Openness

I've recently been working on a project with the Open Data Institute. One of the (many) things I've enjoyed is the open development; working in the public does seem awkward/hard at first, though I found myself writing better code, getting useful feedback, and being able to collaborate much more easily.

With shaker.io - I'm going to make an effort to make development as transparent and open as possible. My hope is that people will be able to get involved with discussions on issues & pull requests. My dream is that people can get involved with improving the tool and adding functionality that I didn't think of.

So far I've been trying to make the code as friendly and accessible as possible (no obscure templating languages, no fancy asset processing). Nodejitsu have provided the hosting under their "free for open source".

White October

One of the things I'm most excited about is that White October (my employer) will be putting some time toward this project as part of an ongoing crusade of being totally awesome.

This week some of us will be looking at the project from a development, ux and design point of view. We'll be throwing around a lot of ideas and feeding them into the site, so do keep an eye on the project and feel free to jump in on discussions or pull requests.

You can find the project at http://shaker.io and on github under benfoxall/shaker.

CSS matches

2013-05-09T00:00:00Z

I was working with a large css codebase and wanted to see if our rules were becoming more specific as the css source grew, so I built css.benjaminbenben.com to look at how css rules are applied to a page.

Active rules

This shows how many of the selectors are being used on a page, you can toggle to show only the active ones.

Overview

The '-' link on the bottom right scales the rules so that they fit the height of the window. This is to show the how the impact changes as rules are added to the css.

An example (with notes) of the jsOxford site is below:

### How it works

The main part of this is a PhantomJS script which

loads the page
extracts all stylesheet rules
finds matching elements for each rule and gets the positions of them
takes a screenshot

All this is sent back to the client in a json object (including the image as a data-uri).

The source code is now online at github.com/benfoxall/wtcss

Example pages

google.com - some styled elements are offscreen
hacker news - only 31 css selectors!
facebook - only 5% of rules match on landing page
css.benjaminbenben.com - yup, you can do that
white october - we used a custom bootstrap build, though you can see the gaps in the scaffolding sizes we didn't use
jsoxford - you can see the rules at the bottom that we added to target specific elements

Cross window communication

2013-04-24T00:00:00Z

I was part of the "Rising Stars" track at the jQuery UK conference this year where I talked about sending messages between browser windows. This post covers the first half of my talk - sending events between local windows.

My slides are now online, though they are more prompts for me to talk, rather than being full of information. The demos wouldn't really work with it being publicly accessible, so I'm going to cover each of the techniques I mentioned on this blog.

The websockets/binaryJS/webRTC things are on the way - just working on getting the server side part hosted nicely.

postMessage

When you have a window that you can reference from with js - either by getting an iframe from the DOM, or as returned by window.open() - you can use postMessage to communicate with that window (crucially, even if that window has a different origin).

// Send messages from parent window
var win = window.open('http://benjaminbenben.com/pink.html','','width=200');
document.onselectionchange = function(e){
	win.postMessage(document.getSelection().toString(), '*' );
}

// (on the target window) listen for messages
window.addEventListener('message', function(e){
	echo.textContent = e.data;
});

>demo opens a window and sends it the text selection from this page

For more information about postMessage - check out the entry on MDN and on John Resigs blog post about it.

Storage Events

When you aren't able to access a window directly, but it shares the same origin - you can use storage events to synchronise data between windows.

A storage event is fired when another window changes the localStorage for that page. By listening to these events - you can keep objects in sync across windows.

// listen for changes from other windows
window.addEventListener("storage", function(e){
	if(e.key == 'example') $('#el').css(JSON.parse(e.newValue));
}, false);

// update a local element and notify other windows of the change
$('#el').css({color:"red"});
localStorage.setItem('example','{color:"red"}');

A nice side effect of this is that you have the state of an element persisted in localStorage, so you could render that on page load. See this gist for a general way of doing this.

This approach can become particularly interesting when the data being synced is displayed in different ways in different windows - in my talk I showed how the reveal.js slide deck could be viewed in both overview and normal views at the same time (see this gist to see how that can be implemented).

demo move your mouse over the area below, any other windows open on this page will update

Image processing + web workers

2013-04-14T00:00:00Z

tl;dr - examples:

>with web worker (should be smoother)

>without web worker

Web workers let you to take JavaScript execution off the main UI thread - which can be really useful if you are doing complex things with video

I came across a javascript qr-code reader a few days ago. When I started using it to scan from a getUserMedia stream - it worked fine, but the extra processing was blocking the ui, which was particularly noticable when you're displaying the video.

I thought it was a pretty good candidate for taking the processing off to a web worker; which turned out pretty well.

Once you've got the imageData from your canvas, you can run it through jsqrcode by setting attributes of the qrcode object, then call .render():

qrcode.imagedata = imagedata;
qrcode.width = imagedata.width;
qrcode.height = imagedata.height;

var content = qrcode.process();

It was pretty straightforward to pull the code into a web worker, I spent a bit of time before I realised that console.logs were making it fall over. Here's the interface for responding to messages with the worker:

self.onmessage = function(event) {
    var imagedata = event.data;
    qrcode.imagedata = imagedata;
    qrcode.width = imagedata.width;
    qrcode.height = imagedata.height;

    var resp;
    try{
        resp = qrcode.process();
    } catch(e){
        resp = ''; // *mostly* "no code found"
    }
    postMessage(resp);
};

Back in the original page, you can creater the worker and deliver messages to it using the .postMessage function. You can optionally list Transferable objects to efficiently move them to the web worker.

var worker = new Worker("jsqrcode/worker.js"),
worker.onmessage = function(event) {
    console.log("qr code is:" + event.data);
}

// imagedata = ctx.getImageData(…)
worker.postMessage(imagedata, [imagedata.data.buffer]);

Jsqrcode is on github, as is my fork with the starts of the worker interface. You can either view source on the examples above, or view them on github.

Tweet Globe

2013-04-07T00:00:00Z

Plotting geocoded tweets on a globe with canvas

I gathered a few hours of geocoded tweets from the twitter streaming api (using the maptime code as a base). This was to explore some ideas that we'd been talking about at White October.

Drawing the globe is relatively straightforward. The lat/long pairs are converted into position vectors, which are then transformed based on the mouse position. The Sylvester library was pretty handy for transforming the points (Pete talked about Sylvester at jsoxford recently).

The original plan was to animate this over a period of time, though it looked quite random/noisy so I went for this static view instead.

Gareth pointed me to a post about the effects in tron legacy which makes me want to make this a lot more awesome!

lllocal

2013-04-05T00:00:00Z

I built lllocal - which lets you find and listen to bands that will be playing in your area soon.

I've been thinking of this idea for a long time and I put up a public version a couple of months ago. It's brilliant to get feedback from people and I've also got tickets to two events while testing it out for myself (Keaton Henson was great and we're off to see Daughter in a couple of weeks).

My motivation for lllocal came from:

my personalised last.fm 'events near Oxford' are almost all in London. Granted, these bands are ones who I'd really like to see and Oxford is pretty close to London - though I wanted to see more live music without spending evenings on a bus.
we got tickets send out spotify playlists to their mailing lists. I love this style of suggesting people to go and listen to.

The gig listings come from the last.fm api and the Spotify web apis are used to find and play the bands on spotify.

There is still a huge amount to do - though I'm really happy to have something online. If you have any feedback or suggestions I'd love if you got in touch or left a message on the feedback page.

Lllocal is online at lllocal.com.

Maptime

2012-12-05T00:00:00Z

In time for our first JS Oxford meet - I put together a small node app which reads geocoded tweets from the twitter streaming api and pushes them to the browser to display on a map.

This is a stripped down version of a project that I worked on at White October this summer. This version is not at all for production use (your browser will grind to a halt if you leave it running for a while!), though I hope it's a good/interesting example of linking up server and client js.

The code is all up at the jsoxford github account, I'll go over a few bits of it:

app.js

This is the main node.js file it brings in some external packages:

ntwitter - for accessing the twitter streaming api
express - to serve some static files
faye - for sending messages between the server and the browser

Once these have been brought in, you can connect to the streaming api using ntwitter. This gives you access to a stream object, which you can add listeners for new tweets using the stream.on() function (see eventEmitter docs for more details).

twit.stream('statuses/filter', filterParams, function(stream) {
  // stream.on('data', yayFn)
});

We then want to serve some static files for our client side pages/scripts, you can use express to do this (express can do a whole lot more - if you want to have a look, I'd recommend using the executable to generate a basic app).

We also want to send data to the browser using faye, this has a really nice pubsub api based on the bayeux protocol. Attaching this to the http server will listen for websocket/ajax long-polling requests and serve a client js wrapper at /faye.js.

var app = express();
app.use(express.static(__dirname + '/public'));

var bayeux = new faye.NodeAdapter({mount: '/faye'});

var server = http.createServer(app);
bayeux.attach(server);
server.listen(3000);

Now, to link it all together - you can listen for events on the twitter stream, then publish them to a faye channel with the following code.

stream.on('data', function(data){
  if(data.geo)
    bayeux.getClient()
      .publish('/tweet', {
        geo: data.geo,
        text: data.text
      });
});

markers.html

Moving clientside (this code is in ./public and will be served to the browser), we first want to connect to the faye pubsub. To do this, we include the faye client library and connect to the endpoint that we mounted faye at on the server using Faye.Client.

<script type="text/javascript" src="/faye.js"></script>
<script type="text/javascript">
var client = new Faye.Client('/faye');
</script>

We're using the google maps api to display the map and place the markers. The majority of the code for this is straight from the simple-markers example. (To get more of an introduction - have a look at the tutorial).

To get the tweet data from Faye, you use the client.subscribe function to listen to a channel - in this case we broadcast them over '/tweet' from node.

var mapOptions = {
	// ...
};
var map = new google.maps.Map(document.getElementById("map"),mapOptions);

client.subscribe('/tweet', function(message) {
  if(message.geo && message.geo.coordinates){
    placeMarker(message.geo.coordinates);
  }
});

function placeMarker(coords){
  var latlng = new google.maps.LatLng(coords[0],coords[1]);
  new google.maps.Marker({
    position: latlng,
    map: map,
    title:""
  });
}

And that's it! Have a look at the code on github (I've missed out a little bit of the surrounding bumf above) and have a play with it.

Also, if you are based around Oxford - come along to our next JSOxford meet on the 17th of January.

Lastfm Canvas Streamgraph

2012-11-04T00:00:00Z

A browser based last.fm streamgraph using canvas.

This is based on Lee Byrons listening histories project. I love this project - it's a really interesting and engaging visualisation, and the last.fm data makes it really personal (I can't think of any other services that give as much personalised data as last.fm).

There are services that let you download a pdf streamgraph: lastgraph.aeracode.org & last.fm playground (if you're a subscriber).

My version is different as all the api requesting and graph drawing are done in the browser - this lets you see your graph as soon as any data is ready.

I originally started creating a large svg for the whole chart, though this became quite slow, so I used separate canvas elements for each week of data and . This is slightly limiting - I couldn't sort or colour the artists based on when they appear in your history (as the original does).

Out With The Old

2012-09-25T00:00:00Z

After more than a year of no posts - I've left my old blog behind.

This new one is built with Jekyll and put it online with GitHub Pages. The source is on github.

It seems that a lot of Jekyll sites start with a post about the interesting way that they have been deployed. So, for the record, I kept it simple and went for the github jekyll generator.