IoT Light Sensor: There is Data!

One of my goals for the IoT light sensor was to explore graphing live data. Phant, the Sparkfun service I’m using to post my data online, has an example of graphing live data with Google Charts. Since I know nothing about coding for the web save some outdated html and css knowledge, I used their example code as a template for what I wanted to do.

Before we continue: I don’t fully understand it. I’m starting to get it but javascript is weird and so are asynchronous calls.

The key seems to be this ajax call.

// JSONP request
var jsonData = $.ajax({
    url: '' + public_key + '.json',
    data: {page: 1},
    dataType: 'jsonp',
}).done(function (results) {
    // do something here with the data returned

So a couple things. First of all, and this was not obvious to me at first, this asks for data from my Sparkfun Phant stream and then keeps going doing other things, like any code that comes afterwards, while it waits. When the data gets in, it executes what’s in the done function. So if I make a bunch of JSONP requests one after another, what I want done with the data returned won’t necessarily happen in the order I make the requests.

The next thing is that my data stream is really big. I post seven variables every 30 seconds and my sensor has been doing this for months. I have about 15MB of data hanging out. As far as I can tell there is only one way to limit the amount of data that gets returned:

data: {page: 1},

Sparkfun talks about this in their fairly bare-bones documentation. The relevant part is below:

By default, returns all of your logged data in the format you requested. If there is a need, you can also request output in 250 kilobyte chunks by setting the page parameter in the query string portion of the URL. This can be very helpful if you just want to check out the latest logged data without requesting the entire data set.

So if I don’t include a page, I’m calling all of my data which takes forever. Plus then I end up graphing way more than I’d like. But if I do include a page, I’m only getting a couple hours of data. Ideally I’d be graphing a couple days worth of data. As far as I can tell, there isn’t a way to ask for a range of pages at once. I tried a couple different ways of trying to do this, {page: 1-5}, {page: 1:5}, {page: [1,2,3,4,5]} but none of them worked. {page: [1,2,3,4,5]} appears to return the most recent two pages. I couldn’t figure out what I needed to return an arbitrary range nor even where to find this information.

Screen Shot 2016-01-16 at 11.02.54 AM
Takes way too long to load. That said, this graph is really interesting. It indicates that my office gets very little natural light. The smaller bumps are days when no one came into the office. The steps up and down from the high sections mark when the lights were turned on and off. It also shows that we have much less blue in our lightbulbs than daylight does.
Screen Shot 2016-01-16 at 11.04.06 AM
Loads quickly, but not enough data to be interesting.

That’s when I tried making multiple calls and realized that what I did with the result wouldn’t be done in order. This is not necessarily a problem, assuming I could do something once they were all done, i.e. sort and correctly order the data, but I didn’t know how to do that. The only thing I could think of was making additional calls within the done function, but that seems like the wrong way to do it.

Regardless I feel I got pretty far and learned a lot. If anyone knows the correct way to return an arbitrary amount of data from my data stream, let me know! Until them I’m psyched to have gotten a sensor-to-internet-graph system up and running with relatively little overhead.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s