WORDS. On the internet. Oh my.


On April 8th I stopped redirecting to Bike Calgary[1] to start showing off the aggregated data that I was pulling together from the 3 Eco-Counter installations. With the source on GitHub, I thought it'd be worth explaining a little of the why and how.

At the start of January, the City of Calgary made public the web page for bike counter on the Peace Bridge with promises of making more available including at least 10 more during the upcoming cycle track pilot. The Peace Bridge counter had data stretching back to April 24th, 2014 and by default was always showing the entire daily data set.

My first curiousity was whether I can could have a bookmark to just show the last week or so worth of numbers which led me to figuring out how the webapp worked. (Good ol' WebKit developer tools)

After that in tandem with some projects I was looking into for work I decided to start seeing about scrapping the data and storing it somewhere to compare numbers (different installations, averages, weather) more easily. So a big thank you for the people at the City and Eco-Counter for not telling me to "get lost and don't use things inappropriately".

As for how - the Python scripts just ask Environment Canada and the counters once a day for their last day's worth of new data (if possible) and store it in Graphite. Interacting with the data is Grafana 2 behind nginx. All hosted on a tiny instance on some publicly available free compute resources that I just happen to also manage as part of my day job. Funnily, most of the script writing was done during an all nighter at a Denny's in Kamloops waiting for 4 AM to roll around so I could swap some power cables in a maintenance window.

It's nothing fancy but it's fun to see what might come of it when data is made available.

1 - I had registered the domain last year and figured that was a good place to point until I had a better idea of how to use it.

Trying to make sense of when to use Docker vs. LXC

While working on some side projects the past couple weeks I kept confusing myself on how things worked behind the scenes between Linux Containers and Docker. They both leverage the Linux kernel's cgroups to function on Linux (and in Docker's case - similiar technologies in other OSes), but differ completely in terms of how you interact with them.

While Linux Containers can best be thought of a super lightweight VM to run a whole VM, Docker contains a slew of other features that blur the lines between it acting like a super lightweight VM and being a full platform to build off of. Docker plays closer to the idea of a process/group of processes (application) under a chroot versus LXC's idea of a whole OS/machine in a chroot jail.

So it's misleading to think of a Docker container the same way as a LXC container. Same technology behind the scenes but completely different approaches. For Docker it's all in how you set up your container to run - you can have all the other services you normally get in a VM if you so wish.

For example with LXC setting up MySQL would consist of making the container, running the command to install MySQL and setting the service to go. You can then log in or attach and run other commands as well if necessary.

Docker on the other hand involves similar steps with the flexibility of having Docker do the install and run the service when the container starts (defined in the Dockerfile). However if you want to attach to that container and run more commands you have to have set access to do that up ahead of time (eg. supervisord, runit), create a new container with that command, or try and force your way into the container. (you can try lxc-attach but if you want a new TTY and you're attaching to a mysqld instance? Not going to work)

After figuring that out - the use of Puppet in Docker started to make more sense. Have Puppet configure your image and then save/commit that state or kick off the supervisord process to keep the container "alive". Docker lends itself more to recreating/iterating whenever a new update is needed over updating settings.

In summary - LXC container is analagous to a VM, while Docker a very supercharged sandbox for running a process or group of processes. Use LXC when you're wanting a separate "server" without the extra overhead, Docker when you're wanting to run a "service".

I also recommend reading the FAQ - primarily the what Docker "adds to LXC". In the end it's left me more leery of using Docker - it's a bit of a paradigm shift I'm not ready to do just yet.

On one last sidenote, IPv6 support also looks like a lot of pain - but not any worse than LXC.

IPv6 and Systems

Last Monday I was part of a team presenting a workshop at BCNET's 2014 conference about Configuring IPv6 for Networks and Systems. The network walkthrough and slides put together by BCNET are available on their wiki while the Systems portion I worked has the slides and workshop examples on Github.

Thanks to everyone who came out.

Moving to Ghost

Alongside changing hosts (moving from TextDrive to my own VPS), adding IPv6 support for my websites and taking far too long to do it I've also swapped WordPress for Ghost. Slightly involved installation but much nicer. Most importantly no comment spam.

Quick and Dirty: Disabling Firefox's Updates with a CCK

With Firefox's new 6 week major release cycle combined with the fact Firefox won't silently update to major versions like Chrome I got a lot of complaints about Firefox always asking for an update.


Unfortunately Firefox doesn't deal with plist files like most other Mac apps (part of it's cross platform nature) so MCX isn't an option. However you can manage Firefox by either editing the Application file itself (annoying) or by installing a global browser customization extension (easy).

If you want to low down on this and to also stand on the shoulders of giants; read Greg Neagle's posts about customizing Firefox:

The quick and dirty steps to disable updates:

  1. Install the Firefox Client Customization Kit:
  2. Launch the CCK Wizard from the Tools menu
  3. Create a new configuration
  4. Follow the Wizard through to the Customize Preferences section (the 12th or 13th screen). Be sure to remember the Unique ID (first screen) you gave the extension.
  5. Add both the app.update.enabled and app.update.autoUpdateEnabled keys and set them to false. I also lock these preferences because I update the applications using munki
  6. Save your CCK .xpi. The next extra steps involved here are so the user is not prompted to install the .xpi to their local extension folder when they launch Firefox. (I figured this out on my own and then noticed it was in the comments on the More Firefox Customization post and even has it's own post.)
  7. Install the .xpi file it creates on your local profile (should just be able to double click on it or just drag it onto Firefox).
  8. Go to your local Profile's extension folder (~/Library/Application Support/Firefox/Profiles/SOMETHING/extensions)
  9. Find the folder that corresponds to the Unique ID you gave it on the first screen of the CCK. In my case it was an email address.
  10. Copy that folder to /Library/Application Support/Mozilla/{ec8030f7-c20a-464f-9b0e-13a3a9e97384} (where global extensions are stored)
  11. Open up Firefox and test to see if it is installed (check the Extensions loaded or if you used that menu item see if it exists)
  12. Read Greg Neagle's post on More Firefox Customization for more details.

That folder you copied to the Global Extensions Folder can now be distributed to all your computers that need the CCK applied; either as a package (ideal) or using Remote Desktop.

Of note, extra things I like to do are to disable the welcome and upgrade pages, and add an item for our home page to the Help menu (also helps verify the extension is working) but I do allow users to disable the CCK if need be (though I've yet to know one to care enough to do so)

Calgary FCPUG: Outputting to Blu-ray, DVD and the Web

I've now uploaded the slides from the talk I gave to the Calgary FCPUG about outputting to Blu-ray, DVD and the Web. You can grab them from the following link:

I do have an audio recording of the first presentation I've given since high school but have yet had a chance to listen and edit it as necessary.

I hope everyone found it useful.

Using Gmail as your SMTP server When Using your ISP's Email

NOTE: You're going to be using Google's service to send the email but for all intents and purposes it's completely transparent to both you and your recipient. It's also a world lot better than using some random SMTP server (having to find out the local one and always change it) or finding all your email you sent doesn't even arrive in your recipient's inbox because it's been marked as spam because of the server used. I'd recommend looking for an IMAP host instead for the long run.

For brevity I'm leaving out the exact steps to hook this up with your favourite mail client but you can find that out fairly easily as it's only changing the SMTP server (or check my post about setting up Shaw's SMTP service) and change to and using your Google login instead of say Shaw's in the section about changing your SMTP server).

  1. Set up a Google Account. If you have one you're good to go.
  2. Log into Gmail
  3. Go to Settings (link is in the top right)
  4. Go to Accounts and Import
  5. Under "Send mail as:" section click "Send mail from another address"
  6. Enter your email address you want to use (eg. [email protected]) and press Next
  7. Choose to use Gmail's servers, press Next and choose Send Verification
  8. Click on the link in the verification email. This will verify the email address so you can move onto step 9. You may need to check your Junk Mail folder.
  9. Back at the "Send mail as" section (you may need to refresh the browser) click the "make default" link for the email address you set up and be sure that below it "Always reply from default address" is selected.
  10. Now be sure to change your SMTP settings on your computer/mobile device accordingly. This varies from device to device as to the steps but is the most important step. If not set correctly (eg. not turning off other SMTP servers on an iOS device) will make everything we've done for naught.
  11. Send an email to yourself to test and reply to it and make sure it gets to the right address. The only times I've ever seen an error here is if the SMTP wasn't set up correctly, step 9 wasn't followed or the carrier's SMTP server was enabled again (yes it's repeated because it accounts for 99% of errors I've seen).

Not difficult, but something I can grab when writing an email on how to do it. :-)

Hooking Up with Shaw's New "Remote SMTP" Service

Update (February 2012 - Webmail 2.0 is completely up with all it's Exchange goodness - so check out Shaw's new instructions )

Please join me in welcoming Shaw's new feature of actually allowing Shaw email users to send email while travelling without resorting to webmail or trying to find the local ISP's SMTP server address (or seeing that Telus' mobile SMTP server is blacklisted AGAIN marking all your email as spam). This is of course ignoring that I don't recommend anyone actually use their ISP provided email address but instead use something a bit more dedicated like your own domain or an actual email service. It's still better than an AOL address.

Previously I'd been setting clients up to use a Gmail account as their proxy sending address when they have a Shaw or Telus email address - this makes it easier for Shaw clients. It's also a lot simpler than the Gmail approach (which I have yet to post here).

Coles Notes

  1. Turn on Mobile Access using the new Webmail beta:
  2. Change your SMTP settings to point to using port 587, STARTTLS, and use your username and password as the authentication

More elaborate instructions

Setting Up Shaw's End:

  1. Sign into
  2. Click on Preferences (right side of screen - it’s a text link beside Feedback)
  3. Click on the ‘tab’ that says ‘Mobile Access’
  4. Set it to Enabled (click the radio button beside it)
  5. You may need to change your password to meet their new security requirements.
  6. Press Save. It will say “Preferences saved” in a small yellow box at the top of the page if it’s successful.


  1. Go to the Tools menu and choose Account Settings
  2. On the left side on that window click on “Outgoing Server (SMTP)”. You may need to scroll as it’s always the last item.
  3. There should be an item associated with the Shaw account you just turned on. Whichever account you turned on Mobile Access for and click on it and then click the button that says “Edit...”
  4. In the window that appears change the server name to instead of shawmail or that it was set to. The port number should be changed from 25 to 587 and the Connection Security to STARTTLS. Authentication: Normal password and then enter your username
  5. Press OK
  6. If you did not change your password when setting up Shaw then press OK and you’re done. If you did change your password when setting up Mobile Access the next time you check email it will ask you for a new password and you can enter it.

  1. Go to the Preferences and click on the Accounts section
  2. Click on your Shaw account on the left side
  3. Where it says Outgoing Mail Server (SMTP): click on the drop down menu and choose "Edit SMTP Server List".
  4. Find the Shaw SMTP server in that list and click on it.
  5. Change Server Name to
  6. Click on the Advanced tab
  7. Check off Use Secure Sockets Layer
  8. Change the Authentication drop down menu to Password
  9. Enter your username and password and then press OK.
  10. Close the accounts preferences window and say yes to saving it if necessary.

Shaw's already provided instructions on setting up an account anew for iOS, Android and Blackberry devices:

Using AWStats 7.0 with S3 Logs

The necessary code changes from Amazon's Tutorial on using AWStats with S3 Logs take place at line 17,764 with version 7.0 and not 10,657.

The hint? Just do a search for # HTTP request. Keep only GET, POST, HEAD for any version of AWStats.

Of note, as of the published date AWStats 7.0 does not count the files viewed correctly. I'm not sure why.

Using Compressor to Make H.264 MP4s

In April I pushed to GitHub my RewraptoMP4 Script I put together to help assist in creating proper MPEG-4 container files while being able to use the x264 QuickTime component in Compressor. Compressor only allows you to specify the codec being used when exporting to a QuickTime file, however it is possible to use QuickTime Player after the fact to convert a QuickTime Movie to an MPEG-4 without transcoding so long as the codecs are supported in the MPEG-4 container spec.

My primary reason for the extra work is that Google Chrome will not recognize a .mov file as a valid wrapper for video in HTML 5's <video> tags.

Using Compressor

You need to make your Compressor preset using x264 as a normal QuickTime movie preset (use the table below to help with settings if necessary). You'll then want to grab the script from GitHub and add it as a script to your preset.

Scripting Compressor isn't very straight forward, while you can use AppleScript or launch a script using Compressor they fail to mention that the script must be saved as an application and the file is accessed by using on open.

Helpful Table of Limitations

DeviceMax ResMax Bit RateH.264 Settings
iPhone640x4802.5 MbpsCan only use Baseline profile Level 3.0 with CAVLC
iPod touch640x4802.5 MbpsCan only use Baseline profile Level 3.0 with CAVLC
iPhone 3G640x4802.5 MbpsCan only use Baseline profile Level 3.0 with CAVLC
iPod touch 2G640x4802.5 MbpsCan only use Baseline profile Level 3.0 with CAVLC
iPhone 3G S640x4802.5 MbpsCan only use Baseline profile Level 3.0 with CAVLC
iPod touch 3G640x4802.5 MbpsCan only use Baseline profile Level 3.0 with CAVLC
iPad1280x720"Unlimited"Can only use up to Main Profile Level 3.1
iPhone 41280x720"Unlimited"Can only use up to Main Profile Level 3.1
iPod touch 4G1280x720"Unlimited"Can only use up to Main Profile Level 3.1
G1480x320600Lack of documentation for anything Android
Droid X1280x720?Lack of documentation for anything Android - can't play more than 24FPS

Android information is rather limited. Official Android information is near non-existent.

Thanks to: