Dev Diary - yycbike_count - Python 3 and GitHub Actions

A couple weeks back, a Twitter bot I run stopped working correctly. It’s been on my list for a long time to revamp and clean up - if only because the server that it was running on was very overdue for being rebuilt. So when Eco Counter changed their private API and broke the script it provided a great chance to rewrite.

Fun items learned along the way:

Python 3 Upgrade Notes

  • Changing print is straight forward.
  • f strings are awesome
  • Cleaning up data is super time consuming, but cleans things up so nicely.

GitHub Actions

As part of the rewrite I wanted to explore ways to stop managing a server in order to run the bot. As it turns out GitHub actions has a schedule feature and could work quite well for both testing and running the script. They also provide a fantastic learning tutorial at https://lab.github.com/githubtraining/github-actions:-hello-world.

It gave me a great easy chance to get into actually setting up a simplistic CD system for myself, and to get my hands dirty. This coupled with act made it easy to test and run the script through being rewritten.

The whole process is summarized in a sentence, but it took some time and I hope will pay off in spades in the future.

You can find the workflow used on GitHub.

Repurposing or Extending yycbike_count

For those who are interested in taking the work in yycbike_count to use for their own city - please do. There are a couple things to mention if you want to re-use most of the work I did.

Counter Config

The counter config is hard coded into twitterBot.py - sorry.

GitHub Actions

If you want to use the workflow you’ll need to make 4 GitHub secrets correspondding to the environment variables (TWITTER_TOKEN, TWITTER_TOKEN_KEY, etc.). If you want to use act, you’ll also need to add them to the .secrets file.

act

If you want to use act to run the workflow locally you’ll need to add the following to your .actrc file:

-P ubuntu-latest=catthehacker/ubuntu:act-latest

Quick Looking a .csv File

On ServerFault someone was asked a question why .csv files were unable to be viewed in Quick Look. For some reason .csv files are not defined as plain text by the operating system which leads to odd conflicts between applications that can deal with .csv files that causes QuickLook to report that it can’t find a generator for the file and just show the icon.

The one way to force it to work is to roll your own, with a bit of help after reading the QuickLook Programming Reference it was rather simple to roll together a proof of concept QuickLook generator for .csv files.

It’s not pretty, it may break at any time (if only because it’s using a dynamic UTI since one is not defined anywhere, however .csv files have been using the UTI “dyn.age80g650” for quite a while - as far back as prior to 10.0) but it works and forces the .csv to display properly.

The code and a download of CSVQL is available on GitHub:

Source: http://github.com/Chealion/chealion/tree/master

Download: http://cloud.github.com/downloads/Chealion/chealion/CSVQL.zip

Check out Pascal’s comment for a much nicer looking version - actually puts the data into tables.