Dev Diary - Community Development Map

The Problem

The City of Calgary has a number of resources available for finding out what development is happening in your community. The primary resource is the Development Map, which while comprehensive is significantly off putting because of the amount of information presented. It also only covers the major developments - land use changes and development permits, that are open to some community consultation.

My particular community - Sunalta - has an additional problem because we joined a project with the City of Calgary removing the need for most businesses to apply for a development permit for change of use on our Main Streets (10th Ave. and 14th St). While we are very supportive of the project, we’ve always struggled with notification or knowing when a new business has moved in so we could welcome them to the community. The lack of development permit makes this a bit harder.

The Data

Thankfully, the City provides this data via their open data portal:

However the City also releases the information for more minor information to help answer resident’s questions that are sometimes directed to us.

But can I use it?

So cool, we have some data. How do we make this easier to use? The Open Data Portal does present the data and even allows you to do some visualization but you’re unable to share your visualizations and filtered datasets without logging in. So using is only good for exploration. It also does not provide a great way for less technically minded folks on our board to get this data or even glance at it.

So the next best thing is to look at setting something up to read the data and create the visualizations we want!

The “solution”

From here I’ll share my notes about how I arrived at my solution. It’s far from the only solution, but is the result of my trial, error, and realizing there are a LOT of options I’m simply not even aware of.


The first step was to try and filter the data from the 4 datasets to just the area I cared about - turns out this is easy with Socrata’s Query Language. Adding ?communityname=SUNALTA to my queries and the python py-soda library and I’m off to the races. The land use dataset however does NOT include community fields, so I bet heavily on that 1000 land use items is sufficient for the last 365 days worth of data.

Visualization (Streamlit)

The next tech I dove into - was how can I show off this data on a map and in a table. At work another team has started using Streamlit and it was incredibly easy to get started with. It also had a fantastic feature where it will cache data removing my worry about needing to jump through hoops to avoid hitting too many times and needing to create a caching layer myself.

The biggest gotchas with streamlit I have found is that the default map is nowhere near as featureful as using the plotly library to put dots on the map. Plotly’s table feature while supported has terrible performance and causes everything to drag to a halt. As such I went with Plotly for showing the map, but used Streamlit’s native table view to show off the data.


While Streamlit is easy to run and put a reverse proxy in front of (for SSL and caching), they do also run a free service to deploy your Streamlit app to Heroku for showing off. Streamlit Sharing is free for public repositories (like this project). It will also hook up to GitHub to redeploy your app whenever you push an update without needing to code this in GitHub workflows yourself. Very convenient.

Dev Diary - yycbike_count - Python 3 and GitHub Actions

A couple weeks back, a Twitter bot I run stopped working correctly. It’s been on my list for a long time to revamp and clean up - if only because the server that it was running on was very overdue for being rebuilt. So when Eco Counter changed their private API and broke the script it provided a great chance to rewrite.

Fun items learned along the way:

Python 3 Upgrade Notes

  • Changing print is straight forward.
  • f strings are awesome
  • Cleaning up data is super time consuming, but cleans things up so nicely.

GitHub Actions

As part of the rewrite I wanted to explore ways to stop managing a server in order to run the bot. As it turns out GitHub actions has a schedule feature and could work quite well for both testing and running the script. They also provide a fantastic learning tutorial at

It gave me a great easy chance to get into actually setting up a simplistic CD system for myself, and to get my hands dirty. This coupled with act made it easy to test and run the script through being rewritten.

The whole process is summarized in a sentence, but it took some time and I hope will pay off in spades in the future.

You can find the workflow used on GitHub.

Repurposing or Extending yycbike_count

For those who are interested in taking the work in yycbike_count to use for their own city - please do. There are a couple things to mention if you want to re-use most of the work I did.

Counter Config

The counter config is hard coded into - sorry.

GitHub Actions

If you want to use the workflow you’ll need to make 4 GitHub secrets correspondding to the environment variables (TWITTER_TOKEN, TWITTER_TOKEN_KEY, etc.). If you want to use act, you’ll also need to add them to the .secrets file.


If you want to use act to run the workflow locally you’ll need to add the following to your .actrc file:

-P ubuntu-latest=catthehacker/ubuntu:act-latest