A Strava dashboard on a Raspberry Pi (Part 3): The Strava API

This is part 3 of a series of blogposts on how I created a Strava dashboard on a Inky Impression e-ink display with a Raspberry Pi.

OAuth2

This was the part that I expected to be the hard part: getting my data from Strava. Or, to be more precise: getting the connection right so the Strava API would allow me to get that data. Because it requires authentication via the OAuth2 protocol and I’ve tried a similar thing a few years back with a Google API and I just didn’t get it. But now I do.

Strava API documentation

It requires a whole “dance” between your computer code and the Strava API where you exchange all kinds of tokens back and forth. Strava’s Getting Started with the Strava API document explains it quite well. And this blogpost by Graziano Fuccio helped me a lot with the Python code: http://www.grace-dev.com/python-apis/strava-api/.

Frustratingly I still didn’t get it to work though. The reason I found out, is because the URL of the authentication has changed. From https://www.strava.com/oauth/token it became  https://www.strava.com/api/v3/oauth/token. I found this elsewhere in the Stava API documentation, where the correct URL was shown. I’ve told Strava that their Getting Started documentation is outdated. They asked me to create a ticket and I’ve done so, but I don’t think they changed their document yet. But Graziano Fuccio did though.

Continue reading
Posted in Howto, Python | Tagged , , , , , | 2 Comments

A Strava dashboard on a Raspberry Pi (Part 2): Installing software

In last blogpost we set up the Raspberry Pi, attached the Inky Impression display and got the Raspberry Pi ready for remote access.

Time to get the Inky Impression software installed and make the Inky Impression screen display something.

Your SSH connection of choice

For this we’re going to have to run some commands via remote SSH. There are multiple ways to log in remotely. You can use a tool like Putty or the terminal on MacOS (I like iTerm2). That’s actually simpler.

But I chose to use Visual Studio Code because you can edit Python code remotely via SSH straight on the Raspberry Pi.

To do this you must install Visual Studio Code. Visual Studio Code has all kinds of extensions. Here we will install the Remote – SSH extension. And while you’re at it, maybe install the Python extension as well, because we will be writing some Python later.

Installing the Remote-SSH extension in Visual Studio Code
Continue reading
Posted in Howto, Python | Tagged , , , , | 2 Comments

A Strava dashboard on a Raspberry Pi (Part 1): Setting up the Raspberry Pi

This is the list of hardware I’ve used:

  • An Inky Impression 5.7 inch e-ink display.
  • (The Inky Impression comes with a 40-pin female header included to boost height for full-size Pis and standoffs included to securely attach to your Pi)
  • A Raspberry Pi 3 model B+ (I had lying around) + power supply
  • A micro SD card with 8 GB storage or more.
  • Initially: keyboard, mouse and monitor (but if you configure the WiFi on the Raspberry Pi and configure it to allow remote SSH, you can connect to it via WiFi from the convenience of your regular computer)

For those who don’t know a Raspberry Pi: this is a very small and quite cheap computer. The Raspberry Pi 3B+ I’ve used for example is about 40 euros. But you can spend even less, because my Strava dashboard doesn’t exactly require a lot of computing power.

So you could instead use a Raspberry Pi Zero 2 W (15-25 euros), which takes up less space also. But I believe this will require soldering to attach the GPIO. And it seems to be out of stock on a lot of sites.

Continue reading
Posted in Howto, Python | Tagged , , , | 2 Comments

Building a Strava dashboard on a Raspberry Pi with an e-ink display

Let’s face it: my purchase of the Pimoroni Inky Impression 5.7 inch display was a solution looking for a problem. I saw a video about it and I was sold on the idea of having an e-ink display on one of my Raspberry Pi’s.

The Pimoroni Inky Impression on a Raspberry Pi 3B

While having a 7-colour e-ink display is cool and all, I had to come up with a good plan to utilize one. So it wouldn’t end up in a drawer after a short experiment.

You can use it to display images, but it is 7-colour. So you have to “dither” full colour images to have it display well. Actually comic book style images are displayed much better than the average dithered photo. The resolution is quite low (600×448) and the refresh rate is quite slow (10-20 seconds). But for some applications this is just fine.

Continue reading
Posted in Howto, Python | Tagged , , , | 1 Comment

Using Stable Diffusion to create images for a presentation

Have you heard about text-to-image models like DALL-E 2, Stable Diffusion and MidJourney? These are AI algorithms that take in text (the “prompt”) that describes what kind of picture you want as input and as output the algorithm creates that picture, based on billions of images.

An example could be “an astronaut on a bicycle on the moon by Van Gogh”. And this would be one of the results:

{“prompt”: {“software”: “imaginairy”, “prompts”: [[1, “an astronaut on a bicycle on the moon in the style of Van Gogh”]], “prompt_strength”: 7.5, “init_image”: “None”, “init_image_strength”: 0.6, “seed”: 938321671, “steps”: 40, “height”: 512, “width”: 512, “upscale”: false, “fix_faces”: false, “sampler_type”: “plms”}}

I got access to DALL-E 2 in July this year. DALL-E 2 is a closed source algorithm made by OpenAI. You can sign up to request access to DALL-E 2. Once you get access you can use it for free for a limited of runs. After that you have to pay to use it more.

Continue reading
Posted in Weird experiments | Tagged , , , , | Leave a comment

The blog is back

Well, that was scary. Just before I went on holiday I switched providers for my marcel-jan.eu domain. And while I had some time build in before going on vacation, there were problems with the transfer code not working. Because apparently the .eu domain is different from the regular .nl domain.

In the end I managed to get my marcel-jan.eu mail working just the evening before leaving. But I saw no way to migrate the blog while packing my bags. So the blog was down for more than 2 weeks. Did anybody miss it?

After getting back home I had to piece back the WordPress blog with a .zip backup and a backup of the filesystem. Never done such a thing before. And the original WordPress blog on my old provider’s site was already gone. So there were no more alternatives to do a better export.

Importing did not go as planned

I started by installing WordPress at my new provider’s site. And I went to PHPMyAdmin, which is the tool to work with the database behind WordPress. I imported the .zip (with a .sql file in it). And.. no blogposts. A further look with PHPMyAdmin in the database showed that there were several xxx_posts tables. The one the WordPress site was looking in, was wplx_posts. My imported tables where called wp_posts and 4a2vK12BOL_posts. wp_posts contained old stuff. The 4a2vK12BOL_posts table turned out to have all my posts.

Time to play dirty with SQL

So how do I point WordPress to the right data? It’s good to have some SQL skills. What if.. hear me out.. I read the .sql file I got from the export, pick out the SQL to import the 4a2vK12BOL_posts table. Search and replace in the SQL text the term “4a2vK12BOL_posts” for “wplx_posts” in a text editor? And then import that? It’s dirty, I grant you that.

But it turns out, it works. As long as you don’t create any new posts beforehand that use the same ID as the ones you try to import. A quick removal of the Hello World post made sure of that.

And it worked. I got my posts back. Okay, that’s something. I don’t have to type all my writings from 2017 to now again.

I did something similar for the comments. Make sure you do that before the first comment spam arrives. Because it will overlap the ID in the comment table with the ones you try to import.

Now I need some images

I was not really surprised that restoring table contents did nothing for my images. Pretty sure that had to come from the filesystem. Luckily I had made a backup of all that. But where to get the image files and where to put them?

Well, looking over the sql for the posts table, I found references to image files like this one: https://marcel-jan.eu/datablog/wp-content/uploads/2017/11/Heart-Reanimation-65992.gif. So somewhere there should be a path with something like wp-content/uploads in the name and a lot of gifs and jpgs in it. I found that, uploaded the directories to the new site and now I had my images back.

That one time I used TablePress

My article about Lion’s Mane is one of the most popular blogposts for some reason. Lots of people who want to gain cognitive enhancement. (I wished my post about becoming a skeptic was just as popular. Oh well.) In that post was my one use of a TablePress table. How to get that back?

It turns out the data can be found in the options table. But I had some doubts whether importing it would mess other things up and whether TablePress would find it. So I dug in the Internet Archive to find the contents of the table, and used Excel to create a csv file of that table. Imported that in TablePress and hey presto: we got ourselves our table back.

Tags and categories

One thing I noticed that my categories and tags were gone. The categories were a big mess after 5 years of blogging. Actually it wasn’t a big loss. More like a good moment to rethink them. As for tags: it would be nice to retrieve them somehow.

Fortunately there is documentation on the data model of WordPress’ database. Like this site: https://wp-staging.com/docs/the-wordpress-database-structure/

From this I learned what tables I needed to import to get my tags back. It turns out it’s wplx_term_taxonomy and wplx_term_relationships. In wplx_term_taxonomy there were already 3 IDs taken. ID 2 and 3 were now a wp_theme, where in my old table they were categories.

I decided to remove ID 1, 2 and 3 from my insert statement and import that. If I’m missing 2 categories, that won’t hurt me a lot.

Anything else?

From the wp-staging article I learned I probably won’t be needing much more from the import. Maybe I will me missing some stuff from the options table, because there’s all kind of stuff that plugins put there. But I’m not going to open that can of worms.

I certainly learned a lot on WordPress and its database.. forcefully. Glad the blog is back on the road at my new provider.

Coverart by DALL-E 2

Posted in Howto | Tagged , , , , | Leave a comment

I started vlogging about data mesh (and other things)

Last June I made a short video while walking in the park next to the DIKW Intelligence office. And I posted it on LinkedIn. To my surprise it did very well. So I thought: why not make more of these short videos on data topics? And why not make them in somewhere in nature?

I’m on my bike almost every day this time of year. Surely I could make a short stop and do a little talk? I┬ástarted to make them in Dutch and then also in English. Continue reading

Posted in Active Learning, Data engineering | Tagged , , | Leave a comment

Adding the track of my bike ride on a Folium map

Having markers of videos and photos taken during my bike ride is cool and all, but how about having a track of the bike ride itself? All my bike rides are registered on Strava, the cycling and running app. Strava has an API for developers, but it requires connecting via OAuth 2.0 and knowledge of the API. I decided to go an easier route: because I’m Strava Premium member, I can download the GPX track of any ride, including my own.

These .gpx track files are of the same XML structure as we saw embedded in video files in my last blogpost. I can just open the file and use almost the same Python code to read the locations.

Continue reading

Posted in Howto, Python | Tagged , , , , , , , | Leave a comment

Photo locations, marker icons and displaying photos on my map

When I was finished last week with creating my video location map in Python, I thought “shame I can’t plot photo locations”. That’s because my Fuji X-T30 camera doesn’t store GPS info. When I bought the camera I assumed every modern camera had GPS tagging, so I didn’t even checked that feature. Too bad. But I also made some photo’s during my vacations with my humble iPhone 8. And it does have GPS tags. So let’s plot some photo locations.

Continue reading

Posted in Data engineering, Howto, Python | Tagged , , , , , , , | Leave a comment