Category: Data Engineering
-
Database and tools
I had been thinking about having a dedicated relational database server at home, and found an unused old PC that was laying around and fit for the purpose. Changing to Ubuntu and installing the chosen open source database was quite fast. Then there was setting up a secure connection between the server and my main…
-
Underground data
Public transit offers some nice open data sets. I was searching for subway data, and found out that New York releases a large data batch of metro arrival times every morning. Not knowing much on the NYC metro, there was some research to be done, going through the metro map, route time tables, station locations…
-
Outliers?
Happy holidays! I started working on an anomaly detection system that thought I need. The first step was to have data downloads from an API, and some functions storing data for later use. The API returns data in JSON format, but here you effectively deal with a list of tuples. Instead of using plain tuples,…