Happy holidays! I started working on an anomaly detection system that thought I need. The first step was to have data downloads from an API, and some functions storing data for later use. The API returns data in JSON format, but here you effectively deal with a list of tuples. Instead of using plain tuples, I wanted to have the records as List[MyTuple], and decided to use the NamedTuple class from typing package (not @dataclass) for better usability. A small detail, but I felt it worked out nice actually.
In my own projects I’m often using databases like MySQL, Snowflake, MongoDB, SQLite3, or maybe the large cloud database providers: Azure, AWS, GCP. Some time ago I had heard about new ArcticDB, and wanted to try it out. There’s two things that caught my attention: it’s serverless, and it works seamlessly with Pandas dataframes. Sounds like it can really be the no hassle solution, and they guarantee it for petabyte scale. First tests and running the workflow went well, and I’m waiting forward to see how it looks later when I have the anomaly detection part ready.