Home » dataset » dataset:懶人的資料庫
2016-08-30 dataset, Python 尚無留言

這裡有個有趣的小東西,有在寫 Python 程式者,可能會覺得很受用。。。


dataset: databases for lazy people

 Although managing data in relational database has plenty of benefits, they’re rarely used in day-to-day work with small to medium scale datasets. But why is that? Why do we see an awful lot of data stored in static files in CSV or JSON format, even though they are hard to query and update incrementally?

The answer is that programmers are lazy, and thus they tend to prefer the easiest solution they find. And in Python, a database isn’t the simplest solution for storing a bunch of structured data. This is what dataset is going to change!

dataset provides two key functions that make using SQL databases in Python a breeze:

  • A simple abstraction layer removes most direct SQL statements without the necessity for a full ORM model – essentially, databases can be used like a JSON file or NoSQL store.
  • Database contents can be exported (frozen) using a sophisticated plain file generatorwith JSON and CSV support. Exports can be configured to include metadata and dynamic file names depending on the exported data. The exporter can also be used as a command-line tool, datafreeze.

A simple data loading script using dataset might look like this:

Here is similar code, without dataset.


  • Automatic schema: If a table or column is written that does not exist in the database, it will be created automatically.
  • Upserts: Records are either created or updated, depending on whether an existing version can be found.
  • Query helpers for simple queries such as all rows in a table or all distinct values across a set of columns.
  • Compatibility: Being built on top of SQLAlchemy, dataset works with all major databases, such as SQLite, PostgreSQL and MySQL.
  • Scripted exports: Data can be exported based on a scripted configuration, making the process easy and replicable.


9  +    =  seventeen

這個網站採用 Akismet 服務減少垃圾留言。進一步了解 Akismet 如何處理網站訪客的留言資料