metertimeseriesdataformat
Differences
This shows you the differences between two versions of the page.
|
metertimeseriesdataformat [2011/01/12 15:57] kleinmat |
metertimeseriesdataformat [2012/10/30 10:27] (current) |
||
|---|---|---|---|
| Line 1: | Line 1: | ||
| - | ==== Overview ==== | + | ==== Implementation ==== |
| - | The aim is to specifiy and create a generic data format for timeseries with measured data. | + | An implementation is being developed here: [[https://github.com/gonium/libklio|libklio (Klio: Greek muse of history)]] |
| + | |||
| + | ==== Goal ==== | ||
| + | |||
| + | The aim is to specifiy and create a generic software library for timeseries with measured data. The library must be able to store timeseries with associated metadata. The library should consist of two parts: | ||
| + | |||
| + | * An interface for other components that would like to store or access timeseries data. The library must enforce certain restrictions on incoming data, such as a valid timestamp. For accessing the data an database-cursor like interface seems appropriate. | ||
| + | * The library must be able to characterize the data based on an empirical cumulative distribution function (ecdf). This means that min/max, average, median etc. should be represented based on the full data, not only the data that has been reduced in resolution. NOTE: This is currently an open issue, to be solved using complex event processing techniques. See e.g. [[http://subs.emis.de/LNI/Proceedings/Proceedings65/GI-Proceedings.65-24.pdf]]. | ||
| + | * The library must store metadata for a given sensor, including free-text comments. | ||
| + | * The library must perform sanity checks. | ||
| ==== Requirements ==== | ==== Requirements ==== | ||
| + | |||
| + | The library is to be used in a wide range of applications, ranging from recording data locally (on a small embedded device like the Chumby) to the webservers of the project. It must therefore be written in C++, as portable as possible. It is safe to assume a POSIX architecture, the boost libraries can also be used. | ||
| === Metainformation === | === Metainformation === | ||
| Line 13: | Line 24: | ||
| * Scaling Factor (e.g. SCALING_FACTOR = 10.0) | * Scaling Factor (e.g. SCALING_FACTOR = 10.0) | ||
| * Offset/Bias | * Offset/Bias | ||
| - | * Missing Value (e.g. MISSING_VALUE = -1) | + | * Missing Value (e.g. MISSING_VALUE = -1) or (better) don't store missing values. Whats not stored is not available. |
| === Data === | === Data === | ||
| - | * Timestamp (unixtimestamp?) | + | * Timestamp (unixtimestamp) |
| - | * Value (int with scalingfactor or floatingpoints?) | + | * Value (floatingpoints) |
| + | |||
| + | ==== Version 0.1 ==== | ||
| + | |||
| + | The development of this library will be an ongoing process. In the first version, a simple usecase should be satisfied: the data measurements of individual devices (measured by the plugwise system) must be stored. Basically, this means that 10 individual sensors must be stored with synchronized timesteps. The measurement interval will be in the 1-2 second range. Since the data will be used to develop algorithms, a portable and compact (single-file) storage is adequate. Performance of the storage mechanism is not a critical requirement. | ||
| + | |||
| + | Basic operations for this usecase: | ||
| + | |||
| + | * Create an empty storage container file | ||
| + | * add/remove a sensor from the container (including dataset and metadata) | ||
| + | * add a reading to a sensor's storage | ||
| + | * dump all readings of a sensor | ||
| + | |||
| + | |||
metertimeseriesdataformat.1294844231.txt.gz · Last modified: 2012/10/30 10:36 (external edit)
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Noncommercial-Share Alike 3.0 Unported
