metertimeseriesdataformat
Differences
This shows you the differences between two versions of the page.
|
metertimeseriesdataformat [2011/01/24 11:43] gonium |
metertimeseriesdataformat [2012/10/30 10:27] (current) |
||
|---|---|---|---|
| Line 1: | Line 1: | ||
| + | ==== Implementation ==== | ||
| + | |||
| + | An implementation is being developed here: [[https://github.com/gonium/libklio|libklio (Klio: Greek muse of history)]] | ||
| + | |||
| ==== Goal ==== | ==== Goal ==== | ||
| Line 4: | Line 8: | ||
| * An interface for other components that would like to store or access timeseries data. The library must enforce certain restrictions on incoming data, such as a valid timestamp. For accessing the data an database-cursor like interface seems appropriate. | * An interface for other components that would like to store or access timeseries data. The library must enforce certain restrictions on incoming data, such as a valid timestamp. For accessing the data an database-cursor like interface seems appropriate. | ||
| - | * The library must be able to characterize the data based on an empirical cumulative distribution function (ecdf). This means that min/max, average, median etc. should be represented based on the full data, not only the data that has been reduced in resolution. NOTE: This is currently an open issue, to be solved using complex stream processing techniques. | + | * The library must be able to characterize the data based on an empirical cumulative distribution function (ecdf). This means that min/max, average, median etc. should be represented based on the full data, not only the data that has been reduced in resolution. NOTE: This is currently an open issue, to be solved using complex event processing techniques. See e.g. [[http://subs.emis.de/LNI/Proceedings/Proceedings65/GI-Proceedings.65-24.pdf]]. |
| - | * The library must store metadata for a given sensor. | + | * The library must store metadata for a given sensor, including free-text comments. |
| + | * The library must perform sanity checks. | ||
| ==== Requirements ==== | ==== Requirements ==== | ||
| + | |||
| + | The library is to be used in a wide range of applications, ranging from recording data locally (on a small embedded device like the Chumby) to the webservers of the project. It must therefore be written in C++, as portable as possible. It is safe to assume a POSIX architecture, the boost libraries can also be used. | ||
| === Metainformation === | === Metainformation === | ||
| Line 17: | Line 24: | ||
| * Scaling Factor (e.g. SCALING_FACTOR = 10.0) | * Scaling Factor (e.g. SCALING_FACTOR = 10.0) | ||
| * Offset/Bias | * Offset/Bias | ||
| - | * Missing Value (e.g. MISSING_VALUE = -1) | + | * Missing Value (e.g. MISSING_VALUE = -1) or (better) don't store missing values. Whats not stored is not available. |
| === Data === | === Data === | ||
| Line 23: | Line 30: | ||
| * Timestamp (unixtimestamp) | * Timestamp (unixtimestamp) | ||
| * Value (floatingpoints) | * Value (floatingpoints) | ||
| + | |||
| + | ==== Version 0.1 ==== | ||
| + | |||
| + | The development of this library will be an ongoing process. In the first version, a simple usecase should be satisfied: the data measurements of individual devices (measured by the plugwise system) must be stored. Basically, this means that 10 individual sensors must be stored with synchronized timesteps. The measurement interval will be in the 1-2 second range. Since the data will be used to develop algorithms, a portable and compact (single-file) storage is adequate. Performance of the storage mechanism is not a critical requirement. | ||
| + | |||
| + | Basic operations for this usecase: | ||
| + | |||
| + | * Create an empty storage container file | ||
| + | * add/remove a sensor from the container (including dataset and metadata) | ||
| + | * add a reading to a sensor's storage | ||
| + | * dump all readings of a sensor | ||
| + | |||
| + | |||
| + | |||
metertimeseriesdataformat.1295865839.txt.gz · Last modified: 2012/10/30 10:43 (external edit)
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Noncommercial-Share Alike 3.0 Unported
