Timeseries data structure

georgelza · 2024-10-02T09:28:04.394Z

Hi all.
I’d like to verify/validate the following statement:
with regards to time series data…

It does introduce a concept that needs to be consider, every record is primarily comprised out of 3 components.

- timestamp of the event
- metadata about the event
    this can be a array of key/value pairs.
- measurement/value of the event

   {
	  "timestamp" : "2024-09-27T20:23:14.869Z",
	  "metadata" : {
		"siteId" : 1009,
		"deviceId" : 1042,
		"sensorId" : 10180,
		"unit" : "Psi"
	  },
	  "measurement" : 1015.3997
	}

Would everyone agree that this is correct,
For me a a record/payload that have multiple key’s with measurement/values is not a time series specific record, it falls under data that sits better in a normal collection.

G

georgelza · 2024-10-02T10:27:48.156Z

guess this is also a valid structure.

{
  timestamp: 2024-10-01T02:00:00.000Z,
  metadata: {
    deviceId: 1002,
    sensorId: 90005,
    siteId: 1001,
    unit: 'celcius'
  },
  symbol: 'BTC-USD',
  price: 34104.1145,
  measurement: 12
}

implying multiple values for the time stamp and the metadata set provided.

steevej · 2024-10-02T12:45:12.701Z

georgelza:

Would everyone agree that this is correct

That is in line with the documentation.

georgelza:

Would everyone agree that this is correct

I do

georgelza:

For me a a record/payload that have multiple key’s with measurement/values is not a time series specific record, it falls under data that sits better in a normal collection

No one is enforcing the use of this collection type. You may use normal collection if they fit better your use-case. Time series offer some benefits that may or may not be useful for your use-case. Storing measurements in buckets is one such benefits for space efficiency in particular.

georgelza:

guess this is also a valid structure.

{
  timestamp: 2024-10-01T02:00:00.000Z,
  metadata: {
    deviceId: 1002,
    sensorId: 90005,
    siteId: 1001,
    unit: 'celcius'
  },
  symbol: 'BTC-USD',
  price: 34104.1145,
  measurement: 12
}

Yes it is valid but not logical as you have a single unit in metadata but multiple values as your measurements. Personally, I would not put unit in the metadata but I would do put symbol in the metadata. I see the price as a series of time events related to a specific stock price.

My experience is quite limited with TS so I may be completely wrong. But it is an interesting discussion that I wanted to participate.

georgelza · 2024-10-02T17:21:36.502Z

the extended values in the measurement sections is problem just bad examples.
agree on unit, as the unit directly mapped to measurement.
my question was more the presence of secondary “tag” : numeric value in the post metadata section
G

steevej · 2024-10-03T13:00:35.929Z

I do not understand what you mean by

georgelza:

the presence of secondary “tag” : numeric value in the post metadata section