GroveStreams

Timing of derived streams412

ctmorrison private msg quote post Address this user
In an effort to reduce my data usage, I've been looking at concatenating multiple inputs (analog/digital) into a single string stream with the expectation of parsing this out in Grovestreams into derived streams. Some testing proves this will work OK, but I am a bit concerned about the time stamps and possibly time delay on seeing the resulting derived data streams.

How often/quickly does the derivation engine run? Is the timing of the resulting streams deterministic? Quite often, we need to know exact (per minute) temporal relationships of the data we're capturing.
Post 1 IP   flag post
MikeMills private msg quote post Address this user
Derivation is currently running about every 1-2 minutes.

The resulting sample will have the same time as the Regular stream sample's time being derived from.

We may include derived samples/intervals in our pricing data I/O someday. We're starting to see a lot more derived streams than we anticipated. Some customers have far more derived streams than device populated streams. No decision has been made yet, but it's something to consider.
Post 2 IP   flag post
ctmorrison private msg quote post Address this user
So...here's where I'm coming from. I'm looking at sending the analog six inputs (temperatures) from a Particle Electron as a string and then parsing out the individual readings at Grovestreams. This not only helps reduce the Electron data usage, but also the Grovestreams data usage. For example I would send the string "a=0764|1423|0609|1854|0701|0999" as opposed to "a1=76.4&a2=142.3&a3=60.9&a4=185.4&a5=70.1&a6=99.9".

The digital could be more of a reduction, such as "d=110101" as opposed to "d1=1&d2=1&d3=0&d4=1&d5=0&d6=1".

In order to make the Electron's data plan affordable, I'm also likely increasing the data push period from 1 minute to 5 minutes unless we need more detailed data for a short period of time and then shorten it, again.

Does this strategy makes sense from a Grovestreams' perspective or am I "robbing Peter to pay Paul?" I don't want to go through the development effort if it just doesn't make sense from a GS perspective.
Post 3 IP   flag post
ctmorrison private msg quote post Address this user
The timestamps of the derived streams appear to be "close" to the timestamp of the stream from which they're derived. However, I'm seeing as much as a 59 second difference between stream timestamps that are based upon the same stream. When you said they'd have "the same time" did you mean just very close or is something not working as expected? When we're studying equipment and trying to determine inter-relationships and functionality, timing can be critical.

I've included a screenshot of some of the streams. I don't yet know why ai6 is not being derived.

Post 4 IP   flag post
MikeMills private msg quote post Address this user
The derived sample time should match the dependent's sample time for regular streams. Can you attach a screen shot of your derived stream derivation tab?
Post 5 IP   flag post
ctmorrison private msg quote post Address this user
Here's one of the derived stream's derivation tab.

Post 6 IP   flag post
ctmorrison private msg quote post Address this user
OK. I see the error in how I was looking at things. When I double-click on the component and see all the streams, the timestamp I'm seeing is the "Last Updated Date" which is not indicative of the actual timestamp on the stream's data being displayed. However, if I go into each stream, I can see the actual timestamps on each data point and they do, in fact, match across all derived streams and the stream data from which they are derived. Perfect!
Post 7 IP   flag post
2965 7 7
Log in or sign up to compose a reply.