What are billable transactions?65
Pages:
1
|
MikeMills private msg quote post Address this user | |
| We get asked this a lot. A billable transaction is any API call to the GroveStream's servers. This includes calls from devices uploading stream data, calls from the GroveStreams user interface (since it is built with the public API), and calls from GroveStreams embedded widgets. The GroveStreams servers will cache a request for up to 10 seconds. Requests that hit the cache are not counted as a billable transaction. This avoids viral requests from the UI and embedded widgets. Users of the GS UI that have "Editor" rights will not hit the cache. A majority of transaction counts for users will be from device feed uploads. Small customers, with only a few streams, may have a majority of their transaction counts come from the UI or embedded widgets. Typical culprits and workarounds for high transaction counts: 1) A device that uploads at a high frequency sample rate. Decrease the sample rate. 2) A gateway that uploads feeds for many devices/streams. Use a GS feed API that allows for the batching of many stream feeds into one feed PUT. This may require the use of JSON or the advanced feed PUT API. 3) A dashboard with many widgets left open with "Remember me" sign-in enabled and with polling rates set to a low value (such as 5 seconds). Each widget performs it's own transactions. Increase the polling value or don't leave dashboards open over night to lower your transaction count. 4) Embed-able dashboard widgets with low polling values. Increase the polling value. 5) New organization notification checks from the UI studios. These occur once every 30 seconds by default. This can be changed within a studio. Click Admin - Organization Information to change the polling frequency. To check your transaction usage: 1) Click your name, choose Account, and select the Usage tab. Checking your transaction count will increase your transaction count by 1-3 transactions. Like almost all GS calls from the UI, this call too is an API call. To monitor transactions from the UI (your browser): 1) Click F12 (this will bring up the browser debugger window for most browsers such as Chrome or IE). 2) Click the Network tab and select the XHR filter 3) Watch for GS API calls made to the server. To monitor transactions that use an API key (usually device feeds). 1) Click the API key button within a studio 2) Engage API tracing 3) Monitor System notifications to see each transaction |
||
| Post 1 • IP flag post | ||
|
|
MikeMills private msg quote post Address this user | |
| Derived streams will not increase the transaction count during the derivation process when their values are derived and saved. | ||
| Post 2 • IP flag post | ||
|
steveb private msg quote post Address this user | |
| my learnings re transaction counts are that unexpected high usage comes from the dashboard - personally I think the defaults are too aggressive for a "pay per transaction" system - at least for the people at the low end who are trying to stay at or below the 10,000/month free system I found out that the items that were eating my transactions were The dashboard polling frequency for checking for new message notifications was 60 seconds, I bumped this to 60000 seconds Also the default refresh time for each widget on a dashboard is set to 60 seconds So if you have a dashboard with 5 widgets, that's a poll every 10 seconds doing nothing, so it's very easy to rack up a relatively high number of transactions for doing nothing. Of course if you are using millions of transactions then it doesn't matter, it's the low usage user that gets hit, in relative terms. I would rather that both of these for all new items be set to a much bigger number to force people who want near real-time updates to think about it and select an appropriate value, rather than have an aggressively low value for all that can cause a large transaction count. To be honest, the costs are minimal - that doesn't bother me at all - but understanding where the cost is incurred I found the main thing - and the default settings seem tuned to rack up "high" transaction counts without the user being aware of them -Steve |
||
| Post 3 • IP flag post | ||
|
|
MikeMills private msg quote post Address this user | |
| We originally had widget polling default to 5 seconds but bumped it up to 60 seconds to keep transactions lower. We could default it a higher number again. We'll wait a bit and see if we get more feedback around that. | ||
| Post 4 • IP flag post | ||
|
|
MikeMills private msg quote post Address this user | |
| We've changed some defaults around polling and added some administration settings around UI polling. See: https://forum.grovestreams.com/topic/70/new-org-ui-transaction-management-settings/ | ||
| Post 5 • IP flag post | ||
|
draythomp private msg quote post Address this user | |
| We've discussed this previously, but I want to bring it up here to see other opinions. Situation: Having a weather station that uploads, say every minute, feeds for wind speed, temperature, barometric pressure, y'know, weather stuff. I don't want to get too specific and wander off subject. The idea is that this person (ain't me, I measure power) wants to publish this stuff constantly on a web page so his friends in the hobby can build comparison charts and such. Say Bill in the Arizona desert wants to compare his weather to Emil in Southern India by charting both of their data on a single chart. You have great facilities for this, but neither of these guys have much money. This would mean a heck of a lot of API reads to gather the data and chart it on the web for others to look at and comment on. Each of these guys can control their own uploads, but they can't control downloads by other folk. This makes them really hesitant to publish ways to read the data and kind of stymies their enthusiasm. Especially something like posting a widget to a forum to show off a new thingie that may be viewed by a bunch of folk over an indefinite period. They worry about someone displaying data continuously and running up a bill that they have no control over. Yes, they can restrict it, by keeping it secret, but sharing is what these folk want to do. They want to show off their data collection and facilities to each other. I know that the per transaction amount is tiny; actually, it's so small that I have trouble figuring it out, but people worry about some guy out there running up a bill by grabbing data that they really want to share. What can Bill and Emil do to be confident someone doesn't put the screws to them? |
||
| Post 6 • IP flag post | ||
|
|
MikeMills private msg quote post Address this user | |
| Let's work through draythomp's scenario: Bill's upload transactions/month: 1440*30 = 43,200 transactions Emil's upload transactions/month: 1440*30 = 43,200 transactions Bill and Emil's UI transactions/month (best guess) 200,000 transactions So without 3rd party downloading transactions, the total is 286,400 transactions/month (1st 10,000 transactions are free): $1.38/month Now, let's do a best guess on public widget transactions. Assumptions: - A Line graph widget is posted on a forum with one or more streams - Widget Polling is set to zero - Widget posting goes viral The number of GroveStreams transactions to display the widget is 3. One to query for the widget, one to query for all stream metadata, and one to query for all stream feeds. If the widget were polling, there would be one transaction for each poll to get the latest stream feeds. This widget is not polling. GroveStreams servers will cache server requests for 10 seconds. There are some caching exceptions from our studios when the user has Editor rights. We don't bill for queries that hit the cache. Worst case scenario is that this widget will incur 3 billable transactions every 10 seconds or 25,920/day or 777,600/month. This brings a maximum number of transactions to 1,064,000 transactions/month for a cost of $5.32 per month. Not bad for a graph that went viral. My guess is that this example did not expose the real problem with our current billing model. It does reflect how cheap we are, but what we hear from most people is the difficulty and risk with trying to estimate transactions. We know we need to improve our billing model so that we are still affordable but one where there is less risk for us and our customers. Things we've thought about: - Allow the user to set a transaction or billing amount threshold. What happens when the threshold is reached? Do we prevent stream feeds from still arriving - probably not. Maybe we just send email notifications when the limit is being approached and then the org owner can make adjustments to slow their usage down (such as removing the viral widget). - Expose billing and usage metrics as optional GS streams. Then users can graph them and set alerts and such. - Move to billing tiers. Free, small, medium, large, enterprise. Each tier has a monthly cost. But thresholds are still implied and users probably want an email notification when they are crossing a threshold. Users would probably feel more comfortable knowing tiered amounts, but in reality, they would probably end up paying us more. - Any other ideas? - Do our competitors have a better billing solution? |
||
| Post 7 • IP flag post | ||
|
|
draythomp private msg quote post Address this user | |
| That cost is low enough that no one would complain. Well, that's naive of me; there would be complaints, albeit somewhat trivial. However, the problem happens when the discussion between the hobbyist gets going. There would be graph after graph showing different ideas. Several different forums would catch it and point or copy it. People would blog about how they would have done it differently (copying in the code of course) and this could set for years. And, you're totally right, it's the risk that make people worry, not the fact. That's mostly because risk aversion keeps them from doing it and the facts never become apparent. How about billing more for incoming and make outgoing free. By that I mean any transaction that results in data changing or increasing storage cost money, reading back the data costs nothing. Bill and Emil would each be responsible for paying for their costs to get it to your servers, but wouldn't have to worry about the guy in France that sucks it down to his machine to show off how much better his weather station is. I have no clue at all what it costs you to receive, store and deliver the data, so this may be a dumb idea, but is is done by other services. The idea is that makers want to share their conquests with the world. Corporations want to control access to the data. These two things are inherently contradictory. You folk have a tough time trying to resolve it. |
||
| Post 8 • IP flag post | ||
|
|
MikeMills private msg quote post Address this user | |
| I think exposing and controlling the data can be accomplished so we're not too worried about that. In our mind, corporate users aren't too much different than home users. I used to build BI corporate software for IBM and most of those end users couldn't touch a home hobbiest's technical ability. I think the only difference is that larger companies can afford to pay consultants/analysts to do the modeling (and device programming) for them. Small businesses and home owners can't. We just need to keep things simple for all users. Yeah, we're moving in the right direction, but have a ways to go. We could do as you suggested - only charge for Feed PUTs. The problem we face is that we exposed our entire API so that users could build their own interface. We really can't anticipate what they'll build. We also have some users that want to constantly pull out component and stream definitions. If we un-restrict all GETs, users will poll for those types of items frequently. We'll think about this some more. There's got to be a simple solution there. |
||
| Post 9 • IP flag post | ||
Pages:
1