How much will my stream cost to run?

Due to the way DataSift's billing works, this question really needs to be answered in two parts:

How many DPUs will my stream cost to run?

To figure this out, we first need to compile our CSDL filter to get the DPU cost of the filter you wish to consume. On hitting the /compile API endpoint, you will receive a stream hash, and the DPU cost of that stream:

  "hash": "5981063111f95af67afeff15b59bc7e2",
  "created_at": "2014-10-02 11:28:07",
  "dpu": "6.7"

You can also get the DPU cost of an existing stream hash by calling the /dpu API endpoint.

 Once we know the DPU cost of a stream, we know how much that stream costs to run per hour (6.7 DPUs in this case).

Remember; there is no cost to compiling a stream - you have no obligation to use a stream hash once you have compiled it, so if you need to compare the DPU cost of different CSDL filters you can compile as many as you like until you find a DPUI cost that works for you.

How much will the license fees for my stream cost?

Unfortunately DataSift has no way of accurately predicting this, though you could use one of a few methods to help estimate what your license fee spend will be:

  • Run an Historics Preview; this is a cost-efficient statistical analysis of 1% of available data over the timeframe (in the past) you specify. We can return statistics such as how many interactions match your filter in the given timeframe, so you can use this to help predict how many interactions you will receive, and calculate an estimated license fee cost from that
  • You can simply consume your stream for a short period to see roughly how many interactions you receive per minute, and try to extrapolate the overall cost from that
  • Similar to above, you can consume your stream for a short period, but apply a sample value to your CSDL using the interaction.sample target. This will allow you to run the sample filter while receiving only a small sample of the total available interactions (thus costing less in license fees)
Was this article helpful?
0 out of 0 found this helpful
Have more questions? Submit a request