What I receive via DataSift doesn't match what I see on

I wrote an article a while back, explaining how to migrate from Twitter's Streaming API to DataSift - this is a follow up to that article.

We are often asked; "I can see this Tweet appear on, but I didn't receive it via my DataSift stream - why!?" There could be numerous reasons behind this:

When was the Tweet sent?

When running a live stream on DataSift, the interactions you receive will be sent to you in real time (or perhaps with a slight delay, depending on your delivery method). The Tweets you see when you run a search on will often have been created several minutes, or hours ago - you should double check you were actively consuming your DataSift stream when that Tweet was sent to ensure you should have received it.

Should your CSDL have matched that Tweet?

The search analyzes multiple data points within a Tweet when searching for a given keyword; DataSift on the other hand will return interactions based on more specific filters. Take a look at your CSDL, and make sure you are explicitly filtering against the data point you believe should caused that Tweet to be returned.

A list of all our available Twitter and Interaction targets is available on within our Targets Documentation.

Are you filtering just for Tweets, or retweets as well?

Filtering against CSDL targets such as twitter.text or filter against Tweets only - not retweets. We provide twitter.retweet.* targets to filter specifically for retweets, and the interaction.* targets which filter against attributes common to both Tweets and retweets, such as the content of the interaction (interaction.content), or the ID of the user who created the interaction (

Are you filtering for entities such as links, and @mentions correctly?

When using the search in particular, a regular keyword search will return Tweets which contain your keyword as part of the title of a link included in the Tweet, or which @mention or were sent by the user you search for. When using DataSift, we allow you to create very specific filters, so you need to ensure your CSDL is filtering on the entity that should match your keyword.

Have you enabled the Twitter data source?

You will need to ensure you have enabled your data sources at before DataSift will return any interactions from that source.

Was this article helpful?
1 out of 1 found this helpful
Have more questions? Submit a request