[Tutor] Simple flask app querying twitter keeps crashing.

[jimmy connelly]

Hello I have a very simple Flask app for fetching tweets from twitter...

Its very simple
https://twitter-happy-or-sad-b.herokuapp.com/ outputs the word "hello" and
nothing else,

https://twitter-happy-or-sad-b.herokuapp.com/subject-tweets/any text
returns 100 tweets containg 'any text' or 100 tweets containing whatever
text you insert as a query string.

It runs great for a number of hours then crashes.

The / endpoint still outputs hello but the api for fetcing tweets is
completly broken after the crash.

The app works fine again for a few hours after restarting the dynos
manually then it crashes again.

I thought maybe it was because the dyno was going to sleep so I upgraded to
'hobby' service., but it doesn't help matters except it seems to run for a
little longer before crashing.

There is nothing in the logs at the moment because the logs don't go back
far enough to the time of the error.

I'm really stuck and don't know what to do.

Below is the entire code for the app - told you it was simple!


import tweepy
import json
import sys
from objdict import ObjDict
from flask import Flask
from flask import request

from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer
from flask_cors import CORS

app = Flask(__name__)


CORS(app)

@app.route("/")
def helloWorld():
return "Hello"


# Enter authorisations
consumer_key = "my-consumer-key"
consumer_secret = "my-consumer-secret"
access_key = "my-access-key"
access_secret = "my-access-secret"

# Set up your authorisations
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_key, access_secret)

# Set up API call
api = tweepy.API(auth, parser = tweepy.parsers.JSONParser())
#Make another app.route() decorator here that takes in an integer id in the
@app.route('/subject-tweets/<string:query>')
def getTweetsofQuery(query):
if request.method == 'GET':
# Set search query
searchquery = query + '-filter:retweets'
sentences = []
data = api.search(q = searchquery, count = 100, lang = 'en', result_type =
'mixed')
print 'size of data is ' + str(sys.getsizeof(data))
emptyData = False

#gotdata = None
try:
#gotdata = data.values()[1][10]['text']
for x in range(1, 99):
sentences.append(data.values()[1][x]['text'])
except IndexError:
sentences.append('no tweets for ' + query + ' - sorry')
emptyData = True
except Exception as e:
# do something with your exception
print(str(e))



# OLD CODE
#try:
# gotdata = data.values()[1][10]['text']
# for x in range(1, 99):
# sentences.append(data.values()[1][x]['text'])
#except IndexError:
# sentences.append('no tweets for ' + query + ' - sorry')
# emptyData = True




analyzer = SentimentIntensityAnalyzer()
compoundScores = []
for sentence in sentences:
vs = analyzer.polarity_scores(sentence)
compoundScores.append(vs['compound'])
#print(str(vs['compound']))

def mean(compoundScores ):
return float(sum(compoundScores )) / max(len(compoundScores ), 1)
data = ObjDict()
data.subject = query
if emptyData == True:
data.sentiment = 'sorry we dont have a score for ' + query
data.score = "It was impossible to measure sentiment for " + query + " this
time because no one tweeted that phrase in the last 7 days"
data.tweets = ["Sorry we don't have any tweets for " + query ]
else :
data.score = mean(compoundScores)
data.scoreString = ' The sentiment score for ' + query + ' is ' + str
(mean(compoundScores))
if data.score >= 0.5:
#data.sentiment = 'The sentiment rating for '+ query + ' is positive'
data.sentiment = 'Positive'
elif data.score > -0.5 and data.score < 0.5 :
#data.sentiment = 'The sentiment rating for '+ query + ' is neutral'
data.sentiment = 'Neutral'
elif data.score <= -0.5 :
#data.sentiment = 'The sentiment rating for '+ query + ' is negative'
data.sentiment = 'Negative'
data.tweets = sentences
json_data = data.dumps()
return json.dumps(data)

if __name__ == '__main__':
app.debug = True
app.run(host='0.0.0.0', port=5000)