Tag Archives: twitter

Internet, Python, Software

Posted on by Matt Warren

This is a simple Twitter Python script that checks your friends time-line and prints out any links that have been posted. In addition it visits each of the URLs and finds the actual title of the destination page and prints that along side. This simple script demonstrates an easy way to gather some of the hottest trends on the internet the moment they happen.

If you set up a Twitter account within a niche and find a few of the players in that niche to follow then you can simply find any links posted, check them to see if they are on topic (using some keyword/heuristics) and then either notify yourself of the interesting content, or automatically scrape it for use on one of your related websites. That gives you perhaps the most up to date content possible before it hits Google Trends. It also gives you a chance to promote it before the social news sites find it (or be the first to submit it to them).

With a bit more work you could parse out some of the meta tag keywords/description, crawl the website, or find and cut out the content from the page. If it’s a blog you could post a comment.

Example Usage:

$ python TwitterLinks.py
http://bit.ly/s8rQX - Twitter Status - Tweets from users you follow may be missing from your timeline
http://bit.ly/26hiT - Why Link Exchanges Are a Terrible, No-Good Idea - Food Blog Alliance
http://FrankAndTrey.com - Frank and Trey
http://bit.ly/yPRHp - Gallery: Cute animals in the news this week
...

And here’s the python code:

#!/usr/bin/env python
# -*- coding: utf-8 -*-
# (C) 2009 HalOtis Marketing
# written by Matt Warren
# http://halotis.com/
 
try:
   import json
except:
   import simplejson as json # http://undefined.org/python/#simplejson
import twitter     #http://code.google.com/p/python-twitter/
 
from urllib2 import urlopen
import re
 
SETTINGS = {'user':'twitter user name', 'password':'you password here'}
 
def listFriendsURLs(user, password):
    re_pattern='.*?((?:http|https)(?::\\/{2}[\\w]+)(?:[\\/|\\.]?)(?:[^\\s"]*))'	# HTTP URL
    rg = re.compile(re_pattern,re.IGNORECASE|re.DOTALL)
 
    api = twitter.Api(user, password)
    timeline = api.GetFriendsTimeline(user)
 
    for status in timeline:
        m = rg.search(status.text)
        if m:
            httpurl=m.group(1)
            title = getTitle(httpurl)
            print httpurl, '-', title
 
def getTitle(url):
    req = urlopen(url)
    html = req.read()
 
    re_pattern='<title>(.*?)</title>'
    rg = re.compile(re_pattern,re.IGNORECASE|re.DOTALL)
 
    m = rg.search(html)
    if m:
        title = m.group(1)
        return title.strip()
    return None
 
if __name__ == '__main__':
    listFriendsURLs(SETTINGS['user'], SETTINGS['password'])
Internet, Python, Software

Posted on by Matt Warren

I was a bit hesitant to post this script since it is such a powerful marketing tool that it could be used very badly in the hands of a spammer. The basic premise is to directly respond to someone’s tweet if they mention your product or service. So for example I might want to have a tweet that goes out directly to someone who mentions twitter and python in a tweet and let them know about this blog. This will accomplish the same thing as the TwitterHawk service except you won’t have to pay per tweet.

To do this I had a choice. I could use a service like TweetBeep.com and then write a script that responded to the emails in my inbox, or I could use the Twitter Search API directly. The search API is so dead simple that I wanted to try that route.

The other thing to consider is that I don’t want to send a tweet to the same person more than once so I need to keep a list of twitter users that I have responded to. I used pickle to persist that list of usernames to disk so that it sticks around between uses.

The query functionality provided by the Twitter Search API is pretty cool and provides much more power than I have used in this script. For example it is possible to geo-target, lookup hashtags, or reply tweets. You can check out the full spec at http://apiwiki.twitter.com/Twitter-API-Documentation

Lastly, to keep it a bit simpler I’m ignoring the pagination in the search results and this script will only respond to the first page worth of results. Adding a loop per page would be pretty straight forward but I didn’t want to clutter up the code.

Example Usage:

>>> import tweetBack
>>> tweetBack.tweet_back('python twitter', 'Here is a blog with some good Python scripts you might find interesting http://halotis.com', 'twitter_username', 'twitter_password')
@nooble sent message
@ichiro_j sent message
@Ghabrie1 sent message
.....

Here’s the Python Code:

#!/usr/bin/env python
# -*- coding: utf-8 -*-
# (C) 2009 HalOtis Marketing
# written by Matt Warren
# http://halotis.com/
 
try:
   import json as simplejson
except:
   import simplejson  # http://undefined.org/python/#simplejson
import twitter     #http://code.google.com/p/python-twitter/
 
import urllib
import pickle
 
TWITTER_USER = 'username'
TWITTER_PASSWORD = 'password'
 
USER_LIST_FILE = 'tweetback.pck'
 
#read stored list of twitter users that have been responded to already in a file
try:
    f = open(USER_LIST_FILE, 'r')
    user_list = pickle.load(f)
except:
    user_list = []
 
def search_results(query):
    url = 'http://search.twitter.com/search.json?q=' + '+'.join(query.split())
    return simplejson.load(urllib.urlopen(url))
 
def tweet_back(query, tweet_reply, username=TWITTER_USER, password=TWITTER_PASSWORD):
    results = search_results(query)
 
    api = twitter.Api(username, password)
    try:
        for result in results['results']:
            if result['from_user'] not in user_list:
                api.PostUpdate('@' + result['from_user'] + ' ' + tweet_reply)
                print '@' + result['from_user'] + ' sent message'
 
                user_list.append(result['from_user'])
    except:
        print 'Failed to post update. may have gone over the twitter API limit.. please wait and try again'
 
    #write the user_list to disk
    f = open(USER_LIST_FILE, 'w')
    pickle.dump(user_list, f)
 
if __name__=='__main__':
    tweet_back('python twitter', 'Here is a blog with some good Python scripts you might find interesting http://halotis.com')

Update: thanks tante for the simplejson note.

Internet, Python, Software

Posted on by Matt Warren

I noticed that several accounts are spamming the twitter trends. Go to twitter.com and select one of the trends in the right column. You’ll undoubtedly see some tweets that are blatantly inserting words from the trending topics list into unrelated ads.

I was curious just how easy it would be to get the trending topics to target them with tweets. Turns out it is amazingly simple and shows off some of the beauty of Python.

This script doesn’t actually do anything with the trend information. It just simply downloads and prints out the list. But combine this code with the sample code from
RSS Twitter Bot in Python and you’ll have a recipe for some seriously powerful promotion.

import simplejson  # http://undefined.org/python/#simplejson
import urllib
 
result = simplejson.load(urllib.urlopen('http://search.twitter.com/trends.json'))
 
print [trend['name'] for trend in result['trends']]
Business, Internet, Software

Posted on by Matt Warren

This script will create an image on the fly of a users most recent twitter message.  It could be used as an email or forum signature or any place that allows you to embed a custom image such as on a blog or website.

I saw a website that did this the other day and wanted to try to duplicate the functionality.  Turns out it was pretty trivial even for someone with very little PHP experience. So I felt inspired enough to create a new website based on this script and called it TwitSig.us. Check it out.

It creates images something like this:

And here’s the code that does it:

<?php
include "twitter.php"; // from http://twitter.slawcup.com/twitter.class.phps
 
$t = new twitter();
$res = $t->userTimeline($_GET["user"], 1);
 
$my_img = imagecreatefrompng ( "base.png" );
 
$grey = imagecolorallocate( $my_img, 150, 150, 150 );
$red = imagecolorallocate( $my_img, 255, 0,  0 );
$text_colour = imagecolorallocate( $my_img, 0, 0, 0 );
 
if($res===false){
	imagestring( $my_img, 4, 30, 25, "no messages at this time",
	  $text_colour );
} else {
	$newtext = wordwrap($res->status->text, 65, "\n");
	imagettftext( $my_img, 10, 0, 10, 35, $text_colour, "Arial.ttf", $newtext);
	imagettftext( $my_img, 10, 0, 90, 15, $red, "Arial Bold.ttf", "@".$_GET["user"]);
	imagettftext( $my_img, 10, 0, 225, 15, $grey, "Arial.ttf", strftime("%a %d %b %H:%M %Y", strtotime($res->status->created_at)));
}
 
header( "Content-type: image/png" );
imagepng( $my_img );
?>

To get this script working for yourself you’ll need to make sure that you have the two font files and the base.png file for the background image that the text is put on.

Internet, Python, Software

Posted on by Matt Warren

I was a little bored today and decided to write up a simple script that pushes RSS feed information out to Twitter and manages to keep track of the history so that tweets are not sent out more than once.

It was actually a very trivial little script to write but it could actually be useful for something that I’m working on in the future.

The script makes use of an Sqlite database to store history and bit.ly for shortening URLs. I’ve made heavy use of some really nice open source libraries to make for a very short and sweet little script.

Grab the necessary python libraries:
python-twitter
python-bitly
feedparser

You’ll need to sign up for free accounts at Twitter and Bit.ly to use this script.

Hopefully someone out there can take this code example to do something really cool with Twitter and Python.

Update: I’ve added some bit.ly link tracking output to this script. After it twitters the RSS feed it will print out the click count information for every bit.ly link.

from time import strftime
import sqlite3
 
import twitter     #http://code.google.com/p/python-twitter/
import bitly       #http://code.google.com/p/python-bitly/
import feedparser  #available at feedparser.org
 
 
DATABASE = "tweets.sqlite"
 
BITLY_LOGIN = "bitlyUsername"
BITLY_API_KEY = "api key"
 
TWITTER_USER = "username"
TWITTER_PASSWORD = "password"
 
def print_stats():
	conn = sqlite3.connect(DATABASE)
	conn.row_factory = sqlite3.Row
	c = conn.cursor()
 
	b = bitly.Api(login=BITLY_LOGIN,apikey=BITLY_API_KEY)
 
	c.execute('SELECT title, url, short_url from RSSContent')
	all_links = c.fetchall()
 
	for row in all_links:
 
		short_url = row['short_url']
 
		if short_url is None:
			short_url = b.shorten(row['url'])
			c.execute('UPDATE RSSContent SET `short_url`=? WHERE `url`=?',(short_url,row['url']))
 
 
		stats = b.stats(short_url)
		print "%s - User clicks %s, total clicks: %s" % (row['title'], stats.user_clicks,stats.total_clicks)
 
	conn.commit()
 
def tweet_rss(url):
 
	conn = sqlite3.connect(DATABASE)
	conn.row_factory = sqlite3.Row
	c = conn.cursor()
 
	#create the table if it doesn't exist
	c.execute('CREATE TABLE IF NOT EXISTS RSSContent (`url`, `title`, `dateAdded`, `content`, `short_url`)')
 
	api = twitter.Api(username=TWITTER_USER, password=TWITTER_PASSWORD)
	b = bitly.Api(login=BITLY_LOGIN,apikey=BITLY_API_KEY)
 
	d = feedparser.parse(url)
 
	for entry in d.entries:
 
		#check for duplicates
		c.execute('select * from RSSContent where url=?', (entry.link,))
		if not c.fetchall():
 
			tweet_text = "%s - %s" % (entry.title, entry.summary)
 
			shortened_link = b.shorten(entry.link)
 
			t = (entry.link, entry.title, strftime("%Y-%m-%d %H:%M:%S", entry.updated_parsed), entry.summary, shortened_link)
			c.execute('insert into RSSContent (`url`, `title`,`dateAdded`, `content`, `short_url`) values (?,?,?,?,?)', t)
			print "%s.. %s" % (tweet_text[:115], shortened_link)
 
			api.PostUpdate("%s.. %s" % (tweet_text[:115], shortened_link))
 
	conn.commit()
 
if __name__ == '__main__':
  tweet_rss('http://www.halotis.com/feed/')
  print_stats()