Log in

No account? Create an account
Theory blog aggregator: stats and directions - Arvind Narayanan's journal [entries|archive|friends|userinfo]

Theory blog aggregator: stats and directions [Jul. 20th, 2009|12:08 am]
Arvind Narayanan

Traffic to the Theory of Computing Blog Aggregator has more than doubled over the past year, and is now at 16,000 visits/month.

Visits / month

Incidentally, the readership is tech-savvy, as you might expect — more than half the visits come from Firefox, Safari is the second most popular, MSIE barely breaks 10% and Chrome is almost tied with MSIE. Opera traffic is a rounding error, and bots aren't shown on the chart.

Visits by browser: Jun 2009

Given that Theory of Computing is a fairly niche community, the amount of traffic is surprising for a site that was slapped together with a weekend's worth of effort. Combined with the fact that the RSS reader is dying, it seems clear to me that topic aggregators have a great future.

So I'm wondering if it's time to update the aggregator with some new features. The problem with this is that I already worked on a snazzy Web 2.0 UI (hit '?' for keyboard shortcuts), which turned out to be much worse for usability than the current avatar.

Without going that route, here are some of the things I'm thinking of doing:
  • Write special-purpose code to import comments from Wordpress blogs. Currently the aggregator imports Blogger comments but not Wordpress, because the latter works in a different way. Blogger and wordpress cover the vast majority of the blogs, so that's all I really care about.
  • Polish up the recent comments feature (see the Web 2.0 UI link above for a demo) and implement it on the main site.
  • Import TOC-related tweets. I will use a special hash-tag, (#cstheory?) to import. This will be combined with either a blacklist or a whitelist of twitter users, in case anyone tries to abuse it.
  • Set up a bunch of other aggregators.

[User Picture]From: patrickwonders
2009-07-20 01:51 pm (UTC)
Hmmm. I didn't realize you were behind that aggregator. I have been reading it via RSS for three or four months. I suppose that I should look into why RSS readers are dying. :(

And, with the above stats. I should look into aggregating some of the math blogs I read.
(Reply) (Thread)
[User Picture]From: arvindn
2009-07-20 04:52 pm (UTC)
i thought about doing math blogs, but then realized there's a whole spectrum of them in terms of the audience -- k-12 (which has its own spectrum), college-level, research, etc. even if you limit yourself to research blogs, isn't the field so broad that most mathematicians aren't interested in each other's research? do you think you can identify a coherent subset of blogs?

edit. i didn't mention it in the post, but part of my plan is to create a web UI for anyone to use my software to build their own aggregator. you want in?

it will cost money to host, so there will be ads on the page. hopefully since the audience is highly targeted, the ads will be something that's actually useful (e.g., links to books relevant to the topic.) that's the only catch.

Edited at 2009-07-20 04:57 pm (UTC)
(Reply) (Parent) (Thread)
[User Picture]From: patrickwonders
2009-07-20 05:18 pm (UTC)
I have about thirty feeds in my RSS reader in the Math folder. Actually, I have a couple in there that don't mesh so well with the others. I think the rest are pretty coherent though. Part of that is because I generated my list by surfing blogrolls. So, many of the blogs on the list are there because one of the other bloggers on the list likes them.

I did the same with CS-ish sorts of blogs. Of course, I ended up with a bunch of blogs that I now see in both the ToC aggregation and individually. *shrug*

I'm falling behind on my own projects at the moment, and I have more fire for some of those than for this aggregator thing, so I'll have to pass on jumping in here. I'm also not particularly excited about Javascript....

My webhosting site doesn't have Python installed either, so I'm not sure I can be much use in the resources department.
(Reply) (Parent) (Thread)
[User Picture]From: arvindn
2009-07-20 08:38 pm (UTC)
oh, i guess i wasn't very clear. i'm going to make a web page where you can enter a list of blogs and create an aggregator with the push of a button. you wouldn't have to host anything. the UI would let you optionally specify tags for each blog (to filter the posts so that off-topic posts don't show up), and URLs for a thumbnail for each blogger.

if you don't have time that's fine; i'd appreciate if you could send me your list, and i'll try to take care of the rest myself.
(Reply) (Parent) (Thread)
[User Picture]From: patrickwonders
2009-07-20 09:37 pm (UTC)
Ah... I thought you were looking for someone to help you code or market or what-have-you.... Certainly, I can give it all a go when you're ready for someone to do that.
(Reply) (Parent) (Thread)