New version up

We've got a new version out today.

About the biggest new feature is the ability to cluster stories by topic.

If you click the new "cluster" link at the top of the page, the stories on the page will be rearranged into groups by topic. The rearrangement is performed entirely using javascript and does not consume an additional page load. This is taken as a preference and on subsequent views the page will be rearranged automatically as soon as it loads.

Other features include improved sentence parsing, so the context information provided for related links will be clipped so that 1) not too much information will be lost 2) but otherwise will be cut at likely at sentence boundaries. The result is usually much more readable.

We've also included a link so that you can find related stories right from the main page, which previously you needed to click on there "permalink" link for each story in order to see, which most people may not have realized.

The next few enhancements will likely mostly be related to look and feel.

New features

So, we pushed out another new version yesterday.

Here are some of the new features:

• Related links for the exact post — For each of our sources that links to a top story, we now provide a link to the permalink to the exact post (if we're able to detect it) rather than just a link to the whole blog. This makes blogrevolution much, much more useful.


• More related stories — we now calculate the similarity of top stories to each other by their content, as well as by context. We offer a list of both if anything comes up


• Favicons everywhere — because they're cool

Also the RSS feed items now link to the permalink of each story at blogrevolution, instead of linking to the anchor link on the front page.

We should have some more revisions coming down the pike relatively soon.

New version coming soon

Get excited.

It's the little things that count

BlogRevolution now will be able to truncate text at sentence boundaries.

It's surprisingly how much better it looks this way.

RSS Engine update

We've testing the new RSS engine, and it's looking pretty sweet.

Still might be awhile before we see it deployed, however.

Related links and RSS engine updates

Recently our related story links for each story was returning far too many false positives.

The algorithm we came up with was weighted to heavily towards chronological overlap -- what people were linking to at the same times. Which was fraught with error, because our spider can't be everywhere at once, and so as a result too often stories came up as similar because of sheer coincidence.

Today I fixed the related links algorithm up so that it uses a much more robust distance statistic. This will mean that the related links will now be much more accurate. Hopefully.

Internally we've been working a lot on our RSS and Atom engine, which when it goes to production will improve many aspects of the site.

With just a few more features we will be able to go past our undeclared beta stage, and there will be much rejoicing.

New features rolling out today

Everyone loves YouTube, so we added a sidebar for recent YouTube videos today, aside from a number of minor aesthetic changes and internal enhancements.

Added posts similar

If you click on the permalink for any story, you'll now see a list of any stories that are contextually similar.

The method we use to do this is very crude yet it seems to be highly accurate. And, it's very fast and adds almost no time to the time it takes to build an update.

The content of the actual story is not considered. For instance, Barack Obama's web site shows up as highly similar to Hillary Clinton's web site, though if you were looking for similarity based on the content of those pages alone it would be very difficult to detect that relationship.

At some future date we might add a second list of stories similar by content, or we might decide it would be better to combine the two similarity measures.

Skipped an update

We lost power here at BlogRevolution HQ which looked like it caused some database corruption, so we skipped an update while we repaired the database and brought everything back online. Should be back to our regularly scheduled program in a few minutes.

New models cooking

Here are BlogRevolution HQ, we're working on better mathematical models for the article summarizer.

The article summarizer totally borks up a summary about 3% of the time. We're hoping to cut that down by about two thirds.

Javascript/CSS Compression

We've turned on compression on our javascript and css files. This will improve performance a little bit for most users, but could cause problems for Netscape 4 users. Since we appears to get no traffic from Netscape 4 users, (hurrah!) it shouldn't be a problem.

Progress marches on.