NOTICE:
This content is in the "archives" of Gadgetopia. It has been moved to this subdomain as it is no longer considered relevant to the site. It is being hosted here for a indeterminate period of time. Its existence at this URL is not guaranteed, and it may be removed at any time. If you would like to refer to this content in the future, you are encouraged to save it to your local file system.

What Links Here

Originally published by "dbarker" on 2005-12-29 11:02:00Z

We’re trying something new around here. Since we have so many posts, there’s a lot of internal linking between posts. This linking has value because it can point you to other things you may be interested in.

You see, you can only link to something in the past – it needs to exist when you’re writing or editing the page. But what about something you write and then link to multiple times? Knowing what has been written after the item that you’re looking at, that links back to it, is something you may want to know.

To this end, we’ve created a script that runs every night and parses the text of every entry on this site, extracts all the internal hyperlinks, and traces them back to the posts they link to. (Yes, I thought about using referrers. That would probably work too. But that requires someone to click on the link, and that struck me as a fuzzy way of doing things.)

So, in a lot of posts, right under the text and before the comments, you’ll see something like this:

Take a look around and see if you like it. We will have some issues with Markdown appearing in the previews, but it’s largely in place. It seems to have value – it’s very good at pointing you at something related to what you’re looking at which you might be interested in reading. And that’s really the trick to get visitors deeper into the site.

As an added bonus, here are the top five posts on this site by number of internal links pointing to them. The winner is no surprise, if you’ve been around here for a while. Click on any of these things to see lots of “What Links Here” results, if you want examples.

  1. Open and Closed Content Management

  2. Adding User-Defined Fields in Movable Type

  3. Problems No Platform Will Fix

  4. Keyword Categorization: Thinking Out Loud

  5. Bayesian Spam Filter for Outlook

Comment by "Biscuit" on 2005-12-30 12:57:00Z
Cool! Nice simple solution to a fairly universal problem. No idea if you got the idea from a comment I left on one of your other posts ([Automatically changing tense](http://www.gadgetopia.com/post/4714)), but it's always good to see techniques like this being tried out (actually I'm pretty sure you guys already had this idea in your heads given the way you've been developing gadgetopia). As a question though, any ideas on how to differentiate between something that is just a related post/article, versus an update on a current issue (more useful on news sites, for example getting the history of the Sony DRM debacle)... I wonder would it be useful (beyond an academic exercise) to actually automatically create a timeline map of posts? Don't know, are there any products/services out there that are looking into this kind thing? (other than search engines that is)
Comment by "Deane" on 2005-12-30 15:57:00Z
Sadly, there's a bug, it looks like. The "This Post Links To" section on this entry should have five entries. It only has three. Bummer.
Comment by "Darren Chamberlain" on 2005-12-30 16:33:00Z
It would be cool if you took all of this information and graphed it. You could build an interesting map of the site this way. Also, is the script available anywhere? I'd like to see how it works
Comment by "josh" on 2005-12-30 18:36:00Z
Why not create the list when you publish the entry? As the event gets stored into the database parse it there and update the DB. Theres a lot of latency in the system right now.
Comment by "Deane" on 2005-12-30 07:15:00Z
>Why not create the list when you publish the entry? That's not logically possible. "What Links Here" -- by definition -- refers to things that have been published *after* the subject entry. They don't exist when the entry is published.
Comment by "Biscuit" on 2006-01-02 09:58:00Z
@Deane, I think what Josh was refering to is that when you do a post, add it and it's links to a db (and then the links to section is just a query against the db) rather than scanning all existing posts (that that would of needed to be done to fill in the db)... More of an aid to how to work out the links, and then you can do all sorts of funky stuff like graphing the posts and looking at common threads... Cool! :-)