Don’t fret the next time you have to check a large list of links to discover which are dead and which are live, Dead Link Checker can check your list for you in next to no time.

Most regular visitors to JournalXtra are aware of the Webmaster Marketing Tools that I compiled to make it easier to submit websites, blogs and ebooks etc.. to relevant directory, ping service and syndication sites as well as to submit press releases. Websites come and go over time and so links to them expire. Every once in a while I recheck those links, hundreds of them, for activity. If I checked those links manually by clicking them one-by-one it would take me a week or two to keep those tools up to date.

One method used to check for dead links is to use software that reads a webpage or website and attempts to load each link. Such software can be set to check links on the pages to which links on the tested page lead and to further depth if required. For example, these programs will check links on page one, links on the pages lead to by links on page one, links on the pages to which the links on page two lead and so on….

A few of these link check programs are:

• KLinkStatus (Linux)
• Linkchecker
• Xenu’s Link Sleuth (Windows)

A more exhaustive list of URL checker tools is at the bottom of this page along with the download link for Dead Link Checker.

This type of link check program has two huge limitations:

1. they check all links on a specified page. One cannot specify which links are checked and which links are not;
2. the links they check must be placed with in a webpage that is stored and loaded by a web server.

It is not difficult to see how those limitations can become hindrances to anyone wishing to check the activity of a specific set of links within a webpage or website with many links that need not be checked.

Thankfully, there is a perl script called Dead Link Checker which will run through a list of links in a file that is stored on a local machine, test them for activity and write each result to an HTML formatted file.

Dead Link Checker places each link into an HTML file (which it automatically creates) categorized by server response code. Each tested link that returns 3xx or 4xx server codes (for example 301 or 403) is written into the HTML file alongside any page to which it redirects. This makes it easy to decide whether a link is dead or just redirected to an updated version of the page.

Many page redirects are only between the http:// and http://www. URL preference of a site’s webmaster.

Here’s an example of how to use it. Example pages provided. All work is performed from within the same  folder:

1. create a text file (links.txt)
2. paste the collection of URLs into the file (links.txt) – it might look like this
   
3. format the links so that they only describe the URL from HTTP:// to .COM or .NET etc… i.e remove any superfluous text – it might now look like this . Use the Text Manipulation FAQs if you need help to quickly edit large text files.
   
4. open a terminal (Konsole or console) in the folder that holds the text file (links.txt)
5. type the command


deadlinkcheck -HTMLoutput -noCache -Verb links.txt > checked.html


6. that command will provide an HTML file called checked.html out of the links it checks from the file links.txt – it might look like this
   
7. load the created HTML file (checked.html) in a web browser
8. also load the HTMl file (checked.html) in a text editor (I use Kate) – it might look like this
   
9. read the links and manually remove the bad ones from the HTML file loaded in the text editor
   • those with a 2xx result are Live so should be kept
   • those with a 3xx result are redirecting to somewhere else so check that the redirect is not to a totally different site; if it is then that link is probably dead so check them manually to confirm their status
   • those with a 4xx result lead to non found pages. Check them manually to confirm whether they are active or dead. Dead Link Checker relies on the server response, not all servers give a correct response
   • those with a 5xx result will likely never load. Check them manually
10. use the text editor to reformat the links in the HTML file (checked.html) once the bad links have been removed from it (Instructions provided assume the use of Kate)
    • convert all characters to lower case (Tools>Lowercase)
    • remove superfluous characters with sed:
    • open find and replace (ctrl+r)
    • find

    
       


    • replace it with

    
|


    • (or some other character that is not reproduced anywhere within the file)
    • Use Alt+A to replace all occurrences.
    • find

    
<code>|-> </code>


    • replace it with nothing (i.e just remove it)
    • find

    
<b>


    • replace it with nothing (i.e just remove it)
    • find

    
</b>


    • replace it with nothing (i.e just remove it)
    • find

    
</a><br>


    • replace it with

    
</a>


    • Manually remove anything that isn’t between a tags (<a> and </a>)
    • use a terminal (Konsole or Console) – open it from within the folder that holds the HTML file (checked.html) and enter this command

    
sed 's/.*|//g' checked.html > links.txt


    • That command removes anything in a line that is written in front of a pipe “|” (inclusive of the pipe) and places all the links into a the file links.txt
11. The file links.txt will now contain the checked, active URLs from the original list formatted within <a> tags with anchor text equal to the link e.g


<a href="http://journalxtra.com">http://journalxtra.com</a>

It will look similar to this


12. If required, alter the anchor text to remove the http:// and .com (or .net etc…) components.

Intelligent use of Dead Link Checker can transform a task that would usually take a week or two to complete into one that takes less than 20 minutes.

I do not whether DLC works in Windows. Use a Linux Live Disk (Kubuntu or Linux Mint) if it doesn’t.

Download Link

Download Dead Link Checker:

• Sourceforge
• Its user guide is available  here (also at Sourceforge)

List of Link Checker Software

These URL checker tools check links loaded within webpages served online only.

Run from a local PC

• Checklinks
• Dead link check
• gURLChecker
• KLinkStatus
• Linkchecker
• link-checker
• linklint
• webcheck
• webgrep
• Xenu’s Link Sleuth

HTML interface

• W3C Link Checker

  Copyright secured by Digiprove © 2010

In all likelihood you’re reading this because, like me, you’ve been driven to smashing your head against every brick wall around you by the source code for both sdlmame and sdlmess. Yes, like me, I expect you’ve found nothing but disaster every time you’ve tried to compile them.

I gave up on them several times. I resorted to the ZXMameCD – a mame and mess live disc – to get my retro gaming fix; and as that didn’t do everything I needed I created a virtual machine  so I could install them onto a virtual OS that would share my host system’s files (ZXMameCD didn’t). But I never gave up fully and all the tinkering and googling has finally paid off: yes, I have them both installed. And I will bring you back from the brink of insanity by showing how I did it.

I cheated a little with installing sdlmame. I found two repositories that provide sdlmame and sdlmess packages. One of them provides the up-to-date version of sdlmame and the other provides a not-so-up-to-date version of sdlmess and sdlmame. So we can install the up-to-date sdlmame from a repository but we still need to compile sdlmess from source code if we want the latest version of it.

The Up-to-Date SDLMAME Repository

That repository’s home page is here.

The instructions for adding this repository to versions of *ubuntu other than 9.10 are available here.

The Terminal commands for adding the Karmic Koala specific repository are:

wget http://sdlmame4ubuntu.org/repoc/957AC3A6.gpg -O- |  sudo apt-key add -
sudo wget http://sdlmame4ubuntu.org/repoc/karmic.major.list -O  /etc/apt/sources.list.d /sdlmame4ubuntu.major.list
sudo apt-get update
sudo apt-get install sdlmame

Nice, easy and quick.

The Not-So-Up-to-Date SDLMAME and SDLMESS Repository

This repository is aimed at all Debian based distros. At the time of writing this it provides access to an older version of sdlmame than the repository listed above here. Although the writing at the site states that it provides access to an sdlmess package I wasn’t able  install it through any of my package managers (that includes running apt-get from Terminal). You might have better luck than I did. The instructions listed here are for Debian Lenny and should be good for *buntu 8.04 to 9.10 (I think).

The repository’s home page is here. You can add it to your system by typing, or copying and pasting, the following instructions into Terminal:

sudo su
echo deb http://apt.ludomatic.fr lenny non-free >> /etc/apt/sources.list.d/ludomatic.list
echo deb-src http://apt.ludomatic.fr lenny non-free  >> /etc/apt/sources.list.d/ludomatic.list
wget http://apt.ludomatic.fr/ludomatic.key.asc -O - | apt-key add -
apt-get update
apt-get install sdlmame sdlmess
exit

Compiling SDLMAME and SDLMESS from Source Code

In case you want to compile sdlmame and sdlmess for yourself the official home and download page for them is here and the instructions for compiling them are here. Don’t worry that they both say they are for sdlmam because the build instructions are the same for both products.

The installation instructions found at those two pages were not very helpful for me. In fact, they drove me to near insanity because no matter how many times I tried them they never resulted in successful installations. The steps I took to get them to install on my Kubuntu system were:

1. Install the necessary development, header and dependency files using these Terminal commands:

sudo apt-get install build-essential libsdl1.2-dev libgconf2-dev libexpat1-dev zlib1g-dev libxinerama-dev libgtk2.0-dev gcc-4.2
sudo apt-get build-dep sdlmame

2. Download the SDLMESS zip file from here, unzip it and move it to /usr/share/games (see note below)

3. Open  Terminal in the folder that contains the source code (/usr/share/games/sdlmess-xxx)

4. Compile SDLMESS using the appropriate one of these Terminal commands:

64 bit systems

makeCC=@gcc-4.2 LD=@gcc-4.2 PTR64=1

64 bit systems with duel core processors

makeCC=@gcc-4.2 LD=@gcc-4.2 PTR64=1 -j3

32 bit systems

make -f makefile.sdl CC=@gcc-4.2 LD=@gcc-4.2

32 bit systems with duel core processors

make -f makefile.sdl CC=@gcc-4.2 LD=@gcc-4.2 -j3

Guidance for accessing /usr/share/games

You will need to open Terminal and type kdesudo dolphin (or, if you’re using gnome, gksu nautilus) to load your filemanager in super user mode which means you will be able to move files to, edit files in and delete files from areas that are usually access restricted.

Additional SDMAME and SDLMESS Resource

QMC2 is a graphical front end for both sdlmame and sdlmess. It has to be compiled twice: once for sdlmess and once for sdlmame. The home page is here, the download page is here and the installation instructions are here. The QMC2 site offers lots of other mame and mess related downloads too.

This page will be updated often.

I was about to stick these extra repositories into the Karmic Koala Review but thought “No, there are enough of them and they’re useful enough to put into their own little section.” so that’s what I’ve done here. I will update this list of repositories whenever I find a new one that’s worth adding to my sources.list.

First, a little about repositories

Software can be installed into Linux in one of two ways:

1. Each program can be located and downloaded from the Internet or an external storage device and then installed in its own peculiar way; and

2. Each program can be installed using a package manager which queries a locally stored list of items stacked with different software libraries called repositories. This list tells package managers where to find downloadable software.

Manually finding software over the Internet can be a long chore and increases the likelihood of picking up unwanted guests that might rape and pillage an operating system. Using a package manager to find and install software saves time and minimizes the risk of picking up viruses when the repositories it consults are trustworthy.

Different projects use different repositories to host their software which means we often need to add extra repositories to our package managers’ list of them. This list is called sources.list. The Ubuntu distributions store sources.list in /etc/apt/ and additional user added lists in /etc/apt/sources.list.d/(filename.list). If you want to see the content of your repository list then type kate /etc/apt/sources.list (Gnome users should use gedit instead of kate). You won’t be able to edit it unless you view it as root by typing kdesudo or gksudo in front of those commands e.g kdesudo kate /etc/apt/sources.list will open sources.list for editing.

Each repository has a security key (GPG key) associated with it that ensures package managers are aware that the software installed from a repository has been authenticated by the user who added it (that’ll be YOU, generally). Package managers alert a user who is about to install  non-authenticated software and prompt for the user to confirm that the software should be installed.

There are two ways that you can add extra repositories to your sources.list:

1. Copy the content of another machine’s sources.list and paste it over your own or merge them, as you see fit; and

2. Add each extra repository and it’s GPG key individually using either the command line (Terminal) or your package manager’s built-in repository manager.

Here is a video demonstration of how to install a repository and GPG key using Synaptic. The repository came from here and the GPG key came from here (the whole contents of the page was copied into a text file which was then renamed gpgkey.gpg):

Would you like to replay the video or share the link to it with your friends?

Copying and pasting another repository list is not ideal. Different Linux disros require different repositories and, as repositories provide a route into a machine, over-zealous copying of sources.lists from one machine to another could open the gate for malicious software to flood in.

Adding new repositories isn’t difficult and most software providers give good instructions. The easiest way to add a repository is to type or copy and paste a few commands into a Terminal as demonstrated in this next video. The repository used in the video came from here and the GPG key came from here. The actual code obtained from those links is:

deb http://ppa.launchpad.net/kubuntu-ppa/ppa/ubuntu karmic main

and

gpg --keyserver keyserver.ubuntu.com --recv 2836CB0A8AC93F7A

gpg --export --armor 2836CB0A8AC93F7A | sudo apt-key add -

Which I re-formatted to one line which can be entered into Terminal by a super user (type sudo su then the return key to become a super user), thus

echo deb http://ppa.launchpad.net/kubuntu-ppa/ppa/ubuntu karmic main >> /etc/apt/sources.list && gpg --keyserver keyserver.ubuntu.com --recv 2836CB0A8AC93F7A && gpg --export --armor 2836CB0A8AC93F7A | sudo apt-key add -

Would you like to replay the video or share the link to it with your friends?

So, here, as promised, are those extra repositories. I’ve linked to the source website for each repository and provided a few links to some great sources for software that are not hosted in repositories.

Medibuntu

In their own words:

“Medibuntu (Multimedia, Entertainment & Distractions In Ubuntu) is a repository of packages that cannot be included into the Ubuntu distribution for legal reasons (copyright, license, patent, etc).”

This repository contains many of the multimedia codecs required to play patented and restricted formats.

The main site is here.

To add the repository to your own system (all *buntu flavours including 32 and 64 bit versions) copy and paste the following lines of code into Terminal (there’s no need to do them one at a time, just copy and paste the whole lot in one go):

sudo wget http://www.medibuntu.org/sources.list.d/$(lsb_release -cs).list --output-document=/etc/apt/sources.list.d/medibuntu.list
sudo apt-get -q update
sudo apt-get --yes -q --allow-unauthenticated install medibuntu-keyring
sudo apt-get -q update

Use either your package manager or Terminal (using sudo apt-get install package) to install libdvdcss2 (for playing DVDs) and one of the following non-native codec packages. Choose one that is suitable for your computer’s architecture:

w32codecs for 32 bit machines,

w64codecs for 64 bit machines, or

ppc-codecs for power PCs.

Immediately after installing a Linux operating system I use Synaptic to browse for and install codecs and the appropriate “restricted extras” for the distribution. You might choose to do the same.

For a full list of the available packages visit here.

Ubuntu Studio

This one provides a lot of extra multimedia packages aimed specifically at multimedia enthusiasts. In the developers’  own words:

“We provide a suite of the best open-source applications available for multimedia creation. Completely free to use, modify and redistribute. Your only limitation is your imagination.”

The main site is here.

To add the repository to your Karmic Koala system copy and paste the following lines of code into your Terminal:

sudo su
echo deb http://archive.ubuntustudio.org/ubuntustudio karmic main >> /etc/apt/sources.list
wget -q http://archive.ubuntustudio.org/ubuntustudio.gpg -O- | sudo apt-key add –
sudo apt-get update

The GPG key might not work for all systems but the available packages will still be accessible through your package manager. An alternative GPG key can be obtained from here (a public keyserver). You can import it with these Terminal instructions:

gpg --keyserver keyserver.veridis.com --recv-keys 0x2D635AED
gpg --export --armor 0x2D635AED | sudo apt-key add -

To add the repository to versions other than Karmic Koala (e.g Jaunty Jakalope) replace karmic with the other version’s title wherever it occurs in the above code (e.g replace karmic with lucid,  jaunty or gutsy).

Once the Ubuntu Studio repository has been added you can install components from it either individually or by searching for “ubuntustudio” in Synaptic (or another package manager) and selecting to install all the media and audio packages in one go. Please be aware that you might need to reselect KDE as your desktop environment when you next log-in after installing the Ubuntu Studio packages – they have a habit of auto-selecting Gnome.

PlayOnLinux

This is THE compliment to anyone who uses Wine for running Windows programs on Linux. It provides a graphical interface from which Windws software can be selected and installed. For those wanting more information about this program I’ve written a mini-review of it here.

The home website can be found here.

To add the repository to Karmic just copy and paste these few lines into your Terminal:

sudo wget http://deb.playonlinux.com/playonlinux_karmic.list -O /etc/apt/sources.list.d/playonlinux.list
sudo apt-get update
sudo apt-get install playonlinux

Users of other versions of Linux should visit the download page for PlayOnLinux, here, to get the correct code for them.

VirtualBox

Provides the software required to install multiple operating systems onto one machine without the need to edit hard drive partitions or to duel boot. It does this by creating a virtual hardware environment onto which a guest operating system may be installed. The virtual environment is hosted by the real hardware and operating system; and the virtual environment provides the guest operating system access to the host operating system’s hardware without the guest being aware that it is being run within another environment. In the words of the developers:

“VirtualBox is a powerful x86 and AMD64/Intel64 virtualization product for enterprise as well as home use. Not only is VirtualBox an extremely feature rich, high performance product for enterprise customers, it is also the only professional solution that is freely available as Open Source Software under the terms of the GNU General Public License (GPL).”

I use VirtualBox in preference to Qemu and FAUmachine for most of my virtualiseation needs.

The home page for VirtualBox is here.

Copy and paste these lines of code into a Terminal to add the VirtualBox repository to Karmic Koala:

sudo su
echo deb http://download.virtualbox.org/virtualbox/debian karmic non-free >> /etc/apt/sources.list
wget -q http://download.virtualbox.org/virtualbox/debian/sun_vbox.asc -O- | sudo apt-key add -
sudo apt-get update

At this point in time the current version is VirtualBox 3.0 so install it with apt-get install virtualbox-3.0

Users of other Linux distributions should see the download page here for specific repository installation notes.

I recommend you visit the homepage to learn more about this program and its associated part called Guest Additions.

I’ve written a step-by-step text guide with video demonstrations to setting-up VirtualBox and adding shared media and drives. You can view it here.

IPList

Is similar to Moblock and Peerguardian – it helps defend a computer’s downloads from malicious or espying IP addresses. These type of programs are often used by well informed torrent users who need to protect themselves from copyright activists and government snooping. They’re not perfect so cannot be fully relied on. They work by checking the IP address of a download’s source against a blacklist of known mischievous IP addresses. When a blacklisted IP address is found the data it sends is dropped and the sender is unable to prove that a downloader is downloading restricted data.

The program’s homepage is here.

Karmic Koala users can add the repository, GPG key and package to their systems by copying and pasting the following lines of code into their Terminal (this is good for all versions of Karmic (64 and 32 bit versions as well):

sudo wget http://iplist.sf.net/sources.list.d/karmic.list -O /etc/apt/sources.list.d/iplist.list
sudo apt-key adv --recv-keys --keyserver keyserver.ubuntu.com C6E3D905C8BCD56BB02E6E0B39456311108B243F
sudo apt-get update
sudo apt-get install iplist

Users of other Linux distros should visit here to learn how to install it.

GetDeb

This is  small repository of Linux applications and games that have been pre-packaged ready for installation into Ubuntu based systems. There’s a problem with their instructions for adding their GPG key via Terminal so the best way to add their repository is by visiting this page and following step one of their installation guide.

If you really want to use the Terminal method then the code is:

sudo su
echo deb http://archive.getdeb.net/ubuntu karmic-getdeb apps >> /etc/apt/sources.list
wget -q -O- http://archive.getdeb.net/getdeb-archive.key | sudo apt-key add -
sudo apt-get update

GetDeb’s homepage can be found here.
SDLMAME & SDLMESS
These are the up-to-date and maintained versions of MAME (Multiple Arcade Machine Emulator) and MESS (Multiple Emulator Super System). MAME is for playing arcade games on a PC whereas MESS is for running console and non-PC computer software (e.g Spectrum, Atari, Commodore). There are two repositories for these. The first one is specifically for SDLMAME for *ubuntu systems and the second one is for SDLMAME and SDLMESS on all Debian based operating systems (which includes the *ubuntu distros).
The Terminal commands for adding the *ubuntu Karmic Koala specific repository are (these do not install SDLMESS):

wget http://sdlmame4ubuntu.org/repoc/957AC3A6.gpg -O- |  sudo apt-key add -
sudo wget http://sdlmame4ubuntu.org/repoc/karmic.major.list -O  /etc/apt/sources.list.d /sdlmame4ubuntu.major.list
sudo apt-get update
sudo apt-get install sdlmame

Instructions for adding this repository to earlier versions of *ubuntu are available here.
That repository’s home page is here.
The second repository is aimed at all Debian based distros. At the time of writing this it provides access to an older version of SDLMAME than the above repository. I couldn’t install SDLMESS from this one but you might have better luck. The instructions listed here are for Debian Lenny and should be good for *buntu 8.04 to 9.10 (I think).

sudo su
echo deb http://apt.ludomatic.fr lenny non-free >> /etc/apt/sources.list.d/ludomatic.list
echo deb-src http://apt.ludomatic.fr lenny non-free  >> /etc/apt/sources.list.d/ludomatic.list
wget http://apt.ludomatic.fr/ludomatic.key.asc -O - | apt-key add -
apt-get update
apt-get install sdlmame sdlmess
exit

The repository’s home page is here.
In case you want to compile SDLMESS and SDLMAME for yourself the official home and download page for them is here. Instructions for compiling them are here. Don’t worry that they both say they are for SDLMAME because the build instructions are the same for both SDLMESS and SDLMAME (hope that makes sense).
I had a lot of trouble getting SDLMESS to install on Kubuntu 9.10 for 64 bit processors. Here are the steps I took to get it to install. These instructions should work for most *ubuntu versions:

1. Install the necessary development, header and dependency files using these Terminal commands:

sudo apt-get install build-essential libsdl1.2-dev libgconf2-dev libexpat1-dev zlib1g-dev libxinerama-dev libgtk2.0-dev gcc-4.2
sudo apt-get build-dep sdlmame

2. Download the SDLMESS zip file from here, unzip it and move it to /usr/share/games (see note below)

3. Open  Terminal in the folder that contains the source code (/usr/share/games/sdlmess-xxx)

4. Compile SDLMESS using the appropriate one of these Terminal commands:

64 bit systems

makeCC=@gcc-4.2 LD=@gcc-4.2 PTR64=1

64 bit systems with duel core processors

makeCC=@gcc-4.2 LD=@gcc-4.2 PTR64=1 -j3

32 bit systems

make -f makefile.sdl CC=@gcc-4.2 LD=@gcc-4.2

32 bit systems with duel core processors

make -f makefile.sdl CC=@gcc-4.2 LD=@gcc-4.2 -j3

Additional Software Sources

Apt Get is a database of unofficial repositories of Debian software. Be careful whenever you add new repositories to your system. Apt Get is available here.

Gna is a source for open source software. It is available here.

Sourceforge is a huge collection of Linux and non-Linux software. It can be accessed here.

Happy Penguin is a fantastic resource for locating Linux games. Access is this way.

KDE-Apps.org provides los of KDE application software. This way to enter.

KDE-Files.org has loads of plug-ins for applications. You can get to it through here.

Linux App Finder is another huge collection of Linux software. It can be accessed here.

Open Source Living provides links and reviews of some useful Linux and non-Linux software. It can be accessed here.

Opera is available from here.

Google Chrome (beta for Linux) is available here.

Flash for 64 bit systems is available from  here.

Realplayer is available from here.

Key Server List
These are useful for finding GPG keys to authenticate software repositories when the software’s maintainer hasn’t explicitly stated his GPG key:

Veridis Key Server

Additional Repository Guides

The Ubuntu Repository Installation Guide

The Ubuntu Guide (I highly recommend this)

This page will be updated often.